Elastic Search - Chat and Messages

Elasticsearch (ES) is used in Hububb Chat to make thread lists fast, searchable, and always up to date. Instead of reading every message from the database each time, we keep a lightweight copy of each thread in Elasticsearch. This allows users to quickly:

• Search and filter chats by keywords, tags, or participants

• Sort by newest activity, unread status, or importance

• Instantly see updated lists without loading full message histories

In short: Firestore stores the full chat data, and Elasticsearch powers fast searching and sorting for the chat inbox.

Why We Use Elasticsearch

Firestore is excellent for reliable storage and live updates — but not for complex queries or sorting large lists (e.g., “show all unread threads sorted by recent activity and tags”).

Elasticsearch solves this by:

• Indexing threads with fields optimized for search and filtering

• Supporting compound queries (e.g., unread + tagged + support type)

• Returning results quickly even when thousands of threads exist

It turns Firestore’s raw data into a read-optimized view of the chat system.

What We Index (and Why)

We do not index every message — only thread-level information.

Each indexed document represents a thread and contains:

• id: Thread ID (matches Firestore document)

• createdAt / updatedAt: Timestamps for sorting and freshness

• participantIds: User IDs participating in the thread

• type: ONE_TO_ONE, OPERATIONS_GROUP, or OPERATIONS_ONE_TO_ONE

• source: Where the conversation started (hububb, channex, operations, etc.)

• externalThreadId / externalSource: For OTA or channel integrations

• propertyId: Linked property (if any)

• isImportant: Whether the thread is starred or prioritized

• needsAttention: List of user IDs who have unread messages

• tags: Optional labels for categorization or workflow

• guestCoordinationActive: Indicates guest coordination team involvement

• lastMessage (snapshot): A lightweight copy of the most recent message (author, body, timestamp)

Note: Elasticsearch only mirrors summary information — full messages are always loaded from Firestore when opening a thread.

How Elasticsearch Fits Into the Chat Flow

1. Thread Creation

When a new thread is created in Firestore, it’s also added to the threads index in Elasticsearch (ELASTIC_THREADS_INDEX). If ES fails, Firestore remains the source of truth — no user data is lost.

2. New Message

When a message is sent:

• Firestore updates the thread (lastMessage, updatedAt, needsAttention, etc.).

• The updated thread is immediately re-indexed in Elasticsearch.

• The chat list, unread count, and sorting all reflect this change instantly.

If ES fails temporarily, the UI may briefly lag until the next update syncs it.

3. Read / Unread Updates

When a user reads a thread:

• Firestore clears them from needsAttention.

• Elasticsearch updates the same field (and adjusts sorting if needed).

When a user marks a thread unread:

• The user’s ID is re-added to needsAttention via a lightweight ES script update.

4. Tags & Importance

Operations and landlords can tag or star threads:

• Firestore updates tags or isImportant.

• Elasticsearch mirrors the update to keep the chat list accurate.

5. WhatsApp / External Events

When WhatsApp or OTA messages arrive:

• Firestore updates the thread and creates a message.

• Elasticsearch syncs automatically, so unread counts and sorting stay correct.

Reading from Elasticsearch

When you open the chat inbox or the Support Chat:

• The system queries Elasticsearch, not Firestore, for speed.

• Threads are filtered and sorted based on:

  • Role and permissions (operations, landlord, etc.)

  • Support mode or guest-coordinated filters

  • Tags, importance, and unread status

• After results are returned, participant and author info are fetched from the database to enrich the response.

Sorting Modes

Mode	Description
Newest	Sorts by updatedAt descending
Unread First	Unread threads (based on needsAttention) appear first, then by recent activity
Important First	Prioritized threads first, then by recent activity

Example Filtering

• Landlord View: Only threads where their uid is in participantIds.

• Operations View: All threads, but filtered by type (support, group, etc.).

• Guest Coordinator View: Threads with guestCoordinationActive = true.

What’s Not in Elasticsearch

• Full message history: Only lastMessage is stored.

• Attachments and payment content: Not indexed for search.

• Precise audit data: Stored only in Firestore for compliance.

Elasticsearch is eventually consistent — it catches up quickly but may lag by a few seconds after large updates.

Failure & Recovery Behavior

• ES writes happen after Firestore saves. If an ES update fails, the app logs an error but continues normally.

• Inconsistencies (e.g., thread appears old or unread count mismatched) are temporary.

• The next message, tag change, or periodic sync job automatically re-indexes the thread.

• A maintenance utility (updateThreadsInElastic) can re-sync all threads manually if needed.

Security & Role Access

Elasticsearch never exposes private data directly. All queries are filtered at runtime by the current user’s permissions:

Role	Visibility
Operations	Full access to all threads
Landlord	Threads where their ID appears in participantIds
Guest Coordinators	Subset where guestCoordinationActive = true

How It Helps in Practice

• Chat loads instantly even with thousands of threads.

• “Unread” and “Important” filters work in milliseconds.

• Searching for a tag or property surfaces threads immediately.

• Firestore stays lean — it handles message storage and real-time updates, not heavy queries.

Summary

Elasticsearch powers fast, searchable chat experiences in Hububb. It stores compact snapshots of threads — not the full chat history — and syncs automatically with Firestore. This combination delivers both reliability (Firestore) and speed (Elasticsearch), ensuring Support Chat and Guest Chat remain responsive and real-time even as the system scales.