Deepanshu's Diary

Designing Slack-Style Delivery and Threading: Durable Logs and UX Trade-offs

--system-designmessagingreliability

Designing Slack-Style Delivery and Threading: Durable Logs and UX Trade-offs

I was reviewing a message-log export for a restaurant outreach flow when a simple but consequential tension jumped out: product wanted “instant” delivered/read indicators, but the backend must tolerate flaky mobile networks, deduplicate retransmits, and preserve thread context. If I mark messages delivered too aggressively I risk lying to users; if I wait for authoritative acknowledgements the UI feels sluggish. In this article I walk through the mental model I use—conversation as an append-only, server-authoritative log—then show a concrete architecture for durable writes, idempotent clients, delivery aggregation, thread visibility, failure modes, and the audit-safe outreach tooling I implemented.

This article is the mental model and a concrete architecture I use for Slack-style messaging (channel timelines + threaded replies), how delivery and thread visibility are handled, what I implemented for a WhatsApp-style outreach/logging project, and where this breaks.

Mental model: messages are app-level logs with two orthogonal axes

Treat a conversation as a persistent, append-only log (the timeline). Threading is a view over that log: each reply is still an entry in the same log but carries a parent pointer.

There are two independent concerns:

- Durability and ordering (server-side): ensure messages are persisted and ordered per-channel or per-thread. - Delivery state (client-side): indicate whether a recipient has received/seen a message. This is a best-effort, eventually-consistent layer built on top of durability.

Keep the server log authoritative; delivery/read states are derived metadata.

Core data shape

A compact representation keeps operations simple and idempotent.

| Field | Purpose | |---|---| | id (UUID) | Client-supplied idempotency key | | channel_id | Channel or conversation id | | parent_id | null for top-level, else parent message id (threading) | | seq | Server-assigned per-channel sequence for ordering | | client_ts / server_ts | Timestamps for sorting and diagnostics | | status | SENT / DELIVERED / READ (derived) | | edit_version | Monotonic counter for edits |

The key invariant: the server assigns seq and persists the message before broadcasting any delivery claim.

A simple architecture and data flow

Components:

- Client (mobile/web) - API gateway / Auth - Message service (write-ahead log + durable store) - Websocket gateway / push service - Fanout workers (deliver to connected clients, push providers) - Presence & cursor service (tracks who is online and read cursors) - Audit / review pipeline (consent, review UI, message logs)

Send message flow (numbered steps):

1. Client POST /messages with idempotency id, payload, and parent_id if it's a thread reply. 2. API gateway validates, forwards to Message service. 3. Message service writes to WAL and durable DB, assigns seq, records server_ts. 4. Message service returns server ack (message persisted + seq) to sender. 5. Fanout worker fetches subscribers (connected sockets, push tokens) and delivers. 6. Each recipient client, upon receiving over websocket, a) displays immediately as "delivered to device", b) sends back a per-device ack to the server. 7. Server consolidates device acks into per-user delivery state and updates message.status (DELIVERED), then broadcasts status updates. 8. When a client marks message as read (user scrolls or opens thread), client updates read cursor; server stores cursor and marks relevant messages as READ for that user.

Thread view: the UI queries messages where channel_id == X and (parent_id == null OR parent_id == thread_root_id). The seq keeps the original chronological order.

Example: thread reply visibility

- Alice posts M1 in #marketing (seq=100). - Bob replies with parent_id=M1. Server persists reply as M2 (seq=101). - In the channel timeline clients see M1 then M2. Thread UI filters by parent_id==M1 to show only replies. Because server seq preserves order, clocks aren't needed to reconstruct the original ordering.

What I built around this idea

I applied the same log-and-gating pattern while building outreach safety and message logs for WhatsApp-style flows in my repos and local tools. Specifically:

- Implemented durable message logs, tenant-scoped metadata, and protected routes for audit and review UIs (used for storing review packs and consent records) [1]. - Built a review-only pipeline that can assemble a one-page pitch, generate durable local review packs, and attach audit events to message drafts before any send action—this enforces consent/review gates [1]. - Created an adversarial safety suite and local-only approval flows so a human operator must approve before any outbound send. The actual send to real providers is still gated and not executed automatically.

What is implemented vs gated:

| Implemented | Still a gate | |---|---| | Durable logs and audit events, review UI, consent workflows [1] | Live sends to WhatsApp providers (review-only, no outbound sends) | | Local durable export of review packs | Provider acceptance, live streaming, and real push gateways (provider secrets are encrypted but full live integration is gated) |

I link to the portfolio repo for the code and experiment artifacts [1].

Where this breaks (failure modes and mitigations)

1. Network partitions cause split-brain presence: clients see stale presence and inaccurate delivered/read states. - Mitigation: treat presence as advisory, rely on persisted per-device acks for delivery; show “delivered to device(s)” not “delivered to user” until user-level cursors confirm.

2. Duplicate messages from retries (at-least-once from flaky clients). - Mitigation: require client-supplied idempotency keys (UUIDs) and server-side dedupe by id before persisting or assigning seq.

3. Out-of-order delivery to some clients (mobile buffering or delayed push notifications). - Mitigation: server assigns seq and clients reorder by seq on receipt; optionally hold rendering until missing seqs arrive or show placeholders.

4. Lost acknowledgments: client fails to send device ack after receiving due to crash. - Mitigation: heartbeat + reconcilation on reconnect; client sends a sync request containing last-seen seqs to repair state.

5. Thread conflicts on edits/deletes (concurrent edits of the same message). - Mitigation: use edit_version monotonic counters and merge rules (last-writer-wins or operational transforms for rich edits).

6. Fanout bottleneck at high fan-out channels (very large groups). - Mitigation: sharded fanout workers, push gateway offload, and lazy fanout for offline users.

A practical checklist (5–8 checks before you ship a message feature)

1. Enforce client idempotency keys and server dedupe. 2. Persist before ack: WAL -> DB -> ack to sender. 3. Assign per-channel sequence numbers for authoritative ordering. 4. Use device-level acks and consolidate into user-level delivery state. 5. Implement a reconnect sync endpoint that reconciles last-seen seqs and cursors. 6. Put consent/review gates and audit logs in the send path for regulated outreach flows. 7. Load-test fanout with realistic subscriber counts and shard accordingly. 8. Provide clear UI semantics: differentiate device-delivered vs user-read.

Conclusion

Slack-style messaging is simpler when you treat the timeline as an authoritative append-only log and keep delivery/read state as derived, eventually-consistent metadata. Threading becomes just a parent pointer and the UI is a view over the same log. The hard work is around idempotency, reconcilation on reconnect, and clear UI semantics so users aren't misled by optimistic delivery indicators.

What I built around these ideas covers durable logs, review and consent gating, and local review-pack storage—sufficient for safe audit and human-in-the-loop workflows but intentionally gated for live sends to providers [1].

References

[1] Deepanshu Verma — portfolio and repos (message logs, review UI, outreach safety artifacts): https://github.com/deepanshuvermaa/my-portfolio [2] Museum of Failure (experiments and patterns I reference during design): https://github.com/deepanshuvermaa/museum-of-failure [3] "Model Context Protocol" — a pattern for structured request/response (useful for building review/audit metadata systems): https://modelcontextprotocol.io/introduction [4] ArXiv — general system design discussions I used for thinking about guarantees: https://arxiv.org/abs/2005.11401 [5] ArXiv — consistency and design trade-offs (background reading): https://arxiv.org/abs/2307.03172

Copied!
Back to all posts