Email
Research
Email engineering guide

A durable Gmail synchronization design

Gmail sync is a cursor-driven state machine: establish a baseline, advance with history records, renew mailbox watches, and deliberately recover when history is no longer available.

Research surface only. This guide documents architecture and evaluation criteria; it does not describe a released DewEngine connector.
01
Baseline

Begin with a bounded full synchronization

List the messages or threads your product actually needs, fetch required fields, persist provider IDs and the newest history ID, and page until the chosen window is complete. Do not treat the first page as the mailbox.

  • Explicit history window
  • Stable Gmail message and thread IDs
  • Lazy body and attachment retrieval
02
Incremental state

Advance from history, not timestamps

Use history.list with the last committed history ID, apply additions, deletions, and label changes, then store the returned cursor only after the local transaction succeeds. Gmail history IDs increase but are not contiguous.

03
Notifications

Treat Pub/Sub as a wake-up signal

Gmail publishes a mailbox address and new history ID through Cloud Pub/Sub. The notification does not replace history retrieval. Acknowledge deliveries, deduplicate them, and renew each watch before its expiration; Google requires watch renewal at least every seven days.

04
Recovery

Make full resynchronization a normal transition

An old history ID can return HTTP 404, and notifications can be delayed or dropped. Pause incremental processing, rebuild the bounded local view, replace the cursor, and reconcile deletes before resuming.

  • Scheduled reconciliation backstop
  • Per-account sync lease
  • Visible degraded and reconnect states
Questions

Before you build.

Does a Gmail push notification include the changed message?+

No. It identifies the mailbox and a history position. The consumer calls history.list and then retrieves the affected resources it needs.

Can the same history ID be shared across Gmail accounts?+

No. Store synchronization state per connected mailbox and never compare cursors across accounts.

Can I use DewEngine's email connectors today?+

Not yet. Gmail, Microsoft Graph, and IMAP/SMTP connectors are planned. This guide documents the intended architecture and the provider requirements a production release must satisfy.

Build with us

Does this match the workflow your users need?

DewEngine is in development. Real use cases decide what ships first.

Share your use case