Multi-Store Commerce Sync Engine

Overview

A sync engine for businesses running a network of storefronts: changes to a parent catalog fan out to child stores, and sales in the children report inventory deltas back to the parent — in near-real-time, and durably, so a webhook that arrives twice or a sync that fails halfway never corrupts state. (Shown as a sanitized capability template — no client or vendor specifics.)

Architecture

flowchart LR
  A["Parent store<br/>product / inventory change"] --> B["Webhook<br/>HMAC verified"]
  B --> C{"Already<br/>processed?"}
  C -->|"dup id"| X["Skip (idempotent)"]
  C -->|"first delivery"| D["Enqueue job<br/>Postgres queue"]
  D --> E["Worker<br/>retry / backoff / fairness"]
  E --> F["Commerce API<br/>(throttled)"]
  F --> G["Child stores<br/>products + collections"]
  G -.->|"sale"| H["Inventory delta<br/>back to parent"]

Engineering decisions

• Idempotency that survives restarts — instead of an in-memory set, the first thing each webhook does is an atomic unique-constraint insert keyed on the delivery id. First delivery wins; at-least-once duplicates lose. Idempotency becomes a database invariant, not a process memory.
• A durable queue without extra infra — a Postgres-backed job queue (pg-boss, no Redis) with priority lanes, exponential backoff, singleton dedup, and per-merchant fairness groups, running as a standalone worker decoupled from the web tier.
• Conflicts resolved by design — parent-precedence for catalog data, last-write-wins for inventory, plus compensating rollback that reverts a partial multi-step sync (e.g. undo the parent write if a downstream variant write fails).

Highlights

• Production hardening — pooled vs direct DB connections, /healthz liveness + readiness, GraphQL rate-limit throttling.
• Tested for real — an integration harness with a disposable test Postgres and a mocked commerce API, so the sync logic is exercised end-to-end.