SSiddiq.in
Integration as distributed-systems work

"Just call their API" hides rate limits, timeouts, retries, idempotency, contract drift, and a 3am pager.

An integration is a distributed system — two services, a fallible network, and two teams that don't talk to each other. This page is the discipline we apply so that when one side wobbles, your side doesn't fall over.

The retry that doesn't corrupt

Retries are safe only if the operation is idempotent.

Not safe to retry blindly
  • • `POST /invoices` without a client-side id
  • • `POST /email/send` without an idempotency key
  • • Payment intent creation without an intent id
  • • Any write where the second call would create a duplicate
Safe · retry all day
  • • `GET /orders/123` — same result any number of times
  • • `PUT /users/42` with the full new state
  • • `POST` with `Idempotency-Key: uuid-v4-once-per-op`
  • • `DELETE /session/abc` — already-gone is still gone
Backoff schedule we default to
100ms500ms2s8s32s→ circuit break

With ±25% jitter, so a thundering herd doesn't retry in lockstep. After 5 failures, open the circuit for 60s — stop hammering a service that's already having a bad day.

Contract testing · the thing that would have saved you

A snapshot of every API response, checked in CI.

Every integration ships with a small recorded set of real responses (with secrets stripped). CI runs Zod schemas against those recordings. When Stripe/HubSpot/whoever adds an optional field or renames one, the test fails on the next dep bump — not at 2am on Black Friday.

  • ✓ 10–30 recorded fixtures per provider
  • ✓ Zod schemas parse each one on every PR
  • ✓ Weekly re-record job catches upstream drift
  • ✓ Schema diff in the PR when we intentionally bump
// contracts/hubspot-contact.ts
export const HubspotContact = z.object({
  id: z.string(),
  properties: z.object({
    email: z.string().email(),
    firstname: z.string().optional(),
    lastname: z.string().optional(),
    lifecyclestage: z.enum([
      "subscriber", "lead", "mql", "sql",
      "opportunity", "customer", "evangelist",
    ]),
  }),
  createdAt: z.string().datetime(),
});

// tests/hubspot-contract.test.ts
for (const f of fixtures) {
  expect(() => HubspotContact.parse(f)).not.toThrow();
}
Observability defaults

Six signals · so 'it broke' becomes 'we know'.

Request-id header

One id through every hop. `grep req_abc` and you have the full story.

Structured JSON logs

Not `console.log`. Parseable, filterable, shippable.

Error tracker (Sentry)

Every unhandled error routed with breadcrumbs + user.

Uptime probes

Real requests from real regions. Not just `/health`.

Latency histograms

P50, P95, P99 per external dependency.

Circuit-breaker metrics

Alert when a dep is degraded, not just down.

Integration FAQ

Because 'the other API changed' — a field became optional, a rate limit tightened, a webhook payload shape shifted. Contract tests catch these on deploy; monitoring catches them at runtime; graceful degradation stops them from taking down your product.

Retry & idempotency playbook

Seven patterns that separate 'API call' from 'reliable API call'.

Idempotency keys
UUID per logical operation, sent as header. Downstream dedupes. Two retries = one side-effect.
Exponential backoff
Delay doubles per retry: 1s, 2s, 4s, 8s, cap at 60s. Prevents synchronized retry storms.
Jitter (±25%)
Randomize each backoff. Otherwise every client retries at the same second — a thundering herd.
Timeout budgets
Client timeout < gateway timeout < upstream timeout. Never let a slow downstream hold a caller hostage.
Circuit breaker
After N failures, stop calling for M seconds. Fail fast, recover on half-open probes.
Dead-letter queue
Requests that exhaust retries land in DLQ with full context. Humans triage; no silent loss.
Replay protection
Sign every webhook with HMAC + timestamp. Reject payloads older than 5 minutes to block replay attacks.

Get integrations that survive the vendor's bad day.

Scope one
Right tech, honest advice

Not sure if this is the right stack for you?

Tell us your goals — we'll recommend the right technology honestly, even if it isn't this one.

  • Personal reply from Siddiq within 24 hours
  • Honest advice — no pushy sales, no fake urgency
  • Clear next steps, timeline, and estimate

We only use your details to reply to this enquiry. Privacy.