Skip to main content

Webhook Reliability

Webhooks are the production interface between mailbot and your code. Reliability work pays back every day: a flaky endpoint silently loses events. A solid one becomes invisible.

This page covers what your endpoint needs to do, how to verify signatures, how mailbot handles retries, and how to debug deliveries when they fail.

Endpoint design checklist​

A reliable mailbot webhook endpoint:

  • responds with 2xx within 5 seconds to acknowledge receipt
  • does not depend on synchronous downstream work to return a response
  • verifies the signature before trusting the payload
  • treats event_id as idempotent — duplicate deliveries should not change state
  • handles all expected event types and ignores unknown ones gracefully
  • logs the raw body and the signature header for debugging

If your handler does heavy processing, push the event to a queue and return 200 immediately. mailbot's timeout is short by design; long synchronous handlers cause retries that compound load.

Signature verification​

Every webhook delivery includes an X-mailbot-Signature header. The signature is HMAC-SHA256(secret, raw_body), hex-encoded.

import crypto from 'node:crypto';

function verifySignature(rawBody, signatureHeader, secret) {
  const expected = crypto
    .createHmac('sha256', secret)
    .update(rawBody)
    .digest('hex');
  // constant-time compare to resist timing attacks
  const a = Buffer.from(expected, 'utf-8');
  const b = Buffer.from(signatureHeader || '', 'utf-8');
  if (a.length !== b.length) return false;
  return crypto.timingSafeEqual(a, b);
}

Two important details:

  1. Verify against the raw body, not the JSON-parsed object. Re-stringifying after parse changes whitespace and breaks the HMAC.
  2. Use constant-time comparison (timingSafeEqual). Plain === leaks timing information.

In Express, capture the raw body before JSON parsing:

import express from 'express';
const app = express();

app.use('/webhooks/mailbot', express.raw({ type: 'application/json' }));

app.post('/webhooks/mailbot', (req, res) => {
  const raw = req.body; // Buffer
  const signature = req.header('X-mailbot-Signature');
  if (!verifySignature(raw, signature, process.env.MAILBOT_WEBHOOK_SECRET)) {
    return res.status(401).end();
  }
  const event = JSON.parse(raw.toString('utf-8'));
  // handle event ...
  res.status(200).end();
});

Retry behavior​

mailbot retries failed deliveries with exponential backoff. See Event delivery model for the full schedule. Summary:

  • 6 attempts total
  • spread across 6 hours
  • 2xx is success; everything else is failure
  • 3xx redirects are treated as failure (mailbot does not follow them)

After all retries fail, the delivery is marked failed. The event itself remains replayable. See Event replay troubleshooting.

Idempotency in your handler​

Because retries are at-least-once, your handler will sometimes see the same event_id twice. The cheapest defense is a deduplication table:

async function handle(event) {
  const inserted = await db.processed_events.insert(
    { event_id: event.event_id, processed_at: new Date() },
    { onConflict: 'do nothing' },
  );
  if (!inserted.rowCount) return; // already processed
  await applyEvent(event);
}

processed_events becomes a permanent record of every delivery you accepted. Index on event_id and prune older than your retention window.

Common failure modes​

SymptomLikely causeFix
401 from your endpointSignature verification mismatchCheck secret value, verify against raw body, use constant-time compare
TimeoutsSync work in handlerMove work to a queue; return 200 immediately
Duplicate state changesHandler not idempotentAdd event_id dedupe table
Endpoint is up but 4xxStrict body validation rejecting unknown fieldsLoosen schema; ignore unknown fields
Endpoint returns 200 but downstream stays emptySilent error after the responseAdd structured logging on the actual handler logic

Debugging a specific delivery​

You have two views to combine:

  1. mailbot side: list deliveries for the inbox via the deliveries log endpoint. Each row shows event_id, attempt count, last response status, last response body.
  2. Your side: structured logs keyed by event_id.

When a delivery looks failed on mailbot but processed on your side (or vice versa), you immediately know which half is lying.

Practical tips​

  • Use one secret per webhook destination. If you compromise one, rotate just that one.
  • Test with replay first. Before changing handler code in production, replay a known event and verify the response.
  • Alert on failed deliveries, not on attempts. mailbot's retry schedule is generous. A single retry is not an incident; six failed attempts is.
  • Keep Content-Type: application/json strict. Anything else is suspicious.
  • Do not run untrusted code based on payload contents. Treat payload like any other user-controlled input.