Skip to main content

Set Up an Email QA Pipeline

Email is hard to test because production email is hard to mock without diverging from reality. mailbot's sandbox inbox solves that: every test gets a real, throwaway inbox that sends and receives real email, but on the sandbox domain so nothing leaks to a customer.

This guide shows the full pattern with Jest, but the same approach works in any test runner.

Pattern​

beforeAll  → create sandbox inbox via API
test       → send mail, assert it arrived
afterAll   → disable inbox (audit trail preserved)

The inbox is short-lived, scoped to the test run, and isolated from your production traffic.

Prerequisites​

  • mailbot API key set as MAILBOT_API_KEY in test env
  • the Node SDK (@yopiesuryadi/mailbot-sdk)
  • Jest 29+ (or any runner with before/after hooks)

Setup​

// tests/email/setup.js
import { MailBot } from '@yopiesuryadi/mailbot-sdk';

export const mailbot = new MailBot({ apiKey: process.env.MAILBOT_API_KEY });

export async function createTestInbox(label) {
  const inbox = await mailbot.inboxes.create({
    username: `test-${Date.now()}`,
    labels: ['qa', label],
  });
  return inbox;
}

export async function disableTestInbox(inboxId) {
  await mailbot.inboxes.disable(inboxId);
}

A complete test​

// tests/email/onboarding-email.test.js
import { mailbot, createTestInbox, disableTestInbox } from './setup';

describe('onboarding email', () => {
  let inbox;

  beforeAll(async () => {
    inbox = await createTestInbox('onboarding');
  });

  afterAll(async () => {
    await disableTestInbox(inbox.id);
  });

  it('sends a welcome email when a user signs up', async () => {
    // trigger your app's signup flow with the test inbox address
    await fetch('https://yourapp.example.com/signup', {
      method: 'POST',
      body: JSON.stringify({ email: inbox.address }),
      headers: { 'Content-Type': 'application/json' },
    });

    // wait for the email to land
    const message = await waitForMessage(inbox.id, {
      subject: /welcome/i,
      timeoutMs: 30_000,
    });

    expect(message).toBeDefined();
    expect(message.body_text).toContain('Welcome');
    expect(message.from).toBe('hello@yourapp.example.com');
  });
});

The waitForMessage helper​

Email is asynchronous. Poll with a timeout instead of expecting an immediate result.

export async function waitForMessage(inboxId, opts = {}) {
  const { subject, fromContains, timeoutMs = 30_000, pollMs = 1000 } = opts;
  const deadline = Date.now() + timeoutMs;

  while (Date.now() < deadline) {
    const messages = await mailbot.messages.list({ inbox_id: inboxId, direction: 'inbound' });
    for (const m of messages) {
      if (subject && !subject.test(m.subject)) continue;
      if (fromContains && !m.from.includes(fromContains)) continue;
      return m;
    }
    await new Promise((r) => setTimeout(r, pollMs));
  }

  const snapshot = await mailbot.messages.list({ inbox_id: inboxId, direction: 'inbound' });
  throw new Error(
    `waitForMessage timed out. Found ${snapshot.length} messages in inbox ${inboxId}: ` +
      JSON.stringify(snapshot.map((m) => ({ subject: m.subject, from: m.from }))),
  );
}

The error includes a snapshot of what was in the inbox, which makes failures actionable. "Expected welcome email, got password-reset email" is a real bug; without the snapshot you would just see "timed out".

Asserting tracking​

Open and click events are tracked automatically. Test them by sending a message and waiting for the engagement event.

it('tracks the onboarding link click', async () => {
  const sent = await mailbot.messages.send({
    inbox_id: inbox.id,
    to: ['recipient@example.com'],
    subject: 'Onboarding',
    body_html: '<a href="https://app.example.com/onboard">Start</a>',
  });

  // simulate the click (or wait for a real one in an E2E run)
  // ...

  const events = await waitForEvent(sent.id, 'message.clicked');
  expect(events.length).toBeGreaterThan(0);
});

Practical tips​

  • Use one inbox per test file. Sharing inboxes across test files leads to flaky asserts when messages from one test bleed into another.
  • Disable, do not delete. Disabled inboxes preserve audit history. The test address is unreusable but the failure record stays.
  • Run sequentially or label aggressively. The labels field on inbox creation is your friend for filtering during debugging.
  • Cap parallelism. mailbot's sandbox burst limit is 25/min per inbox. For high-parallelism CI, create one inbox per test file rather than per test.
  • Do not test production with sandbox inboxes. The sandbox domain has its own reputation profile. Your real outbound goes through your verified domain.