email testing in ci/cd: playwright, cypress & pytest recipes

Every email test in CI is the same five steps: create a fresh inbox, trigger the flow under test, wait for the message via API, assert on the extracted content, tear down. Below are working recipes for Playwright, Cypress, and pytest, plus the GitHub Actions workflow that ties them together with EPHEMAIL_API_KEY as a secret.

Two things to do once: add the API key as a CI secret, and install the client for your language.

# in your repo settings → secrets
EPHEMAIL_API_KEY = ephm_live_...

# node
npm i -D @ephemail/sdk

# python
pip install ephemail

Never bake the key into the repo. Read it from process.env.EPHEMAIL_API_KEY (or os.environ) so local, CI, and preview environments all use their own.

The five steps, framework-agnostic:

• create — POST /v1/addresses returns a fresh address. One per test, never shared.
• trigger — drive your UI or call your signup API with that address.
• wait — POST /v1/wait-for-message long-polls until a message arrives or your timeout fires. No setTimeout.
• assert — read message.extracted.otp / message.extracted.magicLink, type it, assert success.
• cleanup — delete the inbox (or let TTL do it). On failure, print the inbox URL so a human can poke it later.

Wrap inbox lifecycle in a fixture so every test gets a clean address with zero boilerplate.

// tests/fixtures.ts
import { test as base } from "@playwright/test";
import { Ephemail } from "@ephemail/sdk";

type Fixtures = { inbox: { address: string; id: string }; ephemail: Ephemail };

export const test = base.extend<Fixtures>({
  ephemail: async ({}, use) => {
    await use(new Ephemail({ apiKey: process.env.EPHEMAIL_API_KEY! }));
  },
  inbox: async ({ ephemail }, use, testInfo) => {
    const { id, address } = await ephemail.addresses.create();
    await use({ id, address });
    if (testInfo.status !== testInfo.expectedStatus) {
      console.log(`inbox kept for debugging: https://app.ephemail.io/inbox/${id}`);
      return;
    }
    await ephemail.addresses.delete(id);
  },
});
export { expect } from "@playwright/test";

// tests/signup.spec.ts
import { test, expect } from "./fixtures";

test("user can sign up and verify with otp", async ({ page, inbox, ephemail }) => {
  await page.goto("/signup");
  await page.getByLabel("email").fill(inbox.address);
  await page.getByRole("button", { name: /create account/i }).click();

  const msg = await ephemail.waitForMessage({
    address: inbox.address,
    timeoutMs: 30_000,
    receivedAfter: new Date(Date.now() - 60_000).toISOString(),
  });

  expect(msg.extracted.otp).toMatch(/^\d{6}$/);
  await page.getByLabel("verification code").fill(msg.extracted.otp!);
  await page.getByRole("button", { name: /verify/i }).click();

  await expect(page.getByText(/welcome/i)).toBeVisible();
});

Cypress runs in the browser, so put the API calls in a cy.task that executes in Node. That keeps the API key off the page.

// cypress.config.ts
import { defineConfig } from "cypress";
import { Ephemail } from "@ephemail/sdk";

export default defineConfig({
  e2e: {
    setupNodeEvents(on) {
      const ephemail = new Ephemail({ apiKey: process.env.EPHEMAIL_API_KEY! });
      on("task", {
        async createInbox() {
          return ephemail.addresses.create();
        },
        async waitForMessage({ address, after }: { address: string; after: string }) {
          return ephemail.waitForMessage({
            address,
            timeoutMs: 30_000,
            receivedAfter: after,
          });
        },
        async deleteInbox(id: string) {
          await ephemail.addresses.delete(id);
          return null;
        },
      });
    },
  },
});

// cypress/e2e/signup.cy.ts
describe("signup", () => {
  let inbox: { id: string; address: string };
  const startedAt = new Date().toISOString();

  before(() => cy.task("createInbox").then((i: any) => (inbox = i)));
  after(() => cy.task("deleteInbox", inbox.id));

  it("verifies via otp", () => {
    cy.visit("/signup");
    cy.findByLabelText(/email/i).type(inbox.address);
    cy.findByRole("button", { name: /create account/i }).click();

    cy.task("waitForMessage", { address: inbox.address, after: startedAt }).then(
      (msg: any) => {
        expect(msg.extracted.otp).to.match(/^\d{6}$/);
        cy.findByLabelText(/verification code/i).type(msg.extracted.otp);
        cy.findByRole("button", { name: /verify/i }).click();
        cy.findByText(/welcome/i).should("be.visible");
      },
    );
  });
});

A pytest fixture gives you the same per-test isolation and automatic cleanup.

# tests/conftest.py
import os
from datetime import datetime, timezone
import pytest
from ephemail import Ephemail

@pytest.fixture(scope="session")
def ephemail():
    return Ephemail(api_key=os.environ["EPHEMAIL_API_KEY"])

@pytest.fixture
def inbox(ephemail, request):
    address = ephemail.addresses.create()
    yield address
    if request.node.rep_call.failed:
        print(f"inbox kept: https://app.ephemail.io/inbox/{address.id}")
        return
    ephemail.addresses.delete(address.id)

@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(item):
    outcome = yield
    setattr(item, f"rep_{outcome.get_result().when}", outcome.get_result())

# tests/test_signup.py
from datetime import datetime, timezone

def test_signup_otp(page, inbox, ephemail):
    started = datetime.now(timezone.utc).isoformat()
    page.goto("/signup")
    page.get_by_label("email").fill(inbox.address)
    page.get_by_role("button", name="create account").click()

    msg = ephemail.wait_for_message(
        address=inbox.address,
        timeout_ms=30_000,
        received_after=started,
    )
    assert msg.extracted.otp and msg.extracted.otp.isdigit()

    page.get_by_label("verification code").fill(msg.extracted.otp)
    page.get_by_role("button", name="verify").click()
    page.get_by_text("welcome").wait_for()

One workflow, three jobs in parallel. Each gets the same secret and runs its own framework against the same app.

# .github/workflows/e2e.yml
name: e2e
on: [push, pull_request]

jobs:
  playwright:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20, cache: npm }
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - run: npx playwright test
        env:
          EPHEMAIL_API_KEY: ${{ secrets.EPHEMAIL_API_KEY }}
          BASE_URL: ${{ secrets.PREVIEW_URL }}

  cypress:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20, cache: npm }
      - run: npm ci
      - uses: cypress-io/github-action@v6
        with: { browser: chrome }
        env:
          EPHEMAIL_API_KEY: ${{ secrets.EPHEMAIL_API_KEY }}
          CYPRESS_BASE_URL: ${{ secrets.PREVIEW_URL }}

  pytest:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.12" }
      - run: pip install -r requirements.txt
      - run: pytest -q
        env:
          EPHEMAIL_API_KEY: ${{ secrets.EPHEMAIL_API_KEY }}
          BASE_URL: ${{ secrets.PREVIEW_URL }}

• Sharing one inbox across tests. Parallel runs will steal each other's OTPs and you'll blame the network.
• setTimeout instead of long polling. You're either too slow or too flaky. waitForMessage returns the instant the message lands.
• No receivedAfter filter. Without it you'll occasionally match a stale message from a previous run on the same address.
• Tearing down on failure. Keep the inbox when a test fails and print its URL — your future self will thank you.
• Hard-coded timeouts under 10s. Most providers are sub-second, but cold senders can take 5–8s. 30s gives you slack without making green builds slow.

• One fresh inbox per test, always.
• Long-poll with waitForMessage — never setTimeout.
• Filter with receivedAfter to ignore stale messages.
• Keep the inbox on failure and log its URL for human debugging.
• Read EPHEMAIL_API_KEY from secrets — same code, local and CI.

Ready to bolt this into your pipeline? grab an api key and a throwaway inbox and copy a recipe above.

Hero image caption: every green build starts with a fresh inbox at the top of the conveyor.