Files
cloudflare_temp_email/e2e/fixtures/test-helpers.ts
Dream Hunter 7c6d0d7c8a feat(mail): support gzip compressed email storage via ENABLE_MAIL_GZIP (#933)
* feat(mail): support gzip compressed email storage in D1 raw_blob column

Add ENABLE_MAIL_GZIP env var to optionally gzip-compress incoming emails
into a new raw_blob BLOB column, saving D1 storage space. Reading is
backward-compatible: prioritizes raw_blob (decompress) with fallback to
plaintext raw field. Includes DB migration v0.0.7, docs, and changelogs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: gzip fallback on missing column + decouple resolve from handleListQuery

- email/index.ts: gzip INSERT failure now falls back to plaintext INSERT
  instead of silently losing the email (P1: data loss prevention)
- common.ts: add handleMailListQuery for raw_mails-specific list queries
  with resolveRawEmailList, keeping handleListQuery generic
- Replace handleListQuery → handleMailListQuery in mails_api, admin_mail_api,
  user_mail_api (only raw_mails callers)
- Add e2e test infrastructure: worker-gzip service, wrangler.toml.e2e.gzip,
  api-gzip playwright project, mail-gzip.spec.ts with 4 test cases

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address CodeRabbit review feedback for gzip feature

- Use destructuring in resolveRawEmailRow to truly remove raw_blob key
- Narrow fallback scope: only fallback to plaintext on compression failure
  or missing raw_blob column, re-throw other DB errors
- Clean unused imports in e2e gzip test

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add try-catch in resolveRawEmail to prevent single corrupt blob from failing entire list

A corrupted raw_blob would cause decompressBlob to throw, which with
Promise.all in resolveRawEmailList would reject the entire batch query.
Now catches decompression errors and falls back to row.raw field.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(mail): align sendAdminInternalMail with gzip storage path

sendAdminInternalMail now respects ENABLE_MAIL_GZIP: compresses to
raw_blob when enabled, with fallback to plaintext on failure.
Added e2e test verifying admin internal mail is readable under gzip.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(e2e): match admin internal mail by body content instead of encoded subject

mimetext base64-encodes the Subject header, so the raw MIME string
does not contain the literal subject text. Match on body content
(balance: 99) which is plaintext.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(e2e): add WORKER_GZIP_URL guard and length assertions in gzip tests

Address CodeRabbit feedback:
- Skip gzip tests when WORKER_GZIP_URL is not set to prevent false positives
- Assert results array length before accessing [0] for clearer error messages

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(mail): narrow gzip fallback scope and fix webhook query compatibility

- sendAdminInternalMail: separate compress vs DB error handling, only
  fallback to plaintext on compression failure or missing raw_blob
  column, rethrow other DB errors (aligns with email/index.ts)
- Webhook test endpoints: use SELECT * instead of explicit raw_blob
  column reference, so pre-migration databases don't 500
- Docs/changelog: clarify that db_migration must run before enabling
  ENABLE_MAIL_GZIP

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(telegram): use generic Record type for raw_mails query result

Align with other query sites — avoid hardcoding raw_blob in the
TypeScript type annotation so the query works with or without the
column after migration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(models): add RawMailRow type and unify raw_mails query typing

Add RawMailRow type to models with raw_blob as optional field, replacing
ad-hoc Record<string, unknown> and inline type annotations across
webhook test endpoints, telegram API, and gzip utilities.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-04 18:46:39 +08:00

214 lines
6.6 KiB
TypeScript

import { APIRequestContext } from '@playwright/test';
import WebSocket from 'ws';
export const WORKER_URL = process.env.WORKER_URL!;
export const WORKER_URL_SUBDOMAIN = process.env.WORKER_URL_SUBDOMAIN || '';
export const WORKER_URL_ENV_OFF = process.env.WORKER_URL_ENV_OFF || '';
export const WORKER_GZIP_URL = process.env.WORKER_GZIP_URL || '';
export const FRONTEND_URL = process.env.FRONTEND_URL!;
export const MAILPIT_API = process.env.MAILPIT_API!;
export const TEST_DOMAIN = 'test.example.com';
/**
* Create a new email address via the worker API.
* Appends a timestamp suffix to avoid UNIQUE constraint collisions
* with persistent D1 data from previous test runs.
* Returns the JWT and full address string.
*/
export async function createTestAddress(
ctx: APIRequestContext,
name: string,
domain: string = TEST_DOMAIN
): Promise<{ jwt: string; address: string; address_id: number }> {
const uniqueName = `${name}${Date.now()}`;
const res = await ctx.post(`${WORKER_URL}/api/new_address`, {
data: { name: uniqueName, domain },
});
if (!res.ok()) {
throw new Error(`Failed to create address: ${res.status()} ${await res.text()}`);
}
const body = await res.json();
return { jwt: body.jwt, address: body.address, address_id: body.address_id };
}
/**
* Seed a test email by exercising the real worker email() handler
* via the admin test endpoint.
*/
export async function seedTestMail(
ctx: APIRequestContext,
address: string,
opts: { subject?: string; html?: string; text?: string; from?: string }
): Promise<void> {
const from = opts.from || `sender@${TEST_DOMAIN}`;
const subject = opts.subject || 'Test Email';
const boundary = `----E2E${Date.now()}`;
const htmlPart = opts.html || `<p>${opts.text || 'Hello from E2E'}</p>`;
const textPart = opts.text || 'Hello from E2E';
const messageId = `<e2e-${Date.now()}-${Math.random().toString(36).slice(2, 10)}@test>`;
const raw = [
`From: ${from}`,
`To: ${address}`,
`Subject: ${subject}`,
`Message-ID: ${messageId}`,
`MIME-Version: 1.0`,
`Content-Type: multipart/alternative; boundary="${boundary}"`,
``,
`--${boundary}`,
`Content-Type: text/plain; charset=utf-8`,
``,
textPart,
`--${boundary}`,
`Content-Type: text/html; charset=utf-8`,
``,
htmlPart,
`--${boundary}--`,
].join('\r\n');
const res = await ctx.post(`${WORKER_URL}/admin/test/receive_mail`, {
data: { from, to: address, raw },
});
if (!res.ok()) {
throw new Error(`Failed to seed mail: ${res.status()} ${await res.text()}`);
}
const body = await res.json();
if (!body.success) {
throw new Error(`Mail was rejected: ${body.rejected || 'unknown reason'}`);
}
}
/**
* Send a mail via admin/send_mail, which saves to sendbox.
*/
export async function sendTestMail(
ctx: APIRequestContext,
fromAddress: string,
opts: { to_mail: string; subject?: string; content?: string; is_html?: boolean }
): Promise<void> {
const res = await ctx.post(`${WORKER_URL}/admin/send_mail`, {
data: {
from_name: '',
from_mail: fromAddress,
to_name: '',
to_mail: opts.to_mail,
subject: opts.subject || 'Test Sent Mail',
content: opts.content || 'Sent mail body from E2E',
is_html: opts.is_html ?? false,
},
});
if (!res.ok()) {
throw new Error(`Failed to send mail: ${res.status()} ${await res.text()}`);
}
}
/**
* Delete all messages in Mailpit.
*/
export async function deleteAllMailpitMessages(ctx: APIRequestContext) {
const res = await ctx.delete(`${MAILPIT_API}/v1/messages`);
if (!res.ok()) {
throw new Error(`Failed to delete Mailpit messages: ${res.status()} ${await res.text()}`);
}
}
/**
* Derive the Mailpit WebSocket URL from the REST API URL.
* MAILPIT_API is like "http://mailpit:8025/api" → ws://mailpit:8025/api/events
*/
function mailpitWsUrl(): string {
return MAILPIT_API.replace(/^http/, 'ws') + '/events';
}
/**
* Wait for a message matching `predicate` to arrive in Mailpit.
*
* Connects to Mailpit's WebSocket `/api/events` and listens for
* `Type: "new"` events. When a matching message arrives, resolves
* immediately — no polling, no arbitrary sleeps.
*
* Returns `{ ready, message }`:
* - `ready` resolves when the WebSocket connection is open
* - `message` resolves with the matched message summary
*
* Usage: await ready before triggering the send to avoid race conditions.
*/
export function onMailpitMessage(
predicate: (msg: any) => boolean,
{ timeout = 10_000 }: { timeout?: number } = {}
): { ready: Promise<void>; message: Promise<any> } {
let readyResolve: () => void;
let readyReject: (err: Error) => void;
const ready = new Promise<void>((resolve, reject) => {
readyResolve = resolve;
readyReject = reject;
});
const message = new Promise<any>((resolve, reject) => {
let settled = false;
const ws = new WebSocket(mailpitWsUrl());
const timer = setTimeout(() => {
ws.close();
if (!settled) { settled = true; reject(new Error('Mailpit message not received within timeout')); }
}, timeout);
ws.on('open', () => readyResolve());
ws.on('message', (data: WebSocket.Data) => {
try {
const event = JSON.parse(data.toString());
if (event.Type === 'new' && predicate(event.Data)) {
clearTimeout(timer);
ws.close();
if (!settled) { settled = true; resolve(event.Data); }
}
} catch { /* ignore parse errors */ }
});
ws.on('close', () => {
clearTimeout(timer);
if (!settled) { settled = true; reject(new Error('Mailpit WebSocket closed before matching message')); }
});
ws.on('error', (err: Error) => {
clearTimeout(timer);
readyReject(err);
if (!settled) { settled = true; reject(err); }
});
});
return { ready, message };
}
/**
* Request send mail access for an address.
* Must be called before sending mail — creates the address_sender row
* with the DEFAULT_SEND_BALANCE configured in the worker.
*/
export async function requestSendAccess(
ctx: APIRequestContext,
jwt: string
): Promise<void> {
const res = await ctx.post(`${WORKER_URL}/api/request_send_mail_access`, {
headers: { Authorization: `Bearer ${jwt}` },
});
if (!res.ok()) {
throw new Error(`Failed to request send access: ${res.status()} ${await res.text()}`);
}
}
/**
* Delete a test address via its JWT.
*/
export async function deleteAddress(
ctx: APIRequestContext,
jwt: string
): Promise<void> {
const res = await ctx.delete(`${WORKER_URL}/api/delete_address`, {
headers: { Authorization: `Bearer ${jwt}` },
});
if (!res.ok()) {
throw new Error(`Failed to delete address: ${res.status()} ${await res.text()}`);
}
}