Files
cloudflare_temp_email/e2e/scripts/docker-entrypoint.sh
Dream Hunter 7c6d0d7c8a feat(mail): support gzip compressed email storage via ENABLE_MAIL_GZIP (#933)
* feat(mail): support gzip compressed email storage in D1 raw_blob column

Add ENABLE_MAIL_GZIP env var to optionally gzip-compress incoming emails
into a new raw_blob BLOB column, saving D1 storage space. Reading is
backward-compatible: prioritizes raw_blob (decompress) with fallback to
plaintext raw field. Includes DB migration v0.0.7, docs, and changelogs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: gzip fallback on missing column + decouple resolve from handleListQuery

- email/index.ts: gzip INSERT failure now falls back to plaintext INSERT
  instead of silently losing the email (P1: data loss prevention)
- common.ts: add handleMailListQuery for raw_mails-specific list queries
  with resolveRawEmailList, keeping handleListQuery generic
- Replace handleListQuery → handleMailListQuery in mails_api, admin_mail_api,
  user_mail_api (only raw_mails callers)
- Add e2e test infrastructure: worker-gzip service, wrangler.toml.e2e.gzip,
  api-gzip playwright project, mail-gzip.spec.ts with 4 test cases

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address CodeRabbit review feedback for gzip feature

- Use destructuring in resolveRawEmailRow to truly remove raw_blob key
- Narrow fallback scope: only fallback to plaintext on compression failure
  or missing raw_blob column, re-throw other DB errors
- Clean unused imports in e2e gzip test

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add try-catch in resolveRawEmail to prevent single corrupt blob from failing entire list

A corrupted raw_blob would cause decompressBlob to throw, which with
Promise.all in resolveRawEmailList would reject the entire batch query.
Now catches decompression errors and falls back to row.raw field.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(mail): align sendAdminInternalMail with gzip storage path

sendAdminInternalMail now respects ENABLE_MAIL_GZIP: compresses to
raw_blob when enabled, with fallback to plaintext on failure.
Added e2e test verifying admin internal mail is readable under gzip.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(e2e): match admin internal mail by body content instead of encoded subject

mimetext base64-encodes the Subject header, so the raw MIME string
does not contain the literal subject text. Match on body content
(balance: 99) which is plaintext.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(e2e): add WORKER_GZIP_URL guard and length assertions in gzip tests

Address CodeRabbit feedback:
- Skip gzip tests when WORKER_GZIP_URL is not set to prevent false positives
- Assert results array length before accessing [0] for clearer error messages

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(mail): narrow gzip fallback scope and fix webhook query compatibility

- sendAdminInternalMail: separate compress vs DB error handling, only
  fallback to plaintext on compression failure or missing raw_blob
  column, rethrow other DB errors (aligns with email/index.ts)
- Webhook test endpoints: use SELECT * instead of explicit raw_blob
  column reference, so pre-migration databases don't 500
- Docs/changelog: clarify that db_migration must run before enabling
  ENABLE_MAIL_GZIP

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(telegram): use generic Record type for raw_mails query result

Align with other query sites — avoid hardcoding raw_blob in the
TypeScript type annotation so the query works with or without the
column after migration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(models): add RawMailRow type and unify raw_mails query typing

Add RawMailRow type to models with raw_blob as optional field, replacing
ad-hoc Record<string, unknown> and inline type annotations across
webhook test endpoints, telegram API, and gzip utilities.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-04 18:46:39 +08:00

115 lines
3.3 KiB
Bash
Executable File

#!/usr/bin/env bash
set -euo pipefail
echo "==> Waiting for worker at $WORKER_URL ..."
for i in $(seq 1 60); do
if curl -sf "$WORKER_URL/health_check" > /dev/null 2>&1; then
echo " Worker ready after ${i}s"
break
fi
if [ "$i" -eq 60 ]; then
echo "ERROR: Worker not ready after 60s"
exit 1
fi
sleep 1
done
if [ -n "${WORKER_URL_SUBDOMAIN:-}" ]; then
echo "==> Waiting for subdomain worker at $WORKER_URL_SUBDOMAIN ..."
for i in $(seq 1 60); do
if curl -sf "$WORKER_URL_SUBDOMAIN/health_check" > /dev/null 2>&1; then
echo " Subdomain worker ready after ${i}s"
break
fi
if [ "$i" -eq 60 ]; then
echo "ERROR: Subdomain worker not ready after 60s"
exit 1
fi
sleep 1
done
fi
if [ -n "${WORKER_URL_ENV_OFF:-}" ]; then
echo "==> Waiting for env-off worker at $WORKER_URL_ENV_OFF ..."
for i in $(seq 1 60); do
if curl -sf "$WORKER_URL_ENV_OFF/health_check" > /dev/null 2>&1; then
echo " Env-off worker ready after ${i}s"
break
fi
if [ "$i" -eq 60 ]; then
echo "ERROR: Env-off worker not ready after 60s"
exit 1
fi
sleep 1
done
fi
if [ -n "${WORKER_GZIP_URL:-}" ]; then
echo "==> Waiting for worker-gzip at $WORKER_GZIP_URL ..."
for i in $(seq 1 60); do
if curl -sf "$WORKER_GZIP_URL/health_check" > /dev/null 2>&1; then
echo " Worker-gzip ready after ${i}s"
break
fi
if [ "$i" -eq 60 ]; then
echo "ERROR: Worker-gzip not ready after 60s"
exit 1
fi
sleep 1
done
fi
echo "==> Waiting for frontend at $FRONTEND_URL ..."
for i in $(seq 1 60); do
if curl -skf "$FRONTEND_URL" > /dev/null 2>&1; then
echo " Frontend ready after ${i}s"
break
fi
if [ "$i" -eq 60 ]; then
echo "ERROR: Frontend not ready after 60s"
exit 1
fi
sleep 1
done
echo "==> Waiting for smtp-proxy-tls SMTP on $SMTP_PROXY_TLS_HOST:$SMTP_PROXY_TLS_SMTP_PORT ..."
for i in $(seq 1 30); do
if nc -z "$SMTP_PROXY_TLS_HOST" "$SMTP_PROXY_TLS_SMTP_PORT" 2>/dev/null; then
echo " smtp-proxy-tls SMTP ready after ${i}s"
break
fi
if [ "$i" -eq 30 ]; then
echo "WARNING: smtp-proxy-tls SMTP not ready after 30s, continuing anyway"
fi
sleep 1
done
echo "==> Initializing database"
curl -sf -X POST "$WORKER_URL/admin/db_initialize" > /dev/null
curl -sf -X POST "$WORKER_URL/admin/db_migration" > /dev/null
echo " Database initialized"
if [ -n "${WORKER_URL_SUBDOMAIN:-}" ]; then
echo "==> Initializing subdomain worker database"
curl -sf -X POST "$WORKER_URL_SUBDOMAIN/admin/db_initialize" > /dev/null
curl -sf -X POST "$WORKER_URL_SUBDOMAIN/admin/db_migration" > /dev/null
echo " Subdomain worker database initialized"
fi
if [ -n "${WORKER_URL_ENV_OFF:-}" ]; then
echo "==> Initializing env-off worker database"
curl -sf -X POST "$WORKER_URL_ENV_OFF/admin/db_initialize" > /dev/null
curl -sf -X POST "$WORKER_URL_ENV_OFF/admin/db_migration" > /dev/null
echo " Env-off database initialized"
fi
if [ -n "${WORKER_GZIP_URL:-}" ]; then
echo "==> Initializing gzip worker database"
curl -sf -X POST "$WORKER_GZIP_URL/admin/db_initialize" > /dev/null
curl -sf -X POST "$WORKER_GZIP_URL/admin/db_migration" > /dev/null
echo " Gzip worker database initialized"
fi
echo "==> Running Playwright tests"
exec npx playwright test "$@"