Files
cloudflare_temp_email/worker/src/telegram_api/miniapp.ts
T
Dream HunterandClaude Opus 4.6 7c6d0d7c8a feat(mail): support gzip compressed email storage via ENABLE_MAIL_GZIP (#933)
* feat(mail): support gzip compressed email storage in D1 raw_blob column

Add ENABLE_MAIL_GZIP env var to optionally gzip-compress incoming emails
into a new raw_blob BLOB column, saving D1 storage space. Reading is
backward-compatible: prioritizes raw_blob (decompress) with fallback to
plaintext raw field. Includes DB migration v0.0.7, docs, and changelogs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: gzip fallback on missing column + decouple resolve from handleListQuery

- email/index.ts: gzip INSERT failure now falls back to plaintext INSERT
  instead of silently losing the email (P1: data loss prevention)
- common.ts: add handleMailListQuery for raw_mails-specific list queries
  with resolveRawEmailList, keeping handleListQuery generic
- Replace handleListQuery → handleMailListQuery in mails_api, admin_mail_api,
  user_mail_api (only raw_mails callers)
- Add e2e test infrastructure: worker-gzip service, wrangler.toml.e2e.gzip,
  api-gzip playwright project, mail-gzip.spec.ts with 4 test cases

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address CodeRabbit review feedback for gzip feature

- Use destructuring in resolveRawEmailRow to truly remove raw_blob key
- Narrow fallback scope: only fallback to plaintext on compression failure
  or missing raw_blob column, re-throw other DB errors
- Clean unused imports in e2e gzip test

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add try-catch in resolveRawEmail to prevent single corrupt blob from failing entire list

A corrupted raw_blob would cause decompressBlob to throw, which with
Promise.all in resolveRawEmailList would reject the entire batch query.
Now catches decompression errors and falls back to row.raw field.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(mail): align sendAdminInternalMail with gzip storage path

sendAdminInternalMail now respects ENABLE_MAIL_GZIP: compresses to
raw_blob when enabled, with fallback to plaintext on failure.
Added e2e test verifying admin internal mail is readable under gzip.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(e2e): match admin internal mail by body content instead of encoded subject

mimetext base64-encodes the Subject header, so the raw MIME string
does not contain the literal subject text. Match on body content
(balance: 99) which is plaintext.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(e2e): add WORKER_GZIP_URL guard and length assertions in gzip tests

Address CodeRabbit feedback:
- Skip gzip tests when WORKER_GZIP_URL is not set to prevent false positives
- Assert results array length before accessing [0] for clearer error messages

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(mail): narrow gzip fallback scope and fix webhook query compatibility

- sendAdminInternalMail: separate compress vs DB error handling, only
  fallback to plaintext on compression failure or missing raw_blob
  column, rethrow other DB errors (aligns with email/index.ts)
- Webhook test endpoints: use SELECT * instead of explicit raw_blob
  column reference, so pre-migration databases don't 500
- Docs/changelog: clarify that db_migration must run before enabling
  ENABLE_MAIL_GZIP

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(telegram): use generic Record type for raw_mails query result

Align with other query sites — avoid hardcoding raw_blob in the
TypeScript type annotation so the query works with or without the
column after migration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(models): add RawMailRow type and unify raw_mails query typing

Add RawMailRow type to models with raw_blob as optional field, replacing
ad-hoc Record<string, unknown> and inline type annotations across
webhook test endpoints, telegram API, and gzip utilities.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-04 18:46:39 +08:00

185 lines
6.4 KiB
TypeScript

import { Context } from "hono";
import { Jwt } from 'hono/utils/jwt'
import { CONSTANTS } from "../constants";
import { bindTelegramAddress, jwtListToAddressData, tgUserNewAddress, unbindTelegramAddress } from "./common";
import { checkCfTurnstile, checkIsAdmin, getBooleanValue } from "../utils";
import { resolveRawEmailRow } from "../gzip";
import { TelegramSettings } from "./settings";
import i18n from "../i18n";
const encoder = new TextEncoder();
const TG_AUTH_TIMEOUT = 300;
const checkTelegramAuth = async (
c: Context<HonoCustomType>, initData: string
): Promise<string> => {
// check if the request is from telegram
const initDataObj = new URLSearchParams(initData);
initDataObj.sort()
const hash = initDataObj.get('hash');
initDataObj.delete("hash");
const dataToCheck = [...initDataObj.entries()].map(([key, value]) => key + "=" + value).join("\n");
const auth_date = Number(initDataObj.get('auth_date'));
if (auth_date + TG_AUTH_TIMEOUT < (new Date().getTime() / 1000)) {
throw Error("Auth date expired");
}
const user = initDataObj.get('user');
if (!hash || !user) {
throw Error("Invalid initData");
}
const { id: userId } = JSON.parse(user);
const cryptoKey = await crypto.subtle.importKey(
"raw",
encoder.encode("WebAppData"),
{ name: "HMAC", hash: { name: "SHA-256" } },
false,
["sign"]
);
const secretKeyBuffer = await crypto.subtle.sign(
"HMAC", cryptoKey, encoder.encode(c.env.TELEGRAM_BOT_TOKEN)
);
const secretKey = await crypto.subtle.importKey(
"raw",
secretKeyBuffer,
{ name: "HMAC", hash: { name: "SHA-256" } },
false,
["sign", "verify"]
);
const calcHmac = await crypto.subtle.sign(
"HMAC", secretKey, encoder.encode(dataToCheck)
);
const calcHash = Array.from(new Uint8Array(calcHmac))
.map((b) => b.toString(16).padStart(2, "0"))
.join("");
if (calcHash != hash) {
throw Error("Invalid initData");
}
if (typeof userId === "number") {
return userId.toString();
}
return userId;
}
async function getTelegramBindAddress(c: Context<HonoCustomType>): Promise<Response> {
const { initData } = await c.req.json();
try {
const userId = await checkTelegramAuth(c, initData);
// get the address list from the KV
const jwtList = await c.env.KV.get<string[]>(`${CONSTANTS.TG_KV_PREFIX}:${userId}`, 'json') || [];
const res = [];
for (const jwt of jwtList) {
try {
const { address } = await Jwt.verify(jwt, c.env.JWT_SECRET, "HS256");
res.push({ address, jwt });
} catch (e) {
console.error(`failed to verify jwt with error: ${e}`)
continue;
}
}
return c.json(res);
}
catch (e) {
return c.text((e as Error).message, 400);
}
}
async function newTelegramAddress(c: Context<HonoCustomType>): Promise<Response> {
const { initData, address, cf_token, enableRandomSubdomain } = await c.req.json();
const msgs = i18n.getMessagesbyContext(c);
// check cf turnstile
try {
await checkCfTurnstile(c, cf_token);
} catch (error) {
return c.text(msgs.TurnstileCheckFailedMsg, 400)
}
try {
const userId = await checkTelegramAuth(c, initData);
// get the address list from the KV
const res = await tgUserNewAddress(
c,
userId,
address,
msgs,
getBooleanValue(enableRandomSubdomain)
)
return c.json(res);
}
catch (e) {
return c.text((e as Error).message, 400);
}
}
async function bindAddress(c: Context<HonoCustomType>): Promise<Response> {
const { initData, jwt } = await c.req.json();
const msgs = i18n.getMessagesbyContext(c);
try {
const userId = await checkTelegramAuth(c, initData);
await bindTelegramAddress(c, userId, jwt, msgs);
return c.json({ success: true });
}
catch (e) {
return c.text((e as Error).message, 400);
}
}
async function unbindAddress(c: Context<HonoCustomType>): Promise<Response> {
const { initData, address } = await c.req.json();
try {
const userId = await checkTelegramAuth(c, initData);
await unbindTelegramAddress(c, userId, address);
return c.json({ success: true });
}
catch (e) {
return c.text((e as Error).message, 400);
}
}
async function getMail(c: Context<HonoCustomType>): Promise<Response> {
const { initData, mailId } = await c.req.json();
const msgs = i18n.getMessagesbyContext(c);
try {
if (checkIsAdmin(c)) {
const result = await c.env.DB.prepare(
`SELECT * FROM raw_mails where id = ?`
).bind(mailId).first();
if (!result) {
return c.text("Mail not found", 404);
}
return c.json(await resolveRawEmailRow(result));
}
const userId = await checkTelegramAuth(c, initData);
const jwtList = await c.env.KV.get<string[]>(`${CONSTANTS.TG_KV_PREFIX}:${userId}`, 'json') || [];
const { addressList, addressIdMap } = await jwtListToAddressData(c, jwtList, msgs);
const result = await c.env.DB.prepare(
`SELECT * FROM raw_mails where id = ?`
).bind(mailId).first();
if (!result) return c.json(null);
const settings = await c.env.KV.get<TelegramSettings>(CONSTANTS.TG_KV_SETTINGS_KEY, "json");
const superUser = settings?.enableGlobalMailPush && settings?.globalMailPushList.includes(userId);
if (!superUser) {
if (!(result.address as string in addressIdMap)) {
return c.text(msgs.TgNoPermissionViewMailMsg, 403);
}
const address_id = addressIdMap[result.address as string];
const db_address_id = await c.env.DB.prepare(
`SELECT id FROM address where id = ? `
).bind(address_id).first("id");
if (!db_address_id) {
return c.text(msgs.TgNoPermissionViewMailMsg, 403);
}
}
return c.json(await resolveRawEmailRow(result));
}
catch (e) {
return c.text((e as Error).message, 400);
}
}
export default {
getTelegramBindAddress,
newTelegramAddress,
bindAddress,
unbindAddress,
getMail,
}