Compare commits

..

75 Commits

Author SHA1 Message Date
krau
942022969c style: format codes 2026-08-17 19:01:51 +08:00
krau
ab89310325 fix: drop unbounded saved-path cache in telegram storage
Telegram cannot reliably query remote file existence, so the
cache answered a question it could not answer and grew without
bound. Exists returns false again, as before.
2026-08-16 22:05:08 +08:00
krau
b27bb734be style: gofmt test files 2026-08-16 21:56:01 +08:00
krau
fc0c8975a0 style: trim verbose comments
Drop process-style explanations; keep one-line behavior notes.
2026-08-16 21:56:01 +08:00
krau
524a259ae4 test: exercise the alist 401 refresh path
Concurrent uploads now reject the init token, share one
refresh and retry with the new token; token-only stays inert.
2026-08-16 21:51:11 +08:00
krau
e8257c46d2 fix: refresh alist token after startup 401s
The init login no longer satisfies the dedup window, so an
early 401 triggers a real refresh; later 401s reuse it.
Clarify that streaming uploads cannot replay their body.
2026-08-16 21:51:11 +08:00
krau
6a11b8ac8d test: cover cancellation with IgnoreErrors 2026-08-16 21:44:40 +08:00
krau
47d941d851 test: cover notifyProgress tracker call 2026-08-16 21:44:40 +08:00
krau
7e502442c5 fix: propagate cancellation past IgnoreErrors
Cancelled tasks must not be reported as successful: only
ordinary element failures are ignored.
2026-08-16 21:44:35 +08:00
krau
a9425cb5ea fix: call tracker from notifyProgress instead of recursing
The helper called itself, overflowing the stack on any batch
task with a progress tracker.
2026-08-16 21:44:35 +08:00
krau
6ad2a2c884 test: cover alist refresh semantics
Concurrent refresh uses username/password; token-only storage
never attempts a login on 401.
2026-08-16 21:35:21 +08:00
krau
0d42c9f23d fix: skip login refresh for token-only alist storage
401 responses surface the auth error instead of sending a
credential-less login; the refresh window never exceeds TokenExp.
2026-08-16 21:35:21 +08:00
krau
4c8f35ae80 fix: report partial failure in batch done message
IgnoreErrors runs with failed elements show success and failed
counts instead of claiming every file completed.
2026-08-16 21:35:14 +08:00
krau
0a67b8bb57 test: cover nil-tracker task execution 2026-08-16 21:35:14 +08:00
krau
196bb9941f fix: keep album order when filtering failed batch items
Download results are stored by original index so surviving
elements keep their source order.
2026-08-16 21:35:14 +08:00
krau
973f3499a9 fix: guard nil progress trackers in api-created tasks
Batch, telegraph and transfer tasks run without a Telegram tracker
when created through the API; their callbacks must not panic.
2026-08-16 21:34:52 +08:00
krau
d9cb5f9cfc fix: deduplicate concurrent storage initialization
singleflight merges first-time inits so side effects are not duplicated.
2026-08-16 21:20:12 +08:00
krau
ccb702148e fix: localize storage lookup errors in /dir
Use the shared i18n key and escape the dynamic error.
2026-08-16 21:20:12 +08:00
krau
85b1f35dc5 fix: count parsed resources in progress text 2026-08-16 21:20:12 +08:00
krau
9f93a95258 test: cover concurrent alist 401 retry
Ten parallel uploads share a single re-login under -race.
2026-08-16 21:20:12 +08:00
krau
189bf9c736 fix: deduplicate alist token refreshes
Guard token access with a mutex and merge concurrent logins.
Reuse a recent refresh to avoid login storms.
2026-08-16 21:20:11 +08:00
krau
534ed7a7c2 fix: record telegram saved paths only after upload
Skip-large returns a sentinel so skipped files are not marked as saved.
2026-08-16 21:20:11 +08:00
krau
1d4997ba2d fix: upload only downloaded batch elements
Failed elements keep partial cache files and never reach the backend.
Successfully downloaded siblings still upload when one element fails.
2026-08-16 21:20:11 +08:00
krau
f7e532ca7e fix: make cache init idempotent 2026-08-16 20:30:06 +08:00
krau
0957d93da3 chore: ignore cache directory 2026-08-16 20:30:06 +08:00
krau
324d8c100f ci: fix BuildTime formatting and align checkout
Actions format does not format dates; pass the raw timestamp.
2026-08-16 20:28:45 +08:00
krau
4cb23f04b6 docs: document parser plugin config 2026-08-16 20:28:45 +08:00
krau
a92ca75c4f refactor: drop unused hook config 2026-08-16 20:28:45 +08:00
krau
4e99b4bdc9 fix: load remote config without local lookup
Skip the local file search after reading a config URL and add a timeout.
2026-08-16 20:28:45 +08:00
krau
389be59371 refactor: remove dead code from api and bot
Drop the empty ProgressTracker shim, unused token context key and a redundant SetBotCommands call.
2026-08-16 20:28:39 +08:00
krau
0d49ae94af fix: add timeouts and backoff to webhook delivery 2026-08-16 20:28:39 +08:00
krau
48e739f06d fix: fail fast when API listen fails
Bind synchronously and surface errors instead of logging them.
2026-08-16 20:28:39 +08:00
krau
b4fbbae068 fix: notify users on invalid update version 2026-08-16 20:28:33 +08:00
krau
29be3ee90c fix: avoid leaking raw errors in /dir reply 2026-08-16 20:28:33 +08:00
krau
56660a7705 fix: initialize userbot context once
Replace the racy lazy init with sync.OnceValue.
2026-08-16 20:28:33 +08:00
krau
0dfb6af153 fix: require permission for callback handlers 2026-08-16 20:28:33 +08:00
krau
63f53fac7f fix: isolate media groups per user
Key pending groups by chat, user and group id.
2026-08-16 20:28:33 +08:00
krau
607ec8aced fix: guard callback data parsing
Reject malformed callback payloads before indexing split parts.
2026-08-16 20:28:33 +08:00
krau
cd9886d710 fix: show all queued tasks in /task list
Render up to ten tasks and append the truncation note once.
2026-08-16 20:28:33 +08:00
krau
3f7f50133e test: cover invalid plugin version rejection 2026-08-16 20:28:27 +08:00
krau
c7911cc208 refactor: remove commented-out dead code
Drop unused kemono legacy types and commented response structs.
2026-08-16 20:28:27 +08:00
krau
b31d628c19 fix: guard kemono parser against nil fields
Skip sparse preview and attachment entries instead of panicking.
2026-08-16 20:28:27 +08:00
krau
60fd2e04d9 fix: harden JS parser plugin runtime
Validate semver instead of panicking and require canHandle.
Recover plugin worker panics and time out CanHandle calls.
Return a copy from the registry and sanitize install filenames.
2026-08-16 20:28:27 +08:00
krau
b72dd67be9 fix: guard storage registry maps
Protect Storages and UserStorages with mutexes and expose read accessors.
2026-08-16 20:28:21 +08:00
krau
aa25eb1510 fix: enforce telegram album limits and track saved paths
Use tglimit.MaxAlbumItems for album batching and video splitting.
Exists now reports previously saved paths instead of always false.
2026-08-16 20:28:21 +08:00
krau
ba0deababc fix: fail alist init instead of exiting
Replace log.Fatalf with wrapped errors so a bad alist cannot kill the bot.
Cancel token refresh with the init context and re-login on 401.
2026-08-16 20:28:21 +08:00
krau
d1b2dbfe5f fix: kill rclone subprocess on reader close
Prevent cat processes from hanging after the pipe is closed.
2026-08-16 20:28:15 +08:00
krau
bd7da7c31e fix: preserve webdav error causes
Wrap mkdir and write failures with %w and drop dead error values.
2026-08-16 20:28:15 +08:00
krau
1981d8662b fix: sanitize local storage paths
Reject absolute paths and dot-dot escapes in Save.
Check close errors and wrap creation failures.
2026-08-16 20:28:15 +08:00
krau
51fcd17922 refactor: share unique filename logic
Storage backends use fsutil.UniquePath instead of local loops.
2026-08-16 20:28:15 +08:00
krau
610f586d45 refactor: share progress throttling helpers
Move size-tiered and count-based throttling into progressutil.
Localize hardcoded Chinese progress strings.
Drop five duplicated implementations and dead local copies.
2026-08-16 20:28:07 +08:00
krau
6bc12d6feb fix: validate i18n key parity across locales
geni18n fails when a language file misses any key.
Align the syncpeers completion key between en and zh-Hans.
Translate three untranslated parse keys in zh-Hans.
2026-08-16 20:28:03 +08:00
krau
ad41aec22c fix: report streamed upload bytes in batch tasks
Stream downloads report their byte count as the upload total.
2026-08-16 20:27:53 +08:00
krau
4389bfe046 fix: honor IgnoreErrors in batch tasks
Element failures no longer cancel sibling elements or stop later groups.
2026-08-16 20:27:42 +08:00
krau
832eb27d4f fix: harden per-item processing tracking in tasks
Use the resource fingerprint as the dedup key on insert and delete.
Check and set processing entries atomically instead of TOCTOU.
Count only successful downloads.
2026-08-16 20:27:42 +08:00
krau
b6e981282d fix: make resource fingerprints deterministic
Sort map keys before hashing so Resource.ID is stable.
2026-08-16 20:27:42 +08:00
krau
9ed5b5f962 fix: init task queue lazily and close on shutdown
AddTask is safe before Run by initializing the queue once.
Close unblocks workers waiting in Get.
2026-08-16 20:27:26 +08:00
krau
4527022853 fix: prevent queue deadlock after cancelling tasks
Get no longer recurses while holding the mutex.
Cancelled queued tasks leave the map, making their IDs reusable.
Closed empty queues return ErrQueueClosed.
2026-08-16 20:27:26 +08:00
krau
6fd95bbe1b chore: update agents instruction 2026-08-11 09:04:15 +08:00
krau
a1bd2ed762 fix: upgrade deps 2026-08-11 08:57:09 +08:00
krau
7e3e26fcf4 style: format codebase 2026-08-11 08:47:35 +08:00
Haopeng Huo
e49ebef977 fix(telegraph): parse page URLs from message entities (#232) 2026-08-11 08:43:46 +08:00
Haopeng Huo
da38f5bb88 fix: remove cache files after reader closes (#231) 2026-08-11 08:41:40 +08:00
Haopeng Huo
0e6ed66ef5 feat: show upload progress for batch file tasks (#228)
* feat: show upload progress for file tasks

* feat: show upload progress for batch file tasks

* style: distinguish download and upload phases

* style: mark successful task completion

* fix: preserve storage save error context

* fix: serialize single-file progress updates

* fix: stabilize batch upload progress reporting

Correct batch upload totals, completion state, actual file sizes, and synchronized progress snapshots. Add regression coverage for concurrent updates and phase transitions.

* feat: show per-file transfer progress

Format single-file and batch download and upload states with Telegram entities, blockquotes, speeds, transferred sizes, progress bars, concise counters, and accurate confirmation handling. Cover upload retries plus final, error, and cancellation messages with regression tests.

* chore: remove transfer progress tests

* test: restore critical transfer progress coverage

* test: cover interleaved batch transfers

* fix: declare progress styles in locale templates
2026-08-11 08:40:29 +08:00
Haopeng Huo
e4144e73e6 feat(telegram): split oversized videos into playable parts (#229)
* feat(telegram): split oversized videos losslessly

* fix(telegram): respect Premium upload limits

* fix(telegram): keep split videos in one album
2026-08-05 16:57:46 +08:00
Haopeng Huo
1d794a7b9c feat(telegram): preserve source media groups (#226)
* feat(telegram): preserve source media groups

* fix(telegram): rewind batch readers before inspection
2026-08-04 16:46:29 +08:00
irudisca
52f880f0f2 fix(telegram): use seekable file path for thumbnail & metadata extraction (#225)
extractFrameAt() and getVideoMetadata() piped the media into ffmpeg/ffprobe
via pipe:0. A pipe is not seekable, so ffmpeg cannot decode non-faststart MP4s
whose moov atom sits at the END of the file (very common for yt-dlp / HLS-merged
downloads): the frame grab and probe silently fail, and the video is uploaded
with no thumbnail (and, for non-mp4 containers, no dimensions).

Hand ffmpeg/ffprobe a seekable file path instead. When the reader already is an
*os.File we use it directly; otherwise we spool to a temp file and clean it up.

Verified with an A/B upload of a 56 MB non-faststart clip: stock build produced
no thumbnail, patched build produced a correct thumbnail.

Co-authored-by: pennyucloud <valentino@pennyu.co.id>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 18:28:03 +08:00
krau
fc11ca775f fix: update message link parsing to support both t.me and telegram.me domains 2026-07-14 12:53:09 +08:00
krau
056e2fd546 fix: add ca-certificates installation and environment variables for SSL support in Dockerfile, close #222 2026-07-06 11:34:28 +08:00
krau
c9bb6c9e3c feat(docs): add CLI subcommands documentation and file naming strategies
- Introduced new documentation for CLI subcommands: `upload` and `watch`, detailing their usage, flags, and examples.
- Added a section on file naming and conflict strategies, explaining how users can customize file names and handle duplicates via `/config` and `/fnametmpl` commands.
- Translated the new documentation into Chinese for broader accessibility.
2026-06-28 18:00:02 +08:00
krau
2bc460c609 feat: add preset rule import functionality and update related messages 2026-06-26 15:39:14 +08:00
Krau
f02860ff3f feat: implement task event system for progress tracking and reporting (#220) 2026-06-25 21:37:36 +08:00
krau
9c2e70ed43 feat: upgrade deps 2026-06-25 16:30:21 +08:00
krau
3d6cd45909 feat: add configuration options for video download settings 2026-06-25 16:17:03 +08:00
Krau
77ef3154cf feat: add directory watching and auto-upload functionality (#219) 2026-06-25 14:34:59 +08:00
140 changed files with 6839 additions and 1780 deletions

View File

@@ -43,7 +43,7 @@ jobs:
goarch: arm64
steps:
- name: Checkout
uses: actions/checkout@v4
uses: actions/checkout@v6
- name: Extract version from Git Ref
id: extract_version
@@ -64,7 +64,7 @@ jobs:
ldflags: >-
-s -w
-X "github.com/krau/SaveAny-Bot/config.Version=${{ env.VERSION }}"
-X "github.com/krau/SaveAny-Bot/config.BuildTime=${{ format(github.event.repository.updated_at, 'yyyy-MM-dd HH:mm:ss') }}"
-X "github.com/krau/SaveAny-Bot/config.BuildTime=${{ github.event.repository.updated_at }}"
-X "github.com/krau/SaveAny-Bot/config.GitCommit=${{ github.sha }}"
binary_name: saveany-bot
env:

1
.gitignore vendored
View File

@@ -1,5 +1,6 @@
config.toml
logs/
/cache/
tmp/
data/
downloads/

366
AGENTS.md
View File

@@ -1,301 +1,115 @@
# SaveAny-Bot Agent Guidelines
This document provides essential information for AI coding agents working on the SaveAny-Bot project.
# Repository Guidelines
## Project Overview
SaveAny-Bot is a Telegram bot written in Go that saves files/messages from Telegram and various websites to multiple storage backends (local, S3, MinIO, WebDAV, AList, Telegram). It features a plugin system for parsing web content and extensible storage backends.
SaveAny-Bot is a Telegram bot written in Go that saves files and messages from Telegram and websites to multiple storage backends (Local, S3, MinIO, WebDAV, AList, Rclone, Telegram). It supports single-file saves, batch/album saves, streaming, multi-user access, storage rules, cross-storage transfers, yt-dlp and Aria2 downloads, and a Goja-based JavaScript parser plugin system with optional Playwright browser automation.
**Tech Stack**: Go 1.24.2, gotd/td (Telegram MTProto), Cobra (CLI), Viper (config), GORM (ORM), SQLite, Goja (JS runtime), Playwright (browser automation)
**Tech stack**: Go 1.25, gotgproto + gotd/td v0.149.0 (MTProto), Cobra (CLI), Viper (config), GORM + SQLite, Goja (JS runtime), Playwright, charmbracelet/log. License: AGPL-3.0.
## Build & Test Commands
**Note on gotd versions**: `gotd/td` must stay on `v0.149.0` — v0.150+ breaks `gotgproto` (v1.0.0-beta22) compilation (`AsInputDocumentFileLocation` signature, `gotd/log` Logger interface). Do not bump beyond v0.149.
## Architecture & Data Flow
```
Telegram update → client/bot/handlers → core.AddTask(Executable) → pkg/queue (serial workers)
→ core/tasks/* (download via common/tdler) → storage.Storage → progress feedback
```
- **Startup sequence** (`cmd/run.go::initAll`, keep this order): Config → Cache → i18n → Database → Storage → Parser plugins → Userbot → API → Bot. `bot.Init` returns the exit channel; a `SAVEANTBOT-RESTART` error restarts the process (external supervisor).
- **Task pipeline**: handlers build a task (via `core.AddTask`), the queue executes `Executable{Type, Title, TaskID, Execute(ctx)}` with `config.C().Workers` workers. Lifecycle hooks (`TaskBeforeStart/Success/Fail/Cancel`) run around `Execute`. Cancellation = canceling the task's context; tasks must check `ctx.Err()`.
- **Dual progress channels**: (1) `pkg/taskevent` context bus (consumed by `api/` for HTTP/Webhook consumers — `taskevent.WithSink` must be injected for API tasks), (2) Telegram message edits via `ProgressTracker` + `tgutil.ExtFromContext(ctx)` (bot tasks). Upload progress currently reaches the Telegram channel only.
- **Capability interfaces + type assertion fallback** is the core extensibility pattern: `StorageBatchSaver`, `StorageProgressSaver` (upload progress), `StorageListable`, `StorageReadable` are optional; consumers assert and fall back (e.g. wrap reader with `ioutil.NewProgressReader`). New backends only need to implement the interface and register.
- **Config layering**: CLI flag > env `SAVEANY_*` (dots → underscores, e.g. `SAVEANY_TELEGRAM_TOKEN`) > TOML file (path or http(s) URL). `config.C()` returns a **copy** — never mutate it.
- **Storage registration (3 places)**: `pkg/enums/storage` ENUM comment (go-enum), `config/storage/factory.go::storageFactories` (config struct with `Validate()`), `storage/storage.go::storageConstructors` (implementation).
## Key Directories
| Path | Purpose |
|---|---|
| `cmd/` | CLI: `run` (main bot), `upload`, `watch` (standalone subcommands that do NOT run initAll), `geni18n` (i18n key generator) |
| `core/` | `Executable` interface, queue worker loop, hooks; `core/tasks/{tfile,batchtfile,directlinks,parsed,telegraph,transfer,ytdlp,aria2dl}` |
| `client/bot/` | gotgproto client, `handlers/` (all commands + message/callback handlers), `middleware/`, `client/user/` (userbot) |
| `storage/` | 8 backends + `storage.go` (interfaces/registry) + `load.go` (per-user storage resolution) |
| `parsers/` | `parsers.go` (registry), `js/` (Goja plugins, ghttp/playwright injection, build-tagged), `parsers/` (native: twitter, kemono) |
| `config/` | Viper setup, defaults, `storage/` per-type config structs |
| `database/` | GORM models (User/Dir/Rule/WatchChat), AutoMigrate, `syncUsers` |
| `pkg/` | `queue`, `taskevent`, `tcbdata` (callback data), `rule`, `enums/{tasktype,storage,ctxkey,fnamest}`, `storagetypes`, `tfile`, `parser` |
| `common/` | `tdler` (unified downloader), `utils/{tgutil,dlutil,ioutil,fsutil,strutil,tphutil,netutil}`, `i18n` (embedded locales), `cache` (ristretto) |
| `api/` | HTTP API + webhook (task factory with sink injection) |
| `docs/` | Hugo site (hugo-book theme, zh+en mirrored), separate go.mod |
| `plugins/` | JS parser examples + `README.md` (plugin author contract) |
## Development Commands
### Build
```bash
# Standard build
go build -o saveany-bot .
# Run directly
# Build (standard; CGO_ENABLED=0 for static)
CGO_ENABLED=0 go build -trimpath -o saveany-bot .
go run ./cmd
# Docker build (multi-stage, Alpine-based)
docker build -t saveany-bot .
docker compose up -d
```
### Test
```bash
# Run all tests
# Test — known failures: storage/telegram TestCreateSplitZip/TestExtractThumbFrame/TestGetVideoMetadata
# (need gitignored fixtures tests/testfile.dat, tests/testvideo; ffmpeg/ffprobe)
go test ./...
# Run tests in specific package
go test ./pkg/queue
go test ./storage/telegram
# Run tests with verbose output
go test -v ./...
# Run a single test
go test -race ./core/tasks/... ./storage/... ./pkg/queue/... ./common/...
go test -run TestQueueBasic ./pkg/queue
# Run with coverage
go test -cover ./...
```
# Codegen — run after editing locale YAML or enum comments
go generate ./... # geni18n (i18nk keys) + go-enum (pkg/enums/*)
# go-enum is NOT in go.mod; install externally. geni18n runs via go run.
### Lint & Format
```bash
# Format code (standard Go formatting)
go fmt ./...
# Vet code for common issues
# Verify
go vet ./...
# Generate code (i18n keys)
go generate ./...
go fmt ./...
```
### Other Commands
```bash
# Update dependencies
go mod tidy
**Build variants** (Dockerfile.default/micro/pico): `-tags=no_jsparser,no_playwright,no_minio,no_bubbletea,sqlite_glebarez` — each has a `*_stub.go`/`*_glebarez.go` pairing; keep stubs in sync.
# View documentation
cd docs && hugo server -D
```
Docker: `docker build -t saveany-bot .`, `docker compose up -d` (host network, mounts `./data ./config.toml ./downloads ./cache`). CI (`.github/workflows/`) runs **no tests/lint** — only tag-triggered release/docker builds and docs deployment; run `go test ./...` manually before pushing.
## Code Style Guidelines
## Code Conventions & Common Patterns
### Imports
- Standard library first, then third-party, then project-internal
- Group imports with blank lines between groups
- Use explicit import aliases for clarity when needed (e.g., `storconfig`, `storenum`)
- **Imports**: stdlib → third-party → project-internal, blank-line separated. Aliases for clarity (`storconfig`, `storenum`).
- **Naming**: PascalCase exported, camelCase unexported, files `snake_case.go`; **not** ALL_CAPS constants.
- **Errors**: always wrap with `fmt.Errorf("context: %w", err)`; check with `errors.Is/As`; never ignore.
- **Logging**: `log.FromContext(ctx)` with prefixes (`logger.WithPrefix("component")`); never global logger when ctx is available.
- **Context values** (read from the passed ctx, never globals): `log.FromContext`, `tgutil.ExtFromContext` (Telegram ext — **required for message edits; if nil, edits are silently dropped**), `storage.FromContext`, `storagetypes.WithSourceCaption`, `ctxkey.ContentLength` / `ctxkey.OverwriteExisting`.
- **Progress rendering** (#228 convention): i18n templates declare styles with Telegram HTML (`<b>/<code>/<blockquote>/<i>`); dynamic data MUST go through `i18n.T(key, tgutil.EscapeHTMLTemplateData(data))` before `tgutil.RenderHTML`. Never interpolate user data raw, never render-then-substring-search.
- **Progress tracking**: each task package defines its own small `ProgressTracker` interface; optional `UploadProgressTracker` is probed via type assertion (skip if absent). Serialize state + message edits with a mutex; throttle edits (≥1s); aggregate per-item progress monotonically.
- **i18n**: only edit `common/i18n/locale/{zh-Hans,en}.yaml``go generate ./...` → use `i18nk.<Key>` constants. No raw strings in user-facing messages. zh-Hans and en must stay in sync.
- **Registration points** (never forget): new bot command → `client/bot/handlers/register.go::CommandHandlers` (auto-publishes /help menu); new task type → `pkg/enums/tasktype` + `core/tasks/<name>/` + `api/factory.go::CreateTask`; new storage → 3 places above + `docs/content/{en,zh}/deployment/configuration/storages.md`; new enum value → ENUM comment + `go generate`.
- **Concurrency**: `errgroup.WithContext` + `SetLimit(config.C().Workers)`, `atomic.Int64` counters, `sync.Once` for single-shot events, mutex around render state. No lock-in-callback (callbacks fire after unlock).
- **Cancellation**: queue tasks carry a `WithCancel`-derived ctx; check `ctx.Err()` in loops; classify with `errors.Is(err, context.Canceled)`.
- **JS plugins**: `registerParser({metadata, canHandle, parse})`, `version >= 1.0.0`; per-plugin goja VM is single-goroutine (reqCh buffer 10). Changing `pkg/parser.Item/Resource` JSON fields requires updating `plugins/README.md` and example plugins.
- **Message edits**: `ext.EditMessage(chatID, &tg.MessagesEditMessageRequest{...})`; cancel buttons via `tgutil.BuildCancelButton(taskID)`; callback payloads via `pkg/tcbdata` + `common/cache`.
```go
import (
"context"
"fmt"
"github.com/charmbracelet/log"
"github.com/krau/SaveAny-Bot/config"
"github.com/krau/SaveAny-Bot/pkg/enums/storage"
)
```
## Important Files
### Formatting
- Line length: reasonable (no hard limit, but be sensible)
- Organize code with blank lines between logical sections
- Follow standard Go conventions for braces, spacing, etc.
- `main.go``//go:generate` for i18n keys
- `cmd/run.go` — startup sequence `Run/initAll/cleanCache` (cache cleanup on exit, `NoCleanCache` opt-out)
- `core/core.go` — worker loop, hooks, AddTask/CancelTask
- `pkg/queue/queue.go` — generic serial queue (cond/list; duplicate TaskID rejected)
- `storage/storage.go` — interfaces + registry + compile-time capability assertions
- `config/viper.go`, `config.example.toml` — config schema (authoritative field docs)
- `database/db.go` — GORM init, `GetDialect` (build-tag selectable SQLite driver)
- `client/bot/handlers/register.go` — handler dispatch order and CommandHandlers
- `common/tdler/dler.go` — unified download entry
- `core/tasks/batchtfile/item_progress.go` — per-item phase state machine (Downloading/Transferring/Uploading/Retrying/Confirming, FailureStage)
- `parsers/js/plugin.go` — Goja plugin runtime
- `.github/workflows/` — release/docker/docs (no test gate)
### Types & Interfaces
- Use clear, descriptive type names (PascalCase for exported, camelCase for unexported)
- Define interfaces where abstraction is needed (e.g., `Executable`, `StorageConfig`)
- Embed context in method signatures, not structs: `func (s *Service) Do(ctx context.Context) error`
- Prefer composition over inheritance
## Runtime/Tooling Preferences
```go
// Interfaces define behavior
type Executable interface {
Type() tasktype.TaskType
Title() string
TaskID() string
Execute(ctx context.Context) error
}
- **Go 1.25+**: `t.Context()`, `sync.WaitGroup.Go`, `for range n` are available.
- **Runtime binaries**: ffmpeg/ffprobe (media processing/video split), yt-dlp (ytdlp tasks), aria2 optional; Playwright browsers install on demand to `./playwright` (`playwright.Install(chromium, ...)` at first `pw.get()`); Docker images: default has ffmpeg+yt-dlp, micro only curl, pico is scratch static.
- **No Makefile, no golangci.yml, no test/lint CI** — verification is manual (`go vet`, `go test`).
- **go-enum** required externally for enum generation; **geni18n** is in-repo.
- **Docs**: Hugo site in `docs/` (separate go.mod, hugo-book); edit `docs/content/{zh,en}/` — keep both languages mirrored. `docs/public/` is gitignored build output.
- **gitignored fixtures**: `storage/telegram/tests/` (missing — 3 tests fail locally), `data/`, `config.toml`, `playwright/`, `testplugins/`.
// Structs compose behavior
type Local struct {
config config.LocalStorageConfig
logger *log.Logger
}
```
## Testing & QA
### Naming Conventions
- **Packages**: lowercase, single word when possible (avoid underscores)
- **Files**: lowercase with underscores for multiword (e.g., `auth_terminal.go`, `progress_reader.go`)
- **Variables**: camelCase for unexported, PascalCase for exported
- **Constants**: PascalCase for exported, camelCase for unexported (not ALL_CAPS)
- **Functions/Methods**: PascalCase for exported, camelCase for unexported
- **Test files**: `*_test.go` pattern
### Error Handling
- Always handle errors explicitly; never ignore them
- Wrap errors with context using `fmt.Errorf("context: %w", err)`
- Use `errors.Is()` and `errors.As()` for error checking
- Log errors with appropriate level (Error, Warn, Info)
- Return errors from functions rather than panicking (except for truly unrecoverable situations)
```go
// Good error handling
if err := db.Save(user).Error; err != nil {
return fmt.Errorf("failed to save user %d: %w", user.ChatID, err)
}
// Check specific errors
if errors.Is(err, context.Canceled) {
logger.Info("Operation was canceled")
return nil
}
```
### Logging
- Use `github.com/charmbracelet/log` package
- Get logger from context: `log.FromContext(ctx)`
- Create prefixed loggers for components: `logger.WithPrefix("component")`
- Use appropriate levels: Debug, Info, Warn, Error
- Include context in log messages (e.g., task IDs, file names)
```go
logger := log.FromContext(ctx)
logger.Infof("Processing task: %s", task.ID)
logger.Errorf("Failed to save file %s: %v", filename, err)
```
### Concurrency
- Use channels for communication between goroutines
- Protect shared state with `sync.Mutex` or `sync.RWMutex`
- Use `sync.WaitGroup` for coordinating goroutine completion
- Always pass `context.Context` for cancellation support
- Use `context.WithCancel/WithTimeout` for managing goroutine lifetimes
```go
// Example from queue implementation
func (tq *TaskQueue[T]) Add(task *Task[T]) error {
tq.mu.Lock()
defer tq.mu.Unlock()
// ... critical section
tq.cond.Signal()
return nil
}
```
### Comments
- Document exported types, functions, and packages with doc comments
- Start doc comments with the name being documented
- Use `//` for single-line comments
- Explain *why*, not *what* (code should be self-explanatory for "what")
- Add `[NOTE]`, `[WARN]`, `[IMPORTANT]` tags for important clarifications
```go
// GetUserByChatID retrieves a user by their Telegram chat ID.
// Returns an error if the user is not found.
func GetUserByChatID(ctx context.Context, chatID int64) (*User, error) {
```
## Architecture & Conventions
### Application Structure
- **Entry point**: `main.go``cmd.Execute(ctx)`
- **CLI root**: `cmd/root.go` (Cobra), implementation in `cmd/run.go`
- **Startup sequence**: Config → Cache → i18n → Database → Storage → Parsers → Userbot → Bot → Queue
- Follow this order when adding new initialization steps in `cmd/run.go::initAll`
### Configuration (Viper)
- Config defined in `config/viper.go::Config`
- Read from `config.toml` (see `config.example.toml`)
- Environment variables: `SAVEANY_*` prefix (e.g., `SAVEANY_TELEGRAM_TOKEN`)
- Access via `config.C()` (returns a copy, don't modify the return value)
- Storage configs validated via `config/storage/factory.go::LoadStorageConfigs`
### Telegram Client
- **Bot client**: `client/bot/bot.go::Init` (uses gotgproto)
- **Handlers**: Centralized in `client/bot/handlers/` directory
- **Registration**: All handlers registered in `handlers.Register`
- **Commands**: Add to `CommandHandlers` slice for automatic `/help` and bot command list updates
- **Middleware**: Common middleware in `client/middleware/` (floodwait, retry, etc.)
### Tasks & Queue
- **Task interface**: `core/core.go::Executable` (Type, Title, TaskID, Execute methods)
- **Queue**: `pkg/queue.TaskQueue[Executable]` (generic, thread-safe)
- **Workers**: Count from `config.C().Workers`
- **Task types**: Implementations in `core/tasks/**` (tfile, parsed, telegraph, directlinks, batchtfile)
- **Lifecycle hooks**: `TaskBeforeStart`, `TaskSuccess`, `TaskFail`, `TaskCancel` (defined in config)
- **Adding tasks**: Use `core.AddTask(ctx, task)`
### Database (GORM + SQLite)
- **Init**: `database.Init` using `config.C().DB.Path`
- **Models**: User, Dir, Rule, WatchChat (in `database/*.go`)
- **Migrations**: Automatic via `db.AutoMigrate`
- **User sync**: `database.syncUsers` syncs DB with `config.C().Users` (don't manually create/delete users)
- **Context**: Always use `db.WithContext(ctx)` for operations
### Storage Backends
- **Interface**: Defined in `config/storage/types.go` and `storage/`
- **Implementations**: local, alist, s3/minio, webdav, telegram (each in subdirectory)
- **Adding new storage**:
1. Add enum to `pkg/enums/storage`
2. Create config struct in `config/storage/` with `Validate()` method
3. Implement storage in `storage/<name>/`
4. Register in `storageFactories` mapping
5. Update `config.example.toml` with example
### Parser Plugins (JavaScript)
- **Runtime**: Goja (JS runtime) + Playwright (browser automation)
- **Plugin API**: `registerParser({ metadata, canHandle, parse })` in JS
- **Integration**: Defined in `parsers/` directory
- **Documentation**: See `plugins/README.md`
- Plugin `parse` returns `Item`/`Resource` which becomes download/transfer task
### Internationalization (i18n)
- **Usage**: `i18n.T(i18nk.SomeKey, map[string]any{"Name": value})`
- **Locale files**: `common/i18n/locale/*.yaml`
- **Key generation**: Run `go generate ./...` to generate `common/i18n/i18nk/keys.go`
- **Adding new strings**: Add to YAML → run `go generate` → use in code
- All user-facing strings should be internationalized
### Context Usage
- Always pass `context.Context` as first parameter
- Use `log.FromContext(ctx)` to get contextual logger
- Respect context cancellation in long-running operations
- Store request-scoped data in context (e.g., `ctxkey.ContentLength`)
## Special Rules from .github/copilot-instructions.md
1. **Never modify `config.C()` return values** - it returns a copy. Modify config in `config.Init` or via Viper.
2. **Handlers must update `CommandHandlers` slice** - ensures `/help` and bot commands stay in sync.
3. **Task execution must preserve hooks** - don't remove `TaskBeforeStart`, `TaskSuccess`, `TaskFail`, `TaskCancel` hook calls.
4. **User sync is automatic** - don't manually create/delete users in DB; use config-based sync.
5. **Prefer context logger** - use `log.FromContext(ctx)` over global logger when context is available.
6. **Storage factory pattern** - new storage types must register in `storageFactories` mapping.
7. **Plugin API compatibility** - changes to `Item`/`Resource` structures require updating `plugins/README.md`.
## Common Patterns
### Adding a New Command
1. Create handler function in `client/bot/handlers/<name>.go`
2. Add to `CommandHandlers` slice in `register.go`
3. Add i18n key to `common/i18n/locale/*.yaml`
4. Run `go generate ./...`
5. Test with Telegram bot
### Adding a New Task Type
1. Create struct implementing `core.Executable` in `core/tasks/<type>/`
2. Implement `Type()`, `Title()`, `TaskID()`, `Execute(ctx)` methods
3. Add task type enum to `pkg/enums/tasktype`
4. Use `core.AddTask(ctx, task)` to enqueue
### Adding a New Storage Backend
1. Define config struct in `config/storage/<name>.go` with `Validate()` method
2. Implement storage interface in `storage/<name>/<name>.go`
3. Add storage type enum to `pkg/enums/storage`
4. Register factory in `config/storage/factory.go::storageFactories`
5. Update `config.example.toml` with configuration example
## File References
When referencing code locations, use `path/to/file.go:line` format (e.g., `core/core.go:23` for the worker function).
## Testing Guidelines
- Write tests for new functionality (place in `*_test.go` files)
- Test files should be in same package as code being tested
- Use table-driven tests for multiple test cases
- Mock external dependencies (databases, network calls)
- Aim for meaningful tests, not just coverage numbers
## Notes
- Binary size matters: use `CGO_ENABLED=0` for static binaries
- FFmpeg is included in Docker images for media processing
- Build process supports cross-compilation (amd64/arm64, Linux/macOS/Windows)
- Documentation site uses Hugo; edit files in `docs/` directory
- Session data stored in SQLite; delete `data/session.db` if changing bot token
- Pure stdlib `testing` (no testify); table-driven (`[]struct{name...}` + `t.Run`) with `t.Fatalf` got/want assertions. Mock via hand-written interface impls or package-variable replacement (`runMediaTool` in `video_split_test.go`, restored with `t.Cleanup`); in-process services for HTTP (`httptest`), S3 (`gofakes3+s3mem`), WebDAV (`x/net/webdav`).
- **Locale-dependent tests**: pin with `i18n.Init("zh-Hans")` + `t.Cleanup(...)`.
- **Progress/HTML tests**: assert rendered text with `strings.Contains` AND entity counts (`tg.MessageEntityBold/Code/Blockquote/Italic`) — verify style injection stays escaped (`<b>A&B</b>` input must render as literal text).
- **Known failures**: `storage/telegram` `TestCreateSplitZip`, `TestExtractThumbFrame`, `TestGetVideoMetadata` need gitignored fixtures + real ffmpeg — skip with `-skip 'Test(CreateSplitZip|ExtractThumbFrame|GetVideoMetadata)$'`; `api/handlers_test.go` has one `t.Skip` (needs initialized core).
- **Coverage expectations**: pure logic gets table tests (parsers, URL/path utils, progress throttling, grouping); regressions get bug-scenario-named tests (`progress_regression_test.go`). Network/Telegram/Playwright must never be touched by tests.
- When a permanent feature/API change ships: update `config.example.toml` if config, `docs/` if user-facing, `plugins/README.md` if plugin contract, and i18n YAML + `go generate` for new strings.

View File

@@ -7,6 +7,8 @@ ARG BuildTime="Unknown"
WORKDIR /app
RUN apk add --no-cache ca-certificates
COPY go.mod go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod \
go mod download
@@ -31,5 +33,9 @@ FROM scratch
WORKDIR /app
COPY --from=builder /app/saveany-bot .
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
ENV SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
ENV SSL_CERT_DIR=/etc/ssl/certs
ENTRYPOINT ["/app/saveany-bot"]

View File

@@ -1,7 +1,6 @@
package api
import (
"context"
"crypto/subtle"
"net/http"
"strings"
@@ -9,9 +8,6 @@ import (
"github.com/krau/SaveAny-Bot/config"
)
// tokenContextKey 用于在 context 中存储 token
type tokenContextKey struct{}
// AuthMiddleware 返回认证中间件
func AuthMiddleware() func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
@@ -40,9 +36,7 @@ func AuthMiddleware() func(http.Handler) http.Handler {
return
}
// 将 token 添加到 context
ctx := context.WithValue(r.Context(), tokenContextKey{}, token)
next.ServeHTTP(w, r.WithContext(ctx))
next.ServeHTTP(w, r)
})
}
}

View File

@@ -20,6 +20,7 @@ import (
"github.com/krau/SaveAny-Bot/pkg/aria2"
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
"github.com/krau/SaveAny-Bot/pkg/parser"
"github.com/krau/SaveAny-Bot/pkg/taskevent"
"github.com/krau/SaveAny-Bot/pkg/telegraph"
"github.com/krau/SaveAny-Bot/storage"
"github.com/rs/xid"
@@ -38,7 +39,7 @@ func NewTaskFactory(ctx context.Context) *TaskFactory {
// CreateTask 创建任务
func (f *TaskFactory) CreateTask(req *CreateTaskRequest) (*CreateTaskResponse, error) {
// 验证存储
stor, ok := storage.Storages[req.Storage]
stor, ok := storage.GetStorage(req.Storage)
if !ok {
return nil, fmt.Errorf("storage not found: %s", req.Storage)
}
@@ -68,9 +69,14 @@ func (f *TaskFactory) CreateTask(req *CreateTaskRequest) (*CreateTaskResponse, e
func (f *TaskFactory) registerAndEnqueueTask(task core.Executable, taskType tasktype.TaskType, storageName, path, webhook string) error {
taskID := task.TaskID()
RegisterTask(taskID, string(taskType), storageName, path, task.Title(), webhook)
info := RegisterTask(taskID, string(taskType), storageName, path, task.Title(), webhook)
err := core.AddTask(f.ctx, NewExecutableWrapper(task))
// Inject the progress sink into the context so the task's Emit calls update
// the API store (and fire the webhook on terminal states) without the task
// knowing about the API.
taskCtx := taskevent.WithSink(f.ctx, info)
err := core.AddTask(taskCtx, task)
if err != nil {
DeleteTask(taskID)
return fmt.Errorf("failed to add task: %w", err)
@@ -321,12 +327,12 @@ func (f *TaskFactory) createTransferTask(taskID string, createdAt time.Time, req
}
// 验证源存储和目标存储
sourceStor, ok := storage.Storages[params.SourceStorage]
sourceStor, ok := storage.GetStorage(params.SourceStorage)
if !ok {
return nil, fmt.Errorf("source storage not found: %s", params.SourceStorage)
}
targetStor, ok := storage.Storages[params.TargetStorage]
targetStor, ok := storage.GetStorage(params.TargetStorage)
if !ok {
return nil, fmt.Errorf("target storage not found: %s", params.TargetStorage)
}

View File

@@ -4,6 +4,7 @@ import (
"encoding/json"
"net/http"
"strings"
"time"
"github.com/krau/SaveAny-Bot/core"
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
@@ -117,7 +118,7 @@ func (h *Handlers) CancelTaskHandler(w http.ResponseWriter, r *http.Request) {
return
}
// 取消任务
// Cancel the task; the terminal status is set via the task event stream.
if err := core.CancelTask(r.Context(), taskID); err != nil {
WriteError(w, http.StatusInternalServerError, "cancel_failed", "failed to cancel task: "+err.Error())
return
@@ -134,8 +135,9 @@ func (h *Handlers) ListStoragesHandler(w http.ResponseWriter, r *http.Request) {
return
}
storages := make([]StorageInfo, 0, len(storage.Storages))
for name, stor := range storage.Storages {
all := storage.AllStorages()
storages := make([]StorageInfo, 0, len(all))
for name, stor := range all {
storages = append(storages, StorageInfo{
Name: name,
Type: string(stor.Type()),
@@ -184,27 +186,45 @@ func extractTaskIDFromPath(path string) string {
return parts[3]
}
// convertTaskProgressToResponse 将任务进度转换为响应格式
// convertTaskProgressToResponse renders a task's current state, computing
// percent and speed from the snapshot taken under the task's mutex.
func convertTaskProgressToResponse(task *TaskProgressInfo) TaskInfoResponse {
status, total, downloaded, totalFiles, downloadedFiles, startedAt, errMsg, updatedAt := task.snapshot()
resp := TaskInfoResponse{
TaskID: task.TaskID,
Type: tasktype.TaskType(task.Type),
Status: task.Status,
Status: status,
Title: task.Title,
Storage: task.Storage,
Path: task.Path,
Error: task.Error,
Error: errMsg,
CreatedAt: task.CreatedAt,
UpdatedAt: task.UpdatedAt,
UpdatedAt: updatedAt,
}
// 计算进度
if task.TotalBytes > 0 {
percent := float64(task.DownloadedBytes) * 100 / float64(task.TotalBytes)
var percent float64
var speedMBPS float64
if total > 0 {
percent = float64(downloaded) * 100 / float64(total)
} else if totalFiles > 0 {
percent = float64(downloadedFiles) * 100 / float64(totalFiles)
}
if !startedAt.IsZero() {
elapsed := time.Since(startedAt).Seconds()
if elapsed > 0 {
speedMBPS = float64(downloaded) / elapsed / (1024 * 1024)
}
}
if total > 0 || totalFiles > 0 {
resp.Progress = &TaskProgress{
TotalBytes: task.TotalBytes,
DownloadedBytes: task.DownloadedBytes,
TotalBytes: total,
DownloadedBytes: downloaded,
TotalFiles: totalFiles,
DownloadedFiles: downloadedFiles,
Percent: percent,
SpeedMBPS: speedMBPS,
}
}

View File

@@ -13,6 +13,7 @@ import (
"time"
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
"github.com/krau/SaveAny-Bot/pkg/taskevent"
)
// setupTestServer creates a test server with handlers
@@ -403,32 +404,38 @@ func TestConcurrentProgressStore(t *testing.T) {
// TestProgressTrackerConcurrentUpdates tests concurrent progress updates
func TestProgressTrackerConcurrentUpdates(t *testing.T) {
tracker := NewProgressTracker("concurrent-progress", "directlinks", "local", "downloads", "Test", "")
tracker.OnStart(10000, 10)
info := RegisterTask("concurrent-progress", "directlinks", "local", "downloads", "Test", "")
info.Emit(taskevent.Event{TaskID: "concurrent-progress", Phase: taskevent.PhaseStart, TotalBytes: 10000})
var wg sync.WaitGroup
numGoroutines := 50
updatesPerGoroutine := 100
// Concurrent progress updates
// Concurrent progress updates via the Sink interface
for i := range numGoroutines {
wg.Add(1)
go func(id int) {
defer wg.Done()
for j := range updatesPerGoroutine {
tracker.OnProgress(int64(id*updatesPerGoroutine+j), j)
info.Emit(taskevent.Event{
TaskID: "concurrent-progress",
Phase: taskevent.PhaseProgress,
DownloadedBytes: int64(id*updatesPerGoroutine + j),
TotalBytes: 10000,
})
}
}(i)
}
wg.Wait()
info := tracker.GetInfo()
if info.Status != TaskStatusRunning {
t.Errorf("expected status Running after concurrent updates, got %s", info.Status)
status, _, downloaded, _, _, _, _, _ := info.snapshot()
if status != TaskStatusRunning {
t.Errorf("expected status Running after concurrent updates, got %s", status)
}
if downloaded <= 0 {
t.Errorf("expected downloaded bytes > 0 after concurrent updates, got %d", downloaded)
}
// Note: Due to race conditions in the simple implementation,
// we can't reliably check exact values without proper synchronization
}
// TestTaskFactoryValidation tests TaskFactory parameter validation
@@ -526,8 +533,7 @@ func TestEdgeCases(t *testing.T) {
{
name: "Progress tracker with empty webhook",
fn: func(t *testing.T) {
tracker := NewProgressTracker("test", "type", "storage", "path", "title", "")
info := tracker.GetInfo()
info := RegisterTask("test-empty-webhook", "type", "storage", "path", "title", "")
if info.Webhook != "" {
t.Error("expected empty webhook")
}

View File

@@ -2,12 +2,16 @@ package api
import (
"sync"
"sync/atomic"
"time"
"github.com/krau/SaveAny-Bot/pkg/taskevent"
)
// TaskProgressInfo 存储任务的进度信息
// TaskProgressInfo stores the progress of an API-submitted task. All fields are
// guarded by mu. It implements taskevent.Sink so the task layer can update it
// without knowing about the API.
type TaskProgressInfo struct {
mu sync.Mutex
TaskID string
Type string
Status TaskStatus
@@ -21,20 +25,25 @@ type TaskProgressInfo struct {
Error string
CreatedAt time.Time
UpdatedAt time.Time
StartedAt time.Time
Webhook string
webhookNotified bool
}
// progressStore 存储所有 API 任务的进度信息
// progressStore holds all API tasks. Entries are removed a fixed duration after
// they reach a terminal state to bound memory usage.
type progressStore struct {
mu sync.RWMutex
tasks map[string]*TaskProgressInfo
mu sync.RWMutex
tasks map[string]*TaskProgressInfo
retention time.Duration
}
var store = &progressStore{
tasks: make(map[string]*TaskProgressInfo),
tasks: make(map[string]*TaskProgressInfo),
retention: 24 * time.Hour,
}
// RegisterTask 注册一个新的 API 任务
// RegisterTask registers a new API task and returns its progress info.
func RegisterTask(taskID, taskType, storage, path, title, webhook string) *TaskProgressInfo {
info := &TaskProgressInfo{
TaskID: taskID,
@@ -55,7 +64,7 @@ func RegisterTask(taskID, taskType, storage, path, title, webhook string) *TaskP
return info
}
// GetTask 获取任务进度信息
// GetTask returns the progress info for a task.
func GetTask(taskID string) (*TaskProgressInfo, bool) {
store.mu.RLock()
defer store.mu.RUnlock()
@@ -63,7 +72,7 @@ func GetTask(taskID string) (*TaskProgressInfo, bool) {
return info, ok
}
// GetAllTasks 获取所有任务
// GetAllTasks returns all tracked tasks.
func GetAllTasks() []*TaskProgressInfo {
store.mu.RLock()
defer store.mu.RUnlock()
@@ -75,76 +84,117 @@ func GetAllTasks() []*TaskProgressInfo {
return tasks
}
// DeleteTask 删除任务记录
// DeleteTask removes a task record.
func DeleteTask(taskID string) {
store.mu.Lock()
defer store.mu.Unlock()
delete(store.tasks, taskID)
}
// UpdateStatus 更新任务状态
func (t *TaskProgressInfo) UpdateStatus(status TaskStatus) {
t.Status = status
t.UpdatedAt = time.Now()
// CleanupExpired removes tasks that reached a terminal state more than the
// store's retention duration ago. It is safe to call periodically.
func CleanupExpired() {
now := time.Now()
store.mu.Lock()
defer store.mu.Unlock()
for id, info := range store.tasks {
info.mu.Lock()
terminal := info.Status == TaskStatusCompleted || info.Status == TaskStatusFailed || info.Status == TaskStatusCancelled
stale := terminal && now.Sub(info.UpdatedAt) > store.retention
info.mu.Unlock()
if stale {
delete(store.tasks, id)
}
}
}
// SetError 设置错误信息
// StartCleanupLoop runs CleanupExpired on a fixed interval until ctx is done.
// It should be started once during API server initialization.
func StartCleanupLoop(ctx interface{ Done() <-chan struct{} }) {
go func() {
ticker := time.NewTicker(10 * time.Minute)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
CleanupExpired()
}
}
}()
}
// UpdateStatus sets the task status.
func (t *TaskProgressInfo) UpdateStatus(status TaskStatus) {
t.mu.Lock()
t.Status = status
t.UpdatedAt = time.Now()
if status == TaskStatusRunning && t.StartedAt.IsZero() {
t.StartedAt = t.UpdatedAt
}
t.mu.Unlock()
}
// SetError marks the task failed with an error message.
func (t *TaskProgressInfo) SetError(err string) {
t.mu.Lock()
t.Error = err
t.Status = TaskStatusFailed
t.UpdatedAt = time.Now()
t.mu.Unlock()
}
// ProgressTracker 用于 API 任务的进度追踪
type ProgressTracker struct {
info *TaskProgressInfo
// snapshot returns a point-in-time copy of the fields needed to render a
// response, so callers never touch the mutex directly.
func (t *TaskProgressInfo) snapshot() (status TaskStatus, total, downloaded int64, totalFiles, downloadedFiles int, startedAt time.Time, err string, updatedAt time.Time) {
t.mu.Lock()
defer t.mu.Unlock()
return t.Status, t.TotalBytes, t.DownloadedBytes, t.TotalFiles, t.DownloadedFiles, t.StartedAt, t.Error, t.UpdatedAt
}
// NewProgressTracker 创建新的进度追踪器
func NewProgressTracker(taskID, taskType, storage, path, title, webhook string) *ProgressTracker {
info := RegisterTask(taskID, taskType, storage, path, title, webhook)
return &ProgressTracker{info: info}
}
// OnStart 任务开始
func (p *ProgressTracker) OnStart(totalBytes int64, totalFiles int) {
p.info.Status = TaskStatusRunning
p.info.TotalBytes = totalBytes
p.info.TotalFiles = totalFiles
p.info.UpdatedAt = time.Now()
}
// OnProgress 进度更新
func (p *ProgressTracker) OnProgress(downloadedBytes int64, downloadedFiles int) {
atomic.StoreInt64(&p.info.DownloadedBytes, downloadedBytes)
p.info.DownloadedFiles = downloadedFiles
p.info.UpdatedAt = time.Now()
}
// OnDone 任务完成
func (p *ProgressTracker) OnDone(err error) {
if err != nil {
p.info.Status = TaskStatusFailed
p.info.Error = err.Error()
} else {
p.info.Status = TaskStatusCompleted
// Emit implements taskevent.Sink. It translates task lifecycle events into
// status/progress updates and fires the webhook on terminal transitions.
func (t *TaskProgressInfo) Emit(e taskevent.Event) {
t.mu.Lock()
switch e.Phase {
case taskevent.PhaseStart:
t.Status = TaskStatusRunning
if t.StartedAt.IsZero() {
t.StartedAt = time.Now()
}
if e.TotalBytes > 0 {
t.TotalBytes = e.TotalBytes
}
case taskevent.PhaseProgress:
t.Status = TaskStatusRunning
if e.TotalBytes > 0 {
t.TotalBytes = e.TotalBytes
}
t.DownloadedBytes = e.DownloadedBytes
if e.TotalFiles > 0 {
t.TotalFiles = e.TotalFiles
}
if e.DownloadedFiles > 0 {
t.DownloadedFiles = e.DownloadedFiles
}
case taskevent.PhaseDone:
if e.Err != nil {
t.Status = TaskStatusFailed
t.Error = e.Err.Error()
} else {
t.Status = TaskStatusCompleted
}
}
p.info.UpdatedAt = time.Now()
}
t.UpdatedAt = time.Now()
notify := t.Webhook != "" && !t.webhookNotified && (t.Status == TaskStatusCompleted || t.Status == TaskStatusFailed)
if notify {
t.webhookNotified = true
}
t.mu.Unlock()
// GetInfo 获取任务信息
func (p *ProgressTracker) GetInfo() *TaskProgressInfo {
return p.info
}
// UpdateProgressBytes 更新下载字节数
func (p *ProgressTracker) UpdateProgressBytes(bytes int64) {
atomic.StoreInt64(&p.info.DownloadedBytes, bytes)
p.info.UpdatedAt = time.Now()
}
// UpdateProgressFiles 更新下载文件数
func (p *ProgressTracker) UpdateProgressFiles(files int) {
p.info.DownloadedFiles = files
p.info.UpdatedAt = time.Now()
if notify {
payload := CreateWebhookPayload(t.TaskID, t.Type, t.Status, t.Storage, t.Path, e.Err)
SendWebhook(nil, payload)
}
}

View File

@@ -3,6 +3,7 @@ package api
import (
"context"
"fmt"
"net"
"net/http"
"time"
@@ -57,22 +58,19 @@ func NewServer(ctx context.Context) *Server {
// 404 处理
mux.HandleFunc("/", NotFoundHandler)
// 应用中间件
// Apply middleware chain.
var handler http.Handler = mux
// 添加认证中间件
// Apply auth middleware when a token is configured.
token := cfg.Token
if token == "" {
log.FromContext(ctx).Warn("API server is enabled but no token is set, this is insecure!")
}
if token != "" {
handler = AuthMiddleware()(handler)
}
// 添加日志中间件
// Add logging middleware.
handler = loggingMiddleware(handler)
// 添加恢复中间件
// Add recovery middleware.
handler = recoveryMiddleware(handler)
return &Server{
@@ -93,9 +91,15 @@ func (s *Server) Start(ctx context.Context) error {
logger.Infof("Starting API server on %s", s.httpServer.Addr)
// Bind synchronously so listen failures are returned to the caller.
ln, err := net.Listen("tcp", s.httpServer.Addr)
if err != nil {
return fmt.Errorf("failed to listen on %s: %w", s.httpServer.Addr, err)
}
// 在 goroutine 中启动服务器
go func() {
if err := s.httpServer.ListenAndServe(); err != nil && err != http.ErrServerClosed {
if err := s.httpServer.Serve(ln); err != nil && err != http.ErrServerClosed {
logger.Errorf("API server error: %v", err)
}
}()
@@ -151,7 +155,8 @@ func (rw *responseWriter) WriteHeader(code int) {
rw.ResponseWriter.WriteHeader(code)
}
// Start 初始化并启动 API 服务器
// Start initializes and starts the API server. It refuses to start without a
// token, since an open download proxy is a security risk.
func Start(ctx context.Context) error {
cfg := config.C().API
@@ -160,9 +165,13 @@ func Start(ctx context.Context) error {
}
if cfg.Token == "" {
log.FromContext(ctx).Warn("API server is enabled but no token is set, this is insecure!")
return fmt.Errorf("API server is enabled but no token is set; refusing to start insecurely")
}
server := NewServer(ctx)
return server.Start(ctx)
if err := server.Start(ctx); err != nil {
return err
}
StartCleanupLoop(ctx)
return nil
}

View File

@@ -60,6 +60,7 @@ func resolveChatID(_ context.Context, idOrUsername string) (int64, error) {
}
// ParseMessageLink 解析 Telegram 消息链接
// 支持的域名: t.me, telegram.me
// 支持格式:
// - https://t.me/username/123
// - https://t.me/c/123456789/123
@@ -268,5 +269,15 @@ func ExtractFilesFromLinks(ctx context.Context, links []string) ([]tfile.TGFileM
// isValidMessageLink 检查是否是有效的 Telegram 消息链接
func isValidMessageLink(link string) bool {
return strings.HasPrefix(link, "https://t.me/") || strings.HasPrefix(link, "http://t.me/")
for _, prefix := range []string{
"https://t.me/",
"http://t.me/",
"https://telegram.me/",
"http://telegram.me/",
} {
if strings.HasPrefix(link, prefix) {
return true
}
}
return false
}

View File

@@ -40,6 +40,8 @@ type CreateTaskResponse struct {
type TaskProgress struct {
TotalBytes int64 `json:"total_bytes,omitempty"`
DownloadedBytes int64 `json:"downloaded_bytes,omitempty"`
TotalFiles int `json:"total_files,omitempty"`
DownloadedFiles int `json:"downloaded_files,omitempty"`
Percent float64 `json:"percent,omitempty"`
SpeedMBPS float64 `json:"speed_mbps,omitempty"`
}

View File

@@ -4,7 +4,6 @@ import (
"bytes"
"context"
"encoding/json"
"fmt"
"net/http"
"time"
@@ -30,9 +29,17 @@ func SendWebhook(ctx context.Context, payload *WebhookPayload) {
webhookURL := info.Webhook
// 异步发送 webhook
// Async send with retries.
go func() {
logger := log.FromContext(ctx).With("task_id", payload.TaskID)
var logger *log.Logger
if ctx != nil {
logger = log.FromContext(ctx).With("task_id", payload.TaskID)
} else {
logger = log.Default().With("task_id", payload.TaskID)
}
if ctx == nil {
ctx = context.Background()
}
payloadBytes, err := json.Marshal(payload)
if err != nil {
@@ -40,10 +47,15 @@ func SendWebhook(ctx context.Context, payload *WebhookPayload) {
return
}
// 重试 3 次
for i := range 3 {
req, err := http.NewRequestWithContext(context.Background(), http.MethodPost, webhookURL, bytes.NewBuffer(payloadBytes))
// 重试 3 次, 指数退避 (100ms/400ms/1.6s)
const maxAttempts = 3
const requestTimeout = 30 * time.Second
backoff := 100 * time.Millisecond
for i := range maxAttempts {
reqCtx, cancel := context.WithTimeout(ctx, requestTimeout)
req, err := http.NewRequestWithContext(reqCtx, http.MethodPost, webhookURL, bytes.NewBuffer(payloadBytes))
if err != nil {
cancel()
logger.Errorf("Failed to create webhook request: %v", err)
return
}
@@ -52,9 +64,13 @@ func SendWebhook(ctx context.Context, payload *WebhookPayload) {
req.Header.Set("User-Agent", "SaveAny-Bot/1.0")
resp, err := webhookClient.Do(req)
cancel()
if err != nil {
logger.Warnf("Webhook request failed (attempt %d/3): %v", i+1, err)
time.Sleep(time.Second * time.Duration(i+1))
logger.Warnf("Webhook request failed (attempt %d/%d): %v", i+1, maxAttempts, err)
if i < maxAttempts-1 {
time.Sleep(backoff)
}
backoff *= 4
continue
}
resp.Body.Close()
@@ -64,15 +80,18 @@ func SendWebhook(ctx context.Context, payload *WebhookPayload) {
return
}
logger.Warnf("Webhook returned non-2xx status (attempt %d/3): %d", i+1, resp.StatusCode)
time.Sleep(time.Second * time.Duration(i+1))
logger.Warnf("Webhook returned non-2xx status (attempt %d/%d): %d", i+1, maxAttempts, resp.StatusCode)
if i < maxAttempts-1 {
time.Sleep(backoff)
}
backoff *= 4
}
logger.Errorf("Failed to send webhook after 3 attempts")
logger.Errorf("Failed to send webhook after %d attempts", maxAttempts)
}()
}
// CreateWebhookPayload 创建 Webhook 负载
// CreateWebhookPayload creates a Webhook payload.
func CreateWebhookPayload(taskID string, taskType string, status TaskStatus, storage, path string, err error) *WebhookPayload {
payload := &WebhookPayload{
TaskID: taskID,
@@ -93,38 +112,3 @@ func CreateWebhookPayload(taskID string, taskType string, status TaskStatus, sto
return payload
}
// WrapTaskWithWebhook 包装任务执行,添加 webhook 回调
func WrapTaskWithWebhook(ctx context.Context, taskID string, fn func() error) error {
info, ok := GetTask(taskID)
if !ok {
return fmt.Errorf("task not found: %s", taskID)
}
err := fn()
// 确定任务状态
status := TaskStatusCompleted
if err != nil {
if err == context.Canceled {
status = TaskStatusCancelled
} else {
status = TaskStatusFailed
}
}
// 更新任务状态
if err != nil {
info.SetError(err.Error())
} else {
info.UpdateStatus(TaskStatusCompleted)
}
// 发送 webhook
if info.Webhook != "" {
payload := CreateWebhookPayload(taskID, info.Type, status, info.Storage, info.Path, err)
SendWebhook(ctx, payload)
}
return err
}

View File

@@ -1,58 +0,0 @@
package api
import (
"context"
"errors"
"github.com/krau/SaveAny-Bot/core"
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
)
// ExecutableWrapper wraps core.Executable to track task status in the API store and send webhooks.
type ExecutableWrapper struct {
inner core.Executable
}
func NewExecutableWrapper(inner core.Executable) *ExecutableWrapper {
return &ExecutableWrapper{inner: inner}
}
func (w *ExecutableWrapper) Type() tasktype.TaskType { return w.inner.Type() }
func (w *ExecutableWrapper) Title() string { return w.inner.Title() }
func (w *ExecutableWrapper) TaskID() string { return w.inner.TaskID() }
func (w *ExecutableWrapper) Execute(ctx context.Context) error {
taskID := w.inner.TaskID()
if info, ok := GetTask(taskID); ok {
info.UpdateStatus(TaskStatusRunning)
}
err := w.inner.Execute(ctx)
info, ok := GetTask(taskID)
if !ok {
return err
}
var status TaskStatus
if err != nil {
if errors.Is(err, context.Canceled) {
status = TaskStatusCancelled
info.UpdateStatus(TaskStatusCancelled)
} else {
status = TaskStatusFailed
info.SetError(err.Error())
}
} else {
status = TaskStatusCompleted
info.UpdateStatus(TaskStatusCompleted)
}
if info.Webhook != "" {
payload := CreateWebhookPayload(taskID, info.Type, status, info.Storage, info.Path, err)
SendWebhook(ctx, payload)
}
return err
}

View File

@@ -70,9 +70,6 @@ func Init(ctx context.Context) <-chan struct{} {
}{nil, err}
return
}
client.API().BotsSetBotCommands(ctx, &tg.BotsSetBotCommandsRequest{
Scope: &tg.BotCommandScopeDefault{},
})
commands := make([]tg.BotCommand, 0, len(handlers.CommandHandlers))
for _, info := range handlers.CommandHandlers {
commands = append(commands, tg.BotCommand{Command: info.Cmd, Description: i18n.T(info.Desc)})

View File

@@ -23,7 +23,11 @@ import (
)
func handleAddCallback(ctx *ext.Context, update *ext.Update) error {
dataid := strings.Split(string(update.CallbackQuery.Data), " ")[1]
dataParts := strings.Split(string(update.CallbackQuery.Data), " ")
if len(dataParts) < 2 {
return fmt.Errorf("invalid callback data: %q", update.CallbackQuery.Data)
}
dataid := dataParts[1]
data, err := shortcut.GetCallbackDataWithAnswer[tcbdata.Add](ctx, update, dataid)
if err != nil {
return err

View File

@@ -1,6 +1,7 @@
package handlers
import (
"fmt"
"strings"
"github.com/celestix/gotgproto/dispatcher"
@@ -14,7 +15,11 @@ import (
)
func handleCancelCallback(ctx *ext.Context, update *ext.Update) error {
taskid := strings.Split(string(update.CallbackQuery.Data), " ")[1]
dataParts := strings.Split(string(update.CallbackQuery.Data), " ")
if len(dataParts) < 2 {
return fmt.Errorf("invalid callback data: %q", update.CallbackQuery.Data)
}
taskid := dataParts[1]
if err := core.CancelTask(ctx, taskid); err != nil {
log.FromContext(ctx).Errorf("Failed to cancel task %s: %v", taskid, err)
ctx.AnswerCallback(msgelem.AlertCallbackAnswer(update.CallbackQuery.GetQueryID(), i18n.T(i18nk.BotMsgCancelErrorCancelFailed, map[string]any{

View File

@@ -10,6 +10,7 @@ import (
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/msgelem"
"github.com/krau/SaveAny-Bot/common/i18n"
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
"github.com/krau/SaveAny-Bot/database"
"github.com/krau/SaveAny-Bot/storage"
)
@@ -42,7 +43,9 @@ func handleDirCmd(ctx *ext.Context, update *ext.Update) error {
return dispatcher.EndGroups
}
if _, err := storage.GetStorageByUserIDAndName(ctx, user.ChatID, args[2]); err != nil {
ctx.Reply(update, ext.ReplyTextString(err.Error()), nil)
logger.Errorf("Failed to get storage %q: %s", args[2], err)
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgCommonErrorGetStorageFailed,
tgutil.EscapeHTMLTemplateData(map[string]any{"Error": err.Error()}))), nil)
return dispatcher.EndGroups
}

View File

@@ -13,16 +13,24 @@ import (
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/shortcut"
"github.com/krau/SaveAny-Bot/common/i18n"
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
"github.com/krau/SaveAny-Bot/database"
"github.com/krau/SaveAny-Bot/config"
"github.com/krau/SaveAny-Bot/database"
"github.com/krau/SaveAny-Bot/pkg/tcbdata"
"github.com/krau/SaveAny-Bot/pkg/tfile"
"github.com/krau/SaveAny-Bot/storage"
)
// mediaGroupKey uniquely identifies a media group by chat, sender, and group
// ID so files from different users in the same chat can never be mixed.
type mediaGroupKey struct {
chatID int64
userID int64
groupID int64
}
type MediaGroupHandler struct {
groups map[int64][]tfile.TGFileMessage
timers map[int64]*time.Timer
groups map[mediaGroupKey][]tfile.TGFileMessage
timers map[mediaGroupKey]*time.Timer
mu sync.Mutex
timeout time.Duration
setupOnce sync.Once
@@ -39,8 +47,8 @@ func (m *MediaGroupHandler) SetupTimeout(timeoutSec int) {
var (
mediaGroupHandler = &MediaGroupHandler{
groups: make(map[int64][]tfile.TGFileMessage),
timers: make(map[int64]*time.Timer),
groups: make(map[mediaGroupKey][]tfile.TGFileMessage),
timers: make(map[mediaGroupKey]*time.Timer),
mu: sync.Mutex{},
}
)
@@ -66,32 +74,37 @@ func handleGroupMediaMessage(ctx *ext.Context, update *ext.Update, message *tg.M
}
mediaGroupHandler.mu.Lock()
defer mediaGroupHandler.mu.Unlock()
if mediaGroupHandler.groups[groupID] == nil {
mediaGroupHandler.groups[groupID] = make([]tfile.TGFileMessage, 0)
key := mediaGroupKey{
chatID: update.EffectiveChat().GetID(),
userID: userId,
groupID: groupID,
}
mediaGroupHandler.groups[groupID] = append(mediaGroupHandler.groups[groupID], file)
if mediaGroupHandler.groups[key] == nil {
mediaGroupHandler.groups[key] = make([]tfile.TGFileMessage, 0)
}
mediaGroupHandler.groups[key] = append(mediaGroupHandler.groups[key], file)
if timer, exists := mediaGroupHandler.timers[groupID]; exists {
if timer, exists := mediaGroupHandler.timers[key]; exists {
timer.Stop()
}
mediaGroupHandler.timers[groupID] = time.AfterFunc(mediaGroupHandler.timeout, func() {
processMediaGroup(ctx, update, groupID)
mediaGroupHandler.timers[key] = time.AfterFunc(mediaGroupHandler.timeout, func() {
processMediaGroup(ctx, update, key)
})
return dispatcher.EndGroups
}
func processMediaGroup(ctx *ext.Context, update *ext.Update, groupID int64) {
func processMediaGroup(ctx *ext.Context, update *ext.Update, key mediaGroupKey) {
logger := log.FromContext(ctx)
mediaGroupHandler.mu.Lock()
items := mediaGroupHandler.groups[groupID]
delete(mediaGroupHandler.groups, groupID)
delete(mediaGroupHandler.timers, groupID)
items := mediaGroupHandler.groups[key]
delete(mediaGroupHandler.groups, key)
delete(mediaGroupHandler.timers, key)
mediaGroupHandler.mu.Unlock()
if len(items) == 0 {
logger.Warn("No media items to process for group", "groupID", groupID)
logger.Warn("No media items to process for group", "groupID", key.groupID)
return
}
logger.Debugf("Processing media group %d with %d items", groupID, len(items))
logger.Debugf("Processing media group %d with %d items", key.groupID, len(items))
userId := update.GetUserChat().GetID()
msg, err := ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgMediaGroupInfoSavingFiles, nil)), nil)

View File

@@ -22,6 +22,17 @@ func checkPermission(ctx *ext.Context, update *ext.Update) error {
return dispatcher.ContinueGroups
}
// withPermission wraps a callback handler with the same whitelist check used
// for message handlers (checkPermission).
func withPermission(handler func(*ext.Context, *ext.Update) error) func(*ext.Context, *ext.Update) error {
return func(ctx *ext.Context, update *ext.Update) error {
if err := checkPermission(ctx, update); err != nil {
return err
}
return handler(ctx, update)
}
}
func handleSilentMode(next func(*ext.Context, *ext.Update) error, handler func(*ext.Context, *ext.Update) error) func(*ext.Context, *ext.Update) error {
return func(ctx *ext.Context, update *ext.Update) error {
userID := update.GetUserChat().GetID()

View File

@@ -56,11 +56,11 @@ func Register(disp dispatcher.Dispatcher) {
for _, info := range CommandHandlers {
disp.AddHandler(handlers.NewCommand(info.Cmd, info.handler))
}
disp.AddHandler(handlers.NewCallbackQuery(filters.CallbackQuery.Prefix("update"), handleUpdateCallback))
disp.AddHandler(handlers.NewCallbackQuery(filters.CallbackQuery.Prefix(tcbdata.TypeAdd), handleAddCallback))
disp.AddHandler(handlers.NewCallbackQuery(filters.CallbackQuery.Prefix(tcbdata.TypeSetDefault), handleSetDefaultCallback))
disp.AddHandler(handlers.NewCallbackQuery(filters.CallbackQuery.Prefix(tcbdata.TypeCancel), handleCancelCallback))
disp.AddHandler(handlers.NewCallbackQuery(filters.CallbackQuery.Prefix(tcbdata.TypeConfig), handleConfigCallback))
disp.AddHandler(handlers.NewCallbackQuery(filters.CallbackQuery.Prefix("update"), withPermission(handleUpdateCallback)))
disp.AddHandler(handlers.NewCallbackQuery(filters.CallbackQuery.Prefix(tcbdata.TypeAdd), withPermission(handleAddCallback)))
disp.AddHandler(handlers.NewCallbackQuery(filters.CallbackQuery.Prefix(tcbdata.TypeSetDefault), withPermission(handleSetDefaultCallback)))
disp.AddHandler(handlers.NewCallbackQuery(filters.CallbackQuery.Prefix(tcbdata.TypeCancel), withPermission(handleCancelCallback)))
disp.AddHandler(handlers.NewCallbackQuery(filters.CallbackQuery.Prefix(tcbdata.TypeConfig), withPermission(handleConfigCallback)))
disp.AddHandler(handlers.NewMessage(sabotfilters.RegexUrl(regexp.MustCompile(re.TgMessageLinkRegexString)), handleSilentMode(handleMessageLink, handleSilentSaveLink)))
disp.AddHandler(handlers.NewMessage(sabotfilters.RegexUrl(regexp.MustCompile(re.TelegraphUrlRegexString)), handleSilentMode(handleTelegraphUrlMessage, handleSilentSaveTelegraph)))
disp.AddHandler(handlers.NewMessage(filters.Message.Media, handleSilentMode(handleMediaMessage, handleSilentSaveMedia)))

View File

@@ -13,6 +13,7 @@ import (
"github.com/krau/SaveAny-Bot/common/i18n"
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
"github.com/krau/SaveAny-Bot/common/utils/strutil"
"github.com/krau/SaveAny-Bot/config"
"github.com/krau/SaveAny-Bot/database"
"github.com/krau/SaveAny-Bot/pkg/rule"
)
@@ -84,6 +85,46 @@ func handleRuleCmd(ctx *ext.Context, update *ext.Update) error {
return dispatcher.EndGroups
}
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgRuleInfoCreateRuleSuccess, nil)), nil)
case "preset":
// /rule preset <storage> [base_path]
if len(args) < 3 {
ctx.Reply(update, ext.ReplyTextStyledTextArray(msgelem.BuildRuleHelpStyling(user.ApplyRule, user.Rules)), nil)
return dispatcher.EndGroups
}
storageName := args[2]
if !config.C().HasStorage(user.ChatID, storageName) {
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgRuleErrorStorageNotFound, map[string]any{
"Storage": storageName,
})), nil)
return dispatcher.EndGroups
}
basePath := ""
if len(args) >= 4 {
basePath = args[3]
}
presets := rule.PresetCategories(basePath)
imported := 0
for _, p := range presets {
rd := &database.Rule{
Type: rule.FileNameRegex.String(),
Data: p.Regex,
StorageName: storageName,
DirPath: p.Dir,
UserID: user.ID,
}
if err := database.CreateRule(ctx, rd); err != nil {
logger.Errorf("failed to create preset rule %s: %s", p.Name, err)
continue
}
imported++
}
if imported == 0 {
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgRuleErrorCreateRuleFailed, nil)), nil)
return dispatcher.EndGroups
}
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgRuleInfoPresetImported, map[string]any{
"Count": imported,
})), nil)
case "del":
// /rule del <id>
if len(args) < 3 {

View File

@@ -1,6 +1,7 @@
package handlers
import (
"fmt"
"strings"
"github.com/celestix/gotgproto/dispatcher"
@@ -43,7 +44,11 @@ func handleSilentCmd(ctx *ext.Context, update *ext.Update) error {
}
func handleSetDefaultCallback(ctx *ext.Context, update *ext.Update) error {
dataid := strings.Split(string(update.CallbackQuery.Data), " ")[1]
dataParts := strings.Split(string(update.CallbackQuery.Data), " ")
if len(dataParts) < 2 {
return fmt.Errorf("invalid callback data: %q", update.CallbackQuery.Data)
}
dataid := dataParts[1]
data, ok := cache.Get[tcbdata.SetDefaultStorage](dataid)
failedAnswer := func(message string) error {

View File

@@ -89,7 +89,11 @@ func showQueuedTasks(ctx *ext.Context, update *ext.Update) {
styling.Bold(i18n.T(i18nk.BotMsgTasksQueuedTitle)),
styling.Plain(i18n.T(i18nk.BotMsgTasksTotalPrefix, map[string]any{"Count": len(tasks)})),
)
for _, t := range tasks {
const maxShown = 10
for i, t := range tasks {
if i >= maxShown {
break
}
created := t.Created.In(time.Local).Format("2006-01-02 15:04:05")
status := i18n.T(i18nk.BotMsgTasksStatusQueued)
if t.Cancelled {
@@ -105,10 +109,9 @@ func showQueuedTasks(ctx *ext.Context, update *ext.Update) {
styling.Plain("\n"+i18n.T(i18nk.BotMsgTasksFieldStatus)),
styling.Code(status),
)
if len(tasks) > 10 {
opts = append(opts, styling.Plain("\n"+i18n.T(i18nk.BotMsgTasksTruncatedNote, map[string]any{"Count": len(tasks)})))
break
}
}
if len(tasks) > maxShown {
opts = append(opts, styling.Plain("\n"+i18n.T(i18nk.BotMsgTasksTruncatedNote, map[string]any{"Count": len(tasks)})))
}
ctx.Reply(update, ext.ReplyTextStyledTextArray(opts), nil)
}

View File

@@ -10,6 +10,7 @@ import (
"github.com/celestix/gotgproto/ext"
"github.com/gotd/td/telegram/message/html"
"github.com/gotd/td/tg"
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/msgelem"
"github.com/krau/SaveAny-Bot/common/i18n"
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
"github.com/krau/SaveAny-Bot/config"
@@ -100,7 +101,10 @@ func handleUpdateCmd(ctx *ext.Context, u *ext.Update) error {
func handleUpdateCallback(ctx *ext.Context, u *ext.Update) error {
currentV, err := semver.Parse(config.Version)
if err != nil {
return err
ctx.AnswerCallback(msgelem.AlertCallbackAnswer(u.CallbackQuery.GetQueryID(), i18n.T(i18nk.BotMsgUpdateErrorVersionVarInvalid, map[string]any{
"Error": err.Error(),
})))
return dispatcher.EndGroups
}
ctx.EditMessage(u.GetUserChat().GetID(), &tg.MessagesEditMessageRequest{
ID: u.CallbackQuery.GetMsgID(),

View File

@@ -24,6 +24,8 @@ func BuildRuleHelpStyling(enabled bool, rules []database.Rule) []styling.StyledT
styling.Plain(i18n.T(i18nk.BotMsgRuleHelpSwitchSuffix, nil)),
styling.Code("add"),
styling.Plain(i18n.T(i18nk.BotMsgRuleHelpAddSuffix, nil)),
styling.Code("preset"),
styling.Plain(i18n.T(i18nk.BotMsgRuleHelpPresetSuffix, nil)),
styling.Code("del"),
styling.Plain(i18n.T(i18nk.BotMsgRuleHelpDelSuffix, nil)),
styling.Plain(i18n.T(i18nk.BotMsgRuleHelpExistingRulesPrefix, nil)),

View File

@@ -3,8 +3,8 @@ package re
import "regexp"
var (
TgMessageLinkRegexString = `https?://t\.me/(?:c/\d+|[A-Za-z0-9_]+)/\d+(?:/\d+)?(?:\?[^\s#]*[A-Za-z0-9_])?\b`
TgMessageLinkRegexString = `https?://(?:t|telegram)\.me/(?:c/\d+|[A-Za-z0-9_]+)/\d+(?:/\d+)?(?:\?[^\s#]*[A-Za-z0-9_])?\b`
TgMessageLinkRegexp = regexp.MustCompile(TgMessageLinkRegexString)
TelegraphUrlRegexString = `https://telegra.ph/.*`
TelegraphUrlRegexString = `https://telegra\.ph/[^\s]+`
TelegraphUrlRegexp = regexp.MustCompile(TelegraphUrlRegexString)
)

View File

@@ -3,6 +3,7 @@ package shortcut
import (
"encoding/json"
"fmt"
"net/url"
"strings"
@@ -179,21 +180,19 @@ type TelegraphResult struct {
// return replied message, image urls, telegraph path(unescaped), error
func GetTphPicsFromMessageWithReply(ctx *ext.Context, update *ext.Update) (*types.Message, *TelegraphResult, error) {
logger := log.FromContext(ctx)
tphurl := re.TelegraphUrlRegexp.FindString(tgutil.ExtractMessageEntityUrlsText(update.EffectiveMessage.Message))
tphurl := findTelegraphURL(update.EffectiveMessage.Message)
if tphurl == "" {
logger.Warnf("No telegraph url found but called handleTelegraph")
return nil, nil, dispatcher.ContinueGroups
}
pagepath := strings.Split(tphurl, "/")[len(strings.Split(tphurl, "/"))-1]
tphdir, err := url.PathUnescape(pagepath)
pagepath, err := parseTelegraphPagePath(tphurl)
if err != nil {
logger.Errorf("Failed to unescape telegraph path: %s", err)
logger.Errorf("Failed to parse telegraph path: %s", err)
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgCommonErrorParseTelegraphPathFailed, map[string]any{
"Error": err.Error(),
})), nil)
return nil, nil, dispatcher.EndGroups
}
tphdir = strings.TrimSpace(tphdir)
msg, err := ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgCommonInfoFetchingTelegraphPage, nil)), nil)
if err != nil {
logger.Errorf("Failed to reply to update: %s", err)
@@ -244,7 +243,57 @@ func GetTphPicsFromMessageWithReply(ctx *ext.Context, update *ext.Update) (*type
}
return msg, &TelegraphResult{
Pics: imgs,
TphDir: tphdir,
TphDir: pagepath,
Page: page,
}, nil
}
func findTelegraphURL(msg *tg.Message) string {
if msg == nil {
return ""
}
var firstMatch string
findValid := func(text string) string {
for _, tphurl := range re.TelegraphUrlRegexp.FindAllString(text, -1) {
if firstMatch == "" {
firstMatch = tphurl
}
if _, err := parseTelegraphPagePath(tphurl); err == nil {
return tphurl
}
}
return ""
}
for _, entityURL := range tgutil.ExtractMessageEntityUrls(msg) {
if tphurl := findValid(entityURL); tphurl != "" {
return tphurl
}
}
if tphurl := findValid(msg.GetMessage()); tphurl != "" {
return tphurl
}
return firstMatch
}
func parseTelegraphPagePath(pageURL string) (string, error) {
u, err := url.Parse(pageURL)
if err != nil {
return "", fmt.Errorf("invalid telegraph URL: %w", err)
}
if u.Scheme != "https" || !strings.EqualFold(u.Hostname(), "telegra.ph") {
return "", fmt.Errorf("invalid telegraph URL host: %s", u.Host)
}
pagepath := strings.Trim(u.EscapedPath(), "/")
if pagepath == "" || strings.Contains(pagepath, "/") {
return "", fmt.Errorf("invalid telegraph URL path: %s", u.Path)
}
pagepath, err = url.PathUnescape(pagepath)
if err != nil {
return "", fmt.Errorf("failed to unescape telegraph path: %w", err)
}
pagepath = strings.TrimSpace(pagepath)
if pagepath == "" || strings.Contains(pagepath, "/") {
return "", fmt.Errorf("invalid telegraph URL path: %s", u.Path)
}
return pagepath, nil
}

View File

@@ -0,0 +1,163 @@
package shortcut
import (
"testing"
"github.com/gotd/td/tg"
)
func TestFindTelegraphURL(t *testing.T) {
tests := []struct {
name string
msg *tg.Message
want string
}{
{
name: "single URL entity",
msg: &tg.Message{
Message: "https://telegra.ph/Example-01-02",
Entities: []tg.MessageEntityClass{
&tg.MessageEntityURL{Offset: 0, Length: 32},
},
},
want: "https://telegra.ph/Example-01-02",
},
{
name: "Telegraph URL before another URL",
msg: &tg.Message{
Message: "https://telegra.ph/Example-01-02 https://example.com/",
Entities: []tg.MessageEntityClass{
&tg.MessageEntityURL{Offset: 0, Length: 32},
&tg.MessageEntityURL{Offset: 33, Length: 20},
},
},
want: "https://telegra.ph/Example-01-02",
},
{
name: "hidden Telegraph URL",
msg: &tg.Message{
Message: "article",
Entities: []tg.MessageEntityClass{
&tg.MessageEntityTextURL{
Offset: 0,
Length: 7,
URL: "https://telegra.ph/Hidden-01-02",
},
},
},
want: "https://telegra.ph/Hidden-01-02",
},
{
name: "URL entity after non-BMP character",
msg: &tg.Message{
Message: "😀 https://telegra.ph/Emoji-01-02",
Entities: []tg.MessageEntityClass{
&tg.MessageEntityURL{Offset: 3, Length: 30},
},
},
want: "https://telegra.ph/Emoji-01-02",
},
{
name: "valid Telegraph URL after invalid candidate",
msg: &tg.Message{
Message: "https://telegra.ph/nested/Bad https://telegra.ph/Valid-01-02",
Entities: []tg.MessageEntityClass{
&tg.MessageEntityURL{Offset: 0, Length: 29},
&tg.MessageEntityURL{Offset: 30, Length: 30},
},
},
want: "https://telegra.ph/Valid-01-02",
},
{
name: "plain message fallback",
msg: &tg.Message{Message: "read https://telegra.ph/Plain-01-02 now"},
want: "https://telegra.ph/Plain-01-02",
},
{
name: "nil message",
msg: nil,
want: "",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := findTelegraphURL(tt.msg); got != tt.want {
t.Fatalf("findTelegraphURL() = %q, want %q", got, tt.want)
}
})
}
}
func TestParseTelegraphPagePath(t *testing.T) {
tests := []struct {
name string
pageURL string
want string
wantErr bool
}{
{
name: "plain path",
pageURL: "https://telegra.ph/Example-01-02",
want: "Example-01-02",
},
{
name: "escaped path with query and fragment",
pageURL: "https://telegra.ph/%E6%B5%8B%E8%AF%95-01-02?source=telegram#top",
want: "测试-01-02",
},
{
name: "trailing slash",
pageURL: "https://telegra.ph/Example-01-02/",
want: "Example-01-02",
},
{
name: "root URL",
pageURL: "https://telegra.ph/",
wantErr: true,
},
{
name: "wrong host",
pageURL: "https://example.com/Example-01-02",
wantErr: true,
},
{
name: "wrong scheme",
pageURL: "http://telegra.ph/Example-01-02",
wantErr: true,
},
{
name: "invalid percent escape",
pageURL: "https://telegra.ph/Invalid-%zz",
wantErr: true,
},
{
name: "nested path",
pageURL: "https://telegra.ph/nested/Example-01-02",
wantErr: true,
},
{
name: "encoded slash",
pageURL: "https://telegra.ph/nested%2FExample-01-02",
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := parseTelegraphPagePath(tt.pageURL)
if tt.wantErr {
if err == nil {
t.Fatalf("parseTelegraphPagePath(%q) returned no error", tt.pageURL)
}
return
}
if err != nil {
t.Fatalf("parseTelegraphPagePath(%q) failed: %v", tt.pageURL, err)
}
if got != tt.want {
t.Fatalf("parseTelegraphPagePath(%q) = %q, want %q", tt.pageURL, got, tt.want)
}
})
}
}

View File

@@ -2,6 +2,7 @@ package user
import (
"context"
"sync"
"time"
"github.com/celestix/gotgproto"
@@ -20,17 +21,18 @@ import (
)
var uc *gotgproto.Client
var ectx *ext.Context
// getEctx lazily creates the user-client ext.Context exactly once. Guarded by
// sync.OnceValue so concurrent GetCtx calls cannot race on ectx creation.
var getEctx = sync.OnceValue(func() *ext.Context {
return uc.CreateContext()
})
func GetCtx() *ext.Context {
if ectx != nil {
return ectx
}
if uc == nil {
return nil
}
ectx = uc.CreateContext()
return ectx
return getEctx()
}
func Login(ctx context.Context) (*gotgproto.Client, error) {

View File

@@ -22,7 +22,13 @@ func main() {
pkg := flag.String("pkg", "i18nk", "Package name for generated file")
flag.Parse()
type localeFile struct {
path string
keys map[string]struct{}
}
keys := make(map[string]struct{})
var localeFiles []localeFile
err := filepath.WalkDir(*dir, func(path string, d fs.DirEntry, err error) error {
if err != nil {
@@ -42,7 +48,12 @@ func main() {
return fmt.Errorf("failed to parse yaml %s: %w", path, err)
}
collectKeys(content, "", keys)
fileKeys := make(map[string]struct{})
collectKeys(content, "", fileKeys)
localeFiles = append(localeFiles, localeFile{path: path, keys: fileKeys})
for k := range fileKeys {
keys[k] = struct{}{}
}
return nil
})
if err != nil {
@@ -50,6 +61,25 @@ func main() {
os.Exit(1)
}
// 一致性校验: 每个语言文件必须包含全部 key
invalid := false
for _, f := range localeFiles {
var missing []string
for k := range keys {
if _, ok := f.keys[k]; !ok {
missing = append(missing, k)
}
}
if len(missing) > 0 {
invalid = true
sort.Strings(missing)
fmt.Fprintf(os.Stderr, "Error: locale file %s is missing %d key(s): %s\n", f.path, len(missing), strings.Join(missing, ", "))
}
}
if invalid {
os.Exit(1)
}
var list []string
for k := range keys {
list = append(list, k)

View File

@@ -61,6 +61,7 @@ func Run(cmd *cobra.Command, _ []string) {
<-ctx.Done()
logger.Info("Exiting...")
defer logger.Info("Exit complete")
core.Close()
cleanCache()
}
@@ -87,7 +88,7 @@ func initAll(ctx context.Context) (<-chan struct{}, error) {
}
}
if err := api.Start(ctx); err != nil {
logger.Error("Failed to start API server", "error", err)
logger.Fatal("Failed to start API server", "error", err)
}
return bot.Init(ctx), nil
}

View File

@@ -2,6 +2,7 @@ package cache
import (
"fmt"
"sync"
"time"
"github.com/charmbracelet/log"
@@ -9,24 +10,26 @@ import (
"github.com/krau/SaveAny-Bot/config"
)
var cache *ristretto.Cache[string, any]
var (
cache *ristretto.Cache[string, any]
initOnce sync.Once
)
func Init() {
if cache != nil {
panic("cache already initialized")
}
c, err := ristretto.NewCache(&ristretto.Config[string, any]{
NumCounters: config.C().Cache.NumCounters,
MaxCost: config.C().Cache.MaxCost,
BufferItems: 64,
OnReject: func(item *ristretto.Item[any]) {
log.Warnf("Cache item rejected: key=%d, value=%v", item.Key, item.Value)
},
initOnce.Do(func() {
c, err := ristretto.NewCache(&ristretto.Config[string, any]{
NumCounters: config.C().Cache.NumCounters,
MaxCost: config.C().Cache.MaxCost,
BufferItems: 64,
OnReject: func(item *ristretto.Item[any]) {
log.Warnf("Cache item rejected: key=%d, value=%v", item.Key, item.Value)
},
})
if err != nil {
log.Fatalf("failed to create ristretto cache: %v", err)
}
cache = c
})
if err != nil {
log.Fatalf("failed to create ristretto cache: %v", err)
}
cache = c
}
func Set(key string, value any) error {

View File

@@ -84,8 +84,8 @@ const (
BotMsgCommonPromptSelectDefaultDir Key = "bot.msg.common.prompt_select_default_dir"
BotMsgCommonPromptSelectDefaultStorage Key = "bot.msg.common.prompt_select_default_storage"
BotMsgCommonPromptSelectDir Key = "bot.msg.common.prompt_select_dir"
BotMsgConfigButtonFilenameStrategy Key = "bot.msg.config.button_filename_strategy"
BotMsgConfigButtonConflictStrategy Key = "bot.msg.config.button_conflict_strategy"
BotMsgConfigButtonFilenameStrategy Key = "bot.msg.config.button_filename_strategy"
BotMsgConfigConflictStrategyAsk Key = "bot.msg.config.conflict_strategy_ask"
BotMsgConfigConflictStrategyOverwrite Key = "bot.msg.config.conflict_strategy_overwrite"
BotMsgConfigConflictStrategyRename Key = "bot.msg.config.conflict_strategy_rename"
@@ -93,8 +93,8 @@ const (
BotMsgConfigErrorInvalidCallbackData Key = "bot.msg.config.error_invalid_callback_data"
BotMsgConfigErrorInvalidTemplate Key = "bot.msg.config.error_invalid_template"
BotMsgConfigFnametmplHelp Key = "bot.msg.config.fnametmpl_help"
BotMsgConfigInfoCurrentTemplatePrefix Key = "bot.msg.config.info_current_template_prefix"
BotMsgConfigInfoConflictStrategySet Key = "bot.msg.config.info_conflict_strategy_set"
BotMsgConfigInfoCurrentTemplatePrefix Key = "bot.msg.config.info_current_template_prefix"
BotMsgConfigInfoFilenameStrategySet Key = "bot.msg.config.info_filename_strategy_set"
BotMsgConfigInfoTemplateUpdated Key = "bot.msg.config.info_template_updated"
BotMsgConfigPromptSelectConflictStrategy Key = "bot.msg.config.prompt_select_conflict_strategy"
@@ -150,28 +150,52 @@ const (
BotMsgProgressAria2Downloading Key = "bot.msg.progress.aria2_downloading"
BotMsgProgressAria2Start Key = "bot.msg.progress.aria2_start"
BotMsgProgressAvgSpeedPrefix Key = "bot.msg.progress.avg_speed_prefix"
BotMsgProgressBatchDonePrefix Key = "bot.msg.progress.batch_done_prefix"
BotMsgProgressBatchProcessingPrefix Key = "bot.msg.progress.batch_processing_prefix"
BotMsgProgressBatchStartPrefix Key = "bot.msg.progress.batch_start_prefix"
BotMsgProgressBatchCanceled Key = "bot.msg.progress.batch_canceled"
BotMsgProgressBatchDone Key = "bot.msg.progress.batch_done"
BotMsgProgressBatchDoneWithSkipped Key = "bot.msg.progress.batch_done_with_skipped"
BotMsgProgressBatchFailedGroup Key = "bot.msg.progress.batch_failed_group"
BotMsgProgressBatchFailedItem Key = "bot.msg.progress.batch_failed_item"
BotMsgProgressBatchFailedTask Key = "bot.msg.progress.batch_failed_task"
BotMsgProgressBatchFailureStageBatchUpload Key = "bot.msg.progress.batch_failure_stage_batch_upload"
BotMsgProgressBatchFailureStageCache Key = "bot.msg.progress.batch_failure_stage_cache"
BotMsgProgressBatchFailureStageConfirm Key = "bot.msg.progress.batch_failure_stage_confirm"
BotMsgProgressBatchFailureStageDownload Key = "bot.msg.progress.batch_failure_stage_download"
BotMsgProgressBatchFailureStageInternal Key = "bot.msg.progress.batch_failure_stage_internal"
BotMsgProgressBatchFailureStageUpload Key = "bot.msg.progress.batch_failure_stage_upload"
BotMsgProgressBatchItemConfirming Key = "bot.msg.progress.batch_item_confirming"
BotMsgProgressBatchItemDownloading Key = "bot.msg.progress.batch_item_downloading"
BotMsgProgressBatchItemDownloadingUnknown Key = "bot.msg.progress.batch_item_downloading_unknown"
BotMsgProgressBatchItemRetrying Key = "bot.msg.progress.batch_item_retrying"
BotMsgProgressBatchItemTransferring Key = "bot.msg.progress.batch_item_transferring"
BotMsgProgressBatchItemTransferringUnknown Key = "bot.msg.progress.batch_item_transferring_unknown"
BotMsgProgressBatchItemUploading Key = "bot.msg.progress.batch_item_uploading"
BotMsgProgressBatchStatusHeader Key = "bot.msg.progress.batch_status_header"
BotMsgProgressBatchSummaryConfirming Key = "bot.msg.progress.batch_summary_confirming"
BotMsgProgressBatchSummaryFailed Key = "bot.msg.progress.batch_summary_failed"
BotMsgProgressBatchSummaryHiddenActive Key = "bot.msg.progress.batch_summary_hidden_active"
BotMsgProgressBatchSummarySkipped Key = "bot.msg.progress.batch_summary_skipped"
BotMsgProgressCurrentProgressPrefix Key = "bot.msg.progress.current_progress_prefix"
BotMsgProgressCurrentSpeedPrefix Key = "bot.msg.progress.current_speed_prefix"
BotMsgProgressDirectDonePrefix Key = "bot.msg.progress.direct_done_prefix"
BotMsgProgressDirectStart Key = "bot.msg.progress.direct_start"
BotMsgProgressDownloadDonePrefix Key = "bot.msg.progress.download_done_prefix"
BotMsgProgressDownloadFailedPrefix Key = "bot.msg.progress.download_failed_prefix"
BotMsgProgressDownloadedPrefix Key = "bot.msg.progress.downloaded_prefix"
BotMsgProgressDownloadingPrefix Key = "bot.msg.progress.downloading_prefix"
BotMsgProgressErrorPrefix Key = "bot.msg.progress.error_prefix"
BotMsgProgressFileNamePrefix Key = "bot.msg.progress.file_name_prefix"
BotMsgProgressFileProcessingPrefix Key = "bot.msg.progress.file_processing_prefix"
BotMsgProgressFileSizePrefix Key = "bot.msg.progress.file_size_prefix"
BotMsgProgressFileStartPrefix Key = "bot.msg.progress.file_start_prefix"
BotMsgProgressParsedDonePrefix Key = "bot.msg.progress.parsed_done_prefix"
BotMsgProgressParsedStartPrefix Key = "bot.msg.progress.parsed_start_prefix"
BotMsgProgressProcessingListPrefix Key = "bot.msg.progress.processing_list_prefix"
BotMsgProgressProcessingNone Key = "bot.msg.progress.processing_none"
BotMsgProgressSavePathPrefix Key = "bot.msg.progress.save_path_prefix"
BotMsgProgressTaskCanceled Key = "bot.msg.progress.task_canceled"
BotMsgProgressSingleCanceled Key = "bot.msg.progress.single_canceled"
BotMsgProgressSingleDone Key = "bot.msg.progress.single_done"
BotMsgProgressSingleDownloading Key = "bot.msg.progress.single_downloading"
BotMsgProgressSingleDownloadingUnknown Key = "bot.msg.progress.single_downloading_unknown"
BotMsgProgressSingleFailed Key = "bot.msg.progress.single_failed"
BotMsgProgressSingleStatusHeader Key = "bot.msg.progress.single_status_header"
BotMsgProgressSingleUploadRetrying Key = "bot.msg.progress.single_upload_retrying"
BotMsgProgressSingleUploading Key = "bot.msg.progress.single_uploading"
BotMsgProgressSizeWithFiles Key = "bot.msg.progress.size_with_files"
BotMsgProgressSizeWithResources Key = "bot.msg.progress.size_with_resources"
BotMsgProgressTaskCanceledWithId Key = "bot.msg.progress.task_canceled_with_id"
BotMsgProgressTaskFailedWithError Key = "bot.msg.progress.task_failed_with_error"
BotMsgProgressTelegraphDonePrefix Key = "bot.msg.progress.telegraph_done_prefix"
@@ -200,6 +224,7 @@ const (
BotMsgRuleErrorGetUserRulesFailed Key = "bot.msg.rule.error_get_user_rules_failed"
BotMsgRuleErrorInvalidRuleId Key = "bot.msg.rule.error_invalid_rule_id"
BotMsgRuleErrorInvalidRuleType Key = "bot.msg.rule.error_invalid_rule_type"
BotMsgRuleErrorStorageNotFound Key = "bot.msg.rule.error_storage_not_found"
BotMsgRuleErrorUpdateUserFailed Key = "bot.msg.rule.error_update_user_failed"
BotMsgRuleHelpAddSuffix Key = "bot.msg.rule.help_add_suffix"
BotMsgRuleHelpAvailableOps Key = "bot.msg.rule.help_available_ops"
@@ -207,18 +232,20 @@ const (
BotMsgRuleHelpCurrentModeEnabled Key = "bot.msg.rule.help_current_mode_enabled"
BotMsgRuleHelpDelSuffix Key = "bot.msg.rule.help_del_suffix"
BotMsgRuleHelpExistingRulesPrefix Key = "bot.msg.rule.help_existing_rules_prefix"
BotMsgRuleHelpPresetSuffix Key = "bot.msg.rule.help_preset_suffix"
BotMsgRuleHelpSwitchSuffix Key = "bot.msg.rule.help_switch_suffix"
BotMsgRuleHelpUsage Key = "bot.msg.rule.help_usage"
BotMsgRuleInfoCreateRuleSuccess Key = "bot.msg.rule.info_create_rule_success"
BotMsgRuleInfoDeleteRuleSuccess Key = "bot.msg.rule.info_delete_rule_success"
BotMsgRuleInfoPresetImported Key = "bot.msg.rule.info_preset_imported"
BotMsgRuleInfoRuleModeDisabled Key = "bot.msg.rule.info_rule_mode_disabled"
BotMsgRuleInfoRuleModeEnabled Key = "bot.msg.rule.info_rule_mode_enabled"
BotMsgRulePromptProvideRuleId Key = "bot.msg.rule.prompt_provide_rule_id"
BotMsgRulePromptProvideStorageName Key = "bot.msg.rule.prompt_provide_storage_name"
BotMsgSaveErrorInvalidIdOrUsername Key = "bot.msg.save.error_invalid_id_or_username"
BotMsgSaveHelpText Key = "bot.msg.save_help_text"
BotMsgStorageInfoFilenamePrefix Key = "bot.msg.storage.info_filename_prefix"
BotMsgStorageInfoPromptSelectStorage Key = "bot.msg.storage.info_prompt_select_storage"
BotMsgSyncpeersDone Key = "bot.msg.syncpeers.done"
BotMsgSyncpeersFailed Key = "bot.msg.syncpeers.failed"
BotMsgSyncpeersStart Key = "bot.msg.syncpeers.start"
BotMsgSyncpeersSuccess Key = "bot.msg.syncpeers.success"

View File

@@ -196,7 +196,11 @@ bot:
help_switch_suffix: " - Toggle rule mode\n"
help_add_suffix: " <type> <data> <storage_name> <path> - Add rule\n"
help_del_suffix: " <rule_id> - Delete rule\n"
help_preset_suffix: " <storage_name> [base_path] - Import built-in filetype rules (video/image/audio/document/archive)\n"
help_existing_rules_prefix: "\nCurrent rules:\n"
prompt_provide_storage_name: "Please provide a storage name"
error_storage_not_found: "Storage not found: {{.Storage}}"
info_preset_imported: "Imported {{.Count}} built-in classification rules into storage {{.Storage}}"
dir:
error_get_user_dirs_failed: "Failed to get user directories"
error_get_user_failed: "Failed to get user"
@@ -347,32 +351,56 @@ bot:
info_filename_prefix: "Filename: "
info_prompt_select_storage: "\nPlease select storage"
progress:
batch_start_prefix: "Starting batch download task\nTotal size: "
batch_processing_prefix: "Processing batch download task\nTotal size: "
batch_status_header: "<b>📦 Processing</b>\n\nFiles: <code>{{.Total}}</code>\nStatus: ✅ <code>{{.Completed}}</code> | 📥 <code>{{.Downloaded}}</code> | ⏳ <code>{{.Waiting}}</code>\nTotal speed: ⬇️ <code>{{.DownloadSpeed}}</code> | ⬆️ <code>{{.UploadSpeed}}</code>"
batch_item_downloading: "<blockquote><b>⬇️ {{.Index}}/{{.Total}} Downloading</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\nSpeed: <code>{{.Speed}}</code>\nSize: <code>{{.Current}}</code> / <code>{{.Size}}</code></blockquote>"
batch_item_downloading_unknown: "<blockquote><b>⬇️ {{.Index}}/{{.Total}} Downloading</b>\n<code>{{.Name}}</code>\nSpeed: <code>{{.Speed}}</code>\nSize: <code>{{.Current}}</code> / unknown</blockquote>"
batch_item_transferring: "<blockquote><b>↕️ {{.Index}}/{{.Total}} Transferring</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\nSpeed: <code>{{.Speed}}</code>\nSize: <code>{{.Current}}</code> / <code>{{.Size}}</code></blockquote>"
batch_item_transferring_unknown: "<blockquote><b>↕️ {{.Index}}/{{.Total}} Transferring</b>\n<code>{{.Name}}</code>\nSpeed: <code>{{.Speed}}</code>\nSize: <code>{{.Current}}</code> / unknown</blockquote>"
batch_item_uploading: "<blockquote><b>⬆️ {{.Index}}/{{.Total}} Uploading</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\nSpeed: <code>{{.Speed}}</code>\nSize: <code>{{.Current}}</code> / <code>{{.Size}}</code></blockquote>"
batch_item_retrying: "<blockquote><b>🔁 {{.Index}}/{{.Total}} Retrying upload</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\nRetry: <code>{{.Attempt}}/{{.Limit}}</code>\nSpeed before failure: <code>{{.Speed}}</code>\nSize: <code>{{.Current}}</code> / <code>{{.Size}}</code>\nReason: <code>{{.Reason}}</code></blockquote>"
batch_item_confirming: "<blockquote><b>⏳ {{.Index}}/{{.Total}} Waiting</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\n<i>Uploaded, awaiting remote confirmation</i></blockquote>"
batch_summary_hidden_active: "🔄 <code>{{.Count}}</code> more files are active"
batch_summary_confirming: "☁️ Uploaded, awaiting group send: <code>{{.Count}}</code>"
batch_summary_failed: "❌ Failed: <code>{{.Count}}</code>"
batch_summary_skipped: "⏭️ Skipped: <code>{{.Count}}</code>"
batch_done: "<b>✅ Completed</b>\n\nFiles: <code>{{.Count}}</code>\nTotal size: <code>{{.Size}}</code>"
batch_done_with_skipped: "<b>⚠️ Completed</b>\n\nSucceeded: <code>{{.Success}}</code>\nSkipped: <code>{{.Skipped}}</code>\nTotal size: <code>{{.Size}}</code>"
batch_canceled: "<b>🚫 Task canceled</b>\n\nFiles: <code>{{.Total}}</code>\nCompleted: <code>{{.Completed}}</code>\nIncomplete: <code>{{.Incomplete}}</code>\nSkipped: <code>{{.Skipped}}</code>"
batch_failed_item: "<b>❌ Processing failed</b>\n\nFailed file: <code>{{.Index}}. {{.Name}}</code>\nStage: <code>{{.Stage}}</code>\nProgress: <code>{{.Progress}}</code>\nSpeed before failure: <code>{{.Speed}}</code>\nReason: <code>{{.Reason}}</code>\n\n✅ Completed: <code>{{.Completed}}</code>\n❌ Failed: <code>{{.Failed}}</code>\n⏹ Incomplete: <code>{{.Incomplete}}</code>"
batch_failed_group: "<b>❌ Batch upload failed</b>\n\nAffected files: <code>{{.Affected}}</code>\nReason: <code>{{.Reason}}</code>\n\n✅ Completed: <code>{{.Completed}}</code>\n❌ Batch failed: <code>{{.Failed}}</code>\n⏹ Incomplete: <code>{{.Incomplete}}</code>"
batch_failed_task: "<b>❌ Processing failed</b>\n\nReason: <code>{{.Reason}}</code>\n\n✅ Completed: <code>{{.Completed}}</code>\n⏹ Incomplete: <code>{{.Incomplete}}</code>"
batch_failure_stage_download: "download"
batch_failure_stage_cache: "local cache"
batch_failure_stage_upload: "upload"
batch_failure_stage_confirm: "remote confirmation"
batch_failure_stage_batch_upload: "batch upload"
batch_failure_stage_internal: "internal task"
single_status_header: "<b>📦 Processing</b>"
single_downloading: "<blockquote><b>⬇️ Downloading</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\nSpeed: <code>{{.Speed}}</code>\nSize: <code>{{.Current}}</code> / <code>{{.Size}}</code>\nSave to: <code>{{.Destination}}</code></blockquote>"
single_downloading_unknown: "<blockquote><b>⬇️ Downloading</b>\n<code>{{.Name}}</code>\nSpeed: <code>{{.Speed}}</code>\nSize: <code>{{.Current}}</code> / unknown\nSave to: <code>{{.Destination}}</code></blockquote>"
single_uploading: "<blockquote><b>⬆️ Uploading</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\nSpeed: <code>{{.Speed}}</code>\nSize: <code>{{.Current}}</code> / <code>{{.Size}}</code>\nSave to: <code>{{.Destination}}</code></blockquote>"
single_upload_retrying: "<blockquote><b>🔁 Retrying upload</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\nAttempt: <code>{{.Attempt}}</code>\nSpeed: <code>{{.Speed}}</code>\nSize: <code>{{.Current}}</code> / <code>{{.Size}}</code>\nSave to: <code>{{.Destination}}</code></blockquote>"
single_done: "<b>✅ Completed</b>\n\nFilename: <code>{{.Name}}</code>\nTotal size: <code>{{.Size}}</code>\nSave to: <code>{{.Destination}}</code>"
single_canceled: "<b>🚫 Task canceled</b>\n\nFilename: <code>{{.Name}}</code>"
single_failed: "<b>❌ Processing failed</b>\n\nFilename: <code>{{.Name}}</code>\nReason: <code>{{.Reason}}</code>"
downloading_prefix: "Downloading\nTotal size: "
size_with_files: "{{.Size}} ({{.Count}} files)"
size_with_resources: "{{.Size}} ({{.Count}} resources)"
processing_list_prefix: "\nProcessing:\n"
processing_none: " - None"
avg_speed_prefix: "\nAverage speed: "
current_progress_prefix: "\nCurrent progress: "
task_canceled: "Task canceled"
task_canceled_with_id: "Processing canceled: {{.TaskID}}"
task_failed_with_error: "Processing failed: {{.Error}}"
batch_done_prefix: "Completed\nFile count: "
direct_done_prefix: "Completed, file count: "
parsed_start_prefix: "Starting download from {{.Site}}\nTotal size: "
parsed_done_prefix: "Completed, resource count: "
telegraph_start_prefix: "Starting Telegraph download\nImage count: "
telegraph_progress_prefix: "Downloading\nCurrent progress: "
telegraph_done_prefix: "Completed\nImage count: "
file_start_prefix: "Starting download\nFilename: "
file_processing_prefix: "Processing download task\nFilename: "
download_failed_prefix: "Download failed\nFilename: "
download_done_prefix: "Download completed\nFilename: "
file_size_prefix: "\nFile size: "
save_path_prefix: "\nSave path: "
total_size_prefix: "\nTotal size: "
direct_start: "Starting download, total size: {{.SizeMB}} MB ({{.Count}} files)"
file_name_prefix: "Filename: "
error_prefix: "\nError: "
aria2_start: "Waiting for Aria2 to complete download (GID: {{.GID}})..."
aria2_downloading: "Aria2 downloading (GID: {{.GID}})\n"
@@ -398,7 +426,7 @@ bot:
transfer_failed_files_prefix: "\nFailed files: "
syncpeers:
start: "Starting to sync peers..."
done: "Peer sync completed, total {{.Count}} chats synced"
success: "Peer sync completed, total {{.Count}} chats synced"
failed: "Peer sync failed: {{.Error}}"
aria2:
error_aria2_not_enabled: "Aria2 feature is not enabled in the configuration"

View File

@@ -197,7 +197,11 @@ bot:
help_switch_suffix: " - 开关规则模式\n"
help_add_suffix: " <类型> <数据> <存储名> <路径> - 添加规则\n"
help_del_suffix: " <规则ID> - 删除规则\n"
help_preset_suffix: " <存储名> [基础路径] - 导入内置文件类型分类规则(视频/图片/音频/文档/压缩包)\n"
help_existing_rules_prefix: "\n当前已添加的规则:\n"
prompt_provide_storage_name: "请提供存储名称"
error_storage_not_found: "未找到存储: {{.Storage}}"
info_preset_imported: "已导入 {{.Count}} 条内置分类规则到存储 {{.Storage}}"
dir:
error_get_user_dirs_failed: "获取用户文件夹失败"
error_get_user_failed: "获取用户失败"
@@ -234,9 +238,9 @@ bot:
info_install_plugin_success: "插件安装成功: {{.Name}}"
parse:
info_parsing: "正在解析..."
error_parse_text_failed: "Failed to parse text: {{.Error}}"
error_build_storage_select_keyboard_failed: "Failed to build storage selection keyboard: {{.Error}}"
error_build_parsed_text_entity_failed: "Failed to build parsed text entity: {{.Error}}"
error_parse_text_failed: "解析文本失败: {{.Error}}"
error_build_storage_select_keyboard_failed: "构建存储选择键盘失败: {{.Error}}"
error_build_parsed_text_entity_failed: "构建解析文本实体失败: {{.Error}}"
info_link_prefix: "\n链接: "
info_author_prefix: "\n作者: "
info_description_prefix: "\n描述: "
@@ -348,32 +352,56 @@ bot:
info_filename_prefix: "文件名: "
info_prompt_select_storage: "\n请选择存储位置"
progress:
batch_start_prefix: "开始执行批量下载任务\n总大小: "
batch_processing_prefix: "正在处理批量下载任务\n总大小: "
batch_status_header: "<b>📦 正在处理</b>\n\n文件<code>{{.Total}}</code>\n状态✅ <code>{{.Completed}}</code> 📥 <code>{{.Downloaded}}</code> ⏳ <code>{{.Waiting}}</code>\n总速度 <code>{{.DownloadSpeed}}</code> ⬆️ <code>{{.UploadSpeed}}</code>"
batch_item_downloading: "<blockquote><b>⬇️ {{.Index}}/{{.Total}} 下载中</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\n速度<code>{{.Speed}}</code>\n大小<code>{{.Current}}</code> / <code>{{.Size}}</code></blockquote>"
batch_item_downloading_unknown: "<blockquote><b>⬇️ {{.Index}}/{{.Total}} 下载中</b>\n<code>{{.Name}}</code>\n速度<code>{{.Speed}}</code>\n大小<code>{{.Current}}</code> / 未知</blockquote>"
batch_item_transferring: "<blockquote><b>↕️ {{.Index}}/{{.Total}} 传输中</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\n速度<code>{{.Speed}}</code>\n大小<code>{{.Current}}</code> / <code>{{.Size}}</code></blockquote>"
batch_item_transferring_unknown: "<blockquote><b>↕️ {{.Index}}/{{.Total}} 传输中</b>\n<code>{{.Name}}</code>\n速度<code>{{.Speed}}</code>\n大小<code>{{.Current}}</code> / 未知</blockquote>"
batch_item_uploading: "<blockquote><b>⬆️ {{.Index}}/{{.Total}} 上传中</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\n速度<code>{{.Speed}}</code>\n大小<code>{{.Current}}</code> / <code>{{.Size}}</code></blockquote>"
batch_item_retrying: "<blockquote><b>🔁 {{.Index}}/{{.Total}} 上传重试</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\n重试次数<code>{{.Attempt}}/{{.Limit}}</code>\n失败前速度<code>{{.Speed}}</code>\n大小<code>{{.Current}}</code> / <code>{{.Size}}</code>\n原因<code>{{.Reason}}</code></blockquote>"
batch_item_confirming: "<blockquote><b>⏳ {{.Index}}/{{.Total}} 等待中</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\n<i>文件已上传,正在等待远端确认</i></blockquote>"
batch_summary_hidden_active: "🔄 另有 <code>{{.Count}}</code> 个文件正在处理"
batch_summary_confirming: "☁️ 已上传,等待整组发送:<code>{{.Count}}</code>"
batch_summary_failed: "❌ 失败:<code>{{.Count}}</code>"
batch_summary_skipped: "⏭️ 已跳过:<code>{{.Count}}</code>"
batch_done: "<b>✅ 处理完成</b>\n\n文件数: <code>{{.Count}}</code>\n总大小: <code>{{.Size}}</code>"
batch_done_with_skipped: "<b>⚠️ 处理完成</b>\n\n成功: <code>{{.Success}}</code>\n已跳过: <code>{{.Skipped}}</code>\n总大小: <code>{{.Size}}</code>"
batch_canceled: "<b>🚫 任务已取消</b>\n\n文件数: <code>{{.Total}}</code>\n已完成: <code>{{.Completed}}</code>\n未完成: <code>{{.Incomplete}}</code>\n已跳过: <code>{{.Skipped}}</code>"
batch_failed_item: "<b>❌ 处理失败</b>\n\n失败文件: <code>{{.Index}}. {{.Name}}</code>\n失败阶段: <code>{{.Stage}}</code>\n失败进度: <code>{{.Progress}}</code>\n失败前速度: <code>{{.Speed}}</code>\n原因: <code>{{.Reason}}</code>\n\n✅ 已完成: <code>{{.Completed}}</code>\n❌ 失败: <code>{{.Failed}}</code>\n⏹ 未完成: <code>{{.Incomplete}}</code>"
batch_failed_group: "<b>❌ 批量上传失败</b>\n\n受影响文件: <code>{{.Affected}} 个</code>\n原因: <code>{{.Reason}}</code>\n\n✅ 已完成: <code>{{.Completed}}</code>\n❌ 批次失败: <code>{{.Failed}}</code>\n⏹ 未完成: <code>{{.Incomplete}}</code>"
batch_failed_task: "<b>❌ 处理失败</b>\n\n原因: <code>{{.Reason}}</code>\n\n✅ 已完成: <code>{{.Completed}}</code>\n⏹ 未完成: <code>{{.Incomplete}}</code>"
batch_failure_stage_download: "下载"
batch_failure_stage_cache: "本地缓存"
batch_failure_stage_upload: "上传"
batch_failure_stage_confirm: "云端确认"
batch_failure_stage_batch_upload: "批量上传"
batch_failure_stage_internal: "任务内部"
single_status_header: "<b>📦 正在处理</b>"
single_downloading: "<blockquote><b>⬇️ 下载中</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\n速度<code>{{.Speed}}</code>\n大小<code>{{.Current}}</code> / <code>{{.Size}}</code>\n保存至<code>{{.Destination}}</code></blockquote>"
single_downloading_unknown: "<blockquote><b>⬇️ 下载中</b>\n<code>{{.Name}}</code>\n速度<code>{{.Speed}}</code>\n大小<code>{{.Current}}</code> / 未知\n保存至<code>{{.Destination}}</code></blockquote>"
single_uploading: "<blockquote><b>⬆️ 上传中</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\n速度<code>{{.Speed}}</code>\n大小<code>{{.Current}}</code> / <code>{{.Size}}</code>\n保存至<code>{{.Destination}}</code></blockquote>"
single_upload_retrying: "<blockquote><b>🔁 上传重试</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\n尝试次数<code>{{.Attempt}}</code>\n速度<code>{{.Speed}}</code>\n大小<code>{{.Current}}</code> / <code>{{.Size}}</code>\n保存至<code>{{.Destination}}</code></blockquote>"
single_done: "<b>✅ 处理完成</b>\n\n文件名<code>{{.Name}}</code>\n总大小<code>{{.Size}}</code>\n保存至<code>{{.Destination}}</code>"
single_canceled: "<b>🚫 任务已取消</b>\n\n文件名<code>{{.Name}}</code>"
single_failed: "<b>❌ 处理失败</b>\n\n文件名<code>{{.Name}}</code>\n原因<code>{{.Reason}}</code>"
downloading_prefix: "正在下载\n总大小: "
size_with_files: "{{.Size}} ({{.Count}} 个文件)"
size_with_resources: "{{.Size}} ({{.Count}} 个资源)"
processing_list_prefix: "\n正在处理:\n"
processing_none: " - 无"
avg_speed_prefix: "\n平均速度: "
current_progress_prefix: "\n当前进度: "
task_canceled: "任务已取消"
task_canceled_with_id: "处理已取消: {{.TaskID}}"
task_failed_with_error: "处理失败: {{.Error}}"
batch_done_prefix: "处理完成\n文件数: "
direct_done_prefix: "处理完成, 文件数量: "
parsed_start_prefix: "开始下载 {{.Site}} 的资源\n总大小: "
parsed_done_prefix: "处理完成, 资源数量: "
telegraph_start_prefix: "开始下载Telegraph\n图片数量: "
telegraph_progress_prefix: "正在下载\n当前进度: "
telegraph_done_prefix: "处理完成\n图片数量: "
file_start_prefix: "开始下载\n文件名: "
file_processing_prefix: "正在处理下载任务\n文件名: "
download_failed_prefix: "下载失败\n文件名: "
download_done_prefix: "下载完成\n文件名: "
file_size_prefix: "\n文件大小: "
save_path_prefix: "\n保存路径: "
total_size_prefix: "\n总大小: "
direct_start: "开始下载, 总大小: {{.SizeMB}} MB ({{.Count}} 个文件)"
file_name_prefix: "文件名: "
error_prefix: "\n错误: "
aria2_start: "等待 Aria2 下载完成 (GID: {{.GID}})..."
aria2_downloading: "Aria2 正在下载 (GID: {{.GID}})\n"

View File

@@ -0,0 +1,42 @@
package fsutil_test
import (
"errors"
"os"
"path/filepath"
"testing"
"github.com/krau/SaveAny-Bot/common/utils/fsutil"
)
func TestCloseAndRemove(t *testing.T) {
tests := []struct {
name string
preClose bool
}{
{name: "open file"},
{name: "already closed file", preClose: true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
filePath := filepath.Join(t.TempDir(), "cache-file")
file, err := fsutil.CreateFile(filePath)
if err != nil {
t.Fatalf("CreateFile() failed: %v", err)
}
if tt.preClose {
if err := file.Close(); err != nil {
t.Fatalf("Close() failed: %v", err)
}
}
if err := file.CloseAndRemove(); err != nil {
t.Fatalf("CloseAndRemove() failed: %v", err)
}
if _, err := os.Stat(filePath); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("cache file still exists after CloseAndRemove(): %v", err)
}
})
}
}

View File

@@ -1,6 +1,7 @@
package fsutil
import (
"errors"
"os"
"path/filepath"
"strings"
@@ -41,10 +42,11 @@ func (f *File) Remove() error {
}
func (f *File) CloseAndRemove() error {
if err := f.Close(); err != nil {
return err
closeErr := f.Close()
if errors.Is(closeErr, os.ErrClosed) {
closeErr = nil
}
return f.Remove()
return errors.Join(closeErr, f.Remove())
}
func CreateFile(fp string) (*File, error) {

View File

@@ -0,0 +1,27 @@
package fsutil
import (
"fmt"
"path"
"strings"
"github.com/rs/xid"
)
// UniquePath returns a non-taken path under basePath: the name itself, then
// numbered variants, then a random suffix.
func UniquePath(basePath, name string, exists func(candidate string) bool, maxAttempts int) string {
candidate := path.Join(basePath, name)
if !exists(candidate) {
return candidate
}
ext := path.Ext(name)
stem := strings.TrimSuffix(name, ext)
for i := 1; i <= maxAttempts; i++ {
candidate = path.Join(basePath, fmt.Sprintf("%s_%d%s", stem, i, ext))
if !exists(candidate) {
return candidate
}
}
return path.Join(basePath, fmt.Sprintf("%s_%s%s", stem, xid.New().String(), ext))
}

View File

@@ -17,7 +17,11 @@ type ProgressReadSeeker struct {
// Seek implements io.ReadSeeker.
func (pr *ProgressReadSeeker) Seek(offset int64, whence int) (int64, error) {
return pr.reader.Seek(offset, whence)
position, err := pr.reader.Seek(offset, whence)
if err == nil {
pr.read.Store(position)
}
return position, err
}
// NewProgressReader creates a new ProgressReader
@@ -54,7 +58,7 @@ func (pr *ProgressReadSeeker) Progress() float64 {
return float64(pr.read.Load()) / float64(pr.total.Load())
}
// Read returns the number of bytes read so far
// BytesRead returns the current tracked reader position.
func (pr *ProgressReadSeeker) BytesRead() int64 {
return pr.read.Load()
}

View File

@@ -0,0 +1,50 @@
package ioutil
import (
"bytes"
"io"
"testing"
)
func TestProgressReadSeekerTracksReads(t *testing.T) {
var gotRead, gotTotal int64
reader := NewProgressReader(bytes.NewReader([]byte("abcdef")), 6, func(read, total int64) {
gotRead = read
gotTotal = total
})
buffer := make([]byte, 4)
if _, err := io.ReadFull(reader, buffer); err != nil {
t.Fatalf("read failed: %v", err)
}
if gotRead != 4 || gotTotal != 6 {
t.Fatalf("progress = %d/%d, want 4/6", gotRead, gotTotal)
}
if reader.BytesRead() != 4 {
t.Fatalf("BytesRead() = %d, want 4", reader.BytesRead())
}
}
func TestProgressReadSeekerResetsPositionOnSeek(t *testing.T) {
reader := NewProgressReader(bytes.NewReader([]byte("abcdef")), 6, nil)
buffer := make([]byte, 4)
if _, err := io.ReadFull(reader, buffer); err != nil {
t.Fatalf("read failed: %v", err)
}
position, err := reader.Seek(0, io.SeekStart)
if err != nil {
t.Fatalf("seek failed: %v", err)
}
if position != 0 || reader.BytesRead() != 0 {
t.Fatalf("position after seek = %d (tracked %d), want 0", position, reader.BytesRead())
}
if _, err := io.ReadFull(reader, buffer[:2]); err != nil {
t.Fatalf("read after seek failed: %v", err)
}
if reader.BytesRead() != 2 {
t.Fatalf("BytesRead() after seek and read = %d, want 2", reader.BytesRead())
}
}

View File

@@ -0,0 +1,50 @@
// Package progressutil provides shared progress-update throttling for task
// progress trackers.
package progressutil
// updatesLevels picks the percent step by file size.
var updatesLevels = []struct {
size int64 // file size threshold
stepPercent int // minimum percent step between updates
}{
{10 << 20, 100},
{50 << 20, 20},
{200 << 20, 10},
{500 << 20, 5},
}
// ShouldUpdate reports whether a byte-based progress update should be shown.
func ShouldUpdate(total, downloaded int64, lastUpdatePercent int) bool {
if total <= 0 || downloaded <= 0 {
return false
}
percent := int((downloaded * 100) / total)
if percent <= lastUpdatePercent {
return false
}
step := updatesLevels[len(updatesLevels)-1].stepPercent
for _, lvl := range updatesLevels {
if total < lvl.size {
step = lvl.stepPercent
break
}
}
return percent >= lastUpdatePercent+step
}
// ShouldUpdateCount reports whether a count-based progress update (e.g. files
// downloaded so far) should be shown: every 10 units, or when finished.
func ShouldUpdateCount(downloaded, total int64) bool {
if total <= 0 || downloaded <= 0 {
return false
}
const step = int64(10)
if downloaded < step {
return downloaded == total
}
return downloaded%step == 0 || downloaded == total
}

View File

@@ -0,0 +1,35 @@
package tgutil
import (
stdhtml "html"
"github.com/gotd/td/telegram/message/entity"
messagehtml "github.com/gotd/td/telegram/message/html"
"github.com/gotd/td/telegram/message/styling"
"github.com/gotd/td/tg"
)
// EscapeHTMLTemplateData returns a copy of data with string values escaped for
// interpolation into Telegram HTML templates.
func EscapeHTMLTemplateData(data map[string]any) map[string]any {
escaped := make(map[string]any, len(data))
for key, value := range data {
if text, ok := value.(string); ok {
escaped[key] = stdhtml.EscapeString(text)
continue
}
escaped[key] = value
}
return escaped
}
// RenderHTML renders Telegram-compatible HTML into plain text and message
// entities.
func RenderHTML(markup string) (string, []tg.MessageEntityClass, error) {
var builder entity.Builder
if err := styling.Perform(&builder, messagehtml.String(nil, markup)); err != nil {
return "", nil, err
}
text, entities := builder.Complete()
return text, entities, nil
}

View File

@@ -0,0 +1,54 @@
package tgutil
import (
"testing"
"github.com/gotd/td/tg"
)
func TestEscapeHTMLTemplateDataDoesNotMutateInput(t *testing.T) {
input := map[string]any{
"Text": `<b>A&B</b>`,
"Count": 2,
}
escaped := EscapeHTMLTemplateData(input)
if got, want := escaped["Text"], "&lt;b&gt;A&amp;B&lt;/b&gt;"; got != want {
t.Fatalf("escaped text = %q, want %q", got, want)
}
if got := input["Text"]; got != `<b>A&B</b>` {
t.Fatalf("input was mutated: %q", got)
}
if got := escaped["Count"]; got != 2 {
t.Fatalf("non-string value = %v, want 2", got)
}
}
func TestRenderHTMLUsesTemplateStylesAndDecodesValues(t *testing.T) {
data := EscapeHTMLTemplateData(map[string]any{"Name": `<b>A&B</b>.bin`})
markup := `<blockquote><b>Uploading</b>
<code>` + data["Name"].(string) + `</code></blockquote>`
text, entities, err := RenderHTML(markup)
if err != nil {
t.Fatalf("RenderHTML() failed: %v", err)
}
if want := "Uploading\n<b>A&B</b>.bin"; text != want {
t.Fatalf("rendered text = %q, want %q", text, want)
}
var bold, code, blockquote int
for _, messageEntity := range entities {
switch messageEntity.(type) {
case *tg.MessageEntityBold:
bold++
case *tg.MessageEntityCode:
code++
case *tg.MessageEntityBlockquote:
blockquote++
}
}
if bold != 1 || code != 1 || blockquote != 1 {
t.Fatalf("entity counts = bold:%d code:%d blockquote:%d", bold, code, blockquote)
}
}

View File

@@ -2,6 +2,7 @@ package tgutil
import (
"fmt"
"sort"
"strconv"
"strings"
"unicode"
@@ -193,97 +194,6 @@ func getMessagesRange(ctx *ext.Context, chatID int64, minId, maxId int) ([]*tg.M
return result, nil
}
// [TODO]
// type MessageItem struct {
// Message *tg.Message
// Error error
// }
// func IterMessages(ctx *ext.Context, chatID int64, minId, maxId int) (<-chan MessageItem, error) {
// total := maxId - minId + 1
// ch := make(chan MessageItem, 100)
// go func() {
// defer close(ch)
// if !ctx.Self.Bot {
// perr := ctx.PeerStorage.GetInputPeerById(chatID)
// if perr == nil || perr.(*tg.InputPeerEmpty) != nil {
// ch <- MessageItem{
// Error: fmt.Errorf("peer not found: %d", chatID),
// }
// return
// }
// for i := 0; i < total; i += 100 {
// start := minId + i
// end := min(start+100, maxId)
// msgs, err := ctx.Raw.MessagesGetHistory(ctx, &tg.MessagesGetHistoryRequest{
// Peer: perr,
// OffsetID: start,
// AddOffset: start - end,
// Limit: 100,
// })
// if err != nil {
// ch <- MessageItem{
// Error: fmt.Errorf("failed to get messages: %w", err),
// }
// return
// }
// var msgClass []tg.MessageClass
// switch msgsv := msgs.(type) {
// case *tg.MessagesMessages:
// msgClass = msgsv.GetMessages()
// case *tg.MessagesMessagesSlice:
// msgClass = msgsv.GetMessages()
// case *tg.MessagesChannelMessages:
// msgClass = msgsv.GetMessages()
// default:
// ch <- MessageItem{
// Error: fmt.Errorf("unsupported message type: %T", msgsv),
// }
// continue
// }
// for _, msg := range msgClass {
// msg, ok := msg.AsNotEmpty()
// if !ok {
// continue
// }
// switch msg := msg.(type) {
// case *tg.Message:
// key := fmt.Sprintf("tgmsg:%d:%d:%d", ctx.Self.ID, chatID, msg.GetID())
// cache.Set(key, msg)
// ch <- MessageItem{
// Message: msg,
// }
// }
// }
// }
// } else {
// for i := 0; i < total; i += 100 {
// start := minId + i
// end := min(start+100, maxId)
// msgs, err := GetMessagesRange(ctx, chatID, start, end)
// if err != nil {
// ch <- MessageItem{
// Error: fmt.Errorf("failed to get messages: %w", err),
// }
// return
// }
// for _, msg := range msgs {
// if msg == nil {
// continue
// }
// ch <- MessageItem{
// Message: msg,
// }
// }
// }
// }
// }()
// return ch, nil
// }
func getMessageByID(ctx *ext.Context, chatID int64, msgID int) (*tg.Message, error) {
key := fmt.Sprintf("tgmsg:%d:%d:%d", ctx.Self.ID, chatID, msgID)
if msg, ok := cache.Get[*tg.Message](key); ok {
@@ -359,9 +269,16 @@ func GetGroupedMessages(ctx *ext.Context, chatID int64, msg *tg.Message) ([]*tg.
groupedMessages = append(groupedMessages, m)
}
}
sortMessagesByID(groupedMessages)
return groupedMessages, nil
}
func sortMessagesByID(messages []*tg.Message) {
sort.Slice(messages, func(i, j int) bool {
return messages[i].GetID() < messages[j].GetID()
})
}
func ExtractMessageEntityUrls(msg *tg.Message) []string {
if len(msg.Entities) == 0 {
return nil

View File

@@ -0,0 +1,18 @@
package tgutil
import (
"testing"
"github.com/gotd/td/tg"
)
func TestSortMessagesByID(t *testing.T) {
messages := []*tg.Message{{ID: 9}, {ID: 3}, {ID: 7}}
sortMessagesByID(messages)
want := []int{3, 7, 9}
for i := range messages {
if messages[i].GetID() != want[i] {
t.Fatalf("message %d has ID %d, want %d", i, messages[i].GetID(), want[i])
}
}
}

View File

@@ -33,6 +33,36 @@ secret = ""
# 转存完成后删除 Aria2 下载的本地文件
remove_after_transfer = true
# yt-dlp 视频下载配置
[ytdlp]
# 默认下载的最高视频清晰度 (按高度限制), 如 1080, 720, 480; 0 表示不限制 (下载最佳画质)
# 仅在使用 /ytdlp 命令且未手动指定任何参数时生效
max_height = 1080
# 直接指定 yt-dlp format 选择表达式, 留空则使用 max_height
# 设置后优先级高于 max_height, 例如: "bv*[height<=720]+ba/b"
format = ""
# 下载后转封装的视频容器格式, 留空则不转封装. 默认 mp4
recode = "mp4"
# 解析器配置
[parser]
# 启用 JS 解析器插件 (Go 内置解析器默认启用)
plugin_enable = false
# 插件目录, 可以是多个目录
plugin_dirs = ["./plugins"]
# 解析器默认代理
proxy = ""
# Twitter/X 解析器配置
[parser.twitter]
# 自定义 API 域名
api_domain = "api.fxtwitter.com"
# 单独为此解析器指定代理 (留空则使用 [parser] 中的 proxy)
# proxy = "http://127.0.0.1:7890"
# Kemono 解析器配置 (暂无可配置项, 留空即可)
[parser.kemono]
# HTTP API 配置
[api]
# 启用 HTTP API

View File

@@ -10,13 +10,4 @@ type hookExecConfig struct {
TaskSuccess string `toml:"task_success" mapstructure:"task_success" json:"task_success"`
TaskFail string `toml:"task_fail" mapstructure:"task_fail" json:"task_fail"`
TaskCancel string `toml:"task_cancel" mapstructure:"task_cancel" json:"task_cancel"`
// TaskTypes map[string]hookExecOnTypeConfig `toml:"task_types" mapstructure:"task_types" json:"task_types"` // [TODO]
}
// type hookExecOnTypeConfig struct {
// TaskBeforeStart string `toml:"task_before_start" mapstructure:"task_before_start" json:"task_before_start"`
// TaskSuccess string `toml:"task_success" mapstructure:"task_success" json:"task_success"`
// TaskFail string `toml:"task_fail" mapstructure:"task_fail" json:"task_fail"`
// TaskCancel string `toml:"task_cancel" mapstructure:"task_cancel" json:"task_cancel"`
// }

View File

@@ -8,14 +8,16 @@ import (
type TelegramStorageConfig struct {
BaseConfig
ChatID int64 `toml:"chat_id" mapstructure:"chat_id" json:"chat_id"`
ForceFile bool `toml:"force_file" mapstructure:"force_file" json:"force_file"`
RateLimit int `toml:"rate_limit" mapstructure:"rate_limit" json:"rate_limit"`
RateBurst int `toml:"rate_burst" mapstructure:"rate_burst" json:"rate_burst"`
SkipLarge bool `toml:"skip_large" mapstructure:"skip_large" json:"skip_large"` // skip files larger than Telegram limit(2GB)
// split files larger than Telegram limit(2GB) into parts of specified size, in MB, leave 0 to set default(2000MB)
ChatID int64 `toml:"chat_id" mapstructure:"chat_id" json:"chat_id"`
ForceFile bool `toml:"force_file" mapstructure:"force_file" json:"force_file"`
RateLimit int `toml:"rate_limit" mapstructure:"rate_limit" json:"rate_limit"`
RateBurst int `toml:"rate_burst" mapstructure:"rate_burst" json:"rate_burst"`
SkipLarge bool `toml:"skip_large" mapstructure:"skip_large" json:"skip_large"` // skip files larger than Telegram limit(2GB)
SplitLargeVideo bool `toml:"split_large_video" mapstructure:"split_large_video" json:"split_large_video"`
// split files larger than the uploader account limit into parts of specified size, in MB
// leave 0 to use the account limit (2000MB for bots/regular users, 4000MB for Premium users)
// only effective when SkipLarge is false
// use zip when splitting
// use zip when splitting non-video files or when lossless video splitting is disabled/unavailable
SplitSizeMB int64 `toml:"split_size_mb" mapstructure:"split_size_mb" json:"split_size_mb"`
}

View File

@@ -9,6 +9,7 @@ import (
"strings"
"time"
"github.com/charmbracelet/log"
"github.com/duke-git/lancet/v2/slice"
"github.com/krau/SaveAny-Bot/config/storage"
"github.com/spf13/viper"
@@ -35,6 +36,7 @@ type Config struct {
Storages []storage.StorageConfig `toml:"-" mapstructure:"-" json:"storages"`
Parser parserConfig `toml:"parser" mapstructure:"parser" json:"parser"`
Hook hookConfig `toml:"hook" mapstructure:"hook" json:"hook"`
Ytdlp YtdlpConfig `toml:"ytdlp" mapstructure:"ytdlp" json:"ytdlp"`
}
type aria2Config struct {
@@ -67,6 +69,13 @@ func (c Config) GetStorageByName(name string) storage.StorageConfig {
}
func Init(ctx context.Context, configFile ...string) error {
logger := log.FromContext(ctx)
// Reset side tables for re-init.
storages = nil
userIDs = nil
userStorages = make(map[int64][]string)
viper.SetConfigType("toml")
viper.SetEnvPrefix("SAVEANY")
viper.AutomaticEnv()
@@ -75,11 +84,13 @@ func Init(ctx context.Context, configFile ...string) error {
// 如果指定了配置文件路径,则使用指定的配置文件
// 配置文件支持传入一个 http(s) URL 地址
loadedFromURL := false
if len(configFile) > 0 && configFile[0] != "" {
cfg := configFile[0]
if strings.HasPrefix(cfg, "http://") || strings.HasPrefix(cfg, "https://") {
// 使用远程配置文件
resp, err := http.Get(cfg)
client := &http.Client{Timeout: 30 * time.Second}
resp, err := client.Get(cfg)
if err != nil {
return fmt.Errorf("failed to fetch remote config file: %w", err)
}
@@ -90,6 +101,7 @@ func Init(ctx context.Context, configFile ...string) error {
if err := viper.ReadConfig(resp.Body); err != nil {
return fmt.Errorf("failed to read remote config file: %w", err)
}
loadedFromURL = true
} else {
viper.SetConfigFile(cfg)
}
@@ -131,19 +143,24 @@ func Init(ctx context.Context, configFile ...string) error {
"api.host": "0.0.0.0",
"api.port": 8080,
"api.token": "",
// yt-dlp
"ytdlp.recode": "mp4",
}
for key, value := range defaultConfigs {
viper.SetDefault(key, value)
}
if err := viper.ReadInConfig(); err != nil {
fmt.Println("Error reading config file, ", err)
return err
if !loadedFromURL {
if err := viper.ReadInConfig(); err != nil {
logger.Errorf("Error reading config file: %v", err)
return err
}
}
if err := viper.Unmarshal(cfg); err != nil {
fmt.Println("Error unmarshalling config file, ", err)
logger.Errorf("Error unmarshalling config file: %v", err)
return err
}

13
config/ytdlp.go Normal file
View File

@@ -0,0 +1,13 @@
package config
type YtdlpConfig struct {
// MaxHeight limits the video resolution by height in pixels (e.g. 1080, 720).
// 0 means no limit (best available). Ignored when Format is set.
MaxHeight int `toml:"max_height" mapstructure:"max_height" json:"max_height"`
// Format is a raw yt-dlp format selector (-f). When set, it takes precedence
// over MaxHeight and gives the user full control.
Format string `toml:"format" mapstructure:"format" json:"format"`
// Recode is the target video container yt-dlp recodes into (e.g. mp4).
// Empty disables recoding.
Recode string `toml:"recode" mapstructure:"recode" json:"recode"`
}

View File

@@ -3,14 +3,27 @@ package core
import (
"context"
"errors"
"sync"
"github.com/charmbracelet/log"
"github.com/krau/SaveAny-Bot/config"
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
"github.com/krau/SaveAny-Bot/pkg/queue"
"github.com/krau/SaveAny-Bot/pkg/taskevent"
)
var queueInstance *queue.TaskQueue[Executable]
var (
queueOnce sync.Once
queueInstance *queue.TaskQueue[Executable]
)
// initQueue lazily creates the shared task queue.
func initQueue() *queue.TaskQueue[Executable] {
queueOnce.Do(func() {
queueInstance = queue.NewTaskQueue[Executable]()
})
return queueInstance
}
type Executable interface {
Type() tasktype.TaskType
@@ -30,11 +43,14 @@ func worker(ctx context.Context, qe *queue.TaskQueue[Executable], semaphore chan
break // queue closed and empty
}
exe := qtask.Data
taskCtx := qtask.Context()
logger.Infof("Processing task: %s", exe.TaskID())
if err := ExecCommandString(qtask.Context(), execHooks.TaskBeforeStart); err != nil {
taskevent.Emit(taskCtx, taskevent.Event{TaskID: exe.TaskID(), Phase: taskevent.PhaseStart})
if err := ExecCommandString(taskCtx, execHooks.TaskBeforeStart); err != nil {
logger.Errorf("Failed to execute before start hook for task %s: %v", exe.TaskID(), err)
}
if err := exe.Execute(qtask.Context()); err != nil {
err = exe.Execute(taskCtx)
if err != nil {
if errors.Is(err, context.Canceled) {
logger.Infof("Task %s was canceled", exe.TaskID())
if err := ExecCommandString(ctx, execHooks.TaskCancel); err != nil {
@@ -52,6 +68,7 @@ func worker(ctx context.Context, qe *queue.TaskQueue[Executable], semaphore chan
logger.Errorf("Failed to execute success hook for task %s: %v", exe.TaskID(), err)
}
}
taskevent.Emit(taskCtx, taskevent.Event{TaskID: exe.TaskID(), Phase: taskevent.PhaseDone, Err: err})
qe.Done(qtask.ID)
<-semaphore
}
@@ -60,17 +77,22 @@ func worker(ctx context.Context, qe *queue.TaskQueue[Executable], semaphore chan
func Run(ctx context.Context) {
log.FromContext(ctx).Info("Start processing tasks...")
semaphore := make(chan struct{}, config.C().Workers)
if queueInstance == nil {
queueInstance = queue.NewTaskQueue[Executable]()
}
q := initQueue()
for range config.C().Workers {
go worker(ctx, queueInstance, semaphore)
go worker(ctx, q, semaphore)
}
}
// Close stops the queue and unblocks workers in Get.
func Close() {
if q := initQueue(); q != nil {
q.Close()
}
}
func AddTask(ctx context.Context, task Executable) error {
return queueInstance.Add(queue.NewTask(ctx, task.TaskID(), task.Title(), task))
return initQueue().Add(queue.NewTask(ctx, task.TaskID(), task.Title(), task))
}
func CancelTask(ctx context.Context, id string) error {

View File

@@ -6,12 +6,14 @@ import (
"fmt"
"os"
"path/filepath"
"strconv"
"time"
"github.com/charmbracelet/log"
"github.com/krau/SaveAny-Bot/config"
"github.com/krau/SaveAny-Bot/pkg/aria2"
"github.com/krau/SaveAny-Bot/pkg/enums/ctxkey"
"github.com/krau/SaveAny-Bot/pkg/taskevent"
)
// Execute implements core.Executable.
@@ -77,6 +79,12 @@ func (t *Task) waitForDownload(ctx context.Context) error {
if t.Progress != nil {
t.Progress.OnProgress(ctx, t, status)
}
taskevent.Emit(ctx, taskevent.Event{
TaskID: t.ID,
Phase: taskevent.PhaseProgress,
TotalBytes: parseInt64(status.TotalLength),
DownloadedBytes: parseInt64(status.CompletedLength),
})
// Check if download is complete
if status.IsDownloadComplete() {
@@ -248,3 +256,16 @@ func (t *Task) cancelAria2Download() {
logger.Debugf("Failed to remove download result for %s: %v", t.GID, err)
}
}
// parseInt64 parses an aria2 status string (decimal bytes) into int64,
// returning 0 on failure so it can be used directly in progress events.
func parseInt64(s string) int64 {
if s == "" {
return 0
}
n, err := strconv.ParseInt(s, 10, 64)
if err != nil {
return 0
}
return n
}

View File

@@ -2,10 +2,13 @@ package batchtfile
import (
"context"
"errors"
"fmt"
"io"
"os"
"path"
"sync"
"time"
"github.com/charmbracelet/log"
"github.com/duke-git/lancet/v2/retry"
@@ -14,44 +17,318 @@ import (
"github.com/krau/SaveAny-Bot/common/utils/ioutil"
"github.com/krau/SaveAny-Bot/config"
"github.com/krau/SaveAny-Bot/pkg/enums/ctxkey"
"github.com/krau/SaveAny-Bot/pkg/storagetypes"
"github.com/krau/SaveAny-Bot/pkg/taskevent"
"github.com/krau/SaveAny-Bot/storage"
"golang.org/x/sync/errgroup"
)
type executionGroup struct {
elems []*TaskElement
batchSaver storage.StorageBatchSaver
}
func (g executionGroup) usesBatchSaver() bool {
return g.batchSaver != nil
}
func (t *Task) Execute(ctx context.Context) error {
logger := log.FromContext(ctx).WithPrefix(fmt.Sprintf("batch_file[%s]", t.ID))
logger.Info("Starting batch file task")
t.Progress.OnStart(ctx, t)
workers := config.C().Workers
eg, gctx := errgroup.WithContext(ctx)
eg.SetLimit(workers)
for _, elem := range t.elems {
eg.Go(func() error {
t.processingMu.RLock()
if t.processing[elem.ID] != nil {
return fmt.Errorf("element with ID %s is already being processed", elem.ID)
}
t.processingMu.RUnlock()
t.processingMu.Lock()
t.processing[elem.ID] = &elem
t.processingMu.Unlock()
defer func() {
t.processingMu.Lock()
delete(t.processing, elem.ID)
t.processingMu.Unlock()
}()
return t.processElement(gctx, elem)
})
if t.Progress != nil {
t.Progress.OnStart(ctx, t)
}
groups := t.executionGroups()
var err error
for i := 0; i < len(groups); {
if groups[i].usesBatchSaver() {
err = t.processBatch(ctx, groups[i])
i++
} else {
end := i + 1
for end < len(groups) && !groups[end].usesBatchSaver() {
end++
}
elems := make([]*TaskElement, 0, end-i)
for _, group := range groups[i:end] {
elems = append(elems, group.elems...)
}
err = t.processElements(ctx, elems)
i = end
}
if err != nil {
if !t.IgnoreErrors || errors.Is(err, context.Canceled) {
break
}
logger.Warnf("Group processing failed (ignored): %v", err)
err = nil
}
}
err := eg.Wait()
if err != nil {
logger.Errorf("Error during batch file processing: %v", err)
} else {
logger.Info("Batch file task completed successfully")
}
t.Progress.OnDone(ctx, t, err)
t.finishItems(err)
if t.Progress != nil {
t.Progress.OnDone(ctx, t, err)
}
return err
}
// notifyProgress reports a progress update to the optional tracker.
func (t *Task) notifyProgress(ctx context.Context) {
if t.Progress != nil {
t.Progress.OnProgress(ctx, t)
}
}
func (t *Task) executionGroups() []executionGroup {
groups := make([]executionGroup, 0, len(t.elems))
for i := 0; i < len(t.elems); {
elem := &t.elems[i]
batchSaver, batchCapable := elem.Storage.(storage.StorageBatchSaver)
if !batchCapable || elem.sourceGroupKey == "" {
groups = append(groups, executionGroup{elems: []*TaskElement{elem}})
i++
continue
}
end := i + 1
for end < len(t.elems) {
next := &t.elems[end]
if next.Storage != elem.Storage || next.sourceGroupKey != elem.sourceGroupKey {
break
}
end++
}
elems := make([]*TaskElement, 0, end-i)
for j := i; j < end; j++ {
elems = append(elems, &t.elems[j])
}
groups = append(groups, executionGroup{elems: elems, batchSaver: batchSaver})
i = end
}
return groups
}
func (t *Task) processElements(ctx context.Context, elems []*TaskElement) error {
eg, gctx := errgroup.WithContext(ctx)
eg.SetLimit(config.C().Workers)
for _, elem := range elems {
eg.Go(func() error {
if err := t.markProcessing(ctx, elem); err != nil {
return err
}
defer t.unmarkProcessing(elem.ID)
err := t.processElement(gctx, *elem)
if err != nil && t.IgnoreErrors && !errors.Is(err, context.Canceled) {
// Per-item failure: keep siblings running.
log.FromContext(ctx).Warnf("Element %s failed (ignored): %v", elem.ID, err)
return nil
}
return err
})
}
return eg.Wait()
}
func (t *Task) processBatch(ctx context.Context, group executionGroup) error {
defer func() {
for _, elem := range group.elems {
if err := os.Remove(elem.localPath); err != nil && !os.IsNotExist(err) {
log.FromContext(ctx).Warnf("Failed to cleanup batch cache file %s: %v", elem.localPath, err)
}
}
}()
type downloadResult struct {
elem *TaskElement
err error
}
results := make([]downloadResult, len(group.elems))
var resultsMu sync.Mutex
eg, gctx := errgroup.WithContext(ctx)
eg.SetLimit(config.C().Workers)
for i, elem := range group.elems {
eg.Go(func() error {
if err := t.markProcessing(ctx, elem); err != nil {
return err
}
defer t.unmarkProcessing(elem.ID)
err := t.downloadElement(gctx, elem)
// Store by original index.
resultsMu.Lock()
results[i] = downloadResult{elem: elem, err: err}
resultsMu.Unlock()
if err != nil && t.IgnoreErrors && !errors.Is(err, context.Canceled) {
// Per-item failure: keep siblings running.
log.FromContext(ctx).Warnf("Element %s failed (ignored): %v", elem.ID, err)
return nil
}
return err
})
}
if err := eg.Wait(); err != nil {
return err
}
// Upload only successfully downloaded elements.
successElems := make([]*TaskElement, 0, len(group.elems))
for _, r := range results {
if r.err == nil {
successElems = append(successElems, r.elem)
}
}
if len(successElems) == 0 {
return fmt.Errorf("all elements failed to download")
}
items := make([]storagetypes.BatchItem, 0, len(successElems))
openFiles := make([]*os.File, 0, len(successElems))
defer func() {
for _, file := range openFiles {
if err := file.Close(); err != nil {
log.FromContext(ctx).Warnf("Failed to close batch cache file %s: %v", file.Name(), err)
}
}
}()
for _, elem := range successElems {
file, err := os.Open(elem.localPath)
if err != nil {
t.markItemFailed(elem.ID, FailureStageCache, err)
return fmt.Errorf("failed to open cache file: %w", err)
}
stat, err := file.Stat()
if err != nil {
file.Close()
t.markItemFailed(elem.ID, FailureStageCache, err)
return fmt.Errorf("failed to get cache file stat: %w", err)
}
openFiles = append(openFiles, file)
items = append(items, storagetypes.BatchItem{
Reader: file,
StoragePath: elem.Path,
Size: stat.Size(),
SourceGroupKey: elem.sourceGroupKey,
Caption: elem.sourceCaption,
PreserveCaption: elem.preserveCaption,
})
}
for index, item := range items {
t.recordDownloadComplete(successElems[index].ID, item.Size)
}
return t.saveBatchItems(ctx, successElems, items)
}
func (t *Task) saveBatchItems(ctx context.Context, successElems []*TaskElement, items []storagetypes.BatchItem) error {
t.startUpload(ctx)
if progressSaver, ok := successElems[0].Storage.(storage.StorageBatchProgressSaver); ok {
err := progressSaver.SaveBatchWithProgress(ctx, items, func(index int, uploaded, total int64) {
if index < 0 || index >= len(successElems) {
return
}
t.uploadCallback(ctx, successElems[index].ID)(uploaded, total)
})
if err != nil {
for _, elem := range successElems {
t.markItemFailed(elem.ID, FailureStageBatchUpload, err)
}
t.notifyStateChange(ctx)
return fmt.Errorf("failed to save batch: %w", err)
}
for index, elem := range successElems {
t.uploadCallback(ctx, elem.ID)(items[index].Size, items[index].Size)
t.markItemCompleted(elem.ID)
}
t.notifyStateChange(ctx)
return nil
}
for i := range items {
items[i].Reader = ioutil.NewProgressReader(
items[i].Reader,
items[i].Size,
t.uploadCallback(ctx, successElems[i].ID),
)
}
if err := successElems[0].Storage.(storage.StorageBatchSaver).SaveBatch(ctx, items); err != nil {
for _, elem := range successElems {
t.markItemFailed(elem.ID, FailureStageBatchUpload, err)
}
t.notifyStateChange(ctx)
return fmt.Errorf("failed to save batch: %w", err)
}
for index, elem := range successElems {
t.uploadCallback(ctx, elem.ID)(items[index].Size, items[index].Size)
t.markItemCompleted(elem.ID)
}
t.notifyStateChange(ctx)
return nil
}
func (t *Task) markProcessing(ctx context.Context, elem *TaskElement) error {
t.processingMu.Lock()
if t.processing[elem.ID] != nil {
t.processingMu.Unlock()
return fmt.Errorf("element with ID %s is already being processed", elem.ID)
}
t.processing[elem.ID] = elem
t.processingMu.Unlock()
t.markItemActive(elem.ID, elem.stream, time.Now())
t.notifyProgress(ctx)
return nil
}
func (t *Task) unmarkProcessing(id string) {
t.processingMu.Lock()
delete(t.processing, id)
t.processingMu.Unlock()
}
func (t *Task) downloadElement(ctx context.Context, elem *TaskElement) error {
logger := log.FromContext(ctx).WithPrefix(fmt.Sprintf("file[%s]", elem.File.Name()))
logger.Info("Starting file download")
localFile, err := fsutil.CreateFile(elem.localPath)
if err != nil {
t.markItemFailed(elem.ID, FailureStageCache, err)
t.notifyStateChange(ctx)
return fmt.Errorf("failed to create local file: %w", err)
}
wrAt := ioutil.NewProgressWriterAt(localFile, func(n int) {
t.recordItemDownload(elem.ID, int64(n), time.Now())
downloaded := t.downloaded.Add(int64(n))
t.notifyProgress(ctx)
taskevent.Emit(ctx, taskevent.Event{
TaskID: t.ID,
Phase: taskevent.PhaseProgress,
TotalBytes: t.totalSize,
DownloadedBytes: downloaded,
})
})
_, downloadErr := tdler.NewDownloader(elem.File).Parallel(ctx, wrAt)
closeErr := localFile.Close()
if downloadErr != nil {
t.markItemFailed(elem.ID, FailureStageDownload, downloadErr)
t.notifyStateChange(ctx)
return fmt.Errorf("failed to download file: %w", downloadErr)
}
if closeErr != nil {
t.markItemFailed(elem.ID, FailureStageCache, closeErr)
t.notifyStateChange(ctx)
return fmt.Errorf("failed to close cache file: %w", closeErr)
}
logger.Info("File downloaded successfully")
if path.Ext(elem.FileName()) == "" {
if ext := fsutil.DetectFileExt(elem.localPath); ext != "" {
elem.Path += ext
}
}
t.markItemDownloaded(elem.ID)
t.notifyProgress(ctx)
return nil
}
func (t *Task) processElement(ctx context.Context, elem TaskElement) error {
logger := log.FromContext(ctx).WithPrefix(fmt.Sprintf("file[%s]", elem.File.Name()))
if elem.stream {
@@ -59,11 +336,23 @@ func (t *Task) processElement(ctx context.Context, elem TaskElement) error {
defer pr.Close()
errg, uploadCtx := errgroup.WithContext(ctx)
errg.Go(func() error {
return elem.Storage.Save(uploadCtx, pr, elem.Path)
err := elem.Storage.Save(uploadCtx, pr, elem.Path)
if err != nil {
t.markItemFailed(elem.ID, FailureStageUpload, err)
t.notifyStateChange(ctx)
}
return err
})
wr := ioutil.NewProgressWriter(pw, func(n int) {
t.downloaded.Add(int64(n))
t.Progress.OnProgress(ctx, t)
t.recordItemDownload(elem.ID, int64(n), time.Now())
downloaded := t.downloaded.Add(int64(n))
t.notifyProgress(ctx)
taskevent.Emit(ctx, taskevent.Event{
TaskID: t.ID,
Phase: taskevent.PhaseProgress,
TotalBytes: t.totalSize,
DownloadedBytes: downloaded,
})
})
errg.Go(func() error {
defer pw.Close()
@@ -71,6 +360,8 @@ func (t *Task) processElement(ctx context.Context, elem TaskElement) error {
_, err := tdler.NewDownloader(elem.File).Stream(uploadCtx, wr)
if err != nil {
logger.Errorf("Failed to download file: %v", err)
t.markItemFailed(elem.ID, FailureStageDownload, err)
t.notifyStateChange(ctx)
pw.CloseWithError(err)
}
return err
@@ -78,12 +369,22 @@ func (t *Task) processElement(ctx context.Context, elem TaskElement) error {
if err := errg.Wait(); err != nil {
return fmt.Errorf("failed to download file in stream mode: %w", err)
}
// Streamed bytes are the uploaded bytes.
var streamedBytes int64
t.updateItem(elem.ID, func(item *itemProgressState) {
streamedBytes = item.downloaded
})
t.recordDownloadComplete(elem.ID, streamedBytes)
t.markItemCompleted(elem.ID)
t.notifyStateChange(ctx)
logger.Info("File downloaded successfully in stream mode")
return nil
}
logger.Info("Starting file download")
localFile, err := fsutil.CreateFile(elem.localPath)
if err != nil {
t.markItemFailed(elem.ID, FailureStageCache, err)
t.notifyStateChange(ctx)
return fmt.Errorf("failed to create local file: %w", err)
}
defer func() {
@@ -92,11 +393,20 @@ func (t *Task) processElement(ctx context.Context, elem TaskElement) error {
}
}()
wrAt := ioutil.NewProgressWriterAt(localFile, func(n int) {
t.downloaded.Add(int64(n))
t.Progress.OnProgress(ctx, t)
t.recordItemDownload(elem.ID, int64(n), time.Now())
downloaded := t.downloaded.Add(int64(n))
t.notifyProgress(ctx)
taskevent.Emit(ctx, taskevent.Event{
TaskID: t.ID,
Phase: taskevent.PhaseProgress,
TotalBytes: t.totalSize,
DownloadedBytes: downloaded,
})
})
_, err = tdler.NewDownloader(elem.File).Parallel(ctx, wrAt)
if err != nil {
t.markItemFailed(elem.ID, FailureStageDownload, err)
t.notifyStateChange(ctx)
return fmt.Errorf("failed to download file: %w", err)
}
logger.Info("File downloaded successfully")
@@ -109,21 +419,50 @@ func (t *Task) processElement(ctx context.Context, elem TaskElement) error {
var fileStat os.FileInfo
fileStat, err = os.Stat(elem.localPath)
if err != nil {
t.markItemFailed(elem.ID, FailureStageCache, err)
t.notifyStateChange(ctx)
return fmt.Errorf("failed to get file stat: %w", err)
}
t.recordDownloadComplete(elem.ID, fileStat.Size())
vctx := context.WithValue(ctx, ctxkey.ContentLength, fileStat.Size())
t.startUpload(vctx)
onProgress := t.uploadCallback(vctx, elem.ID)
attempt := 0
retryLimit := int(config.C().Retry)
lastFailureStage := FailureStageUpload
err = retry.Retry(func() error {
attempt++
var file *os.File
file, err = os.Open(elem.localPath)
if err != nil {
lastFailureStage = FailureStageCache
t.markItemRetry(elem.ID, lastFailureStage, attempt, retryLimit, err)
t.notifyStateChange(vctx)
return fmt.Errorf("failed to open cache file: %w", err)
}
defer file.Close()
if err = elem.Storage.Save(vctx, file, elem.Path); err != nil {
onProgress(0, fileStat.Size())
if progressSaver, ok := elem.Storage.(storage.StorageProgressSaver); ok {
err = progressSaver.SaveWithProgress(vctx, file, elem.Path, onProgress)
} else {
err = elem.Storage.Save(vctx, ioutil.NewProgressReader(file, fileStat.Size(), onProgress), elem.Path)
}
if err != nil {
logger.Errorf("Failed to save file: %s, retrying...", err)
lastFailureStage = t.itemFailureStage(elem.ID)
t.markItemRetry(elem.ID, lastFailureStage, attempt, retryLimit, err)
t.notifyStateChange(vctx)
return err
}
return nil
}, retry.Context(vctx), retry.RetryTimes(uint(config.C().Retry)))
if err == nil {
onProgress(fileStat.Size(), fileStat.Size())
t.markItemCompleted(elem.ID)
t.notifyStateChange(vctx)
} else {
t.markItemFailed(elem.ID, lastFailureStage, err)
t.notifyStateChange(vctx)
}
return err
}

View File

@@ -0,0 +1,57 @@
package batchtfile
import (
"testing"
"github.com/gotd/td/tg"
"github.com/krau/SaveAny-Bot/pkg/tfile"
tgstorage "github.com/krau/SaveAny-Bot/storage/telegram"
)
func TestExecutionGroupsPreserveSourceAlbums(t *testing.T) {
stor := new(tgstorage.Telegram)
otherStor := new(tgstorage.Telegram)
task := Task{elems: []TaskElement{
{Storage: stor, sourceGroupKey: "album-1"},
{Storage: stor, sourceGroupKey: "album-1"},
{Storage: stor},
{Storage: stor, sourceGroupKey: "album-2"},
{Storage: stor, sourceGroupKey: "album-2"},
{Storage: otherStor, sourceGroupKey: "album-2"},
}}
groups := task.executionGroups()
wantSizes := []int{2, 1, 2, 1}
wantBatch := []bool{true, false, true, true}
if len(groups) != len(wantSizes) {
t.Fatalf("got %d groups, want %d", len(groups), len(wantSizes))
}
for i := range groups {
if got := len(groups[i].elems); got != wantSizes[i] {
t.Errorf("group %d has %d elements, want %d", i, got, wantSizes[i])
}
if got := groups[i].usesBatchSaver(); got != wantBatch[i] {
t.Errorf("group %d batch=%v, want %v", i, got, wantBatch[i])
}
}
}
func TestSourceMetadataPreservesAlbumIdentityAndCaption(t *testing.T) {
msg := &tg.Message{
PeerID: &tg.PeerChannel{ChannelID: 77},
Message: "original caption",
}
msg.SetGroupedID(42)
file := tfile.NewTGFile(nil, nil, 0, "photo.jpg", tfile.WithMessage(msg))
groupKey, caption, preserveCaption := sourceMetadata(file)
if groupKey != "*tg.PeerChannel:77:42" {
t.Fatalf("group key = %q, want %q", groupKey, "*tg.PeerChannel:77:42")
}
if caption != "original caption" {
t.Fatalf("caption = %q, want original caption", caption)
}
if !preserveCaption {
t.Fatal("preserveCaption = false, want true")
}
}

View File

@@ -0,0 +1,360 @@
package batchtfile
import (
"context"
"errors"
"strings"
"time"
)
const (
transferSpeedWindow = 5 * time.Second
transferSamplePeriod = 250 * time.Millisecond
)
// ItemPhase describes the current lifecycle stage of one batch item.
type ItemPhase uint8
const (
ItemPhaseWaiting ItemPhase = iota
ItemPhaseDownloading
ItemPhaseTransferring
ItemPhaseDownloaded
ItemPhaseUploading
ItemPhaseRetrying
ItemPhaseConfirming
ItemPhaseCompleted
ItemPhaseFailed
ItemPhaseStopped
)
// FailureStage identifies the operation that failed for one batch item.
type FailureStage uint8
const (
FailureStageNone FailureStage = iota
FailureStageDownload
FailureStageCache
FailureStageUpload
FailureStageConfirm
FailureStageBatchUpload
FailureStageInternal
)
// TaskItemProgress is an immutable progress snapshot for one batch item.
type TaskItemProgress struct {
Index int
ID string
Name string
Size int64
Downloaded int64
Uploaded int64
DownloadSpeed float64
UploadSpeed float64
Phase ItemPhase
FailureStage FailureStage
RetryAttempt int
RetryLimit int
Error string
}
type transferSample struct {
at time.Time
bytes int64
}
type transferMeter struct {
samples []transferSample
latest transferSample
hasData bool
}
func (m *transferMeter) record(now time.Time, transferred int64) {
if m.hasData && transferred < m.latest.bytes {
m.reset()
}
if m.hasData && now.Before(m.latest.at) {
now = m.latest.at
}
sample := transferSample{at: now, bytes: transferred}
m.latest = sample
m.hasData = true
if len(m.samples) == 0 {
m.samples = append(m.samples, sample)
return
}
if now.Sub(m.samples[len(m.samples)-1].at) >= transferSamplePeriod {
m.samples = append(m.samples, sample)
}
cutoff := now.Add(-transferSpeedWindow)
for len(m.samples) > 1 && m.samples[0].at.Before(cutoff) {
m.samples = m.samples[1:]
}
}
func (m *transferMeter) speed() float64 {
if !m.hasData || len(m.samples) == 0 {
return 0
}
first := m.samples[0]
last := m.latest
elapsed := last.at.Sub(first.at).Seconds()
if elapsed <= 0 || last.bytes <= first.bytes {
return 0
}
return float64(last.bytes-first.bytes) / elapsed
}
func (m *transferMeter) reset() {
m.samples = m.samples[:0]
m.latest = transferSample{}
m.hasData = false
}
type itemProgressState struct {
index int
id string
name string
expectedSize int64
actualSize int64
downloaded int64
uploaded int64
phase ItemPhase
failureStage FailureStage
retryAttempt int
retryLimit int
err string
downloadMeter transferMeter
uploadMeter transferMeter
}
func newItemProgressStates(elems []TaskElement) ([]itemProgressState, map[string]int) {
states := make([]itemProgressState, 0, len(elems))
index := make(map[string]int, len(elems))
for i, elem := range elems {
name := ""
size := int64(0)
if elem.File != nil {
name = elem.File.Name()
size = elem.File.Size()
}
states = append(states, itemProgressState{
index: i + 1,
id: elem.ID,
name: name,
expectedSize: size,
phase: ItemPhaseWaiting,
})
index[elem.ID] = i
}
return states, index
}
func (t *Task) updateItem(id string, update func(*itemProgressState)) bool {
t.itemMu.Lock()
defer t.itemMu.Unlock()
index, ok := t.itemIndex[id]
if !ok || index < 0 || index >= len(t.itemStates) {
return false
}
update(&t.itemStates[index])
return true
}
func (t *Task) markItemActive(id string, stream bool, now time.Time) {
t.updateItem(id, func(item *itemProgressState) {
if stream {
item.phase = ItemPhaseTransferring
} else {
item.phase = ItemPhaseDownloading
}
item.failureStage = FailureStageNone
item.err = ""
item.downloadMeter.record(now, item.downloaded)
if stream {
item.uploadMeter.record(now, item.uploaded)
}
})
}
func (t *Task) recordItemDownload(id string, n int64, now time.Time) {
if n <= 0 {
return
}
t.updateItem(id, func(item *itemProgressState) {
item.downloaded += n
item.downloadMeter.record(now, item.downloaded)
if item.phase == ItemPhaseTransferring {
item.uploaded += n
item.uploadMeter.record(now, item.uploaded)
}
})
}
func (t *Task) markItemDownloaded(id string) {
t.updateItem(id, func(item *itemProgressState) {
item.phase = ItemPhaseDownloaded
})
}
func (t *Task) recordItemDownloaded(id string, actualSize int64) {
t.updateItem(id, func(item *itemProgressState) {
if actualSize > 0 {
item.actualSize = actualSize
}
if item.actualSize == 0 {
item.actualSize = item.downloaded
}
if item.phase != ItemPhaseTransferring {
item.phase = ItemPhaseDownloaded
item.uploadMeter.reset()
}
})
}
func (t *Task) recordItemUpload(id string, uploaded, total int64, now time.Time) bool {
becameConfirming := false
t.updateItem(id, func(item *itemProgressState) {
if uploaded < item.uploaded {
if item.phase != ItemPhaseRetrying {
return
}
item.uploadMeter.reset()
}
if total > 0 {
item.actualSize = total
}
item.uploaded = uploaded
item.uploadMeter.record(now, uploaded)
if total > 0 && uploaded >= total {
becameConfirming = item.phase != ItemPhaseConfirming
item.phase = ItemPhaseConfirming
return
}
item.phase = ItemPhaseUploading
item.failureStage = FailureStageNone
item.err = ""
})
return becameConfirming
}
func (t *Task) markItemRetry(id string, stage FailureStage, attempt, limit int, err error) {
t.updateItem(id, func(item *itemProgressState) {
item.phase = ItemPhaseRetrying
item.failureStage = stage
item.retryAttempt = attempt
item.retryLimit = limit
item.err = compactError(err)
})
}
func (t *Task) markItemFailed(id string, stage FailureStage, err error) {
t.updateItem(id, func(item *itemProgressState) {
if item.phase == ItemPhaseFailed || item.phase == ItemPhaseCompleted {
return
}
if errors.Is(err, context.Canceled) {
item.phase = ItemPhaseStopped
return
}
item.phase = ItemPhaseFailed
item.failureStage = stage
item.err = compactError(err)
})
}
func (t *Task) markItemCompleted(id string) {
t.updateItem(id, func(item *itemProgressState) {
item.phase = ItemPhaseCompleted
item.failureStage = FailureStageNone
item.err = ""
item.retryAttempt = 0
item.retryLimit = 0
if item.actualSize == 0 {
item.actualSize = max(item.downloaded, item.uploaded)
}
})
}
func (t *Task) finishItems(err error) {
if err == nil {
return
}
t.itemMu.Lock()
defer t.itemMu.Unlock()
for i := range t.itemStates {
item := &t.itemStates[i]
if item.phase == ItemPhaseCompleted || item.phase == ItemPhaseFailed {
continue
}
item.phase = ItemPhaseStopped
}
}
func (t *Task) itemFailureStage(id string) FailureStage {
t.itemMu.RLock()
defer t.itemMu.RUnlock()
index, ok := t.itemIndex[id]
if !ok || index < 0 || index >= len(t.itemStates) {
return FailureStageUpload
}
if t.itemStates[index].phase == ItemPhaseConfirming {
return FailureStageConfirm
}
return FailureStageUpload
}
func (t *Task) Items() []TaskItemProgress {
t.itemMu.RLock()
defer t.itemMu.RUnlock()
items := make([]TaskItemProgress, 0, len(t.itemStates))
for i := range t.itemStates {
item := &t.itemStates[i]
size := item.actualSize
if size == 0 {
size = item.expectedSize
}
items = append(items, TaskItemProgress{
Index: item.index,
ID: item.id,
Name: item.name,
Size: size,
Downloaded: item.downloaded,
Uploaded: item.uploaded,
DownloadSpeed: item.downloadMeter.speed(),
UploadSpeed: item.uploadMeter.speed(),
Phase: item.phase,
FailureStage: item.failureStage,
RetryAttempt: item.retryAttempt,
RetryLimit: item.retryLimit,
Error: item.err,
})
}
return items
}
func (t *Task) ActualTotalSize() int64 {
items := t.Items()
var total int64
for _, item := range items {
total += item.Size
}
return total
}
type stateProgressTracker interface {
OnStateChange(ctx context.Context, info TaskInfo)
}
func (t *Task) notifyStateChange(ctx context.Context) {
if tracker, ok := t.Progress.(stateProgressTracker); ok {
tracker.OnStateChange(ctx, t)
}
}
func compactError(err error) string {
if err == nil {
return ""
}
return strings.Join(strings.Fields(err.Error()), " ")
}

View File

@@ -0,0 +1,35 @@
package batchtfile
import (
"context"
"sync/atomic"
"testing"
)
type recordingTracker struct {
calls atomic.Int64
}
func (r *recordingTracker) OnStart(context.Context, TaskInfo) {}
func (r *recordingTracker) OnDone(context.Context, TaskInfo, error) {
}
func (r *recordingTracker) OnProgress(context.Context, TaskInfo) {
r.calls.Add(1)
}
// Regression: notifyProgress must call the tracker, not itself. The previous
// self-call recursed until stack overflow on any batch task with a tracker.
func TestNotifyProgressCallsTracker(t *testing.T) {
tracker := &recordingTracker{}
task := &Task{Progress: tracker}
task.notifyProgress(t.Context())
if tracker.calls.Load() != 1 {
t.Fatalf("expected 1 tracker call, got %d", tracker.calls.Load())
}
}
// The nil tracker path must stay a no-op.
func TestNotifyProgressNilTracker(t *testing.T) {
task := &Task{}
task.notifyProgress(t.Context()) // must not panic
}

View File

@@ -4,20 +4,19 @@ import (
"context"
"errors"
"fmt"
"strconv"
"path"
"strings"
"sync/atomic"
"sync"
"time"
"unicode/utf8"
"github.com/charmbracelet/log"
"github.com/duke-git/lancet/v2/slice"
"github.com/gotd/td/telegram/message/entity"
"github.com/gotd/td/telegram/message/styling"
"github.com/gotd/td/tg"
"github.com/krau/SaveAny-Bot/common/i18n"
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
"github.com/krau/SaveAny-Bot/common/utils/dlutil"
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
"github.com/krau/SaveAny-Bot/config"
)
type ProgressTracker interface {
@@ -27,159 +26,474 @@ type ProgressTracker interface {
}
type Progress struct {
MessageID int
ChatID int64
start time.Time
lastUpdatePercent atomic.Int32
skippedFiles []string
MessageID int
ChatID int64
updateMu sync.Mutex
lastUpdateAt time.Time
lastText string
done bool
skippedFiles []string
}
type renderedBatchMessage struct {
Text string
Entities []tg.MessageEntityClass
Err error
}
const (
progressRenderInterval = time.Second
maxVisibleActiveItems = 5
progressBarWidth = 10
maxDisplayNameRunes = 36
maxDisplayErrorRunes = 240
)
func (p *Progress) OnStart(ctx context.Context, info TaskInfo) {
p.start = time.Now()
p.lastUpdatePercent.Store(0)
log.FromContext(ctx).Debugf("Batch task progress tracking started for message %d in chat %d", p.MessageID, p.ChatID)
entityBuilder := entity.Builder{}
var entities []tg.MessageEntityClass
if err := styling.Perform(&entityBuilder,
styling.Plain(i18n.T(i18nk.BotMsgProgressBatchStartPrefix, nil)),
styling.Code(fmt.Sprintf("%.2f MB (%d个文件)", float64(info.TotalSize())/(1024*1024), info.Count())),
); err != nil {
log.FromContext(ctx).Errorf("Failed to build entities: %s", err)
return
}
text, entities := entityBuilder.Complete()
req := &tg.MessagesEditMessageRequest{
ID: p.MessageID,
}
req.SetMessage(text)
req.SetEntities(entities)
req.SetReplyMarkup(&tg.ReplyInlineMarkup{
Rows: []tg.KeyboardButtonRow{
{
Buttons: []tg.KeyboardButtonClass{
tgutil.BuildCancelButton(info.TaskID()),
},
},
}},
)
ext := tgutil.ExtFromContext(ctx)
if ext != nil {
ext.EditMessage(p.ChatID, req)
return
}
p.render(ctx, info, true)
}
func (p *Progress) OnProgress(ctx context.Context, info TaskInfo) {
if !shouldUpdateProgress(info.TotalSize(), info.Downloaded(), int(p.lastUpdatePercent.Load())) {
p.render(ctx, info, false)
}
func (p *Progress) OnStateChange(ctx context.Context, info TaskInfo) {
p.render(ctx, info, true)
}
func (p *Progress) OnUploadStart(ctx context.Context, info TaskInfo, _ int64) {
p.render(ctx, info, true)
}
func (p *Progress) OnUploadProgress(ctx context.Context, info TaskInfo, _, _ int64) {
p.render(ctx, info, false)
}
func (p *Progress) render(ctx context.Context, info TaskInfo, priority bool) {
p.updateMu.Lock()
defer p.updateMu.Unlock()
if p.done {
return
}
percent := int((info.Downloaded() * 100) / info.TotalSize())
if p.lastUpdatePercent.Load() == int32(percent) {
now := time.Now()
if !priority && !p.lastUpdateAt.IsZero() && now.Sub(p.lastUpdateAt) < progressRenderInterval {
return
}
p.lastUpdatePercent.Store(int32(percent))
log.FromContext(ctx).Debugf("Progress update: %s, %d/%d", info.TaskID(), info.Downloaded(), info.TotalSize())
entityBuilder := entity.Builder{}
var entities []tg.MessageEntityClass
if err := styling.Perform(&entityBuilder,
styling.Plain(i18n.T(i18nk.BotMsgProgressBatchProcessingPrefix, nil)),
styling.Code(fmt.Sprintf("%.2f MB (%d个文件)", float64(info.TotalSize())/(1024*1024), info.Count())),
styling.Plain(i18n.T(i18nk.BotMsgProgressProcessingListPrefix, nil)),
func() styling.StyledTextOption {
var lines []string
for _, elem := range info.Processing() {
lines = append(lines, fmt.Sprintf(" - %s (%.2f MB)", elem.FileName(), float64(elem.FileSize())/(1024*1024)))
}
if len(lines) == 0 {
lines = append(lines, i18n.T(i18nk.BotMsgProgressProcessingNone, nil))
}
return styling.Plain(slice.Join(lines, "\n"))
}(),
styling.Plain(i18n.T(i18nk.BotMsgProgressAvgSpeedPrefix, nil)),
styling.Bold(fmt.Sprintf("%.2f MB/s", dlutil.GetSpeed(info.Downloaded(), p.start)/(1024*1024))),
styling.Plain(i18n.T(i18nk.BotMsgProgressCurrentProgressPrefix, nil)),
styling.Bold(fmt.Sprintf("%.2f%%", float64(info.Downloaded())/float64(info.TotalSize())*100)),
); err != nil {
log.FromContext(ctx).Errorf("Failed to build entities: %s", err)
message := buildBatchProgressMessage(info, p.skippedFiles, visibleActiveItems())
if message.Err != nil {
log.FromContext(ctx).Errorf("Failed to render batch progress message: %v", message.Err)
return
}
text, entities := entityBuilder.Complete()
req := &tg.MessagesEditMessageRequest{
ID: p.MessageID,
}
req.SetMessage(text)
req.SetEntities(entities)
req.SetReplyMarkup(&tg.ReplyInlineMarkup{
Rows: []tg.KeyboardButtonRow{
{
Buttons: []tg.KeyboardButtonClass{
tgutil.BuildCancelButton(info.TaskID()),
},
},
}},
)
ext := tgutil.ExtFromContext(ctx)
if ext != nil {
ext.EditMessage(p.ChatID, req)
if message.Text == p.lastText {
return
}
p.lastText = message.Text
p.lastUpdateAt = now
p.editMessage(ctx, info.TaskID(), message, true)
}
func (p *Progress) OnDone(ctx context.Context, info TaskInfo, err error) {
if err != nil {
log.FromContext(ctx).Errorf("Batch task %s failed: %s", info.TaskID(), err)
} else {
log.FromContext(ctx).Debugf("Batch task %s completed successfully", info.TaskID())
}
entityBuilder := entity.Builder{}
var stylingErr error
if err != nil {
if errors.Is(err, context.Canceled) {
stylingErr = styling.Perform(&entityBuilder,
styling.Plain(i18n.T(i18nk.BotMsgProgressTaskCanceled, nil)),
)
} else {
stylingErr = styling.Perform(&entityBuilder,
styling.Plain(i18n.T(i18nk.BotMsgProgressTaskFailedWithError, map[string]any{
"Error": "",
})),
styling.Code(err.Error()),
)
}
} else {
stylingErr = styling.Perform(&entityBuilder,
styling.Plain(i18n.T(i18nk.BotMsgProgressBatchDonePrefix, nil)),
styling.Code(strconv.Itoa(info.Count())),
styling.Plain(i18n.T(i18nk.BotMsgProgressTotalSizePrefix, nil)),
styling.Code(fmt.Sprintf("%.2f MB", float64(info.TotalSize())/(1024*1024))),
func() styling.StyledTextOption {
if len(p.skippedFiles) == 0 {
return styling.Plain("")
}
return styling.Plain("\n\n" + i18n.T(i18nk.BotMsgCommonInfoConflictFilesSkipped, map[string]any{
"Skipped": strings.Join(p.skippedFiles, "\n"),
}))
}(),
)
}
if stylingErr != nil {
log.FromContext(ctx).Errorf("Failed to build entities: %s", stylingErr)
p.updateMu.Lock()
defer p.updateMu.Unlock()
if p.done {
return
}
text, entities := entityBuilder.Complete()
req := &tg.MessagesEditMessageRequest{
ID: p.MessageID,
p.done = true
message := buildBatchDoneMessage(info, p.skippedFiles, err)
if message.Err != nil {
log.FromContext(ctx).Errorf("Failed to render final batch progress message: %v", message.Err)
return
}
req.SetMessage(text)
req.SetEntities(entities)
p.lastText = message.Text
p.editMessage(ctx, info.TaskID(), message, false)
}
ext := tgutil.ExtFromContext(ctx)
if ext != nil {
ext.EditMessage(p.ChatID, req)
func (p *Progress) editMessage(ctx context.Context, taskID string, message renderedBatchMessage, cancellable bool) {
if message.Err != nil {
log.FromContext(ctx).Errorf("Failed to render batch progress message: %v", message.Err)
return
}
req := buildBatchEditMessageRequest(p.MessageID, taskID, message, cancellable)
if ext := tgutil.ExtFromContext(ctx); ext != nil {
if _, err := ext.EditMessage(p.ChatID, req); err != nil {
log.FromContext(ctx).Errorf("Failed to edit batch progress message: %v", err)
}
}
}
func buildBatchEditMessageRequest(messageID int, taskID string, message renderedBatchMessage, cancellable bool) *tg.MessagesEditMessageRequest {
req := &tg.MessagesEditMessageRequest{ID: messageID}
req.SetMessage(message.Text)
if len(message.Entities) > 0 {
req.SetEntities(message.Entities)
}
if cancellable {
req.SetReplyMarkup(&tg.ReplyInlineMarkup{Rows: []tg.KeyboardButtonRow{{
Buttons: []tg.KeyboardButtonClass{tgutil.BuildCancelButton(taskID)},
}}})
}
return req
}
func buildBatchProgressText(info TaskInfo, skipped []string, activeLimit int) string {
return buildBatchProgressMessage(info, skipped, activeLimit).Text
}
func buildBatchProgressMessage(info TaskInfo, skipped []string, activeLimit int) renderedBatchMessage {
items := info.Items()
completed, waiting, downloaded, failed := itemCounts(items)
downloadSpeed, uploadSpeed := aggregateSpeeds(items)
if activeLimit < 1 {
activeLimit = 1
}
total := len(items) + len(skipped)
downloadSpeedText := formatSpeed(downloadSpeed)
uploadSpeedText := formatSpeed(uploadSpeed)
header := localizedProgressMarkup(i18nk.BotMsgProgressBatchStatusHeader, map[string]any{
"Total": total,
"Completed": completed,
"Downloaded": downloaded,
"Waiting": waiting,
"DownloadSpeed": downloadSpeedText,
"UploadSpeed": uploadSpeedText,
})
var markup strings.Builder
markup.WriteString(header)
visibleItems, hiddenTransfers, summarizedConfirming := visibleBatchItems(items, activeLimit)
for _, item := range visibleItems {
markup.WriteString("\n\n")
markup.WriteString(formatActiveItemMarkup(item, len(items)))
}
if hiddenTransfers > 0 {
markup.WriteString("\n\n")
markup.WriteString(localizedProgressMarkup(i18nk.BotMsgProgressBatchSummaryHiddenActive, map[string]any{"Count": hiddenTransfers}))
}
if summarizedConfirming > 0 {
markup.WriteString("\n\n")
markup.WriteString(localizedProgressMarkup(i18nk.BotMsgProgressBatchSummaryConfirming, map[string]any{"Count": summarizedConfirming}))
}
if failed > 0 {
markup.WriteString("\n")
markup.WriteString(localizedProgressMarkup(i18nk.BotMsgProgressBatchSummaryFailed, map[string]any{"Count": failed}))
}
if len(skipped) > 0 {
markup.WriteString("\n")
markup.WriteString(localizedProgressMarkup(i18nk.BotMsgProgressBatchSummarySkipped, map[string]any{"Count": len(skipped)}))
}
return completeBatchMessage(markup.String())
}
func buildBatchDoneMarkup(info TaskInfo, skipped []string, err error) string {
items := info.Items()
totalSize := info.ActualTotalSize()
if totalSize == 0 {
totalSize = info.TotalSize()
}
if err == nil {
completed, _, _, failed := itemCounts(items)
// Report per-element failures instead of full completion.
totalSkipped := len(skipped) + failed
if totalSkipped > 0 {
return localizedProgressMarkup(i18nk.BotMsgProgressBatchDoneWithSkipped, map[string]any{
"Success": completed,
"Skipped": totalSkipped,
"Size": dlutil.FormatSize(totalSize),
})
}
return localizedProgressMarkup(i18nk.BotMsgProgressBatchDone, map[string]any{
"Count": len(items),
"Size": dlutil.FormatSize(totalSize),
})
}
completed, _, _, failed := itemCounts(items)
incomplete := max(len(items)-completed-failed, 0)
if errors.Is(err, context.Canceled) {
return localizedProgressMarkup(i18nk.BotMsgProgressBatchCanceled, map[string]any{
"Total": len(items) + len(skipped),
"Completed": completed,
"Incomplete": incomplete,
"Skipped": len(skipped),
})
}
failedItems := make([]TaskItemProgress, 0, failed)
for _, item := range items {
if item.Phase == ItemPhaseFailed {
failedItems = append(failedItems, item)
}
}
if len(failedItems) > 1 && failedItems[0].FailureStage == FailureStageBatchUpload {
return localizedProgressMarkup(i18nk.BotMsgProgressBatchFailedGroup, map[string]any{
"Affected": len(failedItems),
"Reason": displayError(firstError(failedItems), err),
"Completed": completed,
"Failed": len(failedItems),
"Incomplete": incomplete,
})
}
if len(failedItems) == 0 {
return localizedProgressMarkup(i18nk.BotMsgProgressBatchFailedTask, map[string]any{
"Reason": displayError("", err),
"Completed": completed,
"Incomplete": incomplete,
})
}
item := failedItems[0]
return localizedProgressMarkup(i18nk.BotMsgProgressBatchFailedItem, map[string]any{
"Index": item.Index,
"Name": truncateFilename(item.Name, maxDisplayNameRunes),
"Stage": failureStageLabel(item.FailureStage),
"Progress": failureProgress(item),
"Speed": failureSpeed(item),
"Reason": displayError(item.Error, err),
"Completed": completed,
"Failed": failed,
"Incomplete": incomplete,
})
}
func buildBatchDoneMessage(info TaskInfo, skipped []string, err error) renderedBatchMessage {
return completeBatchMessage(buildBatchDoneMarkup(info, skipped, err))
}
func formatActiveItemMarkup(item TaskItemProgress, total int) string {
data := map[string]any{
"Index": item.Index,
"Total": total,
"Name": truncateFilename(item.Name, maxDisplayNameRunes),
"Speed": formatSpeed(itemSpeed(item)),
"Progress": itemPercent(item),
"Bar": textProgressBar(itemPercent(item)),
"Current": dlutil.FormatSize(itemBytes(item)),
"Size": dlutil.FormatSize(item.Size),
"Attempt": min(max(item.RetryAttempt, 1), max(item.RetryLimit, 1)),
"Limit": max(item.RetryLimit, 1),
"Reason": truncateRunes(item.Error, maxDisplayErrorRunes),
}
switch item.Phase {
case ItemPhaseDownloading:
if item.Size <= 0 {
return localizedProgressMarkup(i18nk.BotMsgProgressBatchItemDownloadingUnknown, data)
}
return localizedProgressMarkup(i18nk.BotMsgProgressBatchItemDownloading, data)
case ItemPhaseTransferring:
if item.Size <= 0 {
return localizedProgressMarkup(i18nk.BotMsgProgressBatchItemTransferringUnknown, data)
}
return localizedProgressMarkup(i18nk.BotMsgProgressBatchItemTransferring, data)
case ItemPhaseUploading:
return localizedProgressMarkup(i18nk.BotMsgProgressBatchItemUploading, data)
case ItemPhaseRetrying:
return localizedProgressMarkup(i18nk.BotMsgProgressBatchItemRetrying, data)
case ItemPhaseConfirming:
return localizedProgressMarkup(i18nk.BotMsgProgressBatchItemConfirming, data)
default:
return ""
}
}
func localizedProgressMarkup(key i18nk.Key, data map[string]any) string {
return i18n.T(key, tgutil.EscapeHTMLTemplateData(data))
}
func visibleBatchItems(items []TaskItemProgress, limit int) (visible []TaskItemProgress, hiddenTransfers, summarizedConfirming int) {
visible = make([]TaskItemProgress, 0, limit)
transferCount := 0
confirmingCount := 0
for _, item := range items {
switch {
case isTransferPhase(item.Phase):
transferCount++
if len(visible) < limit {
visible = append(visible, item)
}
case item.Phase == ItemPhaseConfirming:
confirmingCount++
}
}
hiddenTransfers = transferCount - len(visible)
if confirmingCount == 1 && len(visible) < limit {
for _, item := range items {
if item.Phase == ItemPhaseConfirming {
visible = append(visible, item)
return visible, hiddenTransfers, 0
}
}
}
return visible, hiddenTransfers, confirmingCount
}
func completeBatchMessage(markup string) renderedBatchMessage {
text, entities, err := tgutil.RenderHTML(markup)
return renderedBatchMessage{Text: text, Entities: entities, Err: err}
}
func itemCounts(items []TaskItemProgress) (completed, waiting, downloaded, failed int) {
for _, item := range items {
switch item.Phase {
case ItemPhaseCompleted:
completed++
case ItemPhaseWaiting:
waiting++
case ItemPhaseDownloaded:
downloaded++
case ItemPhaseFailed:
failed++
}
}
return
}
func aggregateSpeeds(items []TaskItemProgress) (download, upload float64) {
for _, item := range items {
switch item.Phase {
case ItemPhaseDownloading:
download += item.DownloadSpeed
case ItemPhaseTransferring:
download += item.DownloadSpeed
upload += item.UploadSpeed
case ItemPhaseUploading:
upload += item.UploadSpeed
}
}
return
}
func isTransferPhase(phase ItemPhase) bool {
switch phase {
case ItemPhaseDownloading, ItemPhaseTransferring, ItemPhaseUploading, ItemPhaseRetrying:
return true
default:
return false
}
}
func itemBytes(item TaskItemProgress) int64 {
switch item.Phase {
case ItemPhaseDownloading, ItemPhaseTransferring:
return item.Downloaded
default:
return item.Uploaded
}
}
func itemSpeed(item TaskItemProgress) float64 {
switch item.Phase {
case ItemPhaseDownloading, ItemPhaseTransferring:
return item.DownloadSpeed
case ItemPhaseUploading:
return item.UploadSpeed
case ItemPhaseRetrying:
return item.UploadSpeed
default:
return 0
}
}
func itemPercent(item TaskItemProgress) int {
if item.Size <= 0 {
return 0
}
return int(min(itemBytes(item), item.Size) * 100 / item.Size)
}
func textProgressBar(percent int) string {
percent = min(max(percent, 0), 100)
filled := percent * progressBarWidth / 100
return strings.Repeat("🟩", filled) + strings.Repeat("⬜️", progressBarWidth-filled)
}
func formatSpeed(speed float64) string {
if speed <= 0 {
return "0 B/s"
}
return dlutil.FormatSize(int64(speed)) + "/s"
}
func truncateFilename(name string, limit int) string {
if utf8.RuneCountInString(name) <= limit {
return name
}
ext := path.Ext(name)
if utf8.RuneCountInString(ext) >= limit-2 {
return truncateRunes(name, limit-1) + "…"
}
base := strings.TrimSuffix(name, ext)
baseLimit := limit - utf8.RuneCountInString(ext) - 1
return truncateRunes(base, baseLimit) + "…" + ext
}
func truncateRunes(value string, limit int) string {
if limit <= 0 {
return ""
}
runes := []rune(value)
if len(runes) <= limit {
return value
}
return string(runes[:limit])
}
func displayError(itemError string, fallback error) string {
if itemError == "" && fallback != nil {
itemError = compactError(fallback)
}
return truncateRunes(itemError, maxDisplayErrorRunes)
}
func firstError(items []TaskItemProgress) string {
for _, item := range items {
if item.Error != "" {
return item.Error
}
}
return ""
}
func failureStageLabel(stage FailureStage) string {
switch stage {
case FailureStageDownload:
return i18n.T(i18nk.BotMsgProgressBatchFailureStageDownload, nil)
case FailureStageCache:
return i18n.T(i18nk.BotMsgProgressBatchFailureStageCache, nil)
case FailureStageUpload:
return i18n.T(i18nk.BotMsgProgressBatchFailureStageUpload, nil)
case FailureStageConfirm:
return i18n.T(i18nk.BotMsgProgressBatchFailureStageConfirm, nil)
case FailureStageBatchUpload:
return i18n.T(i18nk.BotMsgProgressBatchFailureStageBatchUpload, nil)
default:
return i18n.T(i18nk.BotMsgProgressBatchFailureStageInternal, nil)
}
}
func failureProgress(item TaskItemProgress) string {
if item.Size <= 0 {
return dlutil.FormatSize(failureBytes(item))
}
return fmt.Sprintf("%d%%", min(failureBytes(item), item.Size)*100/item.Size)
}
func failureSpeed(item TaskItemProgress) string {
if item.FailureStage == FailureStageDownload || item.FailureStage == FailureStageCache {
return formatSpeed(item.DownloadSpeed)
}
return formatSpeed(item.UploadSpeed)
}
func failureBytes(item TaskItemProgress) int64 {
if item.FailureStage == FailureStageDownload || item.FailureStage == FailureStageCache {
return item.Downloaded
}
return item.Uploaded
}
func visibleActiveItems() int {
return min(max(config.C().Workers, 1), maxVisibleActiveItems)
}
func NewProgressTracker(messageID int, chatID int64) ProgressTracker {

View File

@@ -0,0 +1,311 @@
package batchtfile
import (
"context"
"errors"
"strings"
"sync"
"testing"
"time"
"github.com/gotd/td/tg"
"github.com/krau/SaveAny-Bot/common/i18n"
"github.com/krau/SaveAny-Bot/pkg/tfile"
)
type progressRegressionRecorder struct {
mu sync.Mutex
startTotal int64
notifications []int64
}
func (*progressRegressionRecorder) OnStart(context.Context, TaskInfo) {}
func (*progressRegressionRecorder) OnProgress(context.Context, TaskInfo) {}
func (*progressRegressionRecorder) OnDone(context.Context, TaskInfo, error) {}
func (r *progressRegressionRecorder) OnUploadStart(_ context.Context, _ TaskInfo, total int64) {
r.mu.Lock()
defer r.mu.Unlock()
r.startTotal = total
}
func (r *progressRegressionRecorder) OnUploadProgress(_ context.Context, _ TaskInfo, uploaded, _ int64) {
r.mu.Lock()
defer r.mu.Unlock()
r.notifications = append(r.notifications, uploaded)
}
type orderedProgressRegressionRecorder struct {
firstEntered chan struct{}
releaseFirst chan struct{}
secondEntered chan struct{}
mu sync.Mutex
notifications []int64
}
func (*orderedProgressRegressionRecorder) OnStart(context.Context, TaskInfo) {}
func (*orderedProgressRegressionRecorder) OnProgress(context.Context, TaskInfo) {}
func (*orderedProgressRegressionRecorder) OnDone(context.Context, TaskInfo, error) {}
func (*orderedProgressRegressionRecorder) OnUploadStart(context.Context, TaskInfo, int64) {
}
func (r *orderedProgressRegressionRecorder) OnUploadProgress(_ context.Context, _ TaskInfo, uploaded, _ int64) {
if uploaded == 100 {
close(r.firstEntered)
<-r.releaseFirst
}
if uploaded == 200 {
close(r.secondEntered)
}
r.mu.Lock()
r.notifications = append(r.notifications, uploaded)
r.mu.Unlock()
}
func TestBatchProgressShowsTransferSpeedAndSize(t *testing.T) {
useProgressRegressionLocale(t)
task := newProgressRegressionTask(nil,
progressRegressionFile{"downloading", 1000},
progressRegressionFile{"uploading", 1000},
progressRegressionFile{"waiting", 1000},
)
started := time.Unix(100, 0)
task.markItemActive("downloading", false, started)
task.recordItemDownload("downloading", 500, started.Add(time.Second))
task.recordItemDownloaded("uploading", 1000)
task.recordItemUpload("uploading", 0, 1000, started.Add(time.Second))
task.recordItemUpload("uploading", 250, 1000, started.Add(2*time.Second))
message := buildBatchProgressMessage(task, nil, 2)
if message.Err != nil {
t.Fatalf("buildBatchProgressMessage() failed: %v", message.Err)
}
assertProgressRegressionContains(t, message.Text,
"状态:✅ 0 📥 0 ⏳ 1",
"⬇️ 1/3 下载中",
"速度500 B/s",
"大小500 B / 1000 B",
"⬆️ 2/3 上传中",
"速度250 B/s",
"大小250 B / 1000 B",
)
bold, _, blockquote, _ := batchEntityCounts(message.Entities)
if bold != 3 || blockquote != 2 {
t.Fatalf("entity counts = bold:%d blockquote:%d, want bold:3 blockquote:2", bold, blockquote)
}
}
func TestBatchProgressLimitsRowsWithoutHidingActiveUpload(t *testing.T) {
useProgressRegressionLocale(t)
task := newProgressRegressionTask(nil,
progressRegressionFile{"confirm-01", 100},
progressRegressionFile{"confirm-02", 100},
progressRegressionFile{"uploading", 100},
progressRegressionFile{"downloading", 100},
)
started := time.Unix(100, 0)
task.recordItemUpload("confirm-01", 100, 100, started)
task.recordItemUpload("confirm-02", 100, 100, started)
task.recordItemUpload("uploading", 40, 100, started.Add(time.Second))
task.markItemActive("downloading", false, started)
message := buildBatchProgressText(task, nil, 2)
assertProgressRegressionContains(t, message,
"uploading.bin",
"downloading.bin",
"☁️ 已上传等待整组发送2",
)
if strings.Contains(message, "confirm-01.bin") || strings.Contains(message, "confirm-02.bin") {
t.Fatalf("confirmation rows displaced active transfers:\n%s", message)
}
}
func TestBatchProgressTemplateOwnsStylesAndEscapesValues(t *testing.T) {
useProgressRegressionLocale(t)
fileID := `<b>A&B</b>`
task := newProgressRegressionTask(nil, progressRegressionFile{fileID, 100})
task.markItemRetry(fileID, FailureStageUpload, 1, 3, errors.New(`<i>remote & failed</i>`))
message := buildBatchProgressMessage(task, nil, 1)
if message.Err != nil {
t.Fatalf("buildBatchProgressMessage() failed: %v", message.Err)
}
assertProgressRegressionContains(t, message.Text,
`<b>A&B</b>.bin`,
`<i>remote & failed</i>`,
)
bold, _, blockquote, italic := batchEntityCounts(message.Entities)
if bold != 2 || blockquote != 1 || italic != 0 {
t.Fatalf("entity counts = bold:%d blockquote:%d italic:%d", bold, blockquote, italic)
}
i18n.Init("en")
english := buildBatchProgressMessage(task, nil, 1)
if english.Err != nil {
t.Fatalf("English batch template failed: %v", english.Err)
}
assertProgressRegressionContains(t, english.Text, "📦 Processing", "Retrying upload", `<b>A&B</b>.bin`)
}
func TestDownloadProgressContinuesAfterUploadStarts(t *testing.T) {
useProgressRegressionLocale(t)
progress := new(Progress)
task := newProgressRegressionTask(progress,
progressRegressionFile{"uploading", 100},
progressRegressionFile{"downloading", 100},
)
progress.OnStart(t.Context(), task)
task.recordDownloadComplete("uploading", 100)
task.uploadCallback(t.Context(), "uploading")(50, 100)
started := time.Unix(100, 0)
task.markItemActive("downloading", false, started)
task.recordItemDownload("downloading", 50, started.Add(time.Second))
progress.updateMu.Lock()
progress.lastUpdateAt = time.Now().Add(-progressRenderInterval)
progress.updateMu.Unlock()
progress.OnProgress(t.Context(), task)
progress.updateMu.Lock()
text := progress.lastText
progress.updateMu.Unlock()
assertProgressRegressionContains(t, text,
"uploading.bin",
"🟩🟩🟩🟩🟩⬜️⬜️⬜️⬜️⬜️ 50%",
"总速度:⬇️ 50 B/s ⬆️ 0 B/s",
"🔄 另有 1 个文件正在处理",
)
}
func TestBatchUploadIgnoresOutOfOrderBytesAndAllowsRetryReset(t *testing.T) {
recorder := new(progressRegressionRecorder)
task := newProgressRegressionTask(recorder, progressRegressionFile{"file", 100})
task.recordDownloadComplete("file", 100)
callback := task.uploadCallback(t.Context(), "file")
callback(80, 100)
callback(10, 100)
if got := task.Items()[0].Uploaded; got != 80 {
t.Fatalf("out-of-order callback regressed item to %d, want 80", got)
}
task.markItemRetry("file", FailureStageUpload, 1, 3, context.DeadlineExceeded)
callback(0, 100)
callback(10, 100)
if got := task.Items()[0].Uploaded; got != 10 {
t.Fatalf("retry did not reset item progress: got %d, want 10", got)
}
recorder.mu.Lock()
defer recorder.mu.Unlock()
for index := 1; index < len(recorder.notifications); index++ {
if recorder.notifications[index] < recorder.notifications[index-1] {
t.Fatalf("aggregate progress regressed: %v", recorder.notifications)
}
}
}
func TestUploadProgressNotificationsRemainOrdered(t *testing.T) {
recorder := &orderedProgressRegressionRecorder{
firstEntered: make(chan struct{}),
releaseFirst: make(chan struct{}),
secondEntered: make(chan struct{}),
}
task := newProgressRegressionTask(recorder,
progressRegressionFile{"first", 100},
progressRegressionFile{"second", 100},
)
task.recordDownloadComplete("first", 100)
task.recordDownloadComplete("second", 100)
first := task.uploadCallback(t.Context(), "first")
second := task.uploadCallback(t.Context(), "second")
var wait sync.WaitGroup
wait.Go(func() {
first(100, 100)
})
<-recorder.firstEntered
wait.Go(func() {
second(100, 100)
})
overtook := false
select {
case <-recorder.secondEntered:
overtook = true
case <-time.After(100 * time.Millisecond):
}
close(recorder.releaseFirst)
wait.Wait()
if overtook {
t.Fatal("later aggregate notification overtook the first callback")
}
recorder.mu.Lock()
defer recorder.mu.Unlock()
if got := recorder.notifications; len(got) != 2 || got[0] != 100 || got[1] != 200 {
t.Fatalf("upload notifications = %v, want [100 200]", got)
}
}
func TestBatchUploadUsesActualSizeWhenMetadataIsUnknown(t *testing.T) {
recorder := new(progressRegressionRecorder)
task := newProgressRegressionTask(recorder, progressRegressionFile{"photo", 0})
task.recordDownloadComplete("photo", 25)
task.uploadCallback(t.Context(), "photo")(25, 25)
recorder.mu.Lock()
defer recorder.mu.Unlock()
if recorder.startTotal != 25 {
t.Fatalf("upload start total = %d, want actual size 25", recorder.startTotal)
}
if got := task.ActualTotalSize(); got != 25 {
t.Fatalf("actual total size = %d, want 25", got)
}
}
type progressRegressionFile struct {
id string
size int64
}
func newProgressRegressionTask(progress ProgressTracker, files ...progressRegressionFile) *Task {
elems := make([]TaskElement, 0, len(files))
for _, file := range files {
elems = append(elems, TaskElement{
ID: file.id,
File: tfile.NewTGFile(nil, nil, file.size, file.id+".bin"),
})
}
return NewBatchTGFileTask("progress-regression", context.Background(), elems, progress, true)
}
func useProgressRegressionLocale(t *testing.T) {
t.Helper()
i18n.Init("zh-Hans")
t.Cleanup(func() { i18n.Init("zh-Hans") })
}
func assertProgressRegressionContains(t *testing.T, value string, wants ...string) {
t.Helper()
for _, want := range wants {
if !strings.Contains(value, want) {
t.Fatalf("text does not contain %q:\n%s", want, value)
}
}
}
func batchEntityCounts(entities []tg.MessageEntityClass) (bold, code, blockquote, italic int) {
for _, messageEntity := range entities {
switch messageEntity.(type) {
case *tg.MessageEntityBold:
bold++
case *tg.MessageEntityCode:
code++
case *tg.MessageEntityBlockquote:
blockquote++
case *tg.MessageEntityItalic:
italic++
}
}
return
}

View File

@@ -7,6 +7,7 @@ import (
"sync"
"sync/atomic"
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
"github.com/krau/SaveAny-Bot/config"
"github.com/krau/SaveAny-Bot/core"
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
@@ -18,25 +19,34 @@ import (
var _ core.Executable = (*Task)(nil)
type TaskElement struct {
ID string
Storage storage.Storage
Path string
File tfile.TGFile
localPath string
stream bool
ID string
Storage storage.Storage
Path string
File tfile.TGFile
localPath string
stream bool
sourceGroupKey string
sourceCaption string
preserveCaption bool
}
type Task struct {
ID string
ctx context.Context
elems []TaskElement
Progress ProgressTracker
IgnoreErrors bool // if true, errors during processing will be ignored
downloaded atomic.Int64
totalSize int64
processing map[string]TaskElementInfo
processingMu sync.RWMutex
failed map[string]error // [TODO] errors for each element
ID string
ctx context.Context
elems []TaskElement
Progress ProgressTracker
IgnoreErrors bool // if true, errors during processing will be ignored
downloaded atomic.Int64
totalSize int64
uploadTotalSize atomic.Int64
processing map[string]TaskElementInfo
processingMu sync.RWMutex
itemStates []itemProgressState
itemIndex map[string]int
itemMu sync.RWMutex
uploadOnce sync.Once
uploadMu sync.Mutex
uploaded map[string]int64
}
// Title implements core.Exectable.
@@ -54,6 +64,7 @@ func NewTaskElement(
file tfile.TGFile,
) (*TaskElement, error) {
id := xid.New().String()
groupKey, caption, preserveCaption := sourceMetadata(file)
_, ok := stor.(storage.StorageCannotStream)
if !config.C().Stream || ok {
cachePath, err := filepath.Abs(filepath.Join(config.C().Temp.BasePath, fmt.Sprintf("%s_%s", id, file.Name())))
@@ -61,22 +72,42 @@ func NewTaskElement(
return nil, fmt.Errorf("failed to get absolute path for cache: %w", err)
}
return &TaskElement{
ID: id,
Storage: stor,
Path: path,
File: file,
localPath: cachePath,
ID: id,
Storage: stor,
Path: path,
File: file,
localPath: cachePath,
sourceGroupKey: groupKey,
sourceCaption: caption,
preserveCaption: preserveCaption,
}, nil
}
return &TaskElement{
ID: id,
Storage: stor,
Path: path,
File: file,
stream: true,
ID: id,
Storage: stor,
Path: path,
File: file,
stream: true,
sourceGroupKey: groupKey,
sourceCaption: caption,
preserveCaption: preserveCaption,
}, nil
}
func sourceMetadata(file tfile.TGFile) (groupKey, caption string, preserveCaption bool) {
messageFile, ok := file.(tfile.TGFileMessage)
if !ok || messageFile.Message() == nil {
return "", "", false
}
msg := messageFile.Message()
groupID, grouped := msg.GetGroupedID()
if !grouped || groupID == 0 {
return "", "", false
}
chatID := tgutil.ChatIdFromPeer(msg.GetPeerID())
return fmt.Sprintf("%T:%d:%d", msg.GetPeerID(), chatID, groupID), msg.GetMessage(), true
}
func NewBatchTGFileTask(
id string,
ctx context.Context,
@@ -84,6 +115,7 @@ func NewBatchTGFileTask(
progress ProgressTracker,
ignoreErrors bool,
) *Task {
itemStates, itemIndex := newItemProgressStates(files)
task := &Task{
ID: id,
ctx: ctx,
@@ -98,9 +130,11 @@ func NewBatchTGFileTask(
return total
}(),
processing: make(map[string]TaskElementInfo),
itemStates: itemStates,
itemIndex: itemIndex,
uploaded: make(map[string]int64),
IgnoreErrors: ignoreErrors,
processingMu: sync.RWMutex{},
failed: make(map[string]error),
}
return task
}

View File

@@ -27,8 +27,10 @@ type TaskInfo interface {
TaskID() string
TotalSize() int64
Downloaded() int64
ActualTotalSize() int64
Count() int
Processing() []TaskElementInfo
Items() []TaskItemProgress
}
func (t *Task) TaskID() string {

View File

@@ -0,0 +1,70 @@
package batchtfile
import (
"context"
"time"
)
// UploadProgressTracker optionally extends a batch progress tracker with a
// distinct aggregate upload phase.
type UploadProgressTracker interface {
OnUploadStart(ctx context.Context, info TaskInfo, total int64)
OnUploadProgress(ctx context.Context, info TaskInfo, uploaded, total int64)
}
func (t *Task) startUpload(ctx context.Context) {
tracker, ok := t.Progress.(UploadProgressTracker)
if !ok {
return
}
t.uploadMu.Lock()
defer t.uploadMu.Unlock()
t.uploadOnce.Do(func() {
tracker.OnUploadStart(ctx, t, t.uploadTotalSize.Load())
})
}
func (t *Task) uploadCallback(ctx context.Context, id string) func(uploaded, total int64) {
return func(uploaded, total int64) {
tracker, ok := t.Progress.(UploadProgressTracker)
if !ok || uploaded < 0 {
return
}
t.startUpload(ctx)
if total > 0 && uploaded > total {
uploaded = total
}
t.uploadMu.Lock()
defer t.uploadMu.Unlock()
becameConfirming := t.recordItemUpload(id, uploaded, total, time.Now())
if t.uploaded == nil {
t.uploaded = make(map[string]int64)
}
previous, tracked := t.uploaded[id]
if !tracked || uploaded > previous {
t.uploaded[id] = uploaded
}
var aggregate int64
for _, current := range t.uploaded {
aggregate += current
}
uploadTotal := t.uploadTotalSize.Load()
if aggregate > uploadTotal {
aggregate = uploadTotal
}
tracker.OnUploadProgress(ctx, t, aggregate, uploadTotal)
if becameConfirming {
t.notifyStateChange(ctx)
}
}
}
func (t *Task) recordDownloadComplete(id string, uploadSize int64) {
t.uploadMu.Lock()
defer t.uploadMu.Unlock()
if uploadSize > 0 {
t.uploadTotalSize.Add(uploadSize)
}
t.recordItemDownloaded(id, uploadSize)
}

View File

@@ -1,32 +0,0 @@
package batchtfile
var progressUpdatesLevels = []struct {
size int64 // 文件大小阈值
stepPercent int // 每多少 % 更新一次
}{
{10 << 20, 100},
{50 << 20, 20},
{200 << 20, 10},
{500 << 20, 5},
}
func shouldUpdateProgress(total, downloaded int64, lastUpdatePercent int) bool {
if total <= 0 || downloaded <= 0 {
return false
}
percent := int((downloaded * 100) / total)
if percent <= lastUpdatePercent {
return false
}
step := progressUpdatesLevels[len(progressUpdatesLevels)-1].stepPercent
for _, lvl := range progressUpdatesLevels {
if total < lvl.size {
step = lvl.stepPercent
break
}
}
return percent >= lastUpdatePercent+step
}

View File

@@ -15,6 +15,7 @@ import (
"github.com/krau/SaveAny-Bot/common/utils/ioutil"
"github.com/krau/SaveAny-Bot/config"
"github.com/krau/SaveAny-Bot/pkg/enums/ctxkey"
"github.com/krau/SaveAny-Bot/pkg/taskevent"
"golang.org/x/sync/errgroup"
)
@@ -75,12 +76,11 @@ func (t *Task) Execute(ctx context.Context) error {
eg.SetLimit(config.C().Workers)
for _, file := range t.files {
eg.Go(func() error {
t.processingMu.RLock()
t.processingMu.Lock()
if _, ok := t.processing[file.URL]; ok {
t.processingMu.Unlock()
return fmt.Errorf("file %s is already being processed", file.URL)
}
t.processingMu.RUnlock()
t.processingMu.Lock()
t.processing[file.URL] = file
t.processingMu.Unlock()
defer func() {
@@ -89,7 +89,6 @@ func (t *Task) Execute(ctx context.Context) error {
t.processingMu.Unlock()
}()
err := t.processLink(gctx, file)
t.downloaded.Add(1)
if errors.Is(err, context.Canceled) {
logger.Debug("Link processing canceled")
return err
@@ -98,6 +97,7 @@ func (t *Task) Execute(ctx context.Context) error {
logger.Errorf("Error processing link %s: %v", file.URL, err)
return fmt.Errorf("failed to process link %s: %w", file.URL, err)
}
t.downloaded.Add(1)
return nil
})
}
@@ -143,10 +143,16 @@ func (t *Task) processLink(ctx context.Context, file *File) error {
}
}()
wr := ioutil.NewProgressWriter(cacheFile, func(n int) {
t.downloadedBytes.Add(int64(n))
downloaded := t.downloadedBytes.Add(int64(n))
if t.Progress != nil {
t.Progress.OnProgress(ctx, t)
}
taskevent.Emit(ctx, taskevent.Event{
TaskID: t.ID,
Phase: taskevent.PhaseProgress,
TotalBytes: t.totalBytes,
DownloadedBytes: downloaded,
})
})
copyResultCh := make(chan error, 1)

View File

@@ -15,6 +15,7 @@ import (
"github.com/krau/SaveAny-Bot/common/i18n"
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
"github.com/krau/SaveAny-Bot/common/utils/dlutil"
"github.com/krau/SaveAny-Bot/common/utils/progressutil"
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
)
@@ -102,7 +103,7 @@ func (p *Progress) OnDone(ctx context.Context, info TaskInfo, err error) {
// OnProgress implements ProgressTracker.
func (p *Progress) OnProgress(ctx context.Context, info TaskInfo) {
if !shouldUpdateProgress(info.TotalBytes(), info.DownloadedBytes(), int(p.lastUpdatePercent.Load())) {
if !progressutil.ShouldUpdate(info.TotalBytes(), info.DownloadedBytes(), int(p.lastUpdatePercent.Load())) {
return
}
percent := int((info.DownloadedBytes() * 100) / info.TotalBytes())
@@ -115,7 +116,10 @@ func (p *Progress) OnProgress(ctx context.Context, info TaskInfo) {
var entities []tg.MessageEntityClass
if err := styling.Perform(&entityBuilder,
styling.Plain(i18n.T(i18nk.BotMsgProgressDownloadingPrefix, nil)),
styling.Code(fmt.Sprintf("%.2f MB (%d个文件)", float64(info.TotalBytes())/(1024*1024), info.TotalFiles())),
styling.Code(i18n.T(i18nk.BotMsgProgressSizeWithFiles, map[string]any{
"Size": fmt.Sprintf("%.2f MB", float64(info.TotalBytes())/(1024*1024)),
"Count": info.TotalFiles(),
})),
styling.Plain(i18n.T(i18nk.BotMsgProgressProcessingListPrefix, nil)),
func() styling.StyledTextOption {
var lines []string

View File

@@ -45,7 +45,6 @@ type Task struct {
downloaded atomic.Int64 // downloaded files count
processing map[string]*File // {"url": File}
processingMu sync.RWMutex
failed map[string]error // [TODO] errors for each file
}
// Title implements core.Exectable.
@@ -127,7 +126,6 @@ func NewTask(
client: http.DefaultClient,
processing: make(map[string]*File),
processingMu: sync.RWMutex{},
failed: make(map[string]error),
totalFiles: int64(len(files)),
}
}

View File

@@ -207,34 +207,3 @@ func parseFilenameFallback(cd string) string {
return decodeFilenameParam(value)
}
var progressUpdatesLevels = []struct {
size int64 // 文件大小阈值
stepPercent int // 每多少 % 更新一次
}{
{10 << 20, 100},
{50 << 20, 50},
{200 << 20, 20},
{500 << 20, 10},
}
func shouldUpdateProgress(total, downloaded int64, lastUpdatePercent int) bool {
if total <= 0 || downloaded <= 0 {
return false
}
percent := int((downloaded * 100) / total)
if percent <= lastUpdatePercent {
return false
}
step := progressUpdatesLevels[len(progressUpdatesLevels)-1].stepPercent
for _, lvl := range progressUpdatesLevels {
if total < lvl.size {
step = lvl.stepPercent
break
}
}
return percent >= lastUpdatePercent+step
}

View File

@@ -16,6 +16,7 @@ import (
"github.com/krau/SaveAny-Bot/config"
"github.com/krau/SaveAny-Bot/pkg/enums/ctxkey"
"github.com/krau/SaveAny-Bot/pkg/parser"
"github.com/krau/SaveAny-Bot/pkg/taskevent"
"golang.org/x/sync/errgroup"
)
@@ -29,21 +30,20 @@ func (t *Task) Execute(ctx context.Context) error {
eg.SetLimit(config.C().Workers)
for _, resource := range t.item.Resources {
eg.Go(func() error {
t.processingMu.RLock()
if t.processing[resource.ID()] != nil {
return fmt.Errorf("resource %s is already being processed", resource.ID())
}
t.processingMu.RUnlock()
resourceID := resource.ID()
t.processingMu.Lock()
t.processing[resource.ID()] = &resource
if t.processing[resourceID] != nil {
t.processingMu.Unlock()
return fmt.Errorf("resource %s is already being processed", resourceID)
}
t.processing[resourceID] = &resource
t.processingMu.Unlock()
defer func() {
t.processingMu.Lock()
delete(t.processing, resource.URL)
delete(t.processing, resourceID)
t.processingMu.Unlock()
}()
err := t.processResource(gctx, resource)
t.downloaded.Add(1)
if errors.Is(err, context.Canceled) {
logger.Debug("Resource processing canceled")
return err
@@ -52,6 +52,7 @@ func (t *Task) Execute(ctx context.Context) error {
logger.Errorf("Error processing resource %s: %v", resource.URL, err)
return fmt.Errorf("failed to process resource %s: %w", resource.URL, err)
}
t.downloaded.Add(1)
return nil
})
}
@@ -107,10 +108,16 @@ func (t *Task) processResource(ctx context.Context, resource parser.Resource) er
}
}()
wr := ioutil.NewProgressWriter(cacheFile, func(n int) {
t.downloadedBytes.Add(int64(n))
downloaded := t.downloadedBytes.Add(int64(n))
if t.progress != nil {
t.progress.OnProgress(ctx, t)
}
taskevent.Emit(ctx, taskevent.Event{
TaskID: t.ID,
Phase: taskevent.PhaseProgress,
TotalBytes: t.totalBytes,
DownloadedBytes: downloaded,
})
})
copyResultCh := make(chan error, 1)

View File

@@ -15,40 +15,10 @@ import (
"github.com/krau/SaveAny-Bot/common/i18n"
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
"github.com/krau/SaveAny-Bot/common/utils/dlutil"
"github.com/krau/SaveAny-Bot/common/utils/progressutil"
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
)
var progressUpdatesLevels = []struct {
size int64 // 文件大小阈值
stepPercent int // 每多少 % 更新一次
}{
{10 << 20, 100},
{50 << 20, 50},
{200 << 20, 20},
{500 << 20, 10},
}
func shouldUpdateProgress(total, downloaded int64, lastUpdatePercent int) bool {
if total <= 0 || downloaded <= 0 {
return false
}
percent := int((downloaded * 100) / total)
if percent <= lastUpdatePercent {
return false
}
step := progressUpdatesLevels[len(progressUpdatesLevels)-1].stepPercent
for _, lvl := range progressUpdatesLevels {
if total < lvl.size {
step = lvl.stepPercent
break
}
}
return percent >= lastUpdatePercent+step
}
type ProgressTracker interface {
OnStart(ctx context.Context, info TaskInfo)
OnProgress(ctx context.Context, info TaskInfo)
@@ -73,7 +43,10 @@ func (p *Progress) OnStart(ctx context.Context, info TaskInfo) {
styling.Plain(i18n.T(i18nk.BotMsgProgressParsedStartPrefix, map[string]any{
"Site": info.Site(),
})),
styling.Code(fmt.Sprintf("%.2f MB (%d个资源)", float64(info.TotalBytes())/(1024*1024), info.TotalResources())),
styling.Code(i18n.T(i18nk.BotMsgProgressSizeWithResources, map[string]any{
"Size": fmt.Sprintf("%.2f MB", float64(info.TotalBytes())/(1024*1024)),
"Count": info.TotalResources(),
})),
); err != nil {
log.FromContext(ctx).Errorf("Failed to build entities: %s", err)
return
@@ -101,7 +74,7 @@ func (p *Progress) OnStart(ctx context.Context, info TaskInfo) {
}
func (p *Progress) OnProgress(ctx context.Context, info TaskInfo) {
if !shouldUpdateProgress(info.TotalBytes(), info.DownloadedBytes(), int(p.lastUpdatePercent.Load())) {
if !progressutil.ShouldUpdate(info.TotalBytes(), info.DownloadedBytes(), int(p.lastUpdatePercent.Load())) {
return
}
percent := int((info.DownloadedBytes() * 100) / info.TotalBytes())
@@ -114,7 +87,10 @@ func (p *Progress) OnProgress(ctx context.Context, info TaskInfo) {
var entities []tg.MessageEntityClass
if err := styling.Perform(&entityBuilder,
styling.Plain(i18n.T(i18nk.BotMsgProgressDownloadingPrefix, nil)),
styling.Code(fmt.Sprintf("%.2f MB (%d个文件)", float64(info.TotalBytes())/(1024*1024), info.TotalResources())),
styling.Code(i18n.T(i18nk.BotMsgProgressSizeWithResources, map[string]any{
"Size": fmt.Sprintf("%.2f MB", float64(info.TotalBytes())/(1024*1024)),
"Count": info.TotalResources(),
})),
styling.Plain(i18n.T(i18nk.BotMsgProgressProcessingListPrefix, nil)),
func() styling.StyledTextOption {
var lines []string

View File

@@ -33,7 +33,6 @@ type Task struct {
downloadedBytes atomic.Int64 // downloaded bytes count
processing map[string]ResourceInfo
processingMu sync.RWMutex
failed map[string]error // [TODO] errors for each resource
}
// Title implements core.Exectable.
@@ -84,6 +83,5 @@ func NewTask(
progress: progressTracker,
processing: make(map[string]ResourceInfo),
processingMu: sync.RWMutex{},
failed: make(map[string]error),
}
}

View File

@@ -11,13 +11,16 @@ import (
"github.com/duke-git/lancet/v2/retry"
"github.com/krau/SaveAny-Bot/common/utils/fsutil"
"github.com/krau/SaveAny-Bot/config"
"github.com/krau/SaveAny-Bot/pkg/taskevent"
"golang.org/x/sync/errgroup"
)
func (t *Task) Execute(ctx context.Context) error {
logger := log.FromContext(ctx)
logger.Infof("Starting Telegraph task %s", t.PhPath)
t.progress.OnStart(ctx, t)
if t.progress != nil {
t.progress.OnStart(ctx, t)
}
eg, gctx := errgroup.WithContext(ctx)
eg.SetLimit(config.C().Workers)
for i, pic := range t.Pics {
@@ -27,8 +30,16 @@ func (t *Task) Execute(ctx context.Context) error {
logger.Errorf("Error processing picture %s: %v", pic, err)
return fmt.Errorf("failed to process picture %s: %w", pic, err)
}
t.downloaded.Add(1)
t.progress.OnProgress(gctx, t)
downloaded := t.downloaded.Add(1)
if t.progress != nil {
t.progress.OnProgress(gctx, t)
}
taskevent.Emit(gctx, taskevent.Event{
TaskID: t.ID,
Phase: taskevent.PhaseProgress,
TotalFiles: t.totalpics,
DownloadedFiles: int(downloaded),
})
return nil
})
}
@@ -38,7 +49,9 @@ func (t *Task) Execute(ctx context.Context) error {
} else {
logger.Infof("Telegraph task %s completed successfully", t.PhPath)
}
t.progress.OnDone(ctx, t, err)
if t.progress != nil {
t.progress.OnDone(ctx, t, err)
}
return err
}

View File

@@ -0,0 +1,31 @@
package telegraph_test
import (
"context"
"io"
"testing"
storconfig "github.com/krau/SaveAny-Bot/config/storage"
"github.com/krau/SaveAny-Bot/core/tasks/telegraph"
storenum "github.com/krau/SaveAny-Bot/pkg/enums/storage"
"github.com/krau/SaveAny-Bot/storage"
)
type mockStorage struct{}
func (mockStorage) Init(context.Context, storconfig.StorageConfig) error { return nil }
func (mockStorage) Type() storenum.StorageType { return storenum.Local }
func (mockStorage) Name() string { return "mock" }
func (mockStorage) Save(context.Context, io.Reader, string) error { return nil }
func (mockStorage) Exists(context.Context, string) bool { return false }
var _ storage.Storage = mockStorage{}
// Regression: API-created tasks run with a nil ProgressTracker (progress goes
// through taskevent); Execute must not panic on the tracker callbacks.
func TestExecuteWithNilTracker(t *testing.T) {
task := telegraph.NewTask("id", t.Context(), "/page", nil, mockStorage{}, "/out", nil, nil)
if err := task.Execute(t.Context()); err != nil {
t.Fatalf("Execute returned error: %v", err)
}
}

View File

@@ -11,6 +11,7 @@ import (
"github.com/gotd/td/tg"
"github.com/krau/SaveAny-Bot/common/i18n"
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
"github.com/krau/SaveAny-Bot/common/utils/progressutil"
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
)
@@ -60,7 +61,7 @@ func (p *Progress) OnStart(ctx context.Context, info TaskInfo) {
}
func (p *Progress) OnProgress(ctx context.Context, info TaskInfo) {
if !shouldUpdateProgress(info.Downloaded(), int64(info.TotalPics())) {
if !progressutil.ShouldUpdateCount(info.Downloaded(), int64(info.TotalPics())) {
return
}
log.FromContext(ctx).Debugf("Progress update: %s, %d/%d", info.TaskID(), info.Downloaded(), info.TotalPics())

View File

@@ -1,13 +0,0 @@
package telegraph
func shouldUpdateProgress(downloaded int64, total int64) bool {
if total <= 0 || downloaded <= 0 {
return false
}
step := int64(10)
if downloaded < step {
return downloaded == total
}
return downloaded%step == 0 || downloaded == total
}

View File

@@ -0,0 +1,42 @@
package tfile
import (
"testing"
"github.com/gotd/td/tg"
tfilepkg "github.com/krau/SaveAny-Bot/pkg/tfile"
)
func TestSourceCaption(t *testing.T) {
tests := []struct {
name string
file tfilepkg.TGFile
want string
ok bool
}{
{
name: "original caption",
file: tfilepkg.NewTGFile(nil, nil, 0, "video.mov", tfilepkg.WithMessage(&tg.Message{Message: "original caption"})),
want: "original caption",
ok: true,
},
{
name: "empty caption suppresses storage fallback",
file: tfilepkg.NewTGFile(nil, nil, 0, "video.mov", tfilepkg.WithMessage(&tg.Message{})),
ok: true,
},
{
name: "file without source message",
file: tfilepkg.NewTGFile(nil, nil, 0, "video.mov"),
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, ok := sourceCaption(tt.file)
if ok != tt.ok || got != tt.want {
t.Fatalf("sourceCaption() = (%q, %v), want (%q, %v)", got, ok, tt.want, tt.ok)
}
})
}
}

View File

@@ -3,6 +3,7 @@ package tfile
import (
"context"
"fmt"
"io"
"os"
"path"
@@ -10,8 +11,12 @@ import (
"github.com/duke-git/lancet/v2/retry"
"github.com/krau/SaveAny-Bot/common/tdler"
"github.com/krau/SaveAny-Bot/common/utils/fsutil"
"github.com/krau/SaveAny-Bot/common/utils/ioutil"
"github.com/krau/SaveAny-Bot/config"
"github.com/krau/SaveAny-Bot/pkg/enums/ctxkey"
"github.com/krau/SaveAny-Bot/pkg/storagetypes"
tfilepkg "github.com/krau/SaveAny-Bot/pkg/tfile"
"github.com/krau/SaveAny-Bot/storage"
)
func (t *Task) Execute(ctx context.Context) error {
@@ -57,13 +62,34 @@ func (t *Task) Execute(ctx context.Context) error {
return fmt.Errorf("failed to get file stat: %w", err)
}
vctx := context.WithValue(ctx, ctxkey.ContentLength, fileStat.Size())
if caption, ok := sourceCaption(t.File); ok {
vctx = storagetypes.WithSourceCaption(vctx, caption)
}
err = retry.Retry(func() error {
file, err := os.Open(t.localPath)
if err != nil {
return fmt.Errorf("failed to open cache file: %w", err)
}
defer file.Close()
if err = t.Storage.Save(vctx, file, t.Path); err != nil {
uploadProgress, tracksUpload := t.Progress.(UploadProgressTracker)
if !tracksUpload {
if err = t.Storage.Save(vctx, file, t.Path); err != nil {
return fmt.Errorf("failed to save file: %w", err)
}
return nil
}
uploadProgress.OnUploadStart(vctx, t, fileStat.Size())
onProgress := func(uploaded, total int64) {
uploadProgress.OnUploadProgress(vctx, t, uploaded, total)
}
if progressSaver, ok := t.Storage.(storage.StorageProgressSaver); ok {
err = progressSaver.SaveWithProgress(vctx, file, t.Path, onProgress)
} else {
var reader io.Reader = ioutil.NewProgressReader(file, fileStat.Size(), onProgress)
err = t.Storage.Save(vctx, reader, t.Path)
}
if err != nil {
return fmt.Errorf("failed to save file: %w", err)
}
return nil
@@ -73,3 +99,11 @@ func (t *Task) Execute(ctx context.Context) error {
}
return nil
}
func sourceCaption(file tfilepkg.TGFile) (string, bool) {
messageFile, ok := file.(tfilepkg.TGFileMessage)
if !ok || messageFile.Message() == nil {
return "", false
}
return messageFile.Message().GetMessage(), true
}

View File

@@ -4,16 +4,17 @@ import (
"context"
"errors"
"fmt"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/charmbracelet/log"
"github.com/gotd/td/telegram/message/entity"
"github.com/gotd/td/telegram/message/styling"
"github.com/gotd/td/tg"
"github.com/krau/SaveAny-Bot/common/i18n"
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
"github.com/krau/SaveAny-Bot/common/utils/dlutil"
"github.com/krau/SaveAny-Bot/common/utils/progressutil"
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
)
@@ -23,153 +24,319 @@ type ProgressTracker interface {
OnDone(ctx context.Context, info TaskInfo, err error)
}
// UploadProgressTracker optionally extends a task progress tracker with a
// distinct upload phase. Keeping it separate preserves compatibility with
// custom download-only trackers.
type UploadProgressTracker interface {
OnUploadStart(ctx context.Context, info TaskInfo, total int64)
OnUploadProgress(ctx context.Context, info TaskInfo, uploaded, total int64)
}
type Progress struct {
MessageID int
ChatID int64
start time.Time
lastUpdatePercent atomic.Int32
lastUpdateAt atomic.Int64
updateMu sync.Mutex
uploadAttempt int
uploadedBytes int64
actualSize int64
hasActualSize bool
}
const (
uploadProgressMinInterval = time.Second
uploadProgressMaxInterval = 3 * time.Second
singleProgressBarWidth = 10
maxSingleErrorRunes = 240
)
type singleProgressPhase int
const (
singlePhaseDownloading singleProgressPhase = iota
singlePhaseUploading
singlePhaseRetrying
)
type renderedSingleMessage struct {
Text string
Entities []tg.MessageEntityClass
Err error
}
func (p *Progress) OnStart(ctx context.Context, info TaskInfo) {
p.updateMu.Lock()
defer p.updateMu.Unlock()
p.start = time.Now()
p.lastUpdatePercent.Store(0)
p.lastUpdateAt.Store(0)
p.uploadAttempt = 0
p.uploadedBytes = 0
p.actualSize = 0
p.hasActualSize = false
log.FromContext(ctx).Debugf("Progress tracking started for message %d in chat %d", p.MessageID, p.ChatID)
entityBuilder := entity.Builder{}
var entities []tg.MessageEntityClass
if err := styling.Perform(&entityBuilder,
styling.Plain(i18n.T(i18nk.BotMsgProgressFileStartPrefix, nil)),
styling.Code(info.FileName()),
styling.Plain(i18n.T(i18nk.BotMsgProgressSavePathPrefix, nil)),
styling.Code(fmt.Sprintf("[%s]:%s", info.StorageName(), info.StoragePath())),
styling.Plain(i18n.T(i18nk.BotMsgProgressFileSizePrefix, nil)),
styling.Code(fmt.Sprintf("%.2f MB", float64(info.FileSize())/(1024*1024))),
); err != nil {
log.FromContext(ctx).Errorf("Failed to build entities: %s", err)
return
}
text, entities := entityBuilder.Complete()
req := &tg.MessagesEditMessageRequest{
ID: p.MessageID,
}
req.SetMessage(text)
req.SetEntities(entities)
req.SetReplyMarkup(&tg.ReplyInlineMarkup{
Rows: []tg.KeyboardButtonRow{
{
Buttons: []tg.KeyboardButtonClass{
tgutil.BuildCancelButton(info.TaskID()),
},
},
}},
)
ext := tgutil.ExtFromContext(ctx)
if ext != nil {
ext.EditMessage(p.ChatID, req)
return
}
p.editMessage(ctx, info.TaskID(), buildSingleProgressMessage(info, singlePhaseDownloading, 0, info.FileSize(), 0, 0), true)
}
func (p *Progress) OnProgress(ctx context.Context, info TaskInfo, downloaded, total int64) {
if !shouldUpdateProgress(total, downloaded, int(p.lastUpdatePercent.Load())) {
p.updateMu.Lock()
defer p.updateMu.Unlock()
now := time.Now()
elapsed := uploadProgressMaxInterval
if lastUpdateAt := p.lastUpdateAt.Load(); lastUpdateAt > 0 {
elapsed = now.Sub(time.Unix(0, lastUpdateAt))
}
if !shouldUpdateSingleDownloadProgress(total, downloaded, int(p.lastUpdatePercent.Load()), elapsed) {
return
}
percent := int32((downloaded * 100) / total)
if p.lastUpdatePercent.Load() == percent {
return
if total > 0 {
percent := int32((downloaded * 100) / total)
if p.lastUpdatePercent.Load() == percent {
return
}
p.lastUpdatePercent.Store(percent)
}
p.lastUpdatePercent.Store(percent)
p.lastUpdateAt.Store(now.UnixNano())
log.FromContext(ctx).Debugf("Progress update: %s, %d/%d", info.FileName(), downloaded, total)
entityBuilder := entity.Builder{}
var entities []tg.MessageEntityClass
if err := styling.Perform(&entityBuilder,
styling.Plain(i18n.T(i18nk.BotMsgProgressFileProcessingPrefix, nil)),
styling.Code(info.FileName()),
styling.Plain(i18n.T(i18nk.BotMsgProgressSavePathPrefix, nil)),
styling.Code(fmt.Sprintf("[%s]:%s", info.StorageName(), info.StoragePath())),
styling.Plain(i18n.T(i18nk.BotMsgProgressFileSizePrefix, nil)),
styling.Code(fmt.Sprintf("%.2f MB", float64(total)/(1024*1024))),
styling.Plain(i18n.T(i18nk.BotMsgProgressAvgSpeedPrefix, nil)),
styling.Bold(fmt.Sprintf("%.2f MB/s", dlutil.GetSpeed(downloaded, p.start)/(1024*1024))),
styling.Plain(i18n.T(i18nk.BotMsgProgressCurrentProgressPrefix, nil)),
styling.Bold(fmt.Sprintf("%.2f%%", float64(downloaded)/float64(total)*100)),
); err != nil {
log.FromContext(ctx).Errorf("Failed to build entities: %s", err)
p.editMessage(ctx, info.TaskID(), buildSingleProgressMessage(
info,
singlePhaseDownloading,
downloaded,
total,
dlutil.GetSpeed(downloaded, p.start),
0,
), true)
}
func shouldUpdateSingleDownloadProgress(total, downloaded int64, lastPercent int, elapsed time.Duration) bool {
if total > 0 {
return progressutil.ShouldUpdate(total, downloaded, lastPercent)
}
return downloaded > 0 && elapsed >= uploadProgressMaxInterval
}
func (p *Progress) OnUploadStart(ctx context.Context, info TaskInfo, total int64) {
p.updateMu.Lock()
defer p.updateMu.Unlock()
p.start = time.Now()
p.lastUpdatePercent.Store(0)
p.lastUpdateAt.Store(p.start.UnixNano())
p.uploadAttempt++
p.uploadedBytes = 0
p.actualSize = max(total, 0)
p.hasActualSize = true
log.FromContext(ctx).Debugf("Upload progress tracking started: %s", info.FileName())
phase := singleUploadPhase(p.uploadAttempt)
p.editMessage(ctx, info.TaskID(), buildSingleProgressMessage(info, phase, 0, total, 0, p.uploadAttempt), true)
}
func (p *Progress) OnUploadProgress(ctx context.Context, info TaskInfo, uploaded, total int64) {
if total <= 0 || uploaded <= 0 {
return
}
text, entities := entityBuilder.Complete()
req := &tg.MessagesEditMessageRequest{
ID: p.MessageID,
p.updateMu.Lock()
defer p.updateMu.Unlock()
if uploaded > total {
uploaded = total
}
req.SetMessage(text)
req.SetEntities(entities)
req.SetReplyMarkup(&tg.ReplyInlineMarkup{
Rows: []tg.KeyboardButtonRow{
{
Buttons: []tg.KeyboardButtonClass{
tgutil.BuildCancelButton(info.TaskID()),
},
},
}},
)
ext := tgutil.ExtFromContext(ctx)
if ext != nil {
ext.EditMessage(p.ChatID, req)
if uploaded < p.uploadedBytes {
return
}
p.uploadedBytes = uploaded
now := time.Now()
lastUpdateAt := time.Unix(0, p.lastUpdateAt.Load())
lastPercent := int(p.lastUpdatePercent.Load())
if !shouldUpdateUploadProgress(total, uploaded, lastPercent, now.Sub(lastUpdateAt)) {
return
}
percent := int32((uploaded * 100) / total)
p.lastUpdatePercent.Store(percent)
p.lastUpdateAt.Store(now.UnixNano())
log.FromContext(ctx).Debugf("Upload progress update: %s, %d/%d", info.FileName(), uploaded, total)
p.editMessage(ctx, info.TaskID(), buildSingleProgressMessage(
info,
singleUploadPhase(p.uploadAttempt),
uploaded,
total,
dlutil.GetSpeed(uploaded, p.start),
p.uploadAttempt,
), true)
}
func shouldUpdateUploadProgress(total, uploaded int64, lastPercent int, elapsed time.Duration) bool {
if total <= 0 || uploaded <= 0 {
return false
}
if uploaded >= total {
return lastPercent < 100 && elapsed >= uploadProgressMinInterval
}
percent := int((uploaded * 100) / total)
if percent < lastPercent {
return false
}
if elapsed < uploadProgressMinInterval {
return false
}
if percent == lastPercent {
return elapsed >= uploadProgressMaxInterval
}
return progressutil.ShouldUpdate(total, uploaded, lastPercent) || elapsed >= uploadProgressMaxInterval
}
func singleUploadPhase(attempt int) singleProgressPhase {
if attempt > 1 {
return singlePhaseRetrying
}
return singlePhaseUploading
}
func (p *Progress) OnDone(ctx context.Context, info TaskInfo, err error) {
p.updateMu.Lock()
defer p.updateMu.Unlock()
if err != nil {
log.FromContext(ctx).Errorf("Progress error for file [%s]: %v", info.FileName(), err)
} else {
log.FromContext(ctx).Debugf("Progress done for file [%s]", info.FileName())
}
entityBuilder := entity.Builder{}
var stylingErr error
p.editMessage(ctx, info.TaskID(), buildSingleDoneMessage(info, p.doneSize(info), err), false)
}
if err != nil {
if errors.Is(err, context.Canceled) {
stylingErr = styling.Perform(&entityBuilder,
styling.Plain(i18n.T(i18nk.BotMsgProgressTaskCanceled, nil)),
styling.Plain("\n"),
styling.Plain(i18n.T(i18nk.BotMsgProgressFileNamePrefix, nil)),
styling.Code(info.FileName()),
)
} else {
stylingErr = styling.Perform(&entityBuilder,
styling.Plain(i18n.T(i18nk.BotMsgProgressDownloadFailedPrefix, nil)),
styling.Code(info.FileName()),
styling.Plain(i18n.T(i18nk.BotMsgProgressErrorPrefix, nil)),
styling.Bold(err.Error()),
)
}
} else {
stylingErr = styling.Perform(&entityBuilder,
styling.Plain(i18n.T(i18nk.BotMsgProgressDownloadDonePrefix, nil)),
styling.Code(info.FileName()),
styling.Plain(i18n.T(i18nk.BotMsgProgressSavePathPrefix, nil)),
styling.Code(fmt.Sprintf("[%s]:%s", info.StorageName(), info.StoragePath())),
)
func (p *Progress) doneSize(info TaskInfo) int64 {
if p.hasActualSize {
return p.actualSize
}
return max(info.FileSize(), 0)
}
if stylingErr != nil {
log.FromContext(ctx).Errorf("Failed to build entities: %s", stylingErr)
func (p *Progress) editMessage(ctx context.Context, taskID string, message renderedSingleMessage, cancellable bool) {
if message.Err != nil {
log.FromContext(ctx).Errorf("Failed to render file progress message: %v", message.Err)
return
}
text, entities := entityBuilder.Complete()
req := &tg.MessagesEditMessageRequest{
ID: p.MessageID,
req := buildSingleEditMessageRequest(p.MessageID, taskID, message, cancellable)
if ext := tgutil.ExtFromContext(ctx); ext != nil {
if _, err := ext.EditMessage(p.ChatID, req); err != nil {
log.FromContext(ctx).Errorf("Failed to edit file progress message: %v", err)
}
}
req.SetMessage(text)
req.SetEntities(entities)
}
ext := tgutil.ExtFromContext(ctx)
if ext != nil {
ext.EditMessage(p.ChatID, req)
func buildSingleEditMessageRequest(messageID int, taskID string, message renderedSingleMessage, cancellable bool) *tg.MessagesEditMessageRequest {
req := &tg.MessagesEditMessageRequest{ID: messageID}
req.SetMessage(message.Text)
if len(message.Entities) > 0 {
req.SetEntities(message.Entities)
}
if cancellable {
req.SetReplyMarkup(&tg.ReplyInlineMarkup{Rows: []tg.KeyboardButtonRow{{
Buttons: []tg.KeyboardButtonClass{tgutil.BuildCancelButton(taskID)},
}}})
}
return req
}
func buildSingleProgressMessage(
info TaskInfo,
phase singleProgressPhase,
current int64,
total int64,
speed float64,
attempt int,
) renderedSingleMessage {
if current < 0 {
current = 0
}
if total > 0 && current > total {
current = total
}
percent := singleProgressPercent(current, total)
destination := fmt.Sprintf("[%s]:%s", info.StorageName(), info.StoragePath())
data := map[string]any{
"Name": info.FileName(),
"Bar": singleProgressBar(percent),
"Progress": percent,
"Speed": singleProgressSpeed(speed),
"Current": dlutil.FormatSize(current),
"Size": dlutil.FormatSize(total),
"Destination": destination,
"Attempt": max(attempt, 1),
}
var key i18nk.Key
switch phase {
case singlePhaseUploading:
key = i18nk.BotMsgProgressSingleUploading
case singlePhaseRetrying:
key = i18nk.BotMsgProgressSingleUploadRetrying
default:
key = i18nk.BotMsgProgressSingleDownloading
if total <= 0 {
key = i18nk.BotMsgProgressSingleDownloadingUnknown
}
}
markup := i18n.T(i18nk.BotMsgProgressSingleStatusHeader, nil) + "\n\n" + localizedProgressMarkup(key, data)
return completeSingleMessage(markup)
}
func buildSingleDoneMessage(info TaskInfo, size int64, err error) renderedSingleMessage {
data := map[string]any{
"Name": info.FileName(),
"Size": dlutil.FormatSize(max(size, 0)),
"Destination": fmt.Sprintf("[%s]:%s", info.StorageName(), info.StoragePath()),
}
var key i18nk.Key
switch {
case err == nil:
key = i18nk.BotMsgProgressSingleDone
case errors.Is(err, context.Canceled):
key = i18nk.BotMsgProgressSingleCanceled
default:
data["Reason"] = truncateSingleError(err.Error())
key = i18nk.BotMsgProgressSingleFailed
}
return completeSingleMessage(localizedProgressMarkup(key, data))
}
func localizedProgressMarkup(key i18nk.Key, data map[string]any) string {
return i18n.T(key, tgutil.EscapeHTMLTemplateData(data))
}
func completeSingleMessage(markup string) renderedSingleMessage {
text, entities, err := tgutil.RenderHTML(markup)
return renderedSingleMessage{Text: text, Entities: entities, Err: err}
}
func singleProgressPercent(current, total int64) int {
if total <= 0 {
return 0
}
return int(min(max(current, 0), total) * 100 / total)
}
func singleProgressBar(percent int) string {
percent = min(max(percent, 0), 100)
filled := percent * singleProgressBarWidth / 100
return strings.Repeat("🟩", filled) + strings.Repeat("⬜️", singleProgressBarWidth-filled)
}
func singleProgressSpeed(speed float64) string {
if speed <= 0 {
return "0 B/s"
}
return dlutil.FormatSize(int64(speed)) + "/s"
}
func truncateSingleError(value string) string {
runes := []rune(value)
if len(runes) <= maxSingleErrorRunes {
return value
}
return string(runes[:maxSingleErrorRunes])
}
type ProgressOption func(*Progress)

View File

@@ -0,0 +1,178 @@
package tfile
import (
"context"
"errors"
"strings"
"sync"
"testing"
"time"
"github.com/gotd/td/tg"
"github.com/krau/SaveAny-Bot/common/i18n"
)
type progressTestTaskInfo struct{}
func (progressTestTaskInfo) TaskID() string { return "task" }
func (progressTestTaskInfo) FileName() string { return "file.bin" }
func (progressTestTaskInfo) FileSize() int64 { return 100 << 20 }
func (progressTestTaskInfo) StoragePath() string { return "file.bin" }
func (progressTestTaskInfo) StorageName() string { return "test" }
func TestShouldUpdateUploadProgress(t *testing.T) {
tests := []struct {
name string
total int64
uploaded int64
lastPercent int
elapsed time.Duration
want bool
}{
{name: "invalid total", total: 0, uploaded: 1, want: false},
{name: "no uploaded bytes", total: 100, uploaded: 0, want: false},
{name: "percentage threshold", total: 100 << 20, uploaded: 10 << 20, elapsed: uploadProgressMinInterval, want: true},
{name: "percentage threshold rate limited", total: 100 << 20, uploaded: 10 << 20, elapsed: uploadProgressMinInterval - time.Millisecond, want: false},
{name: "maximum time threshold", total: 100 << 20, uploaded: 1 << 20, elapsed: uploadProgressMaxInterval, want: true},
{name: "below thresholds", total: 100 << 20, uploaded: 1 << 20, elapsed: uploadProgressMaxInterval - time.Millisecond, want: false},
{name: "completion", total: 100, uploaded: 100, lastPercent: 99, elapsed: uploadProgressMinInterval, want: true},
{name: "completion rate limited", total: 100, uploaded: 100, lastPercent: 99, elapsed: uploadProgressMinInterval - time.Millisecond, want: false},
{name: "completion already reported", total: 100, uploaded: 100, lastPercent: 100, elapsed: uploadProgressMinInterval, want: false},
{name: "out of order callback", total: 100, uploaded: 40, lastPercent: 60, elapsed: uploadProgressMaxInterval, want: false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := shouldUpdateUploadProgress(tt.total, tt.uploaded, tt.lastPercent, tt.elapsed)
if got != tt.want {
t.Fatalf("shouldUpdateUploadProgress() = %v, want %v", got, tt.want)
}
})
}
}
func TestUploadProgressConcurrentCallbacks(t *testing.T) {
progress := new(Progress)
ctx := context.Background()
info := progressTestTaskInfo{}
const total = int64(100 << 20)
progress.OnUploadStart(ctx, info, total)
progress.lastUpdateAt.Store(time.Now().Add(-uploadProgressMaxInterval).UnixNano())
var wg sync.WaitGroup
for uploaded := int64(1 << 20); uploaded <= total; uploaded += 1 << 20 {
uploaded := uploaded
wg.Go(func() {
progress.OnUploadProgress(ctx, info, uploaded, total)
})
}
wg.Wait()
percent := progress.lastUpdatePercent.Load()
if percent <= 0 || percent > 100 {
t.Fatalf("last upload percentage = %d, want a value in (0, 100]", percent)
}
if progress.uploadedBytes != total {
t.Fatalf("maximum uploaded bytes = %d, want %d", progress.uploadedBytes, total)
}
}
func TestSingleUploadRetryKeepsRichProgressLayout(t *testing.T) {
i18n.Init("zh-Hans")
t.Cleanup(func() { i18n.Init("zh-Hans") })
message := buildSingleProgressMessage(
progressTestTaskInfo{},
singleUploadPhase(2),
25<<20,
100<<20,
5<<20,
2,
)
if message.Err != nil {
t.Fatalf("buildSingleProgressMessage() failed: %v", message.Err)
}
for _, want := range []string{
"🔁 上传重试",
"🟩🟩⬜️⬜️⬜️⬜️⬜️⬜️⬜️⬜️ 25%",
"尝试次数2",
"速度5.00 MB/s",
"大小25.00 MB / 100.00 MB",
} {
if !strings.Contains(message.Text, want) {
t.Fatalf("retry progress does not contain %q:\n%s", want, message.Text)
}
}
}
func TestSingleProgressTemplateOwnsStylesAndEscapesValues(t *testing.T) {
i18n.Init("en")
t.Cleanup(func() { i18n.Init("zh-Hans") })
info := htmlProgressTestTaskInfo{}
message := buildSingleProgressMessage(info, singlePhaseDownloading, 50, 100, 25, 0)
if message.Err != nil {
t.Fatalf("buildSingleProgressMessage() failed: %v", message.Err)
}
for _, want := range []string{
`<b>A&B</b>.bin`,
`[store<&>]:dir/<i>x</i>&`,
"Speed: 25 B/s",
} {
if !strings.Contains(message.Text, want) {
t.Fatalf("progress text does not contain %q:\n%s", want, message.Text)
}
}
bold, code, blockquote, italic := singleEntityCounts(message.Entities)
if bold != 2 || code != 6 || blockquote != 1 || italic != 0 {
t.Fatalf("progress entity counts = bold:%d code:%d blockquote:%d italic:%d", bold, code, blockquote, italic)
}
failure := buildSingleDoneMessage(info, 100, errors.New(`<i>remote & failed</i>`))
if failure.Err != nil {
t.Fatalf("buildSingleDoneMessage() failed: %v", failure.Err)
}
if !strings.Contains(failure.Text, `<i>remote & failed</i>`) {
t.Fatalf("failure reason was not preserved literally:\n%s", failure.Text)
}
bold, code, blockquote, italic = singleEntityCounts(failure.Entities)
if bold != 1 || code != 2 || blockquote != 0 || italic != 0 {
t.Fatalf("failure entity counts = bold:%d code:%d blockquote:%d italic:%d", bold, code, blockquote, italic)
}
}
func TestSingleDoneSizeUsesActualUploadSize(t *testing.T) {
progress := new(Progress)
info := progressTestTaskInfo{}
progress.OnStart(context.Background(), info)
progress.OnUploadStart(context.Background(), info, 2048)
if got := progress.doneSize(info); got != 2048 {
t.Fatalf("done size = %d, want actual upload size 2048", got)
}
}
type htmlProgressTestTaskInfo struct{}
func (htmlProgressTestTaskInfo) TaskID() string { return "html-task" }
func (htmlProgressTestTaskInfo) FileName() string { return `<b>A&B</b>.bin` }
func (htmlProgressTestTaskInfo) FileSize() int64 { return 100 }
func (htmlProgressTestTaskInfo) StoragePath() string { return `dir/<i>x</i>&` }
func (htmlProgressTestTaskInfo) StorageName() string { return `store<&>` }
func singleEntityCounts(entities []tg.MessageEntityClass) (bold, code, blockquote, italic int) {
for _, messageEntity := range entities {
switch messageEntity.(type) {
case *tg.MessageEntityBold:
bold++
case *tg.MessageEntityCode:
code++
case *tg.MessageEntityBlockquote:
blockquote++
case *tg.MessageEntityItalic:
italic++
}
}
return
}

View File

@@ -1,32 +0,0 @@
package tfile
var progressUpdatesLevels = []struct {
size int64 // 文件大小阈值
stepPercent int // 每多少 % 更新一次
}{
{10 << 20, 100},
{50 << 20, 20},
{200 << 20, 10},
{500 << 20, 5},
}
func shouldUpdateProgress(total, downloaded int64, lastUpdatePercent int) bool {
if total <= 0 || downloaded <= 0 {
return false
}
percent := int((downloaded * 100) / total)
if percent <= lastUpdatePercent {
return false
}
step := progressUpdatesLevels[len(progressUpdatesLevels)-1].stepPercent
for _, lvl := range progressUpdatesLevels {
if total < lvl.size {
step = lvl.stepPercent
break
}
}
return percent >= lastUpdatePercent+step
}

View File

@@ -4,6 +4,8 @@ import (
"context"
"io"
"sync/atomic"
"github.com/krau/SaveAny-Bot/pkg/taskevent"
)
type ProgressWriterAt struct {
@@ -20,9 +22,16 @@ func (w *ProgressWriterAt) WriteAt(p []byte, off int64) (int, error) {
if err != nil {
return 0, err
}
downloaded := w.downloaded.Add(int64(at))
if w.progress != nil {
w.progress.OnProgress(w.ctx, w.info, w.downloaded.Add(int64(at)), w.total)
w.progress.OnProgress(w.ctx, w.info, downloaded, w.total)
}
taskevent.Emit(w.ctx, taskevent.Event{
TaskID: w.info.TaskID(),
Phase: taskevent.PhaseProgress,
TotalBytes: w.total,
DownloadedBytes: downloaded,
})
return at, nil
}
@@ -56,9 +65,16 @@ func (w *ProgressWriter) Write(p []byte) (int, error) {
if err != nil {
return 0, err
}
downloaded := w.downloaded.Add(int64(at))
if w.progress != nil {
w.progress.OnProgress(w.ctx, w.info, w.downloaded.Add(int64(at)), w.total)
w.progress.OnProgress(w.ctx, w.info, downloaded, w.total)
}
taskevent.Emit(w.ctx, taskevent.Event{
TaskID: w.info.TaskID(),
Phase: taskevent.PhaseProgress,
TotalBytes: w.total,
DownloadedBytes: downloaded,
})
return at, nil
}

View File

@@ -0,0 +1,75 @@
package transfer_test
import (
"context"
"errors"
"io"
"os"
"path/filepath"
"testing"
"github.com/krau/SaveAny-Bot/config"
storconfig "github.com/krau/SaveAny-Bot/config/storage"
"github.com/krau/SaveAny-Bot/core/tasks/transfer"
storenum "github.com/krau/SaveAny-Bot/pkg/enums/storage"
"github.com/krau/SaveAny-Bot/pkg/storagetypes"
"github.com/krau/SaveAny-Bot/storage"
)
// initConfig seeds the global config so task execution reads a sane Workers
// value (the zero default would deadlock errgroup.SetLimit(0)).
func initConfig(t *testing.T) {
t.Helper()
path := filepath.Join(t.TempDir(), "config.toml")
if err := os.WriteFile(path, []byte("workers = 2\n"), 0o644); err != nil {
t.Fatal(err)
}
if err := config.Init(t.Context(), path); err != nil {
t.Fatal(err)
}
}
type cancelSource struct{}
func (cancelSource) Init(context.Context, storconfig.StorageConfig) error { return nil }
func (cancelSource) Type() storenum.StorageType { return storenum.Local }
func (cancelSource) Name() string { return "cancel-source" }
func (cancelSource) Save(context.Context, io.Reader, string) error { return nil }
func (cancelSource) Exists(context.Context, string) bool { return false }
func (cancelSource) ListFiles(context.Context, string) ([]storagetypes.FileInfo, error) {
return nil, nil
}
// OpenFile reports cancellation, as the task context would be cancelled.
func (cancelSource) OpenFile(ctx context.Context, path string) (io.ReadCloser, int64, error) {
return nil, 0, ctx.Err()
}
type voidTarget struct{}
func (voidTarget) Init(context.Context, storconfig.StorageConfig) error { return nil }
func (voidTarget) Type() storenum.StorageType { return storenum.Local }
func (voidTarget) Name() string { return "void-target" }
func (voidTarget) Save(context.Context, io.Reader, string) error { return nil }
func (voidTarget) Exists(context.Context, string) bool { return false }
var (
_ storage.StorageReadable = cancelSource{}
_ storage.Storage = voidTarget{}
)
// Regression: IgnoreErrors must swallow element failures but never a task
// cancellation; Execute must surface context.Canceled.
func TestIgnoreErrorsPropagatesCancel(t *testing.T) {
initConfig(t)
ctx, cancel := context.WithCancel(t.Context())
cancel()
elem := transfer.NewTaskElement(cancelSource{}, storagetypes.FileInfo{Path: "/a", Name: "a", Size: 1}, voidTarget{}, "/out")
task := transfer.NewTransferTask("id", ctx, []transfer.TaskElement{*elem}, nil, true)
err := task.Execute(ctx)
if !errors.Is(err, context.Canceled) {
t.Fatalf("expected context.Canceled, got %v", err)
}
}

View File

@@ -2,6 +2,7 @@ package transfer
import (
"context"
"errors"
"fmt"
"io"
"os"
@@ -11,6 +12,7 @@ import (
"github.com/charmbracelet/log"
"github.com/krau/SaveAny-Bot/config"
"github.com/krau/SaveAny-Bot/pkg/enums/ctxkey"
"github.com/krau/SaveAny-Bot/pkg/taskevent"
"github.com/krau/SaveAny-Bot/storage"
"golang.org/x/sync/errgroup"
)
@@ -19,7 +21,9 @@ import (
func (t *Task) Execute(ctx context.Context) error {
logger := log.FromContext(ctx).WithPrefix(fmt.Sprintf("transfer[%s]", t.ID))
logger.Info("Starting transfer task")
t.Progress.OnStart(ctx, t)
if t.Progress != nil {
t.Progress.OnStart(ctx, t)
}
workers := config.C().Workers
eg, gctx := errgroup.WithContext(ctx)
@@ -27,14 +31,11 @@ func (t *Task) Execute(ctx context.Context) error {
for _, elem := range t.elems {
eg.Go(func() error {
t.processingMu.RLock()
t.processingMu.Lock()
if t.processing[elem.ID] != nil {
t.processingMu.RUnlock()
t.processingMu.Unlock()
return fmt.Errorf("element with ID %s is already being processed", elem.ID)
}
t.processingMu.RUnlock()
t.processingMu.Lock()
t.processing[elem.ID] = &elem
t.processingMu.Unlock()
@@ -45,7 +46,7 @@ func (t *Task) Execute(ctx context.Context) error {
}()
err := t.processElement(gctx, elem)
if err != nil && !t.IgnoreErrors {
if err != nil && (!t.IgnoreErrors || errors.Is(err, context.Canceled)) {
return err
}
if err != nil {
@@ -65,7 +66,9 @@ func (t *Task) Execute(ctx context.Context) error {
logger.Info("Transfer task completed successfully")
}
t.Progress.OnDone(ctx, t, err)
if t.Progress != nil {
t.Progress.OnDone(ctx, t, err)
}
return err
}
@@ -115,7 +118,15 @@ func (t *Task) processElement(ctx context.Context, elem TaskElement) error {
}
t.uploaded.Add(size)
t.Progress.OnProgress(ctx, t)
if t.Progress != nil {
t.Progress.OnProgress(ctx, t)
}
taskevent.Emit(ctx, taskevent.Event{
TaskID: t.ID,
Phase: taskevent.PhaseProgress,
TotalBytes: t.totalSize,
DownloadedBytes: t.uploaded.Load(),
})
logger.Info("File uploaded successfully")
return nil

View File

@@ -14,6 +14,7 @@ import (
"github.com/krau/SaveAny-Bot/common/i18n"
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
"github.com/krau/SaveAny-Bot/common/utils/dlutil"
"github.com/krau/SaveAny-Bot/common/utils/progressutil"
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
)
@@ -83,7 +84,7 @@ func (p *Progress) OnStart(ctx context.Context, info TaskInfo) {
}
func (p *Progress) OnProgress(ctx context.Context, info TaskInfo) {
if !shouldUpdateProgress(info.TotalSize(), info.Uploaded(), int(p.lastUpdatePercent.Load())) {
if !progressutil.ShouldUpdate(info.TotalSize(), info.Uploaded(), int(p.lastUpdatePercent.Load())) {
return
}
percent := int((info.Uploaded() * 100) / info.TotalSize())
@@ -221,14 +222,6 @@ func (p *Progress) OnDone(ctx context.Context, info TaskInfo, err error) {
}
}
func shouldUpdateProgress(total, current int64, lastPercent int) bool {
if total == 0 {
return false
}
currentPercent := int((current * 100) / total)
return currentPercent > lastPercent && currentPercent%5 == 0
}
func formatDuration(d time.Duration) string {
d = d.Round(time.Second)
h := d / time.Hour

View File

@@ -85,12 +85,10 @@ func (t *Task) downloadFiles(ctx context.Context, tempDir string) ([]string, err
cmd := ytdlp.New().
Output(filepath.Join(tempDir, "%(title)s.%(ext)s"))
// If no custom flags are provided, use default behavior
// Apply config-based format/quality defaults only when the user passes no
// custom flags. Any user flag means they take full control of yt-dlp.
if len(t.Flags) == 0 {
cmd = cmd.
FormatSort("res,ext:mp4:m4a").
RecodeVideo("mp4").
RestrictFilenames()
cmd = applyFormatConfig(cmd, config.C().Ytdlp)
}
// Note: If custom flags are provided, users have full control over format/quality
// The output path is always set above to ensure downloads go to the correct directory

View File

@@ -0,0 +1,40 @@
package ytdlp
import (
"strconv"
ytdlp "github.com/lrstanley/go-ytdlp"
"github.com/krau/SaveAny-Bot/config"
)
// buildFormatSelector translates a max height into a yt-dlp format selector.
// It prefers merging the best video+audio within the height limit, then falls
// back to a single muxed stream. An empty result means "no explicit selector".
func buildFormatSelector(maxHeight int) string {
if maxHeight <= 0 {
return ""
}
h := strconv.Itoa(maxHeight)
return "bv*[height<=" + h + "]+ba/b[height<=" + h + "]/b"
}
// applyFormatConfig configures format/quality on the yt-dlp command according to
// the ytdlp config. It is only meant to be called when the user did not supply
// any custom flags, so config-driven defaults never conflict with user input.
func applyFormatConfig(cmd *ytdlp.Command, cfg config.YtdlpConfig) *ytdlp.Command {
switch {
case cfg.Format != "":
cmd = cmd.Format(cfg.Format)
case cfg.MaxHeight > 0:
cmd = cmd.Format(buildFormatSelector(cfg.MaxHeight))
default:
// Preserve the original default: prefer highest resolution mp4/m4a.
cmd = cmd.FormatSort("res,ext:mp4:m4a")
}
if cfg.Recode != "" {
cmd = cmd.RecodeVideo(cfg.Recode)
}
cmd = cmd.RestrictFilenames()
return cmd
}

View File

@@ -0,0 +1,23 @@
package ytdlp
import "testing"
func TestBuildFormatSelector(t *testing.T) {
tests := []struct {
name string
maxHeight int
want string
}{
{"no limit", 0, ""},
{"negative", -1, ""},
{"1080p", 1080, "bv*[height<=1080]+ba/b[height<=1080]/b"},
{"720p", 720, "bv*[height<=720]+ba/b[height<=720]/b"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := buildFormatSelector(tt.maxHeight); got != tt.want {
t.Errorf("buildFormatSelector(%d) = %q, want %q", tt.maxHeight, got, tt.want)
}
})
}
}

View File

@@ -113,6 +113,51 @@ secret = "your-rpc-secret"
remove_after_transfer = true
```
### yt-dlp Configuration
Configures the behavior of the `/ytdlp` command and the `ytdlp` HTTP-API task type when no custom flags are passed.
- `max_height`: Default maximum video resolution by height in pixels (e.g. `1080`, `720`). `0` means no limit (best available). Ignored when `format` is set.
- `format`: A raw yt-dlp format selector (`-f`). When set, it takes precedence over `max_height` and gives you full control, e.g. `bv*[height<=720]+ba/b`.
- `recode`: The target video container yt-dlp recodes into after download (e.g. `mp4`). Leave empty to disable recoding.
{{< hint info >}}
These defaults only apply when using the `/ytdlp` command without passing any custom flags. Passing custom flags on the command (or `flags` in the API) overrides them.
{{< /hint >}}
```toml
[ytdlp]
max_height = 1080
format = "" # e.g. "bv*[height<=720]+ba/b"
recode = "mp4" # empty disables recoding
```
### HTTP API Configuration
When enabled, SaveAny-Bot exposes an HTTP API for creating/querying/canceling tasks programmatically. See [HTTP API](../../usage/api) for the full endpoint reference.
- `enable`: Whether to enable the HTTP API server, default is `false`.
- `host`: Bind address, default `0.0.0.0`.
- `port`: Listen port, default `8080`.
- `token`: Authentication token. **Strongly recommended** — if empty, the API is exposed without any authentication.
```toml
[api]
enable = false
host = "0.0.0.0"
port = 8080
token = "your-token"
```
### Log Configuration
- `level`: Log level. One of `debug`, `info`, `warn`, `error`, `fatal`. Default is `info`.
```toml
[log]
level = "info"
```
### Storage Endpoints List
The storage endpoints list is used to define the storage locations supported by the Bot. Each storage endpoint needs to specify a name, type, and related configuration, using the double bracket syntax `[[storages]]`.

View File

@@ -79,7 +79,8 @@ Stream mode is not supported.
chat_id = "123456789" # Telegram chat ID, the bot will send files to this chat
force_file = false # Force sending as file, default is false
skip_large = false # Skip large files, default is false. If enabled, files exceeding Telegram's limit will not be uploaded.
spilt_size_mb = 2000 # Split size in MB, default is 2000 MB (2 GB). Files larger than this will be split into multiple parts (zip format). Ignored when skip_large is true.
split_large_video = false # Losslessly split oversized videos into one album of up to 10 independently playable parts. Falls back to ZIP parts on failure or when more than 10 parts are required.
split_size_mb = 0 # Split size in MB. 0 uses the uploader account limit: 2000 MB for bots/regular users and 4000 MB for Premium users. Oversized non-video files use ZIP parts. Ignored when skip_large is true.
```
## Rclone
@@ -136,4 +137,4 @@ remote = "myremote"
base_path = "/backup"
config_path = "/path/to/rclone.conf"
flags = ["--progress"]
```
```

View File

@@ -0,0 +1,90 @@
---
title: "CLI Subcommands"
weight: 21
---
# CLI Subcommands
Besides running the Telegram bot with `./saveany-bot` (no subcommand), the binary exposes two helper subcommands for moving local files into a storage backend: `upload` (one-shot) and `watch` (continuous).
These subcommands load the same `config.toml` as the bot, initialize the database and caches, then perform their task. They do **not** start the Telegram bot itself, although storages of type `telegram` will spin up the bot client just for the upload.
## `upload` — Upload a Single File
```
saveany-bot upload -f <file> -s <storage> [-d <dir>] [--no-progress]
```
Flags:
| Flag | Required | Description |
|---|---|---|
| `-f, --file` | Yes | Path to the local file to upload |
| `-s, --storage` | Yes | Target storage name (must exist in `config.toml`) |
| `-d, --dir` | No | Destination directory within the storage. Defaults to the storage's `base_path` |
| `--no-progress` | No | Disable the terminal progress bar |
Examples:
```bash
# Upload a file to the default dir of storage "MyAlist"
./saveany-bot upload -f ./movie.mp4 -s MyAlist
# Upload into a specific subdirectory
./saveany-bot upload -f ./movie.mp4 -s MyAlist -d movies/2026
# Upload via Telegram storage without a progress bar
./saveany-bot upload -f ./photo.jpg -s MyChannel --no-progress
```
## `watch` — Watch a Directory and Auto-Upload
The `watch` subcommand continuously monitors a local directory and uploads created or modified files to a storage backend, preserving the relative directory structure from the watch root.
```
saveany-bot watch -p <path> -s <storage> [-d <dir>] [options]
```
Flags:
| Flag | Default | Description |
|---|---|---|
| `-p, --path` | *(required)* | Local directory to watch |
| `-s, --storage` | *(required)* | Target storage name |
| `-d, --dir` | storage's `base_path` | Destination directory within the storage |
| `-r, --recursive` | `false` | Watch subdirectories recursively |
| `--overwrite` | `false` | Overwrite existing files on the storage instead of skipping them |
| `--initial-scan` | `false` | Upload files already present in the directory on startup |
| `--debounce` | `2s` | How long to wait after the last write before uploading a file |
| `--upload-workers` | `config.workers` | Number of concurrent uploads |
| `--retry-delay` | `3s` | Delay between upload retries |
{{< hint info >}}
Write-completion detection: the watcher debounces per file and only uploads once the file size stays unchanged across the debounce window, so partial/write-in-progress files are not uploaded.
<br />
If a file changes while being uploaded, it is re-uploaded once after the current upload finishes (instead of being queued multiple times).
{{< /hint >}}
Examples:
```bash
# Watch ./inbox and upload new files to "MyAlist" recursively
./saveany-bot watch -p ./inbox -s MyAlist -r
# Watch with a custom destination dir and overwrite
./saveany-bot watch -p ./inbox -s MyAlist -d backup --overwrite
# On startup, also upload everything already in ./inbox
./saveany-bot watch -p ./inbox -s MyAlist --initial-scan
```
### Behavior notes
- Relative directory structure is preserved under the destination directory. A file written to `./inbox/sub/file.txt` with `--path ./inbox` is uploaded to `<dest_dir>/sub/file.txt`.
- `watch` runs until interrupted (e.g. `Ctrl-C` / `SIGINT`); in-flight uploads are drained before exit.
- Retries follow the global `retry` value from `config.toml`, with `--retry-delay` between attempts.
- Telegram-type storages will start the bot client automatically to perform uploads.
{{< hint warning >}}
`watch` is unrelated to the in-bot `/watch` command (which watches Telegram chats). This subcommand watches a **local filesystem directory** and uploads to a storage backend, independent of Telegram.
{{< /hint >}}

View File

@@ -0,0 +1,75 @@
---
title: "File Naming & Conflict Strategies"
weight: 11
---
# File Naming & Conflict Strategies
SaveAny-Bot lets you customize how saved files are named and how collisions with existing files are resolved, directly in Telegram via the `/config` and `/fnametmpl` commands.
## `/config` — User Configuration
The `/config` command opens an inline menu where you can change two per-user settings:
- **Filename strategy** — how the saved file is named
- **Duplicate file strategy** — what happens when a file with the same name already exists in the target storage
Settings are stored per user and apply to all of that user's subsequent save/transfer tasks.
### Filename strategy
| Option | Behavior |
|---|---|
| `Default` | Use the original media filename, or a generated name when no original filename is available |
| `Gen From Msg First` | Generate the filename from the message content (e.g. caption, text) and prefer that over the original filename |
| `Template` | Render the filename from a custom template you define with `/fnametmpl` |
### Duplicate file strategy
| Option | Behavior |
|---|---|
| `Always rename` (default) | Keep the existing file and save the new one with an alternate name |
| `Ask every time` | Prompt you with inline buttons each time a collision occurs |
| `Always overwrite` | Replace the existing file with the new one |
| `Always skip` | Do nothing for conflicting files |
{{< hint info >}}
The conflict strategy only kicks in for storage backends that can detect the existence of a file. Backends that do not support existence checks will fall back to overwriting.
{{< /hint >}}
## `/fnametmpl` — Custom Filename Template
When the filename strategy is set to `Template`, SaveAny-Bot renders each saved file's name using the template configured via `/fnametmpl`.
```
/fnametmpl [template]
```
- Running `/fnametmpl` without arguments shows your current template and the help text.
- Running it with a template string sets that template as your filename template.
The template uses Go [`text/template`](https://pkg.go.dev/text/template) syntax. The available variables are:
| Variable | Description |
|---|---|
| `{{.msgid}}` | Telegram message ID |
| `{{.msgtags}}` | Hashtags found in the message, joined with `_` |
| `{{.msggen}}` | Filename generated from the message |
| `{{.msgdate}}` | Message date, formatted `YYYY-MM-DD_HH-MM-SS` |
| `{{.msgraw}}` | Raw, unprocessed message text |
| `{{.origname}}` | The media's original filename (if any) |
| `{{.chatid}}` | Chat ID of the message |
Examples:
```
# Fixed prefix + message id + date
/fnametmpl Image_{{.msgid}}_{{.msgdate}}.jpg
# Use original name if available, otherwise a generated name
/fnametmpl {{.origname}}
```
{{< hint warning >}}
The template only takes effect when the filename strategy is set to `Template`. If template parsing fails, SaveAny-Bot falls back to the default filename naming logic.
{{< /hint >}}

View File

@@ -27,6 +27,45 @@ Pay attention to spaces; the bot can only parse correctly formatted syntax. Belo
In addition, if `CHOSEN` is used as the storage name in the rule, it means files will be stored under the path of the storage you selected by clicking the inline button.
You can also toggle whether rules are applied with `/rule switch`. When rule mode is off, all files go to the default storage.
## Preset Rules
Manually writing regex rules for common file types is tedious, so the bot ships a built-in set of preset categories (video, image, audio, document, archive) that you can import in one command:
```
/rule preset <storage> [base_path]
```
Parameters:
- `storage`: Target storage name (must exist and be accessible to you)
- `base_path`: Optional. Each preset category's subdirectory is created under this path. If omitted, the default category directory names are used directly.
Examples:
```
# Import preset rules into "MyAlist" with the default directory layout
/rule preset MyAlist
# Import preset rules with a custom base path "downloads/sorted"
/rule preset MyAlist downloads/sorted
```
This will create `FILENAME-REGEX` rules for each category, routing matched files to the corresponding subdirectory under `base_path`:
| Category | Matched extensions | Default directory |
|---|---|---|
| video | mp4, mkv, ts, avi, flv, mov, webm, wmv, rmvb, m2ts | `视频` |
| image | jpg, jpeg, png, gif, webp, bmp | `图片` |
| audio | mp3, flac, wav, aac, m4a, ogg | `音频` |
| document | pdf, doc, docx, xls, xlsx, ppt, pptx, txt, md, csv, epub, mobi, azw3, chm | `文档` |
| archive | zip, rar, 7z, tar, gz, bz2, xz, ... | `压缩包` |
{{< hint info >}}
Preset rules are regular `FILENAME-REGEX` rules once imported. You can view, edit, or delete them individually with `/rule` and `/rule del <id>` like any other rule.
{{< /hint >}}
Rule types:
## FILENAME-REGEX

View File

@@ -30,6 +30,7 @@ base_path = "./downloads"
### 全局配置
- `lang`: Bot 使用的语言, 默认为 `zh-CN` (简体中文), 设为 `en` 则使用英语.
- `stream`: 是否启用 Stream 模式, 默认为 `false`. 启用后 Bot 将直接将文件流式传输到存储端(若存储端支持), 不需要下载到本地
{{< hint warning >}}
Stream 模式对于磁盘空间有限的部署环境十分有用, 但也有一些弊端:
@@ -47,6 +48,7 @@ Stream 模式对于磁盘空间有限的部署环境十分有用, 但也有一
- `proxy`: 全局代理配置, 配置后程序内一切网络连接将会尝试使用该代理, 可选.
```toml
lang = "zh-CN"
stream = false
workers = 3
threads = 4
@@ -111,6 +113,51 @@ secret = "your-rpc-secret"
remove_after_transfer = true
```
### yt-dlp 配置
用于配置 `/ytdlp` 命令以及 HTTP API 中 `ytdlp` 任务类型在未传自定义参数时的默认行为.
- `max_height`: 默认下载的最高视频清晰度 (按高度限制), 如 `1080`, `720`, `480`; `0` 表示不限制 (下载最佳画质). 当设置了 `format` 时此项被忽略.
- `format`: 直接指定 yt-dlp format 选择表达式, 设置后优先级高于 `max_height`, 例如 `bv*[height<=720]+ba/b`.
- `recode`: 下载后转封装的视频容器格式 (如 `mp4`), 留空则不转封装.
{{< hint info >}}
这些默认值仅在使用 `/ytdlp` 命令且未传任何自定义参数时生效. 在命令上传递自定义参数 (或在 API 中传 `flags`) 会覆盖这些默认值.
{{< /hint >}}
```toml
[ytdlp]
max_height = 1080
format = "" # 例如 "bv*[height<=720]+ba/b"
recode = "mp4" # 留空则不转封装
```
### HTTP API 配置
启用后, SaveAny-Bot 会暴露一套 HTTP API, 用于以编程方式创建/查询/取消任务. 完整的接口说明见 [HTTP API](../../usage/api).
- `enable`: 是否启用 HTTP API 服务, 默认为 `false`.
- `host`: 监听地址, 默认 `0.0.0.0`.
- `port`: 监听端口, 默认 `8080`.
- `token`: 鉴权 Token, **强烈建议设置** — 若为空, API 将在无任何鉴权的情况下暴露.
```toml
[api]
enable = false
host = "0.0.0.0"
port = 8080
token = "your-token"
```
### 日志配置
- `level`: 日志级别, 可选 `debug`, `info`, `warn`, `error`, `fatal`. 默认为 `info`.
```toml
[log]
level = "info"
```
### 存储端列表
存储端列表用于定义 Bot 支持的存储位置, 每个存储端需要指定名称、类型和相关配置, 使用双中括号语法 `[[storages]]` 定义.

View File

@@ -82,10 +82,12 @@ chat_id = "123456789"
force_file = false
# 是否跳过大文件, 默认为 false. 如果启用, 超过 Telegram 限制的文件将不会上传.
skip_large = false
# 分卷大小, 单位 MB, 默认为 2000 MB (2 GB).
# 超过该大小的文件将被分割成多个部分上传.(使用 zip 格式)
# 超限视频是否使用 FFmpeg 无损分割成一个媒体组(最多 10 个)内可独立播放的小视频;失败或需要超过 10 段时回退到 ZIP 分卷.
split_large_video = false
# 分卷大小, 单位 MB. 设为 0 时使用实际上传账号的限制Bot/普通用户为 2000 MBPremium 用户为 4000 MB.
# 超过该大小的文件将被分割成多个部分上传;非视频使用 ZIP 格式.
# 当 skip_large 启用时, 该选项无效.
spilt_size_mb = 2000
split_size_mb = 0
```
## Rclone
@@ -142,4 +144,4 @@ remote = "myremote"
base_path = "/backup"
config_path = "/path/to/rclone.conf"
flags = ["--progress"]
```
```

View File

@@ -0,0 +1,90 @@
---
title: "命令行子命令"
weight: 21
---
# 命令行子命令
除了直接运行 `./saveany-bot` (不带子命令) 启动 Telegram Bot 外, 这个二进制文件还提供两个把本地文件上传到存储后端的辅助子命令: `upload` (一次性) 和 `watch` (持续监听).
这些子命令会读取与 Bot 相同的 `config.toml`, 初始化数据库和缓存, 然后执行任务. 它们**不会**启动 Telegram Bot 本身, 但 `telegram` 类型的存储会在需要上传时临时启动 Bot 客户端来执行上传.
## `upload` — 上传单个文件
```
saveany-bot upload -f <文件> -s <存储名> [-d <目录>] [--no-progress]
```
参数:
| 参数 | 必填 | 说明 |
|---|---|---|
| `-f, --file` | 是 | 待上传的本地文件路径 |
| `-s, --storage` | 是 | 目标存储名 (必须存在于 `config.toml`) |
| `-d, --dir` | 否 | 存储中的目标目录, 默认使用存储的 `base_path` |
| `--no-progress` | 否 | 关闭终端进度条 |
示例:
```bash
# 上传文件到 "MyAlist" 的默认目录
./saveany-bot upload -f ./movie.mp4 -s MyAlist
# 上传到指定子目录
./saveany-bot upload -f ./movie.mp4 -s MyAlist -d movies/2026
# 通过 Telegram 存储上传并关闭进度条
./saveany-bot upload -f ./photo.jpg -s MyChannel --no-progress
```
## `watch` — 监听目录并自动上传
`watch` 子命令持续监听一个本地目录, 将新建或修改的文件上传到存储后端, 并保留相对监听根目录的子目录结构.
```
saveany-bot watch -p <路径> -s <存储名> [-d <目录>] [选项]
```
参数:
| 参数 | 默认值 | 说明 |
|---|---|---|
| `-p, --path` | *(必填)* | 要监听的本地目录 |
| `-s, --storage` | *(必填)* | 目标存储名 |
| `-d, --dir` | 存储的 `base_path` | 存储中的目标目录 |
| `-r, --recursive` | `false` | 是否递归监听子目录 |
| `--overwrite` | `false` | 覆盖存储上已有的文件, 而非跳过 |
| `--initial-scan` | `false` | 启动时将目录中已存在的文件也上传 |
| `--debounce` | `2s` | 文件最后一次写入后, 等待多久再上传 |
| `--upload-workers` | `config.workers` | 并发上传数 |
| `--retry-delay` | `3s` | 上传重试之间的延迟 |
{{< hint info >}}
写入完成检测: 监听器会按文件做防抖处理, 仅当文件大小在一个 debounce 窗口内保持不变时才上传, 因此不会上传未写完的半成品文件.
<br />
若某文件在上传过程中又被修改, 它会在当前上传完成后再上传一次, 而不是被重复排队.
{{< /hint >}}
示例:
```bash
# 递归监听 ./inbox 并且把新文件上传到 "MyAlist"
./saveany-bot watch -p ./inbox -s MyAlist -r
# 自定义目标目录并覆盖已有文件
./saveany-bot watch -p ./inbox -s MyAlist -d backup --overwrite
# 启动时把 ./inbox 中已有的内容也一并上传
./saveany-bot watch -p ./inbox -s MyAlist --initial-scan
```
### 行为说明
- 相对子目录结构会被保留: 以 `--path ./inbox` 为例, 写入 `./inbox/sub/file.txt` 的文件会被上传到 `<目标目录>/sub/file.txt`.
- `watch` 会一直运行直到被中断 (如 `Ctrl-C` / `SIGINT`), 退出前会等待所有进行中的上传完成.
- 重试次数遵循 `config.toml` 中的全局 `retry` 值, 各次重试之间间隔 `--retry-delay`.
- `telegram` 类型的存储会自动启动 Bot 客户端来执行上传.
{{< hint warning >}}
`watch` 子命令与 Bot 内的 `/watch` 命令 (监听 Telegram 聊天) 无关. 本子命令监听的是**本地文件系统目录**, 不依赖 Telegram.
{{< /hint >}}

View File

@@ -0,0 +1,75 @@
---
title: "文件命名与重名策略"
weight: 11
---
# 文件命名与重名策略
SaveAny-Bot 支持在 Telegram 中通过 `/config``/fnametmpl` 命令自定义保存文件的命名方式, 以及处理与已存在文件重名时的冲突策略.
## `/config` — 用户配置
`/config` 命令会弹出一个内联菜单, 你可以在其中修改以下两项用户级设置:
- **文件名策略** — 保存文件的命名方式
- **重名文件保存策略** — 目标存储中已存在同名文件时的处理方式
设置按用户分别保存, 对该用户后续所有的保存/转存任务生效.
### 文件名策略
| 选项 | 行为 |
|---|---|
| `默认` | 使用媒体原始文件名, 没有原始文件名时使用生成名 |
| `优先从消息生成` | 优先根据消息内容 (如 caption、文本) 生成文件名, 而非原始文件名 |
| `自定义模板` | 使用 `/fnametmpl` 设置的自定义模板渲染文件名 |
### 重名文件保存策略
| 选项 | 行为 |
|---|---|
| `始终重命名` (默认) | 保留已有文件, 将新文件以另一个名字保存 |
| `每次询问` | 每次遇到重名时通过内联按钮提示你选择 |
| `始终覆盖` | 用新文件替换已有文件 |
| `始终跳过` | 对重名文件不做处理 |
{{< hint info >}}
重名策略仅在能够检测文件是否已存在的存储后端生效. 不支持检测文件是否存在的存储后端会退化为覆盖行为.
{{< /hint >}}
## `/fnametmpl` — 自定义文件名模板
当文件名策略设置为 `自定义模板` 时, SaveAny-Bot 会用 `/fnametmpl` 配置的模板来渲染所保存文件的文件名.
```
/fnametmpl [模板]
```
- 不带参数运行 `/fnametmpl` 会显示当前模板以及帮助说明
- 带模板字符串运行则会把它设为你的文件名模板
模板使用 Go [`text/template`](https://pkg.go.dev/text/template) 语法. 可用变量如下:
| 变量 | 说明 |
|---|---|
| `{{.msgid}}` | Telegram 消息 ID |
| `{{.msgtags}}` | 消息中的标签, 以 `_` 连接输出 |
| `{{.msggen}}` | 根据消息生成的文件名 |
| `{{.msgdate}}` | 消息日期, 格式 `YYYY-MM-DD_HH-MM-SS` |
| `{{.msgraw}}` | 消息的原始文本内容 (不做处理) |
| `{{.origname}}` | 媒体的原始文件名 (如有) |
| `{{.chatid}}` | 消息所在聊天的 ID |
示例:
```
# 固定前缀 + 消息 ID + 日期
/fnametmpl 图片_{{.msgid}}_{{.msgdate}}.jpg
# 优先使用原始文件名, 没有则用生成名
/fnametmpl {{.origname}}
```
{{< hint warning >}}
模板仅在文件名策略设置为 `自定义模板` 时生效. 如果模板解析失败, SaveAny-Bot 会回退到默认的文件名生成逻辑.
{{< /hint >}}

View File

@@ -27,6 +27,45 @@ weight: 3
此外, 规则中的存储名若使用 "CHOSEN" , 则表示存储到点击按钮选择的存储端的路径下
你也可以使用 `/rule switch` 来开关规则模式. 关闭规则模式时, 所有文件都将保存到默认存储.
## 预设规则
为常见文件类型手动编写正则规则比较繁琐, 因此 Bot 内置了一组预设分类 (视频、图片、音频、文档、压缩包), 可以通过一条命令批量导入:
```
/rule preset <存储名> [基础路径]
```
参数:
- `存储名`: 目标存储名 (必须存在且你有权访问)
- `基础路径`: 可选. 各预设分类的子目录会创建在此路径下; 若不填则直接使用默认分类目录名
示例:
```
# 导入预设规则到 "MyAlist", 使用默认目录布局
/rule preset MyAlist
# 在自定义基础路径 "downloads/sorted" 下导入预设规则
/rule preset MyAlist downloads/sorted
```
此命令会为每个分类创建 `FILENAME-REGEX` 规则, 将匹配的文件路由到 `基础路径` 下对应的子目录:
| 分类 | 匹配的扩展名 | 默认目录 |
|---|---|---|
| 视频 | mp4, mkv, ts, avi, flv, mov, webm, wmv, rmvb, m2ts | `视频` |
| 图片 | jpg, jpeg, png, gif, webp, bmp | `图片` |
| 音频 | mp3, flac, wav, aac, m4a, ogg | `音频` |
| 文档 | pdf, doc, docx, xls, xlsx, ppt, pptx, txt, md, csv, epub, mobi, azw3, chm | `文档` |
| 压缩包 | zip, rar, 7z, tar, gz, bz2, xz, ... | `压缩包` |
{{< hint info >}}
导入后的预设规则就是普通的 `FILENAME-REGEX` 规则. 你可以像其他规则一样通过 `/rule` 查看或用 `/rule del <id>` 单独删除/编辑它们.
{{< /hint >}}
规则类型:
## FILENAME-REGEX

File diff suppressed because one or more lines are too long

View File

@@ -1 +1 @@
{"Target":"book.min.a22f4c7d8c2bdc5e3d6e34ba11cb59ab50ea5772594e71305bfd5a595dc78b7e.css","MediaType":"text/css","Data":{"Integrity":"sha256-oi9MfYwr3F49bjS6EctZq1DqV3JZTnEwW/1aWV3Hi34="}}
{"Target":"book.min.a643d39733d3a0ac48d6369128a52703207c5e11a74c3a70cfebfe0c15838ab8.css","MediaType":"text/css","Data":{"Integrity":"sha256-pkPTlzPToKxI1jaRKKUnAyB8XhGnTDpwz+v+DBWDirg="}}

100
go.mod
View File

@@ -11,22 +11,22 @@ require (
github.com/charmbracelet/lipgloss v1.1.0
github.com/charmbracelet/log v1.0.0
github.com/dustin/go-humanize v1.0.1
github.com/gabriel-vasile/mimetype v1.4.13
github.com/gabriel-vasile/mimetype v1.4.15
github.com/goccy/go-yaml v1.19.2
github.com/gotd/contrib v0.21.1
github.com/gotd/td v0.143.0
github.com/gotd/td v0.149.0
github.com/johannesboyne/gofakes3 v0.0.0-20250916175020-ebf3e50324d3
github.com/krau/ffmpeg-go v0.6.0
github.com/lrstanley/go-ytdlp v1.3.5
github.com/minio/minio-go/v7 v7.0.100
github.com/playwright-community/playwright-go v0.5700.1
github.com/minio/minio-go/v7 v7.2.1
github.com/playwright-community/playwright-go v0.6000.0
github.com/rs/xid v1.6.0
github.com/spf13/cobra v1.10.2
github.com/spf13/viper v1.21.0
github.com/unvgo/ghselfupdate v1.0.1
github.com/yapingcat/gomedia v0.0.0-20240906162731-17feea57090c
golang.org/x/net v0.53.0
golang.org/x/term v0.42.0
golang.org/x/net v0.57.0
golang.org/x/term v0.45.0
golang.org/x/time v0.15.0
)
@@ -43,19 +43,19 @@ require (
github.com/charmbracelet/x/term v0.2.2 // indirect
github.com/clipperhouse/displaywidth v0.11.0 // indirect
github.com/clipperhouse/uax29/v2 v2.7.0 // indirect
github.com/cloudflare/circl v1.6.3 // indirect
github.com/coder/websocket v1.8.14 // indirect
github.com/deckarep/golang-set/v2 v2.8.0 // indirect
github.com/dlclark/regexp2 v1.11.5 // indirect
github.com/cloudflare/circl v1.6.5 // indirect
github.com/coder/websocket v1.8.15 // indirect
github.com/deckarep/golang-set/v2 v2.9.0 // indirect
github.com/dlclark/regexp2 v1.12.0 // indirect
github.com/dlclark/regexp2/v2 v2.6.0 // indirect
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f // indirect
github.com/fatih/color v1.19.0 // indirect
github.com/ghodss/yaml v1.0.0 // indirect
github.com/glebarez/go-sqlite v1.22.0 // indirect
github.com/go-faster/errors v0.7.1 // indirect
github.com/glebarez/go-sqlite v1.23.0 // indirect
github.com/go-faster/errors v0.8.0 // indirect
github.com/go-faster/jx v1.2.0 // indirect
github.com/go-faster/xor v1.0.0 // indirect
github.com/go-faster/yaml v0.4.6 // indirect
github.com/go-ini/ini v1.67.0 // indirect
github.com/go-jose/go-jose/v3 v3.0.5 // indirect
github.com/go-logfmt/logfmt v0.6.1 // indirect
github.com/go-sourcemap/sourcemap v2.1.4+incompatible // indirect
@@ -63,29 +63,29 @@ require (
github.com/go-viper/mapstructure/v2 v2.5.0 // indirect
github.com/google/go-github/v30 v30.1.0 // indirect
github.com/google/go-querystring v1.2.0 // indirect
github.com/google/pprof v0.0.0-20260402051712-545e8a4df936 // indirect
github.com/google/pprof v0.0.0-20260802141513-ef3492d7dac3 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/gotd/ige v0.2.2 // indirect
github.com/gotd/ige v0.3.0 // indirect
github.com/gotd/neo v0.1.5 // indirect
github.com/inconshreveable/go-update v0.0.0-20160112193335-8152e7eb6ccf // indirect
github.com/jinzhu/inflection v1.0.0 // indirect
github.com/jinzhu/now v1.1.5 // indirect
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
github.com/klauspost/cpuid/v2 v2.4.0 // indirect
github.com/klauspost/crc32 v1.3.0 // indirect
github.com/lucasb-eyer/go-colorful v1.4.0 // indirect
github.com/mattn/go-colorable v0.1.14 // indirect
github.com/mattn/go-isatty v0.0.21 // indirect
github.com/lucasb-eyer/go-colorful v1.4.1 // indirect
github.com/mattn/go-colorable v0.1.15 // indirect
github.com/mattn/go-isatty v0.0.24 // indirect
github.com/mattn/go-localereader v0.0.1 // indirect
github.com/mattn/go-runewidth v0.0.23 // indirect
github.com/mattn/go-runewidth v0.0.27 // indirect
github.com/minio/crc64nvme v1.1.1 // indirect
github.com/minio/md5-simd v1.1.2 // indirect
github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 // indirect
github.com/muesli/cancelreader v0.2.2 // indirect
github.com/muesli/termenv v0.16.0 // indirect
github.com/ncruces/go-sqlite3-wasm v1.1.1-0.20260409221933-87e4b35a38d0 // indirect
github.com/ncruces/go-sqlite3-wasm/v3 v3.2.35304 // indirect
github.com/ncruces/go-strftime v1.0.0 // indirect
github.com/ncruces/julianday v1.0.0 // indirect
github.com/ogen-go/ogen v1.20.3 // indirect
github.com/ogen-go/ogen v1.24.0 // indirect
github.com/philhofer/fwd v1.2.0 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
@@ -93,49 +93,53 @@ require (
github.com/ryszard/goskiplist v0.0.0-20150312221310-2dfbae5fcf46 // indirect
github.com/segmentio/asm v1.2.1 // indirect
github.com/shopspring/decimal v1.4.0 // indirect
github.com/tinylib/msgp v1.6.3 // indirect
github.com/ulikunitz/xz v0.5.15 // indirect
go.opentelemetry.io/otel v1.43.0 // indirect
go.opentelemetry.io/otel/metric v1.43.0 // indirect
go.opentelemetry.io/otel/trace v1.43.0 // indirect
github.com/tinylib/msgp v1.6.4 // indirect
github.com/ulikunitz/xz v0.5.16 // indirect
github.com/yuin/goldmark v1.8.5 // indirect
github.com/zeebo/xxh3 v1.1.0 // indirect
go.mongodb.org/mongo-driver v1.17.9 // indirect
go.opentelemetry.io/otel v1.45.0 // indirect
go.opentelemetry.io/otel/metric v1.45.0 // indirect
go.opentelemetry.io/otel/trace v1.45.0 // indirect
go.shabbyrobe.org/gocovmerge v0.0.0-20230507111327-fa4f82cfbf4d // indirect
go.uber.org/atomic v1.11.0 // indirect
go.uber.org/zap v1.27.1 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/crypto v0.50.0 // indirect
golang.org/x/mod v0.35.0 // indirect
golang.org/x/tools v0.44.0 // indirect
go.uber.org/zap v1.28.0 // indirect
go.yaml.in/yaml/v3 v3.0.5 // indirect
golang.org/x/crypto v0.54.0 // indirect
golang.org/x/mod v0.39.0 // indirect
golang.org/x/tools v0.48.0 // indirect
gopkg.in/ini.v1 v1.67.3 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
modernc.org/libc v1.72.0 // indirect
modernc.org/libc v1.75.3 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.11.0 // indirect
modernc.org/sqlite v1.48.2 // indirect
modernc.org/memory v1.12.0 // indirect
modernc.org/sqlite v1.56.0 // indirect
rsc.io/qr v0.2.0 // indirect
)
require (
github.com/dgraph-io/ristretto/v2 v2.4.0
github.com/dop251/goja v0.0.0-20260311135729-065cd970411c
github.com/dgraph-io/ristretto/v2 v2.4.2
github.com/dop251/goja v0.0.0-20260806115107-493f22071ef6
github.com/duke-git/lancet/v2 v2.3.9
github.com/fsnotify/fsnotify v1.9.0
github.com/fsnotify/fsnotify v1.10.1
github.com/glebarez/sqlite v1.11.0
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/klauspost/compress v1.18.5 // indirect
github.com/klauspost/compress v1.19.2 // indirect
github.com/mitchellh/mapstructure v1.5.0
github.com/ncruces/go-sqlite3 v0.33.3 // indirect
github.com/ncruces/go-sqlite3/gormlite v0.33.3
github.com/ncruces/go-sqlite3 v0.35.3 // indirect
github.com/ncruces/go-sqlite3/gormlite v0.34.0
github.com/nicksnyder/go-i18n/v2 v2.6.1
github.com/pelletier/go-toml/v2 v2.3.0 // indirect
github.com/pelletier/go-toml/v2 v2.4.3 // indirect
github.com/sagikazarmark/locafero v0.12.0 // indirect
github.com/spf13/afero v1.15.0 // indirect
github.com/spf13/cast v1.10.0 // indirect
github.com/spf13/pflag v1.0.10 // indirect
github.com/subosito/gotenv v1.6.0 // indirect
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
github.com/xo/terminfo v1.0.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f // indirect
golang.org/x/sync v0.20.0
golang.org/x/sys v0.43.0 // indirect
golang.org/x/text v0.36.0
gorm.io/gorm v1.31.1
golang.org/x/exp v0.0.0-20260810151157-a8b543ca52da // indirect
golang.org/x/sync v0.22.0
golang.org/x/sys v0.47.0 // indirect
golang.org/x/text v0.40.0
gorm.io/gorm v1.31.2
)

240
go.sum
View File

@@ -2,8 +2,8 @@ github.com/AnimeKaizoku/cacher v1.0.3 h1:foNAmLfY/DXfA4yEy4uP6WK2Ni7JC+s3QhZv72D
github.com/AnimeKaizoku/cacher v1.0.3/go.mod h1:jw0de/b0K6W7Y3T9rHCMGVKUf6oG7hENNcssxYcZTCc=
github.com/BurntSushi/toml v1.6.0 h1:dRaEfpa2VI55EwlIW72hMRHdWouJeRF7TPYhI+AUQjk=
github.com/BurntSushi/toml v1.6.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
github.com/Masterminds/semver/v3 v3.2.1 h1:RN9w6+7QoMeJVGyfmbcgs28Br8cvmnucEXnY0rYXWg0=
github.com/Masterminds/semver/v3 v3.2.1/go.mod h1:qvl/7zhW3nngYb5+80sSMF+FG2BjYrf8m9wsX0PNOMQ=
github.com/Masterminds/semver/v3 v3.5.0 h1:kQceYJfbupGfZOKZQg0kou0DgAKhzDg2NZPAwZ/2OOE=
github.com/Masterminds/semver/v3 v3.5.0/go.mod h1:4V+yj/TJE1HU9XfppCwVMZq3I84lprf4nC11bSS5beM=
github.com/ProtonMail/go-crypto v1.4.1 h1:9RfcZHqEQUvP8RzecWEUafnZVtEvrBVL9BiF67IQOfM=
github.com/ProtonMail/go-crypto v1.4.1/go.mod h1:e1OaTyu5SYVrO9gKOEhTc+5UcXtTUa+P3uLudwcgPqo=
github.com/aws/aws-sdk-go-v2 v1.36.3 h1:mJoei2CxPutQVxaATCzDUjcZEjVRdpsiiXi2o38yqWM=
@@ -66,24 +66,26 @@ github.com/clipperhouse/displaywidth v0.11.0 h1:lBc6kY44VFw+TDx4I8opi/EtL9m20WSE
github.com/clipperhouse/displaywidth v0.11.0/go.mod h1:bkrFNkf81G8HyVqmKGxsPufD3JhNl3dSqnGhOoSD/o0=
github.com/clipperhouse/uax29/v2 v2.7.0 h1:+gs4oBZ2gPfVrKPthwbMzWZDaAFPGYK72F0NJv2v7Vk=
github.com/clipperhouse/uax29/v2 v2.7.0/go.mod h1:EFJ2TJMRUaplDxHKj1qAEhCtQPW2tJSwu5BF98AuoVM=
github.com/cloudflare/circl v1.6.3 h1:9GPOhQGF9MCYUeXyMYlqTR6a5gTrgR/fBLXvUgtVcg8=
github.com/cloudflare/circl v1.6.3/go.mod h1:2eXP6Qfat4O/Yhh8BznvKnJ+uzEoTQ6jVKJRn81BiS4=
github.com/coder/websocket v1.8.14 h1:9L0p0iKiNOibykf283eHkKUHHrpG7f65OE3BhhO7v9g=
github.com/coder/websocket v1.8.14/go.mod h1:NX3SzP+inril6yawo5CQXx8+fk145lPDC6pumgx0mVg=
github.com/cloudflare/circl v1.6.5 h1:O64F26HEqNhznd/hrC5KZXVKYuKM2rx4deZDTc4ihQA=
github.com/cloudflare/circl v1.6.5/go.mod h1:h5LNyxAc5nTue9DS5jT+48en2PSDYt3zdGnz5OstK6c=
github.com/coder/websocket v1.8.15 h1:6B2JPeOGlpff2Uz6vOEH1Vzpi0iUz20A+lPVhPHtNUA=
github.com/coder/websocket v1.8.15/go.mod h1:NX3SzP+inril6yawo5CQXx8+fk145lPDC6pumgx0mVg=
github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/deckarep/golang-set/v2 v2.8.0 h1:swm0rlPCmdWn9mESxKOjWk8hXSqoxOp+ZlfuyaAdFlQ=
github.com/deckarep/golang-set/v2 v2.8.0/go.mod h1:VAky9rY/yGXJOLEDv3OMci+7wtDpOF4IN+y82NBOac4=
github.com/dgraph-io/ristretto/v2 v2.4.0 h1:I/w09yLjhdcVD2QV192UJcq8dPBaAJb9pOuMyNy0XlU=
github.com/dgraph-io/ristretto/v2 v2.4.0/go.mod h1:0KsrXtXvnv0EqnzyowllbVJB8yBonswa2lTCK2gGo9E=
github.com/deckarep/golang-set/v2 v2.9.0 h1:prva4eP9UysWagLyKrtn074ughi0NnkIf0A4M5yOCKI=
github.com/deckarep/golang-set/v2 v2.9.0/go.mod h1:EWknQXbs0mcFpat2QOoXV0Ee57cD+w6ZEN76BR2JVrM=
github.com/dgraph-io/ristretto/v2 v2.4.2 h1:x0cvjmUKxt764Yxdk2nr94we1AvPPAMh1rh5TQ+Jo80=
github.com/dgraph-io/ristretto/v2 v2.4.2/go.mod h1:0KsrXtXvnv0EqnzyowllbVJB8yBonswa2lTCK2gGo9E=
github.com/dgryski/go-farm v0.0.0-20240924180020-3414d57e47da h1:aIftn67I1fkbMa512G+w+Pxci9hJPB8oMnkcP3iZF38=
github.com/dgryski/go-farm v0.0.0-20240924180020-3414d57e47da/go.mod h1:SqUrOPUnsFjfmXRMNPybcSiG0BgUW2AuFH8PAnS2iTw=
github.com/dlclark/regexp2 v1.11.5 h1:Q/sSnsKerHeCkc/jSTNq1oCm7KiVgUMZRDUoRu0JQZQ=
github.com/dlclark/regexp2 v1.11.5/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/dop251/goja v0.0.0-20260311135729-065cd970411c h1:OcLmPfx1T1RmZVHHFwWMPaZDdRf0DBMZOFMVWJa7Pdk=
github.com/dop251/goja v0.0.0-20260311135729-065cd970411c/go.mod h1:MxLav0peU43GgvwVgNbLAj1s/bSGboKkhuULvq/7hx4=
github.com/dlclark/regexp2 v1.12.0 h1:0j4c5qQmnC6XOWNjP3PIXURXN2gWx76rd3KvgdPkCz8=
github.com/dlclark/regexp2 v1.12.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/dlclark/regexp2/v2 v2.6.0 h1:KugbSrpXcRpziHIcqqFEXwKwi09LbPkTzvLRItf2mo8=
github.com/dlclark/regexp2/v2 v2.6.0/go.mod h1:avUrQvPaLz2DrFNHJF0taWAFFX2C1GMSSoeiqFjcBmU=
github.com/dop251/goja v0.0.0-20260806115107-493f22071ef6 h1:Oh2rRG1un7tLlC3/NJDzKppZ4CeZGkVFJCUOTRwLpfw=
github.com/dop251/goja v0.0.0-20260806115107-493f22071ef6/go.mod h1:LiIEzozrcvNXorsG/3+ypGqdTUAqZryhzSsqi0oU/Qg=
github.com/duke-git/lancet/v2 v2.3.9 h1:ZxUvfoEY7YbsGIeoXRxHWIkRCAt6VN7UBKWgCCqBB3U=
github.com/duke-git/lancet/v2 v2.3.9/go.mod h1:zGa2R4xswg6EG9I6WnyubDbFO/+A/RROxIbXcwryTsc=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
@@ -94,33 +96,30 @@ github.com/fatih/color v1.19.0 h1:Zp3PiM21/9Ld6FzSKyL5c/BULoe/ONr9KlbYVOfG8+w=
github.com/fatih/color v1.19.0/go.mod h1:zNk67I0ZUT1bEGsSGyCZYZNrHuTkJJB+r6Q9VuMi0LE=
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=
github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
github.com/gabriel-vasile/mimetype v1.4.13 h1:46nXokslUBsAJE/wMsp5gtO500a4F3Nkz9Ufpk2AcUM=
github.com/gabriel-vasile/mimetype v1.4.13/go.mod h1:d+9Oxyo1wTzWdyVUPMmXFvp4F9tea18J8ufA774AB3s=
github.com/fsnotify/fsnotify v1.10.1 h1:b0/UzAf9yR5rhf3RPm9gf3ehBPpf0oZKIjtpKrx59Ho=
github.com/fsnotify/fsnotify v1.10.1/go.mod h1:TLheqan6HD6GBK6PrDWyDPBaEV8LspOxvPSjC+bVfgo=
github.com/gabriel-vasile/mimetype v1.4.15 h1:05iP/CYtZ/w455R/KZM6rZ5ieAdh99UPtd+d3YzLmaI=
github.com/gabriel-vasile/mimetype v1.4.15/go.mod h1:azpTcoLcDZRNgFou5j+APrqQx9HqVPWa6ijYQIIVswQ=
github.com/ghodss/yaml v1.0.0 h1:wQHKEahhL6wmXdzwWG11gIVCkOv05bNOh+Rxn0yngAk=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/glebarez/go-sqlite v1.22.0 h1:uAcMJhaA6r3LHMTFgP0SifzgXg46yJkgxqyuyec+ruQ=
github.com/glebarez/go-sqlite v1.22.0/go.mod h1:PlBIdHe0+aUEFn+r2/uthrWq4FxbzugL0L8Li6yQJbc=
github.com/glebarez/go-sqlite v1.23.0 h1:FyhIq4jqmgphQAUlY79zPldYGwISEZikaDfhiGWkkaI=
github.com/glebarez/go-sqlite v1.23.0/go.mod h1:IIYrOH3L0rHY3jb4IXOHoWdklNajSGUN2eJcvK8WrnI=
github.com/glebarez/sqlite v1.11.0 h1:wSG0irqzP6VurnMEpFGer5Li19RpIRi2qvQz++w0GMw=
github.com/glebarez/sqlite v1.11.0/go.mod h1:h8/o8j5wiAsqSPoWELDUdJXhjAhsVliSn7bWZjOhrgQ=
github.com/go-faster/errors v0.7.1 h1:MkJTnDoEdi9pDabt1dpWf7AA8/BaSYZqibYyhZ20AYg=
github.com/go-faster/errors v0.7.1/go.mod h1:5ySTjWFiphBs07IKuiL69nxdfd5+fzh1u7FPGZP2quo=
github.com/go-faster/errors v0.8.0 h1:9T9eJrM+72dFk7n4DfhuaDDe6cyuFCSW2oNUkN77Yqc=
github.com/go-faster/errors v0.8.0/go.mod h1:5ySTjWFiphBs07IKuiL69nxdfd5+fzh1u7FPGZP2quo=
github.com/go-faster/jx v1.2.0 h1:T2YHJPrFaYu21fJtUxC9GzmluKu8rVIFDwwGBKTDseI=
github.com/go-faster/jx v1.2.0/go.mod h1:UWLOVDmMG597a5tBFPLIWJdUxz5/2emOpfsj9Neg0PE=
github.com/go-faster/xor v0.3.0/go.mod h1:x5CaDY9UKErKzqfRfFZdfu+OSTfoZny3w5Ak7UxcipQ=
github.com/go-faster/xor v1.0.0 h1:2o8vTOgErSGHP3/7XwA5ib1FTtUsNtwCoLLBjl31X38=
github.com/go-faster/xor v1.0.0/go.mod h1:x5CaDY9UKErKzqfRfFZdfu+OSTfoZny3w5Ak7UxcipQ=
github.com/go-faster/yaml v0.4.6 h1:lOK/EhI04gCpPgPhgt0bChS6bvw7G3WwI8xxVe0sw9I=
github.com/go-faster/yaml v0.4.6/go.mod h1:390dRIvV4zbnO7qC9FGo6YYutc+wyyUSHBgbXL52eXk=
github.com/go-ini/ini v1.67.0 h1:z6ZrTEZqSWOTyH2FlglNbNgARyHG8oLW9gMELqKr06A=
github.com/go-ini/ini v1.67.0/go.mod h1:ByCAeIL28uOIIG0E3PJtZPDL8WnHpFKFOtgjp+3Ies8=
github.com/go-jose/go-jose/v3 v3.0.5 h1:BLLJWbC4nMZOfuPVxoZIxeYsn6Nl2r1fITaJ78UQlVQ=
github.com/go-jose/go-jose/v3 v3.0.5/go.mod h1:5b+7YgP7ZICgJDBdfjZaIt+H/9L9T/YQrVfLAMboGkQ=
github.com/go-logfmt/logfmt v0.6.1 h1:4hvbpePJKnIzH1B+8OR/JPbTx37NktoI9LE2QZBBkvE=
github.com/go-logfmt/logfmt v0.6.1/go.mod h1:EV2pOAQoZaT1ZXZbqDl5hrymndi4SY9ED9/z6CO0XAk=
github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/logr v1.4.4 h1:tG4xh9yMsRCAiodLVTxyrkzSZ9+o0L1Kg/+cPVcbP/8=
github.com/go-logr/logr v1.4.4/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/go-sourcemap/sourcemap v2.1.4+incompatible h1:a+iTbH5auLKxaNwQFg0B+TCYl6lbukKPc7b5x0n1s6Q=
@@ -141,18 +140,18 @@ github.com/google/go-github/v30 v30.1.0/go.mod h1:n8jBpHl45a/rlBUtRJMOG4GhNADUQF
github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=
github.com/google/go-querystring v1.2.0 h1:yhqkPbu2/OH+V9BfpCVPZkNmUXhb2gBxJArfhIxNtP0=
github.com/google/go-querystring v1.2.0/go.mod h1:8IFJqpSRITyJ8QhQ13bmbeMBDfmeEJZD5A0egEOmkqU=
github.com/google/pprof v0.0.0-20260402051712-545e8a4df936 h1:EwtI+Al+DeppwYX2oXJCETMO23COyaKGP6fHVpkpWpg=
github.com/google/pprof v0.0.0-20260402051712-545e8a4df936/go.mod h1:MxpfABSjhmINe3F1It9d+8exIHFvUqtLIRCdOGNXqiI=
github.com/google/pprof v0.0.0-20260802141513-ef3492d7dac3 h1:LMLX+LgTNWpfvCBdFebv6EsYotImrt/Ppc5cXIriCSo=
github.com/google/pprof v0.0.0-20260802141513-ef3492d7dac3/go.mod h1:jl5iWTm0/hd5PjEYEOuwAJ57L/CibdZfrqZ5XA5GrCk=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gotd/contrib v0.21.1 h1:NSF+0YEnosQ34QEo2o4s6MA5YFDAor1LVvLhN1L3H1M=
github.com/gotd/contrib v0.21.1/go.mod h1:trVJBP9Q/TJbjmJbVnLc0cnX/8T4N0RpQBULVa3BNnE=
github.com/gotd/ige v0.2.2 h1:XQ9dJZwBfDnOGSTxKXBGP4gMud3Qku2ekScRjDWWfEk=
github.com/gotd/ige v0.2.2/go.mod h1:tuCRb+Y5Y3eNTo3ypIfNpQ4MFjrnONiL2jN2AKZXmb0=
github.com/gotd/ige v0.3.0 h1:4f6LEHWsVDLBG0bT9wWG2/9TZb5aWm265G8ZlTXmRRU=
github.com/gotd/ige v0.3.0/go.mod h1:FE9bTaQtvfArizAcZuI4sS6gXaEUBmixdUufVHoCKac=
github.com/gotd/neo v0.1.5 h1:oj0iQfMbGClP8xI59x7fE/uHoTJD7NZH9oV1WNuPukQ=
github.com/gotd/neo v0.1.5/go.mod h1:9A2a4bn9zL6FADufBdt7tZt+WMhvZoc5gWXihOPoiBQ=
github.com/gotd/td v0.143.0 h1:p0U/Nn92zXmAsahDn5CIVzay2kQ36lBBENT/FlWR2nQ=
github.com/gotd/td v0.143.0/go.mod h1:8GA5ecTI5iswLwBAlqf0u6/+j+BqSWUARSrX2Xk1usQ=
github.com/gotd/td v0.149.0 h1:vXzNO99FFzWKKyI0vizvtgkpI7AqmbcXHLWjxkD+69o=
github.com/gotd/td v0.149.0/go.mod h1:+s0fRWlKL+RqoKJAMnCRoj0sWtxzRlWm886elNZEpJQ=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/inconshreveable/go-update v0.0.0-20160112193335-8152e7eb6ccf h1:WfD7VjIE6z8dIvMsI4/s+1qr5EL+zoIGev1BQj1eoJ8=
@@ -165,11 +164,11 @@ github.com/jinzhu/now v1.1.5 h1:/o9tlHleP7gOFmsnYNz3RGnqzefHA47wQpKrrdTIwXQ=
github.com/jinzhu/now v1.1.5/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8=
github.com/johannesboyne/gofakes3 v0.0.0-20250916175020-ebf3e50324d3 h1:2713fQZ560HxoNVgfJH41GKzjMjIG+DW4hH6nYXfXW8=
github.com/johannesboyne/gofakes3 v0.0.0-20250916175020-ebf3e50324d3/go.mod h1:S4S9jGBVlLri0OeqrSSbCGG5vsI6he06UJyuz1WT1EE=
github.com/klauspost/compress v1.18.5 h1:/h1gH5Ce+VWNLSWqPzOVn6XBO+vJbCNGvjoaGBFW2IE=
github.com/klauspost/compress v1.18.5/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/klauspost/compress v1.19.2 h1:hMRETovs/pu/dVWN7zIT1PGG8t509MwT6bO7XSi26R8=
github.com/klauspost/compress v1.19.2/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/klauspost/cpuid/v2 v2.0.1/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/klauspost/cpuid/v2 v2.4.0 h1:S6Hrbc7+ywsr0r+RLapfGBHfyefhCTwEh3A0tV913Dw=
github.com/klauspost/cpuid/v2 v2.4.0/go.mod h1:19jmZ9mjzoF//ddRSUsv0zfBTJWh3QJh9FNxZTMrGxU=
github.com/klauspost/crc32 v1.3.0 h1:sSmTt3gUt81RP655XGZPElI0PelVTZ6YwCRnPSupoFM=
github.com/klauspost/crc32 v1.3.0/go.mod h1:D7kQaZhnkX/Y0tstFGf8VUzv2UofNGqCjnC3zdHB0Hw=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
@@ -180,22 +179,24 @@ github.com/krau/ffmpeg-go v0.6.0 h1:F4HWvOrKXQsfLsFTOnUfP0HY6WISJqOrsAFGSIzkKto=
github.com/krau/ffmpeg-go v0.6.0/go.mod h1:sa7/bWHB6fO9j4lhmxnWQ1U07o+dE1leFjhctotxU7A=
github.com/lrstanley/go-ytdlp v1.3.5 h1:eT+29mK3Lp+XPMQOH25+jVerrrjifYW1o3IkTYJ9SMs=
github.com/lrstanley/go-ytdlp v1.3.5/go.mod h1:VgjnTrvkTf+23JuySjyPq1iQ8ijSovBtTPpXH5XrLtI=
github.com/lucasb-eyer/go-colorful v1.4.0 h1:UtrWVfLdarDgc44HcS7pYloGHJUjHV/4FwW4TvVgFr4=
github.com/lucasb-eyer/go-colorful v1.4.0/go.mod h1:R4dSotOR9KMtayYi1e77YzuveK+i7ruzyGqttikkLy0=
github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
github.com/mattn/go-isatty v0.0.21 h1:xYae+lCNBP7QuW4PUnNG61ffM4hVIfm+zUzDuSzYLGs=
github.com/mattn/go-isatty v0.0.21/go.mod h1:ZXfXG4SQHsB/w3ZeOYbR0PrPwLy+n6xiMrJlRFqopa4=
github.com/lucasb-eyer/go-colorful v1.4.1 h1:1EO+WB73+EH8EVbzlrG3KLAfEypQWVHIBqlTf+2hNss=
github.com/lucasb-eyer/go-colorful v1.4.1/go.mod h1:R4dSotOR9KMtayYi1e77YzuveK+i7ruzyGqttikkLy0=
github.com/mattn/go-colorable v0.1.15 h1:+u9SLTRGnXv73cEsnsmoZBom+dMU88B2M0aDcWy0/jY=
github.com/mattn/go-colorable v0.1.15/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI=
github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A=
github.com/mattn/go-localereader v0.0.1 h1:ygSAOl7ZXTx4RdPYinUpg6W99U8jWvWi9Ye2JC/oIi4=
github.com/mattn/go-localereader v0.0.1/go.mod h1:8fBrzywKY7BI3czFoHkuzRoWE9C+EiG4R1k4Cjx5p88=
github.com/mattn/go-runewidth v0.0.23 h1:7ykA0T0jkPpzSvMS5i9uoNn2Xy3R383f9HDx3RybWcw=
github.com/mattn/go-runewidth v0.0.23/go.mod h1:XBkDxAl56ILZc9knddidhrOlY5R/pDhgLpndooCuJAs=
github.com/mattn/go-runewidth v0.0.27 h1:Feg/Oou5zI/wnpgDF6omIU0OokC9GxLC/WRknhVlIR0=
github.com/mattn/go-runewidth v0.0.27/go.mod h1:3qAiGCV4Koz/yuveO58qUefmUTRm8r0IGEXZ9jeHp/8=
github.com/mattn/go-sqlite3 v1.14.22 h1:2gZY6PC6kBnID23Tichd1K+Z0oS6nE/XwU+Vz/5o4kU=
github.com/mattn/go-sqlite3 v1.14.22/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
github.com/minio/crc64nvme v1.1.1 h1:8dwx/Pz49suywbO+auHCBpCtlW1OfpcLN7wYgVR6wAI=
github.com/minio/crc64nvme v1.1.1/go.mod h1:eVfm2fAzLlxMdUGc0EEBGSMmPwmXD5XiNRpnu9J3bvg=
github.com/minio/md5-simd v1.1.2 h1:Gdi1DZK69+ZVMoNHRXJyNcxrMA4dSxoYHZSQbirFg34=
github.com/minio/md5-simd v1.1.2/go.mod h1:MzdKDxYpY2BT9XQFocsiZf/NKVtR7nkE4RoEpN+20RM=
github.com/minio/minio-go/v7 v7.0.100 h1:ShkWi8Tyj9RtU57OQB2HIXKz4bFgtVib0bbT1sbtLI8=
github.com/minio/minio-go/v7 v7.0.100/go.mod h1:EtGNKtlX20iL2yaYnxEigaIvj0G0GwSDnifnG8ClIdw=
github.com/minio/minio-go/v7 v7.2.1 h1:PfBfwvKB/MmqyN8Vb1G9voWisaM9OrLv+WwOvMwS9Dw=
github.com/minio/minio-go/v7 v7.2.1/go.mod h1:EU9hENAStx/xXduNdrGO5e4X5vk19NtgB+RIPjZO8o0=
github.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY=
github.com/mitchellh/mapstructure v1.5.0/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=
github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 h1:ZK8zHtRHOkbHy6Mmr5D264iyp3TiX5OmNcI5cIARiQI=
@@ -204,36 +205,36 @@ github.com/muesli/cancelreader v0.2.2 h1:3I4Kt4BQjOR54NavqnDogx/MIoWBFa0StPA8ELU
github.com/muesli/cancelreader v0.2.2/go.mod h1:3XuTXfFS2VjM+HTLZY9Ak0l6eUKfijIfMUZ4EgX0QYo=
github.com/muesli/termenv v0.16.0 h1:S5AlUN9dENB57rsbnkPyfdGuWIlkmzJjbFf0Tf5FWUc=
github.com/muesli/termenv v0.16.0/go.mod h1:ZRfOIKPFDYQoDFF4Olj7/QJbW60Ol/kL1pU3VfY/Cnk=
github.com/ncruces/go-sqlite3 v0.33.3 h1:6jCR3KuGvJSEwhaQrkHDGeIe2qCQ6nOUDNsPz7ZIotw=
github.com/ncruces/go-sqlite3 v0.33.3/go.mod h1:t2Osfw0wcKzJTgv2EvrkTtVLqlbKTA5Yvwb2ypAlBcY=
github.com/ncruces/go-sqlite3-wasm v1.1.1-0.20260409221933-87e4b35a38d0 h1:ymE9H30x1AyW5VfMNkJC9teuI2W1jjMsQS7kc6zl6Tg=
github.com/ncruces/go-sqlite3-wasm v1.1.1-0.20260409221933-87e4b35a38d0/go.mod h1:/H3+JykPsfSlvKbOxNSx9kKwm3ecqQGzyCs1e9KkNsU=
github.com/ncruces/go-sqlite3/gormlite v0.33.3 h1:JzLk8XymgvHvy60ib5MtNmd0fIYwGi7FUj2DpRFmnWQ=
github.com/ncruces/go-sqlite3/gormlite v0.33.3/go.mod h1:qDjzlaffXDGg5bhZs2VaaSY0Qb3rsiKq0O4pXkmQfHI=
github.com/ncruces/go-sqlite3 v0.35.3 h1:Ei07Zv1qfV/vyXzelhFsyS5Oh9TArBZHsmFk14Xv3GY=
github.com/ncruces/go-sqlite3 v0.35.3/go.mod h1:i1rhym/NIiB5xeEfzbN+e24Y+i7NGUpf7C2xZ3Dpwks=
github.com/ncruces/go-sqlite3-wasm/v3 v3.2.35304 h1:5NoQAewtgKNK3G4bjNPxVoGXu6F6NzLXWCTdD5FFAEY=
github.com/ncruces/go-sqlite3-wasm/v3 v3.2.35304/go.mod h1:o8gr9w/50fXA5TDskg6bNUjvqmFfw4KaXth4q+yDSjg=
github.com/ncruces/go-sqlite3/gormlite v0.34.0 h1:QLlOy/i7OabsFUQ+d5KyXmq2hw9sMh/CRW435+eQMRY=
github.com/ncruces/go-sqlite3/gormlite v0.34.0/go.mod h1:CMv+6YhqLmPBXYACiQtrWA0q/JLIMTKB4E65SUfLgF0=
github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/ncruces/julianday v1.0.0 h1:fH0OKwa7NWvniGQtxdJRxAgkBMolni2BjDHaWTxqt7M=
github.com/ncruces/julianday v1.0.0/go.mod h1:Dusn2KvZrrovOMJuOt0TNXL6tB7U2E8kvza5fFc9G7g=
github.com/nicksnyder/go-i18n/v2 v2.6.1 h1:JDEJraFsQE17Dut9HFDHzCoAWGEQJom5s0TRd17NIEQ=
github.com/nicksnyder/go-i18n/v2 v2.6.1/go.mod h1:Vee0/9RD3Quc/NmwEjzzD7VTZ+Ir7QbXocrkhOzmUKA=
github.com/ogen-go/ogen v1.20.3 h1:1tvJuJE0BnQ7Nukd6ykiTOP0ucfL0yrAjHUg3S1DCQk=
github.com/ogen-go/ogen v1.20.3/go.mod h1:sJ1pJVp4S1RcSZlYIiMLo0QSMSt2pls4zfrc+hNKnzk=
github.com/pelletier/go-toml/v2 v2.3.0 h1:k59bC/lIZREW0/iVaQR8nDHxVq8OVlIzYCOJf421CaM=
github.com/pelletier/go-toml/v2 v2.3.0/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/ogen-go/ogen v1.24.0 h1:NehBG/8s0JeM6jYHPJeFJntBG3cE/xQgZd1q8BchPNM=
github.com/ogen-go/ogen v1.24.0/go.mod h1:hcg4aTzLcu/MS1SGGahUriPBEOZB3c82RGIC6dYcdFU=
github.com/pelletier/go-toml/v2 v2.4.3 h1:GTRvJQutkOSftxIFD5xw9aepkYNuPWmVJpffdDPYVpY=
github.com/pelletier/go-toml/v2 v2.4.3/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/philhofer/fwd v1.2.0 h1:e6DnBTl7vGY+Gz322/ASL4Gyp1FspeMvx1RNDoToZuM=
github.com/philhofer/fwd v1.2.0/go.mod h1:RqIHx9QI14HlwKwm98g9Re5prTQ6LdeRQn+gXJFxsJM=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/playwright-community/playwright-go v0.5700.1 h1:PNFb1byWqrTT720rEO0JL88C6Ju0EmUnR5deFLvtP/U=
github.com/playwright-community/playwright-go v0.5700.1/go.mod h1:MlSn1dZrx8rszbCxY6x3qK89ZesJUYVx21B2JnkoNF0=
github.com/playwright-community/playwright-go v0.6000.0 h1:R7sENcRI6n0Zd5ZoW8EKdVF1ZVJgvTubfJeqKHNGvsw=
github.com/playwright-community/playwright-go v0.6000.0/go.mod h1:z/YpFVdU4LAi+0f9VPOCkGvmdH6dCrtza9nxnXFXgiE=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
github.com/rogpeppe/go-internal v1.15.0 h1:D0RCU5rMAp+SpgkiNdrjfJ+LX4J1M32V2NeCY7EJ6hc=
github.com/rogpeppe/go-internal v1.15.0/go.mod h1:DrUVZyrJU+txYW5/1kwtXQSMFio52ZOxX7yM1VHvnxs=
github.com/rs/xid v1.6.0 h1:fV591PaemRlL6JfRxGDEPl69wICngIQ3shQtzfy2gxU=
github.com/rs/xid v1.6.0/go.mod h1:7XoLgs4eV+QndskICGsho+ADou8ySMSjJKDIan90Nz0=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
@@ -257,33 +258,47 @@ github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3A
github.com/spf13/viper v1.21.0 h1:x5S+0EU27Lbphp4UKm1C+1oQO+rKx36vfCoaVebLFSU=
github.com/spf13/viper v1.21.0/go.mod h1:P0lhsswPGWD/1lZJ9ny3fYnVqxiegrlNrEmgLjbTCAY=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=
github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU=
github.com/tinylib/msgp v1.6.3 h1:bCSxiTz386UTgyT1i0MSCvdbWjVW+8sG3PjkGsZQt4s=
github.com/tinylib/msgp v1.6.3/go.mod h1:RSp0LW9oSxFut3KzESt5Voq4GVWyS+PSulT77roAqEA=
github.com/ulikunitz/xz v0.5.15 h1:9DNdB5s+SgV3bQ2ApL10xRc35ck0DuIX/isZvIk+ubY=
github.com/ulikunitz/xz v0.5.15/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14=
github.com/tinylib/msgp v1.6.4 h1:mOwYbyYDLPj35mkA2BjjYejgJk9BuHxDdvRnb6v2ZcQ=
github.com/tinylib/msgp v1.6.4/go.mod h1:RSp0LW9oSxFut3KzESt5Voq4GVWyS+PSulT77roAqEA=
github.com/ulikunitz/xz v0.5.16 h1:ld6NyySjx5lowVKwJvMRLnW5nxKX/xnpSiFYZ/Lxur0=
github.com/ulikunitz/xz v0.5.16/go.mod h1:H9Rt/W6/Qj27PGauhQc6nfCDy7vHpzsOThBSaYDoEhw=
github.com/unvgo/ghselfupdate v1.0.1 h1:4clbOkfPbfEmRnnYxVXDSBs0JG12DO+0FfqplJckreU=
github.com/unvgo/ghselfupdate v1.0.1/go.mod h1:3snWV5vEHGXQqqhY7FwKjPOtH6e7cFdHYN7UMAihhxs=
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e h1:JVG44RsyaB9T2KIHavMF/ppJZNG9ZpyihvCd0w101no=
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e/go.mod h1:RbqR21r5mrJuqunuUZ/Dhy/avygyECGrLceyNeo4LiM=
github.com/xo/terminfo v1.0.0 h1:2ZpYzqWzyyytjk3TP6aJVDhkMAkc99/1xKQdA3TDTBY=
github.com/xo/terminfo v1.0.0/go.mod h1:RbqR21r5mrJuqunuUZ/Dhy/avygyECGrLceyNeo4LiM=
github.com/yapingcat/gomedia v0.0.0-20240906162731-17feea57090c h1:xA2TJS9Hu/ivzaZIrDcwvpJ3Fnpsk5fDOJ4iSnL6J0w=
github.com/yapingcat/gomedia v0.0.0-20240906162731-17feea57090c/go.mod h1:WSZ59bidJOO40JSJmLqlkBJrjZCtjbKKkygEMfzY/kc=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
github.com/yuin/goldmark v1.8.5 h1:r6N5afV5qj/5S4UTch8agZHJ8UxNCMwX7WjkkJam2NA=
github.com/yuin/goldmark v1.8.5/go.mod h1:ip/1k0VRfGynBgxOz0yCqHrbZXhcjxyuS66Brc7iBKg=
github.com/zeebo/assert v1.3.0 h1:g7C04CbJuIDKNPFHmsk4hwZDO5O+kntRxzaUoNXj+IQ=
github.com/zeebo/assert v1.3.0/go.mod h1:Pq9JiuJQpG8JLJdtkwrJESF0Foym2/D9XMU5ciN/wJ0=
github.com/zeebo/xxh3 v1.1.0 h1:s7DLGDK45Dyfg7++yxI0khrfwq9661w9EN78eP/UZVs=
github.com/zeebo/xxh3 v1.1.0/go.mod h1:IisAie1LELR4xhVinxWS5+zf1lA4p0MW4T+w+W07F5s=
go.etcd.io/bbolt v1.4.3 h1:dEadXpI6G79deX5prL3QRNP6JB8UxVkqo4UPnHaNXJo=
go.etcd.io/bbolt v1.4.3/go.mod h1:tKQlpPaYCVFctUIgFKFnAlvbmB3tpy1vkTnDWohtc0E=
go.mongodb.org/mongo-driver v1.17.9 h1:IexDdCuuNJ3BHrELgBlyaH9p60JXAvdzWR128q+U5tU=
go.mongodb.org/mongo-driver v1.17.9/go.mod h1:LlOhpH5NUEfhxcAwG0UEkMqwYcc4JU18gtCdGudk/tQ=
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
go.opentelemetry.io/otel v1.43.0 h1:mYIM03dnh5zfN7HautFE4ieIig9amkNANT+xcVxAj9I=
go.opentelemetry.io/otel v1.43.0/go.mod h1:JuG+u74mvjvcm8vj8pI5XiHy1zDeoCS2LB1spIq7Ay0=
go.opentelemetry.io/otel/metric v1.43.0 h1:d7638QeInOnuwOONPp4JAOGfbCEpYb+K6DVWvdxGzgM=
go.opentelemetry.io/otel/metric v1.43.0/go.mod h1:RDnPtIxvqlgO8GRW18W6Z/4P462ldprJtfxHxyKd2PY=
go.opentelemetry.io/otel/trace v1.43.0 h1:BkNrHpup+4k4w+ZZ86CZoHHEkohws8AY+WTX09nk+3A=
go.opentelemetry.io/otel/trace v1.43.0/go.mod h1:/QJhyVBUUswCphDVxq+8mld+AvhXZLhe+8WVFxiFff0=
go.opentelemetry.io/otel v1.45.0 h1:pdrWmLHofpubmArBv1LgFSv1Z0Ie/ppdZzu+kUN5EeU=
go.opentelemetry.io/otel v1.45.0/go.mod h1:XZxIqPapzEYnhNSScF5DIqXhm/rYi0FzCe2XddAwZfQ=
go.opentelemetry.io/otel/metric v1.45.0 h1:7Eg1uH7CJ5cXv9is6tnBe1FI6rj1nwUdbFypRm3br/M=
go.opentelemetry.io/otel/metric v1.45.0/go.mod h1:HAPbm1nd3p1PmFH7v2dR+6BjXxw+Lq4a2+pndMAm08s=
go.opentelemetry.io/otel/trace v1.45.0 h1:l/mP6Uv7oNO7/TblbhpbgMidxhq1uO/rPsikOyVhxag=
go.opentelemetry.io/otel/trace v1.45.0/go.mod h1:qoJJA2xNMnxRrdISU/kLtfUH2wNeQbiv+jhs/CxI8bc=
go.shabbyrobe.org/gocovmerge v0.0.0-20230507111327-fa4f82cfbf4d h1:Ns9kd1Rwzw7t0BR8XMphenji4SmIoNZPn8zhYmaVKP8=
go.shabbyrobe.org/gocovmerge v0.0.0-20230507111327-fa4f82cfbf4d/go.mod h1:92Uoe3l++MlthCm+koNi0tcUCX3anayogF0Pa/sp24k=
go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE=
@@ -292,36 +307,37 @@ go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go.uber.org/zap v1.27.1 h1:08RqriUEv8+ArZRYSTXy1LeBScaMpVSTBhCeaZYfMYc=
go.uber.org/zap v1.27.1/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=
go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
go.uber.org/zap v1.28.0 h1:IZzaP1Fv73/T/pBMLk4VutPl36uNC+OSUh3JLG3FIjo=
go.uber.org/zap v1.28.0/go.mod h1:rDLpOi171uODNm/mxFcuYWxDsqWSAVkFdX4XojSKg/Q=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw=
go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU=
golang.org/x/crypto v0.50.0 h1:zO47/JPrL6vsNkINmLoo/PH1gcxpls50DNogFvB5ZGI=
golang.org/x/crypto v0.50.0/go.mod h1:3muZ7vA7PBCE6xgPX7nkzzjiUq87kRItoJQM1Yo8S+Q=
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f h1:W3F4c+6OLc6H2lb//N1q4WpJkhzJCK5J6kUi1NTVXfM=
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f/go.mod h1:J1xhfL/vlindoeF/aINzNzt2Bket5bjo9sdOYzOsU80=
golang.org/x/crypto v0.54.0 h1:YLIA59K4fiNzHzjnZt2tUJQjQtUWfWbeHBqKtk3eScw=
golang.org/x/crypto v0.54.0/go.mod h1:KWL8ny2AZdGR2cWmzeHrp2azQPGogOv+HeQaVEXC2dk=
golang.org/x/exp v0.0.0-20260810151157-a8b543ca52da h1:YKw1FZDyWyXXZcBxalRHz3CHieUKnKxanrbcO280Zwc=
golang.org/x/exp v0.0.0-20260810151157-a8b543ca52da/go.mod h1:EdfpwwqSu+0Li0mzskwHU6FWDV3t9Q+RZDo3QMUtL3Q=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.35.0 h1:Ww1D637e6Pg+Zb2KrWfHQUnH2dQRLBQyAtpr/haaJeM=
golang.org/x/mod v0.35.0/go.mod h1:+GwiRhIInF8wPm+4AoT6L0FA1QWAad3OMdTRx4tFYlU=
golang.org/x/mod v0.39.0 h1:UF5zwQdCRRUpHfyPwr7d4UrGiVeldIsogtzWVnczL74=
golang.org/x/mod v0.39.0/go.mod h1:bvIbwjQ0HUFFf5AKukeeYQG4ZBUG9yxQbR9aEweIwYY=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
golang.org/x/net v0.53.0 h1:d+qAbo5L0orcWAr0a9JweQpjXF19LMXJE8Ey7hwOdUA=
golang.org/x/net v0.53.0/go.mod h1:JvMuJH7rrdiCfbeHoo3fCQU24Lf5JJwT9W3sJFulfgs=
golang.org/x/net v0.57.0 h1:K5+3DljvIuDG9/Jv9rvyMywYNFCQ9RSUY6OOTTkT+tE=
golang.org/x/net v0.57.0/go.mod h1:KpXc8iv+r3XplLAG/f7Jsf9RPszJzdR0f58q9vGOuEU=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek=
golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
@@ -331,36 +347,38 @@ golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI=
golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.8.0/go.mod h1:xPskH00ivmX89bAKVGSKKtLOWNx2+17Eiy94tnKShWo=
golang.org/x/term v0.17.0/go.mod h1:lLRBjIVuehSbZlaOtGMbcMncT+aqLLLmKrsjNrUguwk=
golang.org/x/term v0.42.0 h1:UiKe+zDFmJobeJ5ggPwOshJIVt6/Ft0rcfrXZDLWAWY=
golang.org/x/term v0.42.0/go.mod h1:Dq/D+snpsbazcBG5+F9Q1n2rXV8Ma+71xEjTRufARgY=
golang.org/x/term v0.45.0 h1:NwWyBmoJCbfTHpxrWoZ9C6/VxOf7ic219I8xZZFdrf0=
golang.org/x/term v0.45.0/go.mod h1:9aqxs0blBcrm/n0L9QW0aRVD+ktan8ssZromtqJC43w=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.36.0 h1:JfKh3XmcRPqZPKevfXVpI1wXPTqbkE5f7JA92a55Yxg=
golang.org/x/text v0.36.0/go.mod h1:NIdBknypM8iqVmPiuco0Dh6P5Jcdk8lJL0CUebqK164=
golang.org/x/text v0.40.0 h1:Ub2Z6/xjgF1WrYQz2nuITOEegKFtiIy+rieRJ5lHZKs=
golang.org/x/text v0.40.0/go.mod h1:hpnzDAfGV753zIKo+wk3u1bVKCGPbrnF7+7LBF/UHVY=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/tools v0.44.0 h1:UP4ajHPIcuMjT1GqzDWRlalUEoY+uzoZKnhOjbIPD2c=
golang.org/x/tools v0.44.0/go.mod h1:KA0AfVErSdxRZIsOVipbv3rQhVXTnlU6UhKxHd1seDI=
golang.org/x/tools v0.48.0 h1:3+hClM1aLL5mjMKm5ovokw9epgRXPuu2tILgismM6RE=
golang.org/x/tools v0.48.0/go.mod h1:08xX0orndb/F7jJxGDicx061tyd5pcMto75YMAXr6lk=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/ini.v1 v1.67.3 h1:iM9Lhz5MRSGhHVGGwCuzG9KO8PoirCXj/m/qTmOJJQw=
gopkg.in/ini.v1 v1.67.3/go.mod h1:x/cyOwCgZqOkJoDIJ3c1KNHMo10+nLGAhh+kn3Zizss=
gopkg.in/mgo.v2 v2.0.0-20180705113604-9856a29383ce h1:xcEWjVhvbDy+nHP67nPDDpbYrY+ILlfndk4bRioVHaU=
gopkg.in/mgo.v2 v2.0.0-20180705113604-9856a29383ce/go.mod h1:yeKp02qBN3iKW1OzL3MGk2IdtZzaj7SFntXj72NppTA=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
@@ -369,32 +387,34 @@ gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gorm.io/gorm v1.31.1 h1:7CA8FTFz/gRfgqgpeKIBcervUn3xSyPUmr6B2WXJ7kg=
gorm.io/gorm v1.31.1/go.mod h1:XyQVbO2k6YkOis7C2437jSit3SsDK72s7n7rsSHd+Gs=
modernc.org/cc/v4 v4.27.3 h1:uNCgn37E5U09mTv1XgskEVUJ8ADKpmFMPxzGJ0TSo+U=
modernc.org/cc/v4 v4.27.3/go.mod h1:3YjcbCqhoTTHPycJDRl2WZKKFj0nwcOIPBfEZK0Hdk8=
modernc.org/ccgo/v4 v4.32.4 h1:L5OB8rpEX4ZsXEQwGozRfJyJSFHbbNVOoQ59DU9/KuU=
modernc.org/ccgo/v4 v4.32.4/go.mod h1:lY7f+fiTDHfcv6YlRgSkxYfhs+UvOEEzj49jAn2TOx0=
gorm.io/driver/sqlite v1.6.0 h1:WHRRrIiulaPiPFmDcod6prc4l2VGVWHz80KspNsxSfQ=
gorm.io/driver/sqlite v1.6.0/go.mod h1:AO9V1qIQddBESngQUKWL9yoH93HIeA1X6V633rBwyT8=
gorm.io/gorm v1.31.2 h1:3o8FXNo9v9S858gil+3LlZA1LkCOzgb4g5BL64FgaCo=
gorm.io/gorm v1.31.2/go.mod h1:XyQVbO2k6YkOis7C2437jSit3SsDK72s7n7rsSHd+Gs=
modernc.org/cc/v4 v4.29.2 h1:h6+9ciCnPKutf4I03CvheAvDLX7+IHlqR6Iy6J+cgd8=
modernc.org/cc/v4 v4.29.2/go.mod h1:OnovgIhbbMXMu1aISnJ0wvVD1KnW+cAUJkIrAWh+kVI=
modernc.org/ccgo/v4 v4.35.0 h1:F+TUsmw09QxLzmi3aeYYGxjAXarmZaKgj3mKQHNaA8w=
modernc.org/ccgo/v4 v4.35.0/go.mod h1:qrVGs9S3Sr2Ztcg9ve+kTAYMp5a3YvWjo+SoN06kJ5I=
modernc.org/fileutil v1.4.0 h1:j6ZzNTftVS054gi281TyLjHPp6CPHr2KCxEXjEbD6SM=
modernc.org/fileutil v1.4.0/go.mod h1:EqdKFDxiByqxLk8ozOxObDSfcVOv/54xDs/DUHdvCUU=
modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI=
modernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito=
modernc.org/gc/v3 v3.1.2 h1:ZtDCnhonXSZexk/AYsegNRV1lJGgaNZJuKjJSWKyEqo=
modernc.org/gc/v3 v3.1.2/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY=
modernc.org/gc/v3 v3.1.5 h1:21ldfPfRYE31Tb7B3mwAK8gy1AxP4+dKjrOQPfqakoc=
modernc.org/gc/v3 v3.1.5/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY=
modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=
modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
modernc.org/libc v1.72.0 h1:IEu559v9a0XWjw0DPoVKtXpO2qt5NVLAnFaBbjq+n8c=
modernc.org/libc v1.72.0/go.mod h1:tTU8DL8A+XLVkEY3x5E/tO7s2Q/q42EtnNWda/L5QhQ=
modernc.org/libc v1.75.3 h1:vCqT5+R0jPXMnvMkGo0T2zXvFNth+lYXVCx5X7CCX/g=
modernc.org/libc v1.75.3/go.mod h1:MjAX68G+0oufI+hNuh0QXcK+Ap+sL8bNPPcIC6EqOfo=
modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
modernc.org/memory v1.11.0 h1:o4QC8aMQzmcwCK3t3Ux/ZHmwFPzE6hf2Y5LbkRs+hbI=
modernc.org/memory v1.11.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
modernc.org/opt v0.1.4 h1:2kNGMRiUjrp4LcaPuLY2PzUfqM/w9N23quVwhKt5Qm8=
modernc.org/opt v0.1.4/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
modernc.org/memory v1.12.0 h1:twkmYNkGXCvtYWzoux02jtK6eovjZbdI0uHFUYp6kuU=
modernc.org/memory v1.12.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
modernc.org/opt v0.2.0 h1:tGyef5ApycA7FSEOMraay9SaTk5zmbx7Tu+cJs4QKZg=
modernc.org/opt v0.2.0/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
modernc.org/sqlite v1.48.2 h1:5CnW4uP8joZtA0LedVqLbZV5GD7F/0x91AXeSyjoh5c=
modernc.org/sqlite v1.48.2/go.mod h1:hWjRO6Tj/5Ik8ieqxQybiEOUXy0NJFNp2tpvVpKlvig=
modernc.org/sqlite v1.56.0 h1:/D8e2RfFqoy/Zc6PuC76U28zFwmI/sYx1Kjm4yEn9e0=
modernc.org/sqlite v1.56.0/go.mod h1:yCJ2cmAaIkHQ25oXWrF8H4O1lIfPYPR26yCEDj2P3pQ=
modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=

Some files were not shown because too many files have changed in this diff Show More