mirror of
https://github.com/krau/SaveAny-Bot.git
synced 2026-08-27 03:10:06 +08:00
Compare commits
85 Commits
v0.46.2
...
feat/resum
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
03d749619d | ||
|
|
8bdafd115c | ||
|
|
991f454096 | ||
|
|
54dc4caafe | ||
|
|
784888f44c | ||
|
|
c86867ff8b | ||
|
|
7c4c7ef3c7 | ||
|
|
21a519fdbe | ||
|
|
1d431dbc88 | ||
|
|
bd1926d200 | ||
|
|
b0850fb5e5 | ||
|
|
75d83b0a3e | ||
|
|
c2f8ab3c01 | ||
|
|
6fd95bbe1b | ||
|
|
a1bd2ed762 | ||
|
|
7e3e26fcf4 | ||
|
|
e49ebef977 | ||
|
|
da38f5bb88 | ||
|
|
0e6ed66ef5 | ||
|
|
e4144e73e6 | ||
|
|
1d794a7b9c | ||
|
|
52f880f0f2 | ||
|
|
fc11ca775f | ||
|
|
056e2fd546 | ||
|
|
c9bb6c9e3c | ||
|
|
2bc460c609 | ||
|
|
f02860ff3f | ||
|
|
9c2e70ed43 | ||
|
|
3d6cd45909 | ||
|
|
77ef3154cf | ||
|
|
88b170acaa | ||
|
|
8059e27978 | ||
|
|
bfab4c85c8 | ||
|
|
62e4a08e28 | ||
|
|
0982abe7bc | ||
|
|
e74839b8e9 | ||
|
|
58ce8275b2 | ||
|
|
8f67f778a3 | ||
|
|
159dba6224 | ||
|
|
22d773da10 | ||
|
|
69ccfa664f | ||
|
|
38355dfd14 | ||
|
|
0940258b4d | ||
|
|
602fc251d8 | ||
|
|
af28738235 | ||
|
|
3eb3b6e3c8 | ||
|
|
f377ee3ca4 | ||
|
|
70f7172162 | ||
|
|
091f581881 | ||
|
|
8b86330f5c | ||
|
|
b431fa08e2 | ||
|
|
a02e8a8d90 | ||
|
|
4d2c345003 | ||
|
|
33a886fac9 | ||
|
|
57539ec3da | ||
|
|
82e1efb518 | ||
|
|
9b52a3e0ce | ||
|
|
6990543c9f | ||
|
|
dd0dea8cb5 | ||
|
|
3d20fbd0fe | ||
|
|
e6d8cc775a | ||
|
|
17e340fff1 | ||
|
|
f92c43b9c8 | ||
|
|
3e20dc2c5f | ||
|
|
3ce00884a0 | ||
|
|
cd7cf4964d | ||
|
|
bc3c841d1d | ||
|
|
743c15f1a5 | ||
|
|
b05d86509c | ||
|
|
f17a380579 | ||
|
|
cabe1189e2 | ||
|
|
a988d05e24 | ||
|
|
1af2c1f7c7 | ||
|
|
7b36fb45f5 | ||
|
|
62cceee592 | ||
|
|
6d315f7af2 | ||
|
|
5352491c76 | ||
|
|
3f914f7a64 | ||
|
|
8972d8a169 | ||
|
|
1339c69dbf | ||
|
|
63aeabb39b | ||
|
|
e60e983229 | ||
|
|
75e5fd10ea | ||
|
|
c8d8a2e0eb | ||
|
|
044e732084 |
6
.github/ISSUE_TEMPLATE/bug.yml
vendored
6
.github/ISSUE_TEMPLATE/bug.yml
vendored
@@ -5,6 +5,12 @@ labels:
|
||||
assignees:
|
||||
- krau
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
# Please Search Before Submitting / 提交前请搜索
|
||||
Please make sure to search existing issues before submitting a new bug report.
|
||||
提交新的 Bug 报告前请务必搜索已有的 issue,避免重复
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: "👾 Description"
|
||||
|
||||
66
.github/ISSUE_TEMPLATE/feature.yml
vendored
66
.github/ISSUE_TEMPLATE/feature.yml
vendored
@@ -8,7 +8,69 @@ body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
# Please describe the feature you want in detail
|
||||
Please describe the feature you want in detail.
|
||||
请详细描述你想要的功能。
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ IMPORTANT NOTICE / 说明
|
||||
|
||||
Save Any Bot supports multiple storage backends, **including Telegram**.
|
||||
However, **all backends are treated equally**, keep this in mind when submitting feature requests.
|
||||
|
||||
Save Any Bot 支持多种存储后端,**包括 Telegram**。
|
||||
但**所有后端在设计上是平等的**,请在提出功能请求前务必理解这一点。
|
||||
|
||||
### ❌ Out of scope requests / 不在项目范围内的请求
|
||||
The following requests are **out of scope** and will be closed without discussion:
|
||||
|
||||
以下请求**不属于本项目设计范围**,将被直接关闭,不再讨论:
|
||||
|
||||
- Adding **Telegram-specific behaviors or exceptions**
|
||||
添加 **仅针对 Telegram 的特殊行为或例外逻辑**
|
||||
- Treating Telegram as anything other than a **generic file storage backend**
|
||||
将 Telegram 视为非“通用文件存储后端”的特殊存在
|
||||
- Saving or syncing **non-file content** (text messages, chat history, etc.)
|
||||
保存或同步 **非文件内容**(文本消息、聊天记录等)
|
||||
- Preserving or reconstructing original messages (e.g. 1:1 forwarding)
|
||||
保留或还原原始消息形态(例如 1:1 转发)
|
||||
- Perform special reprocessing on files to adapt to specific storage backends
|
||||
(e.g. splitting, re-encoding, transforming, etc.)
|
||||
为适配特定存储后端而对文件进行特殊处理
|
||||
(如分割、转码、重编码、转换格式等)
|
||||
- Any request that requires different logic *only because the backend is Telegram*
|
||||
任何**仅因后端是 Telegram 而需要不同逻辑**的请求
|
||||
|
||||
### ❌ Abuse-leaning or high-risk requests / 滥用倾向的请求
|
||||
Requests that may **enable or encourage** the following will NOT be accepted:
|
||||
|
||||
可能**促成或鼓励**以下行为的请求将不会被接受:
|
||||
|
||||
- Violating Telegram Terms of Service
|
||||
违反 Telegram 服务条款
|
||||
- Building traffic, mirror, or profit-oriented channels using third-party content
|
||||
利用第三方内容构建引流、镜像或牟利用途的频道
|
||||
|
||||
### ⚖️ Design principle / 设计原则
|
||||
Save Any Bot follows a **backend-agnostic design**:
|
||||
|
||||
Save Any Bot 遵循 **后端无关(backend-agnostic)** 的设计原则:
|
||||
|
||||
- If a feature cannot be implemented **uniformly across all backends**, it will not be added.
|
||||
如果某个功能无法在 **所有后端** 中统一实现,则不会被添加。
|
||||
- No backend-specific hacks or special cases will be introduced.
|
||||
不会引入任何后端特有的 hack 或特殊处理逻辑。
|
||||
|
||||
---
|
||||
|
||||
If your request falls into any of the categories above, please do not open an issue.
|
||||
Such issues will be closed.
|
||||
|
||||
如果你的请求符合以上任一情况,请不要提交 issue,
|
||||
相关 issue 将被直接关闭。
|
||||
|
||||
Thank you for respecting the scope and design principles of this project.
|
||||
感谢你的理解与支持。
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: "⭐️ Feature description"
|
||||
@@ -30,4 +92,4 @@ body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
## Thank you for contributing to the project :slightly_smiling_face:
|
||||
## Thank you for contributing to the project :slightly_smiling_face:
|
||||
|
||||
86
.github/copilot-instructions.md
vendored
86
.github/copilot-instructions.md
vendored
@@ -1,86 +0,0 @@
|
||||
# SaveAny-Bot AI 协作说明
|
||||
|
||||
本项目是一个将 Telegram 文件/消息转存到多种存储端的 Bot,主要用 Go 实现,CLI 入口在 `cmd/`,核心逻辑分布在 `client/`、`core/`、`config/`、`database/`、`storage/` 等目录。下面是针对本仓库的专用约定,供 AI 编码助手参考。
|
||||
|
||||
## 总体架构与入口
|
||||
- **CLI 入口**:
|
||||
- 二进制入口:`main.go` 调用 `cmd.Execute(ctx)`。
|
||||
- 根命令:`cmd/root.go` 使用 `cobra` 定义 `saveany-bot`,`Run` 实现在 `cmd/run.go`。
|
||||
- **应用启动流程(非常重要)**:
|
||||
- `cmd/run.go::Run` 中按顺序完成:读取配置 `config.Init` → 初始化缓存 `common/cache` → 初始化 i18n `common/i18n` → 初始化数据库 `database.Init` → 加载存储 `storage.LoadStorages` → 加载解析器插件 `parsers.LoadPlugins` → (可选)Userbot 登录 → 启动 Telegram Bot `client/bot.Init` → 启动核心任务队列消费 `core.Run`。
|
||||
- 添加新的初始化步骤时,请遵循该顺序并放在 `initAll` 中,而不是分散在各处。
|
||||
|
||||
## 配置与约定
|
||||
- **配置系统**:
|
||||
- 使用 `viper` 读取 `config.toml`,核心结构体定义在 `config/viper.go::Config`。
|
||||
- 默认值在 `config.Init` 中通过 `viper.SetDefault` 定义,如 `workers`、`retry`、`telegram.*`、`db.*` 等。
|
||||
- `Config.C()` 返回的是全局配置副本(值类型),不要在返回值上修改字段;如果需要修改配置流程,应在 `config.Init` 内或通过 `viper` 进行。
|
||||
- 存储配置通过 `config/storage/factory.go::LoadStorageConfigs` 加载并校验,新增存储类型需:
|
||||
- 在 `pkg/enums/storage` 中增加枚举。
|
||||
- 在 `config/storage/` 下新增具体 `StorageConfig` 实现并实现 `Validate`。
|
||||
- 在 `storageFactories` 映射中注册工厂方法。
|
||||
- **环境变量**:
|
||||
- 所有配置键会被 `SAVEANY_` 前缀的环境变量覆盖,`.` 会被 `_` 替换(`SAVEANY_TELEGRAM_APP_ID` 等)。
|
||||
|
||||
## Telegram 客户端与中间件
|
||||
- **Bot 客户端**:
|
||||
- 入口在 `client/bot/bot.go::Init`,使用 `gotgproto.NewClient` 创建,Session 使用 `database.GetDialect(config.C().DB.Session)`,错误处理通过 `ErrorHandler` 回调完成。
|
||||
- Handlers 注册集中在 `client/bot/handlers` 目录,`handlers.Register` 负责统一挂载;新增命令/消息处理逻辑时优先在该目录按功能拆分文件,实现后在 `handlers.Register` 中注册。
|
||||
- Bot 命令列表依赖 `handlers.CommandHandlers` 来自动注册到 Telegram;新增命令时务必更新该切片,以保持 `/help` 与 Bot 命令列表一致。
|
||||
- **中间件**:
|
||||
- 通用中间件位于 `client/middleware/`,包含 floodwait、防崩溃、重试等;`middleware.NewDefaultMiddlewares` 在 `client/bot/bot.go` 中统一挂载。
|
||||
- 新增跨所有更新生效的行为(如日志、统计)时,应优先实现为中间件。
|
||||
|
||||
## 核心任务与队列
|
||||
- **任务接口与队列**:
|
||||
- 核心接口:`core/core.go::Executable`,包含 `Type() TaskType`、`Title()`、`TaskID()`、`Execute(ctx)`。
|
||||
- 任务队列:`pkg/queue.TaskQueue[Executable]`,由 `core.Run` 使用;`Workers` 数量来自配置 `config.C().Workers`。
|
||||
- 任务类型与实现示例位于 `core/tasks/**`,例如文件任务、Telegraph 任务等;新增任务类型应放在对应子目录并实现 `Executable` 接口,然后通过 `core.AddTask` 入队。
|
||||
- **生命周期 Hook**:
|
||||
- `core.worker` 在执行任务前后会根据 `config.C().Hook.Exec` 调用外部命令(`TaskBeforeStart` / `TaskSuccess` / `TaskFail` / `TaskCancel`)。
|
||||
- 修改任务执行流程时需保留这些 Hook 调用,以免破坏用户已有集成。
|
||||
|
||||
## 数据库与持久化
|
||||
- **数据库初始化**:
|
||||
- `database.Init` 使用配置 `config.C().DB.Path` 创建并连接 SQLite,使用 `GetDialect` 抽象驱动(见 `database/driver_*.go`)。
|
||||
- Migration 通过 `db.AutoMigrate(&User{}, &Dir{}, &Rule{}, &WatchChat{})` 完成,模型定义在 `database/*.go` 中。
|
||||
- **用户同步约定**:
|
||||
- `database.syncUsers` 会根据 `config.C().Users` 同步数据库用户表:在配置中新增/删除用户会自动在 DB 中创建/删除对应记录。
|
||||
- 开发涉及用户表逻辑时,请考虑该同步行为,避免在其他地方直接创建/删除用户记录而与配置冲突。
|
||||
|
||||
## 存储后端
|
||||
- **存储抽象**:
|
||||
- 抽象接口在 `config/storage/types.go` 与 `storage/` 顶层(以及子目录)中;`config/storage/*.go` 处理配置解析,`storage/*` 处理真正的上传/下载实现。
|
||||
- 现有实现包括 `local`、`alist`、`s3/minio`、`webdav`、`telegram` 等,每个后端都有对应子目录和配置结构体。
|
||||
- **新增存储实现的推荐路径**:
|
||||
- 在 `config/storage/` 下添加配置结构体 + `Validate`。
|
||||
- 在 `storage/` 下添加具体实现(例如 `storage/foo/`)。
|
||||
- 在 `pkg/enums/storage` 与 `storageFactories` 中注册,并确保 `storages` 配置示例被更新(`config.example.toml` / 文档)。
|
||||
|
||||
## 解析器插件(JS)
|
||||
- **插件运行时**:
|
||||
- 解析器接口和插件文档在 `plugins/README.md`,Go 端入口为 `parsers/` 目录,使用 `goja` 与 `playwright-go`。
|
||||
- 插件通过 `registerParser({ metadata, canHandle, parse })` 注册,使用 `ghttp`/`playwright` 进行 HTTP/浏览器请求。
|
||||
- **与核心交互约定**:
|
||||
- 插件 `parse` 返回的 `Item`/`Resource` 会被转化为内部任务(通常是下载/转存任务)并进入 `core` 队列。
|
||||
- 修改 `Item`/`Resource` 结构或解析逻辑时,要确保保持向后兼容,或在 `plugins/README.md` 中同步更新字段说明和示例。
|
||||
|
||||
## i18n 与日志
|
||||
- **国际化**:
|
||||
- 所有用户可见字符串(尤其是错误与提示)应使用 `common/i18n`:`i18n.T(i18nk.SomeKey, map[string]any{"Name": name})`。
|
||||
- 语言文件位于 `common/i18n/locale/`,`go:generate` 指令在 `main.go` 中生成 `i18nk/keys.go`;新增文案时需:添加到 YAML、运行 `go generate ./...`、再在代码中引用新 key。
|
||||
- **日志**:
|
||||
- 使用 `github.com/charmbracelet/log`,在 `cmd/run.go::Run` 中通过 `log.WithContext` 将 logger 注入 `context.Context`;后续代码优先通过 `log.FromContext(ctx)` 获取 logger。
|
||||
- 编写新代码时,如已有 `ctx`,请使用 `log.FromContext(ctx)` 而不是全局 logger。
|
||||
|
||||
## 开发与运行
|
||||
- **本地运行**:
|
||||
- 直接运行:`go run ./cmd`(`cmd/root.go` + `cmd/run.go`)。
|
||||
- 或通过 Docker:参见根目录 `README.md` 中的 `docker run ...` 示例及 `docker-compose.yml`。
|
||||
- **代码生成与文档**:
|
||||
- i18n key 生成:`go generate ./...` 会执行 `main.go` 顶部的 `//go:generate`,使用 `cmd/geni18n/main.go` 生成 `common/i18n/i18nk/keys.go`。
|
||||
- 文档站点在 `docs/`(Hugo),通常不需要在核心代码改动时同步修改,除非涉及文档内容。
|
||||
|
||||
---
|
||||
|
||||
如果你在实现某个功能时发现以上规则不够具体(例如某类任务在 `core/tasks` 中到底如何落地,或某个存储/解析器的边界不清晰),请在对应章节下扩展更精确的说明,并在 PR 描述中标注。也欢迎你告诉我有哪些部分需要补充或澄清,我可以进一步细化。
|
||||
4
.github/workflows/build-release.yml
vendored
4
.github/workflows/build-release.yml
vendored
@@ -43,7 +43,7 @@ jobs:
|
||||
goarch: arm64
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
uses: actions/checkout@v6
|
||||
|
||||
- name: Extract version from Git Ref
|
||||
id: extract_version
|
||||
@@ -64,7 +64,7 @@ jobs:
|
||||
ldflags: >-
|
||||
-s -w
|
||||
-X "github.com/krau/SaveAny-Bot/config.Version=${{ env.VERSION }}"
|
||||
-X "github.com/krau/SaveAny-Bot/config.BuildTime=${{ format(github.event.repository.updated_at, 'yyyy-MM-dd HH:mm:ss') }}"
|
||||
-X "github.com/krau/SaveAny-Bot/config.BuildTime=${{ github.event.repository.updated_at }}"
|
||||
-X "github.com/krau/SaveAny-Bot/config.GitCommit=${{ github.sha }}"
|
||||
binary_name: saveany-bot
|
||||
env:
|
||||
|
||||
6
.gitignore
vendored
6
.gitignore
vendored
@@ -1,5 +1,6 @@
|
||||
config.toml
|
||||
logs/
|
||||
/cache/
|
||||
tmp/
|
||||
data/
|
||||
downloads/
|
||||
@@ -9,4 +10,7 @@ cache.db
|
||||
temp/
|
||||
.hugo_build.lock
|
||||
playwright/
|
||||
testplugins/
|
||||
testplugins/
|
||||
*.exe
|
||||
tmp-*
|
||||
saveany-bot
|
||||
115
AGENTS.md
Normal file
115
AGENTS.md
Normal file
@@ -0,0 +1,115 @@
|
||||
# Repository Guidelines
|
||||
|
||||
## Project Overview
|
||||
|
||||
SaveAny-Bot is a Telegram bot written in Go that saves files and messages from Telegram and websites to multiple storage backends (Local, S3, MinIO, WebDAV, AList, Rclone, Telegram). It supports single-file saves, batch/album saves, streaming, multi-user access, storage rules, cross-storage transfers, yt-dlp and Aria2 downloads, and a Goja-based JavaScript parser plugin system with optional Playwright browser automation.
|
||||
|
||||
**Tech stack**: Go 1.25, gotgproto + gotd/td v0.149.0 (MTProto), Cobra (CLI), Viper (config), GORM + SQLite, Goja (JS runtime), Playwright, charmbracelet/log. License: AGPL-3.0.
|
||||
|
||||
**Note on gotd versions**: `gotd/td` must stay on `v0.149.0` — v0.150+ breaks `gotgproto` (v1.0.0-beta22) compilation (`AsInputDocumentFileLocation` signature, `gotd/log` Logger interface). Do not bump beyond v0.149.
|
||||
|
||||
## Architecture & Data Flow
|
||||
|
||||
```
|
||||
Telegram update → client/bot/handlers → core.AddTask(Executable) → pkg/queue (serial workers)
|
||||
→ core/tasks/* (download via common/tdler) → storage.Storage → progress feedback
|
||||
```
|
||||
|
||||
- **Startup sequence** (`cmd/run.go::initAll`, keep this order): Config → Cache → i18n → Database → Storage → Parser plugins → Userbot → API → Bot. `bot.Init` returns the exit channel; a `SAVEANTBOT-RESTART` error restarts the process (external supervisor).
|
||||
- **Task pipeline**: handlers build a task (via `core.AddTask`), the queue executes `Executable{Type, Title, TaskID, Execute(ctx)}` with `config.C().Workers` workers. Lifecycle hooks (`TaskBeforeStart/Success/Fail/Cancel`) run around `Execute`. Cancellation = canceling the task's context; tasks must check `ctx.Err()`.
|
||||
- **Dual progress channels**: (1) `pkg/taskevent` context bus (consumed by `api/` for HTTP/Webhook consumers — `taskevent.WithSink` must be injected for API tasks), (2) Telegram message edits via `ProgressTracker` + `tgutil.ExtFromContext(ctx)` (bot tasks). Upload progress currently reaches the Telegram channel only.
|
||||
- **Capability interfaces + type assertion fallback** is the core extensibility pattern: `StorageBatchSaver`, `StorageProgressSaver` (upload progress), `StorageListable`, `StorageReadable` are optional; consumers assert and fall back (e.g. wrap reader with `ioutil.NewProgressReader`). New backends only need to implement the interface and register.
|
||||
- **Config layering**: CLI flag > env `SAVEANY_*` (dots → underscores, e.g. `SAVEANY_TELEGRAM_TOKEN`) > TOML file (path or http(s) URL). `config.C()` returns a **copy** — never mutate it.
|
||||
- **Storage registration (3 places)**: `pkg/enums/storage` ENUM comment (go-enum), `config/storage/factory.go::storageFactories` (config struct with `Validate()`), `storage/storage.go::storageConstructors` (implementation).
|
||||
|
||||
## Key Directories
|
||||
|
||||
| Path | Purpose |
|
||||
|---|---|
|
||||
| `cmd/` | CLI: `run` (main bot), `upload`, `watch` (standalone subcommands that do NOT run initAll), `geni18n` (i18n key generator) |
|
||||
| `core/` | `Executable` interface, queue worker loop, hooks; `core/tasks/{tfile,batchtfile,directlinks,parsed,telegraph,transfer,ytdlp,aria2dl}` |
|
||||
| `client/bot/` | gotgproto client, `handlers/` (all commands + message/callback handlers), `middleware/`, `client/user/` (userbot) |
|
||||
| `storage/` | 8 backends + `storage.go` (interfaces/registry) + `load.go` (per-user storage resolution) |
|
||||
| `parsers/` | `parsers.go` (registry), `js/` (Goja plugins, ghttp/playwright injection, build-tagged), `parsers/` (native: twitter, kemono) |
|
||||
| `config/` | Viper setup, defaults, `storage/` per-type config structs |
|
||||
| `database/` | GORM models (User/Dir/Rule/WatchChat), AutoMigrate, `syncUsers` |
|
||||
| `pkg/` | `queue`, `taskevent`, `tcbdata` (callback data), `rule`, `enums/{tasktype,storage,ctxkey,fnamest}`, `storagetypes`, `tfile`, `parser` |
|
||||
| `common/` | `tdler` (unified downloader), `utils/{tgutil,dlutil,ioutil,fsutil,strutil,tphutil,netutil}`, `i18n` (embedded locales), `cache` (ristretto) |
|
||||
| `api/` | HTTP API + webhook (task factory with sink injection) |
|
||||
| `docs/` | Hugo site (hugo-book theme, zh+en mirrored), separate go.mod |
|
||||
| `plugins/` | JS parser examples + `README.md` (plugin author contract) |
|
||||
|
||||
## Development Commands
|
||||
|
||||
```bash
|
||||
# Build (standard; CGO_ENABLED=0 for static)
|
||||
CGO_ENABLED=0 go build -trimpath -o saveany-bot .
|
||||
go run ./cmd
|
||||
|
||||
# Test — known failures: storage/telegram TestCreateSplitZip/TestExtractThumbFrame/TestGetVideoMetadata
|
||||
# (need gitignored fixtures tests/testfile.dat, tests/testvideo; ffmpeg/ffprobe)
|
||||
go test ./...
|
||||
go test -race ./core/tasks/... ./storage/... ./pkg/queue/... ./common/...
|
||||
go test -run TestQueueBasic ./pkg/queue
|
||||
|
||||
# Codegen — run after editing locale YAML or enum comments
|
||||
go generate ./... # geni18n (i18nk keys) + go-enum (pkg/enums/*)
|
||||
# go-enum is NOT in go.mod; install externally. geni18n runs via go run.
|
||||
|
||||
# Verify
|
||||
go vet ./...
|
||||
go fmt ./...
|
||||
```
|
||||
|
||||
**Build variants** (Dockerfile.default/micro/pico): `-tags=no_jsparser,no_playwright,no_minio,no_bubbletea,sqlite_glebarez` — each has a `*_stub.go`/`*_glebarez.go` pairing; keep stubs in sync.
|
||||
|
||||
Docker: `docker build -t saveany-bot .`, `docker compose up -d` (host network, mounts `./data ./config.toml ./downloads ./cache`). CI (`.github/workflows/`) runs **no tests/lint** — only tag-triggered release/docker builds and docs deployment; run `go test ./...` manually before pushing.
|
||||
|
||||
## Code Conventions & Common Patterns
|
||||
|
||||
- **Imports**: stdlib → third-party → project-internal, blank-line separated. Aliases for clarity (`storconfig`, `storenum`).
|
||||
- **Naming**: PascalCase exported, camelCase unexported, files `snake_case.go`; **not** ALL_CAPS constants.
|
||||
- **Errors**: always wrap with `fmt.Errorf("context: %w", err)`; check with `errors.Is/As`; never ignore.
|
||||
- **Logging**: `log.FromContext(ctx)` with prefixes (`logger.WithPrefix("component")`); never global logger when ctx is available.
|
||||
- **Context values** (read from the passed ctx, never globals): `log.FromContext`, `tgutil.ExtFromContext` (Telegram ext — **required for message edits; if nil, edits are silently dropped**), `storage.FromContext`, `storagetypes.WithSourceCaption`, `ctxkey.ContentLength` / `ctxkey.OverwriteExisting`.
|
||||
- **Progress rendering** (#228 convention): i18n templates declare styles with Telegram HTML (`<b>/<code>/<blockquote>/<i>`); dynamic data MUST go through `i18n.T(key, tgutil.EscapeHTMLTemplateData(data))` before `tgutil.RenderHTML`. Never interpolate user data raw, never render-then-substring-search.
|
||||
- **Progress tracking**: each task package defines its own small `ProgressTracker` interface; optional `UploadProgressTracker` is probed via type assertion (skip if absent). Serialize state + message edits with a mutex; throttle edits (≥1s); aggregate per-item progress monotonically.
|
||||
- **i18n**: only edit `common/i18n/locale/{zh-Hans,en}.yaml` → `go generate ./...` → use `i18nk.<Key>` constants. No raw strings in user-facing messages. zh-Hans and en must stay in sync.
|
||||
- **Registration points** (never forget): new bot command → `client/bot/handlers/register.go::CommandHandlers` (auto-publishes /help menu); new task type → `pkg/enums/tasktype` + `core/tasks/<name>/` + `api/factory.go::CreateTask`; new storage → 3 places above + `docs/content/{en,zh}/deployment/configuration/storages.md`; new enum value → ENUM comment + `go generate`.
|
||||
- **Concurrency**: `errgroup.WithContext` + `SetLimit(config.C().Workers)`, `atomic.Int64` counters, `sync.Once` for single-shot events, mutex around render state. No lock-in-callback (callbacks fire after unlock).
|
||||
- **Cancellation**: queue tasks carry a `WithCancel`-derived ctx; check `ctx.Err()` in loops; classify with `errors.Is(err, context.Canceled)`.
|
||||
- **JS plugins**: `registerParser({metadata, canHandle, parse})`, `version >= 1.0.0`; per-plugin goja VM is single-goroutine (reqCh buffer 10). Changing `pkg/parser.Item/Resource` JSON fields requires updating `plugins/README.md` and example plugins.
|
||||
- **Message edits**: `ext.EditMessage(chatID, &tg.MessagesEditMessageRequest{...})`; cancel buttons via `tgutil.BuildCancelButton(taskID)`; callback payloads via `pkg/tcbdata` + `common/cache`.
|
||||
|
||||
## Important Files
|
||||
|
||||
- `main.go` — `//go:generate` for i18n keys
|
||||
- `cmd/run.go` — startup sequence `Run/initAll/cleanCache` (cache cleanup on exit, `NoCleanCache` opt-out)
|
||||
- `core/core.go` — worker loop, hooks, AddTask/CancelTask
|
||||
- `pkg/queue/queue.go` — generic serial queue (cond/list; duplicate TaskID rejected)
|
||||
- `storage/storage.go` — interfaces + registry + compile-time capability assertions
|
||||
- `config/viper.go`, `config.example.toml` — config schema (authoritative field docs)
|
||||
- `database/db.go` — GORM init, `GetDialect` (build-tag selectable SQLite driver)
|
||||
- `client/bot/handlers/register.go` — handler dispatch order and CommandHandlers
|
||||
- `common/tdler/dler.go` — unified download entry
|
||||
- `core/tasks/batchtfile/item_progress.go` — per-item phase state machine (Downloading/Transferring/Uploading/Retrying/Confirming, FailureStage)
|
||||
- `parsers/js/plugin.go` — Goja plugin runtime
|
||||
- `.github/workflows/` — release/docker/docs (no test gate)
|
||||
|
||||
## Runtime/Tooling Preferences
|
||||
|
||||
- **Go 1.25+**: `t.Context()`, `sync.WaitGroup.Go`, `for range n` are available.
|
||||
- **Runtime binaries**: ffmpeg/ffprobe (media processing/video split), yt-dlp (ytdlp tasks), aria2 optional; Playwright browsers install on demand to `./playwright` (`playwright.Install(chromium, ...)` at first `pw.get()`); Docker images: default has ffmpeg+yt-dlp, micro only curl, pico is scratch static.
|
||||
- **No Makefile, no golangci.yml, no test/lint CI** — verification is manual (`go vet`, `go test`).
|
||||
- **go-enum** required externally for enum generation; **geni18n** is in-repo.
|
||||
- **Docs**: Hugo site in `docs/` (separate go.mod, hugo-book); edit `docs/content/{zh,en}/` — keep both languages mirrored. `docs/public/` is gitignored build output.
|
||||
- **gitignored fixtures**: `storage/telegram/tests/` (missing — 3 tests fail locally), `data/`, `config.toml`, `playwright/`, `testplugins/`.
|
||||
|
||||
## Testing & QA
|
||||
|
||||
- Pure stdlib `testing` (no testify); table-driven (`[]struct{name...}` + `t.Run`) with `t.Fatalf` got/want assertions. Mock via hand-written interface impls or package-variable replacement (`runMediaTool` in `video_split_test.go`, restored with `t.Cleanup`); in-process services for HTTP (`httptest`), S3 (`gofakes3+s3mem`), WebDAV (`x/net/webdav`).
|
||||
- **Locale-dependent tests**: pin with `i18n.Init("zh-Hans")` + `t.Cleanup(...)`.
|
||||
- **Progress/HTML tests**: assert rendered text with `strings.Contains` AND entity counts (`tg.MessageEntityBold/Code/Blockquote/Italic`) — verify style injection stays escaped (`<b>A&B</b>` input must render as literal text).
|
||||
- **Known failures**: `storage/telegram` `TestCreateSplitZip`, `TestExtractThumbFrame`, `TestGetVideoMetadata` need gitignored fixtures + real ffmpeg — skip with `-skip 'Test(CreateSplitZip|ExtractThumbFrame|GetVideoMetadata)$'`; `api/handlers_test.go` has one `t.Skip` (needs initialized core).
|
||||
- **Coverage expectations**: pure logic gets table tests (parsers, URL/path utils, progress throttling, grouping); regressions get bug-scenario-named tests (`progress_regression_test.go`). Network/Telegram/Playwright must never be touched by tests.
|
||||
- When a permanent feature/API change ships: update `config.example.toml` if config, `docs/` if user-facing, `plugins/README.md` if plugin contract, and i18n YAML + `go generate` for new strings.
|
||||
172
CLAUDE.md
172
CLAUDE.md
@@ -1,172 +0,0 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Project Overview
|
||||
|
||||
SaveAny-Bot is a Telegram bot written in Go that saves files from Telegram and external websites to various storage backends (Local, Alist, S3, WebDAV, Telegram). It supports bypassing "restrict saving content" media, batch downloads, auto-organization, and JS parser plugins for extracting content from websites.
|
||||
|
||||
## Commands
|
||||
|
||||
### Development
|
||||
```bash
|
||||
# Run the bot directly
|
||||
go run ./cmd
|
||||
|
||||
# Run with specific config file
|
||||
go run ./cmd --config /path/to/config.toml
|
||||
|
||||
# Generate i18n keys after modifying locale files
|
||||
go generate ./...
|
||||
|
||||
# Run tests
|
||||
go test ./...
|
||||
|
||||
# Run specific package tests
|
||||
go test ./storage/s3
|
||||
go test ./pkg/queue
|
||||
```
|
||||
|
||||
### Docker
|
||||
```bash
|
||||
# Build and run locally
|
||||
docker-compose -f docker-compose.local.yml up
|
||||
|
||||
# Production deployment
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
### Application Initialization Flow (CRITICAL)
|
||||
|
||||
The application follows a strict initialization sequence in `cmd/run.go::initAll`:
|
||||
|
||||
1. **Config** (`config.Init`) - Load config.toml using viper
|
||||
2. **Cache** (`common/cache`) - Initialize in-memory cache
|
||||
3. **i18n** (`common/i18n`) - Load localization files from `common/i18n/locale/`
|
||||
4. **Database** (`database.Init`) - Connect to SQLite, auto-migrate models
|
||||
5. **Storage** (`storage.LoadStorages`) - Initialize all configured storage backends
|
||||
6. **Parsers** (`parsers.LoadPlugins`) - Load JS parser plugins if enabled
|
||||
7. **Userbot** (optional) - Login to Telegram userbot if configured
|
||||
8. **Bot** (`client/bot.Init`) - Initialize Telegram bot client
|
||||
9. **Core** (`core.Run`) - Start task queue workers
|
||||
|
||||
**When adding new initialization steps, add them to `initAll` in this order.**
|
||||
|
||||
### Core Components
|
||||
|
||||
#### Task Queue System (`core/`)
|
||||
- Central abstraction: `core.Executable` interface with `Type()`, `Title()`, `TaskID()`, `Execute(ctx)`
|
||||
- Task queue: `pkg/queue.TaskQueue[Executable]` managed by `core.Run`
|
||||
- Worker pool size controlled by `config.Workers`
|
||||
- Task implementations in `core/tasks/`:
|
||||
- `tfile/` - Telegram file downloads
|
||||
- `batchtfile/` - Batch downloads
|
||||
- `telegraph/` - Telegraph page saving
|
||||
- `parsed/` - Content from parser plugins
|
||||
- `directlinks/` - Direct URL downloads
|
||||
- Tasks support lifecycle hooks: `TaskBeforeStart`, `TaskSuccess`, `TaskFail`, `TaskCancel` (configured via `config.Hook.Exec`)
|
||||
|
||||
#### Telegram Client (`client/`)
|
||||
- **Bot**: `client/bot/bot.go` uses `gotgproto.NewClient`, handlers in `client/bot/handlers/`
|
||||
- **Userbot**: `client/user/` for bypassing restricted content (optional)
|
||||
- **Middleware**: `client/middleware/` handles flood wait, recovery, retry
|
||||
- Handler registration: All handlers registered in `handlers.Register()`, command list in `handlers.CommandHandlers`
|
||||
|
||||
#### Storage Backends (`storage/`)
|
||||
- Abstract interface: `Storage` with `Init()`, `Save()`, `Exists()`, `JoinStoragePath()`
|
||||
- Implementations: `local/`, `alist/`, `s3/`, `minio/`, `webdav/`, `telegram/`
|
||||
- Storage registry: `storage.Storages` map, constructed via `storageConstructors`
|
||||
- Add new storage: Define enum in `pkg/enums/storage`, config in `config/storage/`, implementation in `storage/`, register in `storageConstructors`
|
||||
|
||||
#### Parser Plugins (`parsers/`)
|
||||
- Native parsers: `parsers/native/twitter/`, `parsers/native/kemono/`
|
||||
- JS plugin runtime: `parsers/js/` using `goja` VM and `playwright-go`
|
||||
- Plugin interface: `registerParser({ metadata, canHandle, parse })` in JS
|
||||
- Plugins return `parser.Item` with `Resources[]`, converted to `core/tasks/parsed` tasks
|
||||
- Plugin docs: `plugins/README.md`
|
||||
|
||||
#### Configuration (`config/`)
|
||||
- Main config: `config/viper.go::Config` loaded via `viper`
|
||||
- Storage configs: `config/storage/factory.go::LoadStorageConfigs` with type-specific validation
|
||||
- Environment override: Prefix `SAVEANY_`, dots become underscores (e.g., `SAVEANY_TELEGRAM_TOKEN`)
|
||||
- **Important**: `config.C()` returns a copy, don't modify its fields; modify via `viper` in `config.Init`
|
||||
|
||||
#### Database (`database/`)
|
||||
- SQLite with GORM
|
||||
- Models: `User`, `Dir`, `Rule`, `WatchChat`
|
||||
- User sync: `database.syncUsers` syncs `config.Users` to DB (creates/deletes users based on config)
|
||||
- Don't create/delete users manually; use config file
|
||||
|
||||
#### Internationalization (`common/i18n/`)
|
||||
- Locale files: `common/i18n/locale/*.yaml`
|
||||
- Key generation: `go generate ./...` runs `cmd/geni18n/main.go` to generate `common/i18n/i18nk/keys.go`
|
||||
- Usage: `i18n.T(i18nk.KeyName, map[string]any{"Param": value})`
|
||||
- Add new strings: Edit YAML → run `go generate` → use new key in code
|
||||
|
||||
#### Logging
|
||||
- Uses `github.com/charmbracelet/log`
|
||||
- Logger injected into context in `cmd/run.go::Run` via `log.WithContext`
|
||||
- Prefer `log.FromContext(ctx)` over global logger when context is available
|
||||
|
||||
### Data Flow
|
||||
|
||||
1. User sends message/file to Telegram bot
|
||||
2. Handler in `client/bot/handlers/` processes update
|
||||
3. Handler creates task (implements `core.Executable`) and calls `core.AddTask()`
|
||||
4. Task enters `pkg/queue.TaskQueue`
|
||||
5. Worker goroutine picks up task, calls `Execute(ctx)`
|
||||
6. Task downloads content (from Telegram or parser) and saves to storage backend via `Storage.Save()`
|
||||
7. Task completion triggers hooks, sends result message back to user
|
||||
|
||||
For parsed content (URLs):
|
||||
1. Handler extracts URL, calls `parsers.ParseWithContext()`
|
||||
2. Parser (native or JS plugin) returns `parser.Item` with `Resources[]`
|
||||
3. `core/tasks/parsed.Task` created with resources
|
||||
4. Task downloads each resource and saves to storage
|
||||
|
||||
## Project-Specific Guidelines
|
||||
|
||||
### Adding New Storage Backend
|
||||
|
||||
1. Define enum in `pkg/enums/storage/storage.go`
|
||||
2. Create config struct in `config/storage/yourtype.go` implementing `StorageConfig` with `Validate()`
|
||||
3. Implement storage in `storage/yourtype/yourtype.go` with `Storage` interface
|
||||
4. Register in `config/storage/factory.go::storageFactories` and `storage/storage.go::storageConstructors`
|
||||
5. Update `config.example.toml` with example configuration
|
||||
|
||||
### Adding New Task Type
|
||||
|
||||
1. Create package in `core/tasks/yourtype/`
|
||||
2. Define struct implementing `core.Executable` interface
|
||||
3. Implement `Type()`, `Title()`, `TaskID()`, `Execute(ctx)`
|
||||
4. Add task type enum in `pkg/enums/tasktype/`
|
||||
5. Create task from handler and call `core.AddTask(ctx, task)`
|
||||
|
||||
### Adding New Bot Command
|
||||
|
||||
1. Create handler file in `client/bot/handlers/yourcommand.go`
|
||||
2. Implement handler function with signature matching gotgproto patterns
|
||||
3. Register in `handlers.Register()` function
|
||||
4. Add to `handlers.CommandHandlers` slice for command list
|
||||
5. Add i18n strings for command description and responses
|
||||
|
||||
### Adding i18n Strings
|
||||
|
||||
1. Edit locale files in `common/i18n/locale/` (e.g., `zh-Hans.yaml`, `en.yaml`)
|
||||
2. Run `go generate ./...` to generate `common/i18n/i18nk/keys.go`
|
||||
3. Use in code: `i18n.T(i18nk.YourNewKey, map[string]any{"Variable": value})`
|
||||
|
||||
### Working with Middleware
|
||||
|
||||
New cross-cutting concerns (logging, metrics, rate limiting) should be implemented as middleware in `client/middleware/`:
|
||||
1. Create middleware file with function signature matching gotgproto middleware
|
||||
2. Add to `middleware.NewDefaultMiddlewares()` in `client/bot/bot.go`
|
||||
|
||||
### Parser Plugin Development
|
||||
|
||||
- JS plugins go in directories specified by `config.parser.plugin_dirs` (default: `./plugins/`)
|
||||
- See `plugins/README.md` for plugin API documentation
|
||||
- Plugins have access to `ghttp` for HTTP requests and `playwright` for browser automation
|
||||
- Test plugins in `./testplugins/` directory
|
||||
@@ -26,7 +26,7 @@ RUN --mount=type=cache,target=/root/.cache/go-build \
|
||||
|
||||
FROM alpine:latest
|
||||
|
||||
RUN apk add --no-cache curl ffmpeg
|
||||
RUN apk add --no-cache curl ffmpeg yt-dlp
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
|
||||
@@ -7,6 +7,8 @@ ARG BuildTime="Unknown"
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
RUN apk add --no-cache ca-certificates
|
||||
|
||||
COPY go.mod go.sum ./
|
||||
RUN --mount=type=cache,target=/go/pkg/mod \
|
||||
go mod download
|
||||
@@ -31,5 +33,9 @@ FROM scratch
|
||||
WORKDIR /app
|
||||
|
||||
COPY --from=builder /app/saveany-bot .
|
||||
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
|
||||
|
||||
ENV SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
|
||||
ENV SSL_CERT_DIR=/etc/ssl/certs
|
||||
|
||||
ENTRYPOINT ["/app/saveany-bot"]
|
||||
|
||||
@@ -26,12 +26,16 @@
|
||||
- Multi-user support
|
||||
- Auto organize files based on storage rules
|
||||
- Watch specified chats and auto-save messages, with filters
|
||||
- Transfer files between different storage backends
|
||||
- Integrate with yt-dlp to download and save media from 1000+ websites
|
||||
- Aria2 integration to download files from URLs/magnets and save to storages
|
||||
- Write JS parser plugins to save files from almost any website
|
||||
- Storage backends:
|
||||
- Alist
|
||||
- S3
|
||||
- WebDAV
|
||||
- Local filesystem
|
||||
- Rclone (via command line)
|
||||
- Telegram (re-upload to specified chats)
|
||||
|
||||
## 📦 Quick Start
|
||||
@@ -39,10 +43,11 @@
|
||||
Create a `config.toml` file with the following content:
|
||||
|
||||
```toml
|
||||
lang = "en" # Language setting, "en" for English
|
||||
[telegram]
|
||||
token = "" # Your bot token, obtained from @BotFather
|
||||
[telegram.proxy]
|
||||
# Enable proxy for Telegram, currently only SOCKS5 is supported
|
||||
# Enable proxy for Telegram
|
||||
enable = false
|
||||
url = "socks5://127.0.0.1:7890"
|
||||
|
||||
|
||||
@@ -24,12 +24,16 @@
|
||||
- 多用户使用
|
||||
- 基于存储规则的自动整理
|
||||
- 监听并自动转存指定聊天的消息, 支持过滤
|
||||
- 在不同存储端之间转存文件
|
||||
- 集成 yt-dlp, 从所支持的网站下载并转存媒体文件
|
||||
- 集成 Aria2, 支持直链/磁力下载和转存
|
||||
- 使用 js 编写解析器插件以转存任意网站的文件
|
||||
- 存储端支持:
|
||||
- Alist
|
||||
- S3
|
||||
- WebDAV
|
||||
- 本地磁盘
|
||||
- Rclone
|
||||
- Telegram (重传回指定聊天)
|
||||
|
||||
## 快速开始
|
||||
@@ -40,7 +44,7 @@
|
||||
[telegram]
|
||||
token = "" # 你的 Bot Token, 在 @BotFather 获取
|
||||
[telegram.proxy]
|
||||
# 启用代理连接 telegram, 当前只支持 socks5
|
||||
# 启用代理连接 telegram
|
||||
enable = false
|
||||
url = "socks5://127.0.0.1:7890"
|
||||
|
||||
|
||||
42
api/auth.go
Normal file
42
api/auth.go
Normal file
@@ -0,0 +1,42 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"crypto/subtle"
|
||||
"net/http"
|
||||
"strings"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
)
|
||||
|
||||
// AuthMiddleware 返回认证中间件
|
||||
func AuthMiddleware() func(http.Handler) http.Handler {
|
||||
return func(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
cfg := config.C().API
|
||||
|
||||
// 从请求头获取 token
|
||||
authHeader := r.Header.Get("Authorization")
|
||||
if authHeader == "" {
|
||||
WriteError(w, http.StatusUnauthorized, "unauthorized", "missing authorization header")
|
||||
return
|
||||
}
|
||||
|
||||
// 提取 Bearer token
|
||||
parts := strings.SplitN(authHeader, " ", 2)
|
||||
if len(parts) != 2 || strings.ToLower(parts[0]) != "bearer" {
|
||||
WriteError(w, http.StatusUnauthorized, "unauthorized", "invalid authorization header format")
|
||||
return
|
||||
}
|
||||
|
||||
token := parts[1]
|
||||
|
||||
// 验证 token
|
||||
if subtle.ConstantTimeCompare([]byte(token), []byte(cfg.Token)) != 1 {
|
||||
WriteError(w, http.StatusUnauthorized, "unauthorized", "invalid token")
|
||||
return
|
||||
}
|
||||
|
||||
next.ServeHTTP(w, r)
|
||||
})
|
||||
}
|
||||
}
|
||||
382
api/factory.go
Normal file
382
api/factory.go
Normal file
@@ -0,0 +1,382 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
"github.com/krau/SaveAny-Bot/core"
|
||||
"github.com/krau/SaveAny-Bot/core/tasks/aria2dl"
|
||||
"github.com/krau/SaveAny-Bot/core/tasks/batchtfile"
|
||||
"github.com/krau/SaveAny-Bot/core/tasks/directlinks"
|
||||
"github.com/krau/SaveAny-Bot/core/tasks/parsed"
|
||||
tphtask "github.com/krau/SaveAny-Bot/core/tasks/telegraph"
|
||||
"github.com/krau/SaveAny-Bot/core/tasks/tfile"
|
||||
"github.com/krau/SaveAny-Bot/core/tasks/transfer"
|
||||
"github.com/krau/SaveAny-Bot/core/tasks/ytdlp"
|
||||
"github.com/krau/SaveAny-Bot/parsers/parsers"
|
||||
"github.com/krau/SaveAny-Bot/pkg/aria2"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
|
||||
"github.com/krau/SaveAny-Bot/pkg/parser"
|
||||
"github.com/krau/SaveAny-Bot/pkg/taskevent"
|
||||
"github.com/krau/SaveAny-Bot/pkg/telegraph"
|
||||
"github.com/krau/SaveAny-Bot/storage"
|
||||
"github.com/rs/xid"
|
||||
)
|
||||
|
||||
// TaskFactory 任务工厂
|
||||
type TaskFactory struct {
|
||||
ctx context.Context
|
||||
}
|
||||
|
||||
// NewTaskFactory 创建任务工厂
|
||||
func NewTaskFactory(ctx context.Context) *TaskFactory {
|
||||
return &TaskFactory{ctx: ctx}
|
||||
}
|
||||
|
||||
// CreateTask 创建任务
|
||||
func (f *TaskFactory) CreateTask(req *CreateTaskRequest) (*CreateTaskResponse, error) {
|
||||
// 验证存储
|
||||
stor, ok := storage.GetStorage(req.Storage)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("storage not found: %s", req.Storage)
|
||||
}
|
||||
|
||||
taskID := xid.New().String()
|
||||
createdAt := time.Now()
|
||||
|
||||
switch req.Type {
|
||||
case tasktype.TaskTypeDirectlinks:
|
||||
return f.createDirectLinksTask(taskID, createdAt, req, stor)
|
||||
case tasktype.TaskTypeYtdlp:
|
||||
return f.createYTDLPTask(taskID, createdAt, req, stor)
|
||||
case tasktype.TaskTypeAria2:
|
||||
return f.createAria2Task(taskID, createdAt, req, stor)
|
||||
case tasktype.TaskTypeParseditem:
|
||||
return f.createParsedTask(taskID, createdAt, req, stor)
|
||||
case tasktype.TaskTypeTgfiles:
|
||||
return f.createTGFilesTask(taskID, createdAt, req, stor)
|
||||
case tasktype.TaskTypeTphpics:
|
||||
return f.createTPHPicsTask(taskID, createdAt, req, stor)
|
||||
case tasktype.TaskTypeTransfer:
|
||||
return f.createTransferTask(taskID, createdAt, req)
|
||||
default:
|
||||
return nil, fmt.Errorf("unsupported task type: %s", req.Type)
|
||||
}
|
||||
}
|
||||
|
||||
func (f *TaskFactory) registerAndEnqueueTask(task core.Executable, taskType tasktype.TaskType, storageName, path, webhook string) error {
|
||||
taskID := task.TaskID()
|
||||
info := RegisterTask(taskID, string(taskType), storageName, path, task.Title(), webhook)
|
||||
|
||||
// Inject the progress sink into the context so the task's Emit calls update
|
||||
// the API store (and fire the webhook on terminal states) without the task
|
||||
// knowing about the API.
|
||||
taskCtx := taskevent.WithSink(f.ctx, info)
|
||||
|
||||
err := core.AddTask(taskCtx, task)
|
||||
if err != nil {
|
||||
DeleteTask(taskID)
|
||||
return fmt.Errorf("failed to add task: %w", err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// createDirectLinksTask 创建直链下载任务
|
||||
func (f *TaskFactory) createDirectLinksTask(taskID string, createdAt time.Time, req *CreateTaskRequest, stor storage.Storage) (*CreateTaskResponse, error) {
|
||||
var params DirectLinksParams
|
||||
if err := json.Unmarshal(req.Params, ¶ms); err != nil {
|
||||
return nil, fmt.Errorf("invalid params: %w", err)
|
||||
}
|
||||
|
||||
if len(params.URLs) == 0 {
|
||||
return nil, fmt.Errorf("no URLs provided")
|
||||
}
|
||||
|
||||
task := directlinks.NewTask(taskID, f.ctx, params.URLs, stor, req.Path, nil)
|
||||
|
||||
err := f.registerAndEnqueueTask(task, tasktype.TaskTypeDirectlinks, req.Storage, req.Path, req.Webhook)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &CreateTaskResponse{
|
||||
TaskID: taskID,
|
||||
Type: tasktype.TaskTypeDirectlinks,
|
||||
Status: TaskStatusQueued,
|
||||
CreatedAt: createdAt,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// createYTDLPTask 创建 yt-dlp 任务
|
||||
func (f *TaskFactory) createYTDLPTask(taskID string, createdAt time.Time, req *CreateTaskRequest, stor storage.Storage) (*CreateTaskResponse, error) {
|
||||
var params YTDLPParams
|
||||
if err := json.Unmarshal(req.Params, ¶ms); err != nil {
|
||||
return nil, fmt.Errorf("invalid params: %w", err)
|
||||
}
|
||||
|
||||
if len(params.URLs) == 0 {
|
||||
return nil, fmt.Errorf("no URLs provided")
|
||||
}
|
||||
|
||||
task := ytdlp.NewTask(taskID, f.ctx, params.URLs, params.Flags, stor, req.Path, nil)
|
||||
|
||||
err := f.registerAndEnqueueTask(task, tasktype.TaskTypeYtdlp, req.Storage, req.Path, req.Webhook)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &CreateTaskResponse{
|
||||
TaskID: taskID,
|
||||
Type: tasktype.TaskTypeYtdlp,
|
||||
Status: TaskStatusQueued,
|
||||
CreatedAt: createdAt,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// createAria2Task 创建 Aria2 任务
|
||||
func (f *TaskFactory) createAria2Task(taskID string, createdAt time.Time, req *CreateTaskRequest, stor storage.Storage) (*CreateTaskResponse, error) {
|
||||
var params Aria2Params
|
||||
if err := json.Unmarshal(req.Params, ¶ms); err != nil {
|
||||
return nil, fmt.Errorf("invalid params: %w", err)
|
||||
}
|
||||
|
||||
if len(params.URLs) == 0 {
|
||||
return nil, fmt.Errorf("no URLs provided")
|
||||
}
|
||||
|
||||
// 检查 Aria2 是否启用
|
||||
cfg := config.C().Aria2
|
||||
if !cfg.Enable {
|
||||
return nil, fmt.Errorf("aria2 is not enabled")
|
||||
}
|
||||
|
||||
aria2Client, err := aria2.NewClient(cfg.Url, cfg.Secret)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to create aria2 client: %w", err)
|
||||
}
|
||||
|
||||
// 添加下载任务到 Aria2
|
||||
gid, err := aria2Client.AddURI(f.ctx, params.URLs, nil)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to add aria2 task: %w", err)
|
||||
}
|
||||
|
||||
task := aria2dl.NewTask(taskID, f.ctx, gid, params.URLs, aria2Client, stor, req.Path, nil)
|
||||
|
||||
err = f.registerAndEnqueueTask(task, tasktype.TaskTypeAria2, req.Storage, req.Path, req.Webhook)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &CreateTaskResponse{
|
||||
TaskID: taskID,
|
||||
Type: tasktype.TaskTypeAria2,
|
||||
Status: TaskStatusQueued,
|
||||
CreatedAt: createdAt,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// createParsedTask 创建解析任务
|
||||
func (f *TaskFactory) createParsedTask(taskID string, createdAt time.Time, req *CreateTaskRequest, stor storage.Storage) (*CreateTaskResponse, error) {
|
||||
var params ParsedParams
|
||||
if err := json.Unmarshal(req.Params, ¶ms); err != nil {
|
||||
return nil, fmt.Errorf("invalid params: %w", err)
|
||||
}
|
||||
|
||||
if params.URL == "" {
|
||||
return nil, fmt.Errorf("no URL provided")
|
||||
}
|
||||
|
||||
// 查找合适的解析器
|
||||
var p parser.Parser
|
||||
for _, parserItem := range parsers.Get() {
|
||||
if parserItem.CanHandle(params.URL) {
|
||||
p = parserItem
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
if p == nil {
|
||||
return nil, fmt.Errorf("no parser found for URL: %s", params.URL)
|
||||
}
|
||||
|
||||
// 解析 URL
|
||||
item, err := p.Parse(f.ctx, params.URL)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to parse URL: %w", err)
|
||||
}
|
||||
|
||||
task := parsed.NewTask(taskID, f.ctx, stor, req.Path, item, nil)
|
||||
|
||||
err = f.registerAndEnqueueTask(task, tasktype.TaskTypeParseditem, req.Storage, req.Path, req.Webhook)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &CreateTaskResponse{
|
||||
TaskID: taskID,
|
||||
Type: tasktype.TaskTypeParseditem,
|
||||
Status: TaskStatusQueued,
|
||||
CreatedAt: createdAt,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// createTGFilesTask 创建 Telegram 文件下载任务
|
||||
func (f *TaskFactory) createTGFilesTask(taskID string, createdAt time.Time, req *CreateTaskRequest, stor storage.Storage) (*CreateTaskResponse, error) {
|
||||
var params TGFilesParams
|
||||
if err := json.Unmarshal(req.Params, ¶ms); err != nil {
|
||||
return nil, fmt.Errorf("invalid params: %w", err)
|
||||
}
|
||||
|
||||
if len(params.MessageLinks) == 0 {
|
||||
return nil, fmt.Errorf("no message links provided")
|
||||
}
|
||||
|
||||
// 提取文件
|
||||
files, err := ExtractFilesFromLinks(f.ctx, params.MessageLinks)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to extract files: %w", err)
|
||||
}
|
||||
|
||||
if len(files) == 0 {
|
||||
return nil, fmt.Errorf("no files found in provided links")
|
||||
}
|
||||
|
||||
var task core.Executable
|
||||
|
||||
if len(files) == 1 {
|
||||
// 单个文件任务
|
||||
tfileTask, err := tfile.NewTGFileTask(taskID, f.ctx, files[0], stor, req.Path, nil)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to create tfile task: %w", err)
|
||||
}
|
||||
task = tfileTask
|
||||
} else {
|
||||
// 批量文件任务
|
||||
elems := make([]batchtfile.TaskElement, 0, len(files))
|
||||
for _, file := range files {
|
||||
elem, err := batchtfile.NewTaskElement(stor, req.Path, file)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to create task element: %w", err)
|
||||
}
|
||||
elems = append(elems, *elem)
|
||||
}
|
||||
|
||||
task = batchtfile.NewBatchTGFileTask(taskID, f.ctx, elems, nil, true)
|
||||
}
|
||||
|
||||
err = f.registerAndEnqueueTask(task, tasktype.TaskTypeTgfiles, req.Storage, req.Path, req.Webhook)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &CreateTaskResponse{
|
||||
TaskID: taskID,
|
||||
Type: tasktype.TaskTypeTgfiles,
|
||||
Status: TaskStatusQueued,
|
||||
CreatedAt: createdAt,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// createTPHPicsTask 创建 Telegraph 图片下载任务
|
||||
func (f *TaskFactory) createTPHPicsTask(taskID string, createdAt time.Time, req *CreateTaskRequest, stor storage.Storage) (*CreateTaskResponse, error) {
|
||||
var params TPHPicsParams
|
||||
if err := json.Unmarshal(req.Params, ¶ms); err != nil {
|
||||
return nil, fmt.Errorf("invalid params: %w", err)
|
||||
}
|
||||
|
||||
if params.TelegraphURL == "" {
|
||||
return nil, fmt.Errorf("no telegraph URL provided")
|
||||
}
|
||||
|
||||
// 提取图片
|
||||
pics, phPath, err := ExtractTelegraphImages(f.ctx, params.TelegraphURL)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to extract telegraph images: %w", err)
|
||||
}
|
||||
|
||||
if len(pics) == 0 {
|
||||
return nil, fmt.Errorf("no images found in telegraph page")
|
||||
}
|
||||
|
||||
client := telegraph.NewClient()
|
||||
task := tphtask.NewTask(taskID, f.ctx, phPath, pics, stor, req.Path, client, nil)
|
||||
|
||||
err = f.registerAndEnqueueTask(task, tasktype.TaskTypeTphpics, req.Storage, req.Path, req.Webhook)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &CreateTaskResponse{
|
||||
TaskID: taskID,
|
||||
Type: tasktype.TaskTypeTphpics,
|
||||
Status: TaskStatusQueued,
|
||||
CreatedAt: createdAt,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// createTransferTask 创建存储间传输任务
|
||||
func (f *TaskFactory) createTransferTask(taskID string, createdAt time.Time, req *CreateTaskRequest) (*CreateTaskResponse, error) {
|
||||
var params TransferParams
|
||||
if err := json.Unmarshal(req.Params, ¶ms); err != nil {
|
||||
return nil, fmt.Errorf("invalid params: %w", err)
|
||||
}
|
||||
|
||||
// 验证源存储和目标存储
|
||||
sourceStor, ok := storage.GetStorage(params.SourceStorage)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("source storage not found: %s", params.SourceStorage)
|
||||
}
|
||||
|
||||
targetStor, ok := storage.GetStorage(params.TargetStorage)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("target storage not found: %s", params.TargetStorage)
|
||||
}
|
||||
|
||||
// 检查源存储是否可读
|
||||
sourceReadable, ok := sourceStor.(storage.StorageReadable)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("source storage does not support reading: %s", params.SourceStorage)
|
||||
}
|
||||
|
||||
// 检查源存储是否可列
|
||||
sourceListable, ok := sourceStor.(storage.StorageListable)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("source storage does not support listing: %s", params.SourceStorage)
|
||||
}
|
||||
|
||||
// 列出源文件
|
||||
files, err := sourceListable.ListFiles(f.ctx, params.SourcePath)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to list source files: %w", err)
|
||||
}
|
||||
|
||||
if len(files) == 0 {
|
||||
return nil, fmt.Errorf("no files found at source path: %s", params.SourcePath)
|
||||
}
|
||||
|
||||
// 创建传输元素
|
||||
elems := make([]transfer.TaskElement, 0, len(files))
|
||||
for _, file := range files {
|
||||
elem := transfer.NewTaskElement(sourceReadable, file, targetStor, params.TargetPath)
|
||||
elems = append(elems, *elem)
|
||||
}
|
||||
|
||||
task := transfer.NewTransferTask(taskID, f.ctx, elems, nil, true)
|
||||
|
||||
err = f.registerAndEnqueueTask(task, tasktype.TaskTypeTransfer, params.TargetStorage, params.TargetPath, req.Webhook)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &CreateTaskResponse{
|
||||
TaskID: taskID,
|
||||
Type: tasktype.TaskTypeTransfer,
|
||||
Status: TaskStatusQueued,
|
||||
CreatedAt: createdAt,
|
||||
}, nil
|
||||
}
|
||||
242
api/handlers.go
Normal file
242
api/handlers.go
Normal file
@@ -0,0 +1,242 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/core"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
|
||||
"github.com/krau/SaveAny-Bot/storage"
|
||||
)
|
||||
|
||||
// Handlers 处理器结构体
|
||||
type Handlers struct {
|
||||
factory *TaskFactory
|
||||
}
|
||||
|
||||
// NewHandlers 创建处理器
|
||||
func NewHandlers(factory *TaskFactory) *Handlers {
|
||||
return &Handlers{factory: factory}
|
||||
}
|
||||
|
||||
// CreateTaskHandler 创建任务处理器
|
||||
func (h *Handlers) CreateTaskHandler(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodPost {
|
||||
WriteError(w, http.StatusMethodNotAllowed, "method_not_allowed", "only POST method is allowed")
|
||||
return
|
||||
}
|
||||
|
||||
var req CreateTaskRequest
|
||||
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
||||
WriteError(w, http.StatusBadRequest, "invalid_request", "failed to decode request body: "+err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
// 验证请求
|
||||
if req.Type == "" {
|
||||
WriteError(w, http.StatusBadRequest, "invalid_request", "task type is required")
|
||||
return
|
||||
}
|
||||
|
||||
if req.Storage == "" {
|
||||
WriteError(w, http.StatusBadRequest, "invalid_request", "storage is required")
|
||||
return
|
||||
}
|
||||
|
||||
// 创建任务
|
||||
resp, err := h.factory.CreateTask(&req)
|
||||
if err != nil {
|
||||
WriteError(w, http.StatusBadRequest, "task_creation_failed", err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
WriteJSON(w, http.StatusCreated, resp)
|
||||
}
|
||||
|
||||
// ListTasksHandler 列出任务处理器
|
||||
func (h *Handlers) ListTasksHandler(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodGet {
|
||||
WriteError(w, http.StatusMethodNotAllowed, "method_not_allowed", "only GET method is allowed")
|
||||
return
|
||||
}
|
||||
|
||||
tasks := GetAllTasks()
|
||||
response := make([]TaskInfoResponse, 0, len(tasks))
|
||||
|
||||
for _, task := range tasks {
|
||||
info := convertTaskProgressToResponse(task)
|
||||
response = append(response, info)
|
||||
}
|
||||
|
||||
WriteJSON(w, http.StatusOK, TasksListResponse{
|
||||
Tasks: response,
|
||||
Total: len(response),
|
||||
})
|
||||
}
|
||||
|
||||
// GetTaskHandler 获取单个任务处理器
|
||||
func (h *Handlers) GetTaskHandler(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodGet {
|
||||
WriteError(w, http.StatusMethodNotAllowed, "method_not_allowed", "only GET method is allowed")
|
||||
return
|
||||
}
|
||||
|
||||
taskID := extractTaskIDFromPath(r.URL.Path)
|
||||
if taskID == "" {
|
||||
WriteError(w, http.StatusBadRequest, "invalid_request", "task ID is required")
|
||||
return
|
||||
}
|
||||
|
||||
task, ok := GetTask(taskID)
|
||||
if !ok {
|
||||
WriteError(w, http.StatusNotFound, "task_not_found", "task not found: "+taskID)
|
||||
return
|
||||
}
|
||||
|
||||
resp := convertTaskProgressToResponse(task)
|
||||
WriteJSON(w, http.StatusOK, resp)
|
||||
}
|
||||
|
||||
// CancelTaskHandler 取消任务处理器
|
||||
func (h *Handlers) CancelTaskHandler(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodDelete {
|
||||
WriteError(w, http.StatusMethodNotAllowed, "method_not_allowed", "only DELETE method is allowed")
|
||||
return
|
||||
}
|
||||
|
||||
taskID := extractTaskIDFromPath(r.URL.Path)
|
||||
if taskID == "" {
|
||||
WriteError(w, http.StatusBadRequest, "invalid_request", "task ID is required")
|
||||
return
|
||||
}
|
||||
|
||||
task, ok := GetTask(taskID)
|
||||
if !ok {
|
||||
WriteError(w, http.StatusNotFound, "task_not_found", "task not found: "+taskID)
|
||||
return
|
||||
}
|
||||
|
||||
// Cancel the task; the terminal status is set via the task event stream.
|
||||
if err := core.CancelTask(r.Context(), taskID); err != nil {
|
||||
WriteError(w, http.StatusInternalServerError, "cancel_failed", "failed to cancel task: "+err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
task.UpdateStatus(TaskStatusCancelled)
|
||||
WriteJSON(w, http.StatusOK, map[string]string{"message": "task cancelled successfully"})
|
||||
}
|
||||
|
||||
// ListStoragesHandler 列出存储处理器
|
||||
func (h *Handlers) ListStoragesHandler(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodGet {
|
||||
WriteError(w, http.StatusMethodNotAllowed, "method_not_allowed", "only GET method is allowed")
|
||||
return
|
||||
}
|
||||
|
||||
all := storage.AllStorages()
|
||||
storages := make([]StorageInfo, 0, len(all))
|
||||
for name, stor := range all {
|
||||
storages = append(storages, StorageInfo{
|
||||
Name: name,
|
||||
Type: string(stor.Type()),
|
||||
})
|
||||
}
|
||||
|
||||
WriteJSON(w, http.StatusOK, StoragesResponse{Storages: storages})
|
||||
}
|
||||
|
||||
// GetTaskTypesHandler 获取支持的任务类型
|
||||
func (h *Handlers) GetTaskTypesHandler(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodGet {
|
||||
WriteError(w, http.StatusMethodNotAllowed, "method_not_allowed", "only GET method is allowed")
|
||||
return
|
||||
}
|
||||
|
||||
types := []tasktype.TaskType{
|
||||
tasktype.TaskTypeDirectlinks,
|
||||
tasktype.TaskTypeYtdlp,
|
||||
tasktype.TaskTypeAria2,
|
||||
tasktype.TaskTypeParseditem,
|
||||
tasktype.TaskTypeTgfiles,
|
||||
tasktype.TaskTypeTphpics,
|
||||
tasktype.TaskTypeTransfer,
|
||||
}
|
||||
|
||||
WriteJSON(w, http.StatusOK, map[string]any{
|
||||
"types": types,
|
||||
})
|
||||
}
|
||||
|
||||
// HealthCheckHandler 健康检查处理器
|
||||
func (h *Handlers) HealthCheckHandler(w http.ResponseWriter, r *http.Request) {
|
||||
WriteJSON(w, http.StatusOK, map[string]string{
|
||||
"status": "ok",
|
||||
})
|
||||
}
|
||||
|
||||
// extractTaskIDFromPath 从路径中提取任务 ID
|
||||
// 路径格式: /api/v1/tasks/:id
|
||||
func extractTaskIDFromPath(path string) string {
|
||||
parts := strings.Split(strings.Trim(path, "/"), "/")
|
||||
if len(parts) < 4 {
|
||||
return ""
|
||||
}
|
||||
return parts[3]
|
||||
}
|
||||
|
||||
// convertTaskProgressToResponse renders a task's current state, computing
|
||||
// percent and speed from the snapshot taken under the task's mutex.
|
||||
func convertTaskProgressToResponse(task *TaskProgressInfo) TaskInfoResponse {
|
||||
status, total, downloaded, totalFiles, downloadedFiles, startedAt, errMsg, updatedAt := task.snapshot()
|
||||
|
||||
resp := TaskInfoResponse{
|
||||
TaskID: task.TaskID,
|
||||
Type: tasktype.TaskType(task.Type),
|
||||
Status: status,
|
||||
Title: task.Title,
|
||||
Storage: task.Storage,
|
||||
Path: task.Path,
|
||||
Error: errMsg,
|
||||
CreatedAt: task.CreatedAt,
|
||||
UpdatedAt: updatedAt,
|
||||
}
|
||||
|
||||
var percent float64
|
||||
var speedMBPS float64
|
||||
if total > 0 {
|
||||
percent = float64(downloaded) * 100 / float64(total)
|
||||
} else if totalFiles > 0 {
|
||||
percent = float64(downloadedFiles) * 100 / float64(totalFiles)
|
||||
}
|
||||
if !startedAt.IsZero() {
|
||||
elapsed := time.Since(startedAt).Seconds()
|
||||
if elapsed > 0 {
|
||||
speedMBPS = float64(downloaded) / elapsed / (1024 * 1024)
|
||||
}
|
||||
}
|
||||
|
||||
if total > 0 || totalFiles > 0 {
|
||||
resp.Progress = &TaskProgress{
|
||||
TotalBytes: total,
|
||||
DownloadedBytes: downloaded,
|
||||
TotalFiles: totalFiles,
|
||||
DownloadedFiles: downloadedFiles,
|
||||
Percent: percent,
|
||||
SpeedMBPS: speedMBPS,
|
||||
}
|
||||
}
|
||||
|
||||
return resp
|
||||
}
|
||||
|
||||
// NotFoundHandler 404 处理器
|
||||
func NotFoundHandler(w http.ResponseWriter, r *http.Request) {
|
||||
WriteError(w, http.StatusNotFound, "not_found", "endpoint not found: "+r.URL.Path)
|
||||
}
|
||||
|
||||
// MethodNotAllowedHandler 405 处理器
|
||||
func MethodNotAllowedHandler(w http.ResponseWriter, r *http.Request) {
|
||||
WriteError(w, http.StatusMethodNotAllowed, "method_not_allowed", "method not allowed: "+r.Method)
|
||||
}
|
||||
695
api/handlers_test.go
Normal file
695
api/handlers_test.go
Normal file
@@ -0,0 +1,695 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
|
||||
"github.com/krau/SaveAny-Bot/pkg/taskevent"
|
||||
)
|
||||
|
||||
// setupTestServer creates a test server with handlers
|
||||
func setupTestServer(t *testing.T) (*Handlers, *TaskFactory) {
|
||||
factory := NewTaskFactory(t.Context())
|
||||
handlers := NewHandlers(factory)
|
||||
return handlers, factory
|
||||
}
|
||||
|
||||
// TestCreateTaskHandler tests the create task endpoint
|
||||
func TestCreateTaskHandler(t *testing.T) {
|
||||
handlers, _ := setupTestServer(t)
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
method string
|
||||
body any
|
||||
wantStatus int
|
||||
wantErr bool
|
||||
}{
|
||||
{
|
||||
name: "Method not allowed",
|
||||
method: http.MethodGet,
|
||||
body: nil,
|
||||
wantStatus: http.StatusMethodNotAllowed,
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "Invalid JSON body",
|
||||
method: http.MethodPost,
|
||||
body: "invalid json",
|
||||
wantStatus: http.StatusBadRequest,
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "Empty request body",
|
||||
method: http.MethodPost,
|
||||
body: CreateTaskRequest{},
|
||||
// Will fail validation for missing type
|
||||
wantStatus: http.StatusBadRequest,
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "Missing type",
|
||||
method: http.MethodPost,
|
||||
body: CreateTaskRequest{
|
||||
Storage: "test-storage",
|
||||
Path: "downloads",
|
||||
},
|
||||
wantStatus: http.StatusBadRequest,
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "Missing storage",
|
||||
method: http.MethodPost,
|
||||
body: CreateTaskRequest{
|
||||
Type: tasktype.TaskTypeDirectlinks,
|
||||
Path: "downloads",
|
||||
},
|
||||
wantStatus: http.StatusBadRequest,
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "Storage not found",
|
||||
method: http.MethodPost,
|
||||
body: CreateTaskRequest{
|
||||
Type: tasktype.TaskTypeDirectlinks,
|
||||
Storage: "non-existent-storage",
|
||||
Path: "downloads",
|
||||
Params: json.RawMessage(`{"urls":["https://example.com/file.zip"]}`),
|
||||
},
|
||||
wantStatus: http.StatusBadRequest,
|
||||
wantErr: true,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
var bodyBytes []byte
|
||||
var err error
|
||||
|
||||
if tt.body != nil {
|
||||
switch v := tt.body.(type) {
|
||||
case string:
|
||||
bodyBytes = []byte(v)
|
||||
default:
|
||||
bodyBytes, err = json.Marshal(tt.body)
|
||||
if err != nil {
|
||||
t.Fatalf("failed to marshal body: %v", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
req := httptest.NewRequest(tt.method, "/api/v1/tasks", bytes.NewReader(bodyBytes))
|
||||
if tt.body != nil {
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
}
|
||||
|
||||
rr := httptest.NewRecorder()
|
||||
handlers.CreateTaskHandler(rr, req)
|
||||
|
||||
if rr.Code != tt.wantStatus {
|
||||
t.Errorf("expected status %d, got %d", tt.wantStatus, rr.Code)
|
||||
}
|
||||
|
||||
if tt.wantErr {
|
||||
var errResp ErrorResponse
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &errResp); err != nil {
|
||||
t.Errorf("expected error response, got: %s", rr.Body.String())
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestListTasksHandler tests the list tasks endpoint
|
||||
func TestListTasksHandler(t *testing.T) {
|
||||
handlers, _ := setupTestServer(t)
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
method string
|
||||
wantStatus int
|
||||
}{
|
||||
{
|
||||
name: "Method not allowed",
|
||||
method: http.MethodPost,
|
||||
wantStatus: http.StatusMethodNotAllowed,
|
||||
},
|
||||
{
|
||||
name: "Success",
|
||||
method: http.MethodGet,
|
||||
wantStatus: http.StatusOK,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
req := httptest.NewRequest(tt.method, "/api/v1/tasks", nil)
|
||||
rr := httptest.NewRecorder()
|
||||
handlers.ListTasksHandler(rr, req)
|
||||
|
||||
if rr.Code != tt.wantStatus {
|
||||
t.Errorf("expected status %d, got %d", tt.wantStatus, rr.Code)
|
||||
}
|
||||
|
||||
if tt.method == http.MethodGet {
|
||||
var resp TasksListResponse
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
|
||||
t.Errorf("failed to unmarshal response: %v", err)
|
||||
}
|
||||
if resp.Tasks == nil {
|
||||
t.Error("expected non-nil tasks array")
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestGetTaskHandler tests the get task endpoint
|
||||
func TestGetTaskHandler(t *testing.T) {
|
||||
handlers, _ := setupTestServer(t)
|
||||
|
||||
// Register a test task
|
||||
testTaskID := "test-get-task"
|
||||
RegisterTask(testTaskID, "directlinks", "local", "downloads", "Test", "")
|
||||
defer DeleteTask(testTaskID)
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
method string
|
||||
path string
|
||||
wantStatus int
|
||||
wantFound bool
|
||||
}{
|
||||
{
|
||||
name: "Method not allowed",
|
||||
method: http.MethodPost,
|
||||
path: "/api/v1/tasks/test-id",
|
||||
wantStatus: http.StatusMethodNotAllowed,
|
||||
},
|
||||
{
|
||||
name: "Missing task ID",
|
||||
method: http.MethodGet,
|
||||
path: "/api/v1/tasks",
|
||||
wantStatus: http.StatusBadRequest,
|
||||
},
|
||||
{
|
||||
name: "Task not found",
|
||||
method: http.MethodGet,
|
||||
path: "/api/v1/tasks/non-existent-task",
|
||||
wantStatus: http.StatusNotFound,
|
||||
},
|
||||
{
|
||||
name: "Task found",
|
||||
method: http.MethodGet,
|
||||
path: "/api/v1/tasks/" + testTaskID,
|
||||
wantStatus: http.StatusOK,
|
||||
wantFound: true,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
req := httptest.NewRequest(tt.method, tt.path, nil)
|
||||
rr := httptest.NewRecorder()
|
||||
handlers.GetTaskHandler(rr, req)
|
||||
|
||||
if rr.Code != tt.wantStatus {
|
||||
t.Errorf("expected status %d, got %d", tt.wantStatus, rr.Code)
|
||||
}
|
||||
|
||||
if tt.wantFound {
|
||||
var resp TaskInfoResponse
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
|
||||
t.Errorf("failed to unmarshal response: %v", err)
|
||||
}
|
||||
if resp.TaskID != testTaskID {
|
||||
t.Errorf("expected task ID %s, got %s", testTaskID, resp.TaskID)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestCancelTaskHandler tests the cancel task endpoint
|
||||
func TestCancelTaskHandler(t *testing.T) {
|
||||
handlers, _ := setupTestServer(t)
|
||||
|
||||
// Register a test task
|
||||
testTaskID := "test-cancel-task"
|
||||
RegisterTask(testTaskID, "directlinks", "local", "downloads", "Test", "")
|
||||
defer DeleteTask(testTaskID)
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
method string
|
||||
path string
|
||||
wantStatus int
|
||||
skipCore bool // Skip if core is not initialized
|
||||
}{
|
||||
{
|
||||
name: "Method not allowed",
|
||||
method: http.MethodGet,
|
||||
path: "/api/v1/tasks/test-id",
|
||||
wantStatus: http.StatusMethodNotAllowed,
|
||||
},
|
||||
{
|
||||
name: "Missing task ID",
|
||||
method: http.MethodDelete,
|
||||
path: "/api/v1/tasks",
|
||||
wantStatus: http.StatusBadRequest,
|
||||
},
|
||||
{
|
||||
name: "Task not found",
|
||||
method: http.MethodDelete,
|
||||
path: "/api/v1/tasks/non-existent-task",
|
||||
wantStatus: http.StatusNotFound,
|
||||
},
|
||||
{
|
||||
name: "Cancel task",
|
||||
method: http.MethodDelete,
|
||||
path: "/api/v1/tasks/" + testTaskID,
|
||||
wantStatus: http.StatusOK,
|
||||
skipCore: true, // Requires initialized core task queue
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
if tt.skipCore {
|
||||
t.Skip("Skipping test: requires initialized core")
|
||||
}
|
||||
|
||||
req := httptest.NewRequest(tt.method, tt.path, nil)
|
||||
rr := httptest.NewRecorder()
|
||||
handlers.CancelTaskHandler(rr, req)
|
||||
|
||||
if rr.Code != tt.wantStatus {
|
||||
t.Errorf("expected status %d, got %d", tt.wantStatus, rr.Code)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestListStoragesHandler tests the list storages endpoint
|
||||
func TestListStoragesHandler(t *testing.T) {
|
||||
handlers, _ := setupTestServer(t)
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
method string
|
||||
wantStatus int
|
||||
}{
|
||||
{
|
||||
name: "Method not allowed",
|
||||
method: http.MethodPost,
|
||||
wantStatus: http.StatusMethodNotAllowed,
|
||||
},
|
||||
{
|
||||
name: "Success",
|
||||
method: http.MethodGet,
|
||||
wantStatus: http.StatusOK,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
req := httptest.NewRequest(tt.method, "/api/v1/storages", nil)
|
||||
rr := httptest.NewRecorder()
|
||||
handlers.ListStoragesHandler(rr, req)
|
||||
|
||||
if rr.Code != tt.wantStatus {
|
||||
t.Errorf("expected status %d, got %d", tt.wantStatus, rr.Code)
|
||||
}
|
||||
|
||||
if tt.method == http.MethodGet {
|
||||
var resp StoragesResponse
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
|
||||
t.Errorf("failed to unmarshal response: %v", err)
|
||||
}
|
||||
if resp.Storages == nil {
|
||||
t.Error("expected non-nil storages array")
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestConcurrentProgressStore tests concurrent access to progress store
|
||||
func TestConcurrentProgressStore(t *testing.T) {
|
||||
// Clear store before test
|
||||
t.Cleanup(func() {
|
||||
tasks := GetAllTasks()
|
||||
for _, task := range tasks {
|
||||
if strings.HasPrefix(task.TaskID, "concurrent-test-") {
|
||||
DeleteTask(task.TaskID)
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
var wg sync.WaitGroup
|
||||
numGoroutines := 100
|
||||
|
||||
// Concurrent registrations
|
||||
for i := range numGoroutines {
|
||||
wg.Add(1)
|
||||
go func(id int) {
|
||||
defer wg.Done()
|
||||
taskID := fmt.Sprintf("concurrent-test-%d", id)
|
||||
RegisterTask(taskID, "directlinks", "local", "downloads", "Test", "")
|
||||
}(i)
|
||||
}
|
||||
|
||||
// Concurrent reads
|
||||
for i := range numGoroutines {
|
||||
wg.Add(1)
|
||||
go func(id int) {
|
||||
defer wg.Done()
|
||||
taskID := fmt.Sprintf("concurrent-test-%d", id)
|
||||
GetTask(taskID)
|
||||
}(i)
|
||||
}
|
||||
|
||||
// Concurrent updates
|
||||
for i := range numGoroutines {
|
||||
wg.Add(1)
|
||||
go func(id int) {
|
||||
defer wg.Done()
|
||||
taskID := fmt.Sprintf("concurrent-test-%d", id)
|
||||
info, ok := GetTask(taskID)
|
||||
if ok {
|
||||
info.UpdateStatus(TaskStatusRunning)
|
||||
}
|
||||
}(i)
|
||||
}
|
||||
|
||||
wg.Wait()
|
||||
|
||||
// Verify all tasks exist
|
||||
for i := range numGoroutines {
|
||||
taskID := fmt.Sprintf("concurrent-test-%d", i)
|
||||
if _, ok := GetTask(taskID); !ok {
|
||||
t.Errorf("task %s not found after concurrent operations", taskID)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestProgressTrackerConcurrentUpdates tests concurrent progress updates
|
||||
func TestProgressTrackerConcurrentUpdates(t *testing.T) {
|
||||
info := RegisterTask("concurrent-progress", "directlinks", "local", "downloads", "Test", "")
|
||||
info.Emit(taskevent.Event{TaskID: "concurrent-progress", Phase: taskevent.PhaseStart, TotalBytes: 10000})
|
||||
|
||||
var wg sync.WaitGroup
|
||||
numGoroutines := 50
|
||||
updatesPerGoroutine := 100
|
||||
|
||||
// Concurrent progress updates via the Sink interface
|
||||
for i := range numGoroutines {
|
||||
wg.Add(1)
|
||||
go func(id int) {
|
||||
defer wg.Done()
|
||||
for j := range updatesPerGoroutine {
|
||||
info.Emit(taskevent.Event{
|
||||
TaskID: "concurrent-progress",
|
||||
Phase: taskevent.PhaseProgress,
|
||||
DownloadedBytes: int64(id*updatesPerGoroutine + j),
|
||||
TotalBytes: 10000,
|
||||
})
|
||||
}
|
||||
}(i)
|
||||
}
|
||||
|
||||
wg.Wait()
|
||||
|
||||
status, _, downloaded, _, _, _, _, _ := info.snapshot()
|
||||
if status != TaskStatusRunning {
|
||||
t.Errorf("expected status Running after concurrent updates, got %s", status)
|
||||
}
|
||||
if downloaded <= 0 {
|
||||
t.Errorf("expected downloaded bytes > 0 after concurrent updates, got %d", downloaded)
|
||||
}
|
||||
}
|
||||
|
||||
// TestTaskFactoryValidation tests TaskFactory parameter validation
|
||||
func TestTaskFactoryValidation(t *testing.T) {
|
||||
factory := NewTaskFactory(context.Background())
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
request *CreateTaskRequest
|
||||
wantErr bool
|
||||
errMsg string
|
||||
}{
|
||||
{
|
||||
name: "Storage not found",
|
||||
request: &CreateTaskRequest{
|
||||
Type: tasktype.TaskTypeDirectlinks,
|
||||
Storage: "non-existent",
|
||||
Path: "downloads",
|
||||
Params: json.RawMessage(`{"urls":["https://example.com/file.zip"]}`),
|
||||
},
|
||||
wantErr: true,
|
||||
errMsg: "storage not found",
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
_, err := factory.CreateTask(tt.request)
|
||||
if (err != nil) != tt.wantErr {
|
||||
t.Errorf("CreateTask() error = %v, wantErr %v", err, tt.wantErr)
|
||||
return
|
||||
}
|
||||
if err != nil && tt.errMsg != "" {
|
||||
if !strings.Contains(strings.ToLower(err.Error()), tt.errMsg) {
|
||||
t.Errorf("error message %q does not contain %q", err.Error(), tt.errMsg)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestEdgeCases tests various edge cases
|
||||
func TestEdgeCases(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
fn func(t *testing.T)
|
||||
}{
|
||||
{
|
||||
name: "Empty request body",
|
||||
fn: func(t *testing.T) {
|
||||
handlers, _ := setupTestServer(t)
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/v1/tasks", nil)
|
||||
rr := httptest.NewRecorder()
|
||||
handlers.CreateTaskHandler(rr, req)
|
||||
if rr.Code != http.StatusBadRequest {
|
||||
t.Errorf("expected %d, got %d", http.StatusBadRequest, rr.Code)
|
||||
}
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "Very long task ID in path",
|
||||
fn: func(t *testing.T) {
|
||||
handlers, _ := setupTestServer(t)
|
||||
longID := strings.Repeat("a", 1000)
|
||||
req := httptest.NewRequest(http.MethodGet, "/api/v1/tasks/"+longID, nil)
|
||||
rr := httptest.NewRecorder()
|
||||
handlers.GetTaskHandler(rr, req)
|
||||
if rr.Code != http.StatusNotFound {
|
||||
t.Errorf("expected %d, got %d", http.StatusNotFound, rr.Code)
|
||||
}
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "Path with special characters",
|
||||
fn: func(t *testing.T) {
|
||||
path := "/api/v1/tasks/test%20id/with/slashes"
|
||||
got := extractTaskIDFromPath(path)
|
||||
expected := "test%20id"
|
||||
if got != expected {
|
||||
t.Errorf("expected %q, got %q", expected, got)
|
||||
}
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "Double slashes in path",
|
||||
fn: func(t *testing.T) {
|
||||
path := "/api/v1/tasks//task-id"
|
||||
got := extractTaskIDFromPath(path)
|
||||
expected := ""
|
||||
if got != expected {
|
||||
t.Errorf("expected %q, got %q", expected, got)
|
||||
}
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "Progress tracker with empty webhook",
|
||||
fn: func(t *testing.T) {
|
||||
info := RegisterTask("test-empty-webhook", "type", "storage", "path", "title", "")
|
||||
if info.Webhook != "" {
|
||||
t.Error("expected empty webhook")
|
||||
}
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, tt.fn)
|
||||
}
|
||||
}
|
||||
|
||||
// TestHealthCheckHandler tests the health check endpoint
|
||||
func TestHealthCheckHandler(t *testing.T) {
|
||||
handlers, _ := setupTestServer(t)
|
||||
|
||||
req := httptest.NewRequest(http.MethodGet, "/health", nil)
|
||||
rr := httptest.NewRecorder()
|
||||
handlers.HealthCheckHandler(rr, req)
|
||||
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Errorf("expected status %d, got %d", http.StatusOK, rr.Code)
|
||||
}
|
||||
|
||||
var resp map[string]string
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
|
||||
t.Fatalf("failed to unmarshal response: %v", err)
|
||||
}
|
||||
|
||||
if resp["status"] != "ok" {
|
||||
t.Errorf("expected status 'ok', got %q", resp["status"])
|
||||
}
|
||||
}
|
||||
|
||||
// TestGetTaskTypesHandler tests the task types endpoint
|
||||
func TestGetTaskTypesHandler(t *testing.T) {
|
||||
handlers, _ := setupTestServer(t)
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
method string
|
||||
wantStatus int
|
||||
}{
|
||||
{
|
||||
name: "Method not allowed",
|
||||
method: http.MethodPost,
|
||||
wantStatus: http.StatusMethodNotAllowed,
|
||||
},
|
||||
{
|
||||
name: "Success",
|
||||
method: http.MethodGet,
|
||||
wantStatus: http.StatusOK,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
req := httptest.NewRequest(tt.method, "/api/v1/task-types", nil)
|
||||
rr := httptest.NewRecorder()
|
||||
handlers.GetTaskTypesHandler(rr, req)
|
||||
|
||||
if rr.Code != tt.wantStatus {
|
||||
t.Errorf("expected status %d, got %d", tt.wantStatus, rr.Code)
|
||||
}
|
||||
|
||||
if tt.method == http.MethodGet {
|
||||
var resp map[string]any
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
|
||||
t.Errorf("failed to unmarshal response: %v", err)
|
||||
}
|
||||
if _, ok := resp["types"]; !ok {
|
||||
t.Error("expected 'types' field in response")
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestNotFoundHandler tests the 404 handler
|
||||
func TestNotFoundHandler(t *testing.T) {
|
||||
req := httptest.NewRequest(http.MethodGet, "/non-existent-path", nil)
|
||||
rr := httptest.NewRecorder()
|
||||
NotFoundHandler(rr, req)
|
||||
|
||||
if rr.Code != http.StatusNotFound {
|
||||
t.Errorf("expected status %d, got %d", http.StatusNotFound, rr.Code)
|
||||
}
|
||||
|
||||
var resp ErrorResponse
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
|
||||
t.Fatalf("failed to unmarshal response: %v", err)
|
||||
}
|
||||
|
||||
if resp.Error != "not_found" {
|
||||
t.Errorf("expected error 'not_found', got %q", resp.Error)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMethodNotAllowedHandler tests the 405 handler
|
||||
func TestMethodNotAllowedHandler(t *testing.T) {
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/v1/tasks", nil)
|
||||
rr := httptest.NewRecorder()
|
||||
MethodNotAllowedHandler(rr, req)
|
||||
|
||||
if rr.Code != http.StatusMethodNotAllowed {
|
||||
t.Errorf("expected status %d, got %d", http.StatusMethodNotAllowed, rr.Code)
|
||||
}
|
||||
|
||||
var resp ErrorResponse
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
|
||||
t.Fatalf("failed to unmarshal response: %v", err)
|
||||
}
|
||||
|
||||
if resp.Error != "method_not_allowed" {
|
||||
t.Errorf("expected error 'method_not_allowed', got %q", resp.Error)
|
||||
}
|
||||
}
|
||||
|
||||
// TestTaskProgressInfoTimeUpdate tests that timestamps are updated correctly
|
||||
func TestTaskProgressInfoTimeUpdate(t *testing.T) {
|
||||
info := RegisterTask("time-test", "directlinks", "local", "downloads", "Test", "")
|
||||
defer DeleteTask("time-test")
|
||||
|
||||
originalTime := info.UpdatedAt
|
||||
time.Sleep(10 * time.Millisecond) // Ensure time difference
|
||||
|
||||
info.UpdateStatus(TaskStatusRunning)
|
||||
if !info.UpdatedAt.After(originalTime) {
|
||||
t.Error("expected UpdatedAt to be updated")
|
||||
}
|
||||
}
|
||||
|
||||
// TestWebhookPayloadWithNilCompletedAt tests webhook payload with nil completed_at
|
||||
func TestWebhookPayloadWithNilCompletedAt(t *testing.T) {
|
||||
payload := WebhookPayload{
|
||||
TaskID: "test-id",
|
||||
Type: "directlinks",
|
||||
Status: TaskStatusRunning,
|
||||
Storage: "local",
|
||||
Path: "downloads/file.zip",
|
||||
CompletedAt: nil,
|
||||
Error: "",
|
||||
}
|
||||
|
||||
data, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
t.Fatalf("failed to marshal: %v", err)
|
||||
}
|
||||
|
||||
var decoded map[string]any
|
||||
if err := json.Unmarshal(data, &decoded); err != nil {
|
||||
t.Fatalf("failed to unmarshal: %v", err)
|
||||
}
|
||||
|
||||
// completed_at should be omitted when nil
|
||||
if _, ok := decoded["completed_at"]; ok {
|
||||
t.Error("expected completed_at to be omitted when nil")
|
||||
}
|
||||
}
|
||||
201
api/progress.go
Normal file
201
api/progress.go
Normal file
@@ -0,0 +1,201 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/pkg/taskevent"
|
||||
)
|
||||
|
||||
// TaskProgressInfo stores the progress of an API-submitted task. All fields are
|
||||
// guarded by mu. It implements taskevent.Sink so the task layer can update it
|
||||
// without knowing about the API.
|
||||
type TaskProgressInfo struct {
|
||||
mu sync.Mutex
|
||||
TaskID string
|
||||
Type string
|
||||
Status TaskStatus
|
||||
Title string
|
||||
TotalBytes int64
|
||||
DownloadedBytes int64
|
||||
TotalFiles int
|
||||
DownloadedFiles int
|
||||
Storage string
|
||||
Path string
|
||||
Error string
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
StartedAt time.Time
|
||||
Webhook string
|
||||
webhookNotified bool
|
||||
}
|
||||
|
||||
// progressStore holds all API tasks. Entries are removed a fixed duration after
|
||||
// they reach a terminal state to bound memory usage.
|
||||
type progressStore struct {
|
||||
mu sync.RWMutex
|
||||
tasks map[string]*TaskProgressInfo
|
||||
retention time.Duration
|
||||
}
|
||||
|
||||
var store = &progressStore{
|
||||
tasks: make(map[string]*TaskProgressInfo),
|
||||
retention: 24 * time.Hour,
|
||||
}
|
||||
|
||||
// RegisterTask registers a new API task and returns its progress info.
|
||||
func RegisterTask(taskID, taskType, storage, path, title, webhook string) *TaskProgressInfo {
|
||||
info := &TaskProgressInfo{
|
||||
TaskID: taskID,
|
||||
Type: taskType,
|
||||
Status: TaskStatusQueued,
|
||||
Title: title,
|
||||
Storage: storage,
|
||||
Path: path,
|
||||
CreatedAt: time.Now(),
|
||||
UpdatedAt: time.Now(),
|
||||
Webhook: webhook,
|
||||
}
|
||||
|
||||
store.mu.Lock()
|
||||
store.tasks[taskID] = info
|
||||
store.mu.Unlock()
|
||||
|
||||
return info
|
||||
}
|
||||
|
||||
// GetTask returns the progress info for a task.
|
||||
func GetTask(taskID string) (*TaskProgressInfo, bool) {
|
||||
store.mu.RLock()
|
||||
defer store.mu.RUnlock()
|
||||
info, ok := store.tasks[taskID]
|
||||
return info, ok
|
||||
}
|
||||
|
||||
// GetAllTasks returns all tracked tasks.
|
||||
func GetAllTasks() []*TaskProgressInfo {
|
||||
store.mu.RLock()
|
||||
defer store.mu.RUnlock()
|
||||
|
||||
tasks := make([]*TaskProgressInfo, 0, len(store.tasks))
|
||||
for _, info := range store.tasks {
|
||||
tasks = append(tasks, info)
|
||||
}
|
||||
return tasks
|
||||
}
|
||||
|
||||
// DeleteTask removes a task record.
|
||||
func DeleteTask(taskID string) {
|
||||
store.mu.Lock()
|
||||
defer store.mu.Unlock()
|
||||
delete(store.tasks, taskID)
|
||||
}
|
||||
|
||||
// CleanupExpired removes tasks that reached a terminal state more than the
|
||||
// store's retention duration ago. It is safe to call periodically.
|
||||
func CleanupExpired() {
|
||||
now := time.Now()
|
||||
store.mu.Lock()
|
||||
defer store.mu.Unlock()
|
||||
for id, info := range store.tasks {
|
||||
info.mu.Lock()
|
||||
terminal := info.Status == TaskStatusCompleted || info.Status == TaskStatusFailed || info.Status == TaskStatusCancelled
|
||||
stale := terminal && now.Sub(info.UpdatedAt) > store.retention
|
||||
info.mu.Unlock()
|
||||
if stale {
|
||||
delete(store.tasks, id)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// StartCleanupLoop runs CleanupExpired on a fixed interval until ctx is done.
|
||||
// It should be started once during API server initialization.
|
||||
func StartCleanupLoop(ctx interface{ Done() <-chan struct{} }) {
|
||||
go func() {
|
||||
ticker := time.NewTicker(10 * time.Minute)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-ticker.C:
|
||||
CleanupExpired()
|
||||
}
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
// UpdateStatus sets the task status.
|
||||
func (t *TaskProgressInfo) UpdateStatus(status TaskStatus) {
|
||||
t.mu.Lock()
|
||||
t.Status = status
|
||||
t.UpdatedAt = time.Now()
|
||||
if status == TaskStatusRunning && t.StartedAt.IsZero() {
|
||||
t.StartedAt = t.UpdatedAt
|
||||
}
|
||||
t.mu.Unlock()
|
||||
}
|
||||
|
||||
// SetError marks the task failed with an error message.
|
||||
func (t *TaskProgressInfo) SetError(err string) {
|
||||
t.mu.Lock()
|
||||
t.Error = err
|
||||
t.Status = TaskStatusFailed
|
||||
t.UpdatedAt = time.Now()
|
||||
t.mu.Unlock()
|
||||
}
|
||||
|
||||
// snapshot returns a point-in-time copy of the fields needed to render a
|
||||
// response, so callers never touch the mutex directly.
|
||||
func (t *TaskProgressInfo) snapshot() (status TaskStatus, total, downloaded int64, totalFiles, downloadedFiles int, startedAt time.Time, err string, updatedAt time.Time) {
|
||||
t.mu.Lock()
|
||||
defer t.mu.Unlock()
|
||||
return t.Status, t.TotalBytes, t.DownloadedBytes, t.TotalFiles, t.DownloadedFiles, t.StartedAt, t.Error, t.UpdatedAt
|
||||
}
|
||||
|
||||
// Emit implements taskevent.Sink. It translates task lifecycle events into
|
||||
// status/progress updates and fires the webhook on terminal transitions.
|
||||
func (t *TaskProgressInfo) Emit(e taskevent.Event) {
|
||||
t.mu.Lock()
|
||||
switch e.Phase {
|
||||
case taskevent.PhaseStart:
|
||||
t.Status = TaskStatusRunning
|
||||
if t.StartedAt.IsZero() {
|
||||
t.StartedAt = time.Now()
|
||||
}
|
||||
if e.TotalBytes > 0 {
|
||||
t.TotalBytes = e.TotalBytes
|
||||
}
|
||||
case taskevent.PhaseProgress:
|
||||
t.Status = TaskStatusRunning
|
||||
if e.TotalBytes > 0 {
|
||||
t.TotalBytes = e.TotalBytes
|
||||
}
|
||||
t.DownloadedBytes = e.DownloadedBytes
|
||||
if e.TotalFiles > 0 {
|
||||
t.TotalFiles = e.TotalFiles
|
||||
}
|
||||
if e.DownloadedFiles > 0 {
|
||||
t.DownloadedFiles = e.DownloadedFiles
|
||||
}
|
||||
case taskevent.PhaseDone:
|
||||
if e.Err != nil {
|
||||
t.Status = TaskStatusFailed
|
||||
t.Error = e.Err.Error()
|
||||
} else {
|
||||
t.Status = TaskStatusCompleted
|
||||
}
|
||||
}
|
||||
t.UpdatedAt = time.Now()
|
||||
notify := t.Webhook != "" && !t.webhookNotified && (t.Status == TaskStatusCompleted || t.Status == TaskStatusFailed)
|
||||
if notify {
|
||||
t.webhookNotified = true
|
||||
}
|
||||
t.mu.Unlock()
|
||||
|
||||
if notify {
|
||||
payload := CreateWebhookPayload(t.TaskID, t.Type, t.Status, t.Storage, t.Path, e.Err)
|
||||
SendWebhook(context.Background(), payload)
|
||||
}
|
||||
}
|
||||
177
api/server.go
Normal file
177
api/server.go
Normal file
@@ -0,0 +1,177 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
)
|
||||
|
||||
// Server API 服务器
|
||||
type Server struct {
|
||||
httpServer *http.Server
|
||||
factory *TaskFactory
|
||||
}
|
||||
|
||||
// NewServer 创建新的 API 服务器
|
||||
func NewServer(ctx context.Context) *Server {
|
||||
cfg := config.C().API
|
||||
|
||||
factory := NewTaskFactory(ctx)
|
||||
handlers := NewHandlers(factory)
|
||||
|
||||
// 设置路由
|
||||
mux := http.NewServeMux()
|
||||
|
||||
// 健康检查
|
||||
mux.HandleFunc("/health", handlers.HealthCheckHandler)
|
||||
|
||||
// API v1 路由
|
||||
mux.HandleFunc("/api/v1/tasks", func(w http.ResponseWriter, r *http.Request) {
|
||||
switch r.Method {
|
||||
case http.MethodGet:
|
||||
handlers.ListTasksHandler(w, r)
|
||||
case http.MethodPost:
|
||||
handlers.CreateTaskHandler(w, r)
|
||||
default:
|
||||
MethodNotAllowedHandler(w, r)
|
||||
}
|
||||
})
|
||||
mux.HandleFunc("/api/v1/tasks/", func(w http.ResponseWriter, r *http.Request) {
|
||||
// 根据方法和路径分发
|
||||
switch r.Method {
|
||||
case http.MethodGet:
|
||||
handlers.GetTaskHandler(w, r)
|
||||
case http.MethodDelete:
|
||||
handlers.CancelTaskHandler(w, r)
|
||||
default:
|
||||
MethodNotAllowedHandler(w, r)
|
||||
}
|
||||
})
|
||||
mux.HandleFunc("/api/v1/storages", handlers.ListStoragesHandler)
|
||||
mux.HandleFunc("/api/v1/task-types", handlers.GetTaskTypesHandler)
|
||||
|
||||
// 404 处理
|
||||
mux.HandleFunc("/", NotFoundHandler)
|
||||
|
||||
// Apply middleware chain.
|
||||
var handler http.Handler = mux
|
||||
|
||||
// Apply auth middleware when a token is configured.
|
||||
token := cfg.Token
|
||||
if token != "" {
|
||||
handler = AuthMiddleware()(handler)
|
||||
}
|
||||
|
||||
// Add logging middleware.
|
||||
handler = loggingMiddleware(handler)
|
||||
|
||||
// Add recovery middleware.
|
||||
handler = recoveryMiddleware(handler)
|
||||
|
||||
return &Server{
|
||||
httpServer: &http.Server{
|
||||
Addr: fmt.Sprintf("%s:%d", cfg.Host, cfg.Port),
|
||||
Handler: handler,
|
||||
ReadTimeout: 30 * time.Second,
|
||||
WriteTimeout: 30 * time.Second,
|
||||
IdleTimeout: 120 * time.Second,
|
||||
},
|
||||
factory: factory,
|
||||
}
|
||||
}
|
||||
|
||||
// Start 启动服务器
|
||||
func (s *Server) Start(ctx context.Context) error {
|
||||
logger := log.FromContext(ctx).With("module", "api")
|
||||
|
||||
logger.Infof("Starting API server on %s", s.httpServer.Addr)
|
||||
|
||||
// Bind synchronously so listen failures are returned to the caller.
|
||||
ln, err := net.Listen("tcp", s.httpServer.Addr)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to listen on %s: %w", s.httpServer.Addr, err)
|
||||
}
|
||||
|
||||
// 在 goroutine 中启动服务器
|
||||
go func() {
|
||||
if err := s.httpServer.Serve(ln); err != nil && err != http.ErrServerClosed {
|
||||
logger.Errorf("API server error: %v", err)
|
||||
}
|
||||
}()
|
||||
|
||||
// 监听 context 取消
|
||||
go func() {
|
||||
<-ctx.Done()
|
||||
shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
if err := s.httpServer.Shutdown(shutdownCtx); err != nil {
|
||||
logger.Errorf("API server shutdown error: %v", err)
|
||||
}
|
||||
}()
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// loggingMiddleware 日志中间件
|
||||
func loggingMiddleware(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
start := time.Now()
|
||||
|
||||
// 包装 ResponseWriter 以获取状态码
|
||||
wrapped := &responseWriter{ResponseWriter: w, statusCode: http.StatusOK}
|
||||
|
||||
next.ServeHTTP(wrapped, r)
|
||||
|
||||
log.Infof("%s %s %d %s", r.Method, r.URL.Path, wrapped.statusCode, time.Since(start))
|
||||
})
|
||||
}
|
||||
|
||||
// recoveryMiddleware 恢复中间件
|
||||
func recoveryMiddleware(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
defer func() {
|
||||
if err := recover(); err != nil {
|
||||
log.Errorf("Panic recovered: %v", err)
|
||||
WriteError(w, http.StatusInternalServerError, "internal_error", "internal server error")
|
||||
}
|
||||
}()
|
||||
next.ServeHTTP(w, r)
|
||||
})
|
||||
}
|
||||
|
||||
// responseWriter 包装 http.ResponseWriter 以捕获状态码
|
||||
type responseWriter struct {
|
||||
http.ResponseWriter
|
||||
statusCode int
|
||||
}
|
||||
|
||||
func (rw *responseWriter) WriteHeader(code int) {
|
||||
rw.statusCode = code
|
||||
rw.ResponseWriter.WriteHeader(code)
|
||||
}
|
||||
|
||||
// Start initializes and starts the API server. It refuses to start without a
|
||||
// token, since an open download proxy is a security risk.
|
||||
func Start(ctx context.Context) error {
|
||||
cfg := config.C().API
|
||||
|
||||
if !cfg.Enable {
|
||||
return nil
|
||||
}
|
||||
|
||||
if cfg.Token == "" {
|
||||
return fmt.Errorf("API server is enabled but no token is set; refusing to start insecurely")
|
||||
}
|
||||
|
||||
server := NewServer(ctx)
|
||||
if err := server.Start(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
StartCleanupLoop(ctx)
|
||||
return nil
|
||||
}
|
||||
283
api/tgfiles.go
Normal file
283
api/tgfiles.go
Normal file
@@ -0,0 +1,283 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/url"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"github.com/celestix/gotgproto/ext"
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/gotd/td/tg"
|
||||
"github.com/krau/SaveAny-Bot/client/bot"
|
||||
userclient "github.com/krau/SaveAny-Bot/client/user"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
|
||||
"github.com/krau/SaveAny-Bot/pkg/tfile"
|
||||
)
|
||||
|
||||
// MessageContext 保存消息和获取它所用的 context
|
||||
type MessageContext struct {
|
||||
Message *tg.Message
|
||||
Client *ext.Context
|
||||
}
|
||||
|
||||
// getClientContext 获取可用的客户端上下文
|
||||
// 优先使用 Bot,失败后回退到 Userbot
|
||||
func getClientContext() (*ext.Context, error) {
|
||||
// 首先尝试获取 Bot context
|
||||
if botCtx := bot.ExtContext(); botCtx != nil {
|
||||
return botCtx, nil
|
||||
}
|
||||
|
||||
// 回退到 Userbot
|
||||
if uc := userclient.GetCtx(); uc != nil {
|
||||
return uc, nil
|
||||
}
|
||||
|
||||
return nil, fmt.Errorf("no client available (bot and userbot are not initialized)")
|
||||
}
|
||||
|
||||
// resolveChatID 解析聊天 ID
|
||||
func resolveChatID(_ context.Context, idOrUsername string) (int64, error) {
|
||||
// 如果是数字 ID
|
||||
if id, err := strconv.ParseInt(idOrUsername, 10, 64); err == nil {
|
||||
// 私有频道 ID 需要加上 -100 前缀
|
||||
if id > 0 {
|
||||
return -1000000000000 - id, nil
|
||||
}
|
||||
return id, nil
|
||||
}
|
||||
|
||||
// 获取可用的客户端上下文
|
||||
clientCtx, err := getClientContext()
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
// 使用 tgutil 的 ParseChatID
|
||||
return tgutil.ParseChatID(clientCtx, idOrUsername)
|
||||
}
|
||||
|
||||
// ParseMessageLink 解析 Telegram 消息链接
|
||||
// 支持的域名: t.me, telegram.me
|
||||
// 支持格式:
|
||||
// - https://t.me/username/123
|
||||
// - https://t.me/c/123456789/123
|
||||
// - https://t.me/c/123456789/111/456 (topic id)
|
||||
// - https://t.me/username/123?comment=2 (评论)
|
||||
func ParseMessageLink(ctx context.Context, link string) (int64, int, error) {
|
||||
u, err := url.Parse(link)
|
||||
if err != nil {
|
||||
return 0, 0, fmt.Errorf("invalid URL: %w", err)
|
||||
}
|
||||
paths := strings.Split(strings.TrimPrefix(u.Path, "/"), "/")
|
||||
|
||||
if cmt := u.Query().Get("comment"); cmt != "" {
|
||||
// 频道评论的消息链接
|
||||
if len(paths) < 1 {
|
||||
return 0, 0, fmt.Errorf("invalid message link format: %s", link)
|
||||
}
|
||||
// 简化处理:返回错误,提示不支持评论链接
|
||||
return 0, 0, fmt.Errorf("comment links are not supported")
|
||||
}
|
||||
|
||||
switch len(paths) {
|
||||
case 2: // https://t.me/username/123
|
||||
chatID, err := resolveChatID(ctx, paths[0])
|
||||
if err != nil {
|
||||
return 0, 0, fmt.Errorf("failed to resolve chat ID: %w", err)
|
||||
}
|
||||
msgID, err := strconv.Atoi(paths[1])
|
||||
if err != nil {
|
||||
return 0, 0, fmt.Errorf("failed to parse message ID: %w", err)
|
||||
}
|
||||
return chatID, msgID, nil
|
||||
case 3:
|
||||
// https://t.me/c/123456789/123
|
||||
// https://t.me/username/123/456 , 123: topic id
|
||||
chatPart, msgPart := paths[1], paths[2]
|
||||
if paths[0] != "c" {
|
||||
chatPart = paths[0]
|
||||
}
|
||||
chatID, err := resolveChatID(ctx, chatPart)
|
||||
if err != nil {
|
||||
return 0, 0, fmt.Errorf("failed to resolve chat ID: %w", err)
|
||||
}
|
||||
msgID, err := strconv.Atoi(msgPart)
|
||||
if err != nil {
|
||||
return 0, 0, fmt.Errorf("failed to parse message ID: %w", err)
|
||||
}
|
||||
return chatID, msgID, nil
|
||||
case 4:
|
||||
// https://t.me/c/123456789/111/456 111: topic id
|
||||
if paths[0] != "c" {
|
||||
return 0, 0, fmt.Errorf("invalid message link format: %s", link)
|
||||
}
|
||||
chatID, err := resolveChatID(ctx, paths[1])
|
||||
if err != nil {
|
||||
return 0, 0, fmt.Errorf("failed to resolve chat ID: %w", err)
|
||||
}
|
||||
msgID, err := strconv.Atoi(paths[3])
|
||||
if err != nil {
|
||||
return 0, 0, fmt.Errorf("failed to parse message ID: %w", err)
|
||||
}
|
||||
return chatID, msgID, nil
|
||||
}
|
||||
return 0, 0, fmt.Errorf("invalid message link format: %s", link)
|
||||
}
|
||||
|
||||
// getMessageWithContext 通过 ID 获取消息,返回消息和使用的 context
|
||||
// 确保消息获取和后续文件创建使用同一个 context
|
||||
func getMessageWithContext(_ context.Context, chatID int64, msgID int) (*MessageContext, error) {
|
||||
// 首先尝试使用 Bot
|
||||
if botCtx := bot.ExtContext(); botCtx != nil {
|
||||
msg, err := tgutil.GetMessageByID(botCtx, chatID, msgID)
|
||||
if err == nil {
|
||||
return &MessageContext{Message: msg, Client: botCtx}, nil
|
||||
}
|
||||
}
|
||||
|
||||
// 回退到 Userbot
|
||||
uc := userclient.GetCtx()
|
||||
if uc == nil {
|
||||
return nil, fmt.Errorf("userbot not initialized and bot cannot access this message")
|
||||
}
|
||||
|
||||
msg, err := tgutil.GetMessageByID(uc, chatID, msgID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &MessageContext{Message: msg, Client: uc}, nil
|
||||
}
|
||||
|
||||
// getGroupedMessagesWithContext 获取媒体组消息,返回消息列表和使用的 context
|
||||
// 确保消息获取和后续文件创建使用同一个 context
|
||||
func getGroupedMessagesWithContext(ctx *MessageContext, chatID int64) ([]*tg.Message, error) {
|
||||
msg := ctx.Message
|
||||
clientCtx := ctx.Client
|
||||
|
||||
groupID, ok := msg.GetGroupedID()
|
||||
if !ok || groupID == 0 {
|
||||
return []*tg.Message{msg}, nil
|
||||
}
|
||||
|
||||
// 使用获取原始消息的同一个 client 获取媒体组
|
||||
msgs, err := tgutil.GetGroupedMessages(clientCtx, chatID, msg)
|
||||
if err != nil || len(msgs) == 0 {
|
||||
// 如果获取失败,至少返回原始消息
|
||||
return []*tg.Message{msg}, nil
|
||||
}
|
||||
|
||||
return msgs, nil
|
||||
}
|
||||
|
||||
// ExtractFilesFromLinks 从消息链接中提取文件
|
||||
// 每个文件的处理流程:解析链接 -> 获取消息 -> 获取媒体组 -> 创建文件对象
|
||||
// 对于单个文件,全程使用同一个 client context,不会交叉
|
||||
func ExtractFilesFromLinks(ctx context.Context, links []string) ([]tfile.TGFileMessage, error) {
|
||||
logger := log.FromContext(ctx)
|
||||
var files []tfile.TGFileMessage
|
||||
|
||||
for _, link := range links {
|
||||
link = strings.TrimSpace(link)
|
||||
if link == "" {
|
||||
continue
|
||||
}
|
||||
|
||||
// 验证链接格式
|
||||
if !isValidMessageLink(link) {
|
||||
logger.Errorf("Invalid message link format: %s", link)
|
||||
continue
|
||||
}
|
||||
|
||||
chatID, msgID, err := ParseMessageLink(ctx, link)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to parse message link %s: %v", link, err)
|
||||
continue
|
||||
}
|
||||
|
||||
// 解析链接 URL 检查是否有 single 参数
|
||||
u, _ := url.Parse(link)
|
||||
single := u != nil && u.Query().Has("single")
|
||||
|
||||
// 获取消息和使用的 context(Bot 优先,失败回退 Userbot)
|
||||
msgCtx, err := getMessageWithContext(ctx, chatID, msgID)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to get message %d from chat %d: %v", msgID, chatID, err)
|
||||
continue
|
||||
}
|
||||
|
||||
msg := msgCtx.Message
|
||||
clientCtx := msgCtx.Client
|
||||
|
||||
if msg.Media == nil {
|
||||
logger.Warnf("Message %d has no media", msgID)
|
||||
continue
|
||||
}
|
||||
|
||||
media, ok := msg.GetMedia()
|
||||
if !ok {
|
||||
logger.Warnf("Failed to get media from message %d", msgID)
|
||||
continue
|
||||
}
|
||||
|
||||
// 检查是否是媒体组
|
||||
groupID, isGroup := msg.GetGroupedID()
|
||||
if isGroup && groupID != 0 && !single {
|
||||
// 使用同一个 client context 获取媒体组
|
||||
groupMsgs, err := getGroupedMessagesWithContext(msgCtx, chatID)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to get grouped messages: %v", err)
|
||||
} else {
|
||||
for _, gmsg := range groupMsgs {
|
||||
if gmsg.Media == nil {
|
||||
continue
|
||||
}
|
||||
gmedia, ok := gmsg.GetMedia()
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
// 使用获取消息时使用的同一个 client context 创建文件
|
||||
file, err := tfile.FromMediaMessage(gmedia, clientCtx.Raw, gmsg)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to create file from media: %v", err)
|
||||
continue
|
||||
}
|
||||
files = append(files, file)
|
||||
}
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
// 单个文件 - 使用获取消息时使用的同一个 client context 创建文件
|
||||
file, err := tfile.FromMediaMessage(media, clientCtx.Raw, msg)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to create file from media: %v", err)
|
||||
continue
|
||||
}
|
||||
files = append(files, file)
|
||||
}
|
||||
|
||||
if len(files) == 0 {
|
||||
return nil, fmt.Errorf("no files found in provided links")
|
||||
}
|
||||
|
||||
return files, nil
|
||||
}
|
||||
|
||||
// isValidMessageLink 检查是否是有效的 Telegram 消息链接
|
||||
func isValidMessageLink(link string) bool {
|
||||
for _, prefix := range []string{
|
||||
"https://t.me/",
|
||||
"http://t.me/",
|
||||
"https://telegram.me/",
|
||||
"http://telegram.me/",
|
||||
} {
|
||||
if strings.HasPrefix(link, prefix) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
80
api/tphpics.go
Normal file
80
api/tphpics.go
Normal file
@@ -0,0 +1,80 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/url"
|
||||
"strings"
|
||||
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/tphutil"
|
||||
"github.com/krau/SaveAny-Bot/pkg/telegraph"
|
||||
)
|
||||
|
||||
// ExtractTelegraphImages 从 Telegraph URL 提取图片
|
||||
func ExtractTelegraphImages(ctx context.Context, pageURL string) ([]string, string, error) {
|
||||
logger := log.FromContext(ctx)
|
||||
|
||||
// 验证 URL 格式
|
||||
if !isValidTelegraphURL(pageURL) {
|
||||
return nil, "", fmt.Errorf("invalid telegraph URL format: %s", pageURL)
|
||||
}
|
||||
|
||||
// 解析 URL 获取页面路径
|
||||
pagepath, err := parseTelegraphPath(pageURL)
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
|
||||
logger.Debugf("Fetching telegraph page: %s", pagepath)
|
||||
|
||||
client := telegraph.NewClient()
|
||||
page, err := client.GetPage(ctx, pagepath)
|
||||
if err != nil {
|
||||
return nil, "", fmt.Errorf("failed to get telegraph page: %w", err)
|
||||
}
|
||||
|
||||
var imgs []string
|
||||
for _, elem := range page.Content {
|
||||
imgs = append(imgs, tphutil.GetNodeImages(elem)...)
|
||||
}
|
||||
|
||||
if len(imgs) == 0 {
|
||||
return nil, "", fmt.Errorf("no images found in telegraph page")
|
||||
}
|
||||
|
||||
return imgs, pagepath, nil
|
||||
}
|
||||
|
||||
// parseTelegraphPath 解析 Telegraph URL 获取页面路径
|
||||
func parseTelegraphPath(pageURL string) (string, error) {
|
||||
u, err := url.Parse(pageURL)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("invalid telegraph URL: %w", err)
|
||||
}
|
||||
|
||||
if !strings.HasSuffix(u.Host, "telegra.ph") && !strings.HasSuffix(u.Host, "telegraph.co") {
|
||||
return "", fmt.Errorf("invalid telegraph URL host: %s", u.Host)
|
||||
}
|
||||
|
||||
paths := strings.Split(strings.TrimPrefix(u.Path, "/"), "/")
|
||||
if len(paths) == 0 || paths[0] == "" {
|
||||
return "", fmt.Errorf("invalid telegraph URL path: %s", u.Path)
|
||||
}
|
||||
|
||||
pagepath := paths[len(paths)-1]
|
||||
pagepath, err = url.PathUnescape(pagepath)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to unescape telegraph path: %w", err)
|
||||
}
|
||||
|
||||
return strings.TrimSpace(pagepath), nil
|
||||
}
|
||||
|
||||
// isValidTelegraphURL 检查是否是有效的 Telegraph URL
|
||||
func isValidTelegraphURL(url string) bool {
|
||||
return strings.HasPrefix(url, "https://telegra.ph/") ||
|
||||
strings.HasPrefix(url, "http://telegra.ph/") ||
|
||||
strings.HasPrefix(url, "https://telegraph.co/") ||
|
||||
strings.HasPrefix(url, "http://telegraph.co/")
|
||||
}
|
||||
163
api/types.go
Normal file
163
api/types.go
Normal file
@@ -0,0 +1,163 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
|
||||
)
|
||||
|
||||
// TaskStatus 表示任务状态
|
||||
type TaskStatus string
|
||||
|
||||
const (
|
||||
TaskStatusQueued TaskStatus = "queued"
|
||||
TaskStatusRunning TaskStatus = "running"
|
||||
TaskStatusCompleted TaskStatus = "completed"
|
||||
TaskStatusFailed TaskStatus = "failed"
|
||||
TaskStatusCancelled TaskStatus = "cancelled"
|
||||
)
|
||||
|
||||
// CreateTaskRequest 创建任务请求
|
||||
type CreateTaskRequest struct {
|
||||
Type tasktype.TaskType `json:"type"`
|
||||
Storage string `json:"storage"`
|
||||
Path string `json:"path"`
|
||||
Webhook string `json:"webhook,omitempty"`
|
||||
Params json.RawMessage `json:"params"`
|
||||
}
|
||||
|
||||
// CreateTaskResponse 创建任务响应
|
||||
type CreateTaskResponse struct {
|
||||
TaskID string `json:"task_id"`
|
||||
Type tasktype.TaskType `json:"type"`
|
||||
Status TaskStatus `json:"status"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
}
|
||||
|
||||
// TaskProgress 任务进度
|
||||
type TaskProgress struct {
|
||||
TotalBytes int64 `json:"total_bytes,omitempty"`
|
||||
DownloadedBytes int64 `json:"downloaded_bytes,omitempty"`
|
||||
TotalFiles int `json:"total_files,omitempty"`
|
||||
DownloadedFiles int `json:"downloaded_files,omitempty"`
|
||||
Percent float64 `json:"percent,omitempty"`
|
||||
SpeedMBPS float64 `json:"speed_mbps,omitempty"`
|
||||
}
|
||||
|
||||
// TaskInfoResponse 任务信息响应
|
||||
type TaskInfoResponse struct {
|
||||
TaskID string `json:"task_id"`
|
||||
Type tasktype.TaskType `json:"type"`
|
||||
Status TaskStatus `json:"status"`
|
||||
Title string `json:"title"`
|
||||
Progress *TaskProgress `json:"progress,omitempty"`
|
||||
Storage string `json:"storage"`
|
||||
Path string `json:"path"`
|
||||
Error string `json:"error,omitempty"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
}
|
||||
|
||||
// TasksListResponse 任务列表响应
|
||||
type TasksListResponse struct {
|
||||
Tasks []TaskInfoResponse `json:"tasks"`
|
||||
Total int `json:"total"`
|
||||
}
|
||||
|
||||
// StoragesResponse 存储列表响应
|
||||
type StoragesResponse struct {
|
||||
Storages []StorageInfo `json:"storages"`
|
||||
}
|
||||
|
||||
// StorageInfo 存储信息
|
||||
type StorageInfo struct {
|
||||
Name string `json:"name"`
|
||||
Type string `json:"type"`
|
||||
}
|
||||
|
||||
// WebhookPayload Webhook 回调负载
|
||||
type WebhookPayload struct {
|
||||
TaskID string `json:"task_id"`
|
||||
Type string `json:"type"`
|
||||
Status TaskStatus `json:"status"`
|
||||
Storage string `json:"storage"`
|
||||
Path string `json:"path"`
|
||||
CompletedAt *time.Time `json:"completed_at,omitempty"`
|
||||
Error string `json:"error,omitempty"`
|
||||
}
|
||||
|
||||
// ErrorResponse 错误响应
|
||||
type ErrorResponse struct {
|
||||
Error string `json:"error"`
|
||||
Message string `json:"message,omitempty"`
|
||||
}
|
||||
|
||||
// APIError API 错误
|
||||
type APIError struct {
|
||||
StatusCode int
|
||||
ErrorCode string
|
||||
Message string
|
||||
}
|
||||
|
||||
func (e *APIError) Error() string {
|
||||
return e.Message
|
||||
}
|
||||
|
||||
// WriteJSON 写入 JSON 响应
|
||||
func WriteJSON(w http.ResponseWriter, statusCode int, data any) error {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(statusCode)
|
||||
return json.NewEncoder(w).Encode(data)
|
||||
}
|
||||
|
||||
// WriteError 写入错误响应
|
||||
func WriteError(w http.ResponseWriter, statusCode int, errCode, message string) error {
|
||||
return WriteJSON(w, statusCode, ErrorResponse{
|
||||
Error: errCode,
|
||||
Message: message,
|
||||
})
|
||||
}
|
||||
|
||||
// Task 参数结构体
|
||||
|
||||
// DirectLinksParams directlinks 任务参数
|
||||
type DirectLinksParams struct {
|
||||
URLs []string `json:"urls"`
|
||||
}
|
||||
|
||||
// YTDLPParams ytdlp 任务参数
|
||||
type YTDLPParams struct {
|
||||
URLs []string `json:"urls"`
|
||||
Flags []string `json:"flags,omitempty"`
|
||||
}
|
||||
|
||||
// Aria2Params aria2 任务参数
|
||||
type Aria2Params struct {
|
||||
URLs []string `json:"urls"`
|
||||
Options map[string]string `json:"options,omitempty"`
|
||||
}
|
||||
|
||||
// ParsedParams parsed 任务参数
|
||||
type ParsedParams struct {
|
||||
URL string `json:"url"`
|
||||
}
|
||||
|
||||
// TransferParams transfer 任务参数
|
||||
type TransferParams struct {
|
||||
SourceStorage string `json:"source_storage"`
|
||||
SourcePath string `json:"source_path"`
|
||||
TargetStorage string `json:"target_storage"`
|
||||
TargetPath string `json:"target_path"`
|
||||
}
|
||||
|
||||
// TGFilesParams tgfiles 任务参数
|
||||
type TGFilesParams struct {
|
||||
MessageLinks []string `json:"message_links"`
|
||||
}
|
||||
|
||||
// TPHPicsParams tphpics 任务参数
|
||||
type TPHPicsParams struct {
|
||||
TelegraphURL string `json:"telegraph_url"`
|
||||
}
|
||||
114
api/webhook.go
Normal file
114
api/webhook.go
Normal file
@@ -0,0 +1,114 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"github.com/charmbracelet/log"
|
||||
)
|
||||
|
||||
// webhookClient Webhook 客户端
|
||||
var webhookClient = &http.Client{
|
||||
Timeout: 30 * time.Second,
|
||||
}
|
||||
|
||||
// SendWebhook 发送 Webhook 回调
|
||||
func SendWebhook(ctx context.Context, payload *WebhookPayload) {
|
||||
if payload == nil || payload.TaskID == "" {
|
||||
return
|
||||
}
|
||||
|
||||
// 获取任务信息以获取 webhook URL
|
||||
info, ok := GetTask(payload.TaskID)
|
||||
if !ok || info.Webhook == "" {
|
||||
return
|
||||
}
|
||||
|
||||
webhookURL := info.Webhook
|
||||
|
||||
// Async send with retries.
|
||||
go func() {
|
||||
var logger *log.Logger
|
||||
if ctx != nil {
|
||||
logger = log.FromContext(ctx).With("task_id", payload.TaskID)
|
||||
} else {
|
||||
logger = log.Default().With("task_id", payload.TaskID)
|
||||
}
|
||||
if ctx == nil {
|
||||
ctx = context.Background()
|
||||
}
|
||||
|
||||
payloadBytes, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to marshal webhook payload: %v", err)
|
||||
return
|
||||
}
|
||||
|
||||
// 重试 3 次, 指数退避 (100ms/400ms/1.6s)
|
||||
const maxAttempts = 3
|
||||
const requestTimeout = 30 * time.Second
|
||||
backoff := 100 * time.Millisecond
|
||||
for i := range maxAttempts {
|
||||
reqCtx, cancel := context.WithTimeout(ctx, requestTimeout)
|
||||
req, err := http.NewRequestWithContext(reqCtx, http.MethodPost, webhookURL, bytes.NewBuffer(payloadBytes))
|
||||
if err != nil {
|
||||
cancel()
|
||||
logger.Errorf("Failed to create webhook request: %v", err)
|
||||
return
|
||||
}
|
||||
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("User-Agent", "SaveAny-Bot/1.0")
|
||||
|
||||
resp, err := webhookClient.Do(req)
|
||||
cancel()
|
||||
if err != nil {
|
||||
logger.Warnf("Webhook request failed (attempt %d/%d): %v", i+1, maxAttempts, err)
|
||||
if i < maxAttempts-1 {
|
||||
time.Sleep(backoff)
|
||||
}
|
||||
backoff *= 4
|
||||
continue
|
||||
}
|
||||
resp.Body.Close()
|
||||
|
||||
if resp.StatusCode >= 200 && resp.StatusCode < 300 {
|
||||
logger.Debugf("Webhook sent successfully: %s", webhookURL)
|
||||
return
|
||||
}
|
||||
|
||||
logger.Warnf("Webhook returned non-2xx status (attempt %d/%d): %d", i+1, maxAttempts, resp.StatusCode)
|
||||
if i < maxAttempts-1 {
|
||||
time.Sleep(backoff)
|
||||
}
|
||||
backoff *= 4
|
||||
}
|
||||
|
||||
logger.Errorf("Failed to send webhook after %d attempts", maxAttempts)
|
||||
}()
|
||||
}
|
||||
|
||||
// CreateWebhookPayload creates a Webhook payload.
|
||||
func CreateWebhookPayload(taskID string, taskType string, status TaskStatus, storage, path string, err error) *WebhookPayload {
|
||||
payload := &WebhookPayload{
|
||||
TaskID: taskID,
|
||||
Type: taskType,
|
||||
Status: status,
|
||||
Storage: storage,
|
||||
Path: path,
|
||||
}
|
||||
|
||||
if status == TaskStatusCompleted || status == TaskStatusFailed {
|
||||
now := time.Now()
|
||||
payload.CompletedAt = &now
|
||||
}
|
||||
|
||||
if err != nil {
|
||||
payload.Error = err.Error()
|
||||
}
|
||||
|
||||
return payload
|
||||
}
|
||||
@@ -70,9 +70,6 @@ func Init(ctx context.Context) <-chan struct{} {
|
||||
}{nil, err}
|
||||
return
|
||||
}
|
||||
client.API().BotsSetBotCommands(ctx, &tg.BotsSetBotCommandsRequest{
|
||||
Scope: &tg.BotCommandScopeDefault{},
|
||||
})
|
||||
commands := make([]tg.BotCommand, 0, len(handlers.CommandHandlers))
|
||||
for _, info := range handlers.CommandHandlers {
|
||||
commands = append(commands, tg.BotCommand{Command: info.Cmd, Description: i18n.T(info.Desc)})
|
||||
|
||||
@@ -23,7 +23,11 @@ import (
|
||||
)
|
||||
|
||||
func handleAddCallback(ctx *ext.Context, update *ext.Update) error {
|
||||
dataid := strings.Split(string(update.CallbackQuery.Data), " ")[1]
|
||||
dataParts := strings.Split(string(update.CallbackQuery.Data), " ")
|
||||
if len(dataParts) < 2 {
|
||||
return fmt.Errorf("invalid callback data: %q", update.CallbackQuery.Data)
|
||||
}
|
||||
dataid := dataParts[1]
|
||||
data, err := shortcut.GetCallbackDataWithAnswer[tcbdata.Add](ctx, update, dataid)
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -73,14 +77,16 @@ func handleAddCallback(ctx *ext.Context, update *ext.Update) error {
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
dirPath = dir.Path
|
||||
} else if data.SelectedDirPath != "" {
|
||||
dirPath = data.SelectedDirPath
|
||||
}
|
||||
|
||||
switch data.TaskType {
|
||||
case tasktype.TaskTypeTgfiles:
|
||||
if data.AsBatch {
|
||||
return shortcut.CreateAndAddBatchTGFileTaskWithEdit(ctx, userID, selectedStorage, dirPath, data.Files, msgID)
|
||||
return shortcut.CreateAndAddBatchTGFileTaskWithEdit(ctx, userID, selectedStorage, dirPath, data.Files, msgID, data.ConflictStrategy)
|
||||
}
|
||||
return shortcut.CreateAndAddTGFileTaskWithEdit(ctx, userID, selectedStorage, dirPath, data.Files[0], msgID)
|
||||
return shortcut.CreateAndAddTGFileTaskWithEdit(ctx, userID, selectedStorage, dirPath, data.Files[0], msgID, data.ConflictStrategy)
|
||||
case tasktype.TaskTypeTphpics:
|
||||
return shortcut.CreateAndAddtelegraphWithEdit(ctx, userID, data.TphPageNode, data.TphDirPath, data.TphPics, selectedStorage, msgID)
|
||||
case tasktype.TaskTypeParseditem:
|
||||
@@ -90,6 +96,19 @@ func handleAddCallback(ctx *ext.Context, update *ext.Update) error {
|
||||
shortcut.CreateAndAddParsedTaskWithEdit(ctx, selectedStorage, dirPath, data.ParsedItem, msgID, userID)
|
||||
case tasktype.TaskTypeDirectlinks:
|
||||
shortcut.CreateAndAddDirectTaskWithEdit(ctx, selectedStorage, dirPath, data.DirectLinks, msgID, userID)
|
||||
case tasktype.TaskTypeAria2:
|
||||
client := GetAria2Client()
|
||||
if client == nil {
|
||||
ctx.AnswerCallback(msgelem.AlertCallbackAnswer(queryID, i18n.T(i18nk.BotMsgAria2ErrorAria2ClientInitFailed, map[string]any{
|
||||
"Error": "aria2 client not initialized",
|
||||
})))
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
shortcut.CreateAndAddAria2TaskWithEdit(ctx, selectedStorage, dirPath, data.Aria2URIs, client, msgID, userID)
|
||||
case tasktype.TaskTypeYtdlp:
|
||||
shortcut.CreateAndAddYtdlpTaskWithEdit(ctx, selectedStorage, dirPath, data.YtdlpURLs, data.YtdlpFlags, msgID, userID)
|
||||
case tasktype.TaskTypeTransfer:
|
||||
return handleTransferCallback(ctx, userID, selectedStorage, dirPath, data, msgID)
|
||||
default:
|
||||
return fmt.Errorf("unexcept task type: %s", data.TaskType)
|
||||
}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/celestix/gotgproto/dispatcher"
|
||||
@@ -14,7 +15,11 @@ import (
|
||||
)
|
||||
|
||||
func handleCancelCallback(ctx *ext.Context, update *ext.Update) error {
|
||||
taskid := strings.Split(string(update.CallbackQuery.Data), " ")[1]
|
||||
dataParts := strings.Split(string(update.CallbackQuery.Data), " ")
|
||||
if len(dataParts) < 2 {
|
||||
return fmt.Errorf("invalid callback data: %q", update.CallbackQuery.Data)
|
||||
}
|
||||
taskid := dataParts[1]
|
||||
if err := core.CancelTask(ctx, taskid); err != nil {
|
||||
log.FromContext(ctx).Errorf("Failed to cancel task %s: %v", taskid, err)
|
||||
ctx.AnswerCallback(msgelem.AlertCallbackAnswer(update.CallbackQuery.GetQueryID(), i18n.T(i18nk.BotMsgCancelErrorCancelFailed, map[string]any{
|
||||
|
||||
@@ -8,8 +8,10 @@ import (
|
||||
"github.com/celestix/gotgproto/dispatcher"
|
||||
"github.com/celestix/gotgproto/ext"
|
||||
"github.com/gotd/td/tg"
|
||||
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/conflictutil"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
"github.com/krau/SaveAny-Bot/database"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/fnamest"
|
||||
"github.com/krau/SaveAny-Bot/pkg/tcbdata"
|
||||
@@ -25,6 +27,10 @@ func handleConfigCmd(ctx *ext.Context, update *ext.Update) error {
|
||||
Text: i18n.T(i18nk.BotMsgConfigButtonFilenameStrategy),
|
||||
Data: fmt.Appendf(nil, "%s %s", tcbdata.TypeConfig, "fnamest"),
|
||||
},
|
||||
&tg.KeyboardButtonCallback{
|
||||
Text: i18n.T(i18nk.BotMsgConfigButtonConflictStrategy),
|
||||
Data: fmt.Appendf(nil, "%s %s", tcbdata.TypeConfig, "conflictst"),
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
@@ -50,6 +56,8 @@ func handleConfigCallback(ctx *ext.Context, update *ext.Update) error {
|
||||
switch args[1] {
|
||||
case "fnamest":
|
||||
return handleConfigFnameSTCallback(ctx, update)
|
||||
case "conflictst":
|
||||
return handleConfigConflictSTCallback(ctx, update)
|
||||
default:
|
||||
return invaildDataAnswer()
|
||||
}
|
||||
@@ -73,9 +81,9 @@ func handleConfigFnameSTCallback(ctx *ext.Context, update *ext.Update) error {
|
||||
return err
|
||||
}
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: update.CallbackQuery.GetMsgID(),
|
||||
ID: update.CallbackQuery.GetMsgID(),
|
||||
Message: i18n.T(i18nk.BotMsgConfigInfoFilenameStrategySet, map[string]any{
|
||||
"Strategy": fnamest.FnameSTDisplay[st],
|
||||
"Strategy": fnamest.GetDisplay(st, config.C().Lang),
|
||||
}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
@@ -84,7 +92,7 @@ func handleConfigFnameSTCallback(ctx *ext.Context, update *ext.Update) error {
|
||||
buttons := make([]tg.KeyboardButtonClass, 0, len(opts))
|
||||
for _, opt := range opts {
|
||||
buttons = append(buttons, &tg.KeyboardButtonCallback{
|
||||
Text: fnamest.FnameSTDisplay[opt],
|
||||
Text: fnamest.GetDisplay(opt, config.C().Lang),
|
||||
Data: fmt.Appendf(nil, "%s %s %s", tcbdata.TypeConfig, "fnamest", opt),
|
||||
})
|
||||
}
|
||||
@@ -100,9 +108,58 @@ func handleConfigFnameSTCallback(ctx *ext.Context, update *ext.Update) error {
|
||||
currentSt = fnamest.Default
|
||||
}
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: update.CallbackQuery.GetMsgID(),
|
||||
Message: i18n.T(i18nk.BotMsgConfigPromptSelectFilenameStrategy, map[string]any{
|
||||
"Strategy": fnamest.FnameSTDisplay[currentSt],
|
||||
ID: update.CallbackQuery.GetMsgID(),
|
||||
Message: i18n.T(i18nk.BotMsgConfigPromptSelectFilenameStrategy, map[string]any{
|
||||
"Strategy": fnamest.GetDisplay(currentSt, config.C().Lang),
|
||||
}),
|
||||
ReplyMarkup: markup,
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
func handleConfigConflictSTCallback(ctx *ext.Context, update *ext.Update) error {
|
||||
userID := update.CallbackQuery.GetUserID()
|
||||
user, err := database.GetUserByChatID(ctx, userID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
args := strings.Fields(string(update.CallbackQuery.Data))
|
||||
if len(args) == 3 {
|
||||
selected := args[2]
|
||||
if !tcbdata.IsConflictStrategy(selected) {
|
||||
return fmt.Errorf("invalid conflict strategy: %s", selected)
|
||||
}
|
||||
user.ConflictStrategy = selected
|
||||
if err := database.UpdateUser(ctx, user); err != nil {
|
||||
return err
|
||||
}
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: update.CallbackQuery.GetMsgID(),
|
||||
Message: i18n.T(i18nk.BotMsgConfigInfoConflictStrategySet, map[string]any{
|
||||
"Strategy": conflictutil.Display(selected),
|
||||
}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
opts := tcbdata.ConflictStrategyValues()
|
||||
rows := make([]tg.KeyboardButtonRow, 0, len(opts))
|
||||
for _, opt := range opts {
|
||||
rows = append(rows, tg.KeyboardButtonRow{
|
||||
Buttons: []tg.KeyboardButtonClass{
|
||||
&tg.KeyboardButtonCallback{
|
||||
Text: conflictutil.Display(opt),
|
||||
Data: fmt.Appendf(nil, "%s %s %s", tcbdata.TypeConfig, "conflictst", opt),
|
||||
},
|
||||
},
|
||||
})
|
||||
}
|
||||
markup := &tg.ReplyInlineMarkup{Rows: rows}
|
||||
currentSt := conflictutil.EffectiveStrategy(user)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: update.CallbackQuery.GetMsgID(),
|
||||
Message: i18n.T(i18nk.BotMsgConfigPromptSelectConflictStrategy, map[string]any{
|
||||
"Strategy": conflictutil.Display(currentSt),
|
||||
}),
|
||||
ReplyMarkup: markup,
|
||||
})
|
||||
|
||||
@@ -10,6 +10,7 @@ import (
|
||||
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/msgelem"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
|
||||
"github.com/krau/SaveAny-Bot/database"
|
||||
"github.com/krau/SaveAny-Bot/storage"
|
||||
)
|
||||
@@ -42,7 +43,9 @@ func handleDirCmd(ctx *ext.Context, update *ext.Update) error {
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
if _, err := storage.GetStorageByUserIDAndName(ctx, user.ChatID, args[2]); err != nil {
|
||||
ctx.Reply(update, ext.ReplyTextString(err.Error()), nil)
|
||||
logger.Errorf("Failed to get storage %q: %s", args[2], err)
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgCommonErrorGetStorageFailed,
|
||||
tgutil.EscapeHTMLTemplateData(map[string]any{"Error": err.Error()}))), nil)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
|
||||
@@ -3,6 +3,7 @@ package handlers
|
||||
import (
|
||||
"net/url"
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
"github.com/celestix/gotgproto/ext"
|
||||
"github.com/charmbracelet/log"
|
||||
@@ -10,6 +11,8 @@ import (
|
||||
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/msgelem"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
"github.com/krau/SaveAny-Bot/pkg/aria2"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
|
||||
"github.com/krau/SaveAny-Bot/pkg/tcbdata"
|
||||
"github.com/krau/SaveAny-Bot/storage"
|
||||
@@ -50,3 +53,61 @@ func handleDlCmd(ctx *ext.Context, update *ext.Update) error {
|
||||
})
|
||||
return nil
|
||||
}
|
||||
|
||||
var aria2ClientInitOnce sync.Once
|
||||
var aria2ClientInitErr error
|
||||
var aria2Client *aria2.Client
|
||||
|
||||
// GetAria2Client returns the shared aria2 client instance
|
||||
func GetAria2Client() *aria2.Client {
|
||||
return aria2Client
|
||||
}
|
||||
|
||||
func handleAria2DlCmd(ctx *ext.Context, update *ext.Update) error {
|
||||
if !config.C().Aria2.Enable {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgAria2ErrorAria2NotEnabled)), nil)
|
||||
return nil
|
||||
}
|
||||
logger := log.FromContext(ctx)
|
||||
args := strings.Split(update.EffectiveMessage.Text, " ")
|
||||
if len(args) < 2 {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgDlUsage)), nil)
|
||||
return nil
|
||||
}
|
||||
links := args[1:]
|
||||
for i, link := range links {
|
||||
links[i] = strings.TrimSpace(link)
|
||||
}
|
||||
links = slice.Compact(links)
|
||||
if len(links) == 0 {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgDlErrorNoValidLinks)), nil)
|
||||
return nil
|
||||
}
|
||||
logger.Debug("Preparing aria2 download", "links", links)
|
||||
|
||||
// Initialize aria2 client to check connection
|
||||
aria2ClientInitOnce.Do(func() {
|
||||
aria2Client, aria2ClientInitErr = aria2.NewClient(config.C().Aria2.Url, config.C().Aria2.Secret)
|
||||
})
|
||||
if aria2ClientInitErr != nil {
|
||||
logger.Error("Failed to initialize aria2 client", "error", aria2ClientInitErr)
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgAria2ErrorAria2ClientInitFailed, map[string]any{
|
||||
"Error": aria2ClientInitErr.Error(),
|
||||
})), nil)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Build storage selection keyboard (don't add to aria2 yet)
|
||||
markup, err := msgelem.BuildAddSelectStorageKeyboard(storage.GetUserStorages(ctx, update.GetUserChat().GetID()), tcbdata.Add{
|
||||
TaskType: tasktype.TaskTypeAria2,
|
||||
Aria2URIs: links,
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgAria2InfoSelectStorage)), &ext.ReplyOpts{
|
||||
Markup: markup,
|
||||
})
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -13,16 +13,24 @@ import (
|
||||
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/shortcut"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
"github.com/krau/SaveAny-Bot/database"
|
||||
"github.com/krau/SaveAny-Bot/pkg/tcbdata"
|
||||
"github.com/krau/SaveAny-Bot/pkg/tfile"
|
||||
"github.com/krau/SaveAny-Bot/storage"
|
||||
)
|
||||
|
||||
// mediaGroupKey uniquely identifies a media group by chat, sender, and group
|
||||
// ID so files from different users in the same chat can never be mixed.
|
||||
type mediaGroupKey struct {
|
||||
chatID int64
|
||||
userID int64
|
||||
groupID int64
|
||||
}
|
||||
|
||||
type MediaGroupHandler struct {
|
||||
groups map[int64][]tfile.TGFileMessage
|
||||
timers map[int64]*time.Timer
|
||||
groups map[mediaGroupKey][]tfile.TGFileMessage
|
||||
timers map[mediaGroupKey]*time.Timer
|
||||
mu sync.Mutex
|
||||
timeout time.Duration
|
||||
setupOnce sync.Once
|
||||
@@ -39,8 +47,8 @@ func (m *MediaGroupHandler) SetupTimeout(timeoutSec int) {
|
||||
|
||||
var (
|
||||
mediaGroupHandler = &MediaGroupHandler{
|
||||
groups: make(map[int64][]tfile.TGFileMessage),
|
||||
timers: make(map[int64]*time.Timer),
|
||||
groups: make(map[mediaGroupKey][]tfile.TGFileMessage),
|
||||
timers: make(map[mediaGroupKey]*time.Timer),
|
||||
mu: sync.Mutex{},
|
||||
}
|
||||
)
|
||||
@@ -53,41 +61,50 @@ func handleGroupMediaMessage(ctx *ext.Context, update *ext.Update, message *tg.M
|
||||
if !supported {
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
file, err := tfile.FromMediaMessage(media, ctx.Raw, message, tfile.WithNameIfEmpty(
|
||||
tgutil.GenFileNameFromMessage(*message),
|
||||
))
|
||||
userId := update.GetUserChat().GetID()
|
||||
userDB, err := database.GetUserByChatID(ctx, userId)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
tfOpts := mediautil.TfileOptions(ctx, userDB, message)
|
||||
file, err := tfile.FromMediaMessage(media, ctx.Raw, message, tfOpts...)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to get file from media: %s", err)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
mediaGroupHandler.mu.Lock()
|
||||
defer mediaGroupHandler.mu.Unlock()
|
||||
if mediaGroupHandler.groups[groupID] == nil {
|
||||
mediaGroupHandler.groups[groupID] = make([]tfile.TGFileMessage, 0)
|
||||
key := mediaGroupKey{
|
||||
chatID: update.EffectiveChat().GetID(),
|
||||
userID: userId,
|
||||
groupID: groupID,
|
||||
}
|
||||
mediaGroupHandler.groups[groupID] = append(mediaGroupHandler.groups[groupID], file)
|
||||
if mediaGroupHandler.groups[key] == nil {
|
||||
mediaGroupHandler.groups[key] = make([]tfile.TGFileMessage, 0)
|
||||
}
|
||||
mediaGroupHandler.groups[key] = append(mediaGroupHandler.groups[key], file)
|
||||
|
||||
if timer, exists := mediaGroupHandler.timers[groupID]; exists {
|
||||
if timer, exists := mediaGroupHandler.timers[key]; exists {
|
||||
timer.Stop()
|
||||
}
|
||||
mediaGroupHandler.timers[groupID] = time.AfterFunc(mediaGroupHandler.timeout, func() {
|
||||
processMediaGroup(ctx, update, groupID)
|
||||
mediaGroupHandler.timers[key] = time.AfterFunc(mediaGroupHandler.timeout, func() {
|
||||
processMediaGroup(ctx, update, key)
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
func processMediaGroup(ctx *ext.Context, update *ext.Update, groupID int64) {
|
||||
func processMediaGroup(ctx *ext.Context, update *ext.Update, key mediaGroupKey) {
|
||||
logger := log.FromContext(ctx)
|
||||
mediaGroupHandler.mu.Lock()
|
||||
items := mediaGroupHandler.groups[groupID]
|
||||
delete(mediaGroupHandler.groups, groupID)
|
||||
delete(mediaGroupHandler.timers, groupID)
|
||||
items := mediaGroupHandler.groups[key]
|
||||
delete(mediaGroupHandler.groups, key)
|
||||
delete(mediaGroupHandler.timers, key)
|
||||
mediaGroupHandler.mu.Unlock()
|
||||
if len(items) == 0 {
|
||||
logger.Warn("No media items to process for group", "groupID", groupID)
|
||||
logger.Warn("No media items to process for group", "groupID", key.groupID)
|
||||
return
|
||||
}
|
||||
logger.Debugf("Processing media group %d with %d items", groupID, len(items))
|
||||
logger.Debugf("Processing media group %d with %d items", key.groupID, len(items))
|
||||
|
||||
userId := update.GetUserChat().GetID()
|
||||
msg, err := ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgMediaGroupInfoSavingFiles, nil)), nil)
|
||||
@@ -114,7 +131,7 @@ func processMediaGroup(ctx *ext.Context, update *ext.Update, groupID int64) {
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to build storage selection keyboard: %s", err)
|
||||
ctx.EditMessage(userId, &tg.MessagesEditMessageRequest{
|
||||
ID: msg.ID,
|
||||
ID: msg.ID,
|
||||
Message: i18n.T(i18nk.BotMsgMediaGroupErrorBuildStorageSelectKeyboardFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
|
||||
@@ -1,10 +1,13 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"errors"
|
||||
|
||||
"github.com/celestix/gotgproto/dispatcher"
|
||||
"github.com/celestix/gotgproto/ext"
|
||||
"github.com/duke-git/lancet/v2/slice"
|
||||
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/dirutil"
|
||||
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/msgelem"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
@@ -12,16 +15,42 @@ import (
|
||||
"github.com/krau/SaveAny-Bot/storage"
|
||||
)
|
||||
|
||||
// responsibleUserID returns the sender's ID. Callback queries carry it
|
||||
// natively; message updates resolve it through the entity map.
|
||||
func responsibleUserID(u *ext.Update) int64 {
|
||||
if u.CallbackQuery != nil {
|
||||
return u.CallbackQuery.GetUserID()
|
||||
}
|
||||
return u.GetUserChat().GetID()
|
||||
}
|
||||
|
||||
func checkPermission(ctx *ext.Context, update *ext.Update) error {
|
||||
userID := update.GetUserChat().GetID()
|
||||
userID := responsibleUserID(update)
|
||||
if !slice.Contain(config.C().GetUsersID(), userID) {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgCommonErrorNoPermission, nil)), nil)
|
||||
if cbq := update.CallbackQuery; cbq != nil {
|
||||
ctx.AnswerCallback(msgelem.AlertCallbackAnswer(cbq.GetQueryID(), i18n.T(i18nk.BotMsgCommonErrorNoPermission, nil)))
|
||||
} else {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgCommonErrorNoPermission, nil)), nil)
|
||||
}
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
return dispatcher.ContinueGroups
|
||||
}
|
||||
|
||||
// withPermission wraps a callback handler with the same whitelist check used
|
||||
// for message handlers (checkPermission). ContinueGroups is the dispatcher's
|
||||
// success sentinel, not an error: only real failures and EndGroups stop the
|
||||
// chain before the wrapped handler runs.
|
||||
func withPermission(handler func(*ext.Context, *ext.Update) error) func(*ext.Context, *ext.Update) error {
|
||||
return func(ctx *ext.Context, update *ext.Update) error {
|
||||
if err := checkPermission(ctx, update); err != nil && !errors.Is(err, dispatcher.ContinueGroups) {
|
||||
return err
|
||||
}
|
||||
return handler(ctx, update)
|
||||
}
|
||||
}
|
||||
|
||||
func handleSilentMode(next func(*ext.Context, *ext.Update) error, handler func(*ext.Context, *ext.Update) error) func(*ext.Context, *ext.Update) error {
|
||||
return func(ctx *ext.Context, update *ext.Update) error {
|
||||
userID := update.GetUserChat().GetID()
|
||||
|
||||
85
client/bot/handlers/middleware_test.go
Normal file
85
client/bot/handlers/middleware_test.go
Normal file
@@ -0,0 +1,85 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/celestix/gotgproto/ext"
|
||||
"github.com/celestix/gotgproto/types"
|
||||
"github.com/gotd/td/tg"
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
)
|
||||
|
||||
// Regression: callback queries usually arrive as updateShort without entity
|
||||
// maps, so resolving the sender through the entity map yields ID 0 and every
|
||||
// click was denied by the whitelist check. Callback updates must use the
|
||||
// native UserID field.
|
||||
func TestResponsibleUserID(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
update *ext.Update
|
||||
want int64
|
||||
}{
|
||||
{
|
||||
name: "callback query uses native user id",
|
||||
update: &ext.Update{CallbackQuery: &tg.UpdateBotCallbackQuery{UserID: 42}},
|
||||
want: 42,
|
||||
},
|
||||
{
|
||||
name: "message resolves through entity map",
|
||||
update: &ext.Update{
|
||||
EffectiveMessage: &types.Message{Message: &tg.Message{PeerID: &tg.PeerUser{UserID: 7}}},
|
||||
Entities: &tg.Entities{Users: map[int64]*tg.User{7: {ID: 7}}},
|
||||
},
|
||||
want: 7,
|
||||
},
|
||||
{
|
||||
name: "callback query ignores entity map",
|
||||
update: &ext.Update{
|
||||
CallbackQuery: &tg.UpdateBotCallbackQuery{UserID: 9},
|
||||
Entities: &tg.Entities{Users: map[int64]*tg.User{8: {ID: 8}}},
|
||||
},
|
||||
want: 9,
|
||||
},
|
||||
{
|
||||
name: "unresolvable update yields zero",
|
||||
update: &ext.Update{},
|
||||
want: 0,
|
||||
},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
if got := responsibleUserID(tt.update); got != tt.want {
|
||||
t.Fatalf("responsibleUserID() = %d, want %d", got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Regression: withPermission must treat ContinueGroups (the dispatcher's
|
||||
// success sentinel) as a pass and invoke the wrapped handler. v0.60.1 treated
|
||||
// it as an error, so every permitted callback was swallowed before the real
|
||||
// handler ran.
|
||||
func TestWithPermissionInvokesHandler(t *testing.T) {
|
||||
path := filepath.Join(t.TempDir(), "config.toml")
|
||||
if err := os.WriteFile(path, []byte("workers = 2\n\n[[users]]\nid = 42\n"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := config.Init(t.Context(), path); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
update := &ext.Update{CallbackQuery: &tg.UpdateBotCallbackQuery{UserID: 42}}
|
||||
called := false
|
||||
handler := withPermission(func(ctx *ext.Context, u *ext.Update) error {
|
||||
called = true
|
||||
return nil
|
||||
})
|
||||
if err := handler(&ext.Context{}, update); err != nil {
|
||||
t.Fatalf("withPermission returned error: %v", err)
|
||||
}
|
||||
if !called {
|
||||
t.Fatal("withPermission did not invoke the wrapped handler")
|
||||
}
|
||||
}
|
||||
@@ -29,15 +29,19 @@ var CommandHandlers = []DescCommandHandler{
|
||||
{"rule", i18nk.BotMsgCmdRule, handleRuleCmd},
|
||||
{"save", i18nk.BotMsgCmdSave, handleSilentMode(handleSaveCmd, handleSilentSaveReplied)},
|
||||
{"dl", i18nk.BotMsgCmdDl, handleDlCmd},
|
||||
{"aria2dl", i18nk.BotMsgCmdAria2dl, handleAria2DlCmd},
|
||||
{"ytdlp", i18nk.BotMsgCmdYtdlp, handleYtdlpCmd},
|
||||
{"transfer", i18nk.BotMsgCmdTransfer, handleTransferCmd},
|
||||
{"task", i18nk.BotMsgCmdTask, handleTaskCmd},
|
||||
{"cancel", i18nk.BotMsgCmdCancel, handleCancelCmd},
|
||||
{"watch", i18nk.BotMsgCmdWatch, handleWatchCmd},
|
||||
{"unwatch", i18nk.BotMsgCmdUnwatch, handleUnwatchCmd},
|
||||
{"lswatch", i18nk.BotMsgCmdLswatch, handleLswatchCmd},
|
||||
{"config", i18nk.BotMsgCmdConfig, handleConfigCmd},
|
||||
{"fnametmpl", i18nk.BotMsgCmdFnametmpl, handleConfigFnameTmpl},
|
||||
{"help", i18nk.BotMsgCmdHelp, handleHelpCmd},
|
||||
{"parser", i18nk.BotMsgCmdParser, handleParserCmd},
|
||||
{"watch", i18nk.BotMsgCmdWatch, handleWatchCmd},
|
||||
{"unwatch", i18nk.BotMsgCmdUnwatch, handleUnwatchCmd},
|
||||
{"lswatch", i18nk.BotMsgCmdLswatch, handleLswatchCmd},
|
||||
{"syncpeers", i18nk.BotMsgCmdSyncpeers, handleSyncpeersCmd},
|
||||
{"update", i18nk.BotMsgCmdUpdate, handleUpdateCmd},
|
||||
}
|
||||
|
||||
@@ -52,11 +56,11 @@ func Register(disp dispatcher.Dispatcher) {
|
||||
for _, info := range CommandHandlers {
|
||||
disp.AddHandler(handlers.NewCommand(info.Cmd, info.handler))
|
||||
}
|
||||
disp.AddHandler(handlers.NewCallbackQuery(filters.CallbackQuery.Prefix("update"), handleUpdateCallback))
|
||||
disp.AddHandler(handlers.NewCallbackQuery(filters.CallbackQuery.Prefix(tcbdata.TypeAdd), handleAddCallback))
|
||||
disp.AddHandler(handlers.NewCallbackQuery(filters.CallbackQuery.Prefix(tcbdata.TypeSetDefault), handleSetDefaultCallback))
|
||||
disp.AddHandler(handlers.NewCallbackQuery(filters.CallbackQuery.Prefix(tcbdata.TypeCancel), handleCancelCallback))
|
||||
disp.AddHandler(handlers.NewCallbackQuery(filters.CallbackQuery.Prefix(tcbdata.TypeConfig), handleConfigCallback))
|
||||
disp.AddHandler(handlers.NewCallbackQuery(filters.CallbackQuery.Prefix("update"), withPermission(handleUpdateCallback)))
|
||||
disp.AddHandler(handlers.NewCallbackQuery(filters.CallbackQuery.Prefix(tcbdata.TypeAdd), withPermission(handleAddCallback)))
|
||||
disp.AddHandler(handlers.NewCallbackQuery(filters.CallbackQuery.Prefix(tcbdata.TypeSetDefault), withPermission(handleSetDefaultCallback)))
|
||||
disp.AddHandler(handlers.NewCallbackQuery(filters.CallbackQuery.Prefix(tcbdata.TypeCancel), withPermission(handleCancelCallback)))
|
||||
disp.AddHandler(handlers.NewCallbackQuery(filters.CallbackQuery.Prefix(tcbdata.TypeConfig), withPermission(handleConfigCallback)))
|
||||
disp.AddHandler(handlers.NewMessage(sabotfilters.RegexUrl(regexp.MustCompile(re.TgMessageLinkRegexString)), handleSilentMode(handleMessageLink, handleSilentSaveLink)))
|
||||
disp.AddHandler(handlers.NewMessage(sabotfilters.RegexUrl(regexp.MustCompile(re.TelegraphUrlRegexString)), handleSilentMode(handleTelegraphUrlMessage, handleSilentSaveTelegraph)))
|
||||
disp.AddHandler(handlers.NewMessage(filters.Message.Media, handleSilentMode(handleMediaMessage, handleSilentSaveMedia)))
|
||||
|
||||
@@ -13,6 +13,7 @@ import (
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/strutil"
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
"github.com/krau/SaveAny-Bot/database"
|
||||
"github.com/krau/SaveAny-Bot/pkg/rule"
|
||||
)
|
||||
@@ -84,6 +85,46 @@ func handleRuleCmd(ctx *ext.Context, update *ext.Update) error {
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgRuleInfoCreateRuleSuccess, nil)), nil)
|
||||
case "preset":
|
||||
// /rule preset <storage> [base_path]
|
||||
if len(args) < 3 {
|
||||
ctx.Reply(update, ext.ReplyTextStyledTextArray(msgelem.BuildRuleHelpStyling(user.ApplyRule, user.Rules)), nil)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
storageName := args[2]
|
||||
if !config.C().HasStorage(user.ChatID, storageName) {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgRuleErrorStorageNotFound, map[string]any{
|
||||
"Storage": storageName,
|
||||
})), nil)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
basePath := ""
|
||||
if len(args) >= 4 {
|
||||
basePath = args[3]
|
||||
}
|
||||
presets := rule.PresetCategories(basePath)
|
||||
imported := 0
|
||||
for _, p := range presets {
|
||||
rd := &database.Rule{
|
||||
Type: rule.FileNameRegex.String(),
|
||||
Data: p.Regex,
|
||||
StorageName: storageName,
|
||||
DirPath: p.Dir,
|
||||
UserID: user.ID,
|
||||
}
|
||||
if err := database.CreateRule(ctx, rd); err != nil {
|
||||
logger.Errorf("failed to create preset rule %s: %s", p.Name, err)
|
||||
continue
|
||||
}
|
||||
imported++
|
||||
}
|
||||
if imported == 0 {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgRuleErrorCreateRuleFailed, nil)), nil)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgRuleInfoPresetImported, map[string]any{
|
||||
"Count": imported,
|
||||
})), nil)
|
||||
case "del":
|
||||
// /rule del <id>
|
||||
if len(args) < 3 {
|
||||
|
||||
@@ -7,11 +7,13 @@ import (
|
||||
"github.com/celestix/gotgproto/dispatcher"
|
||||
"github.com/celestix/gotgproto/ext"
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/duke-git/lancet/v2/validator"
|
||||
"github.com/gotd/td/tg"
|
||||
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/dirutil"
|
||||
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/mediautil"
|
||||
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/msgelem"
|
||||
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/shortcut"
|
||||
"github.com/krau/SaveAny-Bot/client/user"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/strutil"
|
||||
@@ -105,7 +107,12 @@ func handleBatchSave(ctx *ext.Context, update *ext.Update, args []string) error
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgCommonErrorInvalidMsgIdRange, map[string]any{"Error": err.Error()})), nil)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
chatID, err := tgutil.ParseChatID(ctx, chatArg)
|
||||
tctx := ctx
|
||||
uctx := user.GetCtx()
|
||||
if uctx != nil && validator.IsIntStr(chatArg) {
|
||||
tctx = uctx
|
||||
}
|
||||
chatID, err := tgutil.ParseChatID(tctx, chatArg)
|
||||
if err != nil {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgCommonErrorInvalidIdOrUsername, map[string]any{"Error": err.Error()})), nil)
|
||||
return dispatcher.EndGroups
|
||||
@@ -118,7 +125,7 @@ func handleBatchSave(ctx *ext.Context, update *ext.Update, args []string) error
|
||||
}
|
||||
|
||||
// [TODO]: generator istead of get all messages
|
||||
msgs, err := tgutil.GetMessagesRange(ctx, chatID, int(startID), int(endID))
|
||||
msgs, err := tgutil.GetMessagesRange(tctx, chatID, int(startID), int(endID))
|
||||
if err != nil {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgCommonErrorGetMessagesFailed, map[string]any{"Error": err.Error()})), nil)
|
||||
return dispatcher.EndGroups
|
||||
@@ -141,7 +148,7 @@ func handleBatchSave(ctx *ext.Context, update *ext.Update, args []string) error
|
||||
if !supported {
|
||||
continue
|
||||
}
|
||||
file, err := tfile.FromMediaMessage(media, ctx.Raw, msg, tfile.WithNameIfEmpty(tgutil.GenFileNameFromMessage(*msg)))
|
||||
file, err := tfile.FromMediaMessage(media, tctx.Raw, msg, tfile.WithNameIfEmpty(tgutil.GenFileNameFromMessage(*msg)))
|
||||
if err != nil {
|
||||
log.FromContext(ctx).Errorf("Failed to get file from message: %s", err)
|
||||
continue
|
||||
@@ -172,14 +179,14 @@ func handleBatchSave(ctx *ext.Context, update *ext.Update, args []string) error
|
||||
if err != nil {
|
||||
log.FromContext(ctx).Errorf("Failed to build storage selection keyboard: %s", err)
|
||||
ctx.EditMessage(update.EffectiveChat().GetID(), &tg.MessagesEditMessageRequest{
|
||||
ID: replied.ID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorBuildStorageSelectKeyboardFailed, map[string]any{"Error": err.Error()}),
|
||||
ID: replied.ID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorBuildStorageSelectKeyboardFailed, map[string]any{"Error": err.Error()}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
ctx.EditMessage(update.EffectiveChat().GetID(), &tg.MessagesEditMessageRequest{
|
||||
ID: replied.ID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonInfoFoundFilesSelectStorage, map[string]any{"Count": len(files)}),
|
||||
ID: replied.ID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonInfoFoundFilesSelectStorage, map[string]any{"Count": len(files)}),
|
||||
ReplyMarkup: markup,
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/celestix/gotgproto/dispatcher"
|
||||
@@ -43,7 +44,11 @@ func handleSilentCmd(ctx *ext.Context, update *ext.Update) error {
|
||||
}
|
||||
|
||||
func handleSetDefaultCallback(ctx *ext.Context, update *ext.Update) error {
|
||||
dataid := strings.Split(string(update.CallbackQuery.Data), " ")[1]
|
||||
dataParts := strings.Split(string(update.CallbackQuery.Data), " ")
|
||||
if len(dataParts) < 2 {
|
||||
return fmt.Errorf("invalid callback data: %q", update.CallbackQuery.Data)
|
||||
}
|
||||
dataid := dataParts[1]
|
||||
data, ok := cache.Get[tcbdata.SetDefaultStorage](dataid)
|
||||
|
||||
failedAnswer := func(message string) error {
|
||||
|
||||
62
client/bot/handlers/sync_peers.go
Normal file
62
client/bot/handlers/sync_peers.go
Normal file
@@ -0,0 +1,62 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sync"
|
||||
|
||||
"github.com/celestix/gotgproto/dispatcher"
|
||||
"github.com/celestix/gotgproto/ext"
|
||||
"github.com/celestix/gotgproto/storage"
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/gotd/td/telegram/query/dialogs"
|
||||
"github.com/krau/SaveAny-Bot/client/user"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
)
|
||||
|
||||
var syncpeerMu sync.Mutex
|
||||
|
||||
func handleSyncpeersCmd(ctx *ext.Context, u *ext.Update) error {
|
||||
if !config.C().Telegram.Userbot.Enable {
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
syncpeerMu.Lock()
|
||||
defer syncpeerMu.Unlock()
|
||||
uctx := user.GetCtx()
|
||||
if uctx == nil {
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
ctx.Reply(u, ext.ReplyTextString(i18n.T(i18nk.BotMsgSyncpeersStart)), nil)
|
||||
tapi := uctx.Raw
|
||||
peerStorage := uctx.PeerStorage
|
||||
log.FromContext(ctx).Info("Starting to sync peers...")
|
||||
count := 0
|
||||
err := dialogs.NewQueryBuilder(tapi).GetDialogs().BatchSize(50).ForEach(ctx, func(ctx context.Context, e dialogs.Elem) error {
|
||||
for cid, channel := range e.Entities.Channels() {
|
||||
peerStorage.AddPeer(cid, channel.AccessHash, storage.TypeChannel, channel.Username)
|
||||
count++
|
||||
}
|
||||
for uid, user := range e.Entities.Users() {
|
||||
peerStorage.AddPeer(uid, user.AccessHash, storage.TypeUser, user.Username)
|
||||
count++
|
||||
}
|
||||
for gid := range e.Entities.Chats() {
|
||||
peerStorage.AddPeer(gid, storage.DefaultAccessHash, storage.TypeChat, storage.DefaultUsername)
|
||||
count++
|
||||
}
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
log.FromContext(ctx).Error("Failed to sync peers", "error", err)
|
||||
ctx.Reply(u, ext.ReplyTextString(i18n.T(i18nk.BotMsgSyncpeersFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
})), nil)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
log.FromContext(ctx).Info("Finished syncing peers")
|
||||
ctx.Reply(u, ext.ReplyTextString(i18n.T(i18nk.BotMsgSyncpeersSuccess, map[string]any{
|
||||
"Count": count,
|
||||
})), nil)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
@@ -38,7 +38,7 @@ func handleTaskCmd(ctx *ext.Context, update *ext.Update) error {
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
ctx.Reply(update, ext.ReplyTextStyledTextArray([]styling.StyledTextOption{
|
||||
styling.Plain(i18n.T(i18nk.BotMsgTasksCancelRequestedPrefix)),
|
||||
styling.Plain(i18n.T(i18nk.BotMsgTasksCancelRequestedPrefix)),
|
||||
styling.Code(taskID),
|
||||
}), nil)
|
||||
default:
|
||||
@@ -89,7 +89,11 @@ func showQueuedTasks(ctx *ext.Context, update *ext.Update) {
|
||||
styling.Bold(i18n.T(i18nk.BotMsgTasksQueuedTitle)),
|
||||
styling.Plain(i18n.T(i18nk.BotMsgTasksTotalPrefix, map[string]any{"Count": len(tasks)})),
|
||||
)
|
||||
for _, t := range tasks {
|
||||
const maxShown = 10
|
||||
for i, t := range tasks {
|
||||
if i >= maxShown {
|
||||
break
|
||||
}
|
||||
created := t.Created.In(time.Local).Format("2006-01-02 15:04:05")
|
||||
status := i18n.T(i18nk.BotMsgTasksStatusQueued)
|
||||
if t.Cancelled {
|
||||
@@ -105,10 +109,9 @@ func showQueuedTasks(ctx *ext.Context, update *ext.Update) {
|
||||
styling.Plain("\n"+i18n.T(i18nk.BotMsgTasksFieldStatus)),
|
||||
styling.Code(status),
|
||||
)
|
||||
if len(tasks) > 10 {
|
||||
opts = append(opts, styling.Plain("\n"+i18n.T(i18nk.BotMsgTasksTruncatedNote, map[string]any{"Count": len(tasks)})))
|
||||
break
|
||||
}
|
||||
}
|
||||
if len(tasks) > maxShown {
|
||||
opts = append(opts, styling.Plain("\n"+i18n.T(i18nk.BotMsgTasksTruncatedNote, map[string]any{"Count": len(tasks)})))
|
||||
}
|
||||
ctx.Reply(update, ext.ReplyTextStyledTextArray(opts), nil)
|
||||
}
|
||||
|
||||
257
client/bot/handlers/transfer.go
Normal file
257
client/bot/handlers/transfer.go
Normal file
@@ -0,0 +1,257 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"regexp"
|
||||
"strings"
|
||||
|
||||
"github.com/celestix/gotgproto/dispatcher"
|
||||
"github.com/celestix/gotgproto/ext"
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/gotd/td/tg"
|
||||
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/msgelem"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/strutil"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
|
||||
"github.com/krau/SaveAny-Bot/core"
|
||||
"github.com/krau/SaveAny-Bot/core/tasks/transfer"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
|
||||
"github.com/krau/SaveAny-Bot/pkg/storagetypes"
|
||||
"github.com/krau/SaveAny-Bot/pkg/tcbdata"
|
||||
"github.com/krau/SaveAny-Bot/storage"
|
||||
"github.com/rs/xid"
|
||||
)
|
||||
|
||||
func handleTransferCmd(ctx *ext.Context, update *ext.Update) error {
|
||||
logger := log.FromContext(ctx)
|
||||
args := strutil.ParseArgsRespectQuotes(update.EffectiveMessage.Text)
|
||||
|
||||
if len(args) < 2 {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgTransferUsage, nil)), nil)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
// Parse source: storage_name:/path
|
||||
sourceParts := strings.SplitN(args[1], ":", 2)
|
||||
if len(sourceParts) != 2 {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgTransferErrorInvalidSource, nil)), nil)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
sourceStorageName := sourceParts[0]
|
||||
sourcePath := sourceParts[1]
|
||||
|
||||
userID := update.GetUserChat().GetID()
|
||||
|
||||
// Get source storage
|
||||
sourceStorage, err := storage.GetStorageByUserIDAndName(ctx, userID, sourceStorageName)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to get source storage by user ID and name: %s", err)
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgTransferErrorStorageNotFound, map[string]any{
|
||||
"StorageName": sourceStorageName,
|
||||
"Error": err,
|
||||
})), nil)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
// Check if source storage supports listing
|
||||
listable, ok := sourceStorage.(storage.StorageListable)
|
||||
if !ok {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgTransferErrorStorageNotListable, map[string]any{
|
||||
"StorageName": sourceStorageName,
|
||||
})), nil)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
// Check if source storage supports reading
|
||||
_, ok = sourceStorage.(storage.StorageReadable)
|
||||
if !ok {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgTransferErrorStorageNotReadable, map[string]any{
|
||||
"StorageName": sourceStorageName,
|
||||
})), nil)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
// Fetch file list
|
||||
replied, err := ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgTransferInfoFetchingFiles, nil)), nil)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to reply: %s", err)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
files, err := listable.ListFiles(ctx, sourcePath)
|
||||
if err != nil {
|
||||
ctx.EditMessage(update.EffectiveChat().GetID(), &tg.MessagesEditMessageRequest{
|
||||
ID: replied.ID,
|
||||
Message: i18n.T(i18nk.BotMsgTransferErrorListFilesFailed, map[string]any{"Error": err}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
// Optional filter
|
||||
var filter *regexp.Regexp
|
||||
if len(args) >= 3 {
|
||||
filter, err = regexp.Compile(args[2])
|
||||
if err != nil {
|
||||
ctx.EditMessage(update.EffectiveChat().GetID(), &tg.MessagesEditMessageRequest{
|
||||
ID: replied.ID,
|
||||
Message: i18n.T(i18nk.BotMsgTransferErrorInvalidRegex, map[string]any{"Error": err}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
}
|
||||
|
||||
// Filter files
|
||||
filteredFiles := make([]storagetypes.FileInfo, 0)
|
||||
for _, file := range files {
|
||||
if file.IsDir {
|
||||
continue
|
||||
}
|
||||
if filter != nil && !filter.MatchString(file.Name) {
|
||||
continue
|
||||
}
|
||||
filteredFiles = append(filteredFiles, file)
|
||||
}
|
||||
|
||||
if len(filteredFiles) == 0 {
|
||||
ctx.EditMessage(update.EffectiveChat().GetID(), &tg.MessagesEditMessageRequest{
|
||||
ID: replied.ID,
|
||||
Message: i18n.T(i18nk.BotMsgTransferErrorNoFilesToTransfer, nil),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
// Prepare file paths for callback data
|
||||
filePaths := make([]string, 0, len(filteredFiles))
|
||||
var totalSize int64
|
||||
for _, file := range filteredFiles {
|
||||
filePaths = append(filePaths, file.Path)
|
||||
totalSize += file.Size
|
||||
}
|
||||
|
||||
// Build storage selection keyboard
|
||||
markup, err := msgelem.BuildAddSelectStorageKeyboard(storage.GetUserStorages(ctx, userID), tcbdata.Add{
|
||||
TaskType: tasktype.TaskTypeTransfer,
|
||||
TransferSourceStorName: sourceStorageName,
|
||||
TransferSourcePath: sourcePath,
|
||||
TransferFiles: filePaths,
|
||||
})
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to build storage selection keyboard: %s", err)
|
||||
ctx.EditMessage(update.EffectiveChat().GetID(), &tg.MessagesEditMessageRequest{
|
||||
ID: replied.ID,
|
||||
Message: i18n.T(i18nk.BotMsgTransferErrorBuildStorageSelectKeyboardFailed, map[string]any{"Error": err}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
ctx.EditMessage(update.EffectiveChat().GetID(), &tg.MessagesEditMessageRequest{
|
||||
ID: replied.ID,
|
||||
Message: i18n.T(i18nk.BotMsgTransferInfoFilesSelectStorage, map[string]any{
|
||||
"Count": len(filteredFiles),
|
||||
"SizeMB": fmt.Sprintf("%.2f", float64(totalSize)/(1024*1024)),
|
||||
}),
|
||||
ReplyMarkup: markup,
|
||||
})
|
||||
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
func handleTransferCallback(ctx *ext.Context, userID int64, targetStorage storage.Storage, dirPath string, data tcbdata.Add, msgID int) error {
|
||||
logger := log.FromContext(ctx)
|
||||
|
||||
// Get source storage
|
||||
sourceStorage, err := storage.GetStorageByUserIDAndName(ctx, userID, data.TransferSourceStorName)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to get source storage: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgTransferErrorStorageNotFound, map[string]any{"StorageName": data.TransferSourceStorName, "Error": err}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
// Check if source storage supports listing
|
||||
listable, ok := sourceStorage.(storage.StorageListable)
|
||||
if !ok {
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgTransferErrorStorageNotListable, map[string]any{"StorageName": data.TransferSourceStorName}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
// Re-fetch files to get FileInfo (since we only stored paths)
|
||||
// This is necessary to get size and other metadata
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgTransferInfoFetchingFiles, nil),
|
||||
})
|
||||
|
||||
allFiles, err := listable.ListFiles(ctx, data.TransferSourcePath)
|
||||
if err != nil {
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgTransferErrorListFilesFailed, map[string]any{"Error": err}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
// Create a map for quick lookup
|
||||
fileMap := make(map[string]storagetypes.FileInfo)
|
||||
for _, file := range allFiles {
|
||||
fileMap[file.Path] = file
|
||||
}
|
||||
|
||||
// Build task elements for the selected files
|
||||
elems := make([]transfer.TaskElement, 0, len(data.TransferFiles))
|
||||
var totalSize int64
|
||||
for _, filePath := range data.TransferFiles {
|
||||
fileInfo, ok := fileMap[filePath]
|
||||
if !ok {
|
||||
logger.Warnf("File not found in source storage: %s", filePath)
|
||||
continue
|
||||
}
|
||||
elem := transfer.NewTaskElement(sourceStorage, fileInfo, targetStorage, dirPath)
|
||||
elems = append(elems, *elem)
|
||||
totalSize += fileInfo.Size
|
||||
}
|
||||
|
||||
if len(elems) == 0 {
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgTransferErrorNoFilesToTransfer, nil),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
// Create and add task
|
||||
taskID := xid.New().String()
|
||||
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
|
||||
task := transfer.NewTransferTask(
|
||||
taskID,
|
||||
injectCtx,
|
||||
elems,
|
||||
transfer.NewProgressTracker(msgID, userID),
|
||||
true, // IgnoreErrors
|
||||
)
|
||||
|
||||
if err := core.AddTask(injectCtx, task); err != nil {
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgTransferErrorAddTaskFailed, map[string]any{"Error": err}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgTransferInfoTaskAdded, map[string]any{
|
||||
"Count": len(elems),
|
||||
"SizeMB": fmt.Sprintf("%.2f", float64(totalSize)/(1024*1024)),
|
||||
"TaskID": taskID,
|
||||
}),
|
||||
})
|
||||
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
@@ -10,6 +10,7 @@ import (
|
||||
"github.com/celestix/gotgproto/ext"
|
||||
"github.com/gotd/td/telegram/message/html"
|
||||
"github.com/gotd/td/tg"
|
||||
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/msgelem"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
@@ -100,10 +101,13 @@ func handleUpdateCmd(ctx *ext.Context, u *ext.Update) error {
|
||||
func handleUpdateCallback(ctx *ext.Context, u *ext.Update) error {
|
||||
currentV, err := semver.Parse(config.Version)
|
||||
if err != nil {
|
||||
return err
|
||||
ctx.AnswerCallback(msgelem.AlertCallbackAnswer(u.CallbackQuery.GetQueryID(), i18n.T(i18nk.BotMsgUpdateErrorVersionVarInvalid, map[string]any{
|
||||
"Error": err.Error(),
|
||||
})))
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
ctx.EditMessage(u.GetUserChat().GetID(), &tg.MessagesEditMessageRequest{
|
||||
ID: u.CallbackQuery.GetMsgID(),
|
||||
ID: u.CallbackQuery.GetMsgID(),
|
||||
Message: i18n.T(i18nk.BotMsgUpdateInfoUpgradingWithVersion, map[string]any{
|
||||
"Current": config.Version,
|
||||
}),
|
||||
@@ -111,7 +115,7 @@ func handleUpdateCallback(ctx *ext.Context, u *ext.Update) error {
|
||||
latest, err := ghselfupdate.UpdateSelf(currentV, config.GitRepo)
|
||||
if err != nil {
|
||||
ctx.EditMessage(u.GetUserChat().GetID(), &tg.MessagesEditMessageRequest{
|
||||
ID: u.CallbackQuery.GetMsgID(),
|
||||
ID: u.CallbackQuery.GetMsgID(),
|
||||
Message: i18n.T(i18nk.BotMsgUpdateErrorUpgradeFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
@@ -119,7 +123,7 @@ func handleUpdateCallback(ctx *ext.Context, u *ext.Update) error {
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
ctx.EditMessage(u.GetUserChat().GetID(), &tg.MessagesEditMessageRequest{
|
||||
ID: u.CallbackQuery.GetMsgID(),
|
||||
ID: u.CallbackQuery.GetMsgID(),
|
||||
Message: i18n.T(i18nk.BotMsgUpdateInfoUpgradeSuccess, map[string]any{
|
||||
"Version": latest.Version.String(),
|
||||
}),
|
||||
|
||||
55
client/bot/handlers/utils/conflictutil/conflict.go
Normal file
55
client/bot/handlers/utils/conflictutil/conflict.go
Normal file
@@ -0,0 +1,55 @@
|
||||
package conflictutil
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/database"
|
||||
"github.com/krau/SaveAny-Bot/pkg/tcbdata"
|
||||
)
|
||||
|
||||
const maxConflictLines = 10
|
||||
|
||||
func EffectiveStrategy(user *database.User) string {
|
||||
if user != nil && tcbdata.IsConflictStrategy(user.ConflictStrategy) {
|
||||
return user.ConflictStrategy
|
||||
}
|
||||
return tcbdata.ConflictStrategyRename
|
||||
}
|
||||
|
||||
func ResolveStrategy(user *database.User, override string) string {
|
||||
if tcbdata.IsConflictStrategy(override) {
|
||||
return override
|
||||
}
|
||||
return EffectiveStrategy(user)
|
||||
}
|
||||
|
||||
func Display(strategy string) string {
|
||||
switch strategy {
|
||||
case tcbdata.ConflictStrategyRename:
|
||||
return i18n.T(i18nk.BotMsgConfigConflictStrategyRename, nil)
|
||||
case tcbdata.ConflictStrategyAsk:
|
||||
return i18n.T(i18nk.BotMsgConfigConflictStrategyAsk, nil)
|
||||
case tcbdata.ConflictStrategyOverwrite:
|
||||
return i18n.T(i18nk.BotMsgConfigConflictStrategyOverwrite, nil)
|
||||
case tcbdata.ConflictStrategySkip:
|
||||
return i18n.T(i18nk.BotMsgConfigConflictStrategySkip, nil)
|
||||
default:
|
||||
return strategy
|
||||
}
|
||||
}
|
||||
|
||||
func FormatPaths(conflicts []string) string {
|
||||
if len(conflicts) <= maxConflictLines {
|
||||
return strings.Join(conflicts, "\n")
|
||||
}
|
||||
return strings.Join(conflicts[:maxConflictLines], "\n") + "\n" + i18n.T(i18nk.BotMsgCommonPromptConflictMoreFiles, map[string]any{
|
||||
"Count": len(conflicts) - maxConflictLines,
|
||||
})
|
||||
}
|
||||
|
||||
func FormatPath(storageName, storagePath string) string {
|
||||
return fmt.Sprintf("[%s]:%s", storageName, storagePath)
|
||||
}
|
||||
@@ -30,6 +30,7 @@ type FilenameTemplateData struct {
|
||||
MsgTags string `json:"msgtags,omitempty"`
|
||||
MsgGen string `json:"msggen,omitempty"`
|
||||
MsgDate string `json:"msgdate,omitempty"`
|
||||
MsgRaw string `json:"msgraw,omitempty"`
|
||||
OrigName string `json:"origname,omitempty"`
|
||||
ChatID string `json:"chatid,omitempty"`
|
||||
}
|
||||
@@ -39,6 +40,7 @@ func (f FilenameTemplateData) ToMap() map[string]string {
|
||||
"msgid": f.MsgID,
|
||||
"msgtags": f.MsgTags,
|
||||
"msggen": f.MsgGen,
|
||||
"msgraw": f.MsgRaw,
|
||||
"msgdate": f.MsgDate,
|
||||
"origname": f.OrigName,
|
||||
"chatid": f.ChatID,
|
||||
@@ -108,8 +110,10 @@ func BuildFilenameTemplateData(message *tg.Message) map[string]string {
|
||||
t := time.Unix(int64(date), 0)
|
||||
return t.Format("2006-01-02_15-04-05")
|
||||
}(),
|
||||
MsgRaw: message.GetMessage(),
|
||||
ChatID: func() string {
|
||||
// 如果消息是频道的(从消息链接中fetch的) 直接使用其chat id, 无论它是否是从其他来源转发的
|
||||
// 如果消息是频道的(从消息链接中fetch的) 直接使用其chat id,
|
||||
// 无论它是否是从其他来源转发的
|
||||
if message.GetPost() {
|
||||
peer := message.GetPeerID()
|
||||
switch p := peer.(type) {
|
||||
|
||||
@@ -26,7 +26,7 @@ func BuildDirHelpStyling(dirs []database.Dir) []styling.StyledTextOption {
|
||||
styling.Blockquote(func() string {
|
||||
var sb strings.Builder
|
||||
for _, dir := range dirs {
|
||||
sb.WriteString(fmt.Sprintf("%d: ", dir.ID))
|
||||
fmt.Fprintf(&sb, "%d: ", dir.ID)
|
||||
sb.WriteString(dir.StorageName)
|
||||
sb.WriteString(" - ")
|
||||
sb.WriteString(dir.Path)
|
||||
|
||||
@@ -24,6 +24,8 @@ func BuildRuleHelpStyling(enabled bool, rules []database.Rule) []styling.StyledT
|
||||
styling.Plain(i18n.T(i18nk.BotMsgRuleHelpSwitchSuffix, nil)),
|
||||
styling.Code("add"),
|
||||
styling.Plain(i18n.T(i18nk.BotMsgRuleHelpAddSuffix, nil)),
|
||||
styling.Code("preset"),
|
||||
styling.Plain(i18n.T(i18nk.BotMsgRuleHelpPresetSuffix, nil)),
|
||||
styling.Code("del"),
|
||||
styling.Plain(i18n.T(i18nk.BotMsgRuleHelpDelSuffix, nil)),
|
||||
styling.Plain(i18n.T(i18nk.BotMsgRuleHelpExistingRulesPrefix, nil)),
|
||||
|
||||
@@ -38,6 +38,8 @@ func BuildAddSelectStorageKeyboard(stors []storage.Storage, adddata tcbdata.Add)
|
||||
data := tcbdata.Add{
|
||||
TaskType: taskType,
|
||||
SelectedStorName: storage.Name(),
|
||||
SelectedDirPath: adddata.SelectedDirPath,
|
||||
ConflictStrategy: adddata.ConflictStrategy,
|
||||
|
||||
Files: adddata.Files,
|
||||
AsBatch: len(adddata.Files) > 1,
|
||||
@@ -49,6 +51,14 @@ func BuildAddSelectStorageKeyboard(stors []storage.Storage, adddata tcbdata.Add)
|
||||
ParsedItem: adddata.ParsedItem,
|
||||
|
||||
DirectLinks: adddata.DirectLinks,
|
||||
|
||||
Aria2URIs: adddata.Aria2URIs,
|
||||
YtdlpURLs: adddata.YtdlpURLs,
|
||||
YtdlpFlags: adddata.YtdlpFlags,
|
||||
|
||||
TransferSourceStorName: adddata.TransferSourceStorName,
|
||||
TransferSourcePath: adddata.TransferSourcePath,
|
||||
TransferFiles: adddata.TransferFiles,
|
||||
}
|
||||
dataid := xid.New().String()
|
||||
err := cache.Set(dataid, data)
|
||||
@@ -101,6 +111,38 @@ func BuildAddOneSelectStorageMessage(ctx context.Context, stors []storage.Storag
|
||||
}, nil
|
||||
}
|
||||
|
||||
func BuildConflictStrategyMarkup(adddata tcbdata.Add) (*tg.ReplyInlineMarkup, error) {
|
||||
type option struct {
|
||||
text string
|
||||
strategy string
|
||||
}
|
||||
options := []option{
|
||||
{text: i18n.T(i18nk.BotMsgCommonButtonConflictRename, nil), strategy: tcbdata.ConflictStrategyRename},
|
||||
{text: i18n.T(i18nk.BotMsgCommonButtonConflictOverwrite, nil), strategy: tcbdata.ConflictStrategyOverwrite},
|
||||
{text: i18n.T(i18nk.BotMsgCommonButtonConflictSkip, nil), strategy: tcbdata.ConflictStrategySkip},
|
||||
}
|
||||
buttons := make([]tg.KeyboardButtonClass, 0, len(options))
|
||||
for _, opt := range options {
|
||||
data := adddata
|
||||
data.ConflictStrategy = opt.strategy
|
||||
dataid := xid.New().String()
|
||||
if err := cache.Set(dataid, data); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
buttons = append(buttons, &tg.KeyboardButtonCallback{
|
||||
Text: opt.text,
|
||||
Data: fmt.Appendf(nil, "%s %s", tcbdata.TypeAdd, dataid),
|
||||
})
|
||||
}
|
||||
rows := make([]tg.KeyboardButtonRow, 0, len(buttons))
|
||||
for _, button := range buttons {
|
||||
rows = append(rows, tg.KeyboardButtonRow{
|
||||
Buttons: []tg.KeyboardButtonClass{button},
|
||||
})
|
||||
}
|
||||
return &tg.ReplyInlineMarkup{Rows: rows}, nil
|
||||
}
|
||||
|
||||
// Builds the inline keyboard for setting default storage
|
||||
func BuildSetDefaultStorageMarkup(
|
||||
ctx context.Context,
|
||||
|
||||
@@ -3,8 +3,8 @@ package re
|
||||
import "regexp"
|
||||
|
||||
var (
|
||||
TgMessageLinkRegexString = `https?://t\.me/(?:c/\d+|[A-Za-z0-9_]+)/\d+(?:/\d+)?(?:\?[^\s#]*[A-Za-z0-9_])?\b`
|
||||
TgMessageLinkRegexString = `https?://(?:t|telegram)\.me/(?:c/\d+|[A-Za-z0-9_]+)/\d+(?:/\d+)?(?:\?[^\s#]*[A-Za-z0-9_])?\b`
|
||||
TgMessageLinkRegexp = regexp.MustCompile(TgMessageLinkRegexString)
|
||||
TelegraphUrlRegexString = `https://telegra.ph/.*`
|
||||
TelegraphUrlRegexString = `https://telegra\.ph/[^\s]+`
|
||||
TelegraphUrlRegexp = regexp.MustCompile(TelegraphUrlRegexString)
|
||||
)
|
||||
|
||||
65
client/bot/handlers/utils/shortcut/aria2.go
Normal file
65
client/bot/handlers/utils/shortcut/aria2.go
Normal file
@@ -0,0 +1,65 @@
|
||||
package shortcut
|
||||
|
||||
import (
|
||||
"github.com/celestix/gotgproto/dispatcher"
|
||||
"github.com/celestix/gotgproto/ext"
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/gotd/td/tg"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
|
||||
"github.com/krau/SaveAny-Bot/core"
|
||||
"github.com/krau/SaveAny-Bot/core/tasks/aria2dl"
|
||||
"github.com/krau/SaveAny-Bot/pkg/aria2"
|
||||
"github.com/krau/SaveAny-Bot/storage"
|
||||
"github.com/rs/xid"
|
||||
)
|
||||
|
||||
func CreateAndAddAria2TaskWithEdit(ctx *ext.Context, stor storage.Storage, dirPath string, uris []string, aria2Client *aria2.Client, msgID int, userID int64) error {
|
||||
logger := log.FromContext(ctx)
|
||||
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
|
||||
|
||||
// Now add to aria2 after user selected storage
|
||||
logger.Infof("Adding download to aria2, uris type: %T, value: %+v", uris, uris)
|
||||
|
||||
// Ensure uris is valid
|
||||
if len(uris) == 0 {
|
||||
logger.Error("URIs list is empty")
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgDlErrorNoValidLinks, nil),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
gid, err := aria2Client.AddURI(ctx, uris, nil)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to add aria2 download: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgAria2ErrorAddingAria2Download, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
logger.Infof("Aria2 download added with GID: %s", gid)
|
||||
|
||||
// Create task with the GID
|
||||
task := aria2dl.NewTask(xid.New().String(), injectCtx, gid, uris, aria2Client, stor, dirPath, aria2dl.NewProgress(msgID, userID))
|
||||
if err := core.AddTask(injectCtx, task); err != nil {
|
||||
logger.Errorf("Failed to add task: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorTaskAddFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonInfoTaskAdded, nil),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
@@ -16,7 +16,7 @@ import (
|
||||
|
||||
func CreateAndAddDirectTaskWithEdit(ctx *ext.Context, stor storage.Storage, dirPath string, links []string, msgID int, userID int64) error {
|
||||
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
|
||||
task := directlinks.NewTask(xid.New().String(), injectCtx, links, stor, stor.JoinStoragePath(dirPath), directlinks.NewProgress(msgID, userID))
|
||||
task := directlinks.NewTask(xid.New().String(), injectCtx, links, stor, dirPath, directlinks.NewProgress(msgID, userID))
|
||||
if err := core.AddTask(injectCtx, task); err != nil {
|
||||
log.FromContext(ctx).Errorf("Failed to add task: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
|
||||
@@ -3,6 +3,7 @@ package shortcut
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/url"
|
||||
"strings"
|
||||
|
||||
@@ -179,21 +180,19 @@ type TelegraphResult struct {
|
||||
// return replied message, image urls, telegraph path(unescaped), error
|
||||
func GetTphPicsFromMessageWithReply(ctx *ext.Context, update *ext.Update) (*types.Message, *TelegraphResult, error) {
|
||||
logger := log.FromContext(ctx)
|
||||
tphurl := re.TelegraphUrlRegexp.FindString(tgutil.ExtractMessageEntityUrlsText(update.EffectiveMessage.Message))
|
||||
tphurl := findTelegraphURL(update.EffectiveMessage.Message)
|
||||
if tphurl == "" {
|
||||
logger.Warnf("No telegraph url found but called handleTelegraph")
|
||||
return nil, nil, dispatcher.ContinueGroups
|
||||
}
|
||||
pagepath := strings.Split(tphurl, "/")[len(strings.Split(tphurl, "/"))-1]
|
||||
tphdir, err := url.PathUnescape(pagepath)
|
||||
pagepath, err := parseTelegraphPagePath(tphurl)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to unescape telegraph path: %s", err)
|
||||
logger.Errorf("Failed to parse telegraph path: %s", err)
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgCommonErrorParseTelegraphPathFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
})), nil)
|
||||
return nil, nil, dispatcher.EndGroups
|
||||
}
|
||||
tphdir = strings.TrimSpace(tphdir)
|
||||
msg, err := ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgCommonInfoFetchingTelegraphPage, nil)), nil)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to reply to update: %s", err)
|
||||
@@ -244,7 +243,57 @@ func GetTphPicsFromMessageWithReply(ctx *ext.Context, update *ext.Update) (*type
|
||||
}
|
||||
return msg, &TelegraphResult{
|
||||
Pics: imgs,
|
||||
TphDir: tphdir,
|
||||
TphDir: pagepath,
|
||||
Page: page,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func findTelegraphURL(msg *tg.Message) string {
|
||||
if msg == nil {
|
||||
return ""
|
||||
}
|
||||
var firstMatch string
|
||||
findValid := func(text string) string {
|
||||
for _, tphurl := range re.TelegraphUrlRegexp.FindAllString(text, -1) {
|
||||
if firstMatch == "" {
|
||||
firstMatch = tphurl
|
||||
}
|
||||
if _, err := parseTelegraphPagePath(tphurl); err == nil {
|
||||
return tphurl
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
for _, entityURL := range tgutil.ExtractMessageEntityUrls(msg) {
|
||||
if tphurl := findValid(entityURL); tphurl != "" {
|
||||
return tphurl
|
||||
}
|
||||
}
|
||||
if tphurl := findValid(msg.GetMessage()); tphurl != "" {
|
||||
return tphurl
|
||||
}
|
||||
return firstMatch
|
||||
}
|
||||
|
||||
func parseTelegraphPagePath(pageURL string) (string, error) {
|
||||
u, err := url.Parse(pageURL)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("invalid telegraph URL: %w", err)
|
||||
}
|
||||
if u.Scheme != "https" || !strings.EqualFold(u.Hostname(), "telegra.ph") {
|
||||
return "", fmt.Errorf("invalid telegraph URL host: %s", u.Host)
|
||||
}
|
||||
pagepath := strings.Trim(u.EscapedPath(), "/")
|
||||
if pagepath == "" || strings.Contains(pagepath, "/") {
|
||||
return "", fmt.Errorf("invalid telegraph URL path: %s", u.Path)
|
||||
}
|
||||
pagepath, err = url.PathUnescape(pagepath)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to unescape telegraph path: %w", err)
|
||||
}
|
||||
pagepath = strings.TrimSpace(pagepath)
|
||||
if pagepath == "" || strings.Contains(pagepath, "/") {
|
||||
return "", fmt.Errorf("invalid telegraph URL path: %s", u.Path)
|
||||
}
|
||||
return pagepath, nil
|
||||
}
|
||||
|
||||
163
client/bot/handlers/utils/shortcut/message_telegraph_test.go
Normal file
163
client/bot/handlers/utils/shortcut/message_telegraph_test.go
Normal file
@@ -0,0 +1,163 @@
|
||||
package shortcut
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/gotd/td/tg"
|
||||
)
|
||||
|
||||
func TestFindTelegraphURL(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
msg *tg.Message
|
||||
want string
|
||||
}{
|
||||
{
|
||||
name: "single URL entity",
|
||||
msg: &tg.Message{
|
||||
Message: "https://telegra.ph/Example-01-02",
|
||||
Entities: []tg.MessageEntityClass{
|
||||
&tg.MessageEntityURL{Offset: 0, Length: 32},
|
||||
},
|
||||
},
|
||||
want: "https://telegra.ph/Example-01-02",
|
||||
},
|
||||
{
|
||||
name: "Telegraph URL before another URL",
|
||||
msg: &tg.Message{
|
||||
Message: "https://telegra.ph/Example-01-02 https://example.com/",
|
||||
Entities: []tg.MessageEntityClass{
|
||||
&tg.MessageEntityURL{Offset: 0, Length: 32},
|
||||
&tg.MessageEntityURL{Offset: 33, Length: 20},
|
||||
},
|
||||
},
|
||||
want: "https://telegra.ph/Example-01-02",
|
||||
},
|
||||
{
|
||||
name: "hidden Telegraph URL",
|
||||
msg: &tg.Message{
|
||||
Message: "article",
|
||||
Entities: []tg.MessageEntityClass{
|
||||
&tg.MessageEntityTextURL{
|
||||
Offset: 0,
|
||||
Length: 7,
|
||||
URL: "https://telegra.ph/Hidden-01-02",
|
||||
},
|
||||
},
|
||||
},
|
||||
want: "https://telegra.ph/Hidden-01-02",
|
||||
},
|
||||
{
|
||||
name: "URL entity after non-BMP character",
|
||||
msg: &tg.Message{
|
||||
Message: "😀 https://telegra.ph/Emoji-01-02",
|
||||
Entities: []tg.MessageEntityClass{
|
||||
&tg.MessageEntityURL{Offset: 3, Length: 30},
|
||||
},
|
||||
},
|
||||
want: "https://telegra.ph/Emoji-01-02",
|
||||
},
|
||||
{
|
||||
name: "valid Telegraph URL after invalid candidate",
|
||||
msg: &tg.Message{
|
||||
Message: "https://telegra.ph/nested/Bad https://telegra.ph/Valid-01-02",
|
||||
Entities: []tg.MessageEntityClass{
|
||||
&tg.MessageEntityURL{Offset: 0, Length: 29},
|
||||
&tg.MessageEntityURL{Offset: 30, Length: 30},
|
||||
},
|
||||
},
|
||||
want: "https://telegra.ph/Valid-01-02",
|
||||
},
|
||||
{
|
||||
name: "plain message fallback",
|
||||
msg: &tg.Message{Message: "read https://telegra.ph/Plain-01-02 now"},
|
||||
want: "https://telegra.ph/Plain-01-02",
|
||||
},
|
||||
{
|
||||
name: "nil message",
|
||||
msg: nil,
|
||||
want: "",
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
if got := findTelegraphURL(tt.msg); got != tt.want {
|
||||
t.Fatalf("findTelegraphURL() = %q, want %q", got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseTelegraphPagePath(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
pageURL string
|
||||
want string
|
||||
wantErr bool
|
||||
}{
|
||||
{
|
||||
name: "plain path",
|
||||
pageURL: "https://telegra.ph/Example-01-02",
|
||||
want: "Example-01-02",
|
||||
},
|
||||
{
|
||||
name: "escaped path with query and fragment",
|
||||
pageURL: "https://telegra.ph/%E6%B5%8B%E8%AF%95-01-02?source=telegram#top",
|
||||
want: "测试-01-02",
|
||||
},
|
||||
{
|
||||
name: "trailing slash",
|
||||
pageURL: "https://telegra.ph/Example-01-02/",
|
||||
want: "Example-01-02",
|
||||
},
|
||||
{
|
||||
name: "root URL",
|
||||
pageURL: "https://telegra.ph/",
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "wrong host",
|
||||
pageURL: "https://example.com/Example-01-02",
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "wrong scheme",
|
||||
pageURL: "http://telegra.ph/Example-01-02",
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "invalid percent escape",
|
||||
pageURL: "https://telegra.ph/Invalid-%zz",
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "nested path",
|
||||
pageURL: "https://telegra.ph/nested/Example-01-02",
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "encoded slash",
|
||||
pageURL: "https://telegra.ph/nested%2FExample-01-02",
|
||||
wantErr: true,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got, err := parseTelegraphPagePath(tt.pageURL)
|
||||
if tt.wantErr {
|
||||
if err == nil {
|
||||
t.Fatalf("parseTelegraphPagePath(%q) returned no error", tt.pageURL)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatalf("parseTelegraphPagePath(%q) failed: %v", tt.pageURL, err)
|
||||
}
|
||||
if got != tt.want {
|
||||
t.Fatalf("parseTelegraphPagePath(%q) = %q, want %q", tt.pageURL, got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -18,11 +18,11 @@ import (
|
||||
|
||||
func CreateAndAddParsedTaskWithEdit(ctx *ext.Context, stor storage.Storage, dirPath string, item *parser.Item, msgID int, userID int64) error {
|
||||
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
|
||||
task := parsed.NewTask(xid.New().String(), injectCtx, stor, stor.JoinStoragePath(dirPath), item, parsed.NewProgress(msgID, userID))
|
||||
task := parsed.NewTask(xid.New().String(), injectCtx, stor, dirPath, item, parsed.NewProgress(msgID, userID))
|
||||
if err := core.AddTask(injectCtx, task); err != nil {
|
||||
log.FromContext(ctx).Errorf("Failed to add task: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorTaskAddFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
|
||||
@@ -8,6 +8,7 @@ import (
|
||||
"github.com/celestix/gotgproto/ext"
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/gotd/td/tg"
|
||||
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/conflictutil"
|
||||
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/msgelem"
|
||||
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/ruleutil"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
@@ -17,25 +18,29 @@ import (
|
||||
"github.com/krau/SaveAny-Bot/core/tasks/batchtfile"
|
||||
tftask "github.com/krau/SaveAny-Bot/core/tasks/tfile"
|
||||
"github.com/krau/SaveAny-Bot/database"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
|
||||
"github.com/krau/SaveAny-Bot/pkg/tcbdata"
|
||||
"github.com/krau/SaveAny-Bot/pkg/tfile"
|
||||
"github.com/krau/SaveAny-Bot/storage"
|
||||
"github.com/rs/xid"
|
||||
)
|
||||
|
||||
// 创建一个 tfile.TGFileTask 并添加到任务队列中, 以编辑消息的方式反馈结果
|
||||
func CreateAndAddTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor storage.Storage, dirPath string, file tfile.TGFileMessage, trackMsgID int) error {
|
||||
func CreateAndAddTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor storage.Storage, dirPath string, file tfile.TGFileMessage, trackMsgID int, conflictStrategy ...string) error {
|
||||
logger := log.FromContext(ctx)
|
||||
strategy := selectedConflictStrategy(conflictStrategy)
|
||||
user, err := database.GetUserByChatID(ctx, userID)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to get user by chat ID: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: trackMsgID,
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorGetUserWithErrFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
strategy = conflictutil.ResolveStrategy(user, strategy)
|
||||
if user.ApplyRule && user.Rules != nil {
|
||||
matched, matchedStorageName, matchedDirPath := ruleutil.ApplyRule(ctx, user.Rules, ruleutil.NewInput(file))
|
||||
if !matched {
|
||||
@@ -49,7 +54,7 @@ func CreateAndAddTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor storage
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to get storage by user ID and name: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: trackMsgID,
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorGetStorageFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
@@ -59,8 +64,27 @@ func CreateAndAddTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor storage
|
||||
}
|
||||
}
|
||||
startCreateTask:
|
||||
storagePath := stor.JoinStoragePath(path.Join(dirPath, file.Name()))
|
||||
storagePath := path.Join(dirPath, file.Name())
|
||||
if strategy == tcbdata.ConflictStrategyAsk || strategy == tcbdata.ConflictStrategySkip {
|
||||
exists := stor.Exists(ctx, storagePath)
|
||||
if exists && strategy == tcbdata.ConflictStrategyAsk {
|
||||
return promptTGFileConflictStrategy(ctx, userID, stor.Name(), dirPath, []tfile.TGFileMessage{file}, false, []string{conflictutil.FormatPath(stor.Name(), storagePath)}, trackMsgID)
|
||||
}
|
||||
if exists {
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonInfoAllConflictFilesSkipped, map[string]any{
|
||||
"Skipped": file.Name(),
|
||||
}),
|
||||
ReplyMarkup: nil,
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
}
|
||||
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
|
||||
if strategy == tcbdata.ConflictStrategyOverwrite {
|
||||
injectCtx = storage.WithOverwrite(injectCtx)
|
||||
}
|
||||
taskid := xid.New().String()
|
||||
task, err := tftask.NewTGFileTask(taskid, injectCtx, file, stor, storagePath,
|
||||
tftask.NewProgressTrack(
|
||||
@@ -69,7 +93,7 @@ startCreateTask:
|
||||
if err != nil {
|
||||
logger.Errorf("create task failed: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: trackMsgID,
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorTaskCreateFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
@@ -79,7 +103,7 @@ startCreateTask:
|
||||
if err := core.AddTask(injectCtx, task); err != nil {
|
||||
logger.Errorf("add task failed: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: trackMsgID,
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorTaskAddFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
@@ -97,19 +121,21 @@ startCreateTask:
|
||||
}
|
||||
|
||||
// 创建一个 batchtfile.BatchTGFileTask 并添加到任务队列中, 以编辑消息的方式反馈结果
|
||||
func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor storage.Storage, dirPath string, files []tfile.TGFileMessage, trackMsgID int) error {
|
||||
func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor storage.Storage, dirPath string, files []tfile.TGFileMessage, trackMsgID int, conflictStrategy ...string) error {
|
||||
logger := log.FromContext(ctx)
|
||||
strategy := selectedConflictStrategy(conflictStrategy)
|
||||
user, err := database.GetUserByChatID(ctx, userID)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to get user by chat ID: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: trackMsgID,
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorGetUserWithErrFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
strategy = conflictutil.ResolveStrategy(user, strategy)
|
||||
|
||||
useRule := user.ApplyRule && user.Rules != nil
|
||||
|
||||
@@ -128,21 +154,24 @@ func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor st
|
||||
return storname, dirP
|
||||
}
|
||||
|
||||
skipped := make([]string, 0)
|
||||
conflicts := make([]string, 0)
|
||||
elems := make([]batchtfile.TaskElement, 0, len(files))
|
||||
type albumFile struct {
|
||||
file tfile.TGFileMessage
|
||||
storage storage.Storage
|
||||
dirPath string
|
||||
}
|
||||
albumFiles := make(map[int64][]albumFile, 0)
|
||||
for _, file := range files {
|
||||
storName, dirPath := applyRule(file)
|
||||
storName, matchedDirPath := applyRule(file)
|
||||
fileStor := stor
|
||||
if storName != stor.Name() && storName != "" {
|
||||
fileStor, err = storage.GetStorageByUserIDAndName(ctx, user.ChatID, storName)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to get storage by user ID and name: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: trackMsgID,
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorGetStorageFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
@@ -150,16 +179,27 @@ func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor st
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
}
|
||||
if !dirPath.NeedNewForAlbum() {
|
||||
storPath := fileStor.JoinStoragePath(path.Join(dirPath.String(), file.Name()))
|
||||
if !matchedDirPath.NeedNewForAlbum() {
|
||||
storPath := path.Join(matchedDirPath.String(), file.Name())
|
||||
if strategy == tcbdata.ConflictStrategyAsk || strategy == tcbdata.ConflictStrategySkip {
|
||||
exists := fileStor.Exists(ctx, storPath)
|
||||
if exists && strategy == tcbdata.ConflictStrategyAsk {
|
||||
conflicts = append(conflicts, conflictutil.FormatPath(fileStor.Name(), storPath))
|
||||
continue
|
||||
}
|
||||
if exists {
|
||||
skipped = append(skipped, file.Name())
|
||||
continue
|
||||
}
|
||||
}
|
||||
elem, err := batchtfile.NewTaskElement(fileStor, storPath, file)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to create task element: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorTaskCreateFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorTaskCreateFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
@@ -170,12 +210,17 @@ func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor st
|
||||
logger.Warnf("File %s is not in a group, skipping album handling", file.Name())
|
||||
continue
|
||||
}
|
||||
fileDirPath := matchedDirPath.String()
|
||||
if matchedDirPath.NeedNewForAlbum() {
|
||||
fileDirPath = dirPath
|
||||
}
|
||||
if _, ok := albumFiles[groupId]; !ok {
|
||||
albumFiles[groupId] = make([]albumFile, 0)
|
||||
}
|
||||
albumFiles[groupId] = append(albumFiles[groupId], albumFile{
|
||||
file: file,
|
||||
storage: fileStor,
|
||||
dirPath: fileDirPath,
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -188,12 +233,23 @@ func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor st
|
||||
albumDir := strings.TrimSuffix(path.Base(afiles[0].file.Name()), path.Ext(afiles[0].file.Name()))
|
||||
albumStor := afiles[0].storage
|
||||
for _, af := range afiles {
|
||||
afstorPath := af.storage.JoinStoragePath(path.Join(dirPath, albumDir, af.file.Name()))
|
||||
afstorPath := path.Join(af.dirPath, albumDir, af.file.Name())
|
||||
if strategy == tcbdata.ConflictStrategyAsk || strategy == tcbdata.ConflictStrategySkip {
|
||||
exists := albumStor.Exists(ctx, afstorPath)
|
||||
if exists && strategy == tcbdata.ConflictStrategyAsk {
|
||||
conflicts = append(conflicts, conflictutil.FormatPath(albumStor.Name(), afstorPath))
|
||||
continue
|
||||
}
|
||||
if exists {
|
||||
skipped = append(skipped, af.file.Name())
|
||||
continue
|
||||
}
|
||||
}
|
||||
elem, err := batchtfile.NewTaskElement(albumStor, afstorPath, af.file)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to create task element for album file: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: trackMsgID,
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorTaskCreateFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
@@ -204,13 +260,30 @@ func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor st
|
||||
}
|
||||
}
|
||||
|
||||
if strategy == tcbdata.ConflictStrategyAsk && len(conflicts) > 0 {
|
||||
return promptTGFileConflictStrategy(ctx, userID, stor.Name(), dirPath, files, true, conflicts, trackMsgID)
|
||||
}
|
||||
|
||||
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
|
||||
if strategy == tcbdata.ConflictStrategyOverwrite {
|
||||
injectCtx = storage.WithOverwrite(injectCtx)
|
||||
}
|
||||
if len(elems) == 0 {
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonInfoAllConflictFilesSkipped, map[string]any{
|
||||
"Skipped": strings.Join(skipped, "\n"),
|
||||
}),
|
||||
ReplyMarkup: nil,
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
taskid := xid.New().String()
|
||||
task := batchtfile.NewBatchTGFileTask(taskid, injectCtx, elems, batchtfile.NewProgressTracker(trackMsgID, userID), true)
|
||||
task := batchtfile.NewBatchTGFileTask(taskid, injectCtx, elems, batchtfile.NewProgressTrackerWithSkipped(trackMsgID, userID, skipped), true)
|
||||
if err := core.AddTask(injectCtx, task); err != nil {
|
||||
logger.Errorf("Failed to add batch task: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: trackMsgID,
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorTaskAddFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
@@ -219,10 +292,47 @@ func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor st
|
||||
}
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonInfoBatchTasksAdded, map[string]any{
|
||||
"Count": len(files),
|
||||
}),
|
||||
Message: buildBatchAddedMessage(len(elems), skipped),
|
||||
ReplyMarkup: nil,
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
func promptTGFileConflictStrategy(ctx *ext.Context, userID int64, storageName, dirPath string, files []tfile.TGFileMessage, asBatch bool, conflicts []string, trackMsgID int) error {
|
||||
markup, err := msgelem.BuildConflictStrategyMarkup(tcbdata.Add{
|
||||
TaskType: tasktype.TaskTypeTgfiles,
|
||||
SelectedStorName: storageName,
|
||||
SettedDir: true,
|
||||
SelectedDirPath: dirPath,
|
||||
Files: files,
|
||||
AsBatch: asBatch,
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonPromptSelectConflictStrategy, map[string]any{"Files": conflictutil.FormatPaths(conflicts)}),
|
||||
ReplyMarkup: markup,
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
func selectedConflictStrategy(strategies []string) string {
|
||||
if len(strategies) == 0 {
|
||||
return ""
|
||||
}
|
||||
return strategies[0]
|
||||
}
|
||||
|
||||
func buildBatchAddedMessage(count int, skipped []string) string {
|
||||
if len(skipped) == 0 {
|
||||
return i18n.T(i18nk.BotMsgCommonInfoBatchTasksAdded, map[string]any{
|
||||
"Count": count,
|
||||
})
|
||||
}
|
||||
return i18n.T(i18nk.BotMsgCommonInfoBatchTasksAddedWithSkipped, map[string]any{
|
||||
"Count": count,
|
||||
"Skipped": strings.Join(skipped, "\n"),
|
||||
})
|
||||
}
|
||||
|
||||
@@ -25,21 +25,21 @@ func CreateAndAddtelegraphWithEdit(
|
||||
pics []string,
|
||||
stor storage.Storage,
|
||||
trackMsgID int) error {
|
||||
|
||||
|
||||
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
|
||||
task := tphtask.NewTask(xid.New().String(),
|
||||
injectCtx,
|
||||
tphpage.Path,
|
||||
pics,
|
||||
stor,
|
||||
stor.JoinStoragePath(dirPath),
|
||||
dirPath,
|
||||
tphutil.DefaultClient(),
|
||||
tphtask.NewProgress(trackMsgID, userID),
|
||||
)
|
||||
if err := core.AddTask(injectCtx, task); err != nil {
|
||||
log.FromContext(ctx).Errorf("Failed to add task: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: trackMsgID,
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorTaskAddFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
|
||||
63
client/bot/handlers/utils/shortcut/ytdlp.go
Normal file
63
client/bot/handlers/utils/shortcut/ytdlp.go
Normal file
@@ -0,0 +1,63 @@
|
||||
package shortcut
|
||||
|
||||
import (
|
||||
"github.com/celestix/gotgproto/dispatcher"
|
||||
"github.com/celestix/gotgproto/ext"
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/gotd/td/tg"
|
||||
"github.com/rs/xid"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
|
||||
"github.com/krau/SaveAny-Bot/core"
|
||||
"github.com/krau/SaveAny-Bot/core/tasks/ytdlp"
|
||||
"github.com/krau/SaveAny-Bot/storage"
|
||||
)
|
||||
|
||||
func CreateAndAddYtdlpTaskWithEdit(ctx *ext.Context, stor storage.Storage, dirPath string, urls []string, flags []string, msgID int, userID int64) error {
|
||||
logger := log.FromContext(ctx)
|
||||
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
|
||||
|
||||
// Validate URLs
|
||||
if len(urls) == 0 {
|
||||
logger.Error("URLs list is empty")
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgYtdlpErrorNoValidUrls, nil),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
logger.Infof("Creating yt-dlp task for %d URL(s) with %d flag(s)", len(urls), len(flags))
|
||||
|
||||
// Create yt-dlp task
|
||||
task := ytdlp.NewTask(
|
||||
xid.New().String(),
|
||||
injectCtx,
|
||||
urls,
|
||||
flags,
|
||||
stor,
|
||||
dirPath,
|
||||
ytdlp.NewProgress(msgID, userID),
|
||||
)
|
||||
|
||||
// Add task to queue
|
||||
if err := core.AddTask(injectCtx, task); err != nil {
|
||||
logger.Errorf("Failed to add yt-dlp task: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorTaskAddFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonInfoTaskAdded, nil),
|
||||
})
|
||||
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
@@ -5,7 +5,9 @@ import (
|
||||
"path"
|
||||
"regexp"
|
||||
"strings"
|
||||
"sync"
|
||||
"text/template"
|
||||
"time"
|
||||
|
||||
"github.com/celestix/gotgproto/dispatcher"
|
||||
"github.com/celestix/gotgproto/ext"
|
||||
@@ -16,10 +18,12 @@ import (
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
"github.com/krau/SaveAny-Bot/core"
|
||||
"github.com/krau/SaveAny-Bot/core/tasks/tfile"
|
||||
coretfile "github.com/krau/SaveAny-Bot/core/tasks/tfile"
|
||||
"github.com/krau/SaveAny-Bot/database"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/fnamest"
|
||||
"github.com/krau/SaveAny-Bot/pkg/tfile"
|
||||
"github.com/krau/SaveAny-Bot/storage"
|
||||
"github.com/rs/xid"
|
||||
)
|
||||
@@ -60,7 +64,7 @@ func handleWatchCmd(ctx *ext.Context, update *ext.Update) error {
|
||||
filter := ""
|
||||
if len(args) > 2 {
|
||||
filterArg := strings.Join(args[2:], " ")
|
||||
filterType := strings.Split(filterArg, ":")[0]
|
||||
filterType, _, _ := strings.Cut(filterArg, ":")
|
||||
filterData := strings.Split(filterArg, ":")[1]
|
||||
if filterType == "" || filterData == "" {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgWatchErrorFilterFormatInvalid)), nil)
|
||||
@@ -151,6 +155,53 @@ func handleUnwatchCmd(ctx *ext.Context, update *ext.Update) error {
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
type watchMediaGroupHandler struct {
|
||||
groups map[int64]map[uint][]tfile.TGFileMessage // chatID -> userID -> files
|
||||
timers map[int64]map[uint]*time.Timer
|
||||
mu sync.Mutex
|
||||
}
|
||||
|
||||
var watchMediaGroupMgr = &watchMediaGroupHandler{
|
||||
groups: make(map[int64]map[uint][]tfile.TGFileMessage),
|
||||
timers: make(map[int64]map[uint]*time.Timer),
|
||||
}
|
||||
|
||||
func (w *watchMediaGroupHandler) addFile(chatID int64, userID uint, file tfile.TGFileMessage, timeout time.Duration, callback func([]tfile.TGFileMessage)) {
|
||||
w.mu.Lock()
|
||||
defer w.mu.Unlock()
|
||||
|
||||
if w.groups[chatID] == nil {
|
||||
w.groups[chatID] = make(map[uint][]tfile.TGFileMessage)
|
||||
}
|
||||
if w.timers[chatID] == nil {
|
||||
w.timers[chatID] = make(map[uint]*time.Timer)
|
||||
}
|
||||
|
||||
if timer, exists := w.timers[chatID][userID]; exists {
|
||||
timer.Stop()
|
||||
}
|
||||
|
||||
w.groups[chatID][userID] = append(w.groups[chatID][userID], file)
|
||||
|
||||
w.timers[chatID][userID] = time.AfterFunc(timeout, func() {
|
||||
w.mu.Lock()
|
||||
files := w.groups[chatID][userID]
|
||||
delete(w.groups[chatID], userID)
|
||||
delete(w.timers[chatID], userID)
|
||||
if len(w.groups[chatID]) == 0 {
|
||||
delete(w.groups, chatID)
|
||||
}
|
||||
if len(w.timers[chatID]) == 0 {
|
||||
delete(w.timers, chatID)
|
||||
}
|
||||
w.mu.Unlock()
|
||||
|
||||
if len(files) > 0 {
|
||||
callback(files)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func listenMediaMessageEvent(ch chan userclient.MediaMessageEvent) {
|
||||
if userclient.GetCtx() == nil {
|
||||
return
|
||||
@@ -201,6 +252,16 @@ func listenMediaMessageEvent(ch chan userclient.MediaMessageEvent) {
|
||||
logger.Errorf("Failed to get storage by user ID %d and name %s: %v", user.ChatID, user.DefaultStorage, err)
|
||||
continue
|
||||
}
|
||||
// Resolve the default directory path from user.DefaultDir
|
||||
var defaultDirPath string
|
||||
if user.DefaultDir != 0 {
|
||||
dir, err := database.GetDirByID(ctx, user.DefaultDir)
|
||||
if err != nil {
|
||||
logger.Warnf("Failed to get default dir for user %d: %v, using root", user.ChatID, err)
|
||||
} else {
|
||||
defaultDirPath = dir.Path
|
||||
}
|
||||
}
|
||||
switch user.FilenameStrategy {
|
||||
case fnamest.Message.String():
|
||||
file.SetName(tgutil.GenFileNameFromMessage(*file.Message()))
|
||||
@@ -224,7 +285,25 @@ func listenMediaMessageEvent(ch chan userclient.MediaMessageEvent) {
|
||||
}
|
||||
file.SetName(sb.String())
|
||||
}
|
||||
var dirPath string
|
||||
|
||||
// Check if this is a media group and if rules specify NEW-FOR-ALBUM
|
||||
groupID, isGroup := file.Message().GetGroupedID()
|
||||
needAlbumHandling := false
|
||||
if isGroup && groupID != 0 && user.ApplyRule && user.Rules != nil {
|
||||
_, _, matchedDirPath := ruleutil.ApplyRule(ctx, user.Rules, ruleutil.NewInput(file))
|
||||
needAlbumHandling = matchedDirPath.NeedNewForAlbum()
|
||||
}
|
||||
|
||||
if needAlbumHandling {
|
||||
// For media groups with NEW-FOR-ALBUM rule, collect all files of the same group
|
||||
watchMediaGroupMgr.addFile(event.ChatID, user.ID, file, time.Duration(max(config.C().Telegram.MediaGroupTimeout, 1))*time.Second, func(files []tfile.TGFileMessage) {
|
||||
processWatchMediaGroup(ctx, user, stor, defaultDirPath, files)
|
||||
})
|
||||
continue
|
||||
}
|
||||
|
||||
// Process single file or media group without album folder creation
|
||||
dirPath := defaultDirPath
|
||||
if user.ApplyRule && user.Rules != nil {
|
||||
matched, matchedStorageName, matchedDirPath := ruleutil.ApplyRule(ctx, user.Rules, ruleutil.NewInput(file))
|
||||
if !matched {
|
||||
@@ -240,10 +319,10 @@ func listenMediaMessageEvent(ch chan userclient.MediaMessageEvent) {
|
||||
}
|
||||
}
|
||||
startCreateTask:
|
||||
storagePath := stor.JoinStoragePath(path.Join(dirPath, file.Name()))
|
||||
storagePath := path.Join(dirPath, file.Name())
|
||||
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
|
||||
taskid := xid.New().String()
|
||||
task, err := tfile.NewTGFileTask(taskid, injectCtx, file, stor, storagePath, nil)
|
||||
task, err := coretfile.NewTGFileTask(taskid, injectCtx, file, stor, storagePath, nil)
|
||||
if err != nil {
|
||||
logger.Errorf("create task failed: %s", err)
|
||||
continue
|
||||
@@ -256,3 +335,101 @@ func listenMediaMessageEvent(ch chan userclient.MediaMessageEvent) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func processWatchMediaGroup(ctx *ext.Context, user *database.User, stor storage.Storage, dirPath string, files []tfile.TGFileMessage) {
|
||||
logger := log.FromContext(ctx)
|
||||
if len(files) == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
useRule := user.ApplyRule && user.Rules != nil
|
||||
|
||||
applyRule := func(file tfile.TGFileMessage) (string, ruleutil.MatchedDirPath) {
|
||||
if !useRule {
|
||||
return stor.Name(), ruleutil.MatchedDirPath(dirPath)
|
||||
}
|
||||
matched, storName, dirP := ruleutil.ApplyRule(ctx, user.Rules, ruleutil.NewInput(file))
|
||||
if !matched {
|
||||
return stor.Name(), ruleutil.MatchedDirPath(dirPath)
|
||||
}
|
||||
storname := storName.String()
|
||||
if !storName.Usable() {
|
||||
storname = stor.Name()
|
||||
}
|
||||
return storname, dirP
|
||||
}
|
||||
|
||||
type albumFile struct {
|
||||
file tfile.TGFileMessage
|
||||
storage storage.Storage
|
||||
dirPath string
|
||||
}
|
||||
albumFiles := make(map[int64][]albumFile)
|
||||
|
||||
// Collect files by group ID
|
||||
for _, file := range files {
|
||||
storName, ruleDirPath := applyRule(file)
|
||||
fileStor := stor
|
||||
if storName != stor.Name() && storName != "" {
|
||||
var err error
|
||||
fileStor, err = storage.GetStorageByUserIDAndName(ctx, user.ChatID, storName)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to get storage by user ID and name: %s", err)
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
groupId, isGroup := file.Message().GetGroupedID()
|
||||
if !isGroup || groupId == 0 {
|
||||
logger.Warnf("File %s is not in a group, skipping", file.Name())
|
||||
continue
|
||||
}
|
||||
|
||||
// Use the effective dirPath: if rule returns NEW-FOR-ALBUM sentinel, fall back to the
|
||||
// base dirPath passed in (which is defaultDirPath from the caller).
|
||||
effectiveDirPath := string(ruleDirPath)
|
||||
if ruleDirPath.NeedNewForAlbum() {
|
||||
effectiveDirPath = dirPath
|
||||
}
|
||||
|
||||
if _, ok := albumFiles[groupId]; !ok {
|
||||
albumFiles[groupId] = make([]albumFile, 0)
|
||||
}
|
||||
albumFiles[groupId] = append(albumFiles[groupId], albumFile{
|
||||
file: file,
|
||||
storage: fileStor,
|
||||
dirPath: effectiveDirPath,
|
||||
})
|
||||
}
|
||||
|
||||
// Process album files with folder creation
|
||||
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
|
||||
totalTasks := 0
|
||||
for groupID, afiles := range albumFiles {
|
||||
if len(afiles) <= 1 {
|
||||
continue
|
||||
}
|
||||
|
||||
// Use first file's name (without extension) as album folder name
|
||||
albumDir := strings.TrimSuffix(path.Base(afiles[0].file.Name()), path.Ext(afiles[0].file.Name()))
|
||||
albumStor := afiles[0].storage
|
||||
|
||||
logger.Infof("Creating album folder for group %d: %s with %d files", groupID, albumDir, len(afiles))
|
||||
|
||||
for _, af := range afiles {
|
||||
afstorPath := path.Join(af.dirPath, albumDir, af.file.Name())
|
||||
taskid := xid.New().String()
|
||||
task, err := coretfile.NewTGFileTask(taskid, injectCtx, af.file, albumStor, afstorPath, nil)
|
||||
if err != nil {
|
||||
logger.Errorf("create task failed for album file: %s", err)
|
||||
continue
|
||||
}
|
||||
if err := core.AddTask(injectCtx, task); err != nil {
|
||||
logger.Errorf("add task failed: %s", err)
|
||||
continue
|
||||
}
|
||||
totalTasks++
|
||||
}
|
||||
}
|
||||
logger.Infof("Added %d watch media tasks for user %d", totalTasks, user.ChatID)
|
||||
}
|
||||
|
||||
92
client/bot/handlers/ytdlp.go
Normal file
92
client/bot/handlers/ytdlp.go
Normal file
@@ -0,0 +1,92 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"net/url"
|
||||
"strings"
|
||||
|
||||
"github.com/celestix/gotgproto/dispatcher"
|
||||
"github.com/celestix/gotgproto/ext"
|
||||
"github.com/charmbracelet/log"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/msgelem"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
|
||||
"github.com/krau/SaveAny-Bot/pkg/tcbdata"
|
||||
"github.com/krau/SaveAny-Bot/storage"
|
||||
)
|
||||
|
||||
func handleYtdlpCmd(ctx *ext.Context, update *ext.Update) error {
|
||||
logger := log.FromContext(ctx)
|
||||
args := strings.Split(update.EffectiveMessage.Text, " ")
|
||||
if len(args) < 2 {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgYtdlpUsage)), nil)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
// Separate URLs and flags from arguments
|
||||
var urls []string
|
||||
var flags []string
|
||||
|
||||
for i := 1; i < len(args); i++ {
|
||||
arg := strings.TrimSpace(args[i])
|
||||
if arg == "" {
|
||||
continue
|
||||
}
|
||||
|
||||
// Check if it's a flag (starts with - or --)
|
||||
if strings.HasPrefix(arg, "-") {
|
||||
flags = append(flags, arg)
|
||||
// Check if the next argument might be a value for this flag
|
||||
// Don't consume it if it starts with - or looks like a URL with scheme
|
||||
if i+1 < len(args) {
|
||||
nextArg := strings.TrimSpace(args[i+1])
|
||||
if nextArg != "" && !strings.HasPrefix(nextArg, "-") {
|
||||
// Check if it's clearly a URL (has ://)
|
||||
// This handles common video URLs (http://, https://)
|
||||
// For other yt-dlp inputs, users should ensure proper formatting
|
||||
if strings.Contains(nextArg, "://") {
|
||||
// It's a URL, don't consume it as a flag value
|
||||
continue
|
||||
}
|
||||
// Otherwise, treat it as a flag value
|
||||
flags = append(flags, nextArg)
|
||||
i++ // Skip the next argument as it's been consumed
|
||||
}
|
||||
}
|
||||
} else {
|
||||
// Try to parse as URL
|
||||
u, err := url.Parse(arg)
|
||||
if err != nil || u.Scheme == "" || u.Host == "" {
|
||||
logger.Warnf("Invalid URL: %s", arg)
|
||||
continue
|
||||
}
|
||||
urls = append(urls, arg)
|
||||
}
|
||||
}
|
||||
|
||||
if len(urls) == 0 {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgYtdlpErrorNoValidUrls)), nil)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
logger.Debugf("Preparing yt-dlp download for %d URL(s) with %d flag(s)", len(urls), len(flags))
|
||||
|
||||
// Build storage selection keyboard
|
||||
markup, err := msgelem.BuildAddSelectStorageKeyboard(storage.GetUserStorages(ctx, update.GetUserChat().GetID()), tcbdata.Add{
|
||||
TaskType: tasktype.TaskTypeYtdlp,
|
||||
YtdlpURLs: urls,
|
||||
YtdlpFlags: flags,
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgYtdlpInfoUrlsSelectStorage, map[string]any{
|
||||
"Count": len(urls),
|
||||
})), &ext.ReplyOpts{
|
||||
Markup: markup,
|
||||
})
|
||||
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
129
client/bot/handlers/ytdlp_test.go
Normal file
129
client/bot/handlers/ytdlp_test.go
Normal file
@@ -0,0 +1,129 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"net/url"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// TestYtdlpArgumentParsing tests the URL and flag separation logic
|
||||
func TestYtdlpArgumentParsing(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
input string
|
||||
expectedURLs []string
|
||||
expectedFlags []string
|
||||
}{
|
||||
{
|
||||
name: "Single URL without flags",
|
||||
input: "/ytdlp https://example.com/video",
|
||||
expectedURLs: []string{"https://example.com/video"},
|
||||
expectedFlags: []string{},
|
||||
},
|
||||
{
|
||||
name: "Multiple URLs without flags",
|
||||
input: "/ytdlp https://example.com/v1 https://example.com/v2",
|
||||
expectedURLs: []string{"https://example.com/v1", "https://example.com/v2"},
|
||||
expectedFlags: []string{},
|
||||
},
|
||||
{
|
||||
name: "URL with format flag",
|
||||
input: "/ytdlp --format best https://example.com/video",
|
||||
expectedURLs: []string{"https://example.com/video"},
|
||||
expectedFlags: []string{"--format", "best"},
|
||||
},
|
||||
{
|
||||
name: "URL with extract-audio flag",
|
||||
input: "/ytdlp --extract-audio --audio-format mp3 https://example.com/video",
|
||||
expectedURLs: []string{"https://example.com/video"},
|
||||
expectedFlags: []string{"--extract-audio", "--audio-format", "mp3"},
|
||||
},
|
||||
{
|
||||
name: "Multiple URLs with flags",
|
||||
input: "/ytdlp --format best https://example.com/v1 https://example.com/v2",
|
||||
expectedURLs: []string{"https://example.com/v1", "https://example.com/v2"},
|
||||
expectedFlags: []string{"--format", "best"},
|
||||
},
|
||||
{
|
||||
name: "Flags mixed with URLs",
|
||||
input: "/ytdlp https://example.com/v1 --format best https://example.com/v2",
|
||||
expectedURLs: []string{"https://example.com/v1", "https://example.com/v2"},
|
||||
expectedFlags: []string{"--format", "best"},
|
||||
},
|
||||
{
|
||||
name: "Short flag",
|
||||
input: "/ytdlp -f best https://example.com/video",
|
||||
expectedURLs: []string{"https://example.com/video"},
|
||||
expectedFlags: []string{"-f", "best"},
|
||||
},
|
||||
{
|
||||
name: "Boolean flag",
|
||||
input: "/ytdlp --extract-audio https://example.com/video",
|
||||
expectedURLs: []string{"https://example.com/video"},
|
||||
expectedFlags: []string{"--extract-audio"},
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
args := strings.Split(tt.input, " ")
|
||||
|
||||
// Simulate the parsing logic from handleYtdlpCmd
|
||||
var urls []string
|
||||
var flags []string
|
||||
|
||||
for i := 1; i < len(args); i++ {
|
||||
arg := strings.TrimSpace(args[i])
|
||||
if arg == "" {
|
||||
continue
|
||||
}
|
||||
|
||||
// Check if it's a flag (starts with - or --)
|
||||
if strings.HasPrefix(arg, "-") {
|
||||
flags = append(flags, arg)
|
||||
// Check if the next argument might be a value for this flag
|
||||
if i+1 < len(args) {
|
||||
nextArg := strings.TrimSpace(args[i+1])
|
||||
if nextArg != "" && !strings.HasPrefix(nextArg, "-") {
|
||||
// Check if it's clearly a URL (has ://)
|
||||
if strings.Contains(nextArg, "://") {
|
||||
// It's a URL, don't consume it as a flag value
|
||||
continue
|
||||
}
|
||||
// Otherwise, treat it as a flag value
|
||||
flags = append(flags, nextArg)
|
||||
i++ // Skip the next argument as it's been consumed
|
||||
}
|
||||
}
|
||||
} else {
|
||||
// Try to parse as URL
|
||||
u, err := url.Parse(arg)
|
||||
if err != nil || u.Scheme == "" || u.Host == "" {
|
||||
continue
|
||||
}
|
||||
urls = append(urls, arg)
|
||||
}
|
||||
}
|
||||
|
||||
// Verify URLs
|
||||
if len(urls) != len(tt.expectedURLs) {
|
||||
t.Errorf("Expected %d URLs, got %d", len(tt.expectedURLs), len(urls))
|
||||
}
|
||||
for i, expectedURL := range tt.expectedURLs {
|
||||
if i >= len(urls) || urls[i] != expectedURL {
|
||||
t.Errorf("Expected URL[%d] to be '%s', got '%s'", i, expectedURL, urls[i])
|
||||
}
|
||||
}
|
||||
|
||||
// Verify flags
|
||||
if len(flags) != len(tt.expectedFlags) {
|
||||
t.Errorf("Expected %d flags, got %d", len(tt.expectedFlags), len(flags))
|
||||
}
|
||||
for i, expectedFlag := range tt.expectedFlags {
|
||||
if i >= len(flags) || flags[i] != expectedFlag {
|
||||
t.Errorf("Expected flag[%d] to be '%s', got '%s'", i, expectedFlag, flags[i])
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -15,7 +15,7 @@ import (
|
||||
// https://github.com/iyear/tdl/blob/master/core/tclient/tclient.go
|
||||
func NewDefaultMiddlewares(ctx context.Context, timeout time.Duration) []telegram.Middleware {
|
||||
return []telegram.Middleware{
|
||||
recovery.New(ctx, newBackoff(timeout)),
|
||||
recovery.New(ctx, func() backoff.BackOff { return newBackoff(timeout) }),
|
||||
retry.New(config.C().Telegram.RpcRetry),
|
||||
floodwait.NewSimpleWaiter(),
|
||||
}
|
||||
|
||||
@@ -14,19 +14,28 @@ import (
|
||||
)
|
||||
|
||||
type recovery struct {
|
||||
ctx context.Context
|
||||
backoff backoff.BackOff
|
||||
ctx context.Context
|
||||
newBackoff func() backoff.BackOff
|
||||
}
|
||||
|
||||
func New(ctx context.Context, backoff backoff.BackOff) telegram.Middleware {
|
||||
// New returns a recovery middleware.
|
||||
//
|
||||
// newBackoff is a factory that must return a fresh backoff.BackOff on every call: backoff implementations in
|
||||
// cenkalti/backoff/v4 (notably ExponentialBackOff) are not safe for concurrent
|
||||
// use, and the Telegram client invokes RPCs from many goroutines in parallel.
|
||||
//
|
||||
// Sharing a single instance corrupts its internal counters, breaks the
|
||||
// exponential interval, and defeats MaxElapsedTime - see issue #218.
|
||||
func New(ctx context.Context, newBackoff func() backoff.BackOff) telegram.Middleware {
|
||||
return &recovery{
|
||||
ctx: ctx,
|
||||
backoff: backoff,
|
||||
ctx: ctx,
|
||||
newBackoff: newBackoff,
|
||||
}
|
||||
}
|
||||
|
||||
func (r *recovery) Handle(next tg.Invoker) telegram.InvokeFunc {
|
||||
return func(ctx context.Context, input bin.Encoder, output bin.Decoder) error {
|
||||
b := r.newBackoff()
|
||||
|
||||
return backoff.RetryNotify(func() error {
|
||||
if err := next.Invoke(ctx, input, output); err != nil {
|
||||
@@ -38,7 +47,7 @@ func (r *recovery) Handle(next tg.Invoker) telegram.InvokeFunc {
|
||||
}
|
||||
|
||||
return nil
|
||||
}, r.backoff, func(err error, duration time.Duration) {
|
||||
}, b, func(err error, duration time.Duration) {
|
||||
log.FromContext(ctx).Debug("Wait for connection recovery", "error", err, "duration", duration)
|
||||
})
|
||||
}
|
||||
|
||||
@@ -2,6 +2,7 @@ package user
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/celestix/gotgproto"
|
||||
@@ -20,17 +21,18 @@ import (
|
||||
)
|
||||
|
||||
var uc *gotgproto.Client
|
||||
var ectx *ext.Context
|
||||
|
||||
// getEctx lazily creates the user-client ext.Context exactly once. Guarded by
|
||||
// sync.OnceValue so concurrent GetCtx calls cannot race on ectx creation.
|
||||
var getEctx = sync.OnceValue(func() *ext.Context {
|
||||
return uc.CreateContext()
|
||||
})
|
||||
|
||||
func GetCtx() *ext.Context {
|
||||
if ectx != nil {
|
||||
return ectx
|
||||
}
|
||||
if uc == nil {
|
||||
return nil
|
||||
}
|
||||
ectx = uc.CreateContext()
|
||||
return ectx
|
||||
return getEctx()
|
||||
}
|
||||
|
||||
func Login(ctx context.Context) (*gotgproto.Client, error) {
|
||||
|
||||
@@ -22,7 +22,13 @@ func main() {
|
||||
pkg := flag.String("pkg", "i18nk", "Package name for generated file")
|
||||
flag.Parse()
|
||||
|
||||
type localeFile struct {
|
||||
path string
|
||||
keys map[string]struct{}
|
||||
}
|
||||
|
||||
keys := make(map[string]struct{})
|
||||
var localeFiles []localeFile
|
||||
|
||||
err := filepath.WalkDir(*dir, func(path string, d fs.DirEntry, err error) error {
|
||||
if err != nil {
|
||||
@@ -42,7 +48,12 @@ func main() {
|
||||
return fmt.Errorf("failed to parse yaml %s: %w", path, err)
|
||||
}
|
||||
|
||||
collectKeys(content, "", keys)
|
||||
fileKeys := make(map[string]struct{})
|
||||
collectKeys(content, "", fileKeys)
|
||||
localeFiles = append(localeFiles, localeFile{path: path, keys: fileKeys})
|
||||
for k := range fileKeys {
|
||||
keys[k] = struct{}{}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
@@ -50,6 +61,25 @@ func main() {
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
// 一致性校验: 每个语言文件必须包含全部 key
|
||||
invalid := false
|
||||
for _, f := range localeFiles {
|
||||
var missing []string
|
||||
for k := range keys {
|
||||
if _, ok := f.keys[k]; !ok {
|
||||
missing = append(missing, k)
|
||||
}
|
||||
}
|
||||
if len(missing) > 0 {
|
||||
invalid = true
|
||||
sort.Strings(missing)
|
||||
fmt.Fprintf(os.Stderr, "Error: locale file %s is missing %d key(s): %s\n", f.path, len(missing), strings.Join(missing, ", "))
|
||||
}
|
||||
}
|
||||
if invalid {
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
var list []string
|
||||
for k := range keys {
|
||||
list = append(list, k)
|
||||
|
||||
@@ -5,6 +5,7 @@ import (
|
||||
"fmt"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/cmd/upload"
|
||||
"github.com/krau/SaveAny-Bot/cmd/watch"
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
"github.com/spf13/cobra"
|
||||
)
|
||||
@@ -18,6 +19,7 @@ var rootCmd = &cobra.Command{
|
||||
func init() {
|
||||
config.RegisterFlags(rootCmd)
|
||||
upload.Register(rootCmd)
|
||||
watch.Register(rootCmd)
|
||||
}
|
||||
|
||||
func Execute(ctx context.Context) {
|
||||
|
||||
54
cmd/run.go
54
cmd/run.go
@@ -2,19 +2,22 @@ package cmd
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"slices"
|
||||
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/gotd/td/telegram/downloader"
|
||||
"github.com/krau/SaveAny-Bot/api"
|
||||
"github.com/krau/SaveAny-Bot/client/bot"
|
||||
userclient "github.com/krau/SaveAny-Bot/client/user"
|
||||
"github.com/krau/SaveAny-Bot/common/cache"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/fsutil"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
"github.com/krau/SaveAny-Bot/core"
|
||||
"github.com/krau/SaveAny-Bot/database"
|
||||
@@ -26,14 +29,27 @@ import (
|
||||
func Run(cmd *cobra.Command, _ []string) {
|
||||
ctx, cancel := context.WithCancel(cmd.Context())
|
||||
logger := log.NewWithOptions(os.Stdout, log.Options{
|
||||
Level: log.DebugLevel,
|
||||
Level: log.InfoLevel,
|
||||
ReportTimestamp: true,
|
||||
TimeFormat: time.TimeOnly,
|
||||
ReportCaller: true,
|
||||
})
|
||||
log.SetDefault(logger)
|
||||
ctx = log.WithContext(ctx, logger)
|
||||
|
||||
exitChan, err := initAll(ctx, cmd)
|
||||
configFile := config.GetConfigFile(cmd)
|
||||
if err := config.Init(ctx, configFile); err != nil {
|
||||
logger.Fatal("Init failed", "error", err)
|
||||
}
|
||||
|
||||
level, err := log.ParseLevel(strings.TrimSpace(config.C().Log.Level))
|
||||
if err != nil {
|
||||
logger.Warn("Invalid log level, fallback to debug", "level", config.C().Log.Level, "error", err)
|
||||
level = log.DebugLevel
|
||||
}
|
||||
logger.SetLevel(level)
|
||||
|
||||
exitChan, err := initAll(ctx)
|
||||
if err != nil {
|
||||
logger.Fatal("Init failed", "error", err)
|
||||
}
|
||||
@@ -42,19 +58,29 @@ func Run(cmd *cobra.Command, _ []string) {
|
||||
cancel()
|
||||
}()
|
||||
|
||||
core.SetDownloaderProvider(func() downloader.Client {
|
||||
if ectx := bot.ExtContext(); ectx != nil {
|
||||
return ectx.Raw
|
||||
}
|
||||
return nil
|
||||
})
|
||||
// 恢复任务携带 ext 上下文, 让进度编辑/取消按钮在恢复后继续工作。
|
||||
recoverCtx := context.Background()
|
||||
if ectx := bot.ExtContext(); ectx != nil {
|
||||
recoverCtx = tgutil.ExtWithContext(recoverCtx, ectx)
|
||||
}
|
||||
core.RecoverTasks(recoverCtx)
|
||||
|
||||
core.Run(ctx)
|
||||
|
||||
<-ctx.Done()
|
||||
logger.Info("Exiting...")
|
||||
defer logger.Info("Exit complete")
|
||||
core.Close()
|
||||
cleanCache()
|
||||
}
|
||||
|
||||
func initAll(ctx context.Context, cmd *cobra.Command) (<-chan struct{}, error) {
|
||||
configFile := config.GetConfigFile(cmd)
|
||||
if err := config.Init(ctx, configFile); err != nil {
|
||||
return nil, fmt.Errorf("failed to load config: %w", err)
|
||||
}
|
||||
func initAll(ctx context.Context) (<-chan struct{}, error) {
|
||||
cache.Init()
|
||||
logger := log.FromContext(ctx)
|
||||
i18n.Init(config.C().Lang)
|
||||
@@ -76,6 +102,9 @@ func initAll(ctx context.Context, cmd *cobra.Command) (<-chan struct{}, error) {
|
||||
logger.Fatal("User login failed", "error", err)
|
||||
}
|
||||
}
|
||||
if err := api.Start(ctx); err != nil {
|
||||
logger.Fatal("Failed to start API server", "error", err)
|
||||
}
|
||||
return bot.Init(ctx), nil
|
||||
}
|
||||
|
||||
@@ -88,6 +117,15 @@ func cleanCache() {
|
||||
log.Error("Invalid cache directory", "path", config.C().Temp.BasePath)
|
||||
return
|
||||
}
|
||||
unfinished, err := database.CountUnfinishedTasks(context.Background())
|
||||
if err != nil {
|
||||
log.Error("Failed to count unfinished tasks, skipping cache cleanup", "error", err)
|
||||
return
|
||||
}
|
||||
if unfinished > 0 {
|
||||
log.Info("Skipping cache cleanup: unfinished tasks need their cache files for recovery", "tasks", unfinished)
|
||||
return
|
||||
}
|
||||
currentDir, err := os.Getwd()
|
||||
if err != nil {
|
||||
log.Error("Failed to get working directory", "error", err)
|
||||
|
||||
@@ -11,6 +11,7 @@ import (
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/krau/SaveAny-Bot/client/bot"
|
||||
"github.com/krau/SaveAny-Bot/common/cache"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/ioutil"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
@@ -61,6 +62,7 @@ func Upload(cmd *cobra.Command, args []string) error {
|
||||
if err := config.Init(ctx, configFile); err != nil {
|
||||
return fmt.Errorf("failed to load config: %w", err)
|
||||
}
|
||||
i18n.Init(config.C().Lang)
|
||||
cache.Init()
|
||||
database.Init(ctx)
|
||||
|
||||
@@ -88,7 +90,7 @@ func Upload(cmd *cobra.Command, args []string) error {
|
||||
fileName := fileInfo.Name()
|
||||
fileSize := fileInfo.Size()
|
||||
|
||||
uploadPath := stor.JoinStoragePath(path.Join(dirPath, fileName))
|
||||
uploadPath := path.Join(dirPath, fileName)
|
||||
|
||||
ctx = context.WithValue(ctx, ctxkey.ContentLength, fileSize)
|
||||
ctx = tgutil.ExtWithContext(ctx, bot.ExtContext())
|
||||
|
||||
145
cmd/watch/cmd.go
Normal file
145
cmd/watch/cmd.go
Normal file
@@ -0,0 +1,145 @@
|
||||
package watch
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/krau/SaveAny-Bot/client/bot"
|
||||
"github.com/krau/SaveAny-Bot/common/cache"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
"github.com/krau/SaveAny-Bot/database"
|
||||
stortype "github.com/krau/SaveAny-Bot/pkg/enums/storage"
|
||||
"github.com/krau/SaveAny-Bot/storage"
|
||||
"github.com/spf13/cobra"
|
||||
)
|
||||
|
||||
var watchCmd = &cobra.Command{
|
||||
Use: "watch",
|
||||
Short: "watch a local directory and auto-upload changed files to storage",
|
||||
Long: `Watch a local directory and automatically upload created or modified files
|
||||
to the specified storage backend, preserving the relative directory structure.
|
||||
|
||||
Example:
|
||||
saveany-bot watch -p /data/inbox -s mystorage -d backup --recursive`,
|
||||
RunE: runWatch,
|
||||
}
|
||||
|
||||
func Register(root *cobra.Command) {
|
||||
flags := watchCmd.Flags()
|
||||
flags.StringP("path", "p", "", "local directory path to watch")
|
||||
watchCmd.MarkFlagRequired("path")
|
||||
flags.StringP("storage", "s", "", "storage name to upload to")
|
||||
watchCmd.MarkFlagRequired("storage")
|
||||
flags.StringP("dir", "d", "", "storage dir to upload to, default is the base_path of the storage")
|
||||
flags.BoolP("recursive", "r", false, "watch subdirectories recursively")
|
||||
flags.Bool("overwrite", false, "overwrite existing files on storage instead of skipping")
|
||||
flags.Bool("initial-scan", false, "upload existing files in the directory on startup")
|
||||
flags.Duration("debounce", 2*time.Second, "wait time after the last change before uploading a file")
|
||||
flags.Int("upload-workers", 0, "number of concurrent uploads, default is config.workers")
|
||||
flags.Duration("retry-delay", 3*time.Second, "delay between upload retries")
|
||||
root.AddCommand(watchCmd)
|
||||
}
|
||||
|
||||
func runWatch(cmd *cobra.Command, _ []string) error {
|
||||
watchPath, err := cmd.Flags().GetString("path")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
storName, err := cmd.Flags().GetString("storage")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
destDir, err := cmd.Flags().GetString("dir")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
recursive, err := cmd.Flags().GetBool("recursive")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
overwrite, err := cmd.Flags().GetBool("overwrite")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
initialScan, err := cmd.Flags().GetBool("initial-scan")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
debounce, err := cmd.Flags().GetDuration("debounce")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
uploadWorkers, err := cmd.Flags().GetInt("upload-workers")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
retryDelay, err := cmd.Flags().GetDuration("retry-delay")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
ctx := cmd.Context()
|
||||
logger := log.FromContext(ctx)
|
||||
|
||||
configFile := config.GetConfigFile(cmd)
|
||||
if err := config.Init(ctx, configFile); err != nil {
|
||||
return fmt.Errorf("failed to load config: %w", err)
|
||||
}
|
||||
i18n.Init(config.C().Lang)
|
||||
cache.Init()
|
||||
database.Init(ctx)
|
||||
|
||||
stor, err := storage.GetStorageByName(ctx, storName)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to get storage %q: %w", storName, err)
|
||||
}
|
||||
|
||||
// Telegram storage needs the bot client and its ext context injected into ctx.
|
||||
if stor.Type() == stortype.Telegram {
|
||||
bot.Init(ctx)
|
||||
ctx = tgutil.ExtWithContext(ctx, bot.ExtContext())
|
||||
}
|
||||
|
||||
if uploadWorkers < 1 {
|
||||
uploadWorkers = config.C().Workers
|
||||
}
|
||||
|
||||
uploader := NewUploader(ctx, UploaderOptions{
|
||||
Storage: stor,
|
||||
DestDir: destDir,
|
||||
Overwrite: overwrite,
|
||||
Workers: uploadWorkers,
|
||||
Retry: config.C().Retry,
|
||||
RetryDelay: retryDelay,
|
||||
})
|
||||
|
||||
watcher, err := NewWatcher(ctx, WatcherOptions{
|
||||
Root: watchPath,
|
||||
Recursive: recursive,
|
||||
Debounce: debounce,
|
||||
Uploader: uploader,
|
||||
})
|
||||
if err != nil {
|
||||
uploader.Close()
|
||||
return fmt.Errorf("failed to create watcher: %w", err)
|
||||
}
|
||||
|
||||
if initialScan {
|
||||
watcher.ScanExisting(ctx)
|
||||
}
|
||||
|
||||
logger.Infof("watch started: %s -> storage %q dir %q", watchPath, storName, destDir)
|
||||
|
||||
// Run blocks until ctx is cancelled (e.g. SIGINT).
|
||||
runErr := watcher.Run(ctx)
|
||||
|
||||
// Wait for in-flight uploads to finish before exiting.
|
||||
logger.Info("waiting for in-flight uploads to finish...")
|
||||
uploader.Close()
|
||||
logger.Info("watch stopped")
|
||||
|
||||
return runErr
|
||||
}
|
||||
227
cmd/watch/uploader.go
Normal file
227
cmd/watch/uploader.go
Normal file
@@ -0,0 +1,227 @@
|
||||
package watch
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/ctxkey"
|
||||
"github.com/krau/SaveAny-Bot/storage"
|
||||
)
|
||||
|
||||
type uploadJob struct {
|
||||
// localPath is the absolute path of the local file.
|
||||
localPath string
|
||||
// relPath is relative to the watch root, used to preserve directory structure on storage.
|
||||
relPath string
|
||||
}
|
||||
|
||||
// Uploader uploads local files to the target storage via a worker pool.
|
||||
// If a file changes while being uploaded, it is re-uploaded once after the
|
||||
// current upload finishes, instead of being queued multiple times.
|
||||
type Uploader struct {
|
||||
stor storage.Storage
|
||||
destDir string
|
||||
overwrite bool
|
||||
retry int
|
||||
retryDelay time.Duration
|
||||
logger *log.Logger
|
||||
|
||||
jobs chan uploadJob
|
||||
wg sync.WaitGroup
|
||||
|
||||
mu sync.Mutex
|
||||
// inflight maps in-progress (or queued) file paths. A true value means the
|
||||
// file changed during upload and must be re-queued once done.
|
||||
inflight map[string]bool
|
||||
}
|
||||
|
||||
type UploaderOptions struct {
|
||||
Storage storage.Storage
|
||||
DestDir string
|
||||
Overwrite bool
|
||||
Workers int
|
||||
Retry int
|
||||
RetryDelay time.Duration
|
||||
QueueSize int
|
||||
}
|
||||
|
||||
// NewUploader creates and starts an Uploader. The caller must call Close when done.
|
||||
func NewUploader(ctx context.Context, opts UploaderOptions) *Uploader {
|
||||
if opts.Workers < 1 {
|
||||
opts.Workers = 1
|
||||
}
|
||||
if opts.Retry < 1 {
|
||||
opts.Retry = 1
|
||||
}
|
||||
if opts.RetryDelay <= 0 {
|
||||
opts.RetryDelay = 3 * time.Second
|
||||
}
|
||||
if opts.QueueSize < opts.Workers {
|
||||
opts.QueueSize = opts.Workers * 64
|
||||
}
|
||||
|
||||
u := &Uploader{
|
||||
stor: opts.Storage,
|
||||
destDir: opts.DestDir,
|
||||
overwrite: opts.Overwrite,
|
||||
retry: opts.Retry,
|
||||
retryDelay: opts.RetryDelay,
|
||||
logger: log.FromContext(ctx).WithPrefix("uploader"),
|
||||
jobs: make(chan uploadJob, opts.QueueSize),
|
||||
inflight: make(map[string]bool),
|
||||
}
|
||||
|
||||
for i := 0; i < opts.Workers; i++ {
|
||||
u.wg.Add(1)
|
||||
go u.worker(ctx)
|
||||
}
|
||||
|
||||
return u
|
||||
}
|
||||
|
||||
// Submit enqueues an upload job. If the same file is already in flight, it is
|
||||
// marked for re-upload instead of being queued again. Returns false if ctx is
|
||||
// cancelled before the job can be enqueued.
|
||||
func (u *Uploader) Submit(ctx context.Context, job uploadJob) bool {
|
||||
u.mu.Lock()
|
||||
if _, ok := u.inflight[job.localPath]; ok {
|
||||
u.inflight[job.localPath] = true
|
||||
u.mu.Unlock()
|
||||
u.logger.Debugf("file %s already in flight, marked for re-upload", job.localPath)
|
||||
return true
|
||||
}
|
||||
u.inflight[job.localPath] = false
|
||||
u.mu.Unlock()
|
||||
|
||||
select {
|
||||
case u.jobs <- job:
|
||||
return true
|
||||
case <-ctx.Done():
|
||||
u.mu.Lock()
|
||||
delete(u.inflight, job.localPath)
|
||||
u.mu.Unlock()
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
func (u *Uploader) worker(ctx context.Context) {
|
||||
defer u.wg.Done()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case job, ok := <-u.jobs:
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
u.process(ctx, job)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (u *Uploader) process(ctx context.Context, job uploadJob) {
|
||||
if err := u.uploadWithRetry(ctx, job); err != nil {
|
||||
if ctx.Err() != nil {
|
||||
u.clearInflight(job.localPath)
|
||||
return
|
||||
}
|
||||
u.logger.Errorf("failed to upload %s after %d attempt(s): %v", job.localPath, u.retry, err)
|
||||
}
|
||||
|
||||
// Re-queue if the file changed again while it was being uploaded.
|
||||
u.mu.Lock()
|
||||
needReupload := u.inflight[job.localPath]
|
||||
if needReupload {
|
||||
u.inflight[job.localPath] = false
|
||||
} else {
|
||||
delete(u.inflight, job.localPath)
|
||||
}
|
||||
u.mu.Unlock()
|
||||
|
||||
if needReupload {
|
||||
select {
|
||||
case u.jobs <- job:
|
||||
u.logger.Debugf("re-queued %s due to changes during upload", job.localPath)
|
||||
case <-ctx.Done():
|
||||
u.clearInflight(job.localPath)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (u *Uploader) clearInflight(localPath string) {
|
||||
u.mu.Lock()
|
||||
delete(u.inflight, localPath)
|
||||
u.mu.Unlock()
|
||||
}
|
||||
|
||||
func (u *Uploader) uploadWithRetry(ctx context.Context, job uploadJob) error {
|
||||
var lastErr error
|
||||
for attempt := 1; attempt <= u.retry; attempt++ {
|
||||
if ctx.Err() != nil {
|
||||
return ctx.Err()
|
||||
}
|
||||
err := u.upload(ctx, job)
|
||||
if err == nil {
|
||||
return nil
|
||||
}
|
||||
if ctx.Err() != nil {
|
||||
return ctx.Err()
|
||||
}
|
||||
lastErr = err
|
||||
u.logger.Warnf("upload %s failed (attempt %d/%d): %v", job.localPath, attempt, u.retry, err)
|
||||
if attempt < u.retry {
|
||||
select {
|
||||
case <-time.After(u.retryDelay):
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
}
|
||||
}
|
||||
}
|
||||
return lastErr
|
||||
}
|
||||
|
||||
func (u *Uploader) upload(ctx context.Context, job uploadJob) error {
|
||||
file, err := os.Open(filepath.Clean(job.localPath))
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to open file: %w", err)
|
||||
}
|
||||
defer file.Close()
|
||||
|
||||
info, err := file.Stat()
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to stat file: %w", err)
|
||||
}
|
||||
if info.IsDir() {
|
||||
return fmt.Errorf("path is a directory, not a file")
|
||||
}
|
||||
|
||||
// Keep the relative directory structure on the storage side.
|
||||
storagePath := path.Join(u.destDir, filepath.ToSlash(job.relPath))
|
||||
|
||||
uploadCtx := context.WithValue(ctx, ctxkey.ContentLength, info.Size())
|
||||
if u.overwrite {
|
||||
uploadCtx = storage.WithOverwrite(uploadCtx)
|
||||
} else if u.stor.Exists(uploadCtx, storagePath) {
|
||||
u.logger.Infof("skip existing file: %s", storagePath)
|
||||
return nil
|
||||
}
|
||||
|
||||
u.logger.Infof("uploading %s -> %s (%d bytes)", job.localPath, storagePath, info.Size())
|
||||
if err := u.stor.Save(uploadCtx, file, storagePath); err != nil {
|
||||
return fmt.Errorf("failed to save to storage: %w", err)
|
||||
}
|
||||
u.logger.Infof("uploaded %s", storagePath)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Close stops accepting jobs and waits for in-flight uploads to finish.
|
||||
func (u *Uploader) Close() {
|
||||
close(u.jobs)
|
||||
u.wg.Wait()
|
||||
}
|
||||
269
cmd/watch/watcher.go
Normal file
269
cmd/watch/watcher.go
Normal file
@@ -0,0 +1,269 @@
|
||||
package watch
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/fsnotify/fsnotify"
|
||||
)
|
||||
|
||||
// Watcher watches a local directory and submits stable files to the Uploader.
|
||||
//
|
||||
// Write-completion detection: fsnotify emits Write events throughout a write.
|
||||
// Watcher debounces per file and only uploads once the file size stays
|
||||
// unchanged across a debounce window, avoiding uploads of partial files.
|
||||
type Watcher struct {
|
||||
root string
|
||||
recursive bool
|
||||
debounce time.Duration
|
||||
uploader *Uploader
|
||||
logger *log.Logger
|
||||
|
||||
fsw *fsnotify.Watcher
|
||||
|
||||
mu sync.Mutex
|
||||
pending map[string]*time.Timer
|
||||
// lastSize is the last observed file size, used to detect a stable write.
|
||||
lastSize map[string]int64
|
||||
}
|
||||
|
||||
type WatcherOptions struct {
|
||||
Root string
|
||||
Recursive bool
|
||||
Debounce time.Duration
|
||||
Uploader *Uploader
|
||||
}
|
||||
|
||||
// NewWatcher creates a Watcher.
|
||||
func NewWatcher(ctx context.Context, opts WatcherOptions) (*Watcher, error) {
|
||||
if opts.Debounce <= 0 {
|
||||
opts.Debounce = 2 * time.Second
|
||||
}
|
||||
root, err := filepath.Abs(opts.Root)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to resolve root path: %w", err)
|
||||
}
|
||||
info, err := os.Stat(root)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to stat root path: %w", err)
|
||||
}
|
||||
if !info.IsDir() {
|
||||
return nil, fmt.Errorf("watch path must be a directory: %s", root)
|
||||
}
|
||||
|
||||
fsw, err := fsnotify.NewWatcher()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to create fsnotify watcher: %w", err)
|
||||
}
|
||||
|
||||
w := &Watcher{
|
||||
root: root,
|
||||
recursive: opts.Recursive,
|
||||
debounce: opts.Debounce,
|
||||
uploader: opts.Uploader,
|
||||
logger: log.FromContext(ctx).WithPrefix("watcher"),
|
||||
fsw: fsw,
|
||||
pending: make(map[string]*time.Timer),
|
||||
lastSize: make(map[string]int64),
|
||||
}
|
||||
return w, nil
|
||||
}
|
||||
|
||||
// Run starts watching and blocks until ctx is cancelled.
|
||||
func (w *Watcher) Run(ctx context.Context) error {
|
||||
if err := w.addDir(w.root); err != nil {
|
||||
w.fsw.Close()
|
||||
return fmt.Errorf("failed to watch root: %w", err)
|
||||
}
|
||||
w.logger.Infof("watching %s (recursive=%v, debounce=%s)", w.root, w.recursive, w.debounce)
|
||||
|
||||
defer w.cleanup()
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
w.logger.Info("stopping watcher")
|
||||
return nil
|
||||
case event, ok := <-w.fsw.Events:
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
w.handleEvent(ctx, event)
|
||||
case err, ok := <-w.fsw.Errors:
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
w.logger.Errorf("watch error: %v", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (w *Watcher) handleEvent(ctx context.Context, event fsnotify.Event) {
|
||||
// Remove/Rename: cancel any pending upload for this path.
|
||||
if event.Has(fsnotify.Remove) || event.Has(fsnotify.Rename) {
|
||||
w.cancelPending(event.Name)
|
||||
return
|
||||
}
|
||||
|
||||
if !event.Has(fsnotify.Create) && !event.Has(fsnotify.Write) {
|
||||
return
|
||||
}
|
||||
|
||||
info, err := os.Stat(event.Name)
|
||||
if err != nil {
|
||||
// File may have been removed or moved; ignore.
|
||||
return
|
||||
}
|
||||
|
||||
if info.IsDir() {
|
||||
// New directory: watch it recursively and scan files already inside.
|
||||
if event.Has(fsnotify.Create) && w.recursive {
|
||||
if err := w.addDir(event.Name); err != nil {
|
||||
w.logger.Errorf("failed to watch new dir %s: %v", event.Name, err)
|
||||
}
|
||||
w.scanExisting(ctx, event.Name)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
w.scheduleUpload(ctx, event.Name)
|
||||
}
|
||||
|
||||
// scheduleUpload schedules a debounced upload for a file.
|
||||
func (w *Watcher) scheduleUpload(ctx context.Context, file string) {
|
||||
w.mu.Lock()
|
||||
defer w.mu.Unlock()
|
||||
|
||||
if t, ok := w.pending[file]; ok {
|
||||
t.Stop()
|
||||
}
|
||||
w.pending[file] = time.AfterFunc(w.debounce, func() {
|
||||
w.maybeUpload(ctx, file)
|
||||
})
|
||||
}
|
||||
|
||||
// maybeUpload submits the upload once the debounce window passes and the file
|
||||
// size is stable; otherwise it waits another window.
|
||||
func (w *Watcher) maybeUpload(ctx context.Context, file string) {
|
||||
if ctx.Err() != nil {
|
||||
return
|
||||
}
|
||||
|
||||
info, err := os.Stat(file)
|
||||
if err != nil {
|
||||
w.cancelPending(file)
|
||||
return
|
||||
}
|
||||
if info.IsDir() {
|
||||
w.cancelPending(file)
|
||||
return
|
||||
}
|
||||
|
||||
w.mu.Lock()
|
||||
prevSize, seen := w.lastSize[file]
|
||||
curSize := info.Size()
|
||||
if !seen || prevSize != curSize {
|
||||
// Size still changing: likely still being written, wait another window.
|
||||
w.lastSize[file] = curSize
|
||||
w.pending[file] = time.AfterFunc(w.debounce, func() {
|
||||
w.maybeUpload(ctx, file)
|
||||
})
|
||||
w.mu.Unlock()
|
||||
return
|
||||
}
|
||||
// Size stable: treat write as complete.
|
||||
delete(w.pending, file)
|
||||
delete(w.lastSize, file)
|
||||
w.mu.Unlock()
|
||||
|
||||
relPath, err := filepath.Rel(w.root, file)
|
||||
if err != nil {
|
||||
w.logger.Errorf("failed to compute relative path for %s: %v", file, err)
|
||||
return
|
||||
}
|
||||
|
||||
w.uploader.Submit(ctx, uploadJob{localPath: file, relPath: relPath})
|
||||
}
|
||||
|
||||
func (w *Watcher) cancelPending(file string) {
|
||||
w.mu.Lock()
|
||||
defer w.mu.Unlock()
|
||||
if t, ok := w.pending[file]; ok {
|
||||
t.Stop()
|
||||
delete(w.pending, file)
|
||||
}
|
||||
delete(w.lastSize, file)
|
||||
}
|
||||
|
||||
// addDir adds a directory to the watch list, recursively when enabled.
|
||||
func (w *Watcher) addDir(dir string) error {
|
||||
if !w.recursive {
|
||||
return w.fsw.Add(dir)
|
||||
}
|
||||
return filepath.WalkDir(dir, func(p string, d fs.DirEntry, err error) error {
|
||||
if err != nil {
|
||||
w.logger.Warnf("skip path %s: %v", p, err)
|
||||
return nil
|
||||
}
|
||||
if d.IsDir() {
|
||||
if addErr := w.fsw.Add(p); addErr != nil {
|
||||
w.logger.Warnf("failed to watch dir %s: %v", p, addErr)
|
||||
} else {
|
||||
w.logger.Debugf("watching dir %s", p)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
// scanExisting submits files already present under dir (initial sync and new-dir backfill).
|
||||
func (w *Watcher) scanExisting(ctx context.Context, dir string) {
|
||||
walkFn := func(p string, d fs.DirEntry, err error) error {
|
||||
if err != nil {
|
||||
w.logger.Warnf("skip path %s: %v", p, err)
|
||||
return nil
|
||||
}
|
||||
if ctx.Err() != nil {
|
||||
return ctx.Err()
|
||||
}
|
||||
if d.IsDir() {
|
||||
if !w.recursive && p != dir {
|
||||
return fs.SkipDir
|
||||
}
|
||||
return nil
|
||||
}
|
||||
relPath, relErr := filepath.Rel(w.root, p)
|
||||
if relErr != nil {
|
||||
w.logger.Errorf("failed to compute relative path for %s: %v", p, relErr)
|
||||
return nil
|
||||
}
|
||||
w.uploader.Submit(ctx, uploadJob{localPath: p, relPath: relPath})
|
||||
return nil
|
||||
}
|
||||
if err := filepath.WalkDir(dir, walkFn); err != nil && ctx.Err() == nil {
|
||||
w.logger.Errorf("failed to scan dir %s: %v", dir, err)
|
||||
}
|
||||
}
|
||||
|
||||
// ScanExisting triggers a one-time scan and upload of existing files under the watch root.
|
||||
func (w *Watcher) ScanExisting(ctx context.Context) {
|
||||
w.logger.Info("scanning existing files for initial sync")
|
||||
w.scanExisting(ctx, w.root)
|
||||
}
|
||||
|
||||
func (w *Watcher) cleanup() {
|
||||
w.mu.Lock()
|
||||
for _, t := range w.pending {
|
||||
t.Stop()
|
||||
}
|
||||
w.pending = make(map[string]*time.Timer)
|
||||
w.lastSize = make(map[string]int64)
|
||||
w.mu.Unlock()
|
||||
w.fsw.Close()
|
||||
}
|
||||
33
common/cache/ristretto.go
vendored
33
common/cache/ristretto.go
vendored
@@ -2,6 +2,7 @@ package cache
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/charmbracelet/log"
|
||||
@@ -9,24 +10,26 @@ import (
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
)
|
||||
|
||||
var cache *ristretto.Cache[string, any]
|
||||
var (
|
||||
cache *ristretto.Cache[string, any]
|
||||
initOnce sync.Once
|
||||
)
|
||||
|
||||
func Init() {
|
||||
if cache != nil {
|
||||
panic("cache already initialized")
|
||||
}
|
||||
c, err := ristretto.NewCache(&ristretto.Config[string, any]{
|
||||
NumCounters: config.C().Cache.NumCounters,
|
||||
MaxCost: config.C().Cache.MaxCost,
|
||||
BufferItems: 64,
|
||||
OnReject: func(item *ristretto.Item[any]) {
|
||||
log.Warnf("Cache item rejected: key=%d, value=%v", item.Key, item.Value)
|
||||
},
|
||||
initOnce.Do(func() {
|
||||
c, err := ristretto.NewCache(&ristretto.Config[string, any]{
|
||||
NumCounters: config.C().Cache.NumCounters,
|
||||
MaxCost: config.C().Cache.MaxCost,
|
||||
BufferItems: 64,
|
||||
OnReject: func(item *ristretto.Item[any]) {
|
||||
log.Warnf("Cache item rejected: key=%d, value=%v", item.Key, item.Value)
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
log.Fatalf("failed to create ristretto cache: %v", err)
|
||||
}
|
||||
cache = c
|
||||
})
|
||||
if err != nil {
|
||||
log.Fatalf("failed to create ristretto cache: %v", err)
|
||||
}
|
||||
cache = c
|
||||
}
|
||||
|
||||
func Set(key string, value any) error {
|
||||
|
||||
@@ -44,57 +44,7 @@ func Init(lang string) {
|
||||
|
||||
func T(key i18nk.Key, templateData ...map[string]any) string {
|
||||
if localizer == nil || bundle == nil {
|
||||
panic("localizer or bundle is not initialized, call Init() first")
|
||||
}
|
||||
templateDataMap := make(map[string]any)
|
||||
for _, data := range templateData {
|
||||
maps.Copy(templateDataMap, data)
|
||||
}
|
||||
msg, err := localizer.Localize(&i18n.LocalizeConfig{
|
||||
MessageID: string(key),
|
||||
TemplateData: templateDataMap,
|
||||
})
|
||||
if err != nil {
|
||||
return string(key)
|
||||
}
|
||||
return msg
|
||||
}
|
||||
|
||||
func TWithLang(lang, key string, templateData ...map[string]any) string {
|
||||
if bundle == nil {
|
||||
panic("bundle is not initialized, call Init() first")
|
||||
}
|
||||
templateDataMap := make(map[string]any)
|
||||
for _, data := range templateData {
|
||||
maps.Copy(templateDataMap, data)
|
||||
}
|
||||
localizerWithLang := i18n.NewLocalizer(bundle, lang)
|
||||
msg, err := localizerWithLang.Localize(&i18n.LocalizeConfig{
|
||||
MessageID: key,
|
||||
TemplateData: templateDataMap,
|
||||
})
|
||||
if err != nil {
|
||||
return key
|
||||
}
|
||||
return msg
|
||||
}
|
||||
|
||||
// Only use in tests or packages that load before i18n
|
||||
func TWithoutInit(lang string, key i18nk.Key, templateData ...map[string]any) string {
|
||||
bundle := i18n.NewBundle(language.SimplifiedChinese)
|
||||
bundle.RegisterUnmarshalFunc("yaml", yaml.Unmarshal)
|
||||
files, err := localesFS.ReadDir("locale")
|
||||
if err != nil {
|
||||
return string(key)
|
||||
}
|
||||
for _, file := range files {
|
||||
if _, err := bundle.LoadMessageFileFS(localesFS, "locale/"+file.Name()); err != nil {
|
||||
return string(key)
|
||||
}
|
||||
}
|
||||
localizer := i18n.NewLocalizer(bundle, lang)
|
||||
if localizer == nil {
|
||||
return string(key)
|
||||
Init("zh-Hans")
|
||||
}
|
||||
templateDataMap := make(map[string]any)
|
||||
for _, data := range templateData {
|
||||
|
||||
@@ -4,16 +4,24 @@ package i18nk
|
||||
type Key string
|
||||
|
||||
const (
|
||||
BotMsgAria2ErrorAddingAria2Download Key = "bot.msg.aria2.error_adding_aria2_download"
|
||||
BotMsgAria2ErrorAria2ClientInitFailed Key = "bot.msg.aria2.error_aria2_client_init_failed"
|
||||
BotMsgAria2ErrorAria2NotEnabled Key = "bot.msg.aria2.error_aria2_not_enabled"
|
||||
BotMsgAria2InfoAddingAria2Download Key = "bot.msg.aria2.info_adding_aria2_download"
|
||||
BotMsgAria2InfoAria2DownloadAdded Key = "bot.msg.aria2.info_aria2_download_added"
|
||||
BotMsgAria2InfoSelectStorage Key = "bot.msg.aria2.info_select_storage"
|
||||
BotMsgCancelErrorCancelFailed Key = "bot.msg.cancel.error_cancel_failed"
|
||||
BotMsgCancelInfoCancelRequested Key = "bot.msg.cancel.info_cancel_requested"
|
||||
BotMsgCancelInfoCancellingTask Key = "bot.msg.cancel.info_cancelling_task"
|
||||
BotMsgCancelUsage Key = "bot.msg.cancel.usage"
|
||||
BotMsgCmdAria2dl Key = "bot.msg.cmd.aria2dl"
|
||||
BotMsgCmdCancel Key = "bot.msg.cmd.cancel"
|
||||
BotMsgCmdConfig Key = "bot.msg.cmd.config"
|
||||
BotMsgCmdDir Key = "bot.msg.cmd.dir"
|
||||
BotMsgCmdDl Key = "bot.msg.cmd.dl"
|
||||
BotMsgCmdFnametmpl Key = "bot.msg.cmd.fnametmpl"
|
||||
BotMsgCmdHelp Key = "bot.msg.cmd.help"
|
||||
BotMsgCmdImport Key = "bot.msg.cmd.import"
|
||||
BotMsgCmdLswatch Key = "bot.msg.cmd.lswatch"
|
||||
BotMsgCmdParser Key = "bot.msg.cmd.parser"
|
||||
BotMsgCmdRule Key = "bot.msg.cmd.rule"
|
||||
@@ -21,10 +29,16 @@ const (
|
||||
BotMsgCmdSilent Key = "bot.msg.cmd.silent"
|
||||
BotMsgCmdStart Key = "bot.msg.cmd.start"
|
||||
BotMsgCmdStorage Key = "bot.msg.cmd.storage"
|
||||
BotMsgCmdSyncpeers Key = "bot.msg.cmd.syncpeers"
|
||||
BotMsgCmdTask Key = "bot.msg.cmd.task"
|
||||
BotMsgCmdTransfer Key = "bot.msg.cmd.transfer"
|
||||
BotMsgCmdUnwatch Key = "bot.msg.cmd.unwatch"
|
||||
BotMsgCmdUpdate Key = "bot.msg.cmd.update"
|
||||
BotMsgCmdWatch Key = "bot.msg.cmd.watch"
|
||||
BotMsgCmdYtdlp Key = "bot.msg.cmd.ytdlp"
|
||||
BotMsgCommonButtonConflictOverwrite Key = "bot.msg.common.button_conflict_overwrite"
|
||||
BotMsgCommonButtonConflictRename Key = "bot.msg.common.button_conflict_rename"
|
||||
BotMsgCommonButtonConflictSkip Key = "bot.msg.common.button_conflict_skip"
|
||||
BotMsgCommonCancelButtonText Key = "bot.msg.common.cancel_button_text"
|
||||
BotMsgCommonErrorBuildDirSelectKeyboardFailed Key = "bot.msg.common.error_build_dir_select_keyboard_failed"
|
||||
BotMsgCommonErrorBuildStorageSelectKeyboardFailed Key = "bot.msg.common.error_build_storage_select_keyboard_failed"
|
||||
@@ -52,7 +66,10 @@ const (
|
||||
BotMsgCommonErrorTaskAddFailed Key = "bot.msg.common.error_task_add_failed"
|
||||
BotMsgCommonErrorTaskCreateFailed Key = "bot.msg.common.error_task_create_failed"
|
||||
BotMsgCommonErrorUpdateUserInfoFailed Key = "bot.msg.common.error_update_user_info_failed"
|
||||
BotMsgCommonInfoAllConflictFilesSkipped Key = "bot.msg.common.info_all_conflict_files_skipped"
|
||||
BotMsgCommonInfoBatchTasksAdded Key = "bot.msg.common.info_batch_tasks_added"
|
||||
BotMsgCommonInfoBatchTasksAddedWithSkipped Key = "bot.msg.common.info_batch_tasks_added_with_skipped"
|
||||
BotMsgCommonInfoConflictFilesSkipped Key = "bot.msg.common.info_conflict_files_skipped"
|
||||
BotMsgCommonInfoDefaultStorageSet Key = "bot.msg.common.info_default_storage_set"
|
||||
BotMsgCommonInfoDefaultStorageWithDirSet Key = "bot.msg.common.info_default_storage_with_dir_set"
|
||||
BotMsgCommonInfoFetchingFileInfo Key = "bot.msg.common.info_fetching_file_info"
|
||||
@@ -62,16 +79,25 @@ const (
|
||||
BotMsgCommonInfoSilentModeOff Key = "bot.msg.common.info_silent_mode_off"
|
||||
BotMsgCommonInfoSilentModeOn Key = "bot.msg.common.info_silent_mode_on"
|
||||
BotMsgCommonInfoTaskAdded Key = "bot.msg.common.info_task_added"
|
||||
BotMsgCommonPromptConflictMoreFiles Key = "bot.msg.common.prompt_conflict_more_files"
|
||||
BotMsgCommonPromptSelectConflictStrategy Key = "bot.msg.common.prompt_select_conflict_strategy"
|
||||
BotMsgCommonPromptSelectDefaultDir Key = "bot.msg.common.prompt_select_default_dir"
|
||||
BotMsgCommonPromptSelectDefaultStorage Key = "bot.msg.common.prompt_select_default_storage"
|
||||
BotMsgCommonPromptSelectDir Key = "bot.msg.common.prompt_select_dir"
|
||||
BotMsgConfigButtonConflictStrategy Key = "bot.msg.config.button_conflict_strategy"
|
||||
BotMsgConfigButtonFilenameStrategy Key = "bot.msg.config.button_filename_strategy"
|
||||
BotMsgConfigConflictStrategyAsk Key = "bot.msg.config.conflict_strategy_ask"
|
||||
BotMsgConfigConflictStrategyOverwrite Key = "bot.msg.config.conflict_strategy_overwrite"
|
||||
BotMsgConfigConflictStrategyRename Key = "bot.msg.config.conflict_strategy_rename"
|
||||
BotMsgConfigConflictStrategySkip Key = "bot.msg.config.conflict_strategy_skip"
|
||||
BotMsgConfigErrorInvalidCallbackData Key = "bot.msg.config.error_invalid_callback_data"
|
||||
BotMsgConfigErrorInvalidTemplate Key = "bot.msg.config.error_invalid_template"
|
||||
BotMsgConfigFnametmplHelp Key = "bot.msg.config.fnametmpl_help"
|
||||
BotMsgConfigInfoConflictStrategySet Key = "bot.msg.config.info_conflict_strategy_set"
|
||||
BotMsgConfigInfoCurrentTemplatePrefix Key = "bot.msg.config.info_current_template_prefix"
|
||||
BotMsgConfigInfoFilenameStrategySet Key = "bot.msg.config.info_filename_strategy_set"
|
||||
BotMsgConfigInfoTemplateUpdated Key = "bot.msg.config.info_template_updated"
|
||||
BotMsgConfigPromptSelectConflictStrategy Key = "bot.msg.config.prompt_select_conflict_strategy"
|
||||
BotMsgConfigPromptSelectFilenameStrategy Key = "bot.msg.config.prompt_select_filename_strategy"
|
||||
BotMsgConfigPromptSelectOption Key = "bot.msg.config.prompt_select_option"
|
||||
BotMsgDirButtonDefault Key = "bot.msg.dir.button_default"
|
||||
@@ -120,38 +146,85 @@ const (
|
||||
BotMsgParserInfoInstallPluginSuccess Key = "bot.msg.parser.info_install_plugin_success"
|
||||
BotMsgParserPluginNotEnabled Key = "bot.msg.parser.plugin_not_enabled"
|
||||
BotMsgParserPromptReplyWithParserFile Key = "bot.msg.parser.prompt_reply_with_parser_file"
|
||||
BotMsgProgressAria2Done Key = "bot.msg.progress.aria2_done"
|
||||
BotMsgProgressAria2Downloading Key = "bot.msg.progress.aria2_downloading"
|
||||
BotMsgProgressAria2Start Key = "bot.msg.progress.aria2_start"
|
||||
BotMsgProgressAvgSpeedPrefix Key = "bot.msg.progress.avg_speed_prefix"
|
||||
BotMsgProgressBatchDonePrefix Key = "bot.msg.progress.batch_done_prefix"
|
||||
BotMsgProgressBatchProcessingPrefix Key = "bot.msg.progress.batch_processing_prefix"
|
||||
BotMsgProgressBatchStartPrefix Key = "bot.msg.progress.batch_start_prefix"
|
||||
BotMsgProgressBatchCanceled Key = "bot.msg.progress.batch_canceled"
|
||||
BotMsgProgressBatchDone Key = "bot.msg.progress.batch_done"
|
||||
BotMsgProgressBatchDoneWithSkipped Key = "bot.msg.progress.batch_done_with_skipped"
|
||||
BotMsgProgressBatchFailedGroup Key = "bot.msg.progress.batch_failed_group"
|
||||
BotMsgProgressBatchFailedItem Key = "bot.msg.progress.batch_failed_item"
|
||||
BotMsgProgressBatchFailedTask Key = "bot.msg.progress.batch_failed_task"
|
||||
BotMsgProgressBatchFailureStageBatchUpload Key = "bot.msg.progress.batch_failure_stage_batch_upload"
|
||||
BotMsgProgressBatchFailureStageCache Key = "bot.msg.progress.batch_failure_stage_cache"
|
||||
BotMsgProgressBatchFailureStageConfirm Key = "bot.msg.progress.batch_failure_stage_confirm"
|
||||
BotMsgProgressBatchFailureStageDownload Key = "bot.msg.progress.batch_failure_stage_download"
|
||||
BotMsgProgressBatchFailureStageInternal Key = "bot.msg.progress.batch_failure_stage_internal"
|
||||
BotMsgProgressBatchFailureStageUpload Key = "bot.msg.progress.batch_failure_stage_upload"
|
||||
BotMsgProgressBatchItemConfirming Key = "bot.msg.progress.batch_item_confirming"
|
||||
BotMsgProgressBatchItemDownloading Key = "bot.msg.progress.batch_item_downloading"
|
||||
BotMsgProgressBatchItemDownloadingUnknown Key = "bot.msg.progress.batch_item_downloading_unknown"
|
||||
BotMsgProgressBatchItemRetrying Key = "bot.msg.progress.batch_item_retrying"
|
||||
BotMsgProgressBatchItemTransferring Key = "bot.msg.progress.batch_item_transferring"
|
||||
BotMsgProgressBatchItemTransferringUnknown Key = "bot.msg.progress.batch_item_transferring_unknown"
|
||||
BotMsgProgressBatchItemUploading Key = "bot.msg.progress.batch_item_uploading"
|
||||
BotMsgProgressBatchStatusHeader Key = "bot.msg.progress.batch_status_header"
|
||||
BotMsgProgressBatchSummaryConfirming Key = "bot.msg.progress.batch_summary_confirming"
|
||||
BotMsgProgressBatchSummaryFailed Key = "bot.msg.progress.batch_summary_failed"
|
||||
BotMsgProgressBatchSummaryHiddenActive Key = "bot.msg.progress.batch_summary_hidden_active"
|
||||
BotMsgProgressBatchSummarySkipped Key = "bot.msg.progress.batch_summary_skipped"
|
||||
BotMsgProgressCurrentProgressPrefix Key = "bot.msg.progress.current_progress_prefix"
|
||||
BotMsgProgressCurrentSpeedPrefix Key = "bot.msg.progress.current_speed_prefix"
|
||||
BotMsgProgressDirectDonePrefix Key = "bot.msg.progress.direct_done_prefix"
|
||||
BotMsgProgressDirectStart Key = "bot.msg.progress.direct_start"
|
||||
BotMsgProgressDownloadDonePrefix Key = "bot.msg.progress.download_done_prefix"
|
||||
BotMsgProgressDownloadFailedPrefix Key = "bot.msg.progress.download_failed_prefix"
|
||||
BotMsgProgressDownloadedPrefix Key = "bot.msg.progress.downloaded_prefix"
|
||||
BotMsgProgressDownloadingPrefix Key = "bot.msg.progress.downloading_prefix"
|
||||
BotMsgProgressErrorPrefix Key = "bot.msg.progress.error_prefix"
|
||||
BotMsgProgressFileNamePrefix Key = "bot.msg.progress.file_name_prefix"
|
||||
BotMsgProgressFileProcessingPrefix Key = "bot.msg.progress.file_processing_prefix"
|
||||
BotMsgProgressFileSizePrefix Key = "bot.msg.progress.file_size_prefix"
|
||||
BotMsgProgressFileStartPrefix Key = "bot.msg.progress.file_start_prefix"
|
||||
BotMsgProgressParsedDonePrefix Key = "bot.msg.progress.parsed_done_prefix"
|
||||
BotMsgProgressParsedStartPrefix Key = "bot.msg.progress.parsed_start_prefix"
|
||||
BotMsgProgressProcessingListPrefix Key = "bot.msg.progress.processing_list_prefix"
|
||||
BotMsgProgressProcessingNone Key = "bot.msg.progress.processing_none"
|
||||
BotMsgProgressSavePathPrefix Key = "bot.msg.progress.save_path_prefix"
|
||||
BotMsgProgressTaskCanceled Key = "bot.msg.progress.task_canceled"
|
||||
BotMsgProgressSingleCanceled Key = "bot.msg.progress.single_canceled"
|
||||
BotMsgProgressSingleDone Key = "bot.msg.progress.single_done"
|
||||
BotMsgProgressSingleDownloading Key = "bot.msg.progress.single_downloading"
|
||||
BotMsgProgressSingleDownloadingUnknown Key = "bot.msg.progress.single_downloading_unknown"
|
||||
BotMsgProgressSingleFailed Key = "bot.msg.progress.single_failed"
|
||||
BotMsgProgressSingleStatusHeader Key = "bot.msg.progress.single_status_header"
|
||||
BotMsgProgressSingleUploadRetrying Key = "bot.msg.progress.single_upload_retrying"
|
||||
BotMsgProgressSingleUploading Key = "bot.msg.progress.single_uploading"
|
||||
BotMsgProgressSizeWithFiles Key = "bot.msg.progress.size_with_files"
|
||||
BotMsgProgressSizeWithResources Key = "bot.msg.progress.size_with_resources"
|
||||
BotMsgProgressTaskCanceledWithId Key = "bot.msg.progress.task_canceled_with_id"
|
||||
BotMsgProgressTaskFailedWithError Key = "bot.msg.progress.task_failed_with_error"
|
||||
BotMsgProgressTelegraphDonePrefix Key = "bot.msg.progress.telegraph_done_prefix"
|
||||
BotMsgProgressTelegraphProgressPrefix Key = "bot.msg.progress.telegraph_progress_prefix"
|
||||
BotMsgProgressTelegraphStartPrefix Key = "bot.msg.progress.telegraph_start_prefix"
|
||||
BotMsgProgressTotalSizePrefix Key = "bot.msg.progress.total_size_prefix"
|
||||
BotMsgProgressTransferAvgSpeedPrefix Key = "bot.msg.progress.transfer_avg_speed_prefix"
|
||||
BotMsgProgressTransferElapsedTimePrefix Key = "bot.msg.progress.transfer_elapsed_time_prefix"
|
||||
BotMsgProgressTransferFailedFilesPrefix Key = "bot.msg.progress.transfer_failed_files_prefix"
|
||||
BotMsgProgressTransferFailedPrefix Key = "bot.msg.progress.transfer_failed_prefix"
|
||||
BotMsgProgressTransferProcessingMore Key = "bot.msg.progress.transfer_processing_more"
|
||||
BotMsgProgressTransferProcessingPrefix Key = "bot.msg.progress.transfer_processing_prefix"
|
||||
BotMsgProgressTransferProgressPrefix Key = "bot.msg.progress.transfer_progress_prefix"
|
||||
BotMsgProgressTransferRemainingTimePrefix Key = "bot.msg.progress.transfer_remaining_time_prefix"
|
||||
BotMsgProgressTransferSpeedPrefix Key = "bot.msg.progress.transfer_speed_prefix"
|
||||
BotMsgProgressTransferStartPrefix Key = "bot.msg.progress.transfer_start_prefix"
|
||||
BotMsgProgressTransferSuccessPrefix Key = "bot.msg.progress.transfer_success_prefix"
|
||||
BotMsgProgressTransferTotalFilesPrefix Key = "bot.msg.progress.transfer_total_files_prefix"
|
||||
BotMsgProgressTransferTotalSizePrefix Key = "bot.msg.progress.transfer_total_size_prefix"
|
||||
BotMsgProgressTransferUploadedPrefix Key = "bot.msg.progress.transfer_uploaded_prefix"
|
||||
BotMsgProgressYtdlpDone Key = "bot.msg.progress.ytdlp_done"
|
||||
BotMsgProgressYtdlpDownloading Key = "bot.msg.progress.ytdlp_downloading"
|
||||
BotMsgProgressYtdlpStart Key = "bot.msg.progress.ytdlp_start"
|
||||
BotMsgRuleErrorCreateRuleFailed Key = "bot.msg.rule.error_create_rule_failed"
|
||||
BotMsgRuleErrorDeleteRuleFailed Key = "bot.msg.rule.error_delete_rule_failed"
|
||||
BotMsgRuleErrorGetUserRulesFailed Key = "bot.msg.rule.error_get_user_rules_failed"
|
||||
BotMsgRuleErrorInvalidRuleId Key = "bot.msg.rule.error_invalid_rule_id"
|
||||
BotMsgRuleErrorInvalidRuleType Key = "bot.msg.rule.error_invalid_rule_type"
|
||||
BotMsgRuleErrorStorageNotFound Key = "bot.msg.rule.error_storage_not_found"
|
||||
BotMsgRuleErrorUpdateUserFailed Key = "bot.msg.rule.error_update_user_failed"
|
||||
BotMsgRuleHelpAddSuffix Key = "bot.msg.rule.help_add_suffix"
|
||||
BotMsgRuleHelpAvailableOps Key = "bot.msg.rule.help_available_ops"
|
||||
@@ -159,17 +232,23 @@ const (
|
||||
BotMsgRuleHelpCurrentModeEnabled Key = "bot.msg.rule.help_current_mode_enabled"
|
||||
BotMsgRuleHelpDelSuffix Key = "bot.msg.rule.help_del_suffix"
|
||||
BotMsgRuleHelpExistingRulesPrefix Key = "bot.msg.rule.help_existing_rules_prefix"
|
||||
BotMsgRuleHelpPresetSuffix Key = "bot.msg.rule.help_preset_suffix"
|
||||
BotMsgRuleHelpSwitchSuffix Key = "bot.msg.rule.help_switch_suffix"
|
||||
BotMsgRuleHelpUsage Key = "bot.msg.rule.help_usage"
|
||||
BotMsgRuleInfoCreateRuleSuccess Key = "bot.msg.rule.info_create_rule_success"
|
||||
BotMsgRuleInfoDeleteRuleSuccess Key = "bot.msg.rule.info_delete_rule_success"
|
||||
BotMsgRuleInfoPresetImported Key = "bot.msg.rule.info_preset_imported"
|
||||
BotMsgRuleInfoRuleModeDisabled Key = "bot.msg.rule.info_rule_mode_disabled"
|
||||
BotMsgRuleInfoRuleModeEnabled Key = "bot.msg.rule.info_rule_mode_enabled"
|
||||
BotMsgRulePromptProvideRuleId Key = "bot.msg.rule.prompt_provide_rule_id"
|
||||
BotMsgRulePromptProvideStorageName Key = "bot.msg.rule.prompt_provide_storage_name"
|
||||
BotMsgSaveErrorInvalidIdOrUsername Key = "bot.msg.save.error_invalid_id_or_username"
|
||||
BotMsgSaveHelpText Key = "bot.msg.save_help_text"
|
||||
BotMsgStorageInfoFilenamePrefix Key = "bot.msg.storage.info_filename_prefix"
|
||||
BotMsgStorageInfoPromptSelectStorage Key = "bot.msg.storage.info_prompt_select_storage"
|
||||
BotMsgSyncpeersFailed Key = "bot.msg.syncpeers.failed"
|
||||
BotMsgSyncpeersStart Key = "bot.msg.syncpeers.start"
|
||||
BotMsgSyncpeersSuccess Key = "bot.msg.syncpeers.success"
|
||||
BotMsgTasksCancelFailed Key = "bot.msg.tasks.cancel_failed"
|
||||
BotMsgTasksCancelRequestedPrefix Key = "bot.msg.tasks.cancel_requested_prefix"
|
||||
BotMsgTasksFieldCreated Key = "bot.msg.tasks.field_created"
|
||||
@@ -195,6 +274,22 @@ const (
|
||||
BotMsgTelegraphInfoPicCountPrefix Key = "bot.msg.telegraph.info_pic_count_prefix"
|
||||
BotMsgTelegraphInfoPromptSelectStorage Key = "bot.msg.telegraph.info_prompt_select_storage"
|
||||
BotMsgTelegraphInfoTitlePrefix Key = "bot.msg.telegraph.info_title_prefix"
|
||||
BotMsgTransferErrorAddTaskFailed Key = "bot.msg.transfer.error_add_task_failed"
|
||||
BotMsgTransferErrorBuildStorageSelectKeyboardFailed Key = "bot.msg.transfer.error_build_storage_select_keyboard_failed"
|
||||
BotMsgTransferErrorInvalidRegex Key = "bot.msg.transfer.error_invalid_regex"
|
||||
BotMsgTransferErrorInvalidSource Key = "bot.msg.transfer.error_invalid_source"
|
||||
BotMsgTransferErrorInvalidTarget Key = "bot.msg.transfer.error_invalid_target"
|
||||
BotMsgTransferErrorListFilesFailed Key = "bot.msg.transfer.error_list_files_failed"
|
||||
BotMsgTransferErrorNoFilesToTransfer Key = "bot.msg.transfer.error_no_files_to_transfer"
|
||||
BotMsgTransferErrorStorageNotFound Key = "bot.msg.transfer.error_storage_not_found"
|
||||
BotMsgTransferErrorStorageNotListable Key = "bot.msg.transfer.error_storage_not_listable"
|
||||
BotMsgTransferErrorStorageNotReadable Key = "bot.msg.transfer.error_storage_not_readable"
|
||||
BotMsgTransferErrorTargetNotFound Key = "bot.msg.transfer.error_target_not_found"
|
||||
BotMsgTransferInfoFetchingFiles Key = "bot.msg.transfer.info_fetching_files"
|
||||
BotMsgTransferInfoFilesSelectStorage Key = "bot.msg.transfer.info_files_select_storage"
|
||||
BotMsgTransferInfoTaskAdded Key = "bot.msg.transfer.info_task_added"
|
||||
BotMsgTransferStartStats Key = "bot.msg.transfer.start_stats"
|
||||
BotMsgTransferUsage Key = "bot.msg.transfer.usage"
|
||||
BotMsgUpdateButtonUpgrade Key = "bot.msg.update.button_upgrade"
|
||||
BotMsgUpdateErrorCheckLatestFailed Key = "bot.msg.update.error_check_latest_failed"
|
||||
BotMsgUpdateErrorNoReleaseFound Key = "bot.msg.update.error_no_release_found"
|
||||
@@ -218,6 +313,11 @@ const (
|
||||
BotMsgWatchInfoWatchListFilterPrefix Key = "bot.msg.watch.info_watch_list_filter_prefix"
|
||||
BotMsgWatchInfoWatchListHeader Key = "bot.msg.watch.info_watch_list_header"
|
||||
BotMsgWatchHelpText Key = "bot.msg.watch_help_text"
|
||||
BotMsgYtdlpErrorDownloadFailed Key = "bot.msg.ytdlp.error_download_failed"
|
||||
BotMsgYtdlpErrorNoValidUrls Key = "bot.msg.ytdlp.error_no_valid_urls"
|
||||
BotMsgYtdlpInfoDownloading Key = "bot.msg.ytdlp.info_downloading"
|
||||
BotMsgYtdlpInfoUrlsSelectStorage Key = "bot.msg.ytdlp.info_urls_select_storage"
|
||||
BotMsgYtdlpUsage Key = "bot.msg.ytdlp.usage"
|
||||
ConfigErrDuplicateStorageName Key = "config.err.duplicate_storage_name"
|
||||
ConfigErrInvalidCacheDir Key = "config.err.invalid_cache_dir"
|
||||
ErrCleanCacheFailed Key = "err.clean_cache_failed"
|
||||
|
||||
@@ -29,6 +29,7 @@ bot:
|
||||
/silent - Toggle silent mode
|
||||
/storage - Set default storage
|
||||
/save [custom filename] - Save file
|
||||
/import <storage_name> <dir_path> [channel_id] [filter] - Import files from storage to Telegram
|
||||
/dir - Manage storage directories
|
||||
/rule - Manage rules
|
||||
/config - Modify configuration
|
||||
@@ -38,6 +39,7 @@ bot:
|
||||
/watch - Watch chats and auto save (UserBot)
|
||||
/unwatch - Stop watching chats (UserBot)
|
||||
/lswatch - List watched chats (UserBot)
|
||||
/syncpeers - Sync peer chats (UserBot)
|
||||
/update - Check and upgrade to latest version
|
||||
|
||||
Usage guide: https://sabot.unv.app/usage
|
||||
@@ -49,6 +51,10 @@ bot:
|
||||
rule: "Manage auto-save rules"
|
||||
save: "Save files"
|
||||
dl: "Download files from given links"
|
||||
aria2dl: "Download files using Aria2"
|
||||
ytdlp: "Download video/audio using yt-dlp"
|
||||
import: "Import files from storage to Telegram"
|
||||
transfer: "Transfer files between storages"
|
||||
task: "Manage task queue"
|
||||
cancel: "Cancel task"
|
||||
watch: "Watch chats (UserBot)"
|
||||
@@ -59,6 +65,7 @@ bot:
|
||||
help: "Show help"
|
||||
parser: "Manage parsers"
|
||||
update: "Check for updates"
|
||||
syncpeers: "Sync peer chats (UserBot)"
|
||||
save_help_text: |
|
||||
Usage:
|
||||
|
||||
@@ -105,9 +112,17 @@ bot:
|
||||
error_task_add_failed: "Failed to add task: {{.Error}}"
|
||||
info_task_added: "Task added"
|
||||
info_batch_tasks_added: "Batch tasks added, total {{.Count}} files"
|
||||
info_batch_tasks_added_with_skipped: "Batch tasks added, total {{.Count}} files\nSkipped conflicting files:\n{{.Skipped}}"
|
||||
info_all_conflict_files_skipped: "All conflicting files were skipped:\n{{.Skipped}}"
|
||||
info_conflict_files_skipped: "Skipped conflicting files:\n{{.Skipped}}"
|
||||
error_task_create_failed: "Failed to create task: {{.Error}}"
|
||||
error_get_dir_failed: "Failed to get directory: {{.Error}}"
|
||||
prompt_select_dir: "Please select a directory to store to"
|
||||
prompt_select_conflict_strategy: "Files with the same name already exist. Please select a save strategy:\n{{.Files}}"
|
||||
prompt_conflict_more_files: "...and {{.Count}} more files"
|
||||
button_conflict_rename: "Rename"
|
||||
button_conflict_overwrite: "Overwrite"
|
||||
button_conflict_skip: "Skip"
|
||||
prompt_select_default_dir: "Please select a default directory to save to"
|
||||
info_default_storage_set: "Default storage set to: {{.Name}}"
|
||||
info_default_storage_with_dir_set: "Default storage set to: {{.Name}}:/{{.Dir}}"
|
||||
@@ -181,7 +196,11 @@ bot:
|
||||
help_switch_suffix: " - Toggle rule mode\n"
|
||||
help_add_suffix: " <type> <data> <storage_name> <path> - Add rule\n"
|
||||
help_del_suffix: " <rule_id> - Delete rule\n"
|
||||
help_preset_suffix: " <storage_name> [base_path] - Import built-in filetype rules (video/image/audio/document/archive)\n"
|
||||
help_existing_rules_prefix: "\nCurrent rules:\n"
|
||||
prompt_provide_storage_name: "Please provide a storage name"
|
||||
error_storage_not_found: "Storage not found: {{.Storage}}"
|
||||
info_preset_imported: "Imported {{.Count}} built-in classification rules into storage {{.Storage}}"
|
||||
dir:
|
||||
error_get_user_dirs_failed: "Failed to get user directories"
|
||||
error_get_user_failed: "Failed to get user"
|
||||
@@ -259,10 +278,17 @@ bot:
|
||||
config:
|
||||
prompt_select_option: "Please select an option to configure"
|
||||
button_filename_strategy: "Filename strategy"
|
||||
button_conflict_strategy: "Duplicate file strategy"
|
||||
error_invalid_callback_data: "Invalid callback data"
|
||||
error_invalid_template: "Invalid template, please check syntax\n{{.Error}}"
|
||||
info_filename_strategy_set: "Filename strategy set to: {{.Strategy}}"
|
||||
info_conflict_strategy_set: "Duplicate file strategy set to: {{.Strategy}}"
|
||||
prompt_select_filename_strategy: "Please select filename strategy, current strategy: {{.Strategy}}"
|
||||
prompt_select_conflict_strategy: "Please select duplicate file strategy, current strategy: {{.Strategy}}"
|
||||
conflict_strategy_rename: "Always rename"
|
||||
conflict_strategy_ask: "Ask every time"
|
||||
conflict_strategy_overwrite: "Always overwrite"
|
||||
conflict_strategy_skip: "Always skip"
|
||||
fnametmpl_help: |-
|
||||
Use this command to set filename template, for example:
|
||||
/fnametmpl Image_{{"{{.msgid}}"}}_{{"{{.msgdate}}"}}.jpg
|
||||
@@ -272,6 +298,7 @@ bot:
|
||||
- {{"{{.msgtags}}"}}: Tags in the message, joined with underscore
|
||||
- {{"{{.msggen}}"}}: Generated filename from the message
|
||||
- {{"{{.msgdate}}"}}: Message date, format YYYY-MM-DD_HH-MM-SS
|
||||
- {{"{{.msgraw}}"}}: Raw message text (unprocessed)
|
||||
- {{"{{.origname}}"}}: Original media filename (if any)
|
||||
- {{"{{.chatid}}"}}: Chat ID of the message
|
||||
|
||||
@@ -283,6 +310,34 @@ bot:
|
||||
usage: "Usage: /dl <url1> <url2> ..."
|
||||
error_no_valid_links: "No valid links to download"
|
||||
info_files_select_storage: "Total {{.Count}} files, please select storage"
|
||||
ytdlp:
|
||||
usage: "Usage: /ytdlp [OPTIONS] <URL1> [URL2] ...\nExamples:\n /ytdlp https://example.com/video\n /ytdlp --format best https://example.com/video\n /ytdlp --extract-audio --audio-format mp3 https://example.com/video"
|
||||
error_no_valid_urls: "No valid URLs"
|
||||
info_urls_select_storage: "Found {{.Count}} links, please select storage"
|
||||
info_downloading: "Downloading via yt-dlp..."
|
||||
error_download_failed: "yt-dlp download failed: {{.Error}}"
|
||||
transfer:
|
||||
usage: |
|
||||
Usage: /transfer <source_storage>:/<source_path> [filter]
|
||||
Examples:
|
||||
/transfer local1:/downloads
|
||||
/transfer alist1:/media/photos
|
||||
/transfer webdav1:/files ".*\.mp4$"
|
||||
error_invalid_source: "Invalid source path format, should be: storage_name:/path"
|
||||
error_invalid_target: "Invalid target path format, should be: storage_name:/path"
|
||||
error_storage_not_found: "Storage '{{.StorageName}}' not found or access denied: {{.Error}}"
|
||||
error_storage_not_listable: "Storage '{{.StorageName}}' does not support listing files"
|
||||
error_storage_not_readable: "Storage '{{.StorageName}}' does not support reading files"
|
||||
error_target_not_found: "Target storage '{{.StorageName}}' not found or access denied: {{.Error}}"
|
||||
info_fetching_files: "Fetching file list..."
|
||||
error_list_files_failed: "Failed to list files: {{.Error}}"
|
||||
error_invalid_regex: "Invalid regular expression: {{.Error}}"
|
||||
error_no_files_to_transfer: "No files to transfer in directory"
|
||||
error_add_task_failed: "Failed to add task: {{.Error}}"
|
||||
info_task_added: "Added {{.Count}} files to transfer queue\nTotal size: {{.SizeMB}} MB\nTask ID: {{.TaskID}}"
|
||||
start_stats: "Total files: {{.Count}}\nTotal size: {{.SizeMB}} MB"
|
||||
info_files_select_storage: "Total {{.Count}} files ({{.SizeMB}} MB), please select target storage"
|
||||
error_build_storage_select_keyboard_failed: "Failed to build storage selection keyboard: {{.Error}}"
|
||||
cancel:
|
||||
usage: "Usage: /cancel <task_id>"
|
||||
error_cancel_failed: "Failed to cancel task: {{.Error}}"
|
||||
@@ -296,30 +351,87 @@ bot:
|
||||
info_filename_prefix: "Filename: "
|
||||
info_prompt_select_storage: "\nPlease select storage"
|
||||
progress:
|
||||
batch_start_prefix: "Starting batch download task\nTotal size: "
|
||||
batch_processing_prefix: "Processing batch download task\nTotal size: "
|
||||
batch_status_header: "<b>📦 Processing</b>\n\nFiles: <code>{{.Total}}</code> | Total size: <code>{{.TotalSize}}</code>\nStatus: ✅ <code>{{.Completed}}</code> | 📥 <code>{{.Downloaded}}</code> | ⏳ <code>{{.Waiting}}</code>\nTotal speed: ⬇️ <code>{{.DownloadSpeed}}</code> | ⬆️ <code>{{.UploadSpeed}}</code>"
|
||||
batch_item_downloading: "<blockquote><b>⬇️ {{.Index}}/{{.Total}} Downloading</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\nSpeed: <code>{{.Speed}}</code>\nSize: <code>{{.Current}}</code> / <code>{{.Size}}</code></blockquote>"
|
||||
batch_item_downloading_unknown: "<blockquote><b>⬇️ {{.Index}}/{{.Total}} Downloading</b>\n<code>{{.Name}}</code>\nSpeed: <code>{{.Speed}}</code>\nSize: <code>{{.Current}}</code> / unknown</blockquote>"
|
||||
batch_item_transferring: "<blockquote><b>↕️ {{.Index}}/{{.Total}} Transferring</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\nSpeed: <code>{{.Speed}}</code>\nSize: <code>{{.Current}}</code> / <code>{{.Size}}</code></blockquote>"
|
||||
batch_item_transferring_unknown: "<blockquote><b>↕️ {{.Index}}/{{.Total}} Transferring</b>\n<code>{{.Name}}</code>\nSpeed: <code>{{.Speed}}</code>\nSize: <code>{{.Current}}</code> / unknown</blockquote>"
|
||||
batch_item_uploading: "<blockquote><b>⬆️ {{.Index}}/{{.Total}} Uploading</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\nSpeed: <code>{{.Speed}}</code>\nSize: <code>{{.Current}}</code> / <code>{{.Size}}</code></blockquote>"
|
||||
batch_item_retrying: "<blockquote><b>🔁 {{.Index}}/{{.Total}} Retrying upload</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\nRetry: <code>{{.Attempt}}/{{.Limit}}</code>\nSpeed before failure: <code>{{.Speed}}</code>\nSize: <code>{{.Current}}</code> / <code>{{.Size}}</code>\nReason: <code>{{.Reason}}</code></blockquote>"
|
||||
batch_item_confirming: "<blockquote><b>⏳ {{.Index}}/{{.Total}} Waiting</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\n<i>Uploaded, awaiting remote confirmation</i></blockquote>"
|
||||
batch_summary_hidden_active: "🔄 <code>{{.Count}}</code> more files are active"
|
||||
batch_summary_confirming: "☁️ Uploaded, awaiting group send: <code>{{.Count}}</code>"
|
||||
batch_summary_failed: "❌ Failed: <code>{{.Count}}</code>"
|
||||
batch_summary_skipped: "⏭️ Skipped: <code>{{.Count}}</code>"
|
||||
batch_done: "<b>✅ Completed</b>\n\nFiles: <code>{{.Count}}</code>\nTotal size: <code>{{.Size}}</code>"
|
||||
batch_done_with_skipped: "<b>⚠️ Completed</b>\n\nSucceeded: <code>{{.Success}}</code>\nSkipped: <code>{{.Skipped}}</code>\nTotal size: <code>{{.Size}}</code>"
|
||||
batch_canceled: "<b>🚫 Task canceled</b>\n\nFiles: <code>{{.Total}}</code>\nCompleted: <code>{{.Completed}}</code>\nIncomplete: <code>{{.Incomplete}}</code>\nSkipped: <code>{{.Skipped}}</code>"
|
||||
batch_failed_item: "<b>❌ Processing failed</b>\n\nFailed file: <code>{{.Index}}. {{.Name}}</code>\nStage: <code>{{.Stage}}</code>\nProgress: <code>{{.Progress}}</code>\nSpeed before failure: <code>{{.Speed}}</code>\nReason: <code>{{.Reason}}</code>\n\n✅ Completed: <code>{{.Completed}}</code>\n❌ Failed: <code>{{.Failed}}</code>\n⏹️ Incomplete: <code>{{.Incomplete}}</code>"
|
||||
batch_failed_group: "<b>❌ Batch upload failed</b>\n\nAffected files: <code>{{.Affected}}</code>\nReason: <code>{{.Reason}}</code>\n\n✅ Completed: <code>{{.Completed}}</code>\n❌ Batch failed: <code>{{.Failed}}</code>\n⏹️ Incomplete: <code>{{.Incomplete}}</code>"
|
||||
batch_failed_task: "<b>❌ Processing failed</b>\n\nReason: <code>{{.Reason}}</code>\n\n✅ Completed: <code>{{.Completed}}</code>\n⏹️ Incomplete: <code>{{.Incomplete}}</code>"
|
||||
batch_failure_stage_download: "download"
|
||||
batch_failure_stage_cache: "local cache"
|
||||
batch_failure_stage_upload: "upload"
|
||||
batch_failure_stage_confirm: "remote confirmation"
|
||||
batch_failure_stage_batch_upload: "batch upload"
|
||||
batch_failure_stage_internal: "internal task"
|
||||
single_status_header: "<b>📦 Processing</b>"
|
||||
single_downloading: "<blockquote><b>⬇️ Downloading</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\nSpeed: <code>{{.Speed}}</code>\nSize: <code>{{.Current}}</code> / <code>{{.Size}}</code>\nSave to: <code>{{.Destination}}</code></blockquote>"
|
||||
single_downloading_unknown: "<blockquote><b>⬇️ Downloading</b>\n<code>{{.Name}}</code>\nSpeed: <code>{{.Speed}}</code>\nSize: <code>{{.Current}}</code> / unknown\nSave to: <code>{{.Destination}}</code></blockquote>"
|
||||
single_uploading: "<blockquote><b>⬆️ Uploading</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\nSpeed: <code>{{.Speed}}</code>\nSize: <code>{{.Current}}</code> / <code>{{.Size}}</code>\nSave to: <code>{{.Destination}}</code></blockquote>"
|
||||
single_upload_retrying: "<blockquote><b>🔁 Retrying upload</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\nAttempt: <code>{{.Attempt}}</code>\nSpeed: <code>{{.Speed}}</code>\nSize: <code>{{.Current}}</code> / <code>{{.Size}}</code>\nSave to: <code>{{.Destination}}</code></blockquote>"
|
||||
single_done: "<b>✅ Completed</b>\n\nFilename: <code>{{.Name}}</code>\nTotal size: <code>{{.Size}}</code>\nSave to: <code>{{.Destination}}</code>"
|
||||
single_canceled: "<b>🚫 Task canceled</b>\n\nFilename: <code>{{.Name}}</code>"
|
||||
single_failed: "<b>❌ Processing failed</b>\n\nFilename: <code>{{.Name}}</code>\nReason: <code>{{.Reason}}</code>"
|
||||
downloading_prefix: "Downloading\nTotal size: "
|
||||
size_with_files: "{{.Size}} ({{.Count}} files)"
|
||||
size_with_resources: "{{.Size}} ({{.Count}} resources)"
|
||||
processing_list_prefix: "\nProcessing:\n"
|
||||
processing_none: " - None"
|
||||
avg_speed_prefix: "\nAverage speed: "
|
||||
current_progress_prefix: "\nCurrent progress: "
|
||||
task_canceled: "Task canceled"
|
||||
task_canceled_with_id: "Processing canceled: {{.TaskID}}"
|
||||
task_failed_with_error: "Processing failed: {{.Error}}"
|
||||
batch_done_prefix: "Completed\nFile count: "
|
||||
direct_done_prefix: "Completed, file count: "
|
||||
parsed_start_prefix: "Starting download from {{.Site}}\nTotal size: "
|
||||
parsed_done_prefix: "Completed, resource count: "
|
||||
telegraph_start_prefix: "Starting Telegraph download\nImage count: "
|
||||
telegraph_progress_prefix: "Downloading\nCurrent progress: "
|
||||
telegraph_done_prefix: "Completed\nImage count: "
|
||||
file_start_prefix: "Starting download\nFilename: "
|
||||
file_processing_prefix: "Processing download task\nFilename: "
|
||||
download_failed_prefix: "Download failed\nFilename: "
|
||||
download_done_prefix: "Download completed\nFilename: "
|
||||
file_size_prefix: "\nFile size: "
|
||||
save_path_prefix: "\nSave path: "
|
||||
total_size_prefix: "\nTotal size: "
|
||||
direct_start: "Starting download, total size: {{.SizeMB}} MB ({{.Count}} files)"
|
||||
file_name_prefix: "Filename: "
|
||||
error_prefix: "\nError: "
|
||||
aria2_start: "Waiting for Aria2 to complete download (GID: {{.GID}})..."
|
||||
aria2_downloading: "Aria2 downloading (GID: {{.GID}})\n"
|
||||
aria2_done: "Aria2 download completed and transferred (GID: {{.GID}})\n"
|
||||
ytdlp_start: "Starting yt-dlp download ({{.Count}} links)..."
|
||||
ytdlp_downloading: "yt-dlp downloading ({{.Count}} links)\n"
|
||||
ytdlp_done: "yt-dlp download completed and transferred ({{.Count}} files)\n"
|
||||
downloaded_prefix: "\nDownloaded: "
|
||||
current_speed_prefix: "\nCurrent speed: "
|
||||
transfer_start_prefix: "Transfering: "
|
||||
transfer_progress_prefix: "Transfer progress: "
|
||||
transfer_uploaded_prefix: "\nUploaded: "
|
||||
transfer_speed_prefix: "\nSpeed: "
|
||||
transfer_remaining_time_prefix: "\nRemaining time: "
|
||||
transfer_processing_prefix: "\nProcessing:\n"
|
||||
transfer_processing_more: "...and {{.Count}} more files\n"
|
||||
transfer_failed_prefix: "Transfer failed\n"
|
||||
transfer_success_prefix: "Transfer completed\n"
|
||||
transfer_total_files_prefix: "\nTotal files: "
|
||||
transfer_total_size_prefix: "\nTotal size: "
|
||||
transfer_elapsed_time_prefix: "\nElapsed time: "
|
||||
transfer_avg_speed_prefix: "\nAverage speed: "
|
||||
transfer_failed_files_prefix: "\nFailed files: "
|
||||
syncpeers:
|
||||
start: "Starting to sync peers..."
|
||||
success: "Peer sync completed, total {{.Count}} chats synced"
|
||||
failed: "Peer sync failed: {{.Error}}"
|
||||
aria2:
|
||||
error_aria2_not_enabled: "Aria2 feature is not enabled in the configuration"
|
||||
error_aria2_client_init_failed: "Aria2 client initialization failed: {{.Error}}"
|
||||
info_adding_aria2_download: "Adding Aria2 download task..."
|
||||
error_adding_aria2_download: "Failed to add Aria2 download task: {{.Error}}"
|
||||
info_aria2_download_added: "Aria2 download task added, GID: {{.GID}}"
|
||||
info_select_storage: "Please select storage, the task will be added to Aria2 download queue after selection"
|
||||
|
||||
@@ -29,6 +29,8 @@ bot:
|
||||
/silent - 开关静默模式
|
||||
/storage - 设置默认存储位置
|
||||
/save [自定义文件名] - 保存文件
|
||||
/dl <链接1> <链接2> ... - 下载给定链接的文件
|
||||
/import <存储名> <目录路径> [频道ID] [过滤器] - 从存储端导入文件到 Telegram
|
||||
/dir - 管理存储目录
|
||||
/rule - 管理规则
|
||||
/config - 修改配置
|
||||
@@ -38,6 +40,7 @@ bot:
|
||||
/watch - 监听聊天并自动保存 (UserBot)
|
||||
/unwatch - 取消监听聊天 (UserBot)
|
||||
/lswatch - 列出正在监听的聊天 (UserBot)
|
||||
/syncpeers - 同步对话列表 (UserBot)
|
||||
/update - 检查更新并升级
|
||||
|
||||
使用帮助: https://sabot.unv.app/usage
|
||||
@@ -49,11 +52,16 @@ bot:
|
||||
rule: "管理自动存储规则"
|
||||
save: "保存文件"
|
||||
dl: "下载给定链接的文件"
|
||||
aria2dl: "使用 Aria2 下载给定链接的文件"
|
||||
ytdlp: "使用 yt-dlp 下载视频/音频"
|
||||
import: "从存储端导入文件到 Telegram"
|
||||
transfer: "在存储端之间传输文件"
|
||||
task: "管理任务队列"
|
||||
cancel: "取消任务"
|
||||
watch: "监听聊天(UserBot)"
|
||||
unwatch: "取消监听聊天(UserBot)"
|
||||
lswatch: "列出监听的聊天(UserBot)"
|
||||
syncpeers: "同步对话列表(UserBot)"
|
||||
config: "修改配置"
|
||||
fnametmpl: "设置文件命名模板"
|
||||
help: "显示帮助"
|
||||
@@ -105,9 +113,17 @@ bot:
|
||||
error_task_add_failed: "任务添加失败: {{.Error}}"
|
||||
info_task_added: "任务已添加"
|
||||
info_batch_tasks_added: "已添加批量任务, 共 {{.Count}} 个文件"
|
||||
info_batch_tasks_added_with_skipped: "已添加批量任务, 共 {{.Count}} 个文件\n已跳过同名文件:\n{{.Skipped}}"
|
||||
info_all_conflict_files_skipped: "全部同名文件已跳过:\n{{.Skipped}}"
|
||||
info_conflict_files_skipped: "已跳过同名文件:\n{{.Skipped}}"
|
||||
error_task_create_failed: "任务创建失败: {{.Error}}"
|
||||
error_get_dir_failed: "获取目录失败: {{.Error}}"
|
||||
prompt_select_dir: "请选择要存储到的目录"
|
||||
prompt_select_conflict_strategy: "检测到同名文件, 请选择保存策略:\n{{.Files}}"
|
||||
prompt_conflict_more_files: "...还有 {{.Count}} 个文件"
|
||||
button_conflict_rename: "重命名"
|
||||
button_conflict_overwrite: "覆盖"
|
||||
button_conflict_skip: "跳过"
|
||||
prompt_select_default_dir: "请选择要保存到的默认文件夹"
|
||||
info_default_storage_set: "已将默认存储位置设为: {{.Name}}"
|
||||
info_default_storage_with_dir_set: "已将默认存储位置设为: {{.Name}}:/{{.Dir}}"
|
||||
@@ -181,7 +197,11 @@ bot:
|
||||
help_switch_suffix: " - 开关规则模式\n"
|
||||
help_add_suffix: " <类型> <数据> <存储名> <路径> - 添加规则\n"
|
||||
help_del_suffix: " <规则ID> - 删除规则\n"
|
||||
help_preset_suffix: " <存储名> [基础路径] - 导入内置文件类型分类规则(视频/图片/音频/文档/压缩包)\n"
|
||||
help_existing_rules_prefix: "\n当前已添加的规则:\n"
|
||||
prompt_provide_storage_name: "请提供存储名称"
|
||||
error_storage_not_found: "未找到存储: {{.Storage}}"
|
||||
info_preset_imported: "已导入 {{.Count}} 条内置分类规则到存储 {{.Storage}}"
|
||||
dir:
|
||||
error_get_user_dirs_failed: "获取用户文件夹失败"
|
||||
error_get_user_failed: "获取用户失败"
|
||||
@@ -218,9 +238,9 @@ bot:
|
||||
info_install_plugin_success: "插件安装成功: {{.Name}}"
|
||||
parse:
|
||||
info_parsing: "正在解析..."
|
||||
error_parse_text_failed: "Failed to parse text: {{.Error}}"
|
||||
error_build_storage_select_keyboard_failed: "Failed to build storage selection keyboard: {{.Error}}"
|
||||
error_build_parsed_text_entity_failed: "Failed to build parsed text entity: {{.Error}}"
|
||||
error_parse_text_failed: "解析文本失败: {{.Error}}"
|
||||
error_build_storage_select_keyboard_failed: "构建存储选择键盘失败: {{.Error}}"
|
||||
error_build_parsed_text_entity_failed: "构建解析文本实体失败: {{.Error}}"
|
||||
info_link_prefix: "\n链接: "
|
||||
info_author_prefix: "\n作者: "
|
||||
info_description_prefix: "\n描述: "
|
||||
@@ -259,10 +279,17 @@ bot:
|
||||
config:
|
||||
prompt_select_option: "请选择要配置的选项"
|
||||
button_filename_strategy: "文件名策略"
|
||||
button_conflict_strategy: "重名文件保存策略"
|
||||
error_invalid_callback_data: "无效的回调数据"
|
||||
error_invalid_template: "无效的模板, 请检查语法\n{{.Error}}"
|
||||
info_filename_strategy_set: "已将文件名策略设置为: {{.Strategy}}"
|
||||
info_conflict_strategy_set: "已将重名文件保存策略设置为: {{.Strategy}}"
|
||||
prompt_select_filename_strategy: "请选择文件名策略, 当前策略: {{.Strategy}}"
|
||||
prompt_select_conflict_strategy: "请选择重名文件保存策略, 当前策略: {{.Strategy}}"
|
||||
conflict_strategy_rename: "始终重命名"
|
||||
conflict_strategy_ask: "每次询问"
|
||||
conflict_strategy_overwrite: "始终覆盖"
|
||||
conflict_strategy_skip: "始终跳过"
|
||||
fnametmpl_help: |-
|
||||
使用该命令设置文件名模板, 示例:
|
||||
/fnametmpl 图片_{{"{{.msgid}}"}}_{{"{{.msgdate}}"}}.jpg
|
||||
@@ -272,6 +299,7 @@ bot:
|
||||
- {{"{{.msgtags}}"}}: 消息中的标签, 将以下划线分隔输出
|
||||
- {{"{{.msggen}}"}}: 根据消息生成的文件名
|
||||
- {{"{{.msgdate}}"}}: 消息日期, 格式 YYYY-MM-DD_HH-MM-SS
|
||||
- {{"{{.msgraw}}"}}: 消息的原始文本内容 (不经任何处理)
|
||||
- {{"{{.origname}}"}}: 媒体的原始文件名 (如果有)
|
||||
- {{"{{.chatid}}"}}: 消息的聊天ID
|
||||
|
||||
@@ -283,6 +311,34 @@ bot:
|
||||
usage: "用法: /dl <链接1> <链接2> ..."
|
||||
error_no_valid_links: "没有有效的链接可供下载"
|
||||
info_files_select_storage: "共 {{.Count}} 个文件, 请选择存储位置"
|
||||
ytdlp:
|
||||
usage: "用法: /ytdlp [选项] <URL1> [URL2] ...\n示例:\n /ytdlp https://example.com/video\n /ytdlp --format best https://example.com/video\n /ytdlp --extract-audio --audio-format mp3 https://example.com/video"
|
||||
error_no_valid_urls: "没有有效的 URL"
|
||||
info_urls_select_storage: "共 {{.Count}} 个链接, 请选择存储位置"
|
||||
info_downloading: "正在通过 yt-dlp 下载..."
|
||||
error_download_failed: "yt-dlp 下载失败: {{.Error}}"
|
||||
transfer:
|
||||
usage: |
|
||||
用法: /transfer <source_storage>:/<source_path> [filter]
|
||||
示例:
|
||||
/transfer local1:/downloads
|
||||
/transfer alist1:/media/photos
|
||||
/transfer webdav1:/files ".*\.mp4$"
|
||||
error_invalid_source: "源路径格式无效,应为: storage_name:/path"
|
||||
error_invalid_target: "目标路径格式无效,应为: storage_name:/path"
|
||||
error_storage_not_found: "存储端 '{{.StorageName}}' 不存在或您无权访问: {{.Error}}"
|
||||
error_storage_not_listable: "存储端 '{{.StorageName}}' 不支持列举文件功能"
|
||||
error_storage_not_readable: "存储端 '{{.StorageName}}' 不支持读取文件功能"
|
||||
error_target_not_found: "目标存储端 '{{.StorageName}}' 不存在或您无权访问: {{.Error}}"
|
||||
info_fetching_files: "正在获取文件列表..."
|
||||
error_list_files_failed: "获取文件列表失败: {{.Error}}"
|
||||
error_invalid_regex: "正则表达式无效: {{.Error}}"
|
||||
error_no_files_to_transfer: "目录中没有可传输的文件"
|
||||
error_add_task_failed: "添加任务失败: {{.Error}}"
|
||||
info_task_added: "已添加 {{.Count}} 个文件到传输队列\n总大小: {{.SizeMB}} MB\n任务 ID: {{.TaskID}}"
|
||||
start_stats: "总文件数: {{.Count}}\n总大小: {{.SizeMB}} MB"
|
||||
info_files_select_storage: "共 {{.Count}} 个文件 (总大小: {{.SizeMB}} MB),请选择目标存储位置"
|
||||
error_build_storage_select_keyboard_failed: "构建存储选择键盘失败: {{.Error}}"
|
||||
cancel:
|
||||
usage: "用法: /cancel <task_id>"
|
||||
error_cancel_failed: "取消任务失败: {{.Error}}"
|
||||
@@ -296,30 +352,87 @@ bot:
|
||||
info_filename_prefix: "文件名: "
|
||||
info_prompt_select_storage: "\n请选择存储位置"
|
||||
progress:
|
||||
batch_start_prefix: "开始执行批量下载任务\n总大小: "
|
||||
batch_processing_prefix: "正在处理批量下载任务\n总大小: "
|
||||
batch_status_header: "<b>📦 正在处理</b>\n\n文件:<code>{{.Total}}</code> | 总大小:<code>{{.TotalSize}}</code>\n状态:✅ <code>{{.Completed}}</code> | 📥 <code>{{.Downloaded}}</code> | ⏳ <code>{{.Waiting}}</code>\n总速度:⬇️ <code>{{.DownloadSpeed}}</code> | ⬆️ <code>{{.UploadSpeed}}</code>"
|
||||
batch_item_downloading: "<blockquote><b>⬇️ {{.Index}}/{{.Total}} 下载中</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\n速度:<code>{{.Speed}}</code>\n大小:<code>{{.Current}}</code> / <code>{{.Size}}</code></blockquote>"
|
||||
batch_item_downloading_unknown: "<blockquote><b>⬇️ {{.Index}}/{{.Total}} 下载中</b>\n<code>{{.Name}}</code>\n速度:<code>{{.Speed}}</code>\n大小:<code>{{.Current}}</code> / 未知</blockquote>"
|
||||
batch_item_transferring: "<blockquote><b>↕️ {{.Index}}/{{.Total}} 传输中</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\n速度:<code>{{.Speed}}</code>\n大小:<code>{{.Current}}</code> / <code>{{.Size}}</code></blockquote>"
|
||||
batch_item_transferring_unknown: "<blockquote><b>↕️ {{.Index}}/{{.Total}} 传输中</b>\n<code>{{.Name}}</code>\n速度:<code>{{.Speed}}</code>\n大小:<code>{{.Current}}</code> / 未知</blockquote>"
|
||||
batch_item_uploading: "<blockquote><b>⬆️ {{.Index}}/{{.Total}} 上传中</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\n速度:<code>{{.Speed}}</code>\n大小:<code>{{.Current}}</code> / <code>{{.Size}}</code></blockquote>"
|
||||
batch_item_retrying: "<blockquote><b>🔁 {{.Index}}/{{.Total}} 上传重试</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\n重试次数:<code>{{.Attempt}}/{{.Limit}}</code>\n失败前速度:<code>{{.Speed}}</code>\n大小:<code>{{.Current}}</code> / <code>{{.Size}}</code>\n原因:<code>{{.Reason}}</code></blockquote>"
|
||||
batch_item_confirming: "<blockquote><b>⏳ {{.Index}}/{{.Total}} 等待中</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\n<i>文件已上传,正在等待远端确认</i></blockquote>"
|
||||
batch_summary_hidden_active: "🔄 另有 <code>{{.Count}}</code> 个文件正在处理"
|
||||
batch_summary_confirming: "☁️ 已上传,等待整组发送:<code>{{.Count}}</code>"
|
||||
batch_summary_failed: "❌ 失败:<code>{{.Count}}</code>"
|
||||
batch_summary_skipped: "⏭️ 已跳过:<code>{{.Count}}</code>"
|
||||
batch_done: "<b>✅ 处理完成</b>\n\n文件数: <code>{{.Count}}</code>\n总大小: <code>{{.Size}}</code>"
|
||||
batch_done_with_skipped: "<b>⚠️ 处理完成</b>\n\n成功: <code>{{.Success}}</code>\n已跳过: <code>{{.Skipped}}</code>\n总大小: <code>{{.Size}}</code>"
|
||||
batch_canceled: "<b>🚫 任务已取消</b>\n\n文件数: <code>{{.Total}}</code>\n已完成: <code>{{.Completed}}</code>\n未完成: <code>{{.Incomplete}}</code>\n已跳过: <code>{{.Skipped}}</code>"
|
||||
batch_failed_item: "<b>❌ 处理失败</b>\n\n失败文件: <code>{{.Index}}. {{.Name}}</code>\n失败阶段: <code>{{.Stage}}</code>\n失败进度: <code>{{.Progress}}</code>\n失败前速度: <code>{{.Speed}}</code>\n原因: <code>{{.Reason}}</code>\n\n✅ 已完成: <code>{{.Completed}}</code>\n❌ 失败: <code>{{.Failed}}</code>\n⏹️ 未完成: <code>{{.Incomplete}}</code>"
|
||||
batch_failed_group: "<b>❌ 批量上传失败</b>\n\n受影响文件: <code>{{.Affected}} 个</code>\n原因: <code>{{.Reason}}</code>\n\n✅ 已完成: <code>{{.Completed}}</code>\n❌ 批次失败: <code>{{.Failed}}</code>\n⏹️ 未完成: <code>{{.Incomplete}}</code>"
|
||||
batch_failed_task: "<b>❌ 处理失败</b>\n\n原因: <code>{{.Reason}}</code>\n\n✅ 已完成: <code>{{.Completed}}</code>\n⏹️ 未完成: <code>{{.Incomplete}}</code>"
|
||||
batch_failure_stage_download: "下载"
|
||||
batch_failure_stage_cache: "本地缓存"
|
||||
batch_failure_stage_upload: "上传"
|
||||
batch_failure_stage_confirm: "云端确认"
|
||||
batch_failure_stage_batch_upload: "批量上传"
|
||||
batch_failure_stage_internal: "任务内部"
|
||||
single_status_header: "<b>📦 正在处理</b>"
|
||||
single_downloading: "<blockquote><b>⬇️ 下载中</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\n速度:<code>{{.Speed}}</code>\n大小:<code>{{.Current}}</code> / <code>{{.Size}}</code>\n保存至:<code>{{.Destination}}</code></blockquote>"
|
||||
single_downloading_unknown: "<blockquote><b>⬇️ 下载中</b>\n<code>{{.Name}}</code>\n速度:<code>{{.Speed}}</code>\n大小:<code>{{.Current}}</code> / 未知\n保存至:<code>{{.Destination}}</code></blockquote>"
|
||||
single_uploading: "<blockquote><b>⬆️ 上传中</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\n速度:<code>{{.Speed}}</code>\n大小:<code>{{.Current}}</code> / <code>{{.Size}}</code>\n保存至:<code>{{.Destination}}</code></blockquote>"
|
||||
single_upload_retrying: "<blockquote><b>🔁 上传重试</b>\n<code>{{.Name}}</code>\n{{.Bar}} <code>{{.Progress}}%</code>\n尝试次数:<code>{{.Attempt}}</code>\n速度:<code>{{.Speed}}</code>\n大小:<code>{{.Current}}</code> / <code>{{.Size}}</code>\n保存至:<code>{{.Destination}}</code></blockquote>"
|
||||
single_done: "<b>✅ 处理完成</b>\n\n文件名:<code>{{.Name}}</code>\n总大小:<code>{{.Size}}</code>\n保存至:<code>{{.Destination}}</code>"
|
||||
single_canceled: "<b>🚫 任务已取消</b>\n\n文件名:<code>{{.Name}}</code>"
|
||||
single_failed: "<b>❌ 处理失败</b>\n\n文件名:<code>{{.Name}}</code>\n原因:<code>{{.Reason}}</code>"
|
||||
downloading_prefix: "正在下载\n总大小: "
|
||||
size_with_files: "{{.Size}} ({{.Count}} 个文件)"
|
||||
size_with_resources: "{{.Size}} ({{.Count}} 个资源)"
|
||||
processing_list_prefix: "\n正在处理:\n"
|
||||
processing_none: " - 无"
|
||||
avg_speed_prefix: "\n平均速度: "
|
||||
current_progress_prefix: "\n当前进度: "
|
||||
task_canceled: "任务已取消"
|
||||
task_canceled_with_id: "处理已取消: {{.TaskID}}"
|
||||
task_failed_with_error: "处理失败: {{.Error}}"
|
||||
batch_done_prefix: "处理完成\n文件数: "
|
||||
direct_done_prefix: "处理完成, 文件数量: "
|
||||
parsed_start_prefix: "开始下载 {{.Site}} 的资源\n总大小: "
|
||||
parsed_done_prefix: "处理完成, 资源数量: "
|
||||
telegraph_start_prefix: "开始下载Telegraph\n图片数量: "
|
||||
telegraph_progress_prefix: "正在下载\n当前进度: "
|
||||
telegraph_done_prefix: "处理完成\n图片数量: "
|
||||
file_start_prefix: "开始下载\n文件名: "
|
||||
file_processing_prefix: "正在处理下载任务\n文件名: "
|
||||
download_failed_prefix: "下载失败\n文件名: "
|
||||
download_done_prefix: "下载完成\n文件名: "
|
||||
file_size_prefix: "\n文件大小: "
|
||||
save_path_prefix: "\n保存路径: "
|
||||
total_size_prefix: "\n总大小: "
|
||||
direct_start: "开始下载, 总大小: {{.SizeMB}} MB ({{.Count}} 个文件)"
|
||||
file_name_prefix: "文件名: "
|
||||
error_prefix: "\n错误: "
|
||||
aria2_start: "等待 Aria2 下载完成 (GID: {{.GID}})..."
|
||||
aria2_downloading: "Aria2 正在下载 (GID: {{.GID}})\n"
|
||||
aria2_done: "Aria2 下载完成并已转存 (GID: {{.GID}})\n"
|
||||
ytdlp_start: "开始使用 yt-dlp 下载 ({{.Count}} 个链接)..."
|
||||
ytdlp_downloading: "yt-dlp 正在下载 ({{.Count}} 个链接)\n"
|
||||
ytdlp_done: "yt-dlp 下载完成并已转存 ({{.Count}} 个文件)\n"
|
||||
downloaded_prefix: "\n已下载: "
|
||||
current_speed_prefix: "\n当前速度: "
|
||||
transfer_start_prefix: "正在转存: "
|
||||
transfer_progress_prefix: "转存进度: "
|
||||
transfer_uploaded_prefix: "\n已上传: "
|
||||
transfer_speed_prefix: "\n速度: "
|
||||
transfer_remaining_time_prefix: "\n剩余时间: "
|
||||
transfer_processing_prefix: "\n正在处理:\n"
|
||||
transfer_processing_more: "...和其他 {{.Count}} 个文件\n"
|
||||
transfer_failed_prefix: "转存失败\n"
|
||||
transfer_success_prefix: "转存完成\n"
|
||||
transfer_total_files_prefix: "\n总文件数: "
|
||||
transfer_total_size_prefix: "\n总大小: "
|
||||
transfer_elapsed_time_prefix: "\n耗时: "
|
||||
transfer_avg_speed_prefix: "\n平均速度: "
|
||||
transfer_failed_files_prefix: "\n失败文件数: "
|
||||
syncpeers:
|
||||
start: "正在同步对话列表..."
|
||||
success: "对话列表同步完成, 共同步 {{.Count}} 个对话"
|
||||
failed: "对话列表同步失败: {{.Error}}"
|
||||
aria2:
|
||||
error_aria2_not_enabled: "Aria2 功能未启用, 请在配置文件中启用"
|
||||
error_aria2_client_init_failed: "Aria2 客户端初始化失败: {{.Error}}"
|
||||
info_adding_aria2_download: "正在添加 Aria2 下载任务..."
|
||||
error_adding_aria2_download: "添加 Aria2 下载任务失败: {{.Error}}"
|
||||
info_aria2_download_added: "Aria2 下载任务已添加, GID: {{.GID}}"
|
||||
info_select_storage: "请选择存储位置, 选择后将添加到 Aria2 下载队列"
|
||||
|
||||
@@ -1,7 +1,10 @@
|
||||
package tdler
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/gotd/td/telegram/downloader"
|
||||
"github.com/gotd/td/tg"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/dlutil"
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
"github.com/krau/SaveAny-Bot/pkg/consts/tglimit"
|
||||
@@ -10,5 +13,23 @@ import (
|
||||
|
||||
func NewDownloader(file tfile.TGFile) *downloader.Builder {
|
||||
return downloader.NewDownloader().WithPartSize(tglimit.MaxPartSize).
|
||||
Download(file.Dler(), file.Location()).WithThreads(dlutil.BestThreads(file.Size(), config.C().Threads))
|
||||
Download(eofAwareClient{Client: file.Dler(), size: file.Size()}, file.Location()).
|
||||
WithThreads(dlutil.BestThreads(file.Size(), config.C().Threads))
|
||||
}
|
||||
|
||||
// eofAwareClient answers upload.getFile requests at or past the end of the
|
||||
// file with an empty chunk. gotd's downloader is size-unaware: for files
|
||||
// whose size is an exact multiple of the part size it issues one final
|
||||
// request at offset == size and expects an empty chunk, but Telegram rejects
|
||||
// it with 400 OFFSET_INVALID and the whole download fails.
|
||||
type eofAwareClient struct {
|
||||
downloader.Client
|
||||
size int64
|
||||
}
|
||||
|
||||
func (c eofAwareClient) UploadGetFile(ctx context.Context, req *tg.UploadGetFileRequest) (tg.UploadFileClass, error) {
|
||||
if req.Offset >= c.size {
|
||||
return &tg.UploadFile{}, nil
|
||||
}
|
||||
return c.Client.UploadGetFile(ctx, req)
|
||||
}
|
||||
|
||||
112
common/tdler/dler_test.go
Normal file
112
common/tdler/dler_test.go
Normal file
@@ -0,0 +1,112 @@
|
||||
package tdler
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"github.com/gotd/td/tg"
|
||||
"github.com/gotd/td/tgerr"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/pkg/tfile"
|
||||
)
|
||||
|
||||
// serverLikeClient mimics real Telegram upload.getFile behavior: it returns
|
||||
// up to limit bytes per chunk, and answers any offset at or past the end of
|
||||
// the file with 400 OFFSET_INVALID.
|
||||
type serverLikeClient struct {
|
||||
data []byte
|
||||
|
||||
mu sync.Mutex
|
||||
maxOffset int64
|
||||
}
|
||||
|
||||
func (c *serverLikeClient) UploadGetFile(_ context.Context, req *tg.UploadGetFileRequest) (tg.UploadFileClass, error) {
|
||||
c.mu.Lock()
|
||||
if req.Offset > c.maxOffset {
|
||||
c.maxOffset = req.Offset
|
||||
}
|
||||
c.mu.Unlock()
|
||||
if req.Offset >= int64(len(c.data)) {
|
||||
return nil, tgerr.New(400, "OFFSET_INVALID")
|
||||
}
|
||||
end := min(len(c.data), int(req.Offset)+req.Limit)
|
||||
return &tg.UploadFile{Bytes: c.data[req.Offset:end]}, nil
|
||||
}
|
||||
|
||||
func (c *serverLikeClient) UploadGetFileHashes(context.Context, *tg.UploadGetFileHashesRequest) ([]tg.FileHash, error) {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func (c *serverLikeClient) UploadReuploadCDNFile(context.Context, *tg.UploadReuploadCDNFileRequest) ([]tg.FileHash, error) {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func (c *serverLikeClient) UploadGetCDNFileHashes(context.Context, *tg.UploadGetCDNFileHashesRequest) ([]tg.FileHash, error) {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func (c *serverLikeClient) UploadGetWebFile(context.Context, *tg.UploadGetWebFileRequest) (*tg.UploadWebFile, error) {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
type memWriterAt struct {
|
||||
b []byte
|
||||
}
|
||||
|
||||
func (w *memWriterAt) WriteAt(p []byte, off int64) (int, error) {
|
||||
copy(w.b[off:], p)
|
||||
return len(p), nil
|
||||
}
|
||||
|
||||
func TestDownloadServerLikeEOF(t *testing.T) {
|
||||
const partSize = 1024 * 1024
|
||||
tests := []struct {
|
||||
name string
|
||||
size int
|
||||
parallel bool
|
||||
}{
|
||||
{"stream exact multiple of part size", 2 * partSize, false},
|
||||
{"stream non-multiple", 2*partSize + 12345, false},
|
||||
{"stream smaller than part size", 1234, false},
|
||||
{"parallel exact multiple of part size", 2 * partSize, true},
|
||||
{"parallel non-multiple", 2*partSize + 12345, true},
|
||||
{"parallel smaller than part size", 1234, true},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
data := make([]byte, tt.size)
|
||||
for i := range data {
|
||||
data[i] = byte(i % 251)
|
||||
}
|
||||
client := &serverLikeClient{data: data}
|
||||
file := tfile.NewTGFile(
|
||||
&tg.InputDocumentFileLocation{ID: 1, AccessHash: 2},
|
||||
client, int64(tt.size), "test.bin",
|
||||
)
|
||||
|
||||
dl := NewDownloader(file)
|
||||
var got []byte
|
||||
var err error
|
||||
if tt.parallel {
|
||||
buf := make([]byte, tt.size)
|
||||
_, err = dl.WithThreads(4).Parallel(context.Background(), &memWriterAt{b: buf})
|
||||
got = buf
|
||||
} else {
|
||||
var buf bytes.Buffer
|
||||
_, err = dl.Stream(context.Background(), &buf)
|
||||
got = buf.Bytes()
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatalf("download failed: %v", err)
|
||||
}
|
||||
if !bytes.Equal(got, data) {
|
||||
t.Fatalf("downloaded %d bytes, want %d matching bytes", len(got), len(data))
|
||||
}
|
||||
if client.maxOffset >= int64(tt.size) {
|
||||
t.Fatalf("requested offset %d at or past EOF (size %d)", client.maxOffset, tt.size)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
257
common/tdler/resume.go
Normal file
257
common/tdler/resume.go
Normal file
@@ -0,0 +1,257 @@
|
||||
package tdler
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net"
|
||||
"os"
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
"github.com/gotd/td/tg"
|
||||
"github.com/gotd/td/tgerr"
|
||||
"golang.org/x/sync/errgroup"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/pkg/consts/tglimit"
|
||||
"github.com/krau/SaveAny-Bot/pkg/tfile"
|
||||
)
|
||||
|
||||
const maxChunkRetries = 20
|
||||
|
||||
// resumeBitmap records which partSize-aligned blocks of a download have been
|
||||
// durably written, so an interrupted download can continue after a restart.
|
||||
// The bitmap file is rewritten atomically after every completed block.
|
||||
type resumeBitmap struct {
|
||||
PartSize int `json:"part_size"`
|
||||
Size int64 `json:"size"`
|
||||
Blocks []uint64 `json:"blocks"`
|
||||
|
||||
mu sync.Mutex
|
||||
}
|
||||
|
||||
func newResumeBitmap(size int64) *resumeBitmap {
|
||||
b := &resumeBitmap{PartSize: tglimit.MaxPartSize, Size: size}
|
||||
b.ensureBlocks()
|
||||
return b
|
||||
}
|
||||
|
||||
func (b *resumeBitmap) ensureBlocks() {
|
||||
if need := (b.blockCount() + 63) / 64; len(b.Blocks) < need {
|
||||
b.Blocks = make([]uint64, need)
|
||||
}
|
||||
}
|
||||
|
||||
func (b *resumeBitmap) blockCount() int {
|
||||
return int((b.Size + int64(b.PartSize) - 1) / int64(b.PartSize))
|
||||
}
|
||||
|
||||
func (b *resumeBitmap) isDone(block int) bool {
|
||||
return b.Blocks[block/64]&(1<<uint(block%64)) != 0
|
||||
}
|
||||
|
||||
func (b *resumeBitmap) markDone(block int) {
|
||||
b.Blocks[block/64] |= 1 << uint(block%64)
|
||||
}
|
||||
|
||||
func (b *resumeBitmap) complete() bool {
|
||||
for block := 0; block < b.blockCount(); block++ {
|
||||
if !b.isDone(block) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func (b *resumeBitmap) missingBlocks() []int {
|
||||
missing := make([]int, 0, b.blockCount())
|
||||
for block := 0; block < b.blockCount(); block++ {
|
||||
if !b.isDone(block) {
|
||||
missing = append(missing, block)
|
||||
}
|
||||
}
|
||||
return missing
|
||||
}
|
||||
|
||||
func loadResumeBitmap(path string) (*resumeBitmap, error) {
|
||||
data, err := os.ReadFile(path)
|
||||
if errors.Is(err, os.ErrNotExist) {
|
||||
return nil, nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read resume bitmap: %w", err)
|
||||
}
|
||||
var b resumeBitmap
|
||||
if err := json.Unmarshal(data, &b); err != nil {
|
||||
// 无法解析的位图 (外部损坏): 删除并视为不存在, 全量重下自愈。
|
||||
_ = os.Remove(path)
|
||||
return nil, nil
|
||||
}
|
||||
if b.Size <= 0 || b.PartSize <= 0 {
|
||||
// 无效位图 (损坏或旧格式), 视为不存在, 全量重下。
|
||||
_ = os.Remove(path)
|
||||
return nil, nil
|
||||
}
|
||||
b.ensureBlocks()
|
||||
return &b, nil
|
||||
}
|
||||
|
||||
func (b *resumeBitmap) save(path string) error {
|
||||
b.mu.Lock()
|
||||
defer b.mu.Unlock()
|
||||
return b.saveLocked(path)
|
||||
}
|
||||
|
||||
func (b *resumeBitmap) saveLocked(path string) error {
|
||||
data, err := json.Marshal(b)
|
||||
if err != nil {
|
||||
return fmt.Errorf("marshal resume bitmap: %w", err)
|
||||
}
|
||||
tmp := path + ".tmp"
|
||||
if err := os.WriteFile(tmp, data, 0o644); err != nil {
|
||||
return fmt.Errorf("write resume bitmap: %w", err)
|
||||
}
|
||||
return os.Rename(tmp, path)
|
||||
}
|
||||
|
||||
func (b *resumeBitmap) markAndSave(block int, path string) error {
|
||||
b.mu.Lock()
|
||||
defer b.mu.Unlock()
|
||||
b.markDone(block)
|
||||
return b.saveLocked(path)
|
||||
}
|
||||
|
||||
func isRetryableTimeout(ctx context.Context, err error) bool {
|
||||
if err == nil || ctx.Err() != nil {
|
||||
return false
|
||||
}
|
||||
if tgerr.Is(err, tg.ErrTimeout) || errors.Is(err, context.DeadlineExceeded) {
|
||||
return true
|
||||
}
|
||||
var netErr net.Error
|
||||
return errors.As(err, &netErr) && netErr.Timeout()
|
||||
}
|
||||
|
||||
// fetchChunk downloads one partSize-aligned chunk, retrying flood waits and
|
||||
// transient timeouts like gotd's downloader does.
|
||||
func fetchChunk(ctx context.Context, file tfile.TGFile, offset int64, limit int) ([]byte, error) {
|
||||
req := &tg.UploadGetFileRequest{
|
||||
Location: file.Location(),
|
||||
Offset: offset,
|
||||
Limit: limit,
|
||||
}
|
||||
timeoutRetries := 0
|
||||
for {
|
||||
res, err := file.Dler().UploadGetFile(ctx, req)
|
||||
if err == nil {
|
||||
switch r := res.(type) {
|
||||
case *tg.UploadFile:
|
||||
return r.Bytes, nil
|
||||
case *tg.UploadFileCDNRedirect:
|
||||
return nil, fmt.Errorf("CDN redirect is not supported (dc %d)", r.DCID)
|
||||
default:
|
||||
return nil, fmt.Errorf("unexpected upload.getFile response %T", res)
|
||||
}
|
||||
}
|
||||
if flood, ferr := tgerr.FloodWait(ctx, err); ferr != nil {
|
||||
if flood {
|
||||
// FloodWait already slept; retry.
|
||||
continue
|
||||
}
|
||||
if isRetryableTimeout(ctx, ferr) {
|
||||
timeoutRetries++
|
||||
if timeoutRetries >= maxChunkRetries {
|
||||
return nil, fmt.Errorf("get chunk at %d: retry limit reached: %w", offset, ferr)
|
||||
}
|
||||
continue
|
||||
}
|
||||
return nil, fmt.Errorf("get chunk at %d: %w", offset, ferr)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// DownloadResumable downloads file to w in partSize chunks, skipping blocks
|
||||
// already recorded as complete in bitmapPath and persisting every completed
|
||||
// block so an interrupted download can resume. A missing or incompatible
|
||||
// bitmap starts a full download. Requires a known, non-zero file size.
|
||||
func DownloadResumable(
|
||||
ctx context.Context,
|
||||
file tfile.TGFile,
|
||||
w io.WriterAt,
|
||||
threads int,
|
||||
bitmapPath string,
|
||||
) error {
|
||||
if file.Size() <= 0 {
|
||||
return fmt.Errorf("resumable download requires a known size")
|
||||
}
|
||||
bm, err := loadResumeBitmap(bitmapPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
// 位图描述的数据文件 (bitmapPath 去掉 .bitmap 后缀) 必须存在且非空:
|
||||
// 若缺失或为空, 已标记完成的块字节已丢失, 必须重置位图全量重下。
|
||||
if bm != nil {
|
||||
partPath := strings.TrimSuffix(bitmapPath, ".bitmap")
|
||||
if stat, err := os.Stat(partPath); err != nil || stat.Size() == 0 {
|
||||
if err := os.Remove(bitmapPath); err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||
return fmt.Errorf("reset stale resume bitmap: %w", err)
|
||||
}
|
||||
bm = nil
|
||||
}
|
||||
}
|
||||
if bm == nil || bm.PartSize != tglimit.MaxPartSize || bm.Size != file.Size() {
|
||||
bm = newResumeBitmap(file.Size())
|
||||
if err := bm.save(bitmapPath); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
missing := bm.missingBlocks()
|
||||
if len(missing) == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
eg, gctx := errgroup.WithContext(ctx)
|
||||
eg.SetLimit(threads)
|
||||
for _, block := range missing {
|
||||
block := block
|
||||
eg.Go(func() error {
|
||||
offset := int64(block) * int64(bm.PartSize)
|
||||
data, err := fetchChunk(gctx, file, offset, bm.PartSize)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if len(data) == 0 {
|
||||
return fmt.Errorf("file ended early at offset %d (expected size %d)", offset, bm.Size)
|
||||
}
|
||||
if _, err := w.WriteAt(data, offset); err != nil {
|
||||
return fmt.Errorf("write chunk at offset %d: %w", offset, err)
|
||||
}
|
||||
return bm.markAndSave(block, bitmapPath)
|
||||
})
|
||||
}
|
||||
if err := eg.Wait(); err != nil {
|
||||
return err
|
||||
}
|
||||
if !bm.complete() {
|
||||
return fmt.Errorf("download finished with missing blocks")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// RemoveResumeState deletes the bitmap file of a completed download.
|
||||
func RemoveResumeState(bitmapPath string) error {
|
||||
if err := os.Remove(bitmapPath); err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||
return fmt.Errorf("remove resume bitmap: %w", err)
|
||||
}
|
||||
if err := os.Remove(bitmapPath + ".tmp"); err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||
return fmt.Errorf("remove resume bitmap temp: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// ResumeStatePath returns the bitmap path for a download cache file.
|
||||
func ResumeStatePath(cachePath string) string {
|
||||
return cachePath + ".bitmap"
|
||||
}
|
||||
271
common/tdler/resume_test.go
Normal file
271
common/tdler/resume_test.go
Normal file
@@ -0,0 +1,271 @@
|
||||
package tdler
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/gotd/td/tg"
|
||||
"github.com/gotd/td/tgerr"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/pkg/tfile"
|
||||
)
|
||||
|
||||
// failAfterClient serves the first failAfter chunks, then returns err.
|
||||
type failAfterClient struct {
|
||||
*serverLikeClient
|
||||
failAfter int
|
||||
calls int
|
||||
err error
|
||||
}
|
||||
|
||||
func (c *failAfterClient) UploadGetFile(ctx context.Context, req *tg.UploadGetFileRequest) (tg.UploadFileClass, error) {
|
||||
c.calls++
|
||||
if c.calls > c.failAfter {
|
||||
return nil, c.err
|
||||
}
|
||||
return c.serverLikeClient.UploadGetFile(ctx, req)
|
||||
}
|
||||
|
||||
func TestDownloadResumableFull(t *testing.T) {
|
||||
data := make([]byte, 3*1024*1024+123)
|
||||
for i := range data {
|
||||
data[i] = byte(i % 251)
|
||||
}
|
||||
client := &serverLikeClient{data: data}
|
||||
file := tfile.NewTGFile(&tg.InputDocumentFileLocation{ID: 1, AccessHash: 2}, client, int64(len(data)), "test.bin")
|
||||
|
||||
dir := t.TempDir()
|
||||
bitmapPath := filepath.Join(dir, "test.bin.bitmap")
|
||||
w := &memWriterAt{b: make([]byte, len(data))}
|
||||
|
||||
if err := DownloadResumable(context.Background(), file, w, 4, bitmapPath); err != nil {
|
||||
t.Fatalf("download failed: %v", err)
|
||||
}
|
||||
if !bytesEqual(w.b, data) {
|
||||
t.Fatalf("downloaded data mismatch")
|
||||
}
|
||||
bm, err := loadResumeBitmap(bitmapPath)
|
||||
if err != nil {
|
||||
t.Fatalf("load bitmap: %v", err)
|
||||
}
|
||||
if bm == nil || !bm.complete() {
|
||||
t.Fatalf("bitmap not complete after full download")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDownloadResumableInterrupted(t *testing.T) {
|
||||
data := make([]byte, 5*1024*1024) // exactly 5 blocks
|
||||
for i := range data {
|
||||
data[i] = byte(i % 251)
|
||||
}
|
||||
dir := t.TempDir()
|
||||
bitmapPath := filepath.Join(dir, "test.bin.bitmap")
|
||||
w := &memWriterAt{b: make([]byte, len(data))}
|
||||
|
||||
// First run: 3 blocks complete, 4th request fails.
|
||||
flaky := &failAfterClient{
|
||||
serverLikeClient: &serverLikeClient{data: data},
|
||||
failAfter: 3,
|
||||
err: tgerr.New(500, "INTERNAL_SERVER_ERROR"),
|
||||
}
|
||||
file := tfile.NewTGFile(&tg.InputDocumentFileLocation{ID: 1, AccessHash: 2}, flaky, int64(len(data)), "test.bin")
|
||||
err := DownloadResumable(context.Background(), file, w, 1, bitmapPath)
|
||||
if err == nil {
|
||||
t.Fatalf("expected first run to fail")
|
||||
}
|
||||
bm, err := loadResumeBitmap(bitmapPath)
|
||||
if err != nil {
|
||||
t.Fatalf("load bitmap after interruption: %v", err)
|
||||
}
|
||||
if bm == nil {
|
||||
t.Fatalf("bitmap missing after interruption")
|
||||
}
|
||||
if got := bm.blockCount() - len(bm.missingBlocks()); got != 3 {
|
||||
t.Fatalf("expected 3 completed blocks, got %d", got)
|
||||
}
|
||||
|
||||
// Second run: only the missing blocks are requested.
|
||||
healthy := &serverLikeClient{data: data}
|
||||
file = tfile.NewTGFile(&tg.InputDocumentFileLocation{ID: 1, AccessHash: 2}, healthy, int64(len(data)), "test.bin")
|
||||
if err := DownloadResumable(context.Background(), file, w, 1, bitmapPath); err != nil {
|
||||
t.Fatalf("resume failed: %v", err)
|
||||
}
|
||||
if !bytesEqual(w.b, data) {
|
||||
t.Fatalf("resumed data mismatch")
|
||||
}
|
||||
if healthy.maxOffset >= int64(len(data)) {
|
||||
t.Fatalf("resume requested offset %d at or past EOF", healthy.maxOffset)
|
||||
}
|
||||
if bm, err = loadResumeBitmap(bitmapPath); err != nil || bm == nil || !bm.complete() {
|
||||
t.Fatalf("bitmap not complete after resume: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDownloadResumableBitmapResetOnSizeChange(t *testing.T) {
|
||||
data := make([]byte, 2*1024*1024)
|
||||
for i := range data {
|
||||
data[i] = byte(i % 251)
|
||||
}
|
||||
dir := t.TempDir()
|
||||
bitmapPath := filepath.Join(dir, "test.bin.bitmap")
|
||||
w := &memWriterAt{b: make([]byte, len(data))}
|
||||
|
||||
// Record a bitmap claiming the old, larger file is fully downloaded.
|
||||
stale := newResumeBitmap(int64(4 * 1024 * 1024))
|
||||
if err := stale.save(bitmapPath); err != nil {
|
||||
t.Fatalf("save stale bitmap: %v", err)
|
||||
}
|
||||
|
||||
client := &serverLikeClient{data: data}
|
||||
file := tfile.NewTGFile(&tg.InputDocumentFileLocation{ID: 1, AccessHash: 2}, client, int64(len(data)), "test.bin")
|
||||
if err := DownloadResumable(context.Background(), file, w, 1, bitmapPath); err != nil {
|
||||
t.Fatalf("download with stale bitmap failed: %v", err)
|
||||
}
|
||||
if !bytesEqual(w.b, data) {
|
||||
t.Fatalf("data mismatch with stale bitmap")
|
||||
}
|
||||
}
|
||||
|
||||
// TestDownloadResumablePartMissingOrTruncated resets the bitmap: skipped
|
||||
// blocks would otherwise be zero-filled (caller recreates the part file
|
||||
// without its bytes), or the download would wedge forever on a stale
|
||||
// complete bitmap.
|
||||
func TestDownloadResumablePartMissingOrTruncated(t *testing.T) {
|
||||
data := make([]byte, 5*1024*1024)
|
||||
for i := range data {
|
||||
data[i] = byte(i % 251)
|
||||
}
|
||||
dir := t.TempDir()
|
||||
partPath := filepath.Join(dir, "test.bin.part")
|
||||
bitmapPath := ResumeStatePath(partPath)
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
doneBlocks []int
|
||||
createPart bool
|
||||
truncate bool
|
||||
}{
|
||||
{"part missing, partial bitmap", []int{0, 1, 2}, false, false},
|
||||
{"part empty, partial bitmap", []int{0, 1, 2}, true, true},
|
||||
{"part missing, complete bitmap", []int{0, 1, 2, 3, 4}, false, false},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
os.Remove(partPath)
|
||||
os.Remove(bitmapPath)
|
||||
bm := newResumeBitmap(int64(len(data)))
|
||||
for _, block := range tt.doneBlocks {
|
||||
bm.markDone(block)
|
||||
}
|
||||
if err := bm.save(bitmapPath); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if tt.createPart {
|
||||
// Simulate the caller re-creating the part file (truncating).
|
||||
if err := os.WriteFile(partPath, nil, 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if tt.truncate {
|
||||
if err := os.WriteFile(partPath, make([]byte, 0), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
partFile, err := os.OpenFile(partPath, os.O_CREATE|os.O_RDWR, 0o644)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer partFile.Close()
|
||||
client := &serverLikeClient{data: data}
|
||||
file := tfile.NewTGFile(&tg.InputDocumentFileLocation{ID: 1, AccessHash: 2}, client, int64(len(data)), "test.bin")
|
||||
if err := DownloadResumable(context.Background(), file, partFile, 1, bitmapPath); err != nil {
|
||||
t.Fatalf("download failed: %v", err)
|
||||
}
|
||||
got := make([]byte, len(data))
|
||||
if _, err := partFile.ReadAt(got, 0); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !bytesEqual(got, data) {
|
||||
t.Fatalf("downloaded data mismatch (blocks not reset)")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestDownloadResumableInvalidBitmap treats a corrupt bitmap as absent.
|
||||
func TestDownloadResumableInvalidBitmap(t *testing.T) {
|
||||
data := make([]byte, 1024*1024+7)
|
||||
for i := range data {
|
||||
data[i] = byte(i % 251)
|
||||
}
|
||||
dir := t.TempDir()
|
||||
partPath := filepath.Join(dir, "test.bin.part")
|
||||
bitmapPath := ResumeStatePath(partPath)
|
||||
for _, content := range []string{
|
||||
`{"part_size":1048576,"size":-1,"blocks":[]}`,
|
||||
`{"part_size":1048576,"size":9223372036854775807,"blocks":[]}`,
|
||||
`not json`,
|
||||
} {
|
||||
os.Remove(partPath)
|
||||
if err := os.WriteFile(bitmapPath, []byte(content), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
partFile, err := os.OpenFile(partPath, os.O_CREATE|os.O_RDWR, 0o644)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
client := &serverLikeClient{data: data}
|
||||
file := tfile.NewTGFile(&tg.InputDocumentFileLocation{ID: 1, AccessHash: 2}, client, int64(len(data)), "test.bin")
|
||||
err = DownloadResumable(context.Background(), file, partFile, 1, bitmapPath)
|
||||
partFile.Close()
|
||||
if err != nil {
|
||||
t.Fatalf("download with corrupt bitmap %q failed: %v", content, err)
|
||||
}
|
||||
got := make([]byte, len(data))
|
||||
f, err := os.Open(partPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := f.ReadAt(got, 0); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
f.Close()
|
||||
if !bytesEqual(got, data) {
|
||||
t.Fatalf("downloaded data mismatch with corrupt bitmap %q", content)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRemoveResumeState(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "x.bitmap")
|
||||
if err := os.WriteFile(path, []byte("{}"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := RemoveResumeState(path); err != nil {
|
||||
t.Fatalf("RemoveResumeState: %v", err)
|
||||
}
|
||||
if _, err := os.Stat(path); !errors.Is(err, os.ErrNotExist) {
|
||||
t.Fatalf("bitmap still exists: %v", err)
|
||||
}
|
||||
// Removing again must be a no-op.
|
||||
if err := RemoveResumeState(path); err != nil {
|
||||
t.Fatalf("RemoveResumeState second call: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func bytesEqual(a, b []byte) bool {
|
||||
if len(a) != len(b) {
|
||||
return false
|
||||
}
|
||||
for i := range a {
|
||||
if a[i] != b[i] {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
@@ -1,6 +1,9 @@
|
||||
package dlutil
|
||||
|
||||
import "time"
|
||||
import (
|
||||
"fmt"
|
||||
"time"
|
||||
)
|
||||
|
||||
var threadsLevels = []struct {
|
||||
threads int
|
||||
@@ -31,3 +34,23 @@ func GetSpeed(downloaded int64, startTime time.Time) float64 {
|
||||
}
|
||||
return float64(downloaded) / elapsed
|
||||
}
|
||||
|
||||
// FormatSize formats a byte size as a human-readable string
|
||||
func FormatSize(bytes int64) string {
|
||||
const (
|
||||
KB = 1024
|
||||
MB = KB * 1024
|
||||
GB = MB * 1024
|
||||
)
|
||||
|
||||
switch {
|
||||
case bytes >= GB:
|
||||
return fmt.Sprintf("%.2f GB", float64(bytes)/float64(GB))
|
||||
case bytes >= MB:
|
||||
return fmt.Sprintf("%.2f MB", float64(bytes)/float64(MB))
|
||||
case bytes >= KB:
|
||||
return fmt.Sprintf("%.2f KB", float64(bytes)/float64(KB))
|
||||
default:
|
||||
return fmt.Sprintf("%d B", bytes)
|
||||
}
|
||||
}
|
||||
|
||||
42
common/utils/fsutil/file_test.go
Normal file
42
common/utils/fsutil/file_test.go
Normal file
@@ -0,0 +1,42 @@
|
||||
package fsutil_test
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/common/utils/fsutil"
|
||||
)
|
||||
|
||||
func TestCloseAndRemove(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
preClose bool
|
||||
}{
|
||||
{name: "open file"},
|
||||
{name: "already closed file", preClose: true},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
filePath := filepath.Join(t.TempDir(), "cache-file")
|
||||
file, err := fsutil.CreateFile(filePath)
|
||||
if err != nil {
|
||||
t.Fatalf("CreateFile() failed: %v", err)
|
||||
}
|
||||
if tt.preClose {
|
||||
if err := file.Close(); err != nil {
|
||||
t.Fatalf("Close() failed: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
if err := file.CloseAndRemove(); err != nil {
|
||||
t.Fatalf("CloseAndRemove() failed: %v", err)
|
||||
}
|
||||
if _, err := os.Stat(filePath); !errors.Is(err, os.ErrNotExist) {
|
||||
t.Fatalf("cache file still exists after CloseAndRemove(): %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -1,6 +1,7 @@
|
||||
package fsutil
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
@@ -41,10 +42,11 @@ func (f *File) Remove() error {
|
||||
}
|
||||
|
||||
func (f *File) CloseAndRemove() error {
|
||||
if err := f.Close(); err != nil {
|
||||
return err
|
||||
closeErr := f.Close()
|
||||
if errors.Is(closeErr, os.ErrClosed) {
|
||||
closeErr = nil
|
||||
}
|
||||
return f.Remove()
|
||||
return errors.Join(closeErr, f.Remove())
|
||||
}
|
||||
|
||||
func CreateFile(fp string) (*File, error) {
|
||||
|
||||
27
common/utils/fsutil/unique.go
Normal file
27
common/utils/fsutil/unique.go
Normal file
@@ -0,0 +1,27 @@
|
||||
package fsutil
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"path"
|
||||
"strings"
|
||||
|
||||
"github.com/rs/xid"
|
||||
)
|
||||
|
||||
// UniquePath returns a non-taken path under basePath: the name itself, then
|
||||
// numbered variants, then a random suffix.
|
||||
func UniquePath(basePath, name string, exists func(candidate string) bool, maxAttempts int) string {
|
||||
candidate := path.Join(basePath, name)
|
||||
if !exists(candidate) {
|
||||
return candidate
|
||||
}
|
||||
ext := path.Ext(name)
|
||||
stem := strings.TrimSuffix(name, ext)
|
||||
for i := 1; i <= maxAttempts; i++ {
|
||||
candidate = path.Join(basePath, fmt.Sprintf("%s_%d%s", stem, i, ext))
|
||||
if !exists(candidate) {
|
||||
return candidate
|
||||
}
|
||||
}
|
||||
return path.Join(basePath, fmt.Sprintf("%s_%s%s", stem, xid.New().String(), ext))
|
||||
}
|
||||
@@ -17,7 +17,11 @@ type ProgressReadSeeker struct {
|
||||
|
||||
// Seek implements io.ReadSeeker.
|
||||
func (pr *ProgressReadSeeker) Seek(offset int64, whence int) (int64, error) {
|
||||
return pr.reader.Seek(offset, whence)
|
||||
position, err := pr.reader.Seek(offset, whence)
|
||||
if err == nil {
|
||||
pr.read.Store(position)
|
||||
}
|
||||
return position, err
|
||||
}
|
||||
|
||||
// NewProgressReader creates a new ProgressReader
|
||||
@@ -54,7 +58,7 @@ func (pr *ProgressReadSeeker) Progress() float64 {
|
||||
return float64(pr.read.Load()) / float64(pr.total.Load())
|
||||
}
|
||||
|
||||
// Read returns the number of bytes read so far
|
||||
// BytesRead returns the current tracked reader position.
|
||||
func (pr *ProgressReadSeeker) BytesRead() int64 {
|
||||
return pr.read.Load()
|
||||
}
|
||||
|
||||
50
common/utils/ioutil/progress_reader_test.go
Normal file
50
common/utils/ioutil/progress_reader_test.go
Normal file
@@ -0,0 +1,50 @@
|
||||
package ioutil
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"io"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestProgressReadSeekerTracksReads(t *testing.T) {
|
||||
var gotRead, gotTotal int64
|
||||
reader := NewProgressReader(bytes.NewReader([]byte("abcdef")), 6, func(read, total int64) {
|
||||
gotRead = read
|
||||
gotTotal = total
|
||||
})
|
||||
|
||||
buffer := make([]byte, 4)
|
||||
if _, err := io.ReadFull(reader, buffer); err != nil {
|
||||
t.Fatalf("read failed: %v", err)
|
||||
}
|
||||
|
||||
if gotRead != 4 || gotTotal != 6 {
|
||||
t.Fatalf("progress = %d/%d, want 4/6", gotRead, gotTotal)
|
||||
}
|
||||
if reader.BytesRead() != 4 {
|
||||
t.Fatalf("BytesRead() = %d, want 4", reader.BytesRead())
|
||||
}
|
||||
}
|
||||
|
||||
func TestProgressReadSeekerResetsPositionOnSeek(t *testing.T) {
|
||||
reader := NewProgressReader(bytes.NewReader([]byte("abcdef")), 6, nil)
|
||||
buffer := make([]byte, 4)
|
||||
if _, err := io.ReadFull(reader, buffer); err != nil {
|
||||
t.Fatalf("read failed: %v", err)
|
||||
}
|
||||
|
||||
position, err := reader.Seek(0, io.SeekStart)
|
||||
if err != nil {
|
||||
t.Fatalf("seek failed: %v", err)
|
||||
}
|
||||
if position != 0 || reader.BytesRead() != 0 {
|
||||
t.Fatalf("position after seek = %d (tracked %d), want 0", position, reader.BytesRead())
|
||||
}
|
||||
|
||||
if _, err := io.ReadFull(reader, buffer[:2]); err != nil {
|
||||
t.Fatalf("read after seek failed: %v", err)
|
||||
}
|
||||
if reader.BytesRead() != 2 {
|
||||
t.Fatalf("BytesRead() after seek and read = %d, want 2", reader.BytesRead())
|
||||
}
|
||||
}
|
||||
@@ -48,4 +48,4 @@ func NewProgressWriter(
|
||||
wr: wr,
|
||||
onWrite: onWrite,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
50
common/utils/progressutil/progress.go
Normal file
50
common/utils/progressutil/progress.go
Normal file
@@ -0,0 +1,50 @@
|
||||
// Package progressutil provides shared progress-update throttling for task
|
||||
// progress trackers.
|
||||
package progressutil
|
||||
|
||||
// updatesLevels picks the percent step by file size.
|
||||
var updatesLevels = []struct {
|
||||
size int64 // file size threshold
|
||||
stepPercent int // minimum percent step between updates
|
||||
}{
|
||||
{10 << 20, 100},
|
||||
{50 << 20, 20},
|
||||
{200 << 20, 10},
|
||||
{500 << 20, 5},
|
||||
}
|
||||
|
||||
// ShouldUpdate reports whether a byte-based progress update should be shown.
|
||||
func ShouldUpdate(total, downloaded int64, lastUpdatePercent int) bool {
|
||||
if total <= 0 || downloaded <= 0 {
|
||||
return false
|
||||
}
|
||||
|
||||
percent := int((downloaded * 100) / total)
|
||||
if percent <= lastUpdatePercent {
|
||||
return false
|
||||
}
|
||||
|
||||
step := updatesLevels[len(updatesLevels)-1].stepPercent
|
||||
for _, lvl := range updatesLevels {
|
||||
if total < lvl.size {
|
||||
step = lvl.stepPercent
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
return percent >= lastUpdatePercent+step
|
||||
}
|
||||
|
||||
// ShouldUpdateCount reports whether a count-based progress update (e.g. files
|
||||
// downloaded so far) should be shown: every 10 units, or when finished.
|
||||
func ShouldUpdateCount(downloaded, total int64) bool {
|
||||
if total <= 0 || downloaded <= 0 {
|
||||
return false
|
||||
}
|
||||
|
||||
const step = int64(10)
|
||||
if downloaded < step {
|
||||
return downloaded == total
|
||||
}
|
||||
return downloaded%step == 0 || downloaded == total
|
||||
}
|
||||
35
common/utils/tgutil/html.go
Normal file
35
common/utils/tgutil/html.go
Normal file
@@ -0,0 +1,35 @@
|
||||
package tgutil
|
||||
|
||||
import (
|
||||
stdhtml "html"
|
||||
|
||||
"github.com/gotd/td/telegram/message/entity"
|
||||
messagehtml "github.com/gotd/td/telegram/message/html"
|
||||
"github.com/gotd/td/telegram/message/styling"
|
||||
"github.com/gotd/td/tg"
|
||||
)
|
||||
|
||||
// EscapeHTMLTemplateData returns a copy of data with string values escaped for
|
||||
// interpolation into Telegram HTML templates.
|
||||
func EscapeHTMLTemplateData(data map[string]any) map[string]any {
|
||||
escaped := make(map[string]any, len(data))
|
||||
for key, value := range data {
|
||||
if text, ok := value.(string); ok {
|
||||
escaped[key] = stdhtml.EscapeString(text)
|
||||
continue
|
||||
}
|
||||
escaped[key] = value
|
||||
}
|
||||
return escaped
|
||||
}
|
||||
|
||||
// RenderHTML renders Telegram-compatible HTML into plain text and message
|
||||
// entities.
|
||||
func RenderHTML(markup string) (string, []tg.MessageEntityClass, error) {
|
||||
var builder entity.Builder
|
||||
if err := styling.Perform(&builder, messagehtml.String(nil, markup)); err != nil {
|
||||
return "", nil, err
|
||||
}
|
||||
text, entities := builder.Complete()
|
||||
return text, entities, nil
|
||||
}
|
||||
54
common/utils/tgutil/html_test.go
Normal file
54
common/utils/tgutil/html_test.go
Normal file
@@ -0,0 +1,54 @@
|
||||
package tgutil
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/gotd/td/tg"
|
||||
)
|
||||
|
||||
func TestEscapeHTMLTemplateDataDoesNotMutateInput(t *testing.T) {
|
||||
input := map[string]any{
|
||||
"Text": `<b>A&B</b>`,
|
||||
"Count": 2,
|
||||
}
|
||||
escaped := EscapeHTMLTemplateData(input)
|
||||
|
||||
if got, want := escaped["Text"], "<b>A&B</b>"; got != want {
|
||||
t.Fatalf("escaped text = %q, want %q", got, want)
|
||||
}
|
||||
if got := input["Text"]; got != `<b>A&B</b>` {
|
||||
t.Fatalf("input was mutated: %q", got)
|
||||
}
|
||||
if got := escaped["Count"]; got != 2 {
|
||||
t.Fatalf("non-string value = %v, want 2", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRenderHTMLUsesTemplateStylesAndDecodesValues(t *testing.T) {
|
||||
data := EscapeHTMLTemplateData(map[string]any{"Name": `<b>A&B</b>.bin`})
|
||||
markup := `<blockquote><b>Uploading</b>
|
||||
<code>` + data["Name"].(string) + `</code></blockquote>`
|
||||
|
||||
text, entities, err := RenderHTML(markup)
|
||||
if err != nil {
|
||||
t.Fatalf("RenderHTML() failed: %v", err)
|
||||
}
|
||||
if want := "Uploading\n<b>A&B</b>.bin"; text != want {
|
||||
t.Fatalf("rendered text = %q, want %q", text, want)
|
||||
}
|
||||
|
||||
var bold, code, blockquote int
|
||||
for _, messageEntity := range entities {
|
||||
switch messageEntity.(type) {
|
||||
case *tg.MessageEntityBold:
|
||||
bold++
|
||||
case *tg.MessageEntityCode:
|
||||
code++
|
||||
case *tg.MessageEntityBlockquote:
|
||||
blockquote++
|
||||
}
|
||||
}
|
||||
if bold != 1 || code != 1 || blockquote != 1 {
|
||||
t.Fatalf("entity counts = bold:%d code:%d blockquote:%d", bold, code, blockquote)
|
||||
}
|
||||
}
|
||||
@@ -2,6 +2,7 @@ package tgutil
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"unicode"
|
||||
@@ -113,29 +114,28 @@ func InputMessageClassSliceFromInt(ids []int) []tg.InputMessageClass {
|
||||
return result
|
||||
}
|
||||
|
||||
func GetMessagesRange(ctx *ext.Context, chatID int64, minId, maxId int) ([]*tg.Message, error) {
|
||||
if msg, err := getMessagesRange(ctx, chatID, minId, maxId); err == nil {
|
||||
return msg, nil
|
||||
func GetMessagesRange(ctx *ext.Context, chatID int64, minId, maxId int) (msg []*tg.Message, err error) {
|
||||
if msg, err = getMessagesRange(ctx, chatID, minId, maxId); err == nil {
|
||||
return
|
||||
}
|
||||
in := constant.TDLibPeerID(chatID)
|
||||
plain := in.ToPlain()
|
||||
|
||||
var channel constant.TDLibPeerID
|
||||
channel.Channel(plain)
|
||||
if msg, err := getMessagesRange(ctx, int64(channel), minId, maxId); err == nil {
|
||||
return msg, nil
|
||||
if msg, err = getMessagesRange(ctx, int64(channel), minId, maxId); err == nil {
|
||||
return
|
||||
}
|
||||
var userID constant.TDLibPeerID
|
||||
userID.User(plain)
|
||||
if msg, err := getMessagesRange(ctx, int64(userID), minId, maxId); err == nil {
|
||||
return msg, nil
|
||||
if msg, err = getMessagesRange(ctx, int64(userID), minId, maxId); err == nil {
|
||||
return
|
||||
}
|
||||
var chat constant.TDLibPeerID
|
||||
chat.Chat(plain)
|
||||
if msg, err := getMessagesRange(ctx, int64(chat), minId, maxId); err == nil {
|
||||
return msg, nil
|
||||
if msg, err = getMessagesRange(ctx, int64(chat), minId, maxId); err == nil {
|
||||
return
|
||||
}
|
||||
return nil, fmt.Errorf("failed to get messages range for chatID %d", chatID)
|
||||
return nil, fmt.Errorf("failed to get messages range for chat %d: %w", chatID, err)
|
||||
}
|
||||
|
||||
func getMessagesRange(ctx *ext.Context, chatID int64, minId, maxId int) ([]*tg.Message, error) {
|
||||
@@ -194,97 +194,6 @@ func getMessagesRange(ctx *ext.Context, chatID int64, minId, maxId int) ([]*tg.M
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// [TODO]
|
||||
// type MessageItem struct {
|
||||
// Message *tg.Message
|
||||
// Error error
|
||||
// }
|
||||
|
||||
// func IterMessages(ctx *ext.Context, chatID int64, minId, maxId int) (<-chan MessageItem, error) {
|
||||
// total := maxId - minId + 1
|
||||
// ch := make(chan MessageItem, 100)
|
||||
|
||||
// go func() {
|
||||
// defer close(ch)
|
||||
// if !ctx.Self.Bot {
|
||||
// perr := ctx.PeerStorage.GetInputPeerById(chatID)
|
||||
// if perr == nil || perr.(*tg.InputPeerEmpty) != nil {
|
||||
// ch <- MessageItem{
|
||||
// Error: fmt.Errorf("peer not found: %d", chatID),
|
||||
// }
|
||||
// return
|
||||
// }
|
||||
|
||||
// for i := 0; i < total; i += 100 {
|
||||
// start := minId + i
|
||||
// end := min(start+100, maxId)
|
||||
// msgs, err := ctx.Raw.MessagesGetHistory(ctx, &tg.MessagesGetHistoryRequest{
|
||||
// Peer: perr,
|
||||
// OffsetID: start,
|
||||
// AddOffset: start - end,
|
||||
// Limit: 100,
|
||||
// })
|
||||
// if err != nil {
|
||||
// ch <- MessageItem{
|
||||
// Error: fmt.Errorf("failed to get messages: %w", err),
|
||||
// }
|
||||
// return
|
||||
// }
|
||||
// var msgClass []tg.MessageClass
|
||||
// switch msgsv := msgs.(type) {
|
||||
// case *tg.MessagesMessages:
|
||||
// msgClass = msgsv.GetMessages()
|
||||
// case *tg.MessagesMessagesSlice:
|
||||
// msgClass = msgsv.GetMessages()
|
||||
// case *tg.MessagesChannelMessages:
|
||||
// msgClass = msgsv.GetMessages()
|
||||
// default:
|
||||
// ch <- MessageItem{
|
||||
// Error: fmt.Errorf("unsupported message type: %T", msgsv),
|
||||
// }
|
||||
// continue
|
||||
// }
|
||||
// for _, msg := range msgClass {
|
||||
// msg, ok := msg.AsNotEmpty()
|
||||
// if !ok {
|
||||
// continue
|
||||
// }
|
||||
// switch msg := msg.(type) {
|
||||
// case *tg.Message:
|
||||
// key := fmt.Sprintf("tgmsg:%d:%d:%d", ctx.Self.ID, chatID, msg.GetID())
|
||||
// cache.Set(key, msg)
|
||||
// ch <- MessageItem{
|
||||
// Message: msg,
|
||||
// }
|
||||
// }
|
||||
// }
|
||||
// }
|
||||
// } else {
|
||||
// for i := 0; i < total; i += 100 {
|
||||
// start := minId + i
|
||||
// end := min(start+100, maxId)
|
||||
// msgs, err := GetMessagesRange(ctx, chatID, start, end)
|
||||
// if err != nil {
|
||||
// ch <- MessageItem{
|
||||
// Error: fmt.Errorf("failed to get messages: %w", err),
|
||||
// }
|
||||
// return
|
||||
// }
|
||||
// for _, msg := range msgs {
|
||||
// if msg == nil {
|
||||
// continue
|
||||
// }
|
||||
// ch <- MessageItem{
|
||||
// Message: msg,
|
||||
// }
|
||||
// }
|
||||
// }
|
||||
// }
|
||||
// }()
|
||||
|
||||
// return ch, nil
|
||||
// }
|
||||
|
||||
func getMessageByID(ctx *ext.Context, chatID int64, msgID int) (*tg.Message, error) {
|
||||
key := fmt.Sprintf("tgmsg:%d:%d:%d", ctx.Self.ID, chatID, msgID)
|
||||
if msg, ok := cache.Get[*tg.Message](key); ok {
|
||||
@@ -360,9 +269,16 @@ func GetGroupedMessages(ctx *ext.Context, chatID int64, msg *tg.Message) ([]*tg.
|
||||
groupedMessages = append(groupedMessages, m)
|
||||
}
|
||||
}
|
||||
sortMessagesByID(groupedMessages)
|
||||
return groupedMessages, nil
|
||||
}
|
||||
|
||||
func sortMessagesByID(messages []*tg.Message) {
|
||||
sort.Slice(messages, func(i, j int) bool {
|
||||
return messages[i].GetID() < messages[j].GetID()
|
||||
})
|
||||
}
|
||||
|
||||
func ExtractMessageEntityUrls(msg *tg.Message) []string {
|
||||
if len(msg.Entities) == 0 {
|
||||
return nil
|
||||
|
||||
18
common/utils/tgutil/message_test.go
Normal file
18
common/utils/tgutil/message_test.go
Normal file
@@ -0,0 +1,18 @@
|
||||
package tgutil
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/gotd/td/tg"
|
||||
)
|
||||
|
||||
func TestSortMessagesByID(t *testing.T) {
|
||||
messages := []*tg.Message{{ID: 9}, {ID: 3}, {ID: 7}}
|
||||
sortMessagesByID(messages)
|
||||
want := []int{3, 7, 9}
|
||||
for i := range messages {
|
||||
if messages[i].GetID() != want[i] {
|
||||
t.Fatalf("message %d has ID %d, want %d", i, messages[i].GetID(), want[i])
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,6 +1,12 @@
|
||||
package tgutil
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"encoding/base64"
|
||||
"fmt"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/url"
|
||||
|
||||
"github.com/gotd/td/telegram/dcs"
|
||||
@@ -8,24 +14,108 @@ import (
|
||||
"golang.org/x/net/proxy"
|
||||
)
|
||||
|
||||
func newProxyDialer(proxyUrl string) (proxy.Dialer, error) {
|
||||
url, err := url.Parse(proxyUrl)
|
||||
// httpProxyDialer implements proxy.ContextDialer for HTTP CONNECT proxies
|
||||
type httpProxyDialer struct {
|
||||
proxyURL *url.URL
|
||||
forward proxy.Dialer
|
||||
}
|
||||
|
||||
func (d *httpProxyDialer) Dial(network, addr string) (net.Conn, error) {
|
||||
return d.DialContext(context.Background(), network, addr)
|
||||
}
|
||||
|
||||
func (d *httpProxyDialer) DialContext(ctx context.Context, network, addr string) (net.Conn, error) {
|
||||
proxyAddr := d.proxyURL.Host
|
||||
if d.proxyURL.Port() == "" {
|
||||
if d.proxyURL.Scheme == "https" {
|
||||
proxyAddr = net.JoinHostPort(d.proxyURL.Hostname(), "443")
|
||||
} else {
|
||||
proxyAddr = net.JoinHostPort(d.proxyURL.Hostname(), "80")
|
||||
}
|
||||
}
|
||||
|
||||
var conn net.Conn
|
||||
var err error
|
||||
if ctxDialer, ok := d.forward.(proxy.ContextDialer); ok {
|
||||
conn, err = ctxDialer.DialContext(ctx, "tcp", proxyAddr)
|
||||
} else {
|
||||
conn, err = d.forward.Dial("tcp", proxyAddr)
|
||||
}
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to connect to proxy: %w", err)
|
||||
}
|
||||
|
||||
// Send CONNECT request
|
||||
connectReq := &http.Request{
|
||||
Method: "CONNECT",
|
||||
URL: &url.URL{Opaque: addr},
|
||||
Host: addr,
|
||||
Header: make(http.Header),
|
||||
}
|
||||
|
||||
// Add proxy authentication if provided
|
||||
if d.proxyURL.User != nil {
|
||||
username := d.proxyURL.User.Username()
|
||||
password, _ := d.proxyURL.User.Password()
|
||||
auth := base64.StdEncoding.EncodeToString([]byte(username + ":" + password))
|
||||
connectReq.Header.Set("Proxy-Authorization", "Basic "+auth)
|
||||
}
|
||||
|
||||
if err := connectReq.Write(conn); err != nil {
|
||||
conn.Close()
|
||||
return nil, fmt.Errorf("failed to write CONNECT request: %w", err)
|
||||
}
|
||||
|
||||
// Read response
|
||||
br := bufio.NewReader(conn)
|
||||
resp, err := http.ReadResponse(br, connectReq)
|
||||
if err != nil {
|
||||
conn.Close()
|
||||
return nil, fmt.Errorf("failed to read CONNECT response: %w", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
conn.Close()
|
||||
return nil, fmt.Errorf("proxy CONNECT failed with status: %s", resp.Status)
|
||||
}
|
||||
|
||||
return conn, nil
|
||||
}
|
||||
|
||||
func newProxyDialer(proxyUrl string) (proxy.ContextDialer, error) {
|
||||
parsedURL, err := url.Parse(proxyUrl)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return proxy.FromURL(url, proxy.Direct)
|
||||
|
||||
switch parsedURL.Scheme {
|
||||
case "http", "https":
|
||||
return &httpProxyDialer{
|
||||
proxyURL: parsedURL,
|
||||
forward: proxy.Direct,
|
||||
}, nil
|
||||
case "socks5", "socks5h":
|
||||
dialer, err := proxy.FromURL(parsedURL, proxy.Direct)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return dialer.(proxy.ContextDialer), nil
|
||||
default:
|
||||
return nil, fmt.Errorf("unsupported proxy scheme: %s", parsedURL.Scheme)
|
||||
}
|
||||
}
|
||||
|
||||
func NewConfigProxyResolver() (dcs.Resolver, error) {
|
||||
resolver := dcs.DefaultResolver()
|
||||
if config.C().Proxy != "" {
|
||||
// gloabl proxy, which has lower priority
|
||||
// global proxy, which has lower priority
|
||||
dialer, err := newProxyDialer(config.C().Proxy)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
resolver = dcs.Plain(dcs.PlainOptions{
|
||||
Dial: dialer.(proxy.ContextDialer).DialContext,
|
||||
Dial: dialer.DialContext,
|
||||
})
|
||||
}
|
||||
if config.C().Telegram.Proxy.Enable && config.C().Telegram.Proxy.URL != "" {
|
||||
@@ -34,7 +124,7 @@ func NewConfigProxyResolver() (dcs.Resolver, error) {
|
||||
return nil, err
|
||||
}
|
||||
resolver = dcs.Plain(dcs.PlainOptions{
|
||||
Dial: dialer.(proxy.ContextDialer).DialContext,
|
||||
Dial: dialer.DialContext,
|
||||
})
|
||||
}
|
||||
return resolver, nil
|
||||
|
||||
@@ -5,6 +5,10 @@ retry = 3 # 下载失败重试次数
|
||||
threads = 4 # 单个任务下载使用的最大线程数
|
||||
stream = false # 使用流式传输模式, 建议仅在硬盘空间十分有限时使用.
|
||||
|
||||
[log]
|
||||
# 日志级别, 可选: debug, info, warn, error, fatal
|
||||
level = "debug"
|
||||
|
||||
[telegram]
|
||||
# Bot Token
|
||||
# 更换 Bot Token 后请删除会话数据库文件 (默认路径为 data/session.db )
|
||||
@@ -14,10 +18,62 @@ token = ""
|
||||
# app_id = 1025907
|
||||
# app_hash = "452b0359b988148995f22ff0f4229750"
|
||||
[telegram.proxy]
|
||||
# 启用代理连接 telegram, 只支持 socks5
|
||||
# 启用代理连接 telegram
|
||||
enable = false
|
||||
url = "socks5://127.0.0.1:7890"
|
||||
|
||||
# Aria2 配置
|
||||
[aria2]
|
||||
# 启用 Aria2 下载支持
|
||||
enable = false
|
||||
# Aria2 RPC URL
|
||||
url = "http://localhost:6800/jsonrpc"
|
||||
# Aria2 RPC Secret (如果配置了 rpc-secret)
|
||||
secret = ""
|
||||
# 转存完成后删除 Aria2 下载的本地文件
|
||||
remove_after_transfer = true
|
||||
|
||||
# yt-dlp 视频下载配置
|
||||
[ytdlp]
|
||||
# 默认下载的最高视频清晰度 (按高度限制), 如 1080, 720, 480; 0 表示不限制 (下载最佳画质)
|
||||
# 仅在使用 /ytdlp 命令且未手动指定任何参数时生效
|
||||
max_height = 1080
|
||||
# 直接指定 yt-dlp format 选择表达式, 留空则使用 max_height
|
||||
# 设置后优先级高于 max_height, 例如: "bv*[height<=720]+ba/b"
|
||||
format = ""
|
||||
# 下载后转封装的视频容器格式, 留空则不转封装. 默认 mp4
|
||||
recode = "mp4"
|
||||
|
||||
# 解析器配置
|
||||
[parser]
|
||||
# 启用 JS 解析器插件 (Go 内置解析器默认启用)
|
||||
plugin_enable = false
|
||||
# 插件目录, 可以是多个目录
|
||||
plugin_dirs = ["./plugins"]
|
||||
# 解析器默认代理
|
||||
proxy = ""
|
||||
|
||||
# Twitter/X 解析器配置
|
||||
[parser.twitter]
|
||||
# 自定义 API 域名
|
||||
api_domain = "api.fxtwitter.com"
|
||||
# 单独为此解析器指定代理 (留空则使用 [parser] 中的 proxy)
|
||||
# proxy = "http://127.0.0.1:7890"
|
||||
|
||||
# Kemono 解析器配置 (暂无可配置项, 留空即可)
|
||||
[parser.kemono]
|
||||
|
||||
# HTTP API 配置
|
||||
[api]
|
||||
# 启用 HTTP API
|
||||
enable = false
|
||||
# 监听地址
|
||||
host = "0.0.0.0"
|
||||
# 监听端口
|
||||
port = 8080
|
||||
# 认证 Token (必需)
|
||||
token = ""
|
||||
|
||||
# 存储列表
|
||||
[[storages]]
|
||||
# 标识名, 需要唯一
|
||||
@@ -51,4 +107,4 @@ blacklist = true
|
||||
[[users]]
|
||||
id = 123456
|
||||
storages = ["本机1"]
|
||||
blacklist = false # 使用白名单模式,此时,用户 123456 仅可使用标识名为 '本地1' 的存储
|
||||
blacklist = false # 使用白名单模式,此时,用户 123456 仅可使用标识名为 '本地1' 的存储
|
||||
|
||||
@@ -17,6 +17,7 @@ func RegisterFlags(cmd *cobra.Command) {
|
||||
flags.Bool("stream", false, "enable stream mode")
|
||||
flags.Bool("no-clean-cache", false, "do not clean cache on exit")
|
||||
flags.String("proxy", "", "proxy URL (http, https, socks5, socks5h)")
|
||||
flags.String("log-level", "", "log level (trace/debug, info, warn, error, fatal)")
|
||||
|
||||
// Telegram 配置
|
||||
flags.String("telegram-token", "", "telegram bot token")
|
||||
@@ -54,6 +55,7 @@ func bindFlags(cmd *cobra.Command) {
|
||||
viper.BindPFlag("stream", flags.Lookup("stream"))
|
||||
viper.BindPFlag("no_clean_cache", flags.Lookup("no-clean-cache"))
|
||||
viper.BindPFlag("proxy", flags.Lookup("proxy"))
|
||||
viper.BindPFlag("log.level", flags.Lookup("log-level"))
|
||||
|
||||
// Telegram
|
||||
viper.BindPFlag("telegram.token", flags.Lookup("telegram-token"))
|
||||
|
||||
@@ -10,13 +10,4 @@ type hookExecConfig struct {
|
||||
TaskSuccess string `toml:"task_success" mapstructure:"task_success" json:"task_success"`
|
||||
TaskFail string `toml:"task_fail" mapstructure:"task_fail" json:"task_fail"`
|
||||
TaskCancel string `toml:"task_cancel" mapstructure:"task_cancel" json:"task_cancel"`
|
||||
|
||||
// TaskTypes map[string]hookExecOnTypeConfig `toml:"task_types" mapstructure:"task_types" json:"task_types"` // [TODO]
|
||||
}
|
||||
|
||||
// type hookExecOnTypeConfig struct {
|
||||
// TaskBeforeStart string `toml:"task_before_start" mapstructure:"task_before_start" json:"task_before_start"`
|
||||
// TaskSuccess string `toml:"task_success" mapstructure:"task_success" json:"task_success"`
|
||||
// TaskFail string `toml:"task_fail" mapstructure:"task_fail" json:"task_fail"`
|
||||
// TaskCancel string `toml:"task_cancel" mapstructure:"task_cancel" json:"task_cancel"`
|
||||
// }
|
||||
|
||||
5
config/log.go
Normal file
5
config/log.go
Normal file
@@ -0,0 +1,5 @@
|
||||
package config
|
||||
|
||||
type logConfig struct {
|
||||
Level string `toml:"level" mapstructure:"level" json:"level"`
|
||||
}
|
||||
@@ -16,6 +16,7 @@ var storageFactories = map[storenum.StorageType]func(cfg *BaseConfig) (StorageCo
|
||||
storenum.Minio: createStorageConfig(&MinioStorageConfig{}),
|
||||
storenum.S3: createStorageConfig(&S3StorageConfig{}),
|
||||
storenum.Telegram: createStorageConfig(&TelegramStorageConfig{}),
|
||||
storenum.Rclone: createStorageConfig(&RcloneStorageConfig{}),
|
||||
}
|
||||
|
||||
func createStorageConfig(configType StorageConfig) func(cfg *BaseConfig) (StorageConfig, error) {
|
||||
|
||||
33
config/storage/rclone.go
Normal file
33
config/storage/rclone.go
Normal file
@@ -0,0 +1,33 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
|
||||
storenum "github.com/krau/SaveAny-Bot/pkg/enums/storage"
|
||||
)
|
||||
|
||||
type RcloneStorageConfig struct {
|
||||
BaseConfig
|
||||
// The name of the remote as defined in rclone config
|
||||
Remote string `toml:"remote" mapstructure:"remote" json:"remote"`
|
||||
BasePath string `toml:"base_path" mapstructure:"base_path" json:"base_path"`
|
||||
// The path to the rclone config file, if not using the default
|
||||
ConfigPath string `toml:"config_path" mapstructure:"config_path" json:"config_path"`
|
||||
// Additional flags to pass to rclone commands
|
||||
Flags []string `toml:"flags" mapstructure:"flags" json:"flags"`
|
||||
}
|
||||
|
||||
func (r *RcloneStorageConfig) Validate() error {
|
||||
if r.Remote == "" {
|
||||
return fmt.Errorf("remote is required for rclone storage")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *RcloneStorageConfig) GetType() storenum.StorageType {
|
||||
return storenum.Rclone
|
||||
}
|
||||
|
||||
func (r *RcloneStorageConfig) GetName() string {
|
||||
return r.Name
|
||||
}
|
||||
@@ -8,14 +8,16 @@ import (
|
||||
|
||||
type TelegramStorageConfig struct {
|
||||
BaseConfig
|
||||
ChatID int64 `toml:"chat_id" mapstructure:"chat_id" json:"chat_id"`
|
||||
ForceFile bool `toml:"force_file" mapstructure:"force_file" json:"force_file"`
|
||||
RateLimit int `toml:"rate_limit" mapstructure:"rate_limit" json:"rate_limit"`
|
||||
RateBurst int `toml:"rate_burst" mapstructure:"rate_burst" json:"rate_burst"`
|
||||
SkipLarge bool `toml:"skip_large" mapstructure:"skip_large" json:"skip_large"` // skip files larger than Telegram limit(2GB)
|
||||
// split files larger than Telegram limit(2GB) into parts of specified size, in MB, leave 0 to set default(2000MB)
|
||||
ChatID int64 `toml:"chat_id" mapstructure:"chat_id" json:"chat_id"`
|
||||
ForceFile bool `toml:"force_file" mapstructure:"force_file" json:"force_file"`
|
||||
RateLimit int `toml:"rate_limit" mapstructure:"rate_limit" json:"rate_limit"`
|
||||
RateBurst int `toml:"rate_burst" mapstructure:"rate_burst" json:"rate_burst"`
|
||||
SkipLarge bool `toml:"skip_large" mapstructure:"skip_large" json:"skip_large"` // skip files larger than Telegram limit(2GB)
|
||||
SplitLargeVideo bool `toml:"split_large_video" mapstructure:"split_large_video" json:"split_large_video"`
|
||||
// split files larger than the uploader account limit into parts of specified size, in MB
|
||||
// leave 0 to use the account limit (2000MB for bots/regular users, 4000MB for Premium users)
|
||||
// only effective when SkipLarge is false
|
||||
// use zip when splitting
|
||||
// use zip when splitting non-video files or when lossless video splitting is disabled/unavailable
|
||||
SplitSizeMB int64 `toml:"split_size_mb" mapstructure:"split_size_mb" json:"split_size_mb"`
|
||||
}
|
||||
|
||||
|
||||
@@ -9,6 +9,7 @@ import (
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/duke-git/lancet/v2/slice"
|
||||
"github.com/krau/SaveAny-Bot/config/storage"
|
||||
"github.com/spf13/viper"
|
||||
@@ -16,13 +17,16 @@ import (
|
||||
)
|
||||
|
||||
type Config struct {
|
||||
Lang string `toml:"lang" mapstructure:"lang" json:"lang"`
|
||||
Workers int `toml:"workers" mapstructure:"workers"`
|
||||
Retry int `toml:"retry" mapstructure:"retry"`
|
||||
NoCleanCache bool `toml:"no_clean_cache" mapstructure:"no_clean_cache" json:"no_clean_cache"`
|
||||
Threads int `toml:"threads" mapstructure:"threads" json:"threads"`
|
||||
Stream bool `toml:"stream" mapstructure:"stream" json:"stream"`
|
||||
Proxy string `toml:"proxy" mapstructure:"proxy" json:"proxy"`
|
||||
Lang string `toml:"lang" mapstructure:"lang" json:"lang"`
|
||||
Workers int `toml:"workers" mapstructure:"workers"`
|
||||
Retry int `toml:"retry" mapstructure:"retry"`
|
||||
NoCleanCache bool `toml:"no_clean_cache" mapstructure:"no_clean_cache" json:"no_clean_cache"`
|
||||
Threads int `toml:"threads" mapstructure:"threads" json:"threads"`
|
||||
Stream bool `toml:"stream" mapstructure:"stream" json:"stream"`
|
||||
Proxy string `toml:"proxy" mapstructure:"proxy" json:"proxy"`
|
||||
Log logConfig `toml:"log" mapstructure:"log" json:"log"`
|
||||
Aria2 aria2Config `toml:"aria2" mapstructure:"aria2" json:"aria2"`
|
||||
API apiConfig `toml:"api" mapstructure:"api" json:"api"`
|
||||
|
||||
Cache cacheConfig `toml:"cache" mapstructure:"cache" json:"cache"`
|
||||
Users []userConfig `toml:"users" mapstructure:"users" json:"users"`
|
||||
@@ -32,6 +36,21 @@ type Config struct {
|
||||
Storages []storage.StorageConfig `toml:"-" mapstructure:"-" json:"storages"`
|
||||
Parser parserConfig `toml:"parser" mapstructure:"parser" json:"parser"`
|
||||
Hook hookConfig `toml:"hook" mapstructure:"hook" json:"hook"`
|
||||
Ytdlp YtdlpConfig `toml:"ytdlp" mapstructure:"ytdlp" json:"ytdlp"`
|
||||
}
|
||||
|
||||
type aria2Config struct {
|
||||
Enable bool `toml:"enable" mapstructure:"enable" json:"enable"`
|
||||
Url string `toml:"url" mapstructure:"url" json:"url"`
|
||||
Secret string `toml:"secret" mapstructure:"secret" json:"secret"`
|
||||
KeepFile bool `toml:"keep_file" mapstructure:"keep_file" json:"keep_file"`
|
||||
}
|
||||
|
||||
type apiConfig struct {
|
||||
Enable bool `toml:"enable" mapstructure:"enable" json:"enable"`
|
||||
Host string `toml:"host" mapstructure:"host" json:"host"`
|
||||
Port int `toml:"port" mapstructure:"port" json:"port"`
|
||||
Token string `toml:"token" mapstructure:"token" json:"token"`
|
||||
}
|
||||
|
||||
var cfg = &Config{}
|
||||
@@ -50,6 +69,13 @@ func (c Config) GetStorageByName(name string) storage.StorageConfig {
|
||||
}
|
||||
|
||||
func Init(ctx context.Context, configFile ...string) error {
|
||||
logger := log.FromContext(ctx)
|
||||
|
||||
// Reset side tables for re-init.
|
||||
storages = nil
|
||||
userIDs = nil
|
||||
userStorages = make(map[int64][]string)
|
||||
|
||||
viper.SetConfigType("toml")
|
||||
viper.SetEnvPrefix("SAVEANY")
|
||||
viper.AutomaticEnv()
|
||||
@@ -58,11 +84,13 @@ func Init(ctx context.Context, configFile ...string) error {
|
||||
|
||||
// 如果指定了配置文件路径,则使用指定的配置文件
|
||||
// 配置文件支持传入一个 http(s) URL 地址
|
||||
loadedFromURL := false
|
||||
if len(configFile) > 0 && configFile[0] != "" {
|
||||
cfg := configFile[0]
|
||||
if strings.HasPrefix(cfg, "http://") || strings.HasPrefix(cfg, "https://") {
|
||||
// 使用远程配置文件
|
||||
resp, err := http.Get(cfg)
|
||||
client := &http.Client{Timeout: 30 * time.Second}
|
||||
resp, err := client.Get(cfg)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to fetch remote config file: %w", err)
|
||||
}
|
||||
@@ -73,6 +101,7 @@ func Init(ctx context.Context, configFile ...string) error {
|
||||
if err := viper.ReadConfig(resp.Body); err != nil {
|
||||
return fmt.Errorf("failed to read remote config file: %w", err)
|
||||
}
|
||||
loadedFromURL = true
|
||||
} else {
|
||||
viper.SetConfigFile(cfg)
|
||||
}
|
||||
@@ -84,10 +113,11 @@ func Init(ctx context.Context, configFile ...string) error {
|
||||
|
||||
defaultConfigs := map[string]any{
|
||||
// 基础配置
|
||||
"lang": "zh-Hans",
|
||||
"workers": 3,
|
||||
"retry": 3,
|
||||
"threads": 4,
|
||||
"lang": "zh-Hans",
|
||||
"workers": 3,
|
||||
"retry": 3,
|
||||
"threads": 4,
|
||||
"log.level": "debug",
|
||||
|
||||
// 缓存配置
|
||||
"cache.ttl": 86400,
|
||||
@@ -107,25 +137,30 @@ func Init(ctx context.Context, configFile ...string) error {
|
||||
// 数据库
|
||||
"db.path": "data/saveany.db",
|
||||
"db.session": "data/session.db",
|
||||
|
||||
// API
|
||||
"api.enable": false,
|
||||
"api.host": "0.0.0.0",
|
||||
"api.port": 8080,
|
||||
"api.token": "",
|
||||
|
||||
// yt-dlp
|
||||
"ytdlp.recode": "mp4",
|
||||
}
|
||||
|
||||
for key, value := range defaultConfigs {
|
||||
viper.SetDefault(key, value)
|
||||
}
|
||||
|
||||
if err := viper.SafeWriteConfigAs("config.toml"); err != nil {
|
||||
if _, ok := err.(viper.ConfigFileAlreadyExistsError); !ok {
|
||||
return fmt.Errorf("error saving default config: %w", err)
|
||||
if !loadedFromURL {
|
||||
if err := viper.ReadInConfig(); err != nil {
|
||||
logger.Errorf("Error reading config file: %v", err)
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
if err := viper.ReadInConfig(); err != nil {
|
||||
fmt.Println("Error reading config file, ", err)
|
||||
return err
|
||||
}
|
||||
|
||||
if err := viper.Unmarshal(cfg); err != nil {
|
||||
fmt.Println("Error unmarshalling config file, ", err)
|
||||
logger.Errorf("Error unmarshalling config file: %v", err)
|
||||
return err
|
||||
}
|
||||
|
||||
|
||||
13
config/ytdlp.go
Normal file
13
config/ytdlp.go
Normal file
@@ -0,0 +1,13 @@
|
||||
package config
|
||||
|
||||
type YtdlpConfig struct {
|
||||
// MaxHeight limits the video resolution by height in pixels (e.g. 1080, 720).
|
||||
// 0 means no limit (best available). Ignored when Format is set.
|
||||
MaxHeight int `toml:"max_height" mapstructure:"max_height" json:"max_height"`
|
||||
// Format is a raw yt-dlp format selector (-f). When set, it takes precedence
|
||||
// over MaxHeight and gives the user full control.
|
||||
Format string `toml:"format" mapstructure:"format" json:"format"`
|
||||
// Recode is the target video container yt-dlp recodes into (e.g. mp4).
|
||||
// Empty disables recoding.
|
||||
Recode string `toml:"recode" mapstructure:"recode" json:"recode"`
|
||||
}
|
||||
58
core/core.go
58
core/core.go
@@ -3,14 +3,28 @@ package core
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"sync"
|
||||
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
"github.com/krau/SaveAny-Bot/database"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
|
||||
"github.com/krau/SaveAny-Bot/pkg/queue"
|
||||
"github.com/krau/SaveAny-Bot/pkg/taskevent"
|
||||
)
|
||||
|
||||
var queueInstance *queue.TaskQueue[Executable]
|
||||
var (
|
||||
queueOnce sync.Once
|
||||
queueInstance *queue.TaskQueue[Executable]
|
||||
)
|
||||
|
||||
// initQueue lazily creates the shared task queue.
|
||||
func initQueue() *queue.TaskQueue[Executable] {
|
||||
queueOnce.Do(func() {
|
||||
queueInstance = queue.NewTaskQueue[Executable]()
|
||||
})
|
||||
return queueInstance
|
||||
}
|
||||
|
||||
type Executable interface {
|
||||
Type() tasktype.TaskType
|
||||
@@ -30,11 +44,17 @@ func worker(ctx context.Context, qe *queue.TaskQueue[Executable], semaphore chan
|
||||
break // queue closed and empty
|
||||
}
|
||||
exe := qtask.Data
|
||||
taskCtx := qtask.Context()
|
||||
logger.Infof("Processing task: %s", exe.TaskID())
|
||||
if err := ExecCommandString(qtask.Context(), execHooks.TaskBeforeStart); err != nil {
|
||||
if err := database.UpdateTaskStatus(taskCtx, exe.TaskID(), database.TaskStatusRunning, ""); err != nil {
|
||||
logger.Errorf("Failed to mark task %s as running: %v", exe.TaskID(), err)
|
||||
}
|
||||
taskevent.Emit(taskCtx, taskevent.Event{TaskID: exe.TaskID(), Phase: taskevent.PhaseStart})
|
||||
if err := ExecCommandString(taskCtx, execHooks.TaskBeforeStart); err != nil {
|
||||
logger.Errorf("Failed to execute before start hook for task %s: %v", exe.TaskID(), err)
|
||||
}
|
||||
if err := exe.Execute(qtask.Context()); err != nil {
|
||||
err = exe.Execute(taskCtx)
|
||||
if err != nil {
|
||||
if errors.Is(err, context.Canceled) {
|
||||
logger.Infof("Task %s was canceled", exe.TaskID())
|
||||
if err := ExecCommandString(ctx, execHooks.TaskCancel); err != nil {
|
||||
@@ -52,7 +72,13 @@ func worker(ctx context.Context, qe *queue.TaskQueue[Executable], semaphore chan
|
||||
logger.Errorf("Failed to execute success hook for task %s: %v", exe.TaskID(), err)
|
||||
}
|
||||
}
|
||||
taskevent.Emit(taskCtx, taskevent.Event{TaskID: exe.TaskID(), Phase: taskevent.PhaseDone, Err: err})
|
||||
qe.Done(qtask.ID)
|
||||
// 用独立 ctx 删除: 优雅关停时 run ctx 已被取消, 会留下已完成任务的行,
|
||||
// 导致重启后重复执行 (重复上传)。
|
||||
if err := database.DeleteTask(context.Background(), exe.TaskID()); err != nil {
|
||||
logger.Errorf("Failed to delete persisted task %s: %v", exe.TaskID(), err)
|
||||
}
|
||||
<-semaphore
|
||||
}
|
||||
}
|
||||
@@ -60,22 +86,36 @@ func worker(ctx context.Context, qe *queue.TaskQueue[Executable], semaphore chan
|
||||
func Run(ctx context.Context) {
|
||||
log.FromContext(ctx).Info("Start processing tasks...")
|
||||
semaphore := make(chan struct{}, config.C().Workers)
|
||||
if queueInstance == nil {
|
||||
queueInstance = queue.NewTaskQueue[Executable]()
|
||||
}
|
||||
q := initQueue()
|
||||
for range config.C().Workers {
|
||||
go worker(ctx, queueInstance, semaphore)
|
||||
go worker(ctx, q, semaphore)
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
// Close stops the queue and unblocks workers in Get.
|
||||
func Close() {
|
||||
if q := initQueue(); q != nil {
|
||||
q.Close()
|
||||
}
|
||||
}
|
||||
|
||||
func AddTask(ctx context.Context, task Executable) error {
|
||||
return queueInstance.Add(queue.NewTask(ctx, task.TaskID(), task.Title(), task))
|
||||
if err := persistTask(ctx, task); err != nil {
|
||||
log.FromContext(ctx).Errorf("Failed to persist task %s: %v", task.TaskID(), err)
|
||||
}
|
||||
return initQueue().Add(queue.NewTask(ctx, task.TaskID(), task.Title(), task))
|
||||
}
|
||||
|
||||
func CancelTask(ctx context.Context, id string) error {
|
||||
err := queueInstance.CancelTask(id)
|
||||
return err
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := database.DeleteTask(ctx, id); err != nil {
|
||||
log.FromContext(ctx).Errorf("Failed to delete persisted task %s: %v", id, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func GetLength(ctx context.Context) int {
|
||||
|
||||
145
core/persist.go
Normal file
145
core/persist.go
Normal file
@@ -0,0 +1,145 @@
|
||||
package core
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"fmt"
|
||||
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/gotd/td/telegram/downloader"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/database"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
|
||||
)
|
||||
|
||||
// TaskCodec serializes and rebuilds a task from its persisted payload.
|
||||
// Task types without a registered codec are dropped with a warning on
|
||||
// recovery instead of being silently re-enqueued.
|
||||
type TaskCodec interface {
|
||||
Marshal(task Executable) ([]byte, error)
|
||||
Unmarshal(payload []byte) (Executable, error)
|
||||
}
|
||||
|
||||
var (
|
||||
taskCodecsMu sync.RWMutex
|
||||
taskCodecs = make(map[tasktype.TaskType]TaskCodec)
|
||||
|
||||
dlerMu sync.RWMutex
|
||||
dlerProvider func() downloader.Client
|
||||
)
|
||||
|
||||
func RegisterTaskCodec(t tasktype.TaskType, codec TaskCodec) {
|
||||
taskCodecsMu.Lock()
|
||||
defer taskCodecsMu.Unlock()
|
||||
taskCodecs[t] = codec
|
||||
}
|
||||
|
||||
func TaskCodecFor(t tasktype.TaskType) (TaskCodec, bool) {
|
||||
taskCodecsMu.RLock()
|
||||
defer taskCodecsMu.RUnlock()
|
||||
codec, ok := taskCodecs[t]
|
||||
return codec, ok
|
||||
}
|
||||
|
||||
// SetDownloaderProvider registers the download client factory used to
|
||||
// rebuild tfile.TGFile values when recovering tasks.
|
||||
func SetDownloaderProvider(f func() downloader.Client) {
|
||||
dlerMu.Lock()
|
||||
defer dlerMu.Unlock()
|
||||
dlerProvider = f
|
||||
}
|
||||
|
||||
// DownloaderClient returns the registered download client, or nil.
|
||||
func DownloaderClient() downloader.Client {
|
||||
dlerMu.RLock()
|
||||
defer dlerMu.RUnlock()
|
||||
if dlerProvider == nil {
|
||||
return nil
|
||||
}
|
||||
return dlerProvider()
|
||||
}
|
||||
|
||||
func persistTask(ctx context.Context, task Executable) error {
|
||||
codec, ok := TaskCodecFor(task.Type())
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
payload, err := codec.Marshal(task)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return database.UpsertTask(ctx, &database.Task{
|
||||
ID: task.TaskID(),
|
||||
Type: string(task.Type()),
|
||||
Payload: payload,
|
||||
Status: string(database.TaskStatusQueued),
|
||||
})
|
||||
}
|
||||
|
||||
// UpdateTaskPayload atomically mutates the persisted payload of a running
|
||||
// task (e.g. recording per-element upload progress for recovery).
|
||||
func UpdateTaskPayload(ctx context.Context, id string, mutate func(payload []byte) ([]byte, error)) error {
|
||||
row, err := database.GetTask(ctx, id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
updated, err := mutate(row.Payload)
|
||||
if err != nil {
|
||||
return fmt.Errorf("mutate payload: %w", err)
|
||||
}
|
||||
return database.UpdateTaskPayload(ctx, id, updated)
|
||||
}
|
||||
|
||||
// RecoverTasks re-enqueues tasks that were unfinished when the process last
|
||||
// exited. Must be called after storages are loaded and before Run. Tasks
|
||||
// that cannot be recovered are marked failed and kept for visibility.
|
||||
func RecoverTasks(ctx context.Context) {
|
||||
logger := log.FromContext(ctx)
|
||||
if err := database.DeleteStaleFailedTasks(ctx, 24*time.Hour); err != nil {
|
||||
logger.Warnf("Failed to clean stale failed tasks: %v", err)
|
||||
}
|
||||
tasks, err := database.GetUnfinishedTasks(ctx)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to load unfinished tasks: %v", err)
|
||||
return
|
||||
}
|
||||
for _, t := range tasks {
|
||||
codec, ok := TaskCodecFor(tasktype.TaskType(t.Type))
|
||||
if !ok {
|
||||
logger.Warnf("Task %s (type %s) cannot be recovered: no codec registered", t.ID, t.Type)
|
||||
markRecoverFailed(ctx, t, "no codec registered")
|
||||
continue
|
||||
}
|
||||
task, err := codec.Unmarshal(t.Payload)
|
||||
if err != nil {
|
||||
logger.Errorf("Task %s cannot be recovered: failed to rebuild: %v", t.ID, err)
|
||||
markRecoverFailed(ctx, t, err.Error())
|
||||
continue
|
||||
}
|
||||
if initQueue().Contains(task.TaskID()) {
|
||||
// Already live in the queue (e.g. submitted via API during
|
||||
// startup); keep the row as-is.
|
||||
logger.Infof("Task %s already queued, keeping row", t.ID)
|
||||
continue
|
||||
}
|
||||
if err := AddTask(ctx, task); err != nil {
|
||||
logger.Errorf("Task %s cannot be recovered: failed to re-enqueue: %v", t.ID, err)
|
||||
markRecoverFailed(ctx, t, err.Error())
|
||||
continue
|
||||
}
|
||||
// Upsert cleared the original creation time; restore it so
|
||||
// GetUnfinishedTasks ordering stays stable across restarts.
|
||||
if err := database.RestoreTaskCreatedAt(ctx, t.ID, t.CreatedAt); err != nil {
|
||||
logger.Warnf("Failed to restore created_at for task %s: %v", t.ID, err)
|
||||
}
|
||||
logger.Infof("Recovered task %s (%s)", t.ID, t.Type)
|
||||
}
|
||||
}
|
||||
|
||||
func markRecoverFailed(ctx context.Context, t database.Task, reason string) {
|
||||
if err := database.UpdateTaskStatus(ctx, t.ID, database.TaskStatusFailed, reason); err != nil {
|
||||
log.FromContext(ctx).Errorf("Failed to mark task %s as failed: %v", t.ID, err)
|
||||
}
|
||||
}
|
||||
162
core/persist_test.go
Normal file
162
core/persist_test.go
Normal file
@@ -0,0 +1,162 @@
|
||||
package core
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
"github.com/krau/SaveAny-Bot/database"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
|
||||
)
|
||||
|
||||
const testRecoverType = tasktype.TaskType("test-recover")
|
||||
|
||||
type stubTask struct {
|
||||
id string
|
||||
}
|
||||
|
||||
func (s *stubTask) Type() tasktype.TaskType { return testRecoverType }
|
||||
func (s *stubTask) Title() string { return s.id }
|
||||
func (s *stubTask) TaskID() string { return s.id }
|
||||
func (s *stubTask) Execute(context.Context) error { return nil }
|
||||
|
||||
type stubCodec struct{}
|
||||
|
||||
func (stubCodec) Marshal(task Executable) ([]byte, error) {
|
||||
return []byte(task.TaskID()), nil
|
||||
}
|
||||
|
||||
func (stubCodec) Unmarshal(payload []byte) (Executable, error) {
|
||||
if len(payload) == 0 {
|
||||
return nil, fmt.Errorf("empty payload")
|
||||
}
|
||||
return &stubTask{id: string(payload)}, nil
|
||||
}
|
||||
|
||||
func initRecoveryEnv(t *testing.T) context.Context {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
cfgPath := filepath.Join(dir, "config.toml")
|
||||
content := fmt.Sprintf("[db]\npath = %q\n", filepath.Join(dir, "test.db"))
|
||||
if err := os.WriteFile(cfgPath, []byte(content), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := config.Init(context.Background(), cfgPath); err != nil {
|
||||
t.Fatalf("config init: %v", err)
|
||||
}
|
||||
database.Init(context.Background())
|
||||
RegisterTaskCodec(testRecoverType, stubCodec{})
|
||||
return context.Background()
|
||||
}
|
||||
|
||||
func TestRecoverTasksReenqueuesAndMarksUnknownFailed(t *testing.T) {
|
||||
ctx := initRecoveryEnv(t)
|
||||
|
||||
if err := database.CreateTask(ctx, &database.Task{
|
||||
ID: "rec-1", Type: string(testRecoverType), Payload: []byte("rec-1"), Status: string(database.TaskStatusQueued),
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := database.CreateTask(ctx, &database.Task{
|
||||
ID: "rec-2", Type: string(testRecoverType), Payload: []byte("rec-2"), Status: string(database.TaskStatusRunning),
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := database.CreateTask(ctx, &database.Task{
|
||||
ID: "drop-1", Type: "unregistered", Payload: nil, Status: string(database.TaskStatusQueued),
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
RecoverTasks(ctx)
|
||||
|
||||
ids := map[string]bool{}
|
||||
for _, info := range GetQueuedTasks(ctx) {
|
||||
ids[info.ID] = true
|
||||
}
|
||||
if !ids["rec-1"] || !ids["rec-2"] {
|
||||
t.Fatalf("recovered task ids = %v, want rec-1 and rec-2", ids)
|
||||
}
|
||||
|
||||
unfinished, err := database.GetUnfinishedTasks(ctx)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(unfinished) != 2 {
|
||||
t.Fatalf("unfinished rows = %d, want 2", len(unfinished))
|
||||
}
|
||||
for _, task := range unfinished {
|
||||
if task.ID == "drop-1" {
|
||||
t.Fatalf("unregistered task record was not dropped")
|
||||
}
|
||||
if task.Status != string(database.TaskStatusQueued) {
|
||||
t.Fatalf("recovered task status = %s, want queued", task.Status)
|
||||
}
|
||||
}
|
||||
// The unrecoverable task must be kept and marked failed, not silently deleted.
|
||||
drop, err := database.GetTask(ctx, "drop-1")
|
||||
if err != nil {
|
||||
t.Fatalf("dropped task row missing: %v", err)
|
||||
}
|
||||
if drop.Status != string(database.TaskStatusFailed) {
|
||||
t.Fatalf("dropped task status = %s, want failed", drop.Status)
|
||||
}
|
||||
if drop.Error == "" {
|
||||
t.Fatalf("dropped task has no failure reason")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRecoverTasksMarksInvalidPayloadFailed(t *testing.T) {
|
||||
ctx := initRecoveryEnv(t)
|
||||
|
||||
if err := database.CreateTask(ctx, &database.Task{
|
||||
ID: "bad-1", Type: string(testRecoverType), Payload: nil, Status: string(database.TaskStatusQueued),
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
RecoverTasks(ctx)
|
||||
|
||||
// bad-1 must not be enqueued; its row is kept as failed.
|
||||
for _, info := range GetQueuedTasks(ctx) {
|
||||
if info.ID == "bad-1" {
|
||||
t.Fatalf("task with invalid payload was enqueued")
|
||||
}
|
||||
}
|
||||
count, err := database.CountUnfinishedTasks(ctx)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if count != 0 {
|
||||
t.Fatalf("unfinished rows = %d, want 0", count)
|
||||
}
|
||||
bad, err := database.GetTask(ctx, "bad-1")
|
||||
if err != nil {
|
||||
t.Fatalf("failed task row missing: %v", err)
|
||||
}
|
||||
if bad.Status != string(database.TaskStatusFailed) {
|
||||
t.Fatalf("bad task status = %s, want failed", bad.Status)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRecoverTasksSkipsAlreadyQueued(t *testing.T) {
|
||||
ctx := initRecoveryEnv(t)
|
||||
|
||||
// A task submitted during startup is both persisted and in the queue.
|
||||
task := &stubTask{id: "live-1"}
|
||||
if err := AddTask(ctx, task); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
RecoverTasks(ctx)
|
||||
|
||||
// The row must survive with its original status.
|
||||
row, err := database.GetTask(ctx, "live-1")
|
||||
if err != nil {
|
||||
t.Fatalf("row missing for queued task: %v", err)
|
||||
}
|
||||
if row.Status != string(database.TaskStatusQueued) {
|
||||
t.Fatalf("row status = %s, want queued", row.Status)
|
||||
}
|
||||
}
|
||||
271
core/tasks/aria2dl/execute.go
Normal file
271
core/tasks/aria2dl/execute.go
Normal file
@@ -0,0 +1,271 @@
|
||||
package aria2dl
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
"github.com/krau/SaveAny-Bot/pkg/aria2"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/ctxkey"
|
||||
"github.com/krau/SaveAny-Bot/pkg/taskevent"
|
||||
)
|
||||
|
||||
// Execute implements core.Executable.
|
||||
func (t *Task) Execute(ctx context.Context) error {
|
||||
logger := log.FromContext(ctx)
|
||||
logger.Infof("Starting aria2 download task %s (GID: %s)", t.ID, t.GID)
|
||||
|
||||
if t.Progress != nil {
|
||||
t.Progress.OnStart(ctx, t)
|
||||
}
|
||||
|
||||
// Wait for aria2 download to complete
|
||||
if err := t.waitForDownload(ctx); err != nil {
|
||||
// If context was canceled, also cancel the aria2 download
|
||||
if errors.Is(err, context.Canceled) {
|
||||
t.cancelAria2Download()
|
||||
}
|
||||
logger.Errorf("Aria2 download failed: %v", err)
|
||||
if t.Progress != nil {
|
||||
t.Progress.OnDone(ctx, t, err)
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
// Transfer downloaded files to storage
|
||||
if err := t.transferFiles(ctx); err != nil {
|
||||
logger.Errorf("File transfer failed: %v", err)
|
||||
if t.Progress != nil {
|
||||
t.Progress.OnDone(ctx, t, err)
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
logger.Infof("Aria2 task %s completed successfully", t.ID)
|
||||
if t.Progress != nil {
|
||||
t.Progress.OnDone(ctx, t, nil)
|
||||
}
|
||||
|
||||
// Clean up aria2 download result
|
||||
if _, err := t.Aria2Client.RemoveDownloadResult(context.Background(), t.GID); err != nil {
|
||||
logger.Warnf("Failed to remove aria2 download result: %v", err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// waitForDownload waits for aria2 to complete the download
|
||||
func (t *Task) waitForDownload(ctx context.Context) error {
|
||||
logger := log.FromContext(ctx)
|
||||
ticker := time.NewTicker(2 * time.Second)
|
||||
defer ticker.Stop()
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
case <-ticker.C:
|
||||
status, err := t.getStatus(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if t.Progress != nil {
|
||||
t.Progress.OnProgress(ctx, t, status)
|
||||
}
|
||||
taskevent.Emit(ctx, taskevent.Event{
|
||||
TaskID: t.ID,
|
||||
Phase: taskevent.PhaseProgress,
|
||||
TotalBytes: parseInt64(status.TotalLength),
|
||||
DownloadedBytes: parseInt64(status.CompletedLength),
|
||||
})
|
||||
|
||||
// Check if download is complete
|
||||
if status.IsDownloadComplete() {
|
||||
// Handle metadata downloads (torrent/magnet) that spawn follow-up downloads
|
||||
if len(status.FollowedBy) > 0 {
|
||||
logger.Infof("Switching from metadata GID %s to actual download GID: %s", t.GID, status.FollowedBy[0])
|
||||
t.GID = status.FollowedBy[0]
|
||||
continue
|
||||
}
|
||||
logger.Infof("Download completed for GID %s", t.GID)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Check for errors
|
||||
if status.IsDownloadError() {
|
||||
return fmt.Errorf("aria2 download error: %s (code: %s)", status.ErrorMessage, status.ErrorCode)
|
||||
}
|
||||
|
||||
if status.IsDownloadRemoved() {
|
||||
return errors.New("aria2 download was removed")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// getStatus retrieves the current status of the download
|
||||
func (t *Task) getStatus(ctx context.Context) (*aria2.Status, error) {
|
||||
logger := log.FromContext(ctx)
|
||||
|
||||
// Try active/waiting queue first
|
||||
status, err := t.Aria2Client.TellStatus(ctx, t.GID)
|
||||
if err == nil {
|
||||
return status, nil
|
||||
}
|
||||
|
||||
// Check stopped queue
|
||||
logger.Debugf("Task not in active queue, checking stopped queue")
|
||||
stoppedTasks, stopErr := t.Aria2Client.TellStopped(ctx, -1, 100)
|
||||
if stopErr != nil {
|
||||
return nil, fmt.Errorf("failed to get aria2 status: %w", err)
|
||||
}
|
||||
|
||||
for _, task := range stoppedTasks {
|
||||
if task.GID == t.GID {
|
||||
logger.Debugf("Found task in stopped queue with status: %s", task.Status)
|
||||
return &task, nil
|
||||
}
|
||||
}
|
||||
|
||||
return nil, fmt.Errorf("task GID %s not found: %w", t.GID, err)
|
||||
}
|
||||
|
||||
// transferFiles transfers downloaded files from aria2 to storage
|
||||
func (t *Task) transferFiles(ctx context.Context) error {
|
||||
logger := log.FromContext(ctx)
|
||||
|
||||
status, err := t.Aria2Client.TellStatus(ctx, t.GID)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to get final status: %w", err)
|
||||
}
|
||||
|
||||
if len(status.Files) == 0 {
|
||||
return errors.New("no files in aria2 download")
|
||||
}
|
||||
|
||||
logger.Infof("Transferring %d file(s) to storage %s", len(status.Files), t.Storage.Name())
|
||||
transferredCount := 0
|
||||
|
||||
for _, file := range status.Files {
|
||||
if file.Selected != "true" {
|
||||
logger.Debugf("Skipping unselected file: %s", file.Path)
|
||||
continue
|
||||
}
|
||||
|
||||
fileName := filepath.Base(file.Path)
|
||||
|
||||
// Skip torrent metadata files
|
||||
if filepath.Ext(fileName) == ".torrent" {
|
||||
logger.Debugf("Skipping torrent metadata file: %s", fileName)
|
||||
t.removeFileIfNeeded(file.Path)
|
||||
continue
|
||||
}
|
||||
|
||||
if err := t.transferFile(ctx, file.Path); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
transferredCount++
|
||||
t.removeFileIfNeeded(file.Path)
|
||||
}
|
||||
|
||||
if transferredCount == 0 {
|
||||
return errors.New("no files were transferred")
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// transferFile transfers a single file to storage
|
||||
func (t *Task) transferFile(ctx context.Context, filePath string) error {
|
||||
logger := log.FromContext(ctx)
|
||||
|
||||
// Check if file exists
|
||||
fileInfo, err := os.Stat(filePath)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
logger.Warnf("Downloaded file not found: %s", filePath)
|
||||
return nil // Not a fatal error, continue with other files
|
||||
}
|
||||
return fmt.Errorf("failed to stat file %s: %w", filePath, err)
|
||||
}
|
||||
|
||||
// Open file
|
||||
f, err := os.Open(filePath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to open file %s: %w", filePath, err)
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
// Set content length in context for storage
|
||||
ctx = context.WithValue(ctx, ctxkey.ContentLength, fileInfo.Size())
|
||||
|
||||
// Save to storage
|
||||
fileName := filepath.Base(filePath)
|
||||
destPath := filepath.Join(t.StorPath, fileName)
|
||||
|
||||
logger.Infof("Transferring file %s to %s:%s", fileName, t.Storage.Name(), destPath)
|
||||
|
||||
if err := t.Storage.Save(ctx, f, destPath); err != nil {
|
||||
return fmt.Errorf("failed to save file %s to storage: %w", fileName, err)
|
||||
}
|
||||
|
||||
logger.Infof("Successfully transferred file %s", fileName)
|
||||
return nil
|
||||
}
|
||||
|
||||
// removeFileIfNeeded removes a file if RemoveAfterTransfer is enabled
|
||||
func (t *Task) removeFileIfNeeded(filePath string) {
|
||||
if config.C().Aria2.KeepFile {
|
||||
return
|
||||
}
|
||||
|
||||
logger := log.FromContext(t.ctx)
|
||||
if err := os.Remove(filePath); err != nil {
|
||||
logger.Warnf("Failed to remove local file %s: %v", filePath, err)
|
||||
} else {
|
||||
logger.Debugf("Removed local file %s", filePath)
|
||||
}
|
||||
}
|
||||
|
||||
// cancelAria2Download cancels the aria2 download task
|
||||
func (t *Task) cancelAria2Download() {
|
||||
logger := log.FromContext(t.ctx)
|
||||
logger.Infof("Canceling aria2 download GID: %s", t.GID)
|
||||
|
||||
// Use a background context with timeout for cleanup
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer cancel()
|
||||
|
||||
// Try to force remove the download
|
||||
if _, err := t.Aria2Client.ForceRemove(ctx, t.GID); err != nil {
|
||||
logger.Warnf("Failed to cancel aria2 download %s: %v", t.GID, err)
|
||||
} else {
|
||||
logger.Infof("Successfully canceled aria2 download %s", t.GID)
|
||||
}
|
||||
|
||||
// Also remove the download result to clean up
|
||||
if _, err := t.Aria2Client.RemoveDownloadResult(ctx, t.GID); err != nil {
|
||||
logger.Debugf("Failed to remove download result for %s: %v", t.GID, err)
|
||||
}
|
||||
}
|
||||
|
||||
// parseInt64 parses an aria2 status string (decimal bytes) into int64,
|
||||
// returning 0 on failure so it can be used directly in progress events.
|
||||
func parseInt64(s string) int64 {
|
||||
if s == "" {
|
||||
return 0
|
||||
}
|
||||
n, err := strconv.ParseInt(s, 10, 64)
|
||||
if err != nil {
|
||||
return 0
|
||||
}
|
||||
return n
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user