Compare commits
44 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
38355dfd14 | ||
|
|
0940258b4d | ||
|
|
602fc251d8 | ||
|
|
af28738235 | ||
|
|
3eb3b6e3c8 | ||
|
|
f377ee3ca4 | ||
|
|
70f7172162 | ||
|
|
091f581881 | ||
|
|
8b86330f5c | ||
|
|
b431fa08e2 | ||
|
|
a02e8a8d90 | ||
|
|
4d2c345003 | ||
|
|
33a886fac9 | ||
|
|
57539ec3da | ||
|
|
82e1efb518 | ||
|
|
9b52a3e0ce | ||
|
|
6990543c9f | ||
|
|
dd0dea8cb5 | ||
|
|
3d20fbd0fe | ||
|
|
e6d8cc775a | ||
|
|
17e340fff1 | ||
|
|
f92c43b9c8 | ||
|
|
3e20dc2c5f | ||
|
|
3ce00884a0 | ||
|
|
cd7cf4964d | ||
|
|
bc3c841d1d | ||
|
|
743c15f1a5 | ||
|
|
b05d86509c | ||
|
|
f17a380579 | ||
|
|
cabe1189e2 | ||
|
|
a988d05e24 | ||
|
|
1af2c1f7c7 | ||
|
|
7b36fb45f5 | ||
|
|
62cceee592 | ||
|
|
6d315f7af2 | ||
|
|
5352491c76 | ||
|
|
3f914f7a64 | ||
|
|
8972d8a169 | ||
|
|
1339c69dbf | ||
|
|
63aeabb39b | ||
|
|
e60e983229 | ||
|
|
75e5fd10ea | ||
|
|
c8d8a2e0eb | ||
|
|
044e732084 |
6
.github/ISSUE_TEMPLATE/bug.yml
vendored
6
.github/ISSUE_TEMPLATE/bug.yml
vendored
@@ -5,6 +5,12 @@ labels:
|
||||
assignees:
|
||||
- krau
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
# Please Search Before Submitting / 提交前请搜索
|
||||
Please make sure to search existing issues before submitting a new bug report.
|
||||
提交新的 Bug 报告前请务必搜索已有的 issue,避免重复
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: "👾 Description"
|
||||
|
||||
66
.github/ISSUE_TEMPLATE/feature.yml
vendored
66
.github/ISSUE_TEMPLATE/feature.yml
vendored
@@ -8,7 +8,69 @@ body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
# Please describe the feature you want in detail
|
||||
Please describe the feature you want in detail.
|
||||
请详细描述你想要的功能。
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ IMPORTANT NOTICE / 说明
|
||||
|
||||
Save Any Bot supports multiple storage backends, **including Telegram**.
|
||||
However, **all backends are treated equally**, keep this in mind when submitting feature requests.
|
||||
|
||||
Save Any Bot 支持多种存储后端,**包括 Telegram**。
|
||||
但**所有后端在设计上是平等的**,请在提出功能请求前务必理解这一点。
|
||||
|
||||
### ❌ Out of scope requests / 不在项目范围内的请求
|
||||
The following requests are **out of scope** and will be closed without discussion:
|
||||
|
||||
以下请求**不属于本项目设计范围**,将被直接关闭,不再讨论:
|
||||
|
||||
- Adding **Telegram-specific behaviors or exceptions**
|
||||
添加 **仅针对 Telegram 的特殊行为或例外逻辑**
|
||||
- Treating Telegram as anything other than a **generic file storage backend**
|
||||
将 Telegram 视为非“通用文件存储后端”的特殊存在
|
||||
- Saving or syncing **non-file content** (text messages, chat history, etc.)
|
||||
保存或同步 **非文件内容**(文本消息、聊天记录等)
|
||||
- Preserving or reconstructing original messages (e.g. 1:1 forwarding)
|
||||
保留或还原原始消息形态(例如 1:1 转发)
|
||||
- Perform special reprocessing on files to adapt to specific storage backends
|
||||
(e.g. splitting, re-encoding, transforming, etc.)
|
||||
为适配特定存储后端而对文件进行特殊处理
|
||||
(如分割、转码、重编码、转换格式等)
|
||||
- Any request that requires different logic *only because the backend is Telegram*
|
||||
任何**仅因后端是 Telegram 而需要不同逻辑**的请求
|
||||
|
||||
### ❌ Abuse-leaning or high-risk requests / 滥用倾向的请求
|
||||
Requests that may **enable or encourage** the following will NOT be accepted:
|
||||
|
||||
可能**促成或鼓励**以下行为的请求将不会被接受:
|
||||
|
||||
- Violating Telegram Terms of Service
|
||||
违反 Telegram 服务条款
|
||||
- Building traffic, mirror, or profit-oriented channels using third-party content
|
||||
利用第三方内容构建引流、镜像或牟利用途的频道
|
||||
|
||||
### ⚖️ Design principle / 设计原则
|
||||
Save Any Bot follows a **backend-agnostic design**:
|
||||
|
||||
Save Any Bot 遵循 **后端无关(backend-agnostic)** 的设计原则:
|
||||
|
||||
- If a feature cannot be implemented **uniformly across all backends**, it will not be added.
|
||||
如果某个功能无法在 **所有后端** 中统一实现,则不会被添加。
|
||||
- No backend-specific hacks or special cases will be introduced.
|
||||
不会引入任何后端特有的 hack 或特殊处理逻辑。
|
||||
|
||||
---
|
||||
|
||||
If your request falls into any of the categories above, please do not open an issue.
|
||||
Such issues will be closed.
|
||||
|
||||
如果你的请求符合以上任一情况,请不要提交 issue,
|
||||
相关 issue 将被直接关闭。
|
||||
|
||||
Thank you for respecting the scope and design principles of this project.
|
||||
感谢你的理解与支持。
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: "⭐️ Feature description"
|
||||
@@ -30,4 +92,4 @@ body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
## Thank you for contributing to the project :slightly_smiling_face:
|
||||
## Thank you for contributing to the project :slightly_smiling_face:
|
||||
|
||||
86
.github/copilot-instructions.md
vendored
86
.github/copilot-instructions.md
vendored
@@ -1,86 +0,0 @@
|
||||
# SaveAny-Bot AI 协作说明
|
||||
|
||||
本项目是一个将 Telegram 文件/消息转存到多种存储端的 Bot,主要用 Go 实现,CLI 入口在 `cmd/`,核心逻辑分布在 `client/`、`core/`、`config/`、`database/`、`storage/` 等目录。下面是针对本仓库的专用约定,供 AI 编码助手参考。
|
||||
|
||||
## 总体架构与入口
|
||||
- **CLI 入口**:
|
||||
- 二进制入口:`main.go` 调用 `cmd.Execute(ctx)`。
|
||||
- 根命令:`cmd/root.go` 使用 `cobra` 定义 `saveany-bot`,`Run` 实现在 `cmd/run.go`。
|
||||
- **应用启动流程(非常重要)**:
|
||||
- `cmd/run.go::Run` 中按顺序完成:读取配置 `config.Init` → 初始化缓存 `common/cache` → 初始化 i18n `common/i18n` → 初始化数据库 `database.Init` → 加载存储 `storage.LoadStorages` → 加载解析器插件 `parsers.LoadPlugins` → (可选)Userbot 登录 → 启动 Telegram Bot `client/bot.Init` → 启动核心任务队列消费 `core.Run`。
|
||||
- 添加新的初始化步骤时,请遵循该顺序并放在 `initAll` 中,而不是分散在各处。
|
||||
|
||||
## 配置与约定
|
||||
- **配置系统**:
|
||||
- 使用 `viper` 读取 `config.toml`,核心结构体定义在 `config/viper.go::Config`。
|
||||
- 默认值在 `config.Init` 中通过 `viper.SetDefault` 定义,如 `workers`、`retry`、`telegram.*`、`db.*` 等。
|
||||
- `Config.C()` 返回的是全局配置副本(值类型),不要在返回值上修改字段;如果需要修改配置流程,应在 `config.Init` 内或通过 `viper` 进行。
|
||||
- 存储配置通过 `config/storage/factory.go::LoadStorageConfigs` 加载并校验,新增存储类型需:
|
||||
- 在 `pkg/enums/storage` 中增加枚举。
|
||||
- 在 `config/storage/` 下新增具体 `StorageConfig` 实现并实现 `Validate`。
|
||||
- 在 `storageFactories` 映射中注册工厂方法。
|
||||
- **环境变量**:
|
||||
- 所有配置键会被 `SAVEANY_` 前缀的环境变量覆盖,`.` 会被 `_` 替换(`SAVEANY_TELEGRAM_APP_ID` 等)。
|
||||
|
||||
## Telegram 客户端与中间件
|
||||
- **Bot 客户端**:
|
||||
- 入口在 `client/bot/bot.go::Init`,使用 `gotgproto.NewClient` 创建,Session 使用 `database.GetDialect(config.C().DB.Session)`,错误处理通过 `ErrorHandler` 回调完成。
|
||||
- Handlers 注册集中在 `client/bot/handlers` 目录,`handlers.Register` 负责统一挂载;新增命令/消息处理逻辑时优先在该目录按功能拆分文件,实现后在 `handlers.Register` 中注册。
|
||||
- Bot 命令列表依赖 `handlers.CommandHandlers` 来自动注册到 Telegram;新增命令时务必更新该切片,以保持 `/help` 与 Bot 命令列表一致。
|
||||
- **中间件**:
|
||||
- 通用中间件位于 `client/middleware/`,包含 floodwait、防崩溃、重试等;`middleware.NewDefaultMiddlewares` 在 `client/bot/bot.go` 中统一挂载。
|
||||
- 新增跨所有更新生效的行为(如日志、统计)时,应优先实现为中间件。
|
||||
|
||||
## 核心任务与队列
|
||||
- **任务接口与队列**:
|
||||
- 核心接口:`core/core.go::Executable`,包含 `Type() TaskType`、`Title()`、`TaskID()`、`Execute(ctx)`。
|
||||
- 任务队列:`pkg/queue.TaskQueue[Executable]`,由 `core.Run` 使用;`Workers` 数量来自配置 `config.C().Workers`。
|
||||
- 任务类型与实现示例位于 `core/tasks/**`,例如文件任务、Telegraph 任务等;新增任务类型应放在对应子目录并实现 `Executable` 接口,然后通过 `core.AddTask` 入队。
|
||||
- **生命周期 Hook**:
|
||||
- `core.worker` 在执行任务前后会根据 `config.C().Hook.Exec` 调用外部命令(`TaskBeforeStart` / `TaskSuccess` / `TaskFail` / `TaskCancel`)。
|
||||
- 修改任务执行流程时需保留这些 Hook 调用,以免破坏用户已有集成。
|
||||
|
||||
## 数据库与持久化
|
||||
- **数据库初始化**:
|
||||
- `database.Init` 使用配置 `config.C().DB.Path` 创建并连接 SQLite,使用 `GetDialect` 抽象驱动(见 `database/driver_*.go`)。
|
||||
- Migration 通过 `db.AutoMigrate(&User{}, &Dir{}, &Rule{}, &WatchChat{})` 完成,模型定义在 `database/*.go` 中。
|
||||
- **用户同步约定**:
|
||||
- `database.syncUsers` 会根据 `config.C().Users` 同步数据库用户表:在配置中新增/删除用户会自动在 DB 中创建/删除对应记录。
|
||||
- 开发涉及用户表逻辑时,请考虑该同步行为,避免在其他地方直接创建/删除用户记录而与配置冲突。
|
||||
|
||||
## 存储后端
|
||||
- **存储抽象**:
|
||||
- 抽象接口在 `config/storage/types.go` 与 `storage/` 顶层(以及子目录)中;`config/storage/*.go` 处理配置解析,`storage/*` 处理真正的上传/下载实现。
|
||||
- 现有实现包括 `local`、`alist`、`s3/minio`、`webdav`、`telegram` 等,每个后端都有对应子目录和配置结构体。
|
||||
- **新增存储实现的推荐路径**:
|
||||
- 在 `config/storage/` 下添加配置结构体 + `Validate`。
|
||||
- 在 `storage/` 下添加具体实现(例如 `storage/foo/`)。
|
||||
- 在 `pkg/enums/storage` 与 `storageFactories` 中注册,并确保 `storages` 配置示例被更新(`config.example.toml` / 文档)。
|
||||
|
||||
## 解析器插件(JS)
|
||||
- **插件运行时**:
|
||||
- 解析器接口和插件文档在 `plugins/README.md`,Go 端入口为 `parsers/` 目录,使用 `goja` 与 `playwright-go`。
|
||||
- 插件通过 `registerParser({ metadata, canHandle, parse })` 注册,使用 `ghttp`/`playwright` 进行 HTTP/浏览器请求。
|
||||
- **与核心交互约定**:
|
||||
- 插件 `parse` 返回的 `Item`/`Resource` 会被转化为内部任务(通常是下载/转存任务)并进入 `core` 队列。
|
||||
- 修改 `Item`/`Resource` 结构或解析逻辑时,要确保保持向后兼容,或在 `plugins/README.md` 中同步更新字段说明和示例。
|
||||
|
||||
## i18n 与日志
|
||||
- **国际化**:
|
||||
- 所有用户可见字符串(尤其是错误与提示)应使用 `common/i18n`:`i18n.T(i18nk.SomeKey, map[string]any{"Name": name})`。
|
||||
- 语言文件位于 `common/i18n/locale/`,`go:generate` 指令在 `main.go` 中生成 `i18nk/keys.go`;新增文案时需:添加到 YAML、运行 `go generate ./...`、再在代码中引用新 key。
|
||||
- **日志**:
|
||||
- 使用 `github.com/charmbracelet/log`,在 `cmd/run.go::Run` 中通过 `log.WithContext` 将 logger 注入 `context.Context`;后续代码优先通过 `log.FromContext(ctx)` 获取 logger。
|
||||
- 编写新代码时,如已有 `ctx`,请使用 `log.FromContext(ctx)` 而不是全局 logger。
|
||||
|
||||
## 开发与运行
|
||||
- **本地运行**:
|
||||
- 直接运行:`go run ./cmd`(`cmd/root.go` + `cmd/run.go`)。
|
||||
- 或通过 Docker:参见根目录 `README.md` 中的 `docker run ...` 示例及 `docker-compose.yml`。
|
||||
- **代码生成与文档**:
|
||||
- i18n key 生成:`go generate ./...` 会执行 `main.go` 顶部的 `//go:generate`,使用 `cmd/geni18n/main.go` 生成 `common/i18n/i18nk/keys.go`。
|
||||
- 文档站点在 `docs/`(Hugo),通常不需要在核心代码改动时同步修改,除非涉及文档内容。
|
||||
|
||||
---
|
||||
|
||||
如果你在实现某个功能时发现以上规则不够具体(例如某类任务在 `core/tasks` 中到底如何落地,或某个存储/解析器的边界不清晰),请在对应章节下扩展更精确的说明,并在 PR 描述中标注。也欢迎你告诉我有哪些部分需要补充或澄清,我可以进一步细化。
|
||||
5
.gitignore
vendored
5
.gitignore
vendored
@@ -9,4 +9,7 @@ cache.db
|
||||
temp/
|
||||
.hugo_build.lock
|
||||
playwright/
|
||||
testplugins/
|
||||
testplugins/
|
||||
*.exe
|
||||
tmp-*
|
||||
saveany-bot
|
||||
301
AGENTS.md
Normal file
301
AGENTS.md
Normal file
@@ -0,0 +1,301 @@
|
||||
# SaveAny-Bot Agent Guidelines
|
||||
|
||||
This document provides essential information for AI coding agents working on the SaveAny-Bot project.
|
||||
|
||||
## Project Overview
|
||||
|
||||
SaveAny-Bot is a Telegram bot written in Go that saves files/messages from Telegram and various websites to multiple storage backends (local, S3, MinIO, WebDAV, AList, Telegram). It features a plugin system for parsing web content and extensible storage backends.
|
||||
|
||||
**Tech Stack**: Go 1.24.2, gotd/td (Telegram MTProto), Cobra (CLI), Viper (config), GORM (ORM), SQLite, Goja (JS runtime), Playwright (browser automation)
|
||||
|
||||
## Build & Test Commands
|
||||
|
||||
### Build
|
||||
```bash
|
||||
# Standard build
|
||||
go build -o saveany-bot .
|
||||
|
||||
# Run directly
|
||||
go run ./cmd
|
||||
|
||||
# Docker build (multi-stage, Alpine-based)
|
||||
docker build -t saveany-bot .
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
### Test
|
||||
```bash
|
||||
# Run all tests
|
||||
go test ./...
|
||||
|
||||
# Run tests in specific package
|
||||
go test ./pkg/queue
|
||||
go test ./storage/telegram
|
||||
|
||||
# Run tests with verbose output
|
||||
go test -v ./...
|
||||
|
||||
# Run a single test
|
||||
go test -run TestQueueBasic ./pkg/queue
|
||||
|
||||
# Run with coverage
|
||||
go test -cover ./...
|
||||
```
|
||||
|
||||
### Lint & Format
|
||||
```bash
|
||||
# Format code (standard Go formatting)
|
||||
go fmt ./...
|
||||
|
||||
# Vet code for common issues
|
||||
go vet ./...
|
||||
|
||||
# Generate code (i18n keys)
|
||||
go generate ./...
|
||||
```
|
||||
|
||||
### Other Commands
|
||||
```bash
|
||||
# Update dependencies
|
||||
go mod tidy
|
||||
|
||||
# View documentation
|
||||
cd docs && hugo server -D
|
||||
```
|
||||
|
||||
## Code Style Guidelines
|
||||
|
||||
### Imports
|
||||
- Standard library first, then third-party, then project-internal
|
||||
- Group imports with blank lines between groups
|
||||
- Use explicit import aliases for clarity when needed (e.g., `storconfig`, `storenum`)
|
||||
|
||||
```go
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
|
||||
"github.com/charmbracelet/log"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/storage"
|
||||
)
|
||||
```
|
||||
|
||||
### Formatting
|
||||
- Line length: reasonable (no hard limit, but be sensible)
|
||||
- Organize code with blank lines between logical sections
|
||||
- Follow standard Go conventions for braces, spacing, etc.
|
||||
|
||||
### Types & Interfaces
|
||||
- Use clear, descriptive type names (PascalCase for exported, camelCase for unexported)
|
||||
- Define interfaces where abstraction is needed (e.g., `Executable`, `StorageConfig`)
|
||||
- Embed context in method signatures, not structs: `func (s *Service) Do(ctx context.Context) error`
|
||||
- Prefer composition over inheritance
|
||||
|
||||
```go
|
||||
// Interfaces define behavior
|
||||
type Executable interface {
|
||||
Type() tasktype.TaskType
|
||||
Title() string
|
||||
TaskID() string
|
||||
Execute(ctx context.Context) error
|
||||
}
|
||||
|
||||
// Structs compose behavior
|
||||
type Local struct {
|
||||
config config.LocalStorageConfig
|
||||
logger *log.Logger
|
||||
}
|
||||
```
|
||||
|
||||
### Naming Conventions
|
||||
- **Packages**: lowercase, single word when possible (avoid underscores)
|
||||
- **Files**: lowercase with underscores for multiword (e.g., `auth_terminal.go`, `progress_reader.go`)
|
||||
- **Variables**: camelCase for unexported, PascalCase for exported
|
||||
- **Constants**: PascalCase for exported, camelCase for unexported (not ALL_CAPS)
|
||||
- **Functions/Methods**: PascalCase for exported, camelCase for unexported
|
||||
- **Test files**: `*_test.go` pattern
|
||||
|
||||
### Error Handling
|
||||
- Always handle errors explicitly; never ignore them
|
||||
- Wrap errors with context using `fmt.Errorf("context: %w", err)`
|
||||
- Use `errors.Is()` and `errors.As()` for error checking
|
||||
- Log errors with appropriate level (Error, Warn, Info)
|
||||
- Return errors from functions rather than panicking (except for truly unrecoverable situations)
|
||||
|
||||
```go
|
||||
// Good error handling
|
||||
if err := db.Save(user).Error; err != nil {
|
||||
return fmt.Errorf("failed to save user %d: %w", user.ChatID, err)
|
||||
}
|
||||
|
||||
// Check specific errors
|
||||
if errors.Is(err, context.Canceled) {
|
||||
logger.Info("Operation was canceled")
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
### Logging
|
||||
- Use `github.com/charmbracelet/log` package
|
||||
- Get logger from context: `log.FromContext(ctx)`
|
||||
- Create prefixed loggers for components: `logger.WithPrefix("component")`
|
||||
- Use appropriate levels: Debug, Info, Warn, Error
|
||||
- Include context in log messages (e.g., task IDs, file names)
|
||||
|
||||
```go
|
||||
logger := log.FromContext(ctx)
|
||||
logger.Infof("Processing task: %s", task.ID)
|
||||
logger.Errorf("Failed to save file %s: %v", filename, err)
|
||||
```
|
||||
|
||||
### Concurrency
|
||||
- Use channels for communication between goroutines
|
||||
- Protect shared state with `sync.Mutex` or `sync.RWMutex`
|
||||
- Use `sync.WaitGroup` for coordinating goroutine completion
|
||||
- Always pass `context.Context` for cancellation support
|
||||
- Use `context.WithCancel/WithTimeout` for managing goroutine lifetimes
|
||||
|
||||
```go
|
||||
// Example from queue implementation
|
||||
func (tq *TaskQueue[T]) Add(task *Task[T]) error {
|
||||
tq.mu.Lock()
|
||||
defer tq.mu.Unlock()
|
||||
// ... critical section
|
||||
tq.cond.Signal()
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
### Comments
|
||||
- Document exported types, functions, and packages with doc comments
|
||||
- Start doc comments with the name being documented
|
||||
- Use `//` for single-line comments
|
||||
- Explain *why*, not *what* (code should be self-explanatory for "what")
|
||||
- Add `[NOTE]`, `[WARN]`, `[IMPORTANT]` tags for important clarifications
|
||||
|
||||
```go
|
||||
// GetUserByChatID retrieves a user by their Telegram chat ID.
|
||||
// Returns an error if the user is not found.
|
||||
func GetUserByChatID(ctx context.Context, chatID int64) (*User, error) {
|
||||
```
|
||||
|
||||
## Architecture & Conventions
|
||||
|
||||
### Application Structure
|
||||
- **Entry point**: `main.go` → `cmd.Execute(ctx)`
|
||||
- **CLI root**: `cmd/root.go` (Cobra), implementation in `cmd/run.go`
|
||||
- **Startup sequence**: Config → Cache → i18n → Database → Storage → Parsers → Userbot → Bot → Queue
|
||||
- Follow this order when adding new initialization steps in `cmd/run.go::initAll`
|
||||
|
||||
### Configuration (Viper)
|
||||
- Config defined in `config/viper.go::Config`
|
||||
- Read from `config.toml` (see `config.example.toml`)
|
||||
- Environment variables: `SAVEANY_*` prefix (e.g., `SAVEANY_TELEGRAM_TOKEN`)
|
||||
- Access via `config.C()` (returns a copy, don't modify the return value)
|
||||
- Storage configs validated via `config/storage/factory.go::LoadStorageConfigs`
|
||||
|
||||
### Telegram Client
|
||||
- **Bot client**: `client/bot/bot.go::Init` (uses gotgproto)
|
||||
- **Handlers**: Centralized in `client/bot/handlers/` directory
|
||||
- **Registration**: All handlers registered in `handlers.Register`
|
||||
- **Commands**: Add to `CommandHandlers` slice for automatic `/help` and bot command list updates
|
||||
- **Middleware**: Common middleware in `client/middleware/` (floodwait, retry, etc.)
|
||||
|
||||
### Tasks & Queue
|
||||
- **Task interface**: `core/core.go::Executable` (Type, Title, TaskID, Execute methods)
|
||||
- **Queue**: `pkg/queue.TaskQueue[Executable]` (generic, thread-safe)
|
||||
- **Workers**: Count from `config.C().Workers`
|
||||
- **Task types**: Implementations in `core/tasks/**` (tfile, parsed, telegraph, directlinks, batchtfile)
|
||||
- **Lifecycle hooks**: `TaskBeforeStart`, `TaskSuccess`, `TaskFail`, `TaskCancel` (defined in config)
|
||||
- **Adding tasks**: Use `core.AddTask(ctx, task)`
|
||||
|
||||
### Database (GORM + SQLite)
|
||||
- **Init**: `database.Init` using `config.C().DB.Path`
|
||||
- **Models**: User, Dir, Rule, WatchChat (in `database/*.go`)
|
||||
- **Migrations**: Automatic via `db.AutoMigrate`
|
||||
- **User sync**: `database.syncUsers` syncs DB with `config.C().Users` (don't manually create/delete users)
|
||||
- **Context**: Always use `db.WithContext(ctx)` for operations
|
||||
|
||||
### Storage Backends
|
||||
- **Interface**: Defined in `config/storage/types.go` and `storage/`
|
||||
- **Implementations**: local, alist, s3/minio, webdav, telegram (each in subdirectory)
|
||||
- **Adding new storage**:
|
||||
1. Add enum to `pkg/enums/storage`
|
||||
2. Create config struct in `config/storage/` with `Validate()` method
|
||||
3. Implement storage in `storage/<name>/`
|
||||
4. Register in `storageFactories` mapping
|
||||
5. Update `config.example.toml` with example
|
||||
|
||||
### Parser Plugins (JavaScript)
|
||||
- **Runtime**: Goja (JS runtime) + Playwright (browser automation)
|
||||
- **Plugin API**: `registerParser({ metadata, canHandle, parse })` in JS
|
||||
- **Integration**: Defined in `parsers/` directory
|
||||
- **Documentation**: See `plugins/README.md`
|
||||
- Plugin `parse` returns `Item`/`Resource` which becomes download/transfer task
|
||||
|
||||
### Internationalization (i18n)
|
||||
- **Usage**: `i18n.T(i18nk.SomeKey, map[string]any{"Name": value})`
|
||||
- **Locale files**: `common/i18n/locale/*.yaml`
|
||||
- **Key generation**: Run `go generate ./...` to generate `common/i18n/i18nk/keys.go`
|
||||
- **Adding new strings**: Add to YAML → run `go generate` → use in code
|
||||
- All user-facing strings should be internationalized
|
||||
|
||||
### Context Usage
|
||||
- Always pass `context.Context` as first parameter
|
||||
- Use `log.FromContext(ctx)` to get contextual logger
|
||||
- Respect context cancellation in long-running operations
|
||||
- Store request-scoped data in context (e.g., `ctxkey.ContentLength`)
|
||||
|
||||
## Special Rules from .github/copilot-instructions.md
|
||||
|
||||
1. **Never modify `config.C()` return values** - it returns a copy. Modify config in `config.Init` or via Viper.
|
||||
2. **Handlers must update `CommandHandlers` slice** - ensures `/help` and bot commands stay in sync.
|
||||
3. **Task execution must preserve hooks** - don't remove `TaskBeforeStart`, `TaskSuccess`, `TaskFail`, `TaskCancel` hook calls.
|
||||
4. **User sync is automatic** - don't manually create/delete users in DB; use config-based sync.
|
||||
5. **Prefer context logger** - use `log.FromContext(ctx)` over global logger when context is available.
|
||||
6. **Storage factory pattern** - new storage types must register in `storageFactories` mapping.
|
||||
7. **Plugin API compatibility** - changes to `Item`/`Resource` structures require updating `plugins/README.md`.
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### Adding a New Command
|
||||
1. Create handler function in `client/bot/handlers/<name>.go`
|
||||
2. Add to `CommandHandlers` slice in `register.go`
|
||||
3. Add i18n key to `common/i18n/locale/*.yaml`
|
||||
4. Run `go generate ./...`
|
||||
5. Test with Telegram bot
|
||||
|
||||
### Adding a New Task Type
|
||||
1. Create struct implementing `core.Executable` in `core/tasks/<type>/`
|
||||
2. Implement `Type()`, `Title()`, `TaskID()`, `Execute(ctx)` methods
|
||||
3. Add task type enum to `pkg/enums/tasktype`
|
||||
4. Use `core.AddTask(ctx, task)` to enqueue
|
||||
|
||||
### Adding a New Storage Backend
|
||||
1. Define config struct in `config/storage/<name>.go` with `Validate()` method
|
||||
2. Implement storage interface in `storage/<name>/<name>.go`
|
||||
3. Add storage type enum to `pkg/enums/storage`
|
||||
4. Register factory in `config/storage/factory.go::storageFactories`
|
||||
5. Update `config.example.toml` with configuration example
|
||||
|
||||
## File References
|
||||
|
||||
When referencing code locations, use `path/to/file.go:line` format (e.g., `core/core.go:23` for the worker function).
|
||||
|
||||
## Testing Guidelines
|
||||
|
||||
- Write tests for new functionality (place in `*_test.go` files)
|
||||
- Test files should be in same package as code being tested
|
||||
- Use table-driven tests for multiple test cases
|
||||
- Mock external dependencies (databases, network calls)
|
||||
- Aim for meaningful tests, not just coverage numbers
|
||||
|
||||
## Notes
|
||||
|
||||
- Binary size matters: use `CGO_ENABLED=0` for static binaries
|
||||
- FFmpeg is included in Docker images for media processing
|
||||
- Build process supports cross-compilation (amd64/arm64, Linux/macOS/Windows)
|
||||
- Documentation site uses Hugo; edit files in `docs/` directory
|
||||
- Session data stored in SQLite; delete `data/session.db` if changing bot token
|
||||
172
CLAUDE.md
172
CLAUDE.md
@@ -1,172 +0,0 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Project Overview
|
||||
|
||||
SaveAny-Bot is a Telegram bot written in Go that saves files from Telegram and external websites to various storage backends (Local, Alist, S3, WebDAV, Telegram). It supports bypassing "restrict saving content" media, batch downloads, auto-organization, and JS parser plugins for extracting content from websites.
|
||||
|
||||
## Commands
|
||||
|
||||
### Development
|
||||
```bash
|
||||
# Run the bot directly
|
||||
go run ./cmd
|
||||
|
||||
# Run with specific config file
|
||||
go run ./cmd --config /path/to/config.toml
|
||||
|
||||
# Generate i18n keys after modifying locale files
|
||||
go generate ./...
|
||||
|
||||
# Run tests
|
||||
go test ./...
|
||||
|
||||
# Run specific package tests
|
||||
go test ./storage/s3
|
||||
go test ./pkg/queue
|
||||
```
|
||||
|
||||
### Docker
|
||||
```bash
|
||||
# Build and run locally
|
||||
docker-compose -f docker-compose.local.yml up
|
||||
|
||||
# Production deployment
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
### Application Initialization Flow (CRITICAL)
|
||||
|
||||
The application follows a strict initialization sequence in `cmd/run.go::initAll`:
|
||||
|
||||
1. **Config** (`config.Init`) - Load config.toml using viper
|
||||
2. **Cache** (`common/cache`) - Initialize in-memory cache
|
||||
3. **i18n** (`common/i18n`) - Load localization files from `common/i18n/locale/`
|
||||
4. **Database** (`database.Init`) - Connect to SQLite, auto-migrate models
|
||||
5. **Storage** (`storage.LoadStorages`) - Initialize all configured storage backends
|
||||
6. **Parsers** (`parsers.LoadPlugins`) - Load JS parser plugins if enabled
|
||||
7. **Userbot** (optional) - Login to Telegram userbot if configured
|
||||
8. **Bot** (`client/bot.Init`) - Initialize Telegram bot client
|
||||
9. **Core** (`core.Run`) - Start task queue workers
|
||||
|
||||
**When adding new initialization steps, add them to `initAll` in this order.**
|
||||
|
||||
### Core Components
|
||||
|
||||
#### Task Queue System (`core/`)
|
||||
- Central abstraction: `core.Executable` interface with `Type()`, `Title()`, `TaskID()`, `Execute(ctx)`
|
||||
- Task queue: `pkg/queue.TaskQueue[Executable]` managed by `core.Run`
|
||||
- Worker pool size controlled by `config.Workers`
|
||||
- Task implementations in `core/tasks/`:
|
||||
- `tfile/` - Telegram file downloads
|
||||
- `batchtfile/` - Batch downloads
|
||||
- `telegraph/` - Telegraph page saving
|
||||
- `parsed/` - Content from parser plugins
|
||||
- `directlinks/` - Direct URL downloads
|
||||
- Tasks support lifecycle hooks: `TaskBeforeStart`, `TaskSuccess`, `TaskFail`, `TaskCancel` (configured via `config.Hook.Exec`)
|
||||
|
||||
#### Telegram Client (`client/`)
|
||||
- **Bot**: `client/bot/bot.go` uses `gotgproto.NewClient`, handlers in `client/bot/handlers/`
|
||||
- **Userbot**: `client/user/` for bypassing restricted content (optional)
|
||||
- **Middleware**: `client/middleware/` handles flood wait, recovery, retry
|
||||
- Handler registration: All handlers registered in `handlers.Register()`, command list in `handlers.CommandHandlers`
|
||||
|
||||
#### Storage Backends (`storage/`)
|
||||
- Abstract interface: `Storage` with `Init()`, `Save()`, `Exists()`, `JoinStoragePath()`
|
||||
- Implementations: `local/`, `alist/`, `s3/`, `minio/`, `webdav/`, `telegram/`
|
||||
- Storage registry: `storage.Storages` map, constructed via `storageConstructors`
|
||||
- Add new storage: Define enum in `pkg/enums/storage`, config in `config/storage/`, implementation in `storage/`, register in `storageConstructors`
|
||||
|
||||
#### Parser Plugins (`parsers/`)
|
||||
- Native parsers: `parsers/native/twitter/`, `parsers/native/kemono/`
|
||||
- JS plugin runtime: `parsers/js/` using `goja` VM and `playwright-go`
|
||||
- Plugin interface: `registerParser({ metadata, canHandle, parse })` in JS
|
||||
- Plugins return `parser.Item` with `Resources[]`, converted to `core/tasks/parsed` tasks
|
||||
- Plugin docs: `plugins/README.md`
|
||||
|
||||
#### Configuration (`config/`)
|
||||
- Main config: `config/viper.go::Config` loaded via `viper`
|
||||
- Storage configs: `config/storage/factory.go::LoadStorageConfigs` with type-specific validation
|
||||
- Environment override: Prefix `SAVEANY_`, dots become underscores (e.g., `SAVEANY_TELEGRAM_TOKEN`)
|
||||
- **Important**: `config.C()` returns a copy, don't modify its fields; modify via `viper` in `config.Init`
|
||||
|
||||
#### Database (`database/`)
|
||||
- SQLite with GORM
|
||||
- Models: `User`, `Dir`, `Rule`, `WatchChat`
|
||||
- User sync: `database.syncUsers` syncs `config.Users` to DB (creates/deletes users based on config)
|
||||
- Don't create/delete users manually; use config file
|
||||
|
||||
#### Internationalization (`common/i18n/`)
|
||||
- Locale files: `common/i18n/locale/*.yaml`
|
||||
- Key generation: `go generate ./...` runs `cmd/geni18n/main.go` to generate `common/i18n/i18nk/keys.go`
|
||||
- Usage: `i18n.T(i18nk.KeyName, map[string]any{"Param": value})`
|
||||
- Add new strings: Edit YAML → run `go generate` → use new key in code
|
||||
|
||||
#### Logging
|
||||
- Uses `github.com/charmbracelet/log`
|
||||
- Logger injected into context in `cmd/run.go::Run` via `log.WithContext`
|
||||
- Prefer `log.FromContext(ctx)` over global logger when context is available
|
||||
|
||||
### Data Flow
|
||||
|
||||
1. User sends message/file to Telegram bot
|
||||
2. Handler in `client/bot/handlers/` processes update
|
||||
3. Handler creates task (implements `core.Executable`) and calls `core.AddTask()`
|
||||
4. Task enters `pkg/queue.TaskQueue`
|
||||
5. Worker goroutine picks up task, calls `Execute(ctx)`
|
||||
6. Task downloads content (from Telegram or parser) and saves to storage backend via `Storage.Save()`
|
||||
7. Task completion triggers hooks, sends result message back to user
|
||||
|
||||
For parsed content (URLs):
|
||||
1. Handler extracts URL, calls `parsers.ParseWithContext()`
|
||||
2. Parser (native or JS plugin) returns `parser.Item` with `Resources[]`
|
||||
3. `core/tasks/parsed.Task` created with resources
|
||||
4. Task downloads each resource and saves to storage
|
||||
|
||||
## Project-Specific Guidelines
|
||||
|
||||
### Adding New Storage Backend
|
||||
|
||||
1. Define enum in `pkg/enums/storage/storage.go`
|
||||
2. Create config struct in `config/storage/yourtype.go` implementing `StorageConfig` with `Validate()`
|
||||
3. Implement storage in `storage/yourtype/yourtype.go` with `Storage` interface
|
||||
4. Register in `config/storage/factory.go::storageFactories` and `storage/storage.go::storageConstructors`
|
||||
5. Update `config.example.toml` with example configuration
|
||||
|
||||
### Adding New Task Type
|
||||
|
||||
1. Create package in `core/tasks/yourtype/`
|
||||
2. Define struct implementing `core.Executable` interface
|
||||
3. Implement `Type()`, `Title()`, `TaskID()`, `Execute(ctx)`
|
||||
4. Add task type enum in `pkg/enums/tasktype/`
|
||||
5. Create task from handler and call `core.AddTask(ctx, task)`
|
||||
|
||||
### Adding New Bot Command
|
||||
|
||||
1. Create handler file in `client/bot/handlers/yourcommand.go`
|
||||
2. Implement handler function with signature matching gotgproto patterns
|
||||
3. Register in `handlers.Register()` function
|
||||
4. Add to `handlers.CommandHandlers` slice for command list
|
||||
5. Add i18n strings for command description and responses
|
||||
|
||||
### Adding i18n Strings
|
||||
|
||||
1. Edit locale files in `common/i18n/locale/` (e.g., `zh-Hans.yaml`, `en.yaml`)
|
||||
2. Run `go generate ./...` to generate `common/i18n/i18nk/keys.go`
|
||||
3. Use in code: `i18n.T(i18nk.YourNewKey, map[string]any{"Variable": value})`
|
||||
|
||||
### Working with Middleware
|
||||
|
||||
New cross-cutting concerns (logging, metrics, rate limiting) should be implemented as middleware in `client/middleware/`:
|
||||
1. Create middleware file with function signature matching gotgproto middleware
|
||||
2. Add to `middleware.NewDefaultMiddlewares()` in `client/bot/bot.go`
|
||||
|
||||
### Parser Plugin Development
|
||||
|
||||
- JS plugins go in directories specified by `config.parser.plugin_dirs` (default: `./plugins/`)
|
||||
- See `plugins/README.md` for plugin API documentation
|
||||
- Plugins have access to `ghttp` for HTTP requests and `playwright` for browser automation
|
||||
- Test plugins in `./testplugins/` directory
|
||||
@@ -26,7 +26,7 @@ RUN --mount=type=cache,target=/root/.cache/go-build \
|
||||
|
||||
FROM alpine:latest
|
||||
|
||||
RUN apk add --no-cache curl ffmpeg
|
||||
RUN apk add --no-cache curl ffmpeg yt-dlp
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
|
||||
@@ -26,12 +26,16 @@
|
||||
- Multi-user support
|
||||
- Auto organize files based on storage rules
|
||||
- Watch specified chats and auto-save messages, with filters
|
||||
- Transfer files between different storage backends
|
||||
- Integrate with yt-dlp to download and save media from 1000+ websites
|
||||
- Aria2 integration to download files from URLs/magnets and save to storages
|
||||
- Write JS parser plugins to save files from almost any website
|
||||
- Storage backends:
|
||||
- Alist
|
||||
- S3
|
||||
- WebDAV
|
||||
- Local filesystem
|
||||
- Rclone (via command line)
|
||||
- Telegram (re-upload to specified chats)
|
||||
|
||||
## 📦 Quick Start
|
||||
@@ -39,10 +43,11 @@
|
||||
Create a `config.toml` file with the following content:
|
||||
|
||||
```toml
|
||||
lang = "en" # Language setting, "en" for English
|
||||
[telegram]
|
||||
token = "" # Your bot token, obtained from @BotFather
|
||||
[telegram.proxy]
|
||||
# Enable proxy for Telegram, currently only SOCKS5 is supported
|
||||
# Enable proxy for Telegram
|
||||
enable = false
|
||||
url = "socks5://127.0.0.1:7890"
|
||||
|
||||
|
||||
@@ -24,12 +24,16 @@
|
||||
- 多用户使用
|
||||
- 基于存储规则的自动整理
|
||||
- 监听并自动转存指定聊天的消息, 支持过滤
|
||||
- 在不同存储端之间转存文件
|
||||
- 集成 yt-dlp, 从所支持的网站下载并转存媒体文件
|
||||
- 集成 Aria2, 支持直链/磁力下载和转存
|
||||
- 使用 js 编写解析器插件以转存任意网站的文件
|
||||
- 存储端支持:
|
||||
- Alist
|
||||
- S3
|
||||
- WebDAV
|
||||
- 本地磁盘
|
||||
- Rclone
|
||||
- Telegram (重传回指定聊天)
|
||||
|
||||
## 快速开始
|
||||
@@ -40,7 +44,7 @@
|
||||
[telegram]
|
||||
token = "" # 你的 Bot Token, 在 @BotFather 获取
|
||||
[telegram.proxy]
|
||||
# 启用代理连接 telegram, 当前只支持 socks5
|
||||
# 启用代理连接 telegram
|
||||
enable = false
|
||||
url = "socks5://127.0.0.1:7890"
|
||||
|
||||
|
||||
48
api/auth.go
Normal file
48
api/auth.go
Normal file
@@ -0,0 +1,48 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/subtle"
|
||||
"net/http"
|
||||
"strings"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
)
|
||||
|
||||
// tokenContextKey 用于在 context 中存储 token
|
||||
type tokenContextKey struct{}
|
||||
|
||||
// AuthMiddleware 返回认证中间件
|
||||
func AuthMiddleware() func(http.Handler) http.Handler {
|
||||
return func(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
cfg := config.C().API
|
||||
|
||||
// 从请求头获取 token
|
||||
authHeader := r.Header.Get("Authorization")
|
||||
if authHeader == "" {
|
||||
WriteError(w, http.StatusUnauthorized, "unauthorized", "missing authorization header")
|
||||
return
|
||||
}
|
||||
|
||||
// 提取 Bearer token
|
||||
parts := strings.SplitN(authHeader, " ", 2)
|
||||
if len(parts) != 2 || strings.ToLower(parts[0]) != "bearer" {
|
||||
WriteError(w, http.StatusUnauthorized, "unauthorized", "invalid authorization header format")
|
||||
return
|
||||
}
|
||||
|
||||
token := parts[1]
|
||||
|
||||
// 验证 token
|
||||
if subtle.ConstantTimeCompare([]byte(token), []byte(cfg.Token)) != 1 {
|
||||
WriteError(w, http.StatusUnauthorized, "unauthorized", "invalid token")
|
||||
return
|
||||
}
|
||||
|
||||
// 将 token 添加到 context
|
||||
ctx := context.WithValue(r.Context(), tokenContextKey{}, token)
|
||||
next.ServeHTTP(w, r.WithContext(ctx))
|
||||
})
|
||||
}
|
||||
}
|
||||
355
api/factory.go
Normal file
355
api/factory.go
Normal file
@@ -0,0 +1,355 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
"github.com/krau/SaveAny-Bot/core"
|
||||
"github.com/krau/SaveAny-Bot/core/tasks/aria2dl"
|
||||
"github.com/krau/SaveAny-Bot/core/tasks/batchtfile"
|
||||
"github.com/krau/SaveAny-Bot/core/tasks/directlinks"
|
||||
"github.com/krau/SaveAny-Bot/core/tasks/parsed"
|
||||
tphtask "github.com/krau/SaveAny-Bot/core/tasks/telegraph"
|
||||
"github.com/krau/SaveAny-Bot/core/tasks/tfile"
|
||||
"github.com/krau/SaveAny-Bot/core/tasks/transfer"
|
||||
"github.com/krau/SaveAny-Bot/core/tasks/ytdlp"
|
||||
"github.com/krau/SaveAny-Bot/parsers/parsers"
|
||||
"github.com/krau/SaveAny-Bot/pkg/aria2"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
|
||||
"github.com/krau/SaveAny-Bot/pkg/parser"
|
||||
"github.com/krau/SaveAny-Bot/pkg/telegraph"
|
||||
"github.com/krau/SaveAny-Bot/storage"
|
||||
"github.com/rs/xid"
|
||||
)
|
||||
|
||||
// TaskFactory 任务工厂
|
||||
type TaskFactory struct {
|
||||
ctx context.Context
|
||||
}
|
||||
|
||||
// NewTaskFactory 创建任务工厂
|
||||
func NewTaskFactory(ctx context.Context) *TaskFactory {
|
||||
return &TaskFactory{ctx: ctx}
|
||||
}
|
||||
|
||||
// CreateTask 创建任务
|
||||
func (f *TaskFactory) CreateTask(req *CreateTaskRequest) (*CreateTaskResponse, error) {
|
||||
// 验证存储
|
||||
stor, ok := storage.Storages[req.Storage]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("storage not found: %s", req.Storage)
|
||||
}
|
||||
|
||||
taskID := xid.New().String()
|
||||
createdAt := time.Now()
|
||||
|
||||
switch req.Type {
|
||||
case tasktype.TaskTypeDirectlinks:
|
||||
return f.createDirectLinksTask(taskID, createdAt, req, stor)
|
||||
case tasktype.TaskTypeYtdlp:
|
||||
return f.createYTDLPTask(taskID, createdAt, req, stor)
|
||||
case tasktype.TaskTypeAria2:
|
||||
return f.createAria2Task(taskID, createdAt, req, stor)
|
||||
case tasktype.TaskTypeParseditem:
|
||||
return f.createParsedTask(taskID, createdAt, req, stor)
|
||||
case tasktype.TaskTypeTgfiles:
|
||||
return f.createTGFilesTask(taskID, createdAt, req, stor)
|
||||
case tasktype.TaskTypeTphpics:
|
||||
return f.createTPHPicsTask(taskID, createdAt, req, stor)
|
||||
case tasktype.TaskTypeTransfer:
|
||||
return f.createTransferTask(taskID, createdAt, req)
|
||||
default:
|
||||
return nil, fmt.Errorf("unsupported task type: %s", req.Type)
|
||||
}
|
||||
}
|
||||
|
||||
// createDirectLinksTask 创建直链下载任务
|
||||
func (f *TaskFactory) createDirectLinksTask(taskID string, createdAt time.Time, req *CreateTaskRequest, stor storage.Storage) (*CreateTaskResponse, error) {
|
||||
var params DirectLinksParams
|
||||
if err := json.Unmarshal(req.Params, ¶ms); err != nil {
|
||||
return nil, fmt.Errorf("invalid params: %w", err)
|
||||
}
|
||||
|
||||
if len(params.URLs) == 0 {
|
||||
return nil, fmt.Errorf("no URLs provided")
|
||||
}
|
||||
|
||||
task := directlinks.NewTask(taskID, f.ctx, params.URLs, stor, req.Path, nil)
|
||||
|
||||
if err := core.AddTask(f.ctx, task); err != nil {
|
||||
return nil, fmt.Errorf("failed to add task: %w", err)
|
||||
}
|
||||
|
||||
return &CreateTaskResponse{
|
||||
TaskID: taskID,
|
||||
Type: tasktype.TaskTypeDirectlinks,
|
||||
Status: TaskStatusQueued,
|
||||
CreatedAt: createdAt,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// createYTDLPTask 创建 yt-dlp 任务
|
||||
func (f *TaskFactory) createYTDLPTask(taskID string, createdAt time.Time, req *CreateTaskRequest, stor storage.Storage) (*CreateTaskResponse, error) {
|
||||
var params YTDLPParams
|
||||
if err := json.Unmarshal(req.Params, ¶ms); err != nil {
|
||||
return nil, fmt.Errorf("invalid params: %w", err)
|
||||
}
|
||||
|
||||
if len(params.URLs) == 0 {
|
||||
return nil, fmt.Errorf("no URLs provided")
|
||||
}
|
||||
|
||||
task := ytdlp.NewTask(taskID, f.ctx, params.URLs, params.Flags, stor, req.Path, nil)
|
||||
|
||||
if err := core.AddTask(f.ctx, task); err != nil {
|
||||
return nil, fmt.Errorf("failed to add task: %w", err)
|
||||
}
|
||||
|
||||
return &CreateTaskResponse{
|
||||
TaskID: taskID,
|
||||
Type: tasktype.TaskTypeYtdlp,
|
||||
Status: TaskStatusQueued,
|
||||
CreatedAt: createdAt,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// createAria2Task 创建 Aria2 任务
|
||||
func (f *TaskFactory) createAria2Task(taskID string, createdAt time.Time, req *CreateTaskRequest, stor storage.Storage) (*CreateTaskResponse, error) {
|
||||
var params Aria2Params
|
||||
if err := json.Unmarshal(req.Params, ¶ms); err != nil {
|
||||
return nil, fmt.Errorf("invalid params: %w", err)
|
||||
}
|
||||
|
||||
if len(params.URLs) == 0 {
|
||||
return nil, fmt.Errorf("no URLs provided")
|
||||
}
|
||||
|
||||
// 检查 Aria2 是否启用
|
||||
cfg := config.C().Aria2
|
||||
if !cfg.Enable {
|
||||
return nil, fmt.Errorf("aria2 is not enabled")
|
||||
}
|
||||
|
||||
aria2Client, err := aria2.NewClient(cfg.Url, cfg.Secret)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to create aria2 client: %w", err)
|
||||
}
|
||||
|
||||
// 添加下载任务到 Aria2
|
||||
gid, err := aria2Client.AddURI(f.ctx, params.URLs, nil)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to add aria2 task: %w", err)
|
||||
}
|
||||
|
||||
task := aria2dl.NewTask(taskID, f.ctx, gid, params.URLs, aria2Client, stor, req.Path, nil)
|
||||
|
||||
if err := core.AddTask(f.ctx, task); err != nil {
|
||||
return nil, fmt.Errorf("failed to add task: %w", err)
|
||||
}
|
||||
|
||||
return &CreateTaskResponse{
|
||||
TaskID: taskID,
|
||||
Type: tasktype.TaskTypeAria2,
|
||||
Status: TaskStatusQueued,
|
||||
CreatedAt: createdAt,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// createParsedTask 创建解析任务
|
||||
func (f *TaskFactory) createParsedTask(taskID string, createdAt time.Time, req *CreateTaskRequest, stor storage.Storage) (*CreateTaskResponse, error) {
|
||||
var params ParsedParams
|
||||
if err := json.Unmarshal(req.Params, ¶ms); err != nil {
|
||||
return nil, fmt.Errorf("invalid params: %w", err)
|
||||
}
|
||||
|
||||
if params.URL == "" {
|
||||
return nil, fmt.Errorf("no URL provided")
|
||||
}
|
||||
|
||||
// 查找合适的解析器
|
||||
var p parser.Parser
|
||||
for _, parserItem := range parsers.Get() {
|
||||
if parserItem.CanHandle(params.URL) {
|
||||
p = parserItem
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
if p == nil {
|
||||
return nil, fmt.Errorf("no parser found for URL: %s", params.URL)
|
||||
}
|
||||
|
||||
// 解析 URL
|
||||
item, err := p.Parse(f.ctx, params.URL)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to parse URL: %w", err)
|
||||
}
|
||||
|
||||
task := parsed.NewTask(taskID, f.ctx, stor, req.Path, item, nil)
|
||||
|
||||
if err := core.AddTask(f.ctx, task); err != nil {
|
||||
return nil, fmt.Errorf("failed to add task: %w", err)
|
||||
}
|
||||
|
||||
return &CreateTaskResponse{
|
||||
TaskID: taskID,
|
||||
Type: tasktype.TaskTypeParseditem,
|
||||
Status: TaskStatusQueued,
|
||||
CreatedAt: createdAt,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// createTGFilesTask 创建 Telegram 文件下载任务
|
||||
func (f *TaskFactory) createTGFilesTask(taskID string, createdAt time.Time, req *CreateTaskRequest, stor storage.Storage) (*CreateTaskResponse, error) {
|
||||
var params TGFilesParams
|
||||
if err := json.Unmarshal(req.Params, ¶ms); err != nil {
|
||||
return nil, fmt.Errorf("invalid params: %w", err)
|
||||
}
|
||||
|
||||
if len(params.MessageLinks) == 0 {
|
||||
return nil, fmt.Errorf("no message links provided")
|
||||
}
|
||||
|
||||
// 提取文件
|
||||
files, err := ExtractFilesFromLinks(f.ctx, params.MessageLinks)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to extract files: %w", err)
|
||||
}
|
||||
|
||||
if len(files) == 0 {
|
||||
return nil, fmt.Errorf("no files found in provided links")
|
||||
}
|
||||
|
||||
if len(files) == 1 {
|
||||
// 单个文件任务
|
||||
tfileTask, err := tfile.NewTGFileTask(taskID, f.ctx, files[0], stor, req.Path, nil)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to create tfile task: %w", err)
|
||||
}
|
||||
if err := core.AddTask(f.ctx, tfileTask); err != nil {
|
||||
return nil, fmt.Errorf("failed to add task: %w", err)
|
||||
}
|
||||
} else {
|
||||
// 批量文件任务
|
||||
elems := make([]batchtfile.TaskElement, 0, len(files))
|
||||
for _, file := range files {
|
||||
elem, err := batchtfile.NewTaskElement(stor, req.Path, file)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to create task element: %w", err)
|
||||
}
|
||||
elems = append(elems, *elem)
|
||||
}
|
||||
|
||||
task := batchtfile.NewBatchTGFileTask(taskID, f.ctx, elems, nil, true)
|
||||
if err := core.AddTask(f.ctx, task); err != nil {
|
||||
return nil, fmt.Errorf("failed to add task: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
return &CreateTaskResponse{
|
||||
TaskID: taskID,
|
||||
Type: tasktype.TaskTypeTgfiles,
|
||||
Status: TaskStatusQueued,
|
||||
CreatedAt: createdAt,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// createTPHPicsTask 创建 Telegraph 图片下载任务
|
||||
func (f *TaskFactory) createTPHPicsTask(taskID string, createdAt time.Time, req *CreateTaskRequest, stor storage.Storage) (*CreateTaskResponse, error) {
|
||||
var params TPHPicsParams
|
||||
if err := json.Unmarshal(req.Params, ¶ms); err != nil {
|
||||
return nil, fmt.Errorf("invalid params: %w", err)
|
||||
}
|
||||
|
||||
if params.TelegraphURL == "" {
|
||||
return nil, fmt.Errorf("no telegraph URL provided")
|
||||
}
|
||||
|
||||
// 提取图片
|
||||
pics, phPath, err := ExtractTelegraphImages(f.ctx, params.TelegraphURL)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to extract telegraph images: %w", err)
|
||||
}
|
||||
|
||||
if len(pics) == 0 {
|
||||
return nil, fmt.Errorf("no images found in telegraph page")
|
||||
}
|
||||
|
||||
client := telegraph.NewClient()
|
||||
task := tphtask.NewTask(taskID, f.ctx, phPath, pics, stor, req.Path, client, nil)
|
||||
|
||||
if err := core.AddTask(f.ctx, task); err != nil {
|
||||
return nil, fmt.Errorf("failed to add task: %w", err)
|
||||
}
|
||||
|
||||
return &CreateTaskResponse{
|
||||
TaskID: taskID,
|
||||
Type: tasktype.TaskTypeTphpics,
|
||||
Status: TaskStatusQueued,
|
||||
CreatedAt: createdAt,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// createTransferTask 创建存储间传输任务
|
||||
func (f *TaskFactory) createTransferTask(taskID string, createdAt time.Time, req *CreateTaskRequest) (*CreateTaskResponse, error) {
|
||||
var params TransferParams
|
||||
if err := json.Unmarshal(req.Params, ¶ms); err != nil {
|
||||
return nil, fmt.Errorf("invalid params: %w", err)
|
||||
}
|
||||
|
||||
// 验证源存储和目标存储
|
||||
sourceStor, ok := storage.Storages[params.SourceStorage]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("source storage not found: %s", params.SourceStorage)
|
||||
}
|
||||
|
||||
targetStor, ok := storage.Storages[params.TargetStorage]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("target storage not found: %s", params.TargetStorage)
|
||||
}
|
||||
|
||||
// 检查源存储是否可读
|
||||
sourceReadable, ok := sourceStor.(storage.StorageReadable)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("source storage does not support reading: %s", params.SourceStorage)
|
||||
}
|
||||
|
||||
// 检查源存储是否可列
|
||||
sourceListable, ok := sourceStor.(storage.StorageListable)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("source storage does not support listing: %s", params.SourceStorage)
|
||||
}
|
||||
|
||||
// 列出源文件
|
||||
files, err := sourceListable.ListFiles(f.ctx, params.SourcePath)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to list source files: %w", err)
|
||||
}
|
||||
|
||||
if len(files) == 0 {
|
||||
return nil, fmt.Errorf("no files found at source path: %s", params.SourcePath)
|
||||
}
|
||||
|
||||
// 创建传输元素
|
||||
elems := make([]transfer.TaskElement, 0, len(files))
|
||||
for _, file := range files {
|
||||
elem := transfer.NewTaskElement(sourceReadable, file, targetStor, params.TargetPath)
|
||||
elems = append(elems, *elem)
|
||||
}
|
||||
|
||||
task := transfer.NewTransferTask(taskID, f.ctx, elems, nil, true)
|
||||
|
||||
if err := core.AddTask(f.ctx, task); err != nil {
|
||||
return nil, fmt.Errorf("failed to add task: %w", err)
|
||||
}
|
||||
|
||||
return &CreateTaskResponse{
|
||||
TaskID: taskID,
|
||||
Type: tasktype.TaskTypeTransfer,
|
||||
Status: TaskStatusQueued,
|
||||
CreatedAt: createdAt,
|
||||
}, nil
|
||||
}
|
||||
222
api/handlers.go
Normal file
222
api/handlers.go
Normal file
@@ -0,0 +1,222 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"strings"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/core"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
|
||||
"github.com/krau/SaveAny-Bot/storage"
|
||||
)
|
||||
|
||||
// Handlers 处理器结构体
|
||||
type Handlers struct {
|
||||
factory *TaskFactory
|
||||
}
|
||||
|
||||
// NewHandlers 创建处理器
|
||||
func NewHandlers(factory *TaskFactory) *Handlers {
|
||||
return &Handlers{factory: factory}
|
||||
}
|
||||
|
||||
// CreateTaskHandler 创建任务处理器
|
||||
func (h *Handlers) CreateTaskHandler(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodPost {
|
||||
WriteError(w, http.StatusMethodNotAllowed, "method_not_allowed", "only POST method is allowed")
|
||||
return
|
||||
}
|
||||
|
||||
var req CreateTaskRequest
|
||||
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
||||
WriteError(w, http.StatusBadRequest, "invalid_request", "failed to decode request body: "+err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
// 验证请求
|
||||
if req.Type == "" {
|
||||
WriteError(w, http.StatusBadRequest, "invalid_request", "task type is required")
|
||||
return
|
||||
}
|
||||
|
||||
if req.Storage == "" {
|
||||
WriteError(w, http.StatusBadRequest, "invalid_request", "storage is required")
|
||||
return
|
||||
}
|
||||
|
||||
// 创建任务
|
||||
resp, err := h.factory.CreateTask(&req)
|
||||
if err != nil {
|
||||
WriteError(w, http.StatusBadRequest, "task_creation_failed", err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
WriteJSON(w, http.StatusCreated, resp)
|
||||
}
|
||||
|
||||
// ListTasksHandler 列出任务处理器
|
||||
func (h *Handlers) ListTasksHandler(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodGet {
|
||||
WriteError(w, http.StatusMethodNotAllowed, "method_not_allowed", "only GET method is allowed")
|
||||
return
|
||||
}
|
||||
|
||||
tasks := GetAllTasks()
|
||||
response := make([]TaskInfoResponse, 0, len(tasks))
|
||||
|
||||
for _, task := range tasks {
|
||||
info := convertTaskProgressToResponse(task)
|
||||
response = append(response, info)
|
||||
}
|
||||
|
||||
WriteJSON(w, http.StatusOK, TasksListResponse{
|
||||
Tasks: response,
|
||||
Total: len(response),
|
||||
})
|
||||
}
|
||||
|
||||
// GetTaskHandler 获取单个任务处理器
|
||||
func (h *Handlers) GetTaskHandler(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodGet {
|
||||
WriteError(w, http.StatusMethodNotAllowed, "method_not_allowed", "only GET method is allowed")
|
||||
return
|
||||
}
|
||||
|
||||
taskID := extractTaskIDFromPath(r.URL.Path)
|
||||
if taskID == "" {
|
||||
WriteError(w, http.StatusBadRequest, "invalid_request", "task ID is required")
|
||||
return
|
||||
}
|
||||
|
||||
task, ok := GetTask(taskID)
|
||||
if !ok {
|
||||
WriteError(w, http.StatusNotFound, "task_not_found", "task not found: "+taskID)
|
||||
return
|
||||
}
|
||||
|
||||
resp := convertTaskProgressToResponse(task)
|
||||
WriteJSON(w, http.StatusOK, resp)
|
||||
}
|
||||
|
||||
// CancelTaskHandler 取消任务处理器
|
||||
func (h *Handlers) CancelTaskHandler(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodDelete {
|
||||
WriteError(w, http.StatusMethodNotAllowed, "method_not_allowed", "only DELETE method is allowed")
|
||||
return
|
||||
}
|
||||
|
||||
taskID := extractTaskIDFromPath(r.URL.Path)
|
||||
if taskID == "" {
|
||||
WriteError(w, http.StatusBadRequest, "invalid_request", "task ID is required")
|
||||
return
|
||||
}
|
||||
|
||||
task, ok := GetTask(taskID)
|
||||
if !ok {
|
||||
WriteError(w, http.StatusNotFound, "task_not_found", "task not found: "+taskID)
|
||||
return
|
||||
}
|
||||
|
||||
// 取消任务
|
||||
if err := core.CancelTask(r.Context(), taskID); err != nil {
|
||||
WriteError(w, http.StatusInternalServerError, "cancel_failed", "failed to cancel task: "+err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
task.UpdateStatus(TaskStatusCancelled)
|
||||
WriteJSON(w, http.StatusOK, map[string]string{"message": "task cancelled successfully"})
|
||||
}
|
||||
|
||||
// ListStoragesHandler 列出存储处理器
|
||||
func (h *Handlers) ListStoragesHandler(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodGet {
|
||||
WriteError(w, http.StatusMethodNotAllowed, "method_not_allowed", "only GET method is allowed")
|
||||
return
|
||||
}
|
||||
|
||||
storages := make([]StorageInfo, 0, len(storage.Storages))
|
||||
for name, stor := range storage.Storages {
|
||||
storages = append(storages, StorageInfo{
|
||||
Name: name,
|
||||
Type: string(stor.Type()),
|
||||
})
|
||||
}
|
||||
|
||||
WriteJSON(w, http.StatusOK, StoragesResponse{Storages: storages})
|
||||
}
|
||||
|
||||
// GetTaskTypesHandler 获取支持的任务类型
|
||||
func (h *Handlers) GetTaskTypesHandler(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodGet {
|
||||
WriteError(w, http.StatusMethodNotAllowed, "method_not_allowed", "only GET method is allowed")
|
||||
return
|
||||
}
|
||||
|
||||
types := []tasktype.TaskType{
|
||||
tasktype.TaskTypeDirectlinks,
|
||||
tasktype.TaskTypeYtdlp,
|
||||
tasktype.TaskTypeAria2,
|
||||
tasktype.TaskTypeParseditem,
|
||||
tasktype.TaskTypeTgfiles,
|
||||
tasktype.TaskTypeTphpics,
|
||||
tasktype.TaskTypeTransfer,
|
||||
}
|
||||
|
||||
WriteJSON(w, http.StatusOK, map[string]any{
|
||||
"types": types,
|
||||
})
|
||||
}
|
||||
|
||||
// HealthCheckHandler 健康检查处理器
|
||||
func (h *Handlers) HealthCheckHandler(w http.ResponseWriter, r *http.Request) {
|
||||
WriteJSON(w, http.StatusOK, map[string]string{
|
||||
"status": "ok",
|
||||
})
|
||||
}
|
||||
|
||||
// extractTaskIDFromPath 从路径中提取任务 ID
|
||||
// 路径格式: /api/v1/tasks/:id
|
||||
func extractTaskIDFromPath(path string) string {
|
||||
parts := strings.Split(strings.Trim(path, "/"), "/")
|
||||
if len(parts) < 4 {
|
||||
return ""
|
||||
}
|
||||
return parts[3]
|
||||
}
|
||||
|
||||
// convertTaskProgressToResponse 将任务进度转换为响应格式
|
||||
func convertTaskProgressToResponse(task *TaskProgressInfo) TaskInfoResponse {
|
||||
resp := TaskInfoResponse{
|
||||
TaskID: task.TaskID,
|
||||
Type: tasktype.TaskType(task.Type),
|
||||
Status: task.Status,
|
||||
Title: task.Title,
|
||||
Storage: task.Storage,
|
||||
Path: task.Path,
|
||||
Error: task.Error,
|
||||
CreatedAt: task.CreatedAt,
|
||||
UpdatedAt: task.UpdatedAt,
|
||||
}
|
||||
|
||||
// 计算进度
|
||||
if task.TotalBytes > 0 {
|
||||
percent := float64(task.DownloadedBytes) * 100 / float64(task.TotalBytes)
|
||||
resp.Progress = &TaskProgress{
|
||||
TotalBytes: task.TotalBytes,
|
||||
DownloadedBytes: task.DownloadedBytes,
|
||||
Percent: percent,
|
||||
}
|
||||
}
|
||||
|
||||
return resp
|
||||
}
|
||||
|
||||
// NotFoundHandler 404 处理器
|
||||
func NotFoundHandler(w http.ResponseWriter, r *http.Request) {
|
||||
WriteError(w, http.StatusNotFound, "not_found", "endpoint not found: "+r.URL.Path)
|
||||
}
|
||||
|
||||
// MethodNotAllowedHandler 405 处理器
|
||||
func MethodNotAllowedHandler(w http.ResponseWriter, r *http.Request) {
|
||||
WriteError(w, http.StatusMethodNotAllowed, "method_not_allowed", "method not allowed: "+r.Method)
|
||||
}
|
||||
689
api/handlers_test.go
Normal file
689
api/handlers_test.go
Normal file
@@ -0,0 +1,689 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
|
||||
)
|
||||
|
||||
// setupTestServer creates a test server with handlers
|
||||
func setupTestServer(t *testing.T) (*Handlers, *TaskFactory) {
|
||||
factory := NewTaskFactory(t.Context())
|
||||
handlers := NewHandlers(factory)
|
||||
return handlers, factory
|
||||
}
|
||||
|
||||
// TestCreateTaskHandler tests the create task endpoint
|
||||
func TestCreateTaskHandler(t *testing.T) {
|
||||
handlers, _ := setupTestServer(t)
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
method string
|
||||
body any
|
||||
wantStatus int
|
||||
wantErr bool
|
||||
}{
|
||||
{
|
||||
name: "Method not allowed",
|
||||
method: http.MethodGet,
|
||||
body: nil,
|
||||
wantStatus: http.StatusMethodNotAllowed,
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "Invalid JSON body",
|
||||
method: http.MethodPost,
|
||||
body: "invalid json",
|
||||
wantStatus: http.StatusBadRequest,
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "Empty request body",
|
||||
method: http.MethodPost,
|
||||
body: CreateTaskRequest{},
|
||||
// Will fail validation for missing type
|
||||
wantStatus: http.StatusBadRequest,
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "Missing type",
|
||||
method: http.MethodPost,
|
||||
body: CreateTaskRequest{
|
||||
Storage: "test-storage",
|
||||
Path: "downloads",
|
||||
},
|
||||
wantStatus: http.StatusBadRequest,
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "Missing storage",
|
||||
method: http.MethodPost,
|
||||
body: CreateTaskRequest{
|
||||
Type: tasktype.TaskTypeDirectlinks,
|
||||
Path: "downloads",
|
||||
},
|
||||
wantStatus: http.StatusBadRequest,
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "Storage not found",
|
||||
method: http.MethodPost,
|
||||
body: CreateTaskRequest{
|
||||
Type: tasktype.TaskTypeDirectlinks,
|
||||
Storage: "non-existent-storage",
|
||||
Path: "downloads",
|
||||
Params: json.RawMessage(`{"urls":["https://example.com/file.zip"]}`),
|
||||
},
|
||||
wantStatus: http.StatusBadRequest,
|
||||
wantErr: true,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
var bodyBytes []byte
|
||||
var err error
|
||||
|
||||
if tt.body != nil {
|
||||
switch v := tt.body.(type) {
|
||||
case string:
|
||||
bodyBytes = []byte(v)
|
||||
default:
|
||||
bodyBytes, err = json.Marshal(tt.body)
|
||||
if err != nil {
|
||||
t.Fatalf("failed to marshal body: %v", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
req := httptest.NewRequest(tt.method, "/api/v1/tasks", bytes.NewReader(bodyBytes))
|
||||
if tt.body != nil {
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
}
|
||||
|
||||
rr := httptest.NewRecorder()
|
||||
handlers.CreateTaskHandler(rr, req)
|
||||
|
||||
if rr.Code != tt.wantStatus {
|
||||
t.Errorf("expected status %d, got %d", tt.wantStatus, rr.Code)
|
||||
}
|
||||
|
||||
if tt.wantErr {
|
||||
var errResp ErrorResponse
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &errResp); err != nil {
|
||||
t.Errorf("expected error response, got: %s", rr.Body.String())
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestListTasksHandler tests the list tasks endpoint
|
||||
func TestListTasksHandler(t *testing.T) {
|
||||
handlers, _ := setupTestServer(t)
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
method string
|
||||
wantStatus int
|
||||
}{
|
||||
{
|
||||
name: "Method not allowed",
|
||||
method: http.MethodPost,
|
||||
wantStatus: http.StatusMethodNotAllowed,
|
||||
},
|
||||
{
|
||||
name: "Success",
|
||||
method: http.MethodGet,
|
||||
wantStatus: http.StatusOK,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
req := httptest.NewRequest(tt.method, "/api/v1/tasks", nil)
|
||||
rr := httptest.NewRecorder()
|
||||
handlers.ListTasksHandler(rr, req)
|
||||
|
||||
if rr.Code != tt.wantStatus {
|
||||
t.Errorf("expected status %d, got %d", tt.wantStatus, rr.Code)
|
||||
}
|
||||
|
||||
if tt.method == http.MethodGet {
|
||||
var resp TasksListResponse
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
|
||||
t.Errorf("failed to unmarshal response: %v", err)
|
||||
}
|
||||
if resp.Tasks == nil {
|
||||
t.Error("expected non-nil tasks array")
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestGetTaskHandler tests the get task endpoint
|
||||
func TestGetTaskHandler(t *testing.T) {
|
||||
handlers, _ := setupTestServer(t)
|
||||
|
||||
// Register a test task
|
||||
testTaskID := "test-get-task"
|
||||
RegisterTask(testTaskID, "directlinks", "local", "downloads", "Test", "")
|
||||
defer DeleteTask(testTaskID)
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
method string
|
||||
path string
|
||||
wantStatus int
|
||||
wantFound bool
|
||||
}{
|
||||
{
|
||||
name: "Method not allowed",
|
||||
method: http.MethodPost,
|
||||
path: "/api/v1/tasks/test-id",
|
||||
wantStatus: http.StatusMethodNotAllowed,
|
||||
},
|
||||
{
|
||||
name: "Missing task ID",
|
||||
method: http.MethodGet,
|
||||
path: "/api/v1/tasks",
|
||||
wantStatus: http.StatusBadRequest,
|
||||
},
|
||||
{
|
||||
name: "Task not found",
|
||||
method: http.MethodGet,
|
||||
path: "/api/v1/tasks/non-existent-task",
|
||||
wantStatus: http.StatusNotFound,
|
||||
},
|
||||
{
|
||||
name: "Task found",
|
||||
method: http.MethodGet,
|
||||
path: "/api/v1/tasks/" + testTaskID,
|
||||
wantStatus: http.StatusOK,
|
||||
wantFound: true,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
req := httptest.NewRequest(tt.method, tt.path, nil)
|
||||
rr := httptest.NewRecorder()
|
||||
handlers.GetTaskHandler(rr, req)
|
||||
|
||||
if rr.Code != tt.wantStatus {
|
||||
t.Errorf("expected status %d, got %d", tt.wantStatus, rr.Code)
|
||||
}
|
||||
|
||||
if tt.wantFound {
|
||||
var resp TaskInfoResponse
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
|
||||
t.Errorf("failed to unmarshal response: %v", err)
|
||||
}
|
||||
if resp.TaskID != testTaskID {
|
||||
t.Errorf("expected task ID %s, got %s", testTaskID, resp.TaskID)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestCancelTaskHandler tests the cancel task endpoint
|
||||
func TestCancelTaskHandler(t *testing.T) {
|
||||
handlers, _ := setupTestServer(t)
|
||||
|
||||
// Register a test task
|
||||
testTaskID := "test-cancel-task"
|
||||
RegisterTask(testTaskID, "directlinks", "local", "downloads", "Test", "")
|
||||
defer DeleteTask(testTaskID)
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
method string
|
||||
path string
|
||||
wantStatus int
|
||||
skipCore bool // Skip if core is not initialized
|
||||
}{
|
||||
{
|
||||
name: "Method not allowed",
|
||||
method: http.MethodGet,
|
||||
path: "/api/v1/tasks/test-id",
|
||||
wantStatus: http.StatusMethodNotAllowed,
|
||||
},
|
||||
{
|
||||
name: "Missing task ID",
|
||||
method: http.MethodDelete,
|
||||
path: "/api/v1/tasks",
|
||||
wantStatus: http.StatusBadRequest,
|
||||
},
|
||||
{
|
||||
name: "Task not found",
|
||||
method: http.MethodDelete,
|
||||
path: "/api/v1/tasks/non-existent-task",
|
||||
wantStatus: http.StatusNotFound,
|
||||
},
|
||||
{
|
||||
name: "Cancel task",
|
||||
method: http.MethodDelete,
|
||||
path: "/api/v1/tasks/" + testTaskID,
|
||||
wantStatus: http.StatusOK,
|
||||
skipCore: true, // Requires initialized core task queue
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
if tt.skipCore {
|
||||
t.Skip("Skipping test: requires initialized core")
|
||||
}
|
||||
|
||||
req := httptest.NewRequest(tt.method, tt.path, nil)
|
||||
rr := httptest.NewRecorder()
|
||||
handlers.CancelTaskHandler(rr, req)
|
||||
|
||||
if rr.Code != tt.wantStatus {
|
||||
t.Errorf("expected status %d, got %d", tt.wantStatus, rr.Code)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestListStoragesHandler tests the list storages endpoint
|
||||
func TestListStoragesHandler(t *testing.T) {
|
||||
handlers, _ := setupTestServer(t)
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
method string
|
||||
wantStatus int
|
||||
}{
|
||||
{
|
||||
name: "Method not allowed",
|
||||
method: http.MethodPost,
|
||||
wantStatus: http.StatusMethodNotAllowed,
|
||||
},
|
||||
{
|
||||
name: "Success",
|
||||
method: http.MethodGet,
|
||||
wantStatus: http.StatusOK,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
req := httptest.NewRequest(tt.method, "/api/v1/storages", nil)
|
||||
rr := httptest.NewRecorder()
|
||||
handlers.ListStoragesHandler(rr, req)
|
||||
|
||||
if rr.Code != tt.wantStatus {
|
||||
t.Errorf("expected status %d, got %d", tt.wantStatus, rr.Code)
|
||||
}
|
||||
|
||||
if tt.method == http.MethodGet {
|
||||
var resp StoragesResponse
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
|
||||
t.Errorf("failed to unmarshal response: %v", err)
|
||||
}
|
||||
if resp.Storages == nil {
|
||||
t.Error("expected non-nil storages array")
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestConcurrentProgressStore tests concurrent access to progress store
|
||||
func TestConcurrentProgressStore(t *testing.T) {
|
||||
// Clear store before test
|
||||
t.Cleanup(func() {
|
||||
tasks := GetAllTasks()
|
||||
for _, task := range tasks {
|
||||
if strings.HasPrefix(task.TaskID, "concurrent-test-") {
|
||||
DeleteTask(task.TaskID)
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
var wg sync.WaitGroup
|
||||
numGoroutines := 100
|
||||
|
||||
// Concurrent registrations
|
||||
for i := range numGoroutines {
|
||||
wg.Add(1)
|
||||
go func(id int) {
|
||||
defer wg.Done()
|
||||
taskID := fmt.Sprintf("concurrent-test-%d", id)
|
||||
RegisterTask(taskID, "directlinks", "local", "downloads", "Test", "")
|
||||
}(i)
|
||||
}
|
||||
|
||||
// Concurrent reads
|
||||
for i := range numGoroutines {
|
||||
wg.Add(1)
|
||||
go func(id int) {
|
||||
defer wg.Done()
|
||||
taskID := fmt.Sprintf("concurrent-test-%d", id)
|
||||
GetTask(taskID)
|
||||
}(i)
|
||||
}
|
||||
|
||||
// Concurrent updates
|
||||
for i := range numGoroutines {
|
||||
wg.Add(1)
|
||||
go func(id int) {
|
||||
defer wg.Done()
|
||||
taskID := fmt.Sprintf("concurrent-test-%d", id)
|
||||
info, ok := GetTask(taskID)
|
||||
if ok {
|
||||
info.UpdateStatus(TaskStatusRunning)
|
||||
}
|
||||
}(i)
|
||||
}
|
||||
|
||||
wg.Wait()
|
||||
|
||||
// Verify all tasks exist
|
||||
for i := range numGoroutines {
|
||||
taskID := fmt.Sprintf("concurrent-test-%d", i)
|
||||
if _, ok := GetTask(taskID); !ok {
|
||||
t.Errorf("task %s not found after concurrent operations", taskID)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestProgressTrackerConcurrentUpdates tests concurrent progress updates
|
||||
func TestProgressTrackerConcurrentUpdates(t *testing.T) {
|
||||
tracker := NewProgressTracker("concurrent-progress", "directlinks", "local", "downloads", "Test", "")
|
||||
tracker.OnStart(10000, 10)
|
||||
|
||||
var wg sync.WaitGroup
|
||||
numGoroutines := 50
|
||||
updatesPerGoroutine := 100
|
||||
|
||||
// Concurrent progress updates
|
||||
for i := range numGoroutines {
|
||||
wg.Add(1)
|
||||
go func(id int) {
|
||||
defer wg.Done()
|
||||
for j := range updatesPerGoroutine {
|
||||
tracker.OnProgress(int64(id*updatesPerGoroutine+j), j)
|
||||
}
|
||||
}(i)
|
||||
}
|
||||
|
||||
wg.Wait()
|
||||
|
||||
info := tracker.GetInfo()
|
||||
if info.Status != TaskStatusRunning {
|
||||
t.Errorf("expected status Running after concurrent updates, got %s", info.Status)
|
||||
}
|
||||
// Note: Due to race conditions in the simple implementation,
|
||||
// we can't reliably check exact values without proper synchronization
|
||||
}
|
||||
|
||||
// TestTaskFactoryValidation tests TaskFactory parameter validation
|
||||
func TestTaskFactoryValidation(t *testing.T) {
|
||||
factory := NewTaskFactory(context.Background())
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
request *CreateTaskRequest
|
||||
wantErr bool
|
||||
errMsg string
|
||||
}{
|
||||
{
|
||||
name: "Storage not found",
|
||||
request: &CreateTaskRequest{
|
||||
Type: tasktype.TaskTypeDirectlinks,
|
||||
Storage: "non-existent",
|
||||
Path: "downloads",
|
||||
Params: json.RawMessage(`{"urls":["https://example.com/file.zip"]}`),
|
||||
},
|
||||
wantErr: true,
|
||||
errMsg: "storage not found",
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
_, err := factory.CreateTask(tt.request)
|
||||
if (err != nil) != tt.wantErr {
|
||||
t.Errorf("CreateTask() error = %v, wantErr %v", err, tt.wantErr)
|
||||
return
|
||||
}
|
||||
if err != nil && tt.errMsg != "" {
|
||||
if !strings.Contains(strings.ToLower(err.Error()), tt.errMsg) {
|
||||
t.Errorf("error message %q does not contain %q", err.Error(), tt.errMsg)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestEdgeCases tests various edge cases
|
||||
func TestEdgeCases(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
fn func(t *testing.T)
|
||||
}{
|
||||
{
|
||||
name: "Empty request body",
|
||||
fn: func(t *testing.T) {
|
||||
handlers, _ := setupTestServer(t)
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/v1/tasks", nil)
|
||||
rr := httptest.NewRecorder()
|
||||
handlers.CreateTaskHandler(rr, req)
|
||||
if rr.Code != http.StatusBadRequest {
|
||||
t.Errorf("expected %d, got %d", http.StatusBadRequest, rr.Code)
|
||||
}
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "Very long task ID in path",
|
||||
fn: func(t *testing.T) {
|
||||
handlers, _ := setupTestServer(t)
|
||||
longID := strings.Repeat("a", 1000)
|
||||
req := httptest.NewRequest(http.MethodGet, "/api/v1/tasks/"+longID, nil)
|
||||
rr := httptest.NewRecorder()
|
||||
handlers.GetTaskHandler(rr, req)
|
||||
if rr.Code != http.StatusNotFound {
|
||||
t.Errorf("expected %d, got %d", http.StatusNotFound, rr.Code)
|
||||
}
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "Path with special characters",
|
||||
fn: func(t *testing.T) {
|
||||
path := "/api/v1/tasks/test%20id/with/slashes"
|
||||
got := extractTaskIDFromPath(path)
|
||||
expected := "test%20id"
|
||||
if got != expected {
|
||||
t.Errorf("expected %q, got %q", expected, got)
|
||||
}
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "Double slashes in path",
|
||||
fn: func(t *testing.T) {
|
||||
path := "/api/v1/tasks//task-id"
|
||||
got := extractTaskIDFromPath(path)
|
||||
expected := ""
|
||||
if got != expected {
|
||||
t.Errorf("expected %q, got %q", expected, got)
|
||||
}
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "Progress tracker with empty webhook",
|
||||
fn: func(t *testing.T) {
|
||||
tracker := NewProgressTracker("test", "type", "storage", "path", "title", "")
|
||||
info := tracker.GetInfo()
|
||||
if info.Webhook != "" {
|
||||
t.Error("expected empty webhook")
|
||||
}
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, tt.fn)
|
||||
}
|
||||
}
|
||||
|
||||
// TestHealthCheckHandler tests the health check endpoint
|
||||
func TestHealthCheckHandler(t *testing.T) {
|
||||
handlers, _ := setupTestServer(t)
|
||||
|
||||
req := httptest.NewRequest(http.MethodGet, "/health", nil)
|
||||
rr := httptest.NewRecorder()
|
||||
handlers.HealthCheckHandler(rr, req)
|
||||
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Errorf("expected status %d, got %d", http.StatusOK, rr.Code)
|
||||
}
|
||||
|
||||
var resp map[string]string
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
|
||||
t.Fatalf("failed to unmarshal response: %v", err)
|
||||
}
|
||||
|
||||
if resp["status"] != "ok" {
|
||||
t.Errorf("expected status 'ok', got %q", resp["status"])
|
||||
}
|
||||
}
|
||||
|
||||
// TestGetTaskTypesHandler tests the task types endpoint
|
||||
func TestGetTaskTypesHandler(t *testing.T) {
|
||||
handlers, _ := setupTestServer(t)
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
method string
|
||||
wantStatus int
|
||||
}{
|
||||
{
|
||||
name: "Method not allowed",
|
||||
method: http.MethodPost,
|
||||
wantStatus: http.StatusMethodNotAllowed,
|
||||
},
|
||||
{
|
||||
name: "Success",
|
||||
method: http.MethodGet,
|
||||
wantStatus: http.StatusOK,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
req := httptest.NewRequest(tt.method, "/api/v1/task-types", nil)
|
||||
rr := httptest.NewRecorder()
|
||||
handlers.GetTaskTypesHandler(rr, req)
|
||||
|
||||
if rr.Code != tt.wantStatus {
|
||||
t.Errorf("expected status %d, got %d", tt.wantStatus, rr.Code)
|
||||
}
|
||||
|
||||
if tt.method == http.MethodGet {
|
||||
var resp map[string]any
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
|
||||
t.Errorf("failed to unmarshal response: %v", err)
|
||||
}
|
||||
if _, ok := resp["types"]; !ok {
|
||||
t.Error("expected 'types' field in response")
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestNotFoundHandler tests the 404 handler
|
||||
func TestNotFoundHandler(t *testing.T) {
|
||||
req := httptest.NewRequest(http.MethodGet, "/non-existent-path", nil)
|
||||
rr := httptest.NewRecorder()
|
||||
NotFoundHandler(rr, req)
|
||||
|
||||
if rr.Code != http.StatusNotFound {
|
||||
t.Errorf("expected status %d, got %d", http.StatusNotFound, rr.Code)
|
||||
}
|
||||
|
||||
var resp ErrorResponse
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
|
||||
t.Fatalf("failed to unmarshal response: %v", err)
|
||||
}
|
||||
|
||||
if resp.Error != "not_found" {
|
||||
t.Errorf("expected error 'not_found', got %q", resp.Error)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMethodNotAllowedHandler tests the 405 handler
|
||||
func TestMethodNotAllowedHandler(t *testing.T) {
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/v1/tasks", nil)
|
||||
rr := httptest.NewRecorder()
|
||||
MethodNotAllowedHandler(rr, req)
|
||||
|
||||
if rr.Code != http.StatusMethodNotAllowed {
|
||||
t.Errorf("expected status %d, got %d", http.StatusMethodNotAllowed, rr.Code)
|
||||
}
|
||||
|
||||
var resp ErrorResponse
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
|
||||
t.Fatalf("failed to unmarshal response: %v", err)
|
||||
}
|
||||
|
||||
if resp.Error != "method_not_allowed" {
|
||||
t.Errorf("expected error 'method_not_allowed', got %q", resp.Error)
|
||||
}
|
||||
}
|
||||
|
||||
// TestTaskProgressInfoTimeUpdate tests that timestamps are updated correctly
|
||||
func TestTaskProgressInfoTimeUpdate(t *testing.T) {
|
||||
info := RegisterTask("time-test", "directlinks", "local", "downloads", "Test", "")
|
||||
defer DeleteTask("time-test")
|
||||
|
||||
originalTime := info.UpdatedAt
|
||||
time.Sleep(10 * time.Millisecond) // Ensure time difference
|
||||
|
||||
info.UpdateStatus(TaskStatusRunning)
|
||||
if !info.UpdatedAt.After(originalTime) {
|
||||
t.Error("expected UpdatedAt to be updated")
|
||||
}
|
||||
}
|
||||
|
||||
// TestWebhookPayloadWithNilCompletedAt tests webhook payload with nil completed_at
|
||||
func TestWebhookPayloadWithNilCompletedAt(t *testing.T) {
|
||||
payload := WebhookPayload{
|
||||
TaskID: "test-id",
|
||||
Type: "directlinks",
|
||||
Status: TaskStatusRunning,
|
||||
Storage: "local",
|
||||
Path: "downloads/file.zip",
|
||||
CompletedAt: nil,
|
||||
Error: "",
|
||||
}
|
||||
|
||||
data, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
t.Fatalf("failed to marshal: %v", err)
|
||||
}
|
||||
|
||||
var decoded map[string]any
|
||||
if err := json.Unmarshal(data, &decoded); err != nil {
|
||||
t.Fatalf("failed to unmarshal: %v", err)
|
||||
}
|
||||
|
||||
// completed_at should be omitted when nil
|
||||
if _, ok := decoded["completed_at"]; ok {
|
||||
t.Error("expected completed_at to be omitted when nil")
|
||||
}
|
||||
}
|
||||
150
api/progress.go
Normal file
150
api/progress.go
Normal file
@@ -0,0 +1,150 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
)
|
||||
|
||||
// TaskProgressInfo 存储任务的进度信息
|
||||
type TaskProgressInfo struct {
|
||||
TaskID string
|
||||
Type string
|
||||
Status TaskStatus
|
||||
Title string
|
||||
TotalBytes int64
|
||||
DownloadedBytes int64
|
||||
TotalFiles int
|
||||
DownloadedFiles int
|
||||
Storage string
|
||||
Path string
|
||||
Error string
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
Webhook string
|
||||
}
|
||||
|
||||
// progressStore 存储所有 API 任务的进度信息
|
||||
type progressStore struct {
|
||||
mu sync.RWMutex
|
||||
tasks map[string]*TaskProgressInfo
|
||||
}
|
||||
|
||||
var store = &progressStore{
|
||||
tasks: make(map[string]*TaskProgressInfo),
|
||||
}
|
||||
|
||||
// RegisterTask 注册一个新的 API 任务
|
||||
func RegisterTask(taskID, taskType, storage, path, title, webhook string) *TaskProgressInfo {
|
||||
info := &TaskProgressInfo{
|
||||
TaskID: taskID,
|
||||
Type: taskType,
|
||||
Status: TaskStatusQueued,
|
||||
Title: title,
|
||||
Storage: storage,
|
||||
Path: path,
|
||||
CreatedAt: time.Now(),
|
||||
UpdatedAt: time.Now(),
|
||||
Webhook: webhook,
|
||||
}
|
||||
|
||||
store.mu.Lock()
|
||||
store.tasks[taskID] = info
|
||||
store.mu.Unlock()
|
||||
|
||||
return info
|
||||
}
|
||||
|
||||
// GetTask 获取任务进度信息
|
||||
func GetTask(taskID string) (*TaskProgressInfo, bool) {
|
||||
store.mu.RLock()
|
||||
defer store.mu.RUnlock()
|
||||
info, ok := store.tasks[taskID]
|
||||
return info, ok
|
||||
}
|
||||
|
||||
// GetAllTasks 获取所有任务
|
||||
func GetAllTasks() []*TaskProgressInfo {
|
||||
store.mu.RLock()
|
||||
defer store.mu.RUnlock()
|
||||
|
||||
tasks := make([]*TaskProgressInfo, 0, len(store.tasks))
|
||||
for _, info := range store.tasks {
|
||||
tasks = append(tasks, info)
|
||||
}
|
||||
return tasks
|
||||
}
|
||||
|
||||
// DeleteTask 删除任务记录
|
||||
func DeleteTask(taskID string) {
|
||||
store.mu.Lock()
|
||||
defer store.mu.Unlock()
|
||||
delete(store.tasks, taskID)
|
||||
}
|
||||
|
||||
// UpdateStatus 更新任务状态
|
||||
func (t *TaskProgressInfo) UpdateStatus(status TaskStatus) {
|
||||
t.Status = status
|
||||
t.UpdatedAt = time.Now()
|
||||
}
|
||||
|
||||
// SetError 设置错误信息
|
||||
func (t *TaskProgressInfo) SetError(err string) {
|
||||
t.Error = err
|
||||
t.Status = TaskStatusFailed
|
||||
t.UpdatedAt = time.Now()
|
||||
}
|
||||
|
||||
// ProgressTracker 用于 API 任务的进度追踪
|
||||
type ProgressTracker struct {
|
||||
info *TaskProgressInfo
|
||||
}
|
||||
|
||||
// NewProgressTracker 创建新的进度追踪器
|
||||
func NewProgressTracker(taskID, taskType, storage, path, title, webhook string) *ProgressTracker {
|
||||
info := RegisterTask(taskID, taskType, storage, path, title, webhook)
|
||||
return &ProgressTracker{info: info}
|
||||
}
|
||||
|
||||
// OnStart 任务开始
|
||||
func (p *ProgressTracker) OnStart(totalBytes int64, totalFiles int) {
|
||||
p.info.Status = TaskStatusRunning
|
||||
p.info.TotalBytes = totalBytes
|
||||
p.info.TotalFiles = totalFiles
|
||||
p.info.UpdatedAt = time.Now()
|
||||
}
|
||||
|
||||
// OnProgress 进度更新
|
||||
func (p *ProgressTracker) OnProgress(downloadedBytes int64, downloadedFiles int) {
|
||||
atomic.StoreInt64(&p.info.DownloadedBytes, downloadedBytes)
|
||||
p.info.DownloadedFiles = downloadedFiles
|
||||
p.info.UpdatedAt = time.Now()
|
||||
}
|
||||
|
||||
// OnDone 任务完成
|
||||
func (p *ProgressTracker) OnDone(err error) {
|
||||
if err != nil {
|
||||
p.info.Status = TaskStatusFailed
|
||||
p.info.Error = err.Error()
|
||||
} else {
|
||||
p.info.Status = TaskStatusCompleted
|
||||
}
|
||||
p.info.UpdatedAt = time.Now()
|
||||
}
|
||||
|
||||
// GetInfo 获取任务信息
|
||||
func (p *ProgressTracker) GetInfo() *TaskProgressInfo {
|
||||
return p.info
|
||||
}
|
||||
|
||||
// UpdateProgressBytes 更新下载字节数
|
||||
func (p *ProgressTracker) UpdateProgressBytes(bytes int64) {
|
||||
atomic.StoreInt64(&p.info.DownloadedBytes, bytes)
|
||||
p.info.UpdatedAt = time.Now()
|
||||
}
|
||||
|
||||
// UpdateProgressFiles 更新下载文件数
|
||||
func (p *ProgressTracker) UpdateProgressFiles(files int) {
|
||||
p.info.DownloadedFiles = files
|
||||
p.info.UpdatedAt = time.Now()
|
||||
}
|
||||
163
api/server.go
Normal file
163
api/server.go
Normal file
@@ -0,0 +1,163 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
)
|
||||
|
||||
// Server API 服务器
|
||||
type Server struct {
|
||||
httpServer *http.Server
|
||||
factory *TaskFactory
|
||||
}
|
||||
|
||||
// NewServer 创建新的 API 服务器
|
||||
func NewServer(ctx context.Context) *Server {
|
||||
cfg := config.C().API
|
||||
|
||||
factory := NewTaskFactory(ctx)
|
||||
handlers := NewHandlers(factory)
|
||||
|
||||
// 设置路由
|
||||
mux := http.NewServeMux()
|
||||
|
||||
// 健康检查
|
||||
mux.HandleFunc("/health", handlers.HealthCheckHandler)
|
||||
|
||||
// API v1 路由
|
||||
mux.HandleFunc("/api/v1/tasks", handlers.CreateTaskHandler)
|
||||
mux.HandleFunc("/api/v1/tasks/", func(w http.ResponseWriter, r *http.Request) {
|
||||
// 根据方法和路径分发
|
||||
switch r.Method {
|
||||
case http.MethodGet:
|
||||
if r.URL.Path == "/api/v1/tasks" {
|
||||
handlers.ListTasksHandler(w, r)
|
||||
} else {
|
||||
handlers.GetTaskHandler(w, r)
|
||||
}
|
||||
case http.MethodDelete:
|
||||
handlers.CancelTaskHandler(w, r)
|
||||
default:
|
||||
MethodNotAllowedHandler(w, r)
|
||||
}
|
||||
})
|
||||
mux.HandleFunc("/api/v1/storages", handlers.ListStoragesHandler)
|
||||
mux.HandleFunc("/api/v1/task-types", handlers.GetTaskTypesHandler)
|
||||
|
||||
// 404 处理
|
||||
mux.HandleFunc("/", NotFoundHandler)
|
||||
|
||||
// 应用中间件
|
||||
var handler http.Handler = mux
|
||||
|
||||
// 添加认证中间件
|
||||
token := cfg.Token
|
||||
if token == "" {
|
||||
log.FromContext(ctx).Warn("API server is enabled but no token is set, this is insecure!")
|
||||
}
|
||||
if token != "" {
|
||||
handler = AuthMiddleware()(handler)
|
||||
}
|
||||
|
||||
// 添加日志中间件
|
||||
handler = loggingMiddleware(handler)
|
||||
|
||||
// 添加恢复中间件
|
||||
handler = recoveryMiddleware(handler)
|
||||
|
||||
return &Server{
|
||||
httpServer: &http.Server{
|
||||
Addr: fmt.Sprintf("%s:%d", cfg.Host, cfg.Port),
|
||||
Handler: handler,
|
||||
ReadTimeout: 30 * time.Second,
|
||||
WriteTimeout: 30 * time.Second,
|
||||
IdleTimeout: 120 * time.Second,
|
||||
},
|
||||
factory: factory,
|
||||
}
|
||||
}
|
||||
|
||||
// Start 启动服务器
|
||||
func (s *Server) Start(ctx context.Context) error {
|
||||
logger := log.FromContext(ctx).With("module", "api")
|
||||
|
||||
logger.Infof("Starting API server on %s", s.httpServer.Addr)
|
||||
|
||||
// 在 goroutine 中启动服务器
|
||||
go func() {
|
||||
if err := s.httpServer.ListenAndServe(); err != nil && err != http.ErrServerClosed {
|
||||
logger.Errorf("API server error: %v", err)
|
||||
}
|
||||
}()
|
||||
|
||||
// 监听 context 取消
|
||||
go func() {
|
||||
<-ctx.Done()
|
||||
shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
if err := s.httpServer.Shutdown(shutdownCtx); err != nil {
|
||||
logger.Errorf("API server shutdown error: %v", err)
|
||||
}
|
||||
}()
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// loggingMiddleware 日志中间件
|
||||
func loggingMiddleware(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
start := time.Now()
|
||||
|
||||
// 包装 ResponseWriter 以获取状态码
|
||||
wrapped := &responseWriter{ResponseWriter: w, statusCode: http.StatusOK}
|
||||
|
||||
next.ServeHTTP(wrapped, r)
|
||||
|
||||
log.Infof("%s %s %d %s", r.Method, r.URL.Path, wrapped.statusCode, time.Since(start))
|
||||
})
|
||||
}
|
||||
|
||||
// recoveryMiddleware 恢复中间件
|
||||
func recoveryMiddleware(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
defer func() {
|
||||
if err := recover(); err != nil {
|
||||
log.Errorf("Panic recovered: %v", err)
|
||||
WriteError(w, http.StatusInternalServerError, "internal_error", "internal server error")
|
||||
}
|
||||
}()
|
||||
next.ServeHTTP(w, r)
|
||||
})
|
||||
}
|
||||
|
||||
// responseWriter 包装 http.ResponseWriter 以捕获状态码
|
||||
type responseWriter struct {
|
||||
http.ResponseWriter
|
||||
statusCode int
|
||||
}
|
||||
|
||||
func (rw *responseWriter) WriteHeader(code int) {
|
||||
rw.statusCode = code
|
||||
rw.ResponseWriter.WriteHeader(code)
|
||||
}
|
||||
|
||||
// Start 初始化并启动 API 服务器
|
||||
func Start(ctx context.Context) error {
|
||||
cfg := config.C().API
|
||||
|
||||
if !cfg.Enable {
|
||||
return nil
|
||||
}
|
||||
|
||||
if cfg.Token == "" {
|
||||
log.FromContext(ctx).Warn("API server is enabled but no token is set, this is insecure!")
|
||||
}
|
||||
|
||||
server := NewServer(ctx)
|
||||
return server.Start(ctx)
|
||||
}
|
||||
272
api/tgfiles.go
Normal file
272
api/tgfiles.go
Normal file
@@ -0,0 +1,272 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/url"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"github.com/celestix/gotgproto/ext"
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/gotd/td/tg"
|
||||
"github.com/krau/SaveAny-Bot/client/bot"
|
||||
userclient "github.com/krau/SaveAny-Bot/client/user"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
|
||||
"github.com/krau/SaveAny-Bot/pkg/tfile"
|
||||
)
|
||||
|
||||
// MessageContext 保存消息和获取它所用的 context
|
||||
type MessageContext struct {
|
||||
Message *tg.Message
|
||||
Client *ext.Context
|
||||
}
|
||||
|
||||
// getClientContext 获取可用的客户端上下文
|
||||
// 优先使用 Bot,失败后回退到 Userbot
|
||||
func getClientContext() (*ext.Context, error) {
|
||||
// 首先尝试获取 Bot context
|
||||
if botCtx := bot.ExtContext(); botCtx != nil {
|
||||
return botCtx, nil
|
||||
}
|
||||
|
||||
// 回退到 Userbot
|
||||
if uc := userclient.GetCtx(); uc != nil {
|
||||
return uc, nil
|
||||
}
|
||||
|
||||
return nil, fmt.Errorf("no client available (bot and userbot are not initialized)")
|
||||
}
|
||||
|
||||
// resolveChatID 解析聊天 ID
|
||||
func resolveChatID(_ context.Context, idOrUsername string) (int64, error) {
|
||||
// 如果是数字 ID
|
||||
if id, err := strconv.ParseInt(idOrUsername, 10, 64); err == nil {
|
||||
// 私有频道 ID 需要加上 -100 前缀
|
||||
if id > 0 {
|
||||
return -1000000000000 - id, nil
|
||||
}
|
||||
return id, nil
|
||||
}
|
||||
|
||||
// 获取可用的客户端上下文
|
||||
clientCtx, err := getClientContext()
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
// 使用 tgutil 的 ParseChatID
|
||||
return tgutil.ParseChatID(clientCtx, idOrUsername)
|
||||
}
|
||||
|
||||
// ParseMessageLink 解析 Telegram 消息链接
|
||||
// 支持格式:
|
||||
// - https://t.me/username/123
|
||||
// - https://t.me/c/123456789/123
|
||||
// - https://t.me/c/123456789/111/456 (topic id)
|
||||
// - https://t.me/username/123?comment=2 (评论)
|
||||
func ParseMessageLink(ctx context.Context, link string) (int64, int, error) {
|
||||
u, err := url.Parse(link)
|
||||
if err != nil {
|
||||
return 0, 0, fmt.Errorf("invalid URL: %w", err)
|
||||
}
|
||||
paths := strings.Split(strings.TrimPrefix(u.Path, "/"), "/")
|
||||
|
||||
if cmt := u.Query().Get("comment"); cmt != "" {
|
||||
// 频道评论的消息链接
|
||||
if len(paths) < 1 {
|
||||
return 0, 0, fmt.Errorf("invalid message link format: %s", link)
|
||||
}
|
||||
// 简化处理:返回错误,提示不支持评论链接
|
||||
return 0, 0, fmt.Errorf("comment links are not supported")
|
||||
}
|
||||
|
||||
switch len(paths) {
|
||||
case 2: // https://t.me/username/123
|
||||
chatID, err := resolveChatID(ctx, paths[0])
|
||||
if err != nil {
|
||||
return 0, 0, fmt.Errorf("failed to resolve chat ID: %w", err)
|
||||
}
|
||||
msgID, err := strconv.Atoi(paths[1])
|
||||
if err != nil {
|
||||
return 0, 0, fmt.Errorf("failed to parse message ID: %w", err)
|
||||
}
|
||||
return chatID, msgID, nil
|
||||
case 3:
|
||||
// https://t.me/c/123456789/123
|
||||
// https://t.me/username/123/456 , 123: topic id
|
||||
chatPart, msgPart := paths[1], paths[2]
|
||||
if paths[0] != "c" {
|
||||
chatPart = paths[0]
|
||||
}
|
||||
chatID, err := resolveChatID(ctx, chatPart)
|
||||
if err != nil {
|
||||
return 0, 0, fmt.Errorf("failed to resolve chat ID: %w", err)
|
||||
}
|
||||
msgID, err := strconv.Atoi(msgPart)
|
||||
if err != nil {
|
||||
return 0, 0, fmt.Errorf("failed to parse message ID: %w", err)
|
||||
}
|
||||
return chatID, msgID, nil
|
||||
case 4:
|
||||
// https://t.me/c/123456789/111/456 111: topic id
|
||||
if paths[0] != "c" {
|
||||
return 0, 0, fmt.Errorf("invalid message link format: %s", link)
|
||||
}
|
||||
chatID, err := resolveChatID(ctx, paths[1])
|
||||
if err != nil {
|
||||
return 0, 0, fmt.Errorf("failed to resolve chat ID: %w", err)
|
||||
}
|
||||
msgID, err := strconv.Atoi(paths[3])
|
||||
if err != nil {
|
||||
return 0, 0, fmt.Errorf("failed to parse message ID: %w", err)
|
||||
}
|
||||
return chatID, msgID, nil
|
||||
}
|
||||
return 0, 0, fmt.Errorf("invalid message link format: %s", link)
|
||||
}
|
||||
|
||||
// getMessageWithContext 通过 ID 获取消息,返回消息和使用的 context
|
||||
// 确保消息获取和后续文件创建使用同一个 context
|
||||
func getMessageWithContext(_ context.Context, chatID int64, msgID int) (*MessageContext, error) {
|
||||
// 首先尝试使用 Bot
|
||||
if botCtx := bot.ExtContext(); botCtx != nil {
|
||||
msg, err := tgutil.GetMessageByID(botCtx, chatID, msgID)
|
||||
if err == nil {
|
||||
return &MessageContext{Message: msg, Client: botCtx}, nil
|
||||
}
|
||||
}
|
||||
|
||||
// 回退到 Userbot
|
||||
uc := userclient.GetCtx()
|
||||
if uc == nil {
|
||||
return nil, fmt.Errorf("userbot not initialized and bot cannot access this message")
|
||||
}
|
||||
|
||||
msg, err := tgutil.GetMessageByID(uc, chatID, msgID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &MessageContext{Message: msg, Client: uc}, nil
|
||||
}
|
||||
|
||||
// getGroupedMessagesWithContext 获取媒体组消息,返回消息列表和使用的 context
|
||||
// 确保消息获取和后续文件创建使用同一个 context
|
||||
func getGroupedMessagesWithContext(ctx *MessageContext, chatID int64) ([]*tg.Message, error) {
|
||||
msg := ctx.Message
|
||||
clientCtx := ctx.Client
|
||||
|
||||
groupID, ok := msg.GetGroupedID()
|
||||
if !ok || groupID == 0 {
|
||||
return []*tg.Message{msg}, nil
|
||||
}
|
||||
|
||||
// 使用获取原始消息的同一个 client 获取媒体组
|
||||
msgs, err := tgutil.GetGroupedMessages(clientCtx, chatID, msg)
|
||||
if err != nil || len(msgs) == 0 {
|
||||
// 如果获取失败,至少返回原始消息
|
||||
return []*tg.Message{msg}, nil
|
||||
}
|
||||
|
||||
return msgs, nil
|
||||
}
|
||||
|
||||
// ExtractFilesFromLinks 从消息链接中提取文件
|
||||
// 每个文件的处理流程:解析链接 -> 获取消息 -> 获取媒体组 -> 创建文件对象
|
||||
// 对于单个文件,全程使用同一个 client context,不会交叉
|
||||
func ExtractFilesFromLinks(ctx context.Context, links []string) ([]tfile.TGFileMessage, error) {
|
||||
logger := log.FromContext(ctx)
|
||||
var files []tfile.TGFileMessage
|
||||
|
||||
for _, link := range links {
|
||||
link = strings.TrimSpace(link)
|
||||
if link == "" {
|
||||
continue
|
||||
}
|
||||
|
||||
// 验证链接格式
|
||||
if !isValidMessageLink(link) {
|
||||
logger.Errorf("Invalid message link format: %s", link)
|
||||
continue
|
||||
}
|
||||
|
||||
chatID, msgID, err := ParseMessageLink(ctx, link)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to parse message link %s: %v", link, err)
|
||||
continue
|
||||
}
|
||||
|
||||
// 解析链接 URL 检查是否有 single 参数
|
||||
u, _ := url.Parse(link)
|
||||
single := u != nil && u.Query().Has("single")
|
||||
|
||||
// 获取消息和使用的 context(Bot 优先,失败回退 Userbot)
|
||||
msgCtx, err := getMessageWithContext(ctx, chatID, msgID)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to get message %d from chat %d: %v", msgID, chatID, err)
|
||||
continue
|
||||
}
|
||||
|
||||
msg := msgCtx.Message
|
||||
clientCtx := msgCtx.Client
|
||||
|
||||
if msg.Media == nil {
|
||||
logger.Warnf("Message %d has no media", msgID)
|
||||
continue
|
||||
}
|
||||
|
||||
media, ok := msg.GetMedia()
|
||||
if !ok {
|
||||
logger.Warnf("Failed to get media from message %d", msgID)
|
||||
continue
|
||||
}
|
||||
|
||||
// 检查是否是媒体组
|
||||
groupID, isGroup := msg.GetGroupedID()
|
||||
if isGroup && groupID != 0 && !single {
|
||||
// 使用同一个 client context 获取媒体组
|
||||
groupMsgs, err := getGroupedMessagesWithContext(msgCtx, chatID)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to get grouped messages: %v", err)
|
||||
} else {
|
||||
for _, gmsg := range groupMsgs {
|
||||
if gmsg.Media == nil {
|
||||
continue
|
||||
}
|
||||
gmedia, ok := gmsg.GetMedia()
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
// 使用获取消息时使用的同一个 client context 创建文件
|
||||
file, err := tfile.FromMediaMessage(gmedia, clientCtx.Raw, gmsg)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to create file from media: %v", err)
|
||||
continue
|
||||
}
|
||||
files = append(files, file)
|
||||
}
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
// 单个文件 - 使用获取消息时使用的同一个 client context 创建文件
|
||||
file, err := tfile.FromMediaMessage(media, clientCtx.Raw, msg)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to create file from media: %v", err)
|
||||
continue
|
||||
}
|
||||
files = append(files, file)
|
||||
}
|
||||
|
||||
if len(files) == 0 {
|
||||
return nil, fmt.Errorf("no files found in provided links")
|
||||
}
|
||||
|
||||
return files, nil
|
||||
}
|
||||
|
||||
// isValidMessageLink 检查是否是有效的 Telegram 消息链接
|
||||
func isValidMessageLink(link string) bool {
|
||||
return strings.HasPrefix(link, "https://t.me/") || strings.HasPrefix(link, "http://t.me/")
|
||||
}
|
||||
80
api/tphpics.go
Normal file
80
api/tphpics.go
Normal file
@@ -0,0 +1,80 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/url"
|
||||
"strings"
|
||||
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/tphutil"
|
||||
"github.com/krau/SaveAny-Bot/pkg/telegraph"
|
||||
)
|
||||
|
||||
// ExtractTelegraphImages 从 Telegraph URL 提取图片
|
||||
func ExtractTelegraphImages(ctx context.Context, pageURL string) ([]string, string, error) {
|
||||
logger := log.FromContext(ctx)
|
||||
|
||||
// 验证 URL 格式
|
||||
if !isValidTelegraphURL(pageURL) {
|
||||
return nil, "", fmt.Errorf("invalid telegraph URL format: %s", pageURL)
|
||||
}
|
||||
|
||||
// 解析 URL 获取页面路径
|
||||
pagepath, err := parseTelegraphPath(pageURL)
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
|
||||
logger.Debugf("Fetching telegraph page: %s", pagepath)
|
||||
|
||||
client := telegraph.NewClient()
|
||||
page, err := client.GetPage(ctx, pagepath)
|
||||
if err != nil {
|
||||
return nil, "", fmt.Errorf("failed to get telegraph page: %w", err)
|
||||
}
|
||||
|
||||
var imgs []string
|
||||
for _, elem := range page.Content {
|
||||
imgs = append(imgs, tphutil.GetNodeImages(elem)...)
|
||||
}
|
||||
|
||||
if len(imgs) == 0 {
|
||||
return nil, "", fmt.Errorf("no images found in telegraph page")
|
||||
}
|
||||
|
||||
return imgs, pagepath, nil
|
||||
}
|
||||
|
||||
// parseTelegraphPath 解析 Telegraph URL 获取页面路径
|
||||
func parseTelegraphPath(pageURL string) (string, error) {
|
||||
u, err := url.Parse(pageURL)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("invalid telegraph URL: %w", err)
|
||||
}
|
||||
|
||||
if !strings.HasSuffix(u.Host, "telegra.ph") && !strings.HasSuffix(u.Host, "telegraph.co") {
|
||||
return "", fmt.Errorf("invalid telegraph URL host: %s", u.Host)
|
||||
}
|
||||
|
||||
paths := strings.Split(strings.TrimPrefix(u.Path, "/"), "/")
|
||||
if len(paths) == 0 || paths[0] == "" {
|
||||
return "", fmt.Errorf("invalid telegraph URL path: %s", u.Path)
|
||||
}
|
||||
|
||||
pagepath := paths[len(paths)-1]
|
||||
pagepath, err = url.PathUnescape(pagepath)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to unescape telegraph path: %w", err)
|
||||
}
|
||||
|
||||
return strings.TrimSpace(pagepath), nil
|
||||
}
|
||||
|
||||
// isValidTelegraphURL 检查是否是有效的 Telegraph URL
|
||||
func isValidTelegraphURL(url string) bool {
|
||||
return strings.HasPrefix(url, "https://telegra.ph/") ||
|
||||
strings.HasPrefix(url, "http://telegra.ph/") ||
|
||||
strings.HasPrefix(url, "https://telegraph.co/") ||
|
||||
strings.HasPrefix(url, "http://telegraph.co/")
|
||||
}
|
||||
161
api/types.go
Normal file
161
api/types.go
Normal file
@@ -0,0 +1,161 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
|
||||
)
|
||||
|
||||
// TaskStatus 表示任务状态
|
||||
type TaskStatus string
|
||||
|
||||
const (
|
||||
TaskStatusQueued TaskStatus = "queued"
|
||||
TaskStatusRunning TaskStatus = "running"
|
||||
TaskStatusCompleted TaskStatus = "completed"
|
||||
TaskStatusFailed TaskStatus = "failed"
|
||||
TaskStatusCancelled TaskStatus = "cancelled"
|
||||
)
|
||||
|
||||
// CreateTaskRequest 创建任务请求
|
||||
type CreateTaskRequest struct {
|
||||
Type tasktype.TaskType `json:"type"`
|
||||
Storage string `json:"storage"`
|
||||
Path string `json:"path"`
|
||||
Webhook string `json:"webhook,omitempty"`
|
||||
Params json.RawMessage `json:"params"`
|
||||
}
|
||||
|
||||
// CreateTaskResponse 创建任务响应
|
||||
type CreateTaskResponse struct {
|
||||
TaskID string `json:"task_id"`
|
||||
Type tasktype.TaskType `json:"type"`
|
||||
Status TaskStatus `json:"status"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
}
|
||||
|
||||
// TaskProgress 任务进度
|
||||
type TaskProgress struct {
|
||||
TotalBytes int64 `json:"total_bytes,omitempty"`
|
||||
DownloadedBytes int64 `json:"downloaded_bytes,omitempty"`
|
||||
Percent float64 `json:"percent,omitempty"`
|
||||
SpeedMBPS float64 `json:"speed_mbps,omitempty"`
|
||||
}
|
||||
|
||||
// TaskInfoResponse 任务信息响应
|
||||
type TaskInfoResponse struct {
|
||||
TaskID string `json:"task_id"`
|
||||
Type tasktype.TaskType `json:"type"`
|
||||
Status TaskStatus `json:"status"`
|
||||
Title string `json:"title"`
|
||||
Progress *TaskProgress `json:"progress,omitempty"`
|
||||
Storage string `json:"storage"`
|
||||
Path string `json:"path"`
|
||||
Error string `json:"error,omitempty"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
}
|
||||
|
||||
// TasksListResponse 任务列表响应
|
||||
type TasksListResponse struct {
|
||||
Tasks []TaskInfoResponse `json:"tasks"`
|
||||
Total int `json:"total"`
|
||||
}
|
||||
|
||||
// StoragesResponse 存储列表响应
|
||||
type StoragesResponse struct {
|
||||
Storages []StorageInfo `json:"storages"`
|
||||
}
|
||||
|
||||
// StorageInfo 存储信息
|
||||
type StorageInfo struct {
|
||||
Name string `json:"name"`
|
||||
Type string `json:"type"`
|
||||
}
|
||||
|
||||
// WebhookPayload Webhook 回调负载
|
||||
type WebhookPayload struct {
|
||||
TaskID string `json:"task_id"`
|
||||
Type string `json:"type"`
|
||||
Status TaskStatus `json:"status"`
|
||||
Storage string `json:"storage"`
|
||||
Path string `json:"path"`
|
||||
CompletedAt *time.Time `json:"completed_at,omitempty"`
|
||||
Error string `json:"error,omitempty"`
|
||||
}
|
||||
|
||||
// ErrorResponse 错误响应
|
||||
type ErrorResponse struct {
|
||||
Error string `json:"error"`
|
||||
Message string `json:"message,omitempty"`
|
||||
}
|
||||
|
||||
// APIError API 错误
|
||||
type APIError struct {
|
||||
StatusCode int
|
||||
ErrorCode string
|
||||
Message string
|
||||
}
|
||||
|
||||
func (e *APIError) Error() string {
|
||||
return e.Message
|
||||
}
|
||||
|
||||
// WriteJSON 写入 JSON 响应
|
||||
func WriteJSON(w http.ResponseWriter, statusCode int, data any) error {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(statusCode)
|
||||
return json.NewEncoder(w).Encode(data)
|
||||
}
|
||||
|
||||
// WriteError 写入错误响应
|
||||
func WriteError(w http.ResponseWriter, statusCode int, errCode, message string) error {
|
||||
return WriteJSON(w, statusCode, ErrorResponse{
|
||||
Error: errCode,
|
||||
Message: message,
|
||||
})
|
||||
}
|
||||
|
||||
// Task 参数结构体
|
||||
|
||||
// DirectLinksParams directlinks 任务参数
|
||||
type DirectLinksParams struct {
|
||||
URLs []string `json:"urls"`
|
||||
}
|
||||
|
||||
// YTDLPParams ytdlp 任务参数
|
||||
type YTDLPParams struct {
|
||||
URLs []string `json:"urls"`
|
||||
Flags []string `json:"flags,omitempty"`
|
||||
}
|
||||
|
||||
// Aria2Params aria2 任务参数
|
||||
type Aria2Params struct {
|
||||
URLs []string `json:"urls"`
|
||||
Options map[string]string `json:"options,omitempty"`
|
||||
}
|
||||
|
||||
// ParsedParams parsed 任务参数
|
||||
type ParsedParams struct {
|
||||
URL string `json:"url"`
|
||||
}
|
||||
|
||||
// TransferParams transfer 任务参数
|
||||
type TransferParams struct {
|
||||
SourceStorage string `json:"source_storage"`
|
||||
SourcePath string `json:"source_path"`
|
||||
TargetStorage string `json:"target_storage"`
|
||||
TargetPath string `json:"target_path"`
|
||||
}
|
||||
|
||||
// TGFilesParams tgfiles 任务参数
|
||||
type TGFilesParams struct {
|
||||
MessageLinks []string `json:"message_links"`
|
||||
}
|
||||
|
||||
// TPHPicsParams tphpics 任务参数
|
||||
type TPHPicsParams struct {
|
||||
TelegraphURL string `json:"telegraph_url"`
|
||||
}
|
||||
130
api/webhook.go
Normal file
130
api/webhook.go
Normal file
@@ -0,0 +1,130 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"github.com/charmbracelet/log"
|
||||
)
|
||||
|
||||
// webhookClient Webhook 客户端
|
||||
var webhookClient = &http.Client{
|
||||
Timeout: 30 * time.Second,
|
||||
}
|
||||
|
||||
// SendWebhook 发送 Webhook 回调
|
||||
func SendWebhook(ctx context.Context, payload *WebhookPayload) {
|
||||
if payload == nil || payload.TaskID == "" {
|
||||
return
|
||||
}
|
||||
|
||||
// 获取任务信息以获取 webhook URL
|
||||
info, ok := GetTask(payload.TaskID)
|
||||
if !ok || info.Webhook == "" {
|
||||
return
|
||||
}
|
||||
|
||||
webhookURL := info.Webhook
|
||||
|
||||
// 异步发送 webhook
|
||||
go func() {
|
||||
logger := log.FromContext(ctx).With("task_id", payload.TaskID)
|
||||
|
||||
payloadBytes, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to marshal webhook payload: %v", err)
|
||||
return
|
||||
}
|
||||
|
||||
// 重试 3 次
|
||||
for i := range 3 {
|
||||
req, err := http.NewRequestWithContext(context.Background(), http.MethodPost, webhookURL, bytes.NewBuffer(payloadBytes))
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to create webhook request: %v", err)
|
||||
return
|
||||
}
|
||||
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("User-Agent", "SaveAny-Bot/1.0")
|
||||
|
||||
resp, err := webhookClient.Do(req)
|
||||
if err != nil {
|
||||
logger.Warnf("Webhook request failed (attempt %d/3): %v", i+1, err)
|
||||
time.Sleep(time.Second * time.Duration(i+1))
|
||||
continue
|
||||
}
|
||||
resp.Body.Close()
|
||||
|
||||
if resp.StatusCode >= 200 && resp.StatusCode < 300 {
|
||||
logger.Debugf("Webhook sent successfully: %s", webhookURL)
|
||||
return
|
||||
}
|
||||
|
||||
logger.Warnf("Webhook returned non-2xx status (attempt %d/3): %d", i+1, resp.StatusCode)
|
||||
time.Sleep(time.Second * time.Duration(i+1))
|
||||
}
|
||||
|
||||
logger.Errorf("Failed to send webhook after 3 attempts")
|
||||
}()
|
||||
}
|
||||
|
||||
// CreateWebhookPayload 创建 Webhook 负载
|
||||
func CreateWebhookPayload(taskID string, taskType string, status TaskStatus, storage, path string, err error) *WebhookPayload {
|
||||
payload := &WebhookPayload{
|
||||
TaskID: taskID,
|
||||
Type: taskType,
|
||||
Status: status,
|
||||
Storage: storage,
|
||||
Path: path,
|
||||
}
|
||||
|
||||
if status == TaskStatusCompleted || status == TaskStatusFailed {
|
||||
now := time.Now()
|
||||
payload.CompletedAt = &now
|
||||
}
|
||||
|
||||
if err != nil {
|
||||
payload.Error = err.Error()
|
||||
}
|
||||
|
||||
return payload
|
||||
}
|
||||
|
||||
// WrapTaskWithWebhook 包装任务执行,添加 webhook 回调
|
||||
func WrapTaskWithWebhook(ctx context.Context, taskID string, fn func() error) error {
|
||||
info, ok := GetTask(taskID)
|
||||
if !ok {
|
||||
return fmt.Errorf("task not found: %s", taskID)
|
||||
}
|
||||
|
||||
err := fn()
|
||||
|
||||
// 确定任务状态
|
||||
status := TaskStatusCompleted
|
||||
if err != nil {
|
||||
if err == context.Canceled {
|
||||
status = TaskStatusCancelled
|
||||
} else {
|
||||
status = TaskStatusFailed
|
||||
}
|
||||
}
|
||||
|
||||
// 更新任务状态
|
||||
if err != nil {
|
||||
info.SetError(err.Error())
|
||||
} else {
|
||||
info.UpdateStatus(TaskStatusCompleted)
|
||||
}
|
||||
|
||||
// 发送 webhook
|
||||
if info.Webhook != "" {
|
||||
payload := CreateWebhookPayload(taskID, info.Type, status, info.Storage, info.Path, err)
|
||||
SendWebhook(ctx, payload)
|
||||
}
|
||||
|
||||
return err
|
||||
}
|
||||
@@ -90,6 +90,19 @@ func handleAddCallback(ctx *ext.Context, update *ext.Update) error {
|
||||
shortcut.CreateAndAddParsedTaskWithEdit(ctx, selectedStorage, dirPath, data.ParsedItem, msgID, userID)
|
||||
case tasktype.TaskTypeDirectlinks:
|
||||
shortcut.CreateAndAddDirectTaskWithEdit(ctx, selectedStorage, dirPath, data.DirectLinks, msgID, userID)
|
||||
case tasktype.TaskTypeAria2:
|
||||
client := GetAria2Client()
|
||||
if client == nil {
|
||||
ctx.AnswerCallback(msgelem.AlertCallbackAnswer(queryID, i18n.T(i18nk.BotMsgAria2ErrorAria2ClientInitFailed, map[string]any{
|
||||
"Error": "aria2 client not initialized",
|
||||
})))
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
shortcut.CreateAndAddAria2TaskWithEdit(ctx, selectedStorage, dirPath, data.Aria2URIs, client, msgID, userID)
|
||||
case tasktype.TaskTypeYtdlp:
|
||||
shortcut.CreateAndAddYtdlpTaskWithEdit(ctx, selectedStorage, dirPath, data.YtdlpURLs, data.YtdlpFlags, msgID, userID)
|
||||
case tasktype.TaskTypeTransfer:
|
||||
return handleTransferCallback(ctx, userID, selectedStorage, dirPath, data, msgID)
|
||||
default:
|
||||
return fmt.Errorf("unexcept task type: %s", data.TaskType)
|
||||
}
|
||||
|
||||
@@ -10,6 +10,7 @@ import (
|
||||
"github.com/gotd/td/tg"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
"github.com/krau/SaveAny-Bot/database"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/fnamest"
|
||||
"github.com/krau/SaveAny-Bot/pkg/tcbdata"
|
||||
@@ -73,9 +74,9 @@ func handleConfigFnameSTCallback(ctx *ext.Context, update *ext.Update) error {
|
||||
return err
|
||||
}
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: update.CallbackQuery.GetMsgID(),
|
||||
ID: update.CallbackQuery.GetMsgID(),
|
||||
Message: i18n.T(i18nk.BotMsgConfigInfoFilenameStrategySet, map[string]any{
|
||||
"Strategy": fnamest.FnameSTDisplay[st],
|
||||
"Strategy": fnamest.GetDisplay(st, config.C().Lang),
|
||||
}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
@@ -84,7 +85,7 @@ func handleConfigFnameSTCallback(ctx *ext.Context, update *ext.Update) error {
|
||||
buttons := make([]tg.KeyboardButtonClass, 0, len(opts))
|
||||
for _, opt := range opts {
|
||||
buttons = append(buttons, &tg.KeyboardButtonCallback{
|
||||
Text: fnamest.FnameSTDisplay[opt],
|
||||
Text: fnamest.GetDisplay(opt, config.C().Lang),
|
||||
Data: fmt.Appendf(nil, "%s %s %s", tcbdata.TypeConfig, "fnamest", opt),
|
||||
})
|
||||
}
|
||||
@@ -100,9 +101,9 @@ func handleConfigFnameSTCallback(ctx *ext.Context, update *ext.Update) error {
|
||||
currentSt = fnamest.Default
|
||||
}
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: update.CallbackQuery.GetMsgID(),
|
||||
Message: i18n.T(i18nk.BotMsgConfigPromptSelectFilenameStrategy, map[string]any{
|
||||
"Strategy": fnamest.FnameSTDisplay[currentSt],
|
||||
ID: update.CallbackQuery.GetMsgID(),
|
||||
Message: i18n.T(i18nk.BotMsgConfigPromptSelectFilenameStrategy, map[string]any{
|
||||
"Strategy": fnamest.GetDisplay(currentSt, config.C().Lang),
|
||||
}),
|
||||
ReplyMarkup: markup,
|
||||
})
|
||||
|
||||
@@ -3,6 +3,7 @@ package handlers
|
||||
import (
|
||||
"net/url"
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
"github.com/celestix/gotgproto/ext"
|
||||
"github.com/charmbracelet/log"
|
||||
@@ -10,6 +11,8 @@ import (
|
||||
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/msgelem"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
"github.com/krau/SaveAny-Bot/pkg/aria2"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
|
||||
"github.com/krau/SaveAny-Bot/pkg/tcbdata"
|
||||
"github.com/krau/SaveAny-Bot/storage"
|
||||
@@ -50,3 +53,61 @@ func handleDlCmd(ctx *ext.Context, update *ext.Update) error {
|
||||
})
|
||||
return nil
|
||||
}
|
||||
|
||||
var aria2ClientInitOnce sync.Once
|
||||
var aria2ClientInitErr error
|
||||
var aria2Client *aria2.Client
|
||||
|
||||
// GetAria2Client returns the shared aria2 client instance
|
||||
func GetAria2Client() *aria2.Client {
|
||||
return aria2Client
|
||||
}
|
||||
|
||||
func handleAria2DlCmd(ctx *ext.Context, update *ext.Update) error {
|
||||
if !config.C().Aria2.Enable {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgAria2ErrorAria2NotEnabled)), nil)
|
||||
return nil
|
||||
}
|
||||
logger := log.FromContext(ctx)
|
||||
args := strings.Split(update.EffectiveMessage.Text, " ")
|
||||
if len(args) < 2 {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgDlUsage)), nil)
|
||||
return nil
|
||||
}
|
||||
links := args[1:]
|
||||
for i, link := range links {
|
||||
links[i] = strings.TrimSpace(link)
|
||||
}
|
||||
links = slice.Compact(links)
|
||||
if len(links) == 0 {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgDlErrorNoValidLinks)), nil)
|
||||
return nil
|
||||
}
|
||||
logger.Debug("Preparing aria2 download", "links", links)
|
||||
|
||||
// Initialize aria2 client to check connection
|
||||
aria2ClientInitOnce.Do(func() {
|
||||
aria2Client, aria2ClientInitErr = aria2.NewClient(config.C().Aria2.Url, config.C().Aria2.Secret)
|
||||
})
|
||||
if aria2ClientInitErr != nil {
|
||||
logger.Error("Failed to initialize aria2 client", "error", aria2ClientInitErr)
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgAria2ErrorAria2ClientInitFailed, map[string]any{
|
||||
"Error": aria2ClientInitErr.Error(),
|
||||
})), nil)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Build storage selection keyboard (don't add to aria2 yet)
|
||||
markup, err := msgelem.BuildAddSelectStorageKeyboard(storage.GetUserStorages(ctx, update.GetUserChat().GetID()), tcbdata.Add{
|
||||
TaskType: tasktype.TaskTypeAria2,
|
||||
Aria2URIs: links,
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgAria2InfoSelectStorage)), &ext.ReplyOpts{
|
||||
Markup: markup,
|
||||
})
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -114,7 +114,7 @@ func processMediaGroup(ctx *ext.Context, update *ext.Update, groupID int64) {
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to build storage selection keyboard: %s", err)
|
||||
ctx.EditMessage(userId, &tg.MessagesEditMessageRequest{
|
||||
ID: msg.ID,
|
||||
ID: msg.ID,
|
||||
Message: i18n.T(i18nk.BotMsgMediaGroupErrorBuildStorageSelectKeyboardFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
|
||||
@@ -29,15 +29,19 @@ var CommandHandlers = []DescCommandHandler{
|
||||
{"rule", i18nk.BotMsgCmdRule, handleRuleCmd},
|
||||
{"save", i18nk.BotMsgCmdSave, handleSilentMode(handleSaveCmd, handleSilentSaveReplied)},
|
||||
{"dl", i18nk.BotMsgCmdDl, handleDlCmd},
|
||||
{"aria2dl", i18nk.BotMsgCmdAria2dl, handleAria2DlCmd},
|
||||
{"ytdlp", i18nk.BotMsgCmdYtdlp, handleYtdlpCmd},
|
||||
{"transfer", i18nk.BotMsgCmdTransfer, handleTransferCmd},
|
||||
{"task", i18nk.BotMsgCmdTask, handleTaskCmd},
|
||||
{"cancel", i18nk.BotMsgCmdCancel, handleCancelCmd},
|
||||
{"watch", i18nk.BotMsgCmdWatch, handleWatchCmd},
|
||||
{"unwatch", i18nk.BotMsgCmdUnwatch, handleUnwatchCmd},
|
||||
{"lswatch", i18nk.BotMsgCmdLswatch, handleLswatchCmd},
|
||||
{"config", i18nk.BotMsgCmdConfig, handleConfigCmd},
|
||||
{"fnametmpl", i18nk.BotMsgCmdFnametmpl, handleConfigFnameTmpl},
|
||||
{"help", i18nk.BotMsgCmdHelp, handleHelpCmd},
|
||||
{"parser", i18nk.BotMsgCmdParser, handleParserCmd},
|
||||
{"watch", i18nk.BotMsgCmdWatch, handleWatchCmd},
|
||||
{"unwatch", i18nk.BotMsgCmdUnwatch, handleUnwatchCmd},
|
||||
{"lswatch", i18nk.BotMsgCmdLswatch, handleLswatchCmd},
|
||||
{"syncpeers", i18nk.BotMsgCmdSyncpeers, handleSyncpeersCmd},
|
||||
{"update", i18nk.BotMsgCmdUpdate, handleUpdateCmd},
|
||||
}
|
||||
|
||||
|
||||
@@ -7,11 +7,13 @@ import (
|
||||
"github.com/celestix/gotgproto/dispatcher"
|
||||
"github.com/celestix/gotgproto/ext"
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/duke-git/lancet/v2/validator"
|
||||
"github.com/gotd/td/tg"
|
||||
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/dirutil"
|
||||
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/mediautil"
|
||||
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/msgelem"
|
||||
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/shortcut"
|
||||
"github.com/krau/SaveAny-Bot/client/user"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/strutil"
|
||||
@@ -105,7 +107,12 @@ func handleBatchSave(ctx *ext.Context, update *ext.Update, args []string) error
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgCommonErrorInvalidMsgIdRange, map[string]any{"Error": err.Error()})), nil)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
chatID, err := tgutil.ParseChatID(ctx, chatArg)
|
||||
tctx := ctx
|
||||
uctx := user.GetCtx()
|
||||
if uctx != nil && validator.IsIntStr(chatArg) {
|
||||
tctx = uctx
|
||||
}
|
||||
chatID, err := tgutil.ParseChatID(tctx, chatArg)
|
||||
if err != nil {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgCommonErrorInvalidIdOrUsername, map[string]any{"Error": err.Error()})), nil)
|
||||
return dispatcher.EndGroups
|
||||
@@ -118,7 +125,7 @@ func handleBatchSave(ctx *ext.Context, update *ext.Update, args []string) error
|
||||
}
|
||||
|
||||
// [TODO]: generator istead of get all messages
|
||||
msgs, err := tgutil.GetMessagesRange(ctx, chatID, int(startID), int(endID))
|
||||
msgs, err := tgutil.GetMessagesRange(tctx, chatID, int(startID), int(endID))
|
||||
if err != nil {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgCommonErrorGetMessagesFailed, map[string]any{"Error": err.Error()})), nil)
|
||||
return dispatcher.EndGroups
|
||||
@@ -141,7 +148,7 @@ func handleBatchSave(ctx *ext.Context, update *ext.Update, args []string) error
|
||||
if !supported {
|
||||
continue
|
||||
}
|
||||
file, err := tfile.FromMediaMessage(media, ctx.Raw, msg, tfile.WithNameIfEmpty(tgutil.GenFileNameFromMessage(*msg)))
|
||||
file, err := tfile.FromMediaMessage(media, tctx.Raw, msg, tfile.WithNameIfEmpty(tgutil.GenFileNameFromMessage(*msg)))
|
||||
if err != nil {
|
||||
log.FromContext(ctx).Errorf("Failed to get file from message: %s", err)
|
||||
continue
|
||||
@@ -172,14 +179,14 @@ func handleBatchSave(ctx *ext.Context, update *ext.Update, args []string) error
|
||||
if err != nil {
|
||||
log.FromContext(ctx).Errorf("Failed to build storage selection keyboard: %s", err)
|
||||
ctx.EditMessage(update.EffectiveChat().GetID(), &tg.MessagesEditMessageRequest{
|
||||
ID: replied.ID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorBuildStorageSelectKeyboardFailed, map[string]any{"Error": err.Error()}),
|
||||
ID: replied.ID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorBuildStorageSelectKeyboardFailed, map[string]any{"Error": err.Error()}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
ctx.EditMessage(update.EffectiveChat().GetID(), &tg.MessagesEditMessageRequest{
|
||||
ID: replied.ID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonInfoFoundFilesSelectStorage, map[string]any{"Count": len(files)}),
|
||||
ID: replied.ID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonInfoFoundFilesSelectStorage, map[string]any{"Count": len(files)}),
|
||||
ReplyMarkup: markup,
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
|
||||
62
client/bot/handlers/sync_peers.go
Normal file
62
client/bot/handlers/sync_peers.go
Normal file
@@ -0,0 +1,62 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sync"
|
||||
|
||||
"github.com/celestix/gotgproto/dispatcher"
|
||||
"github.com/celestix/gotgproto/ext"
|
||||
"github.com/celestix/gotgproto/storage"
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/gotd/td/telegram/query/dialogs"
|
||||
"github.com/krau/SaveAny-Bot/client/user"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
)
|
||||
|
||||
var syncpeerMu sync.Mutex
|
||||
|
||||
func handleSyncpeersCmd(ctx *ext.Context, u *ext.Update) error {
|
||||
if !config.C().Telegram.Userbot.Enable {
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
syncpeerMu.Lock()
|
||||
defer syncpeerMu.Unlock()
|
||||
uctx := user.GetCtx()
|
||||
if uctx == nil {
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
ctx.Reply(u, ext.ReplyTextString(i18n.T(i18nk.BotMsgSyncpeersStart)), nil)
|
||||
tapi := uctx.Raw
|
||||
peerStorage := uctx.PeerStorage
|
||||
log.FromContext(ctx).Info("Starting to sync peers...")
|
||||
count := 0
|
||||
err := dialogs.NewQueryBuilder(tapi).GetDialogs().BatchSize(50).ForEach(ctx, func(ctx context.Context, e dialogs.Elem) error {
|
||||
for cid, channel := range e.Entities.Channels() {
|
||||
peerStorage.AddPeer(cid, channel.AccessHash, storage.TypeChannel, channel.Username)
|
||||
count++
|
||||
}
|
||||
for uid, user := range e.Entities.Users() {
|
||||
peerStorage.AddPeer(uid, user.AccessHash, storage.TypeUser, user.Username)
|
||||
count++
|
||||
}
|
||||
for gid := range e.Entities.Chats() {
|
||||
peerStorage.AddPeer(gid, storage.DefaultAccessHash, storage.TypeChat, storage.DefaultUsername)
|
||||
count++
|
||||
}
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
log.FromContext(ctx).Error("Failed to sync peers", "error", err)
|
||||
ctx.Reply(u, ext.ReplyTextString(i18n.T(i18nk.BotMsgSyncpeersFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
})), nil)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
log.FromContext(ctx).Info("Finished syncing peers")
|
||||
ctx.Reply(u, ext.ReplyTextString(i18n.T(i18nk.BotMsgSyncpeersSuccess, map[string]any{
|
||||
"Count": count,
|
||||
})), nil)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
@@ -38,7 +38,7 @@ func handleTaskCmd(ctx *ext.Context, update *ext.Update) error {
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
ctx.Reply(update, ext.ReplyTextStyledTextArray([]styling.StyledTextOption{
|
||||
styling.Plain(i18n.T(i18nk.BotMsgTasksCancelRequestedPrefix)),
|
||||
styling.Plain(i18n.T(i18nk.BotMsgTasksCancelRequestedPrefix)),
|
||||
styling.Code(taskID),
|
||||
}), nil)
|
||||
default:
|
||||
|
||||
257
client/bot/handlers/transfer.go
Normal file
257
client/bot/handlers/transfer.go
Normal file
@@ -0,0 +1,257 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"regexp"
|
||||
"strings"
|
||||
|
||||
"github.com/celestix/gotgproto/dispatcher"
|
||||
"github.com/celestix/gotgproto/ext"
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/gotd/td/tg"
|
||||
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/msgelem"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/strutil"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
|
||||
"github.com/krau/SaveAny-Bot/core"
|
||||
"github.com/krau/SaveAny-Bot/core/tasks/transfer"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
|
||||
"github.com/krau/SaveAny-Bot/pkg/storagetypes"
|
||||
"github.com/krau/SaveAny-Bot/pkg/tcbdata"
|
||||
"github.com/krau/SaveAny-Bot/storage"
|
||||
"github.com/rs/xid"
|
||||
)
|
||||
|
||||
func handleTransferCmd(ctx *ext.Context, update *ext.Update) error {
|
||||
logger := log.FromContext(ctx)
|
||||
args := strutil.ParseArgsRespectQuotes(update.EffectiveMessage.Text)
|
||||
|
||||
if len(args) < 2 {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgTransferUsage, nil)), nil)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
// Parse source: storage_name:/path
|
||||
sourceParts := strings.SplitN(args[1], ":", 2)
|
||||
if len(sourceParts) != 2 {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgTransferErrorInvalidSource, nil)), nil)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
sourceStorageName := sourceParts[0]
|
||||
sourcePath := sourceParts[1]
|
||||
|
||||
userID := update.GetUserChat().GetID()
|
||||
|
||||
// Get source storage
|
||||
sourceStorage, err := storage.GetStorageByUserIDAndName(ctx, userID, sourceStorageName)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to get source storage by user ID and name: %s", err)
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgTransferErrorStorageNotFound, map[string]any{
|
||||
"StorageName": sourceStorageName,
|
||||
"Error": err,
|
||||
})), nil)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
// Check if source storage supports listing
|
||||
listable, ok := sourceStorage.(storage.StorageListable)
|
||||
if !ok {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgTransferErrorStorageNotListable, map[string]any{
|
||||
"StorageName": sourceStorageName,
|
||||
})), nil)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
// Check if source storage supports reading
|
||||
_, ok = sourceStorage.(storage.StorageReadable)
|
||||
if !ok {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgTransferErrorStorageNotReadable, map[string]any{
|
||||
"StorageName": sourceStorageName,
|
||||
})), nil)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
// Fetch file list
|
||||
replied, err := ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgTransferInfoFetchingFiles, nil)), nil)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to reply: %s", err)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
files, err := listable.ListFiles(ctx, sourcePath)
|
||||
if err != nil {
|
||||
ctx.EditMessage(update.EffectiveChat().GetID(), &tg.MessagesEditMessageRequest{
|
||||
ID: replied.ID,
|
||||
Message: i18n.T(i18nk.BotMsgTransferErrorListFilesFailed, map[string]any{"Error": err}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
// Optional filter
|
||||
var filter *regexp.Regexp
|
||||
if len(args) >= 3 {
|
||||
filter, err = regexp.Compile(args[2])
|
||||
if err != nil {
|
||||
ctx.EditMessage(update.EffectiveChat().GetID(), &tg.MessagesEditMessageRequest{
|
||||
ID: replied.ID,
|
||||
Message: i18n.T(i18nk.BotMsgTransferErrorInvalidRegex, map[string]any{"Error": err}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
}
|
||||
|
||||
// Filter files
|
||||
filteredFiles := make([]storagetypes.FileInfo, 0)
|
||||
for _, file := range files {
|
||||
if file.IsDir {
|
||||
continue
|
||||
}
|
||||
if filter != nil && !filter.MatchString(file.Name) {
|
||||
continue
|
||||
}
|
||||
filteredFiles = append(filteredFiles, file)
|
||||
}
|
||||
|
||||
if len(filteredFiles) == 0 {
|
||||
ctx.EditMessage(update.EffectiveChat().GetID(), &tg.MessagesEditMessageRequest{
|
||||
ID: replied.ID,
|
||||
Message: i18n.T(i18nk.BotMsgTransferErrorNoFilesToTransfer, nil),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
// Prepare file paths for callback data
|
||||
filePaths := make([]string, 0, len(filteredFiles))
|
||||
var totalSize int64
|
||||
for _, file := range filteredFiles {
|
||||
filePaths = append(filePaths, file.Path)
|
||||
totalSize += file.Size
|
||||
}
|
||||
|
||||
// Build storage selection keyboard
|
||||
markup, err := msgelem.BuildAddSelectStorageKeyboard(storage.GetUserStorages(ctx, userID), tcbdata.Add{
|
||||
TaskType: tasktype.TaskTypeTransfer,
|
||||
TransferSourceStorName: sourceStorageName,
|
||||
TransferSourcePath: sourcePath,
|
||||
TransferFiles: filePaths,
|
||||
})
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to build storage selection keyboard: %s", err)
|
||||
ctx.EditMessage(update.EffectiveChat().GetID(), &tg.MessagesEditMessageRequest{
|
||||
ID: replied.ID,
|
||||
Message: i18n.T(i18nk.BotMsgTransferErrorBuildStorageSelectKeyboardFailed, map[string]any{"Error": err}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
ctx.EditMessage(update.EffectiveChat().GetID(), &tg.MessagesEditMessageRequest{
|
||||
ID: replied.ID,
|
||||
Message: i18n.T(i18nk.BotMsgTransferInfoFilesSelectStorage, map[string]any{
|
||||
"Count": len(filteredFiles),
|
||||
"SizeMB": fmt.Sprintf("%.2f", float64(totalSize)/(1024*1024)),
|
||||
}),
|
||||
ReplyMarkup: markup,
|
||||
})
|
||||
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
func handleTransferCallback(ctx *ext.Context, userID int64, targetStorage storage.Storage, dirPath string, data tcbdata.Add, msgID int) error {
|
||||
logger := log.FromContext(ctx)
|
||||
|
||||
// Get source storage
|
||||
sourceStorage, err := storage.GetStorageByUserIDAndName(ctx, userID, data.TransferSourceStorName)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to get source storage: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgTransferErrorStorageNotFound, map[string]any{"StorageName": data.TransferSourceStorName, "Error": err}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
// Check if source storage supports listing
|
||||
listable, ok := sourceStorage.(storage.StorageListable)
|
||||
if !ok {
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgTransferErrorStorageNotListable, map[string]any{"StorageName": data.TransferSourceStorName}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
// Re-fetch files to get FileInfo (since we only stored paths)
|
||||
// This is necessary to get size and other metadata
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgTransferInfoFetchingFiles, nil),
|
||||
})
|
||||
|
||||
allFiles, err := listable.ListFiles(ctx, data.TransferSourcePath)
|
||||
if err != nil {
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgTransferErrorListFilesFailed, map[string]any{"Error": err}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
// Create a map for quick lookup
|
||||
fileMap := make(map[string]storagetypes.FileInfo)
|
||||
for _, file := range allFiles {
|
||||
fileMap[file.Path] = file
|
||||
}
|
||||
|
||||
// Build task elements for the selected files
|
||||
elems := make([]transfer.TaskElement, 0, len(data.TransferFiles))
|
||||
var totalSize int64
|
||||
for _, filePath := range data.TransferFiles {
|
||||
fileInfo, ok := fileMap[filePath]
|
||||
if !ok {
|
||||
logger.Warnf("File not found in source storage: %s", filePath)
|
||||
continue
|
||||
}
|
||||
elem := transfer.NewTaskElement(sourceStorage, fileInfo, targetStorage, dirPath)
|
||||
elems = append(elems, *elem)
|
||||
totalSize += fileInfo.Size
|
||||
}
|
||||
|
||||
if len(elems) == 0 {
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgTransferErrorNoFilesToTransfer, nil),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
// Create and add task
|
||||
taskID := xid.New().String()
|
||||
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
|
||||
task := transfer.NewTransferTask(
|
||||
taskID,
|
||||
injectCtx,
|
||||
elems,
|
||||
transfer.NewProgressTracker(msgID, userID),
|
||||
true, // IgnoreErrors
|
||||
)
|
||||
|
||||
if err := core.AddTask(injectCtx, task); err != nil {
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgTransferErrorAddTaskFailed, map[string]any{"Error": err}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgTransferInfoTaskAdded, map[string]any{
|
||||
"Count": len(elems),
|
||||
"SizeMB": fmt.Sprintf("%.2f", float64(totalSize)/(1024*1024)),
|
||||
"TaskID": taskID,
|
||||
}),
|
||||
})
|
||||
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
@@ -103,7 +103,7 @@ func handleUpdateCallback(ctx *ext.Context, u *ext.Update) error {
|
||||
return err
|
||||
}
|
||||
ctx.EditMessage(u.GetUserChat().GetID(), &tg.MessagesEditMessageRequest{
|
||||
ID: u.CallbackQuery.GetMsgID(),
|
||||
ID: u.CallbackQuery.GetMsgID(),
|
||||
Message: i18n.T(i18nk.BotMsgUpdateInfoUpgradingWithVersion, map[string]any{
|
||||
"Current": config.Version,
|
||||
}),
|
||||
@@ -111,7 +111,7 @@ func handleUpdateCallback(ctx *ext.Context, u *ext.Update) error {
|
||||
latest, err := ghselfupdate.UpdateSelf(currentV, config.GitRepo)
|
||||
if err != nil {
|
||||
ctx.EditMessage(u.GetUserChat().GetID(), &tg.MessagesEditMessageRequest{
|
||||
ID: u.CallbackQuery.GetMsgID(),
|
||||
ID: u.CallbackQuery.GetMsgID(),
|
||||
Message: i18n.T(i18nk.BotMsgUpdateErrorUpgradeFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
@@ -119,7 +119,7 @@ func handleUpdateCallback(ctx *ext.Context, u *ext.Update) error {
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
ctx.EditMessage(u.GetUserChat().GetID(), &tg.MessagesEditMessageRequest{
|
||||
ID: u.CallbackQuery.GetMsgID(),
|
||||
ID: u.CallbackQuery.GetMsgID(),
|
||||
Message: i18n.T(i18nk.BotMsgUpdateInfoUpgradeSuccess, map[string]any{
|
||||
"Version": latest.Version.String(),
|
||||
}),
|
||||
|
||||
@@ -30,6 +30,7 @@ type FilenameTemplateData struct {
|
||||
MsgTags string `json:"msgtags,omitempty"`
|
||||
MsgGen string `json:"msggen,omitempty"`
|
||||
MsgDate string `json:"msgdate,omitempty"`
|
||||
MsgRaw string `json:"msgraw,omitempty"`
|
||||
OrigName string `json:"origname,omitempty"`
|
||||
ChatID string `json:"chatid,omitempty"`
|
||||
}
|
||||
@@ -39,6 +40,7 @@ func (f FilenameTemplateData) ToMap() map[string]string {
|
||||
"msgid": f.MsgID,
|
||||
"msgtags": f.MsgTags,
|
||||
"msggen": f.MsgGen,
|
||||
"msgraw": f.MsgRaw,
|
||||
"msgdate": f.MsgDate,
|
||||
"origname": f.OrigName,
|
||||
"chatid": f.ChatID,
|
||||
@@ -108,8 +110,10 @@ func BuildFilenameTemplateData(message *tg.Message) map[string]string {
|
||||
t := time.Unix(int64(date), 0)
|
||||
return t.Format("2006-01-02_15-04-05")
|
||||
}(),
|
||||
MsgRaw: message.GetMessage(),
|
||||
ChatID: func() string {
|
||||
// 如果消息是频道的(从消息链接中fetch的) 直接使用其chat id, 无论它是否是从其他来源转发的
|
||||
// 如果消息是频道的(从消息链接中fetch的) 直接使用其chat id,
|
||||
// 无论它是否是从其他来源转发的
|
||||
if message.GetPost() {
|
||||
peer := message.GetPeerID()
|
||||
switch p := peer.(type) {
|
||||
|
||||
@@ -26,7 +26,7 @@ func BuildDirHelpStyling(dirs []database.Dir) []styling.StyledTextOption {
|
||||
styling.Blockquote(func() string {
|
||||
var sb strings.Builder
|
||||
for _, dir := range dirs {
|
||||
sb.WriteString(fmt.Sprintf("%d: ", dir.ID))
|
||||
fmt.Fprintf(&sb, "%d: ", dir.ID)
|
||||
sb.WriteString(dir.StorageName)
|
||||
sb.WriteString(" - ")
|
||||
sb.WriteString(dir.Path)
|
||||
|
||||
@@ -49,6 +49,14 @@ func BuildAddSelectStorageKeyboard(stors []storage.Storage, adddata tcbdata.Add)
|
||||
ParsedItem: adddata.ParsedItem,
|
||||
|
||||
DirectLinks: adddata.DirectLinks,
|
||||
|
||||
Aria2URIs: adddata.Aria2URIs,
|
||||
YtdlpURLs: adddata.YtdlpURLs,
|
||||
YtdlpFlags: adddata.YtdlpFlags,
|
||||
|
||||
TransferSourceStorName: adddata.TransferSourceStorName,
|
||||
TransferSourcePath: adddata.TransferSourcePath,
|
||||
TransferFiles: adddata.TransferFiles,
|
||||
}
|
||||
dataid := xid.New().String()
|
||||
err := cache.Set(dataid, data)
|
||||
|
||||
65
client/bot/handlers/utils/shortcut/aria2.go
Normal file
65
client/bot/handlers/utils/shortcut/aria2.go
Normal file
@@ -0,0 +1,65 @@
|
||||
package shortcut
|
||||
|
||||
import (
|
||||
"github.com/celestix/gotgproto/dispatcher"
|
||||
"github.com/celestix/gotgproto/ext"
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/gotd/td/tg"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
|
||||
"github.com/krau/SaveAny-Bot/core"
|
||||
"github.com/krau/SaveAny-Bot/core/tasks/aria2dl"
|
||||
"github.com/krau/SaveAny-Bot/pkg/aria2"
|
||||
"github.com/krau/SaveAny-Bot/storage"
|
||||
"github.com/rs/xid"
|
||||
)
|
||||
|
||||
func CreateAndAddAria2TaskWithEdit(ctx *ext.Context, stor storage.Storage, dirPath string, uris []string, aria2Client *aria2.Client, msgID int, userID int64) error {
|
||||
logger := log.FromContext(ctx)
|
||||
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
|
||||
|
||||
// Now add to aria2 after user selected storage
|
||||
logger.Infof("Adding download to aria2, uris type: %T, value: %+v", uris, uris)
|
||||
|
||||
// Ensure uris is valid
|
||||
if len(uris) == 0 {
|
||||
logger.Error("URIs list is empty")
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgDlErrorNoValidLinks, nil),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
gid, err := aria2Client.AddURI(ctx, uris, nil)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to add aria2 download: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgAria2ErrorAddingAria2Download, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
logger.Infof("Aria2 download added with GID: %s", gid)
|
||||
|
||||
// Create task with the GID
|
||||
task := aria2dl.NewTask(xid.New().String(), injectCtx, gid, uris, aria2Client, stor, dirPath, aria2dl.NewProgress(msgID, userID))
|
||||
if err := core.AddTask(injectCtx, task); err != nil {
|
||||
logger.Errorf("Failed to add task: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorTaskAddFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonInfoTaskAdded, nil),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
@@ -16,7 +16,7 @@ import (
|
||||
|
||||
func CreateAndAddDirectTaskWithEdit(ctx *ext.Context, stor storage.Storage, dirPath string, links []string, msgID int, userID int64) error {
|
||||
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
|
||||
task := directlinks.NewTask(xid.New().String(), injectCtx, links, stor, stor.JoinStoragePath(dirPath), directlinks.NewProgress(msgID, userID))
|
||||
task := directlinks.NewTask(xid.New().String(), injectCtx, links, stor, dirPath, directlinks.NewProgress(msgID, userID))
|
||||
if err := core.AddTask(injectCtx, task); err != nil {
|
||||
log.FromContext(ctx).Errorf("Failed to add task: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
|
||||
@@ -18,11 +18,11 @@ import (
|
||||
|
||||
func CreateAndAddParsedTaskWithEdit(ctx *ext.Context, stor storage.Storage, dirPath string, item *parser.Item, msgID int, userID int64) error {
|
||||
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
|
||||
task := parsed.NewTask(xid.New().String(), injectCtx, stor, stor.JoinStoragePath(dirPath), item, parsed.NewProgress(msgID, userID))
|
||||
task := parsed.NewTask(xid.New().String(), injectCtx, stor, dirPath, item, parsed.NewProgress(msgID, userID))
|
||||
if err := core.AddTask(injectCtx, task); err != nil {
|
||||
log.FromContext(ctx).Errorf("Failed to add task: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorTaskAddFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
|
||||
@@ -29,7 +29,7 @@ func CreateAndAddTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor storage
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to get user by chat ID: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: trackMsgID,
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorGetUserWithErrFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
@@ -49,7 +49,7 @@ func CreateAndAddTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor storage
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to get storage by user ID and name: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: trackMsgID,
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorGetStorageFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
@@ -59,7 +59,7 @@ func CreateAndAddTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor storage
|
||||
}
|
||||
}
|
||||
startCreateTask:
|
||||
storagePath := stor.JoinStoragePath(path.Join(dirPath, file.Name()))
|
||||
storagePath := path.Join(dirPath, file.Name())
|
||||
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
|
||||
taskid := xid.New().String()
|
||||
task, err := tftask.NewTGFileTask(taskid, injectCtx, file, stor, storagePath,
|
||||
@@ -69,7 +69,7 @@ startCreateTask:
|
||||
if err != nil {
|
||||
logger.Errorf("create task failed: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: trackMsgID,
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorTaskCreateFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
@@ -79,7 +79,7 @@ startCreateTask:
|
||||
if err := core.AddTask(injectCtx, task); err != nil {
|
||||
logger.Errorf("add task failed: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: trackMsgID,
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorTaskAddFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
@@ -103,7 +103,7 @@ func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor st
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to get user by chat ID: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: trackMsgID,
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorGetUserWithErrFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
@@ -142,7 +142,7 @@ func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor st
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to get storage by user ID and name: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: trackMsgID,
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorGetStorageFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
@@ -151,15 +151,15 @@ func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor st
|
||||
}
|
||||
}
|
||||
if !dirPath.NeedNewForAlbum() {
|
||||
storPath := fileStor.JoinStoragePath(path.Join(dirPath.String(), file.Name()))
|
||||
storPath := path.Join(dirPath.String(), file.Name())
|
||||
elem, err := batchtfile.NewTaskElement(fileStor, storPath, file)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to create task element: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorTaskCreateFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorTaskCreateFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
@@ -188,12 +188,12 @@ func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor st
|
||||
albumDir := strings.TrimSuffix(path.Base(afiles[0].file.Name()), path.Ext(afiles[0].file.Name()))
|
||||
albumStor := afiles[0].storage
|
||||
for _, af := range afiles {
|
||||
afstorPath := af.storage.JoinStoragePath(path.Join(dirPath, albumDir, af.file.Name()))
|
||||
afstorPath := path.Join(dirPath, albumDir, af.file.Name())
|
||||
elem, err := batchtfile.NewTaskElement(albumStor, afstorPath, af.file)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to create task element for album file: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: trackMsgID,
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorTaskCreateFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
@@ -210,7 +210,7 @@ func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor st
|
||||
if err := core.AddTask(injectCtx, task); err != nil {
|
||||
logger.Errorf("Failed to add batch task: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: trackMsgID,
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorTaskAddFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
@@ -218,8 +218,8 @@ func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor st
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonInfoBatchTasksAdded, map[string]any{
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonInfoBatchTasksAdded, map[string]any{
|
||||
"Count": len(files),
|
||||
}),
|
||||
ReplyMarkup: nil,
|
||||
|
||||
@@ -25,21 +25,21 @@ func CreateAndAddtelegraphWithEdit(
|
||||
pics []string,
|
||||
stor storage.Storage,
|
||||
trackMsgID int) error {
|
||||
|
||||
|
||||
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
|
||||
task := tphtask.NewTask(xid.New().String(),
|
||||
injectCtx,
|
||||
tphpage.Path,
|
||||
pics,
|
||||
stor,
|
||||
stor.JoinStoragePath(dirPath),
|
||||
dirPath,
|
||||
tphutil.DefaultClient(),
|
||||
tphtask.NewProgress(trackMsgID, userID),
|
||||
)
|
||||
if err := core.AddTask(injectCtx, task); err != nil {
|
||||
log.FromContext(ctx).Errorf("Failed to add task: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: trackMsgID,
|
||||
ID: trackMsgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorTaskAddFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
|
||||
63
client/bot/handlers/utils/shortcut/ytdlp.go
Normal file
63
client/bot/handlers/utils/shortcut/ytdlp.go
Normal file
@@ -0,0 +1,63 @@
|
||||
package shortcut
|
||||
|
||||
import (
|
||||
"github.com/celestix/gotgproto/dispatcher"
|
||||
"github.com/celestix/gotgproto/ext"
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/gotd/td/tg"
|
||||
"github.com/rs/xid"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
|
||||
"github.com/krau/SaveAny-Bot/core"
|
||||
"github.com/krau/SaveAny-Bot/core/tasks/ytdlp"
|
||||
"github.com/krau/SaveAny-Bot/storage"
|
||||
)
|
||||
|
||||
func CreateAndAddYtdlpTaskWithEdit(ctx *ext.Context, stor storage.Storage, dirPath string, urls []string, flags []string, msgID int, userID int64) error {
|
||||
logger := log.FromContext(ctx)
|
||||
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
|
||||
|
||||
// Validate URLs
|
||||
if len(urls) == 0 {
|
||||
logger.Error("URLs list is empty")
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgYtdlpErrorNoValidUrls, nil),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
logger.Infof("Creating yt-dlp task for %d URL(s) with %d flag(s)", len(urls), len(flags))
|
||||
|
||||
// Create yt-dlp task
|
||||
task := ytdlp.NewTask(
|
||||
xid.New().String(),
|
||||
injectCtx,
|
||||
urls,
|
||||
flags,
|
||||
stor,
|
||||
dirPath,
|
||||
ytdlp.NewProgress(msgID, userID),
|
||||
)
|
||||
|
||||
// Add task to queue
|
||||
if err := core.AddTask(injectCtx, task); err != nil {
|
||||
logger.Errorf("Failed to add yt-dlp task: %s", err)
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonErrorTaskAddFailed, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
})
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
|
||||
ID: msgID,
|
||||
Message: i18n.T(i18nk.BotMsgCommonInfoTaskAdded, nil),
|
||||
})
|
||||
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
@@ -5,7 +5,9 @@ import (
|
||||
"path"
|
||||
"regexp"
|
||||
"strings"
|
||||
"sync"
|
||||
"text/template"
|
||||
"time"
|
||||
|
||||
"github.com/celestix/gotgproto/dispatcher"
|
||||
"github.com/celestix/gotgproto/ext"
|
||||
@@ -16,10 +18,12 @@ import (
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
"github.com/krau/SaveAny-Bot/core"
|
||||
"github.com/krau/SaveAny-Bot/core/tasks/tfile"
|
||||
coretfile "github.com/krau/SaveAny-Bot/core/tasks/tfile"
|
||||
"github.com/krau/SaveAny-Bot/database"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/fnamest"
|
||||
"github.com/krau/SaveAny-Bot/pkg/tfile"
|
||||
"github.com/krau/SaveAny-Bot/storage"
|
||||
"github.com/rs/xid"
|
||||
)
|
||||
@@ -151,6 +155,53 @@ func handleUnwatchCmd(ctx *ext.Context, update *ext.Update) error {
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
type watchMediaGroupHandler struct {
|
||||
groups map[int64]map[uint][]tfile.TGFileMessage // chatID -> userID -> files
|
||||
timers map[int64]map[uint]*time.Timer
|
||||
mu sync.Mutex
|
||||
}
|
||||
|
||||
var watchMediaGroupMgr = &watchMediaGroupHandler{
|
||||
groups: make(map[int64]map[uint][]tfile.TGFileMessage),
|
||||
timers: make(map[int64]map[uint]*time.Timer),
|
||||
}
|
||||
|
||||
func (w *watchMediaGroupHandler) addFile(chatID int64, userID uint, file tfile.TGFileMessage, timeout time.Duration, callback func([]tfile.TGFileMessage)) {
|
||||
w.mu.Lock()
|
||||
defer w.mu.Unlock()
|
||||
|
||||
if w.groups[chatID] == nil {
|
||||
w.groups[chatID] = make(map[uint][]tfile.TGFileMessage)
|
||||
}
|
||||
if w.timers[chatID] == nil {
|
||||
w.timers[chatID] = make(map[uint]*time.Timer)
|
||||
}
|
||||
|
||||
if timer, exists := w.timers[chatID][userID]; exists {
|
||||
timer.Stop()
|
||||
}
|
||||
|
||||
w.groups[chatID][userID] = append(w.groups[chatID][userID], file)
|
||||
|
||||
w.timers[chatID][userID] = time.AfterFunc(timeout, func() {
|
||||
w.mu.Lock()
|
||||
files := w.groups[chatID][userID]
|
||||
delete(w.groups[chatID], userID)
|
||||
delete(w.timers[chatID], userID)
|
||||
if len(w.groups[chatID]) == 0 {
|
||||
delete(w.groups, chatID)
|
||||
}
|
||||
if len(w.timers[chatID]) == 0 {
|
||||
delete(w.timers, chatID)
|
||||
}
|
||||
w.mu.Unlock()
|
||||
|
||||
if len(files) > 0 {
|
||||
callback(files)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func listenMediaMessageEvent(ch chan userclient.MediaMessageEvent) {
|
||||
if userclient.GetCtx() == nil {
|
||||
return
|
||||
@@ -201,6 +252,16 @@ func listenMediaMessageEvent(ch chan userclient.MediaMessageEvent) {
|
||||
logger.Errorf("Failed to get storage by user ID %d and name %s: %v", user.ChatID, user.DefaultStorage, err)
|
||||
continue
|
||||
}
|
||||
// Resolve the default directory path from user.DefaultDir
|
||||
var defaultDirPath string
|
||||
if user.DefaultDir != 0 {
|
||||
dir, err := database.GetDirByID(ctx, user.DefaultDir)
|
||||
if err != nil {
|
||||
logger.Warnf("Failed to get default dir for user %d: %v, using root", user.ChatID, err)
|
||||
} else {
|
||||
defaultDirPath = dir.Path
|
||||
}
|
||||
}
|
||||
switch user.FilenameStrategy {
|
||||
case fnamest.Message.String():
|
||||
file.SetName(tgutil.GenFileNameFromMessage(*file.Message()))
|
||||
@@ -224,7 +285,25 @@ func listenMediaMessageEvent(ch chan userclient.MediaMessageEvent) {
|
||||
}
|
||||
file.SetName(sb.String())
|
||||
}
|
||||
var dirPath string
|
||||
|
||||
// Check if this is a media group and if rules specify NEW-FOR-ALBUM
|
||||
groupID, isGroup := file.Message().GetGroupedID()
|
||||
needAlbumHandling := false
|
||||
if isGroup && groupID != 0 && user.ApplyRule && user.Rules != nil {
|
||||
_, _, matchedDirPath := ruleutil.ApplyRule(ctx, user.Rules, ruleutil.NewInput(file))
|
||||
needAlbumHandling = matchedDirPath.NeedNewForAlbum()
|
||||
}
|
||||
|
||||
if needAlbumHandling {
|
||||
// For media groups with NEW-FOR-ALBUM rule, collect all files of the same group
|
||||
watchMediaGroupMgr.addFile(event.ChatID, user.ID, file, time.Duration(max(config.C().Telegram.MediaGroupTimeout, 1))*time.Second, func(files []tfile.TGFileMessage) {
|
||||
processWatchMediaGroup(ctx, user, stor, defaultDirPath, files)
|
||||
})
|
||||
continue
|
||||
}
|
||||
|
||||
// Process single file or media group without album folder creation
|
||||
dirPath := defaultDirPath
|
||||
if user.ApplyRule && user.Rules != nil {
|
||||
matched, matchedStorageName, matchedDirPath := ruleutil.ApplyRule(ctx, user.Rules, ruleutil.NewInput(file))
|
||||
if !matched {
|
||||
@@ -240,10 +319,10 @@ func listenMediaMessageEvent(ch chan userclient.MediaMessageEvent) {
|
||||
}
|
||||
}
|
||||
startCreateTask:
|
||||
storagePath := stor.JoinStoragePath(path.Join(dirPath, file.Name()))
|
||||
storagePath := path.Join(dirPath, file.Name())
|
||||
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
|
||||
taskid := xid.New().String()
|
||||
task, err := tfile.NewTGFileTask(taskid, injectCtx, file, stor, storagePath, nil)
|
||||
task, err := coretfile.NewTGFileTask(taskid, injectCtx, file, stor, storagePath, nil)
|
||||
if err != nil {
|
||||
logger.Errorf("create task failed: %s", err)
|
||||
continue
|
||||
@@ -256,3 +335,101 @@ func listenMediaMessageEvent(ch chan userclient.MediaMessageEvent) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func processWatchMediaGroup(ctx *ext.Context, user *database.User, stor storage.Storage, dirPath string, files []tfile.TGFileMessage) {
|
||||
logger := log.FromContext(ctx)
|
||||
if len(files) == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
useRule := user.ApplyRule && user.Rules != nil
|
||||
|
||||
applyRule := func(file tfile.TGFileMessage) (string, ruleutil.MatchedDirPath) {
|
||||
if !useRule {
|
||||
return stor.Name(), ruleutil.MatchedDirPath(dirPath)
|
||||
}
|
||||
matched, storName, dirP := ruleutil.ApplyRule(ctx, user.Rules, ruleutil.NewInput(file))
|
||||
if !matched {
|
||||
return stor.Name(), ruleutil.MatchedDirPath(dirPath)
|
||||
}
|
||||
storname := storName.String()
|
||||
if !storName.Usable() {
|
||||
storname = stor.Name()
|
||||
}
|
||||
return storname, dirP
|
||||
}
|
||||
|
||||
type albumFile struct {
|
||||
file tfile.TGFileMessage
|
||||
storage storage.Storage
|
||||
dirPath string
|
||||
}
|
||||
albumFiles := make(map[int64][]albumFile)
|
||||
|
||||
// Collect files by group ID
|
||||
for _, file := range files {
|
||||
storName, ruleDirPath := applyRule(file)
|
||||
fileStor := stor
|
||||
if storName != stor.Name() && storName != "" {
|
||||
var err error
|
||||
fileStor, err = storage.GetStorageByUserIDAndName(ctx, user.ChatID, storName)
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to get storage by user ID and name: %s", err)
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
groupId, isGroup := file.Message().GetGroupedID()
|
||||
if !isGroup || groupId == 0 {
|
||||
logger.Warnf("File %s is not in a group, skipping", file.Name())
|
||||
continue
|
||||
}
|
||||
|
||||
// Use the effective dirPath: if rule returns NEW-FOR-ALBUM sentinel, fall back to the
|
||||
// base dirPath passed in (which is defaultDirPath from the caller).
|
||||
effectiveDirPath := string(ruleDirPath)
|
||||
if ruleDirPath.NeedNewForAlbum() {
|
||||
effectiveDirPath = dirPath
|
||||
}
|
||||
|
||||
if _, ok := albumFiles[groupId]; !ok {
|
||||
albumFiles[groupId] = make([]albumFile, 0)
|
||||
}
|
||||
albumFiles[groupId] = append(albumFiles[groupId], albumFile{
|
||||
file: file,
|
||||
storage: fileStor,
|
||||
dirPath: effectiveDirPath,
|
||||
})
|
||||
}
|
||||
|
||||
// Process album files with folder creation
|
||||
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
|
||||
totalTasks := 0
|
||||
for groupID, afiles := range albumFiles {
|
||||
if len(afiles) <= 1 {
|
||||
continue
|
||||
}
|
||||
|
||||
// Use first file's name (without extension) as album folder name
|
||||
albumDir := strings.TrimSuffix(path.Base(afiles[0].file.Name()), path.Ext(afiles[0].file.Name()))
|
||||
albumStor := afiles[0].storage
|
||||
|
||||
logger.Infof("Creating album folder for group %d: %s with %d files", groupID, albumDir, len(afiles))
|
||||
|
||||
for _, af := range afiles {
|
||||
afstorPath := path.Join(af.dirPath, albumDir, af.file.Name())
|
||||
taskid := xid.New().String()
|
||||
task, err := coretfile.NewTGFileTask(taskid, injectCtx, af.file, albumStor, afstorPath, nil)
|
||||
if err != nil {
|
||||
logger.Errorf("create task failed for album file: %s", err)
|
||||
continue
|
||||
}
|
||||
if err := core.AddTask(injectCtx, task); err != nil {
|
||||
logger.Errorf("add task failed: %s", err)
|
||||
continue
|
||||
}
|
||||
totalTasks++
|
||||
}
|
||||
}
|
||||
logger.Infof("Added %d watch media tasks for user %d", totalTasks, user.ChatID)
|
||||
}
|
||||
|
||||
92
client/bot/handlers/ytdlp.go
Normal file
92
client/bot/handlers/ytdlp.go
Normal file
@@ -0,0 +1,92 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"net/url"
|
||||
"strings"
|
||||
|
||||
"github.com/celestix/gotgproto/dispatcher"
|
||||
"github.com/celestix/gotgproto/ext"
|
||||
"github.com/charmbracelet/log"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/msgelem"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
|
||||
"github.com/krau/SaveAny-Bot/pkg/tcbdata"
|
||||
"github.com/krau/SaveAny-Bot/storage"
|
||||
)
|
||||
|
||||
func handleYtdlpCmd(ctx *ext.Context, update *ext.Update) error {
|
||||
logger := log.FromContext(ctx)
|
||||
args := strings.Split(update.EffectiveMessage.Text, " ")
|
||||
if len(args) < 2 {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgYtdlpUsage)), nil)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
// Separate URLs and flags from arguments
|
||||
var urls []string
|
||||
var flags []string
|
||||
|
||||
for i := 1; i < len(args); i++ {
|
||||
arg := strings.TrimSpace(args[i])
|
||||
if arg == "" {
|
||||
continue
|
||||
}
|
||||
|
||||
// Check if it's a flag (starts with - or --)
|
||||
if strings.HasPrefix(arg, "-") {
|
||||
flags = append(flags, arg)
|
||||
// Check if the next argument might be a value for this flag
|
||||
// Don't consume it if it starts with - or looks like a URL with scheme
|
||||
if i+1 < len(args) {
|
||||
nextArg := strings.TrimSpace(args[i+1])
|
||||
if nextArg != "" && !strings.HasPrefix(nextArg, "-") {
|
||||
// Check if it's clearly a URL (has ://)
|
||||
// This handles common video URLs (http://, https://)
|
||||
// For other yt-dlp inputs, users should ensure proper formatting
|
||||
if strings.Contains(nextArg, "://") {
|
||||
// It's a URL, don't consume it as a flag value
|
||||
continue
|
||||
}
|
||||
// Otherwise, treat it as a flag value
|
||||
flags = append(flags, nextArg)
|
||||
i++ // Skip the next argument as it's been consumed
|
||||
}
|
||||
}
|
||||
} else {
|
||||
// Try to parse as URL
|
||||
u, err := url.Parse(arg)
|
||||
if err != nil || u.Scheme == "" || u.Host == "" {
|
||||
logger.Warnf("Invalid URL: %s", arg)
|
||||
continue
|
||||
}
|
||||
urls = append(urls, arg)
|
||||
}
|
||||
}
|
||||
|
||||
if len(urls) == 0 {
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgYtdlpErrorNoValidUrls)), nil)
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
|
||||
logger.Debugf("Preparing yt-dlp download for %d URL(s) with %d flag(s)", len(urls), len(flags))
|
||||
|
||||
// Build storage selection keyboard
|
||||
markup, err := msgelem.BuildAddSelectStorageKeyboard(storage.GetUserStorages(ctx, update.GetUserChat().GetID()), tcbdata.Add{
|
||||
TaskType: tasktype.TaskTypeYtdlp,
|
||||
YtdlpURLs: urls,
|
||||
YtdlpFlags: flags,
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgYtdlpInfoUrlsSelectStorage, map[string]any{
|
||||
"Count": len(urls),
|
||||
})), &ext.ReplyOpts{
|
||||
Markup: markup,
|
||||
})
|
||||
|
||||
return dispatcher.EndGroups
|
||||
}
|
||||
129
client/bot/handlers/ytdlp_test.go
Normal file
129
client/bot/handlers/ytdlp_test.go
Normal file
@@ -0,0 +1,129 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"net/url"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// TestYtdlpArgumentParsing tests the URL and flag separation logic
|
||||
func TestYtdlpArgumentParsing(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
input string
|
||||
expectedURLs []string
|
||||
expectedFlags []string
|
||||
}{
|
||||
{
|
||||
name: "Single URL without flags",
|
||||
input: "/ytdlp https://example.com/video",
|
||||
expectedURLs: []string{"https://example.com/video"},
|
||||
expectedFlags: []string{},
|
||||
},
|
||||
{
|
||||
name: "Multiple URLs without flags",
|
||||
input: "/ytdlp https://example.com/v1 https://example.com/v2",
|
||||
expectedURLs: []string{"https://example.com/v1", "https://example.com/v2"},
|
||||
expectedFlags: []string{},
|
||||
},
|
||||
{
|
||||
name: "URL with format flag",
|
||||
input: "/ytdlp --format best https://example.com/video",
|
||||
expectedURLs: []string{"https://example.com/video"},
|
||||
expectedFlags: []string{"--format", "best"},
|
||||
},
|
||||
{
|
||||
name: "URL with extract-audio flag",
|
||||
input: "/ytdlp --extract-audio --audio-format mp3 https://example.com/video",
|
||||
expectedURLs: []string{"https://example.com/video"},
|
||||
expectedFlags: []string{"--extract-audio", "--audio-format", "mp3"},
|
||||
},
|
||||
{
|
||||
name: "Multiple URLs with flags",
|
||||
input: "/ytdlp --format best https://example.com/v1 https://example.com/v2",
|
||||
expectedURLs: []string{"https://example.com/v1", "https://example.com/v2"},
|
||||
expectedFlags: []string{"--format", "best"},
|
||||
},
|
||||
{
|
||||
name: "Flags mixed with URLs",
|
||||
input: "/ytdlp https://example.com/v1 --format best https://example.com/v2",
|
||||
expectedURLs: []string{"https://example.com/v1", "https://example.com/v2"},
|
||||
expectedFlags: []string{"--format", "best"},
|
||||
},
|
||||
{
|
||||
name: "Short flag",
|
||||
input: "/ytdlp -f best https://example.com/video",
|
||||
expectedURLs: []string{"https://example.com/video"},
|
||||
expectedFlags: []string{"-f", "best"},
|
||||
},
|
||||
{
|
||||
name: "Boolean flag",
|
||||
input: "/ytdlp --extract-audio https://example.com/video",
|
||||
expectedURLs: []string{"https://example.com/video"},
|
||||
expectedFlags: []string{"--extract-audio"},
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
args := strings.Split(tt.input, " ")
|
||||
|
||||
// Simulate the parsing logic from handleYtdlpCmd
|
||||
var urls []string
|
||||
var flags []string
|
||||
|
||||
for i := 1; i < len(args); i++ {
|
||||
arg := strings.TrimSpace(args[i])
|
||||
if arg == "" {
|
||||
continue
|
||||
}
|
||||
|
||||
// Check if it's a flag (starts with - or --)
|
||||
if strings.HasPrefix(arg, "-") {
|
||||
flags = append(flags, arg)
|
||||
// Check if the next argument might be a value for this flag
|
||||
if i+1 < len(args) {
|
||||
nextArg := strings.TrimSpace(args[i+1])
|
||||
if nextArg != "" && !strings.HasPrefix(nextArg, "-") {
|
||||
// Check if it's clearly a URL (has ://)
|
||||
if strings.Contains(nextArg, "://") {
|
||||
// It's a URL, don't consume it as a flag value
|
||||
continue
|
||||
}
|
||||
// Otherwise, treat it as a flag value
|
||||
flags = append(flags, nextArg)
|
||||
i++ // Skip the next argument as it's been consumed
|
||||
}
|
||||
}
|
||||
} else {
|
||||
// Try to parse as URL
|
||||
u, err := url.Parse(arg)
|
||||
if err != nil || u.Scheme == "" || u.Host == "" {
|
||||
continue
|
||||
}
|
||||
urls = append(urls, arg)
|
||||
}
|
||||
}
|
||||
|
||||
// Verify URLs
|
||||
if len(urls) != len(tt.expectedURLs) {
|
||||
t.Errorf("Expected %d URLs, got %d", len(tt.expectedURLs), len(urls))
|
||||
}
|
||||
for i, expectedURL := range tt.expectedURLs {
|
||||
if i >= len(urls) || urls[i] != expectedURL {
|
||||
t.Errorf("Expected URL[%d] to be '%s', got '%s'", i, expectedURL, urls[i])
|
||||
}
|
||||
}
|
||||
|
||||
// Verify flags
|
||||
if len(flags) != len(tt.expectedFlags) {
|
||||
t.Errorf("Expected %d flags, got %d", len(tt.expectedFlags), len(flags))
|
||||
}
|
||||
for i, expectedFlag := range tt.expectedFlags {
|
||||
if i >= len(flags) || flags[i] != expectedFlag {
|
||||
t.Errorf("Expected flag[%d] to be '%s', got '%s'", i, expectedFlag, flags[i])
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -10,6 +10,7 @@ import (
|
||||
"slices"
|
||||
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/krau/SaveAny-Bot/api"
|
||||
"github.com/krau/SaveAny-Bot/client/bot"
|
||||
userclient "github.com/krau/SaveAny-Bot/client/user"
|
||||
"github.com/krau/SaveAny-Bot/common/cache"
|
||||
@@ -76,6 +77,9 @@ func initAll(ctx context.Context, cmd *cobra.Command) (<-chan struct{}, error) {
|
||||
logger.Fatal("User login failed", "error", err)
|
||||
}
|
||||
}
|
||||
if err := api.Start(ctx); err != nil {
|
||||
logger.Error("Failed to start API server", "error", err)
|
||||
}
|
||||
return bot.Init(ctx), nil
|
||||
}
|
||||
|
||||
|
||||
@@ -11,6 +11,7 @@ import (
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/krau/SaveAny-Bot/client/bot"
|
||||
"github.com/krau/SaveAny-Bot/common/cache"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/ioutil"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
@@ -61,6 +62,7 @@ func Upload(cmd *cobra.Command, args []string) error {
|
||||
if err := config.Init(ctx, configFile); err != nil {
|
||||
return fmt.Errorf("failed to load config: %w", err)
|
||||
}
|
||||
i18n.Init(config.C().Lang)
|
||||
cache.Init()
|
||||
database.Init(ctx)
|
||||
|
||||
@@ -88,7 +90,7 @@ func Upload(cmd *cobra.Command, args []string) error {
|
||||
fileName := fileInfo.Name()
|
||||
fileSize := fileInfo.Size()
|
||||
|
||||
uploadPath := stor.JoinStoragePath(path.Join(dirPath, fileName))
|
||||
uploadPath := path.Join(dirPath, fileName)
|
||||
|
||||
ctx = context.WithValue(ctx, ctxkey.ContentLength, fileSize)
|
||||
ctx = tgutil.ExtWithContext(ctx, bot.ExtContext())
|
||||
|
||||
@@ -44,57 +44,7 @@ func Init(lang string) {
|
||||
|
||||
func T(key i18nk.Key, templateData ...map[string]any) string {
|
||||
if localizer == nil || bundle == nil {
|
||||
panic("localizer or bundle is not initialized, call Init() first")
|
||||
}
|
||||
templateDataMap := make(map[string]any)
|
||||
for _, data := range templateData {
|
||||
maps.Copy(templateDataMap, data)
|
||||
}
|
||||
msg, err := localizer.Localize(&i18n.LocalizeConfig{
|
||||
MessageID: string(key),
|
||||
TemplateData: templateDataMap,
|
||||
})
|
||||
if err != nil {
|
||||
return string(key)
|
||||
}
|
||||
return msg
|
||||
}
|
||||
|
||||
func TWithLang(lang, key string, templateData ...map[string]any) string {
|
||||
if bundle == nil {
|
||||
panic("bundle is not initialized, call Init() first")
|
||||
}
|
||||
templateDataMap := make(map[string]any)
|
||||
for _, data := range templateData {
|
||||
maps.Copy(templateDataMap, data)
|
||||
}
|
||||
localizerWithLang := i18n.NewLocalizer(bundle, lang)
|
||||
msg, err := localizerWithLang.Localize(&i18n.LocalizeConfig{
|
||||
MessageID: key,
|
||||
TemplateData: templateDataMap,
|
||||
})
|
||||
if err != nil {
|
||||
return key
|
||||
}
|
||||
return msg
|
||||
}
|
||||
|
||||
// Only use in tests or packages that load before i18n
|
||||
func TWithoutInit(lang string, key i18nk.Key, templateData ...map[string]any) string {
|
||||
bundle := i18n.NewBundle(language.SimplifiedChinese)
|
||||
bundle.RegisterUnmarshalFunc("yaml", yaml.Unmarshal)
|
||||
files, err := localesFS.ReadDir("locale")
|
||||
if err != nil {
|
||||
return string(key)
|
||||
}
|
||||
for _, file := range files {
|
||||
if _, err := bundle.LoadMessageFileFS(localesFS, "locale/"+file.Name()); err != nil {
|
||||
return string(key)
|
||||
}
|
||||
}
|
||||
localizer := i18n.NewLocalizer(bundle, lang)
|
||||
if localizer == nil {
|
||||
return string(key)
|
||||
Init("zh-Hans")
|
||||
}
|
||||
templateDataMap := make(map[string]any)
|
||||
for _, data := range templateData {
|
||||
|
||||
@@ -4,16 +4,24 @@ package i18nk
|
||||
type Key string
|
||||
|
||||
const (
|
||||
BotMsgAria2ErrorAddingAria2Download Key = "bot.msg.aria2.error_adding_aria2_download"
|
||||
BotMsgAria2ErrorAria2ClientInitFailed Key = "bot.msg.aria2.error_aria2_client_init_failed"
|
||||
BotMsgAria2ErrorAria2NotEnabled Key = "bot.msg.aria2.error_aria2_not_enabled"
|
||||
BotMsgAria2InfoAddingAria2Download Key = "bot.msg.aria2.info_adding_aria2_download"
|
||||
BotMsgAria2InfoAria2DownloadAdded Key = "bot.msg.aria2.info_aria2_download_added"
|
||||
BotMsgAria2InfoSelectStorage Key = "bot.msg.aria2.info_select_storage"
|
||||
BotMsgCancelErrorCancelFailed Key = "bot.msg.cancel.error_cancel_failed"
|
||||
BotMsgCancelInfoCancelRequested Key = "bot.msg.cancel.info_cancel_requested"
|
||||
BotMsgCancelInfoCancellingTask Key = "bot.msg.cancel.info_cancelling_task"
|
||||
BotMsgCancelUsage Key = "bot.msg.cancel.usage"
|
||||
BotMsgCmdAria2dl Key = "bot.msg.cmd.aria2dl"
|
||||
BotMsgCmdCancel Key = "bot.msg.cmd.cancel"
|
||||
BotMsgCmdConfig Key = "bot.msg.cmd.config"
|
||||
BotMsgCmdDir Key = "bot.msg.cmd.dir"
|
||||
BotMsgCmdDl Key = "bot.msg.cmd.dl"
|
||||
BotMsgCmdFnametmpl Key = "bot.msg.cmd.fnametmpl"
|
||||
BotMsgCmdHelp Key = "bot.msg.cmd.help"
|
||||
BotMsgCmdImport Key = "bot.msg.cmd.import"
|
||||
BotMsgCmdLswatch Key = "bot.msg.cmd.lswatch"
|
||||
BotMsgCmdParser Key = "bot.msg.cmd.parser"
|
||||
BotMsgCmdRule Key = "bot.msg.cmd.rule"
|
||||
@@ -21,10 +29,13 @@ const (
|
||||
BotMsgCmdSilent Key = "bot.msg.cmd.silent"
|
||||
BotMsgCmdStart Key = "bot.msg.cmd.start"
|
||||
BotMsgCmdStorage Key = "bot.msg.cmd.storage"
|
||||
BotMsgCmdSyncpeers Key = "bot.msg.cmd.syncpeers"
|
||||
BotMsgCmdTask Key = "bot.msg.cmd.task"
|
||||
BotMsgCmdTransfer Key = "bot.msg.cmd.transfer"
|
||||
BotMsgCmdUnwatch Key = "bot.msg.cmd.unwatch"
|
||||
BotMsgCmdUpdate Key = "bot.msg.cmd.update"
|
||||
BotMsgCmdWatch Key = "bot.msg.cmd.watch"
|
||||
BotMsgCmdYtdlp Key = "bot.msg.cmd.ytdlp"
|
||||
BotMsgCommonCancelButtonText Key = "bot.msg.common.cancel_button_text"
|
||||
BotMsgCommonErrorBuildDirSelectKeyboardFailed Key = "bot.msg.common.error_build_dir_select_keyboard_failed"
|
||||
BotMsgCommonErrorBuildStorageSelectKeyboardFailed Key = "bot.msg.common.error_build_storage_select_keyboard_failed"
|
||||
@@ -120,15 +131,20 @@ const (
|
||||
BotMsgParserInfoInstallPluginSuccess Key = "bot.msg.parser.info_install_plugin_success"
|
||||
BotMsgParserPluginNotEnabled Key = "bot.msg.parser.plugin_not_enabled"
|
||||
BotMsgParserPromptReplyWithParserFile Key = "bot.msg.parser.prompt_reply_with_parser_file"
|
||||
BotMsgProgressAria2Done Key = "bot.msg.progress.aria2_done"
|
||||
BotMsgProgressAria2Downloading Key = "bot.msg.progress.aria2_downloading"
|
||||
BotMsgProgressAria2Start Key = "bot.msg.progress.aria2_start"
|
||||
BotMsgProgressAvgSpeedPrefix Key = "bot.msg.progress.avg_speed_prefix"
|
||||
BotMsgProgressBatchDonePrefix Key = "bot.msg.progress.batch_done_prefix"
|
||||
BotMsgProgressBatchProcessingPrefix Key = "bot.msg.progress.batch_processing_prefix"
|
||||
BotMsgProgressBatchStartPrefix Key = "bot.msg.progress.batch_start_prefix"
|
||||
BotMsgProgressCurrentProgressPrefix Key = "bot.msg.progress.current_progress_prefix"
|
||||
BotMsgProgressCurrentSpeedPrefix Key = "bot.msg.progress.current_speed_prefix"
|
||||
BotMsgProgressDirectDonePrefix Key = "bot.msg.progress.direct_done_prefix"
|
||||
BotMsgProgressDirectStart Key = "bot.msg.progress.direct_start"
|
||||
BotMsgProgressDownloadDonePrefix Key = "bot.msg.progress.download_done_prefix"
|
||||
BotMsgProgressDownloadFailedPrefix Key = "bot.msg.progress.download_failed_prefix"
|
||||
BotMsgProgressDownloadedPrefix Key = "bot.msg.progress.downloaded_prefix"
|
||||
BotMsgProgressDownloadingPrefix Key = "bot.msg.progress.downloading_prefix"
|
||||
BotMsgProgressErrorPrefix Key = "bot.msg.progress.error_prefix"
|
||||
BotMsgProgressFileNamePrefix Key = "bot.msg.progress.file_name_prefix"
|
||||
@@ -147,6 +163,23 @@ const (
|
||||
BotMsgProgressTelegraphProgressPrefix Key = "bot.msg.progress.telegraph_progress_prefix"
|
||||
BotMsgProgressTelegraphStartPrefix Key = "bot.msg.progress.telegraph_start_prefix"
|
||||
BotMsgProgressTotalSizePrefix Key = "bot.msg.progress.total_size_prefix"
|
||||
BotMsgProgressTransferAvgSpeedPrefix Key = "bot.msg.progress.transfer_avg_speed_prefix"
|
||||
BotMsgProgressTransferElapsedTimePrefix Key = "bot.msg.progress.transfer_elapsed_time_prefix"
|
||||
BotMsgProgressTransferFailedFilesPrefix Key = "bot.msg.progress.transfer_failed_files_prefix"
|
||||
BotMsgProgressTransferFailedPrefix Key = "bot.msg.progress.transfer_failed_prefix"
|
||||
BotMsgProgressTransferProcessingMore Key = "bot.msg.progress.transfer_processing_more"
|
||||
BotMsgProgressTransferProcessingPrefix Key = "bot.msg.progress.transfer_processing_prefix"
|
||||
BotMsgProgressTransferProgressPrefix Key = "bot.msg.progress.transfer_progress_prefix"
|
||||
BotMsgProgressTransferRemainingTimePrefix Key = "bot.msg.progress.transfer_remaining_time_prefix"
|
||||
BotMsgProgressTransferSpeedPrefix Key = "bot.msg.progress.transfer_speed_prefix"
|
||||
BotMsgProgressTransferStartPrefix Key = "bot.msg.progress.transfer_start_prefix"
|
||||
BotMsgProgressTransferSuccessPrefix Key = "bot.msg.progress.transfer_success_prefix"
|
||||
BotMsgProgressTransferTotalFilesPrefix Key = "bot.msg.progress.transfer_total_files_prefix"
|
||||
BotMsgProgressTransferTotalSizePrefix Key = "bot.msg.progress.transfer_total_size_prefix"
|
||||
BotMsgProgressTransferUploadedPrefix Key = "bot.msg.progress.transfer_uploaded_prefix"
|
||||
BotMsgProgressYtdlpDone Key = "bot.msg.progress.ytdlp_done"
|
||||
BotMsgProgressYtdlpDownloading Key = "bot.msg.progress.ytdlp_downloading"
|
||||
BotMsgProgressYtdlpStart Key = "bot.msg.progress.ytdlp_start"
|
||||
BotMsgRuleErrorCreateRuleFailed Key = "bot.msg.rule.error_create_rule_failed"
|
||||
BotMsgRuleErrorDeleteRuleFailed Key = "bot.msg.rule.error_delete_rule_failed"
|
||||
BotMsgRuleErrorGetUserRulesFailed Key = "bot.msg.rule.error_get_user_rules_failed"
|
||||
@@ -170,6 +203,10 @@ const (
|
||||
BotMsgSaveHelpText Key = "bot.msg.save_help_text"
|
||||
BotMsgStorageInfoFilenamePrefix Key = "bot.msg.storage.info_filename_prefix"
|
||||
BotMsgStorageInfoPromptSelectStorage Key = "bot.msg.storage.info_prompt_select_storage"
|
||||
BotMsgSyncpeersDone Key = "bot.msg.syncpeers.done"
|
||||
BotMsgSyncpeersFailed Key = "bot.msg.syncpeers.failed"
|
||||
BotMsgSyncpeersStart Key = "bot.msg.syncpeers.start"
|
||||
BotMsgSyncpeersSuccess Key = "bot.msg.syncpeers.success"
|
||||
BotMsgTasksCancelFailed Key = "bot.msg.tasks.cancel_failed"
|
||||
BotMsgTasksCancelRequestedPrefix Key = "bot.msg.tasks.cancel_requested_prefix"
|
||||
BotMsgTasksFieldCreated Key = "bot.msg.tasks.field_created"
|
||||
@@ -195,6 +232,22 @@ const (
|
||||
BotMsgTelegraphInfoPicCountPrefix Key = "bot.msg.telegraph.info_pic_count_prefix"
|
||||
BotMsgTelegraphInfoPromptSelectStorage Key = "bot.msg.telegraph.info_prompt_select_storage"
|
||||
BotMsgTelegraphInfoTitlePrefix Key = "bot.msg.telegraph.info_title_prefix"
|
||||
BotMsgTransferErrorAddTaskFailed Key = "bot.msg.transfer.error_add_task_failed"
|
||||
BotMsgTransferErrorBuildStorageSelectKeyboardFailed Key = "bot.msg.transfer.error_build_storage_select_keyboard_failed"
|
||||
BotMsgTransferErrorInvalidRegex Key = "bot.msg.transfer.error_invalid_regex"
|
||||
BotMsgTransferErrorInvalidSource Key = "bot.msg.transfer.error_invalid_source"
|
||||
BotMsgTransferErrorInvalidTarget Key = "bot.msg.transfer.error_invalid_target"
|
||||
BotMsgTransferErrorListFilesFailed Key = "bot.msg.transfer.error_list_files_failed"
|
||||
BotMsgTransferErrorNoFilesToTransfer Key = "bot.msg.transfer.error_no_files_to_transfer"
|
||||
BotMsgTransferErrorStorageNotFound Key = "bot.msg.transfer.error_storage_not_found"
|
||||
BotMsgTransferErrorStorageNotListable Key = "bot.msg.transfer.error_storage_not_listable"
|
||||
BotMsgTransferErrorStorageNotReadable Key = "bot.msg.transfer.error_storage_not_readable"
|
||||
BotMsgTransferErrorTargetNotFound Key = "bot.msg.transfer.error_target_not_found"
|
||||
BotMsgTransferInfoFetchingFiles Key = "bot.msg.transfer.info_fetching_files"
|
||||
BotMsgTransferInfoFilesSelectStorage Key = "bot.msg.transfer.info_files_select_storage"
|
||||
BotMsgTransferInfoTaskAdded Key = "bot.msg.transfer.info_task_added"
|
||||
BotMsgTransferStartStats Key = "bot.msg.transfer.start_stats"
|
||||
BotMsgTransferUsage Key = "bot.msg.transfer.usage"
|
||||
BotMsgUpdateButtonUpgrade Key = "bot.msg.update.button_upgrade"
|
||||
BotMsgUpdateErrorCheckLatestFailed Key = "bot.msg.update.error_check_latest_failed"
|
||||
BotMsgUpdateErrorNoReleaseFound Key = "bot.msg.update.error_no_release_found"
|
||||
@@ -218,6 +271,11 @@ const (
|
||||
BotMsgWatchInfoWatchListFilterPrefix Key = "bot.msg.watch.info_watch_list_filter_prefix"
|
||||
BotMsgWatchInfoWatchListHeader Key = "bot.msg.watch.info_watch_list_header"
|
||||
BotMsgWatchHelpText Key = "bot.msg.watch_help_text"
|
||||
BotMsgYtdlpErrorDownloadFailed Key = "bot.msg.ytdlp.error_download_failed"
|
||||
BotMsgYtdlpErrorNoValidUrls Key = "bot.msg.ytdlp.error_no_valid_urls"
|
||||
BotMsgYtdlpInfoDownloading Key = "bot.msg.ytdlp.info_downloading"
|
||||
BotMsgYtdlpInfoUrlsSelectStorage Key = "bot.msg.ytdlp.info_urls_select_storage"
|
||||
BotMsgYtdlpUsage Key = "bot.msg.ytdlp.usage"
|
||||
ConfigErrDuplicateStorageName Key = "config.err.duplicate_storage_name"
|
||||
ConfigErrInvalidCacheDir Key = "config.err.invalid_cache_dir"
|
||||
ErrCleanCacheFailed Key = "err.clean_cache_failed"
|
||||
|
||||
@@ -29,6 +29,7 @@ bot:
|
||||
/silent - Toggle silent mode
|
||||
/storage - Set default storage
|
||||
/save [custom filename] - Save file
|
||||
/import <storage_name> <dir_path> [channel_id] [filter] - Import files from storage to Telegram
|
||||
/dir - Manage storage directories
|
||||
/rule - Manage rules
|
||||
/config - Modify configuration
|
||||
@@ -38,6 +39,7 @@ bot:
|
||||
/watch - Watch chats and auto save (UserBot)
|
||||
/unwatch - Stop watching chats (UserBot)
|
||||
/lswatch - List watched chats (UserBot)
|
||||
/syncpeers - Sync peer chats (UserBot)
|
||||
/update - Check and upgrade to latest version
|
||||
|
||||
Usage guide: https://sabot.unv.app/usage
|
||||
@@ -49,6 +51,10 @@ bot:
|
||||
rule: "Manage auto-save rules"
|
||||
save: "Save files"
|
||||
dl: "Download files from given links"
|
||||
aria2dl: "Download files using Aria2"
|
||||
ytdlp: "Download video/audio using yt-dlp"
|
||||
import: "Import files from storage to Telegram"
|
||||
transfer: "Transfer files between storages"
|
||||
task: "Manage task queue"
|
||||
cancel: "Cancel task"
|
||||
watch: "Watch chats (UserBot)"
|
||||
@@ -59,6 +65,7 @@ bot:
|
||||
help: "Show help"
|
||||
parser: "Manage parsers"
|
||||
update: "Check for updates"
|
||||
syncpeers: "Sync peer chats (UserBot)"
|
||||
save_help_text: |
|
||||
Usage:
|
||||
|
||||
@@ -272,6 +279,7 @@ bot:
|
||||
- {{"{{.msgtags}}"}}: Tags in the message, joined with underscore
|
||||
- {{"{{.msggen}}"}}: Generated filename from the message
|
||||
- {{"{{.msgdate}}"}}: Message date, format YYYY-MM-DD_HH-MM-SS
|
||||
- {{"{{.msgraw}}"}}: Raw message text (unprocessed)
|
||||
- {{"{{.origname}}"}}: Original media filename (if any)
|
||||
- {{"{{.chatid}}"}}: Chat ID of the message
|
||||
|
||||
@@ -283,6 +291,34 @@ bot:
|
||||
usage: "Usage: /dl <url1> <url2> ..."
|
||||
error_no_valid_links: "No valid links to download"
|
||||
info_files_select_storage: "Total {{.Count}} files, please select storage"
|
||||
ytdlp:
|
||||
usage: "Usage: /ytdlp [OPTIONS] <URL1> [URL2] ...\nExamples:\n /ytdlp https://example.com/video\n /ytdlp --format best https://example.com/video\n /ytdlp --extract-audio --audio-format mp3 https://example.com/video"
|
||||
error_no_valid_urls: "No valid URLs"
|
||||
info_urls_select_storage: "Found {{.Count}} links, please select storage"
|
||||
info_downloading: "Downloading via yt-dlp..."
|
||||
error_download_failed: "yt-dlp download failed: {{.Error}}"
|
||||
transfer:
|
||||
usage: |
|
||||
Usage: /transfer <source_storage>:/<source_path> [filter]
|
||||
Examples:
|
||||
/transfer local1:/downloads
|
||||
/transfer alist1:/media/photos
|
||||
/transfer webdav1:/files ".*\.mp4$"
|
||||
error_invalid_source: "Invalid source path format, should be: storage_name:/path"
|
||||
error_invalid_target: "Invalid target path format, should be: storage_name:/path"
|
||||
error_storage_not_found: "Storage '{{.StorageName}}' not found or access denied: {{.Error}}"
|
||||
error_storage_not_listable: "Storage '{{.StorageName}}' does not support listing files"
|
||||
error_storage_not_readable: "Storage '{{.StorageName}}' does not support reading files"
|
||||
error_target_not_found: "Target storage '{{.StorageName}}' not found or access denied: {{.Error}}"
|
||||
info_fetching_files: "Fetching file list..."
|
||||
error_list_files_failed: "Failed to list files: {{.Error}}"
|
||||
error_invalid_regex: "Invalid regular expression: {{.Error}}"
|
||||
error_no_files_to_transfer: "No files to transfer in directory"
|
||||
error_add_task_failed: "Failed to add task: {{.Error}}"
|
||||
info_task_added: "Added {{.Count}} files to transfer queue\nTotal size: {{.SizeMB}} MB\nTask ID: {{.TaskID}}"
|
||||
start_stats: "Total files: {{.Count}}\nTotal size: {{.SizeMB}} MB"
|
||||
info_files_select_storage: "Total {{.Count}} files ({{.SizeMB}} MB), please select target storage"
|
||||
error_build_storage_select_keyboard_failed: "Failed to build storage selection keyboard: {{.Error}}"
|
||||
cancel:
|
||||
usage: "Usage: /cancel <task_id>"
|
||||
error_cancel_failed: "Failed to cancel task: {{.Error}}"
|
||||
@@ -323,3 +359,36 @@ bot:
|
||||
direct_start: "Starting download, total size: {{.SizeMB}} MB ({{.Count}} files)"
|
||||
file_name_prefix: "Filename: "
|
||||
error_prefix: "\nError: "
|
||||
aria2_start: "Waiting for Aria2 to complete download (GID: {{.GID}})..."
|
||||
aria2_downloading: "Aria2 downloading (GID: {{.GID}})\n"
|
||||
aria2_done: "Aria2 download completed and transferred (GID: {{.GID}})\n"
|
||||
ytdlp_start: "Starting yt-dlp download ({{.Count}} links)..."
|
||||
ytdlp_downloading: "yt-dlp downloading ({{.Count}} links)\n"
|
||||
ytdlp_done: "yt-dlp download completed and transferred ({{.Count}} files)\n"
|
||||
downloaded_prefix: "\nDownloaded: "
|
||||
current_speed_prefix: "\nCurrent speed: "
|
||||
transfer_start_prefix: "Transfering: "
|
||||
transfer_progress_prefix: "Transfer progress: "
|
||||
transfer_uploaded_prefix: "\nUploaded: "
|
||||
transfer_speed_prefix: "\nSpeed: "
|
||||
transfer_remaining_time_prefix: "\nRemaining time: "
|
||||
transfer_processing_prefix: "\nProcessing:\n"
|
||||
transfer_processing_more: "...and {{.Count}} more files\n"
|
||||
transfer_failed_prefix: "Transfer failed\n"
|
||||
transfer_success_prefix: "Transfer completed\n"
|
||||
transfer_total_files_prefix: "\nTotal files: "
|
||||
transfer_total_size_prefix: "\nTotal size: "
|
||||
transfer_elapsed_time_prefix: "\nElapsed time: "
|
||||
transfer_avg_speed_prefix: "\nAverage speed: "
|
||||
transfer_failed_files_prefix: "\nFailed files: "
|
||||
syncpeers:
|
||||
start: "Starting to sync peers..."
|
||||
done: "Peer sync completed, total {{.Count}} chats synced"
|
||||
failed: "Peer sync failed: {{.Error}}"
|
||||
aria2:
|
||||
error_aria2_not_enabled: "Aria2 feature is not enabled in the configuration"
|
||||
error_aria2_client_init_failed: "Aria2 client initialization failed: {{.Error}}"
|
||||
info_adding_aria2_download: "Adding Aria2 download task..."
|
||||
error_adding_aria2_download: "Failed to add Aria2 download task: {{.Error}}"
|
||||
info_aria2_download_added: "Aria2 download task added, GID: {{.GID}}"
|
||||
info_select_storage: "Please select storage, the task will be added to Aria2 download queue after selection"
|
||||
|
||||
@@ -29,6 +29,8 @@ bot:
|
||||
/silent - 开关静默模式
|
||||
/storage - 设置默认存储位置
|
||||
/save [自定义文件名] - 保存文件
|
||||
/dl <链接1> <链接2> ... - 下载给定链接的文件
|
||||
/import <存储名> <目录路径> [频道ID] [过滤器] - 从存储端导入文件到 Telegram
|
||||
/dir - 管理存储目录
|
||||
/rule - 管理规则
|
||||
/config - 修改配置
|
||||
@@ -38,6 +40,7 @@ bot:
|
||||
/watch - 监听聊天并自动保存 (UserBot)
|
||||
/unwatch - 取消监听聊天 (UserBot)
|
||||
/lswatch - 列出正在监听的聊天 (UserBot)
|
||||
/syncpeers - 同步对话列表 (UserBot)
|
||||
/update - 检查更新并升级
|
||||
|
||||
使用帮助: https://sabot.unv.app/usage
|
||||
@@ -49,11 +52,16 @@ bot:
|
||||
rule: "管理自动存储规则"
|
||||
save: "保存文件"
|
||||
dl: "下载给定链接的文件"
|
||||
aria2dl: "使用 Aria2 下载给定链接的文件"
|
||||
ytdlp: "使用 yt-dlp 下载视频/音频"
|
||||
import: "从存储端导入文件到 Telegram"
|
||||
transfer: "在存储端之间传输文件"
|
||||
task: "管理任务队列"
|
||||
cancel: "取消任务"
|
||||
watch: "监听聊天(UserBot)"
|
||||
unwatch: "取消监听聊天(UserBot)"
|
||||
lswatch: "列出监听的聊天(UserBot)"
|
||||
syncpeers: "同步对话列表(UserBot)"
|
||||
config: "修改配置"
|
||||
fnametmpl: "设置文件命名模板"
|
||||
help: "显示帮助"
|
||||
@@ -272,6 +280,7 @@ bot:
|
||||
- {{"{{.msgtags}}"}}: 消息中的标签, 将以下划线分隔输出
|
||||
- {{"{{.msggen}}"}}: 根据消息生成的文件名
|
||||
- {{"{{.msgdate}}"}}: 消息日期, 格式 YYYY-MM-DD_HH-MM-SS
|
||||
- {{"{{.msgraw}}"}}: 消息的原始文本内容 (不经任何处理)
|
||||
- {{"{{.origname}}"}}: 媒体的原始文件名 (如果有)
|
||||
- {{"{{.chatid}}"}}: 消息的聊天ID
|
||||
|
||||
@@ -283,6 +292,34 @@ bot:
|
||||
usage: "用法: /dl <链接1> <链接2> ..."
|
||||
error_no_valid_links: "没有有效的链接可供下载"
|
||||
info_files_select_storage: "共 {{.Count}} 个文件, 请选择存储位置"
|
||||
ytdlp:
|
||||
usage: "用法: /ytdlp [选项] <URL1> [URL2] ...\n示例:\n /ytdlp https://example.com/video\n /ytdlp --format best https://example.com/video\n /ytdlp --extract-audio --audio-format mp3 https://example.com/video"
|
||||
error_no_valid_urls: "没有有效的 URL"
|
||||
info_urls_select_storage: "共 {{.Count}} 个链接, 请选择存储位置"
|
||||
info_downloading: "正在通过 yt-dlp 下载..."
|
||||
error_download_failed: "yt-dlp 下载失败: {{.Error}}"
|
||||
transfer:
|
||||
usage: |
|
||||
用法: /transfer <source_storage>:/<source_path> [filter]
|
||||
示例:
|
||||
/transfer local1:/downloads
|
||||
/transfer alist1:/media/photos
|
||||
/transfer webdav1:/files ".*\.mp4$"
|
||||
error_invalid_source: "源路径格式无效,应为: storage_name:/path"
|
||||
error_invalid_target: "目标路径格式无效,应为: storage_name:/path"
|
||||
error_storage_not_found: "存储端 '{{.StorageName}}' 不存在或您无权访问: {{.Error}}"
|
||||
error_storage_not_listable: "存储端 '{{.StorageName}}' 不支持列举文件功能"
|
||||
error_storage_not_readable: "存储端 '{{.StorageName}}' 不支持读取文件功能"
|
||||
error_target_not_found: "目标存储端 '{{.StorageName}}' 不存在或您无权访问: {{.Error}}"
|
||||
info_fetching_files: "正在获取文件列表..."
|
||||
error_list_files_failed: "获取文件列表失败: {{.Error}}"
|
||||
error_invalid_regex: "正则表达式无效: {{.Error}}"
|
||||
error_no_files_to_transfer: "目录中没有可传输的文件"
|
||||
error_add_task_failed: "添加任务失败: {{.Error}}"
|
||||
info_task_added: "已添加 {{.Count}} 个文件到传输队列\n总大小: {{.SizeMB}} MB\n任务 ID: {{.TaskID}}"
|
||||
start_stats: "总文件数: {{.Count}}\n总大小: {{.SizeMB}} MB"
|
||||
info_files_select_storage: "共 {{.Count}} 个文件 (总大小: {{.SizeMB}} MB),请选择目标存储位置"
|
||||
error_build_storage_select_keyboard_failed: "构建存储选择键盘失败: {{.Error}}"
|
||||
cancel:
|
||||
usage: "用法: /cancel <task_id>"
|
||||
error_cancel_failed: "取消任务失败: {{.Error}}"
|
||||
@@ -323,3 +360,36 @@ bot:
|
||||
direct_start: "开始下载, 总大小: {{.SizeMB}} MB ({{.Count}} 个文件)"
|
||||
file_name_prefix: "文件名: "
|
||||
error_prefix: "\n错误: "
|
||||
aria2_start: "等待 Aria2 下载完成 (GID: {{.GID}})..."
|
||||
aria2_downloading: "Aria2 正在下载 (GID: {{.GID}})\n"
|
||||
aria2_done: "Aria2 下载完成并已转存 (GID: {{.GID}})\n"
|
||||
ytdlp_start: "开始使用 yt-dlp 下载 ({{.Count}} 个链接)..."
|
||||
ytdlp_downloading: "yt-dlp 正在下载 ({{.Count}} 个链接)\n"
|
||||
ytdlp_done: "yt-dlp 下载完成并已转存 ({{.Count}} 个文件)\n"
|
||||
downloaded_prefix: "\n已下载: "
|
||||
current_speed_prefix: "\n当前速度: "
|
||||
transfer_start_prefix: "正在转存: "
|
||||
transfer_progress_prefix: "转存进度: "
|
||||
transfer_uploaded_prefix: "\n已上传: "
|
||||
transfer_speed_prefix: "\n速度: "
|
||||
transfer_remaining_time_prefix: "\n剩余时间: "
|
||||
transfer_processing_prefix: "\n正在处理:\n"
|
||||
transfer_processing_more: "...和其他 {{.Count}} 个文件\n"
|
||||
transfer_failed_prefix: "转存失败\n"
|
||||
transfer_success_prefix: "转存完成\n"
|
||||
transfer_total_files_prefix: "\n总文件数: "
|
||||
transfer_total_size_prefix: "\n总大小: "
|
||||
transfer_elapsed_time_prefix: "\n耗时: "
|
||||
transfer_avg_speed_prefix: "\n平均速度: "
|
||||
transfer_failed_files_prefix: "\n失败文件数: "
|
||||
syncpeers:
|
||||
start: "正在同步对话列表..."
|
||||
success: "对话列表同步完成, 共同步 {{.Count}} 个对话"
|
||||
failed: "对话列表同步失败: {{.Error}}"
|
||||
aria2:
|
||||
error_aria2_not_enabled: "Aria2 功能未启用, 请在配置文件中启用"
|
||||
error_aria2_client_init_failed: "Aria2 客户端初始化失败: {{.Error}}"
|
||||
info_adding_aria2_download: "正在添加 Aria2 下载任务..."
|
||||
error_adding_aria2_download: "添加 Aria2 下载任务失败: {{.Error}}"
|
||||
info_aria2_download_added: "Aria2 下载任务已添加, GID: {{.GID}}"
|
||||
info_select_storage: "请选择存储位置, 选择后将添加到 Aria2 下载队列"
|
||||
|
||||
@@ -1,6 +1,9 @@
|
||||
package dlutil
|
||||
|
||||
import "time"
|
||||
import (
|
||||
"fmt"
|
||||
"time"
|
||||
)
|
||||
|
||||
var threadsLevels = []struct {
|
||||
threads int
|
||||
@@ -31,3 +34,23 @@ func GetSpeed(downloaded int64, startTime time.Time) float64 {
|
||||
}
|
||||
return float64(downloaded) / elapsed
|
||||
}
|
||||
|
||||
// FormatSize formats a byte size as a human-readable string
|
||||
func FormatSize(bytes int64) string {
|
||||
const (
|
||||
KB = 1024
|
||||
MB = KB * 1024
|
||||
GB = MB * 1024
|
||||
)
|
||||
|
||||
switch {
|
||||
case bytes >= GB:
|
||||
return fmt.Sprintf("%.2f GB", float64(bytes)/float64(GB))
|
||||
case bytes >= MB:
|
||||
return fmt.Sprintf("%.2f MB", float64(bytes)/float64(MB))
|
||||
case bytes >= KB:
|
||||
return fmt.Sprintf("%.2f KB", float64(bytes)/float64(KB))
|
||||
default:
|
||||
return fmt.Sprintf("%d B", bytes)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -48,4 +48,4 @@ func NewProgressWriter(
|
||||
wr: wr,
|
||||
onWrite: onWrite,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -113,29 +113,28 @@ func InputMessageClassSliceFromInt(ids []int) []tg.InputMessageClass {
|
||||
return result
|
||||
}
|
||||
|
||||
func GetMessagesRange(ctx *ext.Context, chatID int64, minId, maxId int) ([]*tg.Message, error) {
|
||||
if msg, err := getMessagesRange(ctx, chatID, minId, maxId); err == nil {
|
||||
return msg, nil
|
||||
func GetMessagesRange(ctx *ext.Context, chatID int64, minId, maxId int) (msg []*tg.Message, err error) {
|
||||
if msg, err = getMessagesRange(ctx, chatID, minId, maxId); err == nil {
|
||||
return
|
||||
}
|
||||
in := constant.TDLibPeerID(chatID)
|
||||
plain := in.ToPlain()
|
||||
|
||||
var channel constant.TDLibPeerID
|
||||
channel.Channel(plain)
|
||||
if msg, err := getMessagesRange(ctx, int64(channel), minId, maxId); err == nil {
|
||||
return msg, nil
|
||||
if msg, err = getMessagesRange(ctx, int64(channel), minId, maxId); err == nil {
|
||||
return
|
||||
}
|
||||
var userID constant.TDLibPeerID
|
||||
userID.User(plain)
|
||||
if msg, err := getMessagesRange(ctx, int64(userID), minId, maxId); err == nil {
|
||||
return msg, nil
|
||||
if msg, err = getMessagesRange(ctx, int64(userID), minId, maxId); err == nil {
|
||||
return
|
||||
}
|
||||
var chat constant.TDLibPeerID
|
||||
chat.Chat(plain)
|
||||
if msg, err := getMessagesRange(ctx, int64(chat), minId, maxId); err == nil {
|
||||
return msg, nil
|
||||
if msg, err = getMessagesRange(ctx, int64(chat), minId, maxId); err == nil {
|
||||
return
|
||||
}
|
||||
return nil, fmt.Errorf("failed to get messages range for chatID %d", chatID)
|
||||
return nil, fmt.Errorf("failed to get messages range for chat %d: %w", chatID, err)
|
||||
}
|
||||
|
||||
func getMessagesRange(ctx *ext.Context, chatID int64, minId, maxId int) ([]*tg.Message, error) {
|
||||
|
||||
@@ -1,6 +1,12 @@
|
||||
package tgutil
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"encoding/base64"
|
||||
"fmt"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/url"
|
||||
|
||||
"github.com/gotd/td/telegram/dcs"
|
||||
@@ -8,24 +14,108 @@ import (
|
||||
"golang.org/x/net/proxy"
|
||||
)
|
||||
|
||||
func newProxyDialer(proxyUrl string) (proxy.Dialer, error) {
|
||||
url, err := url.Parse(proxyUrl)
|
||||
// httpProxyDialer implements proxy.ContextDialer for HTTP CONNECT proxies
|
||||
type httpProxyDialer struct {
|
||||
proxyURL *url.URL
|
||||
forward proxy.Dialer
|
||||
}
|
||||
|
||||
func (d *httpProxyDialer) Dial(network, addr string) (net.Conn, error) {
|
||||
return d.DialContext(context.Background(), network, addr)
|
||||
}
|
||||
|
||||
func (d *httpProxyDialer) DialContext(ctx context.Context, network, addr string) (net.Conn, error) {
|
||||
proxyAddr := d.proxyURL.Host
|
||||
if d.proxyURL.Port() == "" {
|
||||
if d.proxyURL.Scheme == "https" {
|
||||
proxyAddr = net.JoinHostPort(d.proxyURL.Hostname(), "443")
|
||||
} else {
|
||||
proxyAddr = net.JoinHostPort(d.proxyURL.Hostname(), "80")
|
||||
}
|
||||
}
|
||||
|
||||
var conn net.Conn
|
||||
var err error
|
||||
if ctxDialer, ok := d.forward.(proxy.ContextDialer); ok {
|
||||
conn, err = ctxDialer.DialContext(ctx, "tcp", proxyAddr)
|
||||
} else {
|
||||
conn, err = d.forward.Dial("tcp", proxyAddr)
|
||||
}
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to connect to proxy: %w", err)
|
||||
}
|
||||
|
||||
// Send CONNECT request
|
||||
connectReq := &http.Request{
|
||||
Method: "CONNECT",
|
||||
URL: &url.URL{Opaque: addr},
|
||||
Host: addr,
|
||||
Header: make(http.Header),
|
||||
}
|
||||
|
||||
// Add proxy authentication if provided
|
||||
if d.proxyURL.User != nil {
|
||||
username := d.proxyURL.User.Username()
|
||||
password, _ := d.proxyURL.User.Password()
|
||||
auth := base64.StdEncoding.EncodeToString([]byte(username + ":" + password))
|
||||
connectReq.Header.Set("Proxy-Authorization", "Basic "+auth)
|
||||
}
|
||||
|
||||
if err := connectReq.Write(conn); err != nil {
|
||||
conn.Close()
|
||||
return nil, fmt.Errorf("failed to write CONNECT request: %w", err)
|
||||
}
|
||||
|
||||
// Read response
|
||||
br := bufio.NewReader(conn)
|
||||
resp, err := http.ReadResponse(br, connectReq)
|
||||
if err != nil {
|
||||
conn.Close()
|
||||
return nil, fmt.Errorf("failed to read CONNECT response: %w", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
conn.Close()
|
||||
return nil, fmt.Errorf("proxy CONNECT failed with status: %s", resp.Status)
|
||||
}
|
||||
|
||||
return conn, nil
|
||||
}
|
||||
|
||||
func newProxyDialer(proxyUrl string) (proxy.ContextDialer, error) {
|
||||
parsedURL, err := url.Parse(proxyUrl)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return proxy.FromURL(url, proxy.Direct)
|
||||
|
||||
switch parsedURL.Scheme {
|
||||
case "http", "https":
|
||||
return &httpProxyDialer{
|
||||
proxyURL: parsedURL,
|
||||
forward: proxy.Direct,
|
||||
}, nil
|
||||
case "socks5", "socks5h":
|
||||
dialer, err := proxy.FromURL(parsedURL, proxy.Direct)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return dialer.(proxy.ContextDialer), nil
|
||||
default:
|
||||
return nil, fmt.Errorf("unsupported proxy scheme: %s", parsedURL.Scheme)
|
||||
}
|
||||
}
|
||||
|
||||
func NewConfigProxyResolver() (dcs.Resolver, error) {
|
||||
resolver := dcs.DefaultResolver()
|
||||
if config.C().Proxy != "" {
|
||||
// gloabl proxy, which has lower priority
|
||||
// global proxy, which has lower priority
|
||||
dialer, err := newProxyDialer(config.C().Proxy)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
resolver = dcs.Plain(dcs.PlainOptions{
|
||||
Dial: dialer.(proxy.ContextDialer).DialContext,
|
||||
Dial: dialer.DialContext,
|
||||
})
|
||||
}
|
||||
if config.C().Telegram.Proxy.Enable && config.C().Telegram.Proxy.URL != "" {
|
||||
@@ -34,7 +124,7 @@ func NewConfigProxyResolver() (dcs.Resolver, error) {
|
||||
return nil, err
|
||||
}
|
||||
resolver = dcs.Plain(dcs.PlainOptions{
|
||||
Dial: dialer.(proxy.ContextDialer).DialContext,
|
||||
Dial: dialer.DialContext,
|
||||
})
|
||||
}
|
||||
return resolver, nil
|
||||
|
||||
@@ -14,10 +14,32 @@ token = ""
|
||||
# app_id = 1025907
|
||||
# app_hash = "452b0359b988148995f22ff0f4229750"
|
||||
[telegram.proxy]
|
||||
# 启用代理连接 telegram, 只支持 socks5
|
||||
# 启用代理连接 telegram
|
||||
enable = false
|
||||
url = "socks5://127.0.0.1:7890"
|
||||
|
||||
# Aria2 配置
|
||||
[aria2]
|
||||
# 启用 Aria2 下载支持
|
||||
enable = false
|
||||
# Aria2 RPC URL
|
||||
url = "http://localhost:6800/jsonrpc"
|
||||
# Aria2 RPC Secret (如果配置了 rpc-secret)
|
||||
secret = ""
|
||||
# 转存完成后删除 Aria2 下载的本地文件
|
||||
remove_after_transfer = true
|
||||
|
||||
# HTTP API 配置
|
||||
[api]
|
||||
# 启用 HTTP API
|
||||
enable = false
|
||||
# 监听地址
|
||||
host = "0.0.0.0"
|
||||
# 监听端口
|
||||
port = 8080
|
||||
# 认证 Token (必需)
|
||||
token = ""
|
||||
|
||||
# 存储列表
|
||||
[[storages]]
|
||||
# 标识名, 需要唯一
|
||||
|
||||
@@ -16,6 +16,7 @@ var storageFactories = map[storenum.StorageType]func(cfg *BaseConfig) (StorageCo
|
||||
storenum.Minio: createStorageConfig(&MinioStorageConfig{}),
|
||||
storenum.S3: createStorageConfig(&S3StorageConfig{}),
|
||||
storenum.Telegram: createStorageConfig(&TelegramStorageConfig{}),
|
||||
storenum.Rclone: createStorageConfig(&RcloneStorageConfig{}),
|
||||
}
|
||||
|
||||
func createStorageConfig(configType StorageConfig) func(cfg *BaseConfig) (StorageConfig, error) {
|
||||
|
||||
33
config/storage/rclone.go
Normal file
33
config/storage/rclone.go
Normal file
@@ -0,0 +1,33 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
|
||||
storenum "github.com/krau/SaveAny-Bot/pkg/enums/storage"
|
||||
)
|
||||
|
||||
type RcloneStorageConfig struct {
|
||||
BaseConfig
|
||||
// The name of the remote as defined in rclone config
|
||||
Remote string `toml:"remote" mapstructure:"remote" json:"remote"`
|
||||
BasePath string `toml:"base_path" mapstructure:"base_path" json:"base_path"`
|
||||
// The path to the rclone config file, if not using the default
|
||||
ConfigPath string `toml:"config_path" mapstructure:"config_path" json:"config_path"`
|
||||
// Additional flags to pass to rclone commands
|
||||
Flags []string `toml:"flags" mapstructure:"flags" json:"flags"`
|
||||
}
|
||||
|
||||
func (r *RcloneStorageConfig) Validate() error {
|
||||
if r.Remote == "" {
|
||||
return fmt.Errorf("remote is required for rclone storage")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *RcloneStorageConfig) GetType() storenum.StorageType {
|
||||
return storenum.Rclone
|
||||
}
|
||||
|
||||
func (r *RcloneStorageConfig) GetName() string {
|
||||
return r.Name
|
||||
}
|
||||
@@ -16,13 +16,15 @@ import (
|
||||
)
|
||||
|
||||
type Config struct {
|
||||
Lang string `toml:"lang" mapstructure:"lang" json:"lang"`
|
||||
Workers int `toml:"workers" mapstructure:"workers"`
|
||||
Retry int `toml:"retry" mapstructure:"retry"`
|
||||
NoCleanCache bool `toml:"no_clean_cache" mapstructure:"no_clean_cache" json:"no_clean_cache"`
|
||||
Threads int `toml:"threads" mapstructure:"threads" json:"threads"`
|
||||
Stream bool `toml:"stream" mapstructure:"stream" json:"stream"`
|
||||
Proxy string `toml:"proxy" mapstructure:"proxy" json:"proxy"`
|
||||
Lang string `toml:"lang" mapstructure:"lang" json:"lang"`
|
||||
Workers int `toml:"workers" mapstructure:"workers"`
|
||||
Retry int `toml:"retry" mapstructure:"retry"`
|
||||
NoCleanCache bool `toml:"no_clean_cache" mapstructure:"no_clean_cache" json:"no_clean_cache"`
|
||||
Threads int `toml:"threads" mapstructure:"threads" json:"threads"`
|
||||
Stream bool `toml:"stream" mapstructure:"stream" json:"stream"`
|
||||
Proxy string `toml:"proxy" mapstructure:"proxy" json:"proxy"`
|
||||
Aria2 aria2Config `toml:"aria2" mapstructure:"aria2" json:"aria2"`
|
||||
API apiConfig `toml:"api" mapstructure:"api" json:"api"`
|
||||
|
||||
Cache cacheConfig `toml:"cache" mapstructure:"cache" json:"cache"`
|
||||
Users []userConfig `toml:"users" mapstructure:"users" json:"users"`
|
||||
@@ -34,6 +36,20 @@ type Config struct {
|
||||
Hook hookConfig `toml:"hook" mapstructure:"hook" json:"hook"`
|
||||
}
|
||||
|
||||
type aria2Config struct {
|
||||
Enable bool `toml:"enable" mapstructure:"enable" json:"enable"`
|
||||
Url string `toml:"url" mapstructure:"url" json:"url"`
|
||||
Secret string `toml:"secret" mapstructure:"secret" json:"secret"`
|
||||
KeepFile bool `toml:"keep_file" mapstructure:"keep_file" json:"keep_file"`
|
||||
}
|
||||
|
||||
type apiConfig struct {
|
||||
Enable bool `toml:"enable" mapstructure:"enable" json:"enable"`
|
||||
Host string `toml:"host" mapstructure:"host" json:"host"`
|
||||
Port int `toml:"port" mapstructure:"port" json:"port"`
|
||||
Token string `toml:"token" mapstructure:"token" json:"token"`
|
||||
}
|
||||
|
||||
var cfg = &Config{}
|
||||
|
||||
func C() Config {
|
||||
@@ -107,6 +123,12 @@ func Init(ctx context.Context, configFile ...string) error {
|
||||
// 数据库
|
||||
"db.path": "data/saveany.db",
|
||||
"db.session": "data/session.db",
|
||||
|
||||
// API
|
||||
"api.enable": false,
|
||||
"api.host": "0.0.0.0",
|
||||
"api.port": 8080,
|
||||
"api.token": "",
|
||||
}
|
||||
|
||||
for key, value := range defaultConfigs {
|
||||
|
||||
250
core/tasks/aria2dl/execute.go
Normal file
250
core/tasks/aria2dl/execute.go
Normal file
@@ -0,0 +1,250 @@
|
||||
package aria2dl
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"time"
|
||||
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
"github.com/krau/SaveAny-Bot/pkg/aria2"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/ctxkey"
|
||||
)
|
||||
|
||||
// Execute implements core.Executable.
|
||||
func (t *Task) Execute(ctx context.Context) error {
|
||||
logger := log.FromContext(ctx)
|
||||
logger.Infof("Starting aria2 download task %s (GID: %s)", t.ID, t.GID)
|
||||
|
||||
if t.Progress != nil {
|
||||
t.Progress.OnStart(ctx, t)
|
||||
}
|
||||
|
||||
// Wait for aria2 download to complete
|
||||
if err := t.waitForDownload(ctx); err != nil {
|
||||
// If context was canceled, also cancel the aria2 download
|
||||
if errors.Is(err, context.Canceled) {
|
||||
t.cancelAria2Download()
|
||||
}
|
||||
logger.Errorf("Aria2 download failed: %v", err)
|
||||
if t.Progress != nil {
|
||||
t.Progress.OnDone(ctx, t, err)
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
// Transfer downloaded files to storage
|
||||
if err := t.transferFiles(ctx); err != nil {
|
||||
logger.Errorf("File transfer failed: %v", err)
|
||||
if t.Progress != nil {
|
||||
t.Progress.OnDone(ctx, t, err)
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
logger.Infof("Aria2 task %s completed successfully", t.ID)
|
||||
if t.Progress != nil {
|
||||
t.Progress.OnDone(ctx, t, nil)
|
||||
}
|
||||
|
||||
// Clean up aria2 download result
|
||||
if _, err := t.Aria2Client.RemoveDownloadResult(context.Background(), t.GID); err != nil {
|
||||
logger.Warnf("Failed to remove aria2 download result: %v", err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// waitForDownload waits for aria2 to complete the download
|
||||
func (t *Task) waitForDownload(ctx context.Context) error {
|
||||
logger := log.FromContext(ctx)
|
||||
ticker := time.NewTicker(2 * time.Second)
|
||||
defer ticker.Stop()
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
case <-ticker.C:
|
||||
status, err := t.getStatus(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if t.Progress != nil {
|
||||
t.Progress.OnProgress(ctx, t, status)
|
||||
}
|
||||
|
||||
// Check if download is complete
|
||||
if status.IsDownloadComplete() {
|
||||
// Handle metadata downloads (torrent/magnet) that spawn follow-up downloads
|
||||
if len(status.FollowedBy) > 0 {
|
||||
logger.Infof("Switching from metadata GID %s to actual download GID: %s", t.GID, status.FollowedBy[0])
|
||||
t.GID = status.FollowedBy[0]
|
||||
continue
|
||||
}
|
||||
logger.Infof("Download completed for GID %s", t.GID)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Check for errors
|
||||
if status.IsDownloadError() {
|
||||
return fmt.Errorf("aria2 download error: %s (code: %s)", status.ErrorMessage, status.ErrorCode)
|
||||
}
|
||||
|
||||
if status.IsDownloadRemoved() {
|
||||
return errors.New("aria2 download was removed")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// getStatus retrieves the current status of the download
|
||||
func (t *Task) getStatus(ctx context.Context) (*aria2.Status, error) {
|
||||
logger := log.FromContext(ctx)
|
||||
|
||||
// Try active/waiting queue first
|
||||
status, err := t.Aria2Client.TellStatus(ctx, t.GID)
|
||||
if err == nil {
|
||||
return status, nil
|
||||
}
|
||||
|
||||
// Check stopped queue
|
||||
logger.Debugf("Task not in active queue, checking stopped queue")
|
||||
stoppedTasks, stopErr := t.Aria2Client.TellStopped(ctx, -1, 100)
|
||||
if stopErr != nil {
|
||||
return nil, fmt.Errorf("failed to get aria2 status: %w", err)
|
||||
}
|
||||
|
||||
for _, task := range stoppedTasks {
|
||||
if task.GID == t.GID {
|
||||
logger.Debugf("Found task in stopped queue with status: %s", task.Status)
|
||||
return &task, nil
|
||||
}
|
||||
}
|
||||
|
||||
return nil, fmt.Errorf("task GID %s not found: %w", t.GID, err)
|
||||
}
|
||||
|
||||
// transferFiles transfers downloaded files from aria2 to storage
|
||||
func (t *Task) transferFiles(ctx context.Context) error {
|
||||
logger := log.FromContext(ctx)
|
||||
|
||||
status, err := t.Aria2Client.TellStatus(ctx, t.GID)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to get final status: %w", err)
|
||||
}
|
||||
|
||||
if len(status.Files) == 0 {
|
||||
return errors.New("no files in aria2 download")
|
||||
}
|
||||
|
||||
logger.Infof("Transferring %d file(s) to storage %s", len(status.Files), t.Storage.Name())
|
||||
transferredCount := 0
|
||||
|
||||
for _, file := range status.Files {
|
||||
if file.Selected != "true" {
|
||||
logger.Debugf("Skipping unselected file: %s", file.Path)
|
||||
continue
|
||||
}
|
||||
|
||||
fileName := filepath.Base(file.Path)
|
||||
|
||||
// Skip torrent metadata files
|
||||
if filepath.Ext(fileName) == ".torrent" {
|
||||
logger.Debugf("Skipping torrent metadata file: %s", fileName)
|
||||
t.removeFileIfNeeded(file.Path)
|
||||
continue
|
||||
}
|
||||
|
||||
if err := t.transferFile(ctx, file.Path); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
transferredCount++
|
||||
t.removeFileIfNeeded(file.Path)
|
||||
}
|
||||
|
||||
if transferredCount == 0 {
|
||||
return errors.New("no files were transferred")
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// transferFile transfers a single file to storage
|
||||
func (t *Task) transferFile(ctx context.Context, filePath string) error {
|
||||
logger := log.FromContext(ctx)
|
||||
|
||||
// Check if file exists
|
||||
fileInfo, err := os.Stat(filePath)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
logger.Warnf("Downloaded file not found: %s", filePath)
|
||||
return nil // Not a fatal error, continue with other files
|
||||
}
|
||||
return fmt.Errorf("failed to stat file %s: %w", filePath, err)
|
||||
}
|
||||
|
||||
// Open file
|
||||
f, err := os.Open(filePath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to open file %s: %w", filePath, err)
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
// Set content length in context for storage
|
||||
ctx = context.WithValue(ctx, ctxkey.ContentLength, fileInfo.Size())
|
||||
|
||||
// Save to storage
|
||||
fileName := filepath.Base(filePath)
|
||||
destPath := filepath.Join(t.StorPath, fileName)
|
||||
|
||||
logger.Infof("Transferring file %s to %s:%s", fileName, t.Storage.Name(), destPath)
|
||||
|
||||
if err := t.Storage.Save(ctx, f, destPath); err != nil {
|
||||
return fmt.Errorf("failed to save file %s to storage: %w", fileName, err)
|
||||
}
|
||||
|
||||
logger.Infof("Successfully transferred file %s", fileName)
|
||||
return nil
|
||||
}
|
||||
|
||||
// removeFileIfNeeded removes a file if RemoveAfterTransfer is enabled
|
||||
func (t *Task) removeFileIfNeeded(filePath string) {
|
||||
if config.C().Aria2.KeepFile {
|
||||
return
|
||||
}
|
||||
|
||||
logger := log.FromContext(t.ctx)
|
||||
if err := os.Remove(filePath); err != nil {
|
||||
logger.Warnf("Failed to remove local file %s: %v", filePath, err)
|
||||
} else {
|
||||
logger.Debugf("Removed local file %s", filePath)
|
||||
}
|
||||
}
|
||||
|
||||
// cancelAria2Download cancels the aria2 download task
|
||||
func (t *Task) cancelAria2Download() {
|
||||
logger := log.FromContext(t.ctx)
|
||||
logger.Infof("Canceling aria2 download GID: %s", t.GID)
|
||||
|
||||
// Use a background context with timeout for cleanup
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer cancel()
|
||||
|
||||
// Try to force remove the download
|
||||
if _, err := t.Aria2Client.ForceRemove(ctx, t.GID); err != nil {
|
||||
logger.Warnf("Failed to cancel aria2 download %s: %v", t.GID, err)
|
||||
} else {
|
||||
logger.Infof("Successfully canceled aria2 download %s", t.GID)
|
||||
}
|
||||
|
||||
// Also remove the download result to clean up
|
||||
if _, err := t.Aria2Client.RemoveDownloadResult(ctx, t.GID); err != nil {
|
||||
logger.Debugf("Failed to remove download result for %s: %v", t.GID, err)
|
||||
}
|
||||
}
|
||||
189
core/tasks/aria2dl/progress.go
Normal file
189
core/tasks/aria2dl/progress.go
Normal file
@@ -0,0 +1,189 @@
|
||||
package aria2dl
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strconv"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/gotd/td/telegram/message/entity"
|
||||
"github.com/gotd/td/telegram/message/styling"
|
||||
"github.com/gotd/td/tg"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/dlutil"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
|
||||
"github.com/krau/SaveAny-Bot/pkg/aria2"
|
||||
)
|
||||
|
||||
type ProgressTracker interface {
|
||||
OnStart(ctx context.Context, task *Task)
|
||||
OnProgress(ctx context.Context, task *Task, status *aria2.Status)
|
||||
OnDone(ctx context.Context, task *Task, err error)
|
||||
}
|
||||
|
||||
type Progress struct {
|
||||
msgID int
|
||||
chatID int64
|
||||
start time.Time
|
||||
lastUpdatePercent atomic.Int32
|
||||
}
|
||||
|
||||
// OnStart implements ProgressTracker.
|
||||
func (p *Progress) OnStart(ctx context.Context, task *Task) {
|
||||
logger := log.FromContext(ctx)
|
||||
p.start = time.Now()
|
||||
p.lastUpdatePercent.Store(0)
|
||||
logger.Infof("Aria2 task started: message_id=%d, chat_id=%d, gid=%s", p.msgID, p.chatID, task.GID)
|
||||
ext := tgutil.ExtFromContext(ctx)
|
||||
if ext == nil {
|
||||
return
|
||||
}
|
||||
entityBuilder := entity.Builder{}
|
||||
if err := styling.Perform(&entityBuilder,
|
||||
styling.Plain(i18n.T(i18nk.BotMsgProgressAria2Start, map[string]any{
|
||||
"GID": task.GID,
|
||||
}))); err != nil {
|
||||
log.FromContext(ctx).Errorf("Failed to build entities: %s", err)
|
||||
return
|
||||
}
|
||||
text, entities := entityBuilder.Complete()
|
||||
req := &tg.MessagesEditMessageRequest{
|
||||
ID: p.msgID,
|
||||
}
|
||||
req.SetMessage(text)
|
||||
req.SetEntities(entities)
|
||||
req.SetReplyMarkup(&tg.ReplyInlineMarkup{
|
||||
Rows: []tg.KeyboardButtonRow{
|
||||
{
|
||||
Buttons: []tg.KeyboardButtonClass{
|
||||
tgutil.BuildCancelButton(task.TaskID()),
|
||||
},
|
||||
},
|
||||
}},
|
||||
)
|
||||
ext.EditMessage(p.chatID, req)
|
||||
}
|
||||
|
||||
// OnProgress implements ProgressTracker.
|
||||
func (p *Progress) OnProgress(ctx context.Context, task *Task, status *aria2.Status) {
|
||||
totalLength, _ := strconv.ParseInt(status.TotalLength, 10, 64)
|
||||
completedLength, _ := strconv.ParseInt(status.CompletedLength, 10, 64)
|
||||
downloadSpeed, _ := strconv.ParseInt(status.DownloadSpeed, 10, 64)
|
||||
|
||||
if totalLength == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
percent := int((completedLength * 100) / totalLength)
|
||||
if p.lastUpdatePercent.Load() == int32(percent) {
|
||||
return
|
||||
}
|
||||
p.lastUpdatePercent.Store(int32(percent))
|
||||
|
||||
log.FromContext(ctx).Debugf("Aria2 progress update: %s, %d/%d", task.GID, completedLength, totalLength)
|
||||
|
||||
entityBuilder := entity.Builder{}
|
||||
if err := styling.Perform(&entityBuilder,
|
||||
styling.Plain(i18n.T(i18nk.BotMsgProgressAria2Downloading, map[string]any{
|
||||
"GID": task.GID,
|
||||
})),
|
||||
styling.Plain(i18n.T(i18nk.BotMsgProgressDownloadedPrefix, nil)),
|
||||
styling.Code(fmt.Sprintf("%.2f MB / %.2f MB", float64(completedLength)/(1024*1024), float64(totalLength)/(1024*1024))),
|
||||
styling.Plain(i18n.T(i18nk.BotMsgProgressCurrentSpeedPrefix, nil)),
|
||||
styling.Bold(fmt.Sprintf("%.2f MB/s", float64(downloadSpeed)/(1024*1024))),
|
||||
styling.Plain(i18n.T(i18nk.BotMsgProgressAvgSpeedPrefix, nil)),
|
||||
styling.Bold(fmt.Sprintf("%.2f MB/s", dlutil.GetSpeed(completedLength, p.start)/(1024*1024))),
|
||||
styling.Plain(i18n.T(i18nk.BotMsgProgressCurrentProgressPrefix, nil)),
|
||||
styling.Bold(fmt.Sprintf("%.2f%%", float64(percent))),
|
||||
); err != nil {
|
||||
log.FromContext(ctx).Errorf("Failed to build entities: %s", err)
|
||||
return
|
||||
}
|
||||
text, entities := entityBuilder.Complete()
|
||||
req := &tg.MessagesEditMessageRequest{
|
||||
ID: p.msgID,
|
||||
}
|
||||
req.SetMessage(text)
|
||||
req.SetEntities(entities)
|
||||
req.SetReplyMarkup(&tg.ReplyInlineMarkup{
|
||||
Rows: []tg.KeyboardButtonRow{
|
||||
{
|
||||
Buttons: []tg.KeyboardButtonClass{
|
||||
tgutil.BuildCancelButton(task.TaskID()),
|
||||
},
|
||||
},
|
||||
}},
|
||||
)
|
||||
ext := tgutil.ExtFromContext(ctx)
|
||||
if ext != nil {
|
||||
ext.EditMessage(p.chatID, req)
|
||||
}
|
||||
}
|
||||
|
||||
// OnDone implements ProgressTracker.
|
||||
func (p *Progress) OnDone(ctx context.Context, task *Task, err error) {
|
||||
logger := log.FromContext(ctx)
|
||||
if err != nil {
|
||||
if errors.Is(err, context.Canceled) {
|
||||
logger.Infof("Aria2 task %s was canceled", task.TaskID())
|
||||
ext := tgutil.ExtFromContext(ctx)
|
||||
if ext != nil {
|
||||
ext.EditMessage(p.chatID, &tg.MessagesEditMessageRequest{
|
||||
ID: p.msgID,
|
||||
Message: i18n.T(i18nk.BotMsgProgressTaskCanceledWithId, map[string]any{
|
||||
"TaskID": task.TaskID(),
|
||||
}),
|
||||
})
|
||||
}
|
||||
} else {
|
||||
logger.Errorf("Aria2 task %s failed: %s", task.TaskID(), err)
|
||||
ext := tgutil.ExtFromContext(ctx)
|
||||
if ext != nil {
|
||||
ext.EditMessage(p.chatID, &tg.MessagesEditMessageRequest{
|
||||
ID: p.msgID,
|
||||
Message: i18n.T(i18nk.BotMsgProgressTaskFailedWithError, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
})
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
logger.Infof("Aria2 task %s completed successfully", task.TaskID())
|
||||
|
||||
entityBuilder := entity.Builder{}
|
||||
if err := styling.Perform(&entityBuilder,
|
||||
styling.Plain(i18n.T(i18nk.BotMsgProgressAria2Done, map[string]any{
|
||||
"GID": task.GID,
|
||||
})),
|
||||
styling.Plain(i18n.T(i18nk.BotMsgProgressSavePathPrefix, nil)),
|
||||
styling.Code(fmt.Sprintf("[%s]:%s", task.Storage.Name(), task.StorPath)),
|
||||
); err != nil {
|
||||
logger.Errorf("Failed to build entities: %s", err)
|
||||
return
|
||||
}
|
||||
text, entities := entityBuilder.Complete()
|
||||
req := &tg.MessagesEditMessageRequest{
|
||||
ID: p.msgID,
|
||||
}
|
||||
req.SetMessage(text)
|
||||
req.SetEntities(entities)
|
||||
|
||||
ext := tgutil.ExtFromContext(ctx)
|
||||
if ext != nil {
|
||||
ext.EditMessage(p.chatID, req)
|
||||
}
|
||||
}
|
||||
|
||||
var _ ProgressTracker = (*Progress)(nil)
|
||||
|
||||
func NewProgress(msgID int, userID int64) ProgressTracker {
|
||||
return &Progress{
|
||||
msgID: msgID,
|
||||
chatID: userID,
|
||||
}
|
||||
}
|
||||
61
core/tasks/aria2dl/task.go
Normal file
61
core/tasks/aria2dl/task.go
Normal file
@@ -0,0 +1,61 @@
|
||||
package aria2dl
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/core"
|
||||
"github.com/krau/SaveAny-Bot/pkg/aria2"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
|
||||
"github.com/krau/SaveAny-Bot/storage"
|
||||
)
|
||||
|
||||
var _ core.Executable = (*Task)(nil)
|
||||
|
||||
type Task struct {
|
||||
ID string
|
||||
ctx context.Context
|
||||
GID string
|
||||
URIs []string
|
||||
Aria2Client *aria2.Client
|
||||
Storage storage.Storage
|
||||
StorPath string
|
||||
Progress ProgressTracker
|
||||
}
|
||||
|
||||
// Title implements core.Executable.
|
||||
func (t *Task) Title() string {
|
||||
return fmt.Sprintf("[%s](Aria2 GID:%s->%s:%s)", t.Type(), t.GID, t.Storage.Name(), t.StorPath)
|
||||
}
|
||||
|
||||
// Type implements core.Executable.
|
||||
func (t *Task) Type() tasktype.TaskType {
|
||||
return tasktype.TaskTypeAria2
|
||||
}
|
||||
|
||||
// TaskID implements core.Executable.
|
||||
func (t *Task) TaskID() string {
|
||||
return t.ID
|
||||
}
|
||||
|
||||
func NewTask(
|
||||
id string,
|
||||
ctx context.Context,
|
||||
gid string,
|
||||
uris []string,
|
||||
aria2Client *aria2.Client,
|
||||
stor storage.Storage,
|
||||
storPath string,
|
||||
progressTracker ProgressTracker,
|
||||
) *Task {
|
||||
return &Task{
|
||||
ID: id,
|
||||
ctx: ctx,
|
||||
GID: gid,
|
||||
URIs: uris,
|
||||
Aria2Client: aria2Client,
|
||||
Storage: stor,
|
||||
StorPath: storPath,
|
||||
Progress: progressTracker,
|
||||
}
|
||||
}
|
||||
209
core/tasks/aria2dl/task_test.go
Normal file
209
core/tasks/aria2dl/task_test.go
Normal file
@@ -0,0 +1,209 @@
|
||||
package aria2dl
|
||||
|
||||
import (
|
||||
"context"
|
||||
"io"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
storconfig "github.com/krau/SaveAny-Bot/config/storage"
|
||||
"github.com/krau/SaveAny-Bot/pkg/aria2"
|
||||
storenum "github.com/krau/SaveAny-Bot/pkg/enums/storage"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
|
||||
)
|
||||
|
||||
type mockStorage struct {
|
||||
name string
|
||||
savePath string
|
||||
}
|
||||
|
||||
func (m *mockStorage) Name() string {
|
||||
return m.name
|
||||
}
|
||||
|
||||
func (m *mockStorage) Type() storenum.StorageType {
|
||||
return storenum.StorageType("mock")
|
||||
}
|
||||
|
||||
func (m *mockStorage) Init(ctx context.Context, config storconfig.StorageConfig) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m *mockStorage) Save(ctx context.Context, reader io.Reader, path string) error {
|
||||
m.savePath = path
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m *mockStorage) Exists(ctx context.Context, path string) bool {
|
||||
return false
|
||||
}
|
||||
|
||||
func (m *mockStorage) JoinStoragePath(path string) string {
|
||||
return path
|
||||
}
|
||||
|
||||
type mockProgress struct {
|
||||
started bool
|
||||
done bool
|
||||
doneErr error
|
||||
progress int
|
||||
}
|
||||
|
||||
func (m *mockProgress) OnStart(ctx context.Context, task *Task) {
|
||||
m.started = true
|
||||
}
|
||||
|
||||
func (m *mockProgress) OnProgress(ctx context.Context, task *Task, status *aria2.Status) {
|
||||
m.progress++
|
||||
}
|
||||
|
||||
func (m *mockProgress) OnDone(ctx context.Context, task *Task, err error) {
|
||||
m.done = true
|
||||
m.doneErr = err
|
||||
}
|
||||
|
||||
func TestTaskCreation(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
mockStor := &mockStorage{name: "test-storage"}
|
||||
mockProg := &mockProgress{}
|
||||
|
||||
task := NewTask(
|
||||
"test-task-id",
|
||||
ctx,
|
||||
"test-gid",
|
||||
[]string{"http://example.com/file.zip"},
|
||||
nil,
|
||||
mockStor,
|
||||
"/test/path",
|
||||
mockProg,
|
||||
)
|
||||
|
||||
if task.ID != "test-task-id" {
|
||||
t.Errorf("Expected task ID to be 'test-task-id', got '%s'", task.ID)
|
||||
}
|
||||
|
||||
if task.GID != "test-gid" {
|
||||
t.Errorf("Expected GID to be 'test-gid', got '%s'", task.GID)
|
||||
}
|
||||
|
||||
if task.Type() != tasktype.TaskTypeAria2 {
|
||||
t.Errorf("Expected task type to be TaskTypeAria2, got '%s'", task.Type())
|
||||
}
|
||||
|
||||
if task.TaskID() != "test-task-id" {
|
||||
t.Errorf("Expected TaskID() to return 'test-task-id', got '%s'", task.TaskID())
|
||||
}
|
||||
|
||||
if task.Storage.Name() != "test-storage" {
|
||||
t.Errorf("Expected storage name to be 'test-storage', got '%s'", task.Storage.Name())
|
||||
}
|
||||
}
|
||||
|
||||
func TestProgressTracker(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
mockStor := &mockStorage{name: "test-storage"}
|
||||
mockProg := &mockProgress{}
|
||||
|
||||
task := NewTask(
|
||||
"test-task-id",
|
||||
ctx,
|
||||
"test-gid",
|
||||
[]string{"http://example.com/file.zip"},
|
||||
nil,
|
||||
mockStor,
|
||||
"/test/path",
|
||||
mockProg,
|
||||
)
|
||||
|
||||
// Test OnStart
|
||||
mockProg.OnStart(ctx, task)
|
||||
if !mockProg.started {
|
||||
t.Error("Expected OnStart to set started to true")
|
||||
}
|
||||
|
||||
// Test OnProgress
|
||||
status := &aria2.Status{
|
||||
GID: "test-gid",
|
||||
Status: "active",
|
||||
TotalLength: "1000000",
|
||||
CompletedLength: "500000",
|
||||
DownloadSpeed: "100000",
|
||||
}
|
||||
mockProg.OnProgress(ctx, task, status)
|
||||
if mockProg.progress != 1 {
|
||||
t.Errorf("Expected progress to be 1, got %d", mockProg.progress)
|
||||
}
|
||||
|
||||
// Test OnDone
|
||||
mockProg.OnDone(ctx, task, nil)
|
||||
if !mockProg.done {
|
||||
t.Error("Expected OnDone to set done to true")
|
||||
}
|
||||
if mockProg.doneErr != nil {
|
||||
t.Errorf("Expected doneErr to be nil, got %v", mockProg.doneErr)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTaskTitle(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
mockStor := &mockStorage{name: "test-storage"}
|
||||
|
||||
task := NewTask(
|
||||
"test-task-id",
|
||||
ctx,
|
||||
"test-gid-123",
|
||||
[]string{"http://example.com/file.zip"},
|
||||
nil,
|
||||
mockStor,
|
||||
"/test/path",
|
||||
nil,
|
||||
)
|
||||
|
||||
title := task.Title()
|
||||
expectedSubstr := "test-gid-123"
|
||||
if len(title) == 0 {
|
||||
t.Error("Expected title to not be empty")
|
||||
}
|
||||
|
||||
// Check if title contains the GID
|
||||
found := false
|
||||
for i := 0; i < len(title)-len(expectedSubstr)+1; i++ {
|
||||
if title[i:i+len(expectedSubstr)] == expectedSubstr {
|
||||
found = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Errorf("Expected title to contain GID '%s', got '%s'", expectedSubstr, title)
|
||||
}
|
||||
}
|
||||
|
||||
func TestContextCancellation(t *testing.T) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
|
||||
defer cancel()
|
||||
|
||||
mockStor := &mockStorage{name: "test-storage"}
|
||||
mockProg := &mockProgress{}
|
||||
|
||||
task := NewTask(
|
||||
"test-task-id",
|
||||
ctx,
|
||||
"test-gid",
|
||||
[]string{"http://example.com/file.zip"},
|
||||
nil, // nil client will cause Execute to fail/timeout
|
||||
mockStor,
|
||||
"/test/path",
|
||||
mockProg,
|
||||
)
|
||||
|
||||
// Just verify the task structure is valid
|
||||
if task.ctx.Err() != nil {
|
||||
t.Error("Context should not be cancelled yet")
|
||||
}
|
||||
|
||||
// Wait for context to timeout
|
||||
<-ctx.Done()
|
||||
if ctx.Err() == nil {
|
||||
t.Error("Context should be cancelled after timeout")
|
||||
}
|
||||
}
|
||||
@@ -45,9 +45,17 @@ func (t *Task) Execute(ctx context.Context) error {
|
||||
fetchedTotalBytes.Add(resp.ContentLength)
|
||||
file.Size = resp.ContentLength
|
||||
if name := resp.Header.Get("Content-Disposition"); name != "" {
|
||||
// Set file name
|
||||
filename := parseFilename(name)
|
||||
file.Name = filename
|
||||
if filename != "" {
|
||||
file.Name = filename
|
||||
}
|
||||
}
|
||||
// extract filename from URL if Content-Disposition is empty or invalid
|
||||
if file.Name == "" {
|
||||
file.Name = parseFilenameFromURL(file.URL)
|
||||
}
|
||||
if file.Name == "" {
|
||||
return fmt.Errorf("failed to determine filename for %s: Content-Disposition header is empty and URL does not contain a valid filename", file.URL)
|
||||
}
|
||||
|
||||
return nil
|
||||
|
||||
@@ -76,6 +76,9 @@ func (t *Task) StorageName() string {
|
||||
|
||||
// StoragePath implements TaskInfo.
|
||||
func (t *Task) StoragePath() string {
|
||||
if len(t.files) == 1 {
|
||||
return t.StorPath + "/" + t.files[0].Name
|
||||
}
|
||||
return t.StorPath
|
||||
}
|
||||
|
||||
|
||||
@@ -144,6 +144,41 @@ func tryDecodeGBK(s string) string {
|
||||
return ""
|
||||
}
|
||||
|
||||
// parseFilenameFromURL extracts filename from URL path
|
||||
// This is used as a fallback when Content-Disposition is not available
|
||||
func parseFilenameFromURL(rawURL string) string {
|
||||
parsed, err := url.Parse(rawURL)
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
|
||||
// Get the path part and extract the last segment
|
||||
path := parsed.Path
|
||||
if path == "" {
|
||||
return ""
|
||||
}
|
||||
|
||||
// URL decode the path first
|
||||
decodedPath, err := url.PathUnescape(path)
|
||||
if err != nil {
|
||||
decodedPath = path
|
||||
}
|
||||
|
||||
// Get the last segment of the path
|
||||
lastSlash := strings.LastIndex(decodedPath, "/")
|
||||
if lastSlash == -1 {
|
||||
return decodedPath
|
||||
}
|
||||
filename := decodedPath[lastSlash+1:]
|
||||
|
||||
// Remove query string if somehow still present
|
||||
if idx := strings.Index(filename, "?"); idx != -1 {
|
||||
filename = filename[:idx]
|
||||
}
|
||||
|
||||
return filename
|
||||
}
|
||||
|
||||
// parseFilenameFallback manually parses filename= when mime.ParseMediaType fails
|
||||
func parseFilenameFallback(cd string) string {
|
||||
// Look for filename= (case-insensitive)
|
||||
|
||||
73
core/tasks/directlinks/util_test.go
Normal file
73
core/tasks/directlinks/util_test.go
Normal file
@@ -0,0 +1,73 @@
|
||||
package directlinks
|
||||
|
||||
import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestParseFilenameFromURL(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
url string
|
||||
expected string
|
||||
}{
|
||||
{
|
||||
name: "simple filename",
|
||||
url: "https://example.com/files/document.pdf",
|
||||
expected: "document.pdf",
|
||||
},
|
||||
{
|
||||
name: "filename with encoded characters",
|
||||
url: "https://example.com/files/%E6%B5%8B%E8%AF%95.zip",
|
||||
expected: "测试.zip",
|
||||
},
|
||||
{
|
||||
name: "filename with query string in URL",
|
||||
url: "https://example.com/files/image.png?token=abc123",
|
||||
expected: "image.png",
|
||||
},
|
||||
{
|
||||
name: "nested path",
|
||||
url: "https://example.com/a/b/c/file.txt",
|
||||
expected: "file.txt",
|
||||
},
|
||||
{
|
||||
name: "URL with port",
|
||||
url: "https://example.com:8080/downloads/archive.tar.gz",
|
||||
expected: "archive.tar.gz",
|
||||
},
|
||||
{
|
||||
name: "empty path",
|
||||
url: "https://example.com",
|
||||
expected: "",
|
||||
},
|
||||
{
|
||||
name: "root path only",
|
||||
url: "https://example.com/",
|
||||
expected: "",
|
||||
},
|
||||
{
|
||||
name: "filename with spaces encoded",
|
||||
url: "https://example.com/my%20file%20name.pdf",
|
||||
expected: "my file name.pdf",
|
||||
},
|
||||
{
|
||||
name: "complex encoded filename",
|
||||
url: "https://example.com/downloads/%E4%B8%AD%E6%96%87%E6%96%87%E4%BB%B6.docx",
|
||||
expected: "中文文件.docx",
|
||||
},
|
||||
{
|
||||
name: "invalid URL",
|
||||
url: "://invalid-url",
|
||||
expected: "",
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
result := parseFilenameFromURL(tt.url)
|
||||
if result != tt.expected {
|
||||
t.Errorf("parseFilenameFromURL(%q) = %q, want %q", tt.url, result, tt.expected)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
142
core/tasks/transfer/execute.go
Normal file
142
core/tasks/transfer/execute.go
Normal file
@@ -0,0 +1,142 @@
|
||||
package transfer
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/ctxkey"
|
||||
"github.com/krau/SaveAny-Bot/storage"
|
||||
"golang.org/x/sync/errgroup"
|
||||
)
|
||||
|
||||
// Execute implements core.Executable.
|
||||
func (t *Task) Execute(ctx context.Context) error {
|
||||
logger := log.FromContext(ctx).WithPrefix(fmt.Sprintf("transfer[%s]", t.ID))
|
||||
logger.Info("Starting transfer task")
|
||||
t.Progress.OnStart(ctx, t)
|
||||
|
||||
workers := config.C().Workers
|
||||
eg, gctx := errgroup.WithContext(ctx)
|
||||
eg.SetLimit(workers)
|
||||
|
||||
for _, elem := range t.elems {
|
||||
eg.Go(func() error {
|
||||
t.processingMu.RLock()
|
||||
if t.processing[elem.ID] != nil {
|
||||
t.processingMu.RUnlock()
|
||||
return fmt.Errorf("element with ID %s is already being processed", elem.ID)
|
||||
}
|
||||
t.processingMu.RUnlock()
|
||||
|
||||
t.processingMu.Lock()
|
||||
t.processing[elem.ID] = &elem
|
||||
t.processingMu.Unlock()
|
||||
|
||||
defer func() {
|
||||
t.processingMu.Lock()
|
||||
delete(t.processing, elem.ID)
|
||||
t.processingMu.Unlock()
|
||||
}()
|
||||
|
||||
err := t.processElement(gctx, elem)
|
||||
if err != nil && !t.IgnoreErrors {
|
||||
return err
|
||||
}
|
||||
if err != nil {
|
||||
t.processingMu.Lock()
|
||||
t.failed[elem.ID] = err
|
||||
t.processingMu.Unlock()
|
||||
logger.Errorf("Failed to process file %s: %v", elem.FileInfo.Name, err)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
err := eg.Wait()
|
||||
if err != nil {
|
||||
logger.Errorf("Error during transfer processing: %v", err)
|
||||
} else {
|
||||
logger.Info("Transfer task completed successfully")
|
||||
}
|
||||
|
||||
t.Progress.OnDone(ctx, t, err)
|
||||
return err
|
||||
}
|
||||
|
||||
func (t *Task) processElement(ctx context.Context, elem TaskElement) error {
|
||||
logger := log.FromContext(ctx).WithPrefix(fmt.Sprintf("file[%s]", elem.FileInfo.Name))
|
||||
|
||||
// Check whether the source storage supports reading
|
||||
readableStorage, ok := elem.SourceStorage.(storage.StorageReadable)
|
||||
if !ok {
|
||||
return fmt.Errorf("source storage %s does not support reading", elem.SourceStorage.Name())
|
||||
}
|
||||
|
||||
logger.Info("Opening file from source storage")
|
||||
reader, size, err := readableStorage.OpenFile(ctx, elem.SourcePath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to open file: %w", err)
|
||||
}
|
||||
defer reader.Close()
|
||||
|
||||
// Build target storage path: /target_path/filename
|
||||
storagePath := path.Join(elem.TargetPath, elem.FileInfo.Name)
|
||||
|
||||
// Inject file size into context
|
||||
ctx = context.WithValue(ctx, ctxkey.ContentLength, size)
|
||||
|
||||
if config.C().Stream {
|
||||
if err := elem.TargetStorage.Save(ctx, reader, storagePath); err != nil {
|
||||
return fmt.Errorf("failed to upload file to storage: %w", err)
|
||||
}
|
||||
} else {
|
||||
logger.Info("Downloading to temporary file for ReadSeeker support")
|
||||
tempFile, err := t.downloadToTemp(reader, elem.FileInfo.Name)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to download to temp: %w", err)
|
||||
}
|
||||
defer os.Remove(tempFile.Name())
|
||||
defer tempFile.Close()
|
||||
|
||||
if _, err := tempFile.Seek(0, io.SeekStart); err != nil {
|
||||
return fmt.Errorf("failed to seek temp file: %w", err)
|
||||
}
|
||||
|
||||
logger.Infof("Uploading file to storage (size: %d bytes)", size)
|
||||
if err := elem.TargetStorage.Save(ctx, tempFile, storagePath); err != nil {
|
||||
return fmt.Errorf("failed to upload file to storage: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
t.uploaded.Add(size)
|
||||
t.Progress.OnProgress(ctx, t)
|
||||
|
||||
logger.Info("File uploaded successfully")
|
||||
return nil
|
||||
}
|
||||
|
||||
func (t *Task) downloadToTemp(reader io.Reader, filename string) (*os.File, error) {
|
||||
tempDir := config.C().Temp.BasePath
|
||||
if tempDir == "" {
|
||||
tempDir = os.TempDir()
|
||||
}
|
||||
|
||||
tempFile, err := os.CreateTemp(tempDir, filepath.Base(filename)+"-*.tmp")
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to create temp file: %w", err)
|
||||
}
|
||||
|
||||
if _, err := io.Copy(tempFile, reader); err != nil {
|
||||
tempFile.Close()
|
||||
os.Remove(tempFile.Name())
|
||||
return nil, fmt.Errorf("failed to copy to temp file: %w", err)
|
||||
}
|
||||
|
||||
return tempFile, nil
|
||||
}
|
||||
247
core/tasks/transfer/progress.go
Normal file
247
core/tasks/transfer/progress.go
Normal file
@@ -0,0 +1,247 @@
|
||||
package transfer
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/gotd/td/telegram/message/entity"
|
||||
"github.com/gotd/td/telegram/message/styling"
|
||||
"github.com/gotd/td/tg"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/dlutil"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
|
||||
)
|
||||
|
||||
type ProgressTracker interface {
|
||||
OnStart(ctx context.Context, info TaskInfo)
|
||||
OnProgress(ctx context.Context, info TaskInfo)
|
||||
OnDone(ctx context.Context, info TaskInfo, err error)
|
||||
}
|
||||
|
||||
type Progress struct {
|
||||
MessageID int
|
||||
ChatID int64
|
||||
start time.Time
|
||||
lastUpdatePercent atomic.Int32
|
||||
}
|
||||
|
||||
func NewProgressTracker(messageID int, chatID int64) ProgressTracker {
|
||||
return &Progress{
|
||||
MessageID: messageID,
|
||||
ChatID: chatID,
|
||||
}
|
||||
}
|
||||
|
||||
func (p *Progress) OnStart(ctx context.Context, info TaskInfo) {
|
||||
p.start = time.Now()
|
||||
p.lastUpdatePercent.Store(0)
|
||||
log.FromContext(ctx).Debugf("Transfer task progress tracking started for message %d in chat %d", p.MessageID, p.ChatID)
|
||||
|
||||
sizeMB := float64(info.TotalSize()) / (1024 * 1024)
|
||||
statsText := i18n.T(i18nk.BotMsgTransferStartStats, map[string]any{
|
||||
"SizeMB": fmt.Sprintf("%.2f", sizeMB),
|
||||
"Count": info.Count(),
|
||||
})
|
||||
|
||||
entityBuilder := entity.Builder{}
|
||||
if err := styling.Perform(&entityBuilder,
|
||||
styling.Plain(i18n.T(i18nk.BotMsgProgressTransferStartPrefix, nil)),
|
||||
styling.Code(statsText),
|
||||
); err != nil {
|
||||
log.FromContext(ctx).Errorf("Failed to build entities: %s", err)
|
||||
return
|
||||
}
|
||||
|
||||
text, entities := entityBuilder.Complete()
|
||||
req := &tg.MessagesEditMessageRequest{
|
||||
ID: p.MessageID,
|
||||
}
|
||||
req.SetMessage(text)
|
||||
req.SetEntities(entities)
|
||||
req.SetReplyMarkup(&tg.ReplyInlineMarkup{
|
||||
Rows: []tg.KeyboardButtonRow{
|
||||
{
|
||||
Buttons: []tg.KeyboardButtonClass{
|
||||
tgutil.BuildCancelButton(info.TaskID()),
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
ext := tgutil.ExtFromContext(ctx)
|
||||
if ext != nil {
|
||||
_, err := ext.EditMessage(p.ChatID, req)
|
||||
if err != nil {
|
||||
log.FromContext(ctx).Errorf("Failed to send progress start message: %s", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (p *Progress) OnProgress(ctx context.Context, info TaskInfo) {
|
||||
if !shouldUpdateProgress(info.TotalSize(), info.Uploaded(), int(p.lastUpdatePercent.Load())) {
|
||||
return
|
||||
}
|
||||
percent := int((info.Uploaded() * 100) / info.TotalSize())
|
||||
if p.lastUpdatePercent.Load() == int32(percent) {
|
||||
return
|
||||
}
|
||||
p.lastUpdatePercent.Store(int32(percent))
|
||||
|
||||
log.FromContext(ctx).Debugf("Progress update: %s, %d/%d", info.TaskID(), info.Uploaded(), info.TotalSize())
|
||||
|
||||
entityBuilder := entity.Builder{}
|
||||
var progressText strings.Builder
|
||||
|
||||
progressText.WriteString(i18n.T(i18nk.BotMsgProgressTransferProgressPrefix, nil))
|
||||
fmt.Fprintf(&progressText, "%d%%", percent)
|
||||
progressText.WriteString(i18n.T(i18nk.BotMsgProgressTransferUploadedPrefix, nil))
|
||||
fmt.Fprintf(&progressText, "%.2f MB / %.2f MB",
|
||||
float64(info.Uploaded())/(1024*1024),
|
||||
float64(info.TotalSize())/(1024*1024))
|
||||
|
||||
if p.start.Unix() > 0 {
|
||||
elapsed := time.Since(p.start)
|
||||
speed := float64(info.Uploaded()) / elapsed.Seconds()
|
||||
progressText.WriteString(i18n.T(i18nk.BotMsgProgressTransferSpeedPrefix, nil))
|
||||
progressText.WriteString(dlutil.FormatSize(int64(speed)) + "/s")
|
||||
|
||||
if info.Uploaded() > 0 {
|
||||
remaining := time.Duration(float64(info.TotalSize()-info.Uploaded()) / speed * float64(time.Second))
|
||||
progressText.WriteString(i18n.T(i18nk.BotMsgProgressTransferRemainingTimePrefix, nil))
|
||||
progressText.WriteString(formatDuration(remaining))
|
||||
}
|
||||
}
|
||||
|
||||
processing := info.Processing()
|
||||
if len(processing) > 0 {
|
||||
progressText.WriteString(i18n.T(i18nk.BotMsgProgressTransferProcessingPrefix, nil))
|
||||
for i, elem := range processing {
|
||||
if i >= 3 {
|
||||
progressText.WriteString(i18n.T(i18nk.BotMsgProgressTransferProcessingMore, map[string]any{"Count": len(processing) - 3}))
|
||||
break
|
||||
}
|
||||
fmt.Fprintf(&progressText, "- %s\n", elem.FileName())
|
||||
}
|
||||
}
|
||||
|
||||
if err := styling.Perform(&entityBuilder,
|
||||
styling.Plain(progressText.String()),
|
||||
); err != nil {
|
||||
log.FromContext(ctx).Errorf("Failed to build entities: %s", err)
|
||||
return
|
||||
}
|
||||
|
||||
text, entities := entityBuilder.Complete()
|
||||
req := &tg.MessagesEditMessageRequest{
|
||||
ID: p.MessageID,
|
||||
}
|
||||
req.SetMessage(text)
|
||||
req.SetEntities(entities)
|
||||
req.SetReplyMarkup(&tg.ReplyInlineMarkup{
|
||||
Rows: []tg.KeyboardButtonRow{
|
||||
{
|
||||
Buttons: []tg.KeyboardButtonClass{
|
||||
tgutil.BuildCancelButton(info.TaskID()),
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
ext := tgutil.ExtFromContext(ctx)
|
||||
if ext != nil {
|
||||
ext.EditMessage(p.ChatID, req)
|
||||
}
|
||||
}
|
||||
|
||||
func (p *Progress) OnDone(ctx context.Context, info TaskInfo, err error) {
|
||||
log.FromContext(ctx).Debugf("Transfer task progress tracking done for message %d in chat %d", p.MessageID, p.ChatID)
|
||||
|
||||
entityBuilder := entity.Builder{}
|
||||
var resultText strings.Builder
|
||||
|
||||
if err != nil {
|
||||
resultText.WriteString(i18n.T(i18nk.BotMsgProgressTransferFailedPrefix, nil))
|
||||
resultText.WriteString(i18n.T(i18nk.BotMsgProgressErrorPrefix, nil))
|
||||
fmt.Fprintf(&resultText, "%v\n", err)
|
||||
} else {
|
||||
resultText.WriteString(i18n.T(i18nk.BotMsgProgressTransferSuccessPrefix, nil))
|
||||
}
|
||||
|
||||
elapsed := time.Since(p.start)
|
||||
resultText.WriteString(i18n.T(i18nk.BotMsgProgressTransferTotalFilesPrefix, nil))
|
||||
fmt.Fprintf(&resultText, "%d\n", info.Count())
|
||||
resultText.WriteString(i18n.T(i18nk.BotMsgProgressTransferTotalSizePrefix, nil))
|
||||
fmt.Fprintf(&resultText, "%.2f MB\n", float64(info.TotalSize())/(1024*1024))
|
||||
resultText.WriteString(i18n.T(i18nk.BotMsgProgressTransferUploadedPrefix, nil))
|
||||
fmt.Fprintf(&resultText, "%.2f MB\n", float64(info.Uploaded())/(1024*1024))
|
||||
resultText.WriteString(i18n.T(i18nk.BotMsgProgressTransferElapsedTimePrefix, nil))
|
||||
fmt.Fprintf(&resultText, "%s\n", formatDuration(elapsed))
|
||||
|
||||
if elapsed.Seconds() > 0 {
|
||||
avgSpeed := float64(info.Uploaded()) / elapsed.Seconds()
|
||||
resultText.WriteString(i18n.T(i18nk.BotMsgProgressTransferAvgSpeedPrefix, nil))
|
||||
fmt.Fprintf(&resultText, "%s/s\n", dlutil.FormatSize(int64(avgSpeed)))
|
||||
}
|
||||
|
||||
failedFiles := info.FailedFiles()
|
||||
if len(failedFiles) > 0 {
|
||||
resultText.WriteString(i18n.T(i18nk.BotMsgProgressTransferFailedFilesPrefix, nil))
|
||||
fmt.Fprintf(&resultText, "%d\n", len(failedFiles))
|
||||
for i, name := range failedFiles {
|
||||
if i >= 5 {
|
||||
resultText.WriteString(i18n.T(i18nk.BotMsgProgressTransferProcessingMore, map[string]any{"Count": len(failedFiles) - 5}))
|
||||
break
|
||||
}
|
||||
fmt.Fprintf(&resultText, "- %s\n", name)
|
||||
}
|
||||
}
|
||||
|
||||
if err := styling.Perform(&entityBuilder,
|
||||
styling.Plain(resultText.String()),
|
||||
); err != nil {
|
||||
log.FromContext(ctx).Errorf("Failed to build entities: %s", err)
|
||||
return
|
||||
}
|
||||
|
||||
text, entities := entityBuilder.Complete()
|
||||
req := &tg.MessagesEditMessageRequest{
|
||||
ID: p.MessageID,
|
||||
}
|
||||
req.SetMessage(text)
|
||||
req.SetEntities(entities)
|
||||
|
||||
ext := tgutil.ExtFromContext(ctx)
|
||||
if ext != nil {
|
||||
ext.EditMessage(p.ChatID, req)
|
||||
}
|
||||
}
|
||||
|
||||
func shouldUpdateProgress(total, current int64, lastPercent int) bool {
|
||||
if total == 0 {
|
||||
return false
|
||||
}
|
||||
currentPercent := int((current * 100) / total)
|
||||
return currentPercent > lastPercent && currentPercent%5 == 0
|
||||
}
|
||||
|
||||
func formatDuration(d time.Duration) string {
|
||||
d = d.Round(time.Second)
|
||||
h := d / time.Hour
|
||||
d -= h * time.Hour
|
||||
m := d / time.Minute
|
||||
d -= m * time.Minute
|
||||
s := d / time.Second
|
||||
|
||||
if h > 0 {
|
||||
return fmt.Sprintf("%dh%dm%ds", h, m, s)
|
||||
}
|
||||
if m > 0 {
|
||||
return fmt.Sprintf("%dm%ds", m, s)
|
||||
}
|
||||
return fmt.Sprintf("%ds", s)
|
||||
}
|
||||
97
core/tasks/transfer/task.go
Normal file
97
core/tasks/transfer/task.go
Normal file
@@ -0,0 +1,97 @@
|
||||
package transfer
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/core"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
|
||||
"github.com/krau/SaveAny-Bot/pkg/storagetypes"
|
||||
"github.com/krau/SaveAny-Bot/storage"
|
||||
"github.com/rs/xid"
|
||||
)
|
||||
|
||||
var _ core.Executable = (*Task)(nil)
|
||||
|
||||
type TaskElement struct {
|
||||
ID string
|
||||
SourceStorage storage.Storage
|
||||
SourcePath string
|
||||
FileInfo storagetypes.FileInfo
|
||||
TargetStorage storage.Storage
|
||||
TargetPath string
|
||||
}
|
||||
|
||||
type Task struct {
|
||||
ID string
|
||||
ctx context.Context
|
||||
elems []TaskElement
|
||||
Progress ProgressTracker
|
||||
IgnoreErrors bool
|
||||
uploaded atomic.Int64
|
||||
totalSize int64
|
||||
processing map[string]TaskElementInfo
|
||||
processingMu sync.RWMutex
|
||||
failed map[string]error
|
||||
}
|
||||
|
||||
// Title implements core.Executable.
|
||||
func (t *Task) Title() string {
|
||||
return fmt.Sprintf("[%s](%d files/%.2fMB)", t.Type(), len(t.elems), float64(t.totalSize)/(1024*1024))
|
||||
}
|
||||
|
||||
// Type implements core.Executable.
|
||||
func (t *Task) Type() tasktype.TaskType {
|
||||
return tasktype.TaskTypeTransfer
|
||||
}
|
||||
|
||||
// TaskID implements core.Executable.
|
||||
func (t *Task) TaskID() string {
|
||||
return t.ID
|
||||
}
|
||||
|
||||
func NewTaskElement(
|
||||
sourceStorage storage.Storage,
|
||||
fileInfo storagetypes.FileInfo,
|
||||
targetStorage storage.Storage,
|
||||
targetPath string,
|
||||
) *TaskElement {
|
||||
id := xid.New().String()
|
||||
return &TaskElement{
|
||||
ID: id,
|
||||
SourceStorage: sourceStorage,
|
||||
SourcePath: fileInfo.Path,
|
||||
FileInfo: fileInfo,
|
||||
TargetStorage: targetStorage,
|
||||
TargetPath: targetPath,
|
||||
}
|
||||
}
|
||||
|
||||
func NewTransferTask(
|
||||
id string,
|
||||
ctx context.Context,
|
||||
elems []TaskElement,
|
||||
progress ProgressTracker,
|
||||
ignoreErrors bool,
|
||||
) *Task {
|
||||
task := &Task{
|
||||
ID: id,
|
||||
ctx: ctx,
|
||||
elems: elems,
|
||||
Progress: progress,
|
||||
uploaded: atomic.Int64{},
|
||||
totalSize: func() int64 {
|
||||
var total int64
|
||||
for _, elem := range elems {
|
||||
total += elem.FileInfo.Size
|
||||
}
|
||||
return total
|
||||
}(),
|
||||
processing: make(map[string]TaskElementInfo),
|
||||
IgnoreErrors: ignoreErrors,
|
||||
failed: make(map[string]error),
|
||||
}
|
||||
return task
|
||||
}
|
||||
73
core/tasks/transfer/taskinfo.go
Normal file
73
core/tasks/transfer/taskinfo.go
Normal file
@@ -0,0 +1,73 @@
|
||||
package transfer
|
||||
|
||||
type TaskElementInfo interface {
|
||||
FileName() string
|
||||
FileSize() int64
|
||||
GetSourcePath() string
|
||||
SourceStorageName() string
|
||||
}
|
||||
|
||||
func (e *TaskElement) FileName() string {
|
||||
return e.FileInfo.Name
|
||||
}
|
||||
|
||||
func (e *TaskElement) FileSize() int64 {
|
||||
return e.FileInfo.Size
|
||||
}
|
||||
|
||||
func (e *TaskElement) GetSourcePath() string {
|
||||
return e.SourcePath
|
||||
}
|
||||
|
||||
func (e *TaskElement) SourceStorageName() string {
|
||||
return e.SourceStorage.Name()
|
||||
}
|
||||
|
||||
type TaskInfo interface {
|
||||
TaskID() string
|
||||
TotalSize() int64
|
||||
Uploaded() int64
|
||||
Count() int
|
||||
Processing() []TaskElementInfo
|
||||
FailedFiles() []string
|
||||
}
|
||||
|
||||
func (t *Task) TotalSize() int64 {
|
||||
return t.totalSize
|
||||
}
|
||||
|
||||
func (t *Task) Uploaded() int64 {
|
||||
return t.uploaded.Load()
|
||||
}
|
||||
|
||||
func (t *Task) Count() int {
|
||||
return len(t.elems)
|
||||
}
|
||||
|
||||
func (t *Task) Processing() []TaskElementInfo {
|
||||
t.processingMu.RLock()
|
||||
defer t.processingMu.RUnlock()
|
||||
|
||||
result := make([]TaskElementInfo, 0, len(t.processing))
|
||||
for _, elem := range t.processing {
|
||||
result = append(result, elem)
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
func (t *Task) FailedFiles() []string {
|
||||
t.processingMu.RLock()
|
||||
defer t.processingMu.RUnlock()
|
||||
|
||||
result := make([]string, 0, len(t.failed))
|
||||
for id := range t.failed {
|
||||
// Find the element by ID
|
||||
for _, elem := range t.elems {
|
||||
if elem.ID == id {
|
||||
result = append(result, elem.FileInfo.Name)
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
return result
|
||||
}
|
||||
194
core/tasks/ytdlp/execute.go
Normal file
194
core/tasks/ytdlp/execute.go
Normal file
@@ -0,0 +1,194 @@
|
||||
package ytdlp
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"github.com/charmbracelet/log"
|
||||
ytdlp "github.com/lrstanley/go-ytdlp"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/config"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/ctxkey"
|
||||
)
|
||||
|
||||
// Execute implements core.Executable.
|
||||
func (t *Task) Execute(ctx context.Context) error {
|
||||
logger := log.FromContext(ctx)
|
||||
logger.Infof("Starting yt-dlp download task %s", t.ID)
|
||||
|
||||
if t.Progress != nil {
|
||||
t.Progress.OnStart(ctx, t)
|
||||
}
|
||||
|
||||
// Create temporary directory for downloads
|
||||
tempDir, err := os.MkdirTemp(config.C().Temp.BasePath, "ytdlp-*")
|
||||
if err != nil {
|
||||
logger.Errorf("Failed to create temp directory: %v", err)
|
||||
if t.Progress != nil {
|
||||
t.Progress.OnDone(ctx, t, err)
|
||||
}
|
||||
return fmt.Errorf("failed to create temp directory: %w", err)
|
||||
}
|
||||
defer os.RemoveAll(tempDir) // Clean up temp directory
|
||||
|
||||
logger.Debugf("Created temp directory: %s", tempDir)
|
||||
|
||||
// Download files using yt-dlp
|
||||
downloadedFiles, err := t.downloadFiles(ctx, tempDir)
|
||||
if err != nil {
|
||||
logger.Errorf("yt-dlp download failed: %v", err)
|
||||
if t.Progress != nil {
|
||||
t.Progress.OnDone(ctx, t, err)
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
if len(downloadedFiles) == 0 {
|
||||
err := errors.New("no files were downloaded")
|
||||
logger.Error(err.Error())
|
||||
if t.Progress != nil {
|
||||
t.Progress.OnDone(ctx, t, err)
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
// Transfer downloaded files to storage
|
||||
logger.Infof("Transferring %d file(s) to storage %s", len(downloadedFiles), t.Storage.Name())
|
||||
for _, filePath := range downloadedFiles {
|
||||
if err := t.transferFile(ctx, filePath); err != nil {
|
||||
logger.Errorf("File transfer failed: %v", err)
|
||||
if t.Progress != nil {
|
||||
t.Progress.OnDone(ctx, t, err)
|
||||
}
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
logger.Infof("yt-dlp task %s completed successfully", t.ID)
|
||||
if t.Progress != nil {
|
||||
t.Progress.OnDone(ctx, t, nil)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// downloadFiles downloads files using yt-dlp and returns the list of downloaded file paths
|
||||
func (t *Task) downloadFiles(ctx context.Context, tempDir string) ([]string, error) {
|
||||
logger := log.FromContext(ctx)
|
||||
|
||||
// Configure yt-dlp command with essential settings
|
||||
// Always set output path to ensure files go to temp directory
|
||||
cmd := ytdlp.New().
|
||||
Output(filepath.Join(tempDir, "%(title)s.%(ext)s"))
|
||||
|
||||
// If no custom flags are provided, use default behavior
|
||||
if len(t.Flags) == 0 {
|
||||
cmd = cmd.
|
||||
FormatSort("res,ext:mp4:m4a").
|
||||
RecodeVideo("mp4").
|
||||
RestrictFilenames()
|
||||
}
|
||||
// Note: If custom flags are provided, users have full control over format/quality
|
||||
// The output path is always set above to ensure downloads go to the correct directory
|
||||
|
||||
if t.Progress != nil {
|
||||
t.Progress.OnProgress(ctx, t, "Downloading...")
|
||||
}
|
||||
|
||||
// Execute download with URLs and custom flags
|
||||
logger.Infof("Executing yt-dlp for %d URL(s) with %d custom flag(s)", len(t.URLs), len(t.Flags))
|
||||
|
||||
// Combine flags and URLs as arguments (flags first, then URLs)
|
||||
// yt-dlp accepts: yt-dlp [OPTIONS] URL [URL...]
|
||||
args := append(t.Flags, t.URLs...)
|
||||
|
||||
// Run with context for cancellation support
|
||||
result, err := cmd.Run(ctx, args...)
|
||||
if err != nil {
|
||||
// Check if context was canceled
|
||||
if errors.Is(err, context.Canceled) {
|
||||
return nil, err
|
||||
}
|
||||
return nil, fmt.Errorf("yt-dlp execution failed: %w", err)
|
||||
}
|
||||
|
||||
if result.ExitCode != 0 {
|
||||
return nil, fmt.Errorf("yt-dlp exited with code %d: %s", result.ExitCode, result.Stderr)
|
||||
}
|
||||
|
||||
// List downloaded files
|
||||
files, err := os.ReadDir(tempDir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to read temp directory: %w", err)
|
||||
}
|
||||
|
||||
var downloadedFiles []string
|
||||
for _, file := range files {
|
||||
if file.IsDir() {
|
||||
continue
|
||||
}
|
||||
fullPath := filepath.Join(tempDir, file.Name())
|
||||
downloadedFiles = append(downloadedFiles, fullPath)
|
||||
logger.Debugf("Downloaded file: %s", file.Name())
|
||||
}
|
||||
|
||||
return downloadedFiles, nil
|
||||
}
|
||||
|
||||
// transferFile transfers a single file to storage
|
||||
func (t *Task) transferFile(ctx context.Context, filePath string) error {
|
||||
logger := log.FromContext(ctx)
|
||||
|
||||
// Check if file exists
|
||||
fileInfo, err := os.Stat(filePath)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
logger.Warnf("Downloaded file not found: %s", filePath)
|
||||
return nil // Not a fatal error
|
||||
}
|
||||
return fmt.Errorf("failed to stat file %s: %w", filePath, err)
|
||||
}
|
||||
|
||||
// Open file
|
||||
f, err := os.Open(filePath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to open file %s: %w", filePath, err)
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
// Set content length in context for storage
|
||||
ctx = context.WithValue(ctx, ctxkey.ContentLength, fileInfo.Size())
|
||||
|
||||
// Save to storage
|
||||
fileName := filepath.Base(filePath)
|
||||
// Remove special characters from filename if needed
|
||||
fileName = sanitizeFilename(fileName)
|
||||
destPath := filepath.Join(t.StorPath, fileName)
|
||||
|
||||
logger.Infof("Transferring file %s to %s:%s", fileName, t.Storage.Name(), destPath)
|
||||
|
||||
if err := t.Storage.Save(ctx, f, destPath); err != nil {
|
||||
return fmt.Errorf("failed to save file %s to storage: %w", fileName, err)
|
||||
}
|
||||
|
||||
logger.Infof("Successfully transferred file %s", fileName)
|
||||
|
||||
if t.Progress != nil {
|
||||
t.Progress.OnProgress(ctx, t, fmt.Sprintf("Transferred: %s", fileName))
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// sanitizeFilename removes or replaces problematic characters in filenames
|
||||
func sanitizeFilename(name string) string {
|
||||
// yt-dlp with --restrict-filenames should already handle most cases
|
||||
// but we can do additional sanitization if needed
|
||||
name = strings.ReplaceAll(name, ":", "_")
|
||||
name = strings.ReplaceAll(name, "\"", "'")
|
||||
return name
|
||||
}
|
||||
183
core/tasks/ytdlp/progress.go
Normal file
183
core/tasks/ytdlp/progress.go
Normal file
@@ -0,0 +1,183 @@
|
||||
package ytdlp
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"github.com/charmbracelet/log"
|
||||
"github.com/gotd/td/telegram/message/entity"
|
||||
"github.com/gotd/td/telegram/message/styling"
|
||||
"github.com/gotd/td/tg"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/common/i18n"
|
||||
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
|
||||
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
|
||||
)
|
||||
|
||||
// ProgressTracker defines the interface for tracking ytdlp task progress
|
||||
type ProgressTracker interface {
|
||||
OnStart(ctx context.Context, task *Task)
|
||||
OnProgress(ctx context.Context, task *Task, status string)
|
||||
OnDone(ctx context.Context, task *Task, err error)
|
||||
}
|
||||
|
||||
type Progress struct {
|
||||
msgID int
|
||||
chatID int64
|
||||
start time.Time
|
||||
lastUpdate atomic.Value // stores time.Time
|
||||
minUpdateInterval time.Duration
|
||||
}
|
||||
|
||||
// OnStart implements ProgressTracker.
|
||||
func (p *Progress) OnStart(ctx context.Context, task *Task) {
|
||||
logger := log.FromContext(ctx)
|
||||
p.start = time.Now()
|
||||
p.lastUpdate.Store(time.Now())
|
||||
p.minUpdateInterval = 2 * time.Second // Avoid too frequent updates
|
||||
logger.Infof("yt-dlp task started: message_id=%d, chat_id=%d, urls=%d", p.msgID, p.chatID, len(task.URLs))
|
||||
ext := tgutil.ExtFromContext(ctx)
|
||||
if ext == nil {
|
||||
return
|
||||
}
|
||||
entityBuilder := entity.Builder{}
|
||||
if err := styling.Perform(&entityBuilder,
|
||||
styling.Plain(i18n.T(i18nk.BotMsgProgressYtdlpStart, map[string]any{
|
||||
"Count": len(task.URLs),
|
||||
})),
|
||||
styling.Plain(i18n.T(i18nk.BotMsgProgressSavePathPrefix, nil)),
|
||||
styling.Code(fmt.Sprintf("[%s]:%s", task.Storage.Name(), task.StorPath)),
|
||||
); err != nil {
|
||||
log.FromContext(ctx).Errorf("Failed to build entities: %s", err)
|
||||
return
|
||||
}
|
||||
text, entities := entityBuilder.Complete()
|
||||
req := &tg.MessagesEditMessageRequest{
|
||||
ID: p.msgID,
|
||||
}
|
||||
req.SetMessage(text)
|
||||
req.SetEntities(entities)
|
||||
req.SetReplyMarkup(&tg.ReplyInlineMarkup{
|
||||
Rows: []tg.KeyboardButtonRow{
|
||||
{
|
||||
Buttons: []tg.KeyboardButtonClass{
|
||||
tgutil.BuildCancelButton(task.TaskID()),
|
||||
},
|
||||
},
|
||||
}},
|
||||
)
|
||||
ext.EditMessage(p.chatID, req)
|
||||
}
|
||||
|
||||
// OnProgress implements ProgressTracker.
|
||||
func (p *Progress) OnProgress(ctx context.Context, task *Task, status string) {
|
||||
// Throttle updates to avoid flooding Telegram API
|
||||
lastUpdateTime := p.lastUpdate.Load().(time.Time)
|
||||
if time.Since(lastUpdateTime) < p.minUpdateInterval {
|
||||
return
|
||||
}
|
||||
p.lastUpdate.Store(time.Now())
|
||||
|
||||
log.FromContext(ctx).Debugf("yt-dlp progress update: %s", status)
|
||||
|
||||
entityBuilder := entity.Builder{}
|
||||
if err := styling.Perform(&entityBuilder,
|
||||
styling.Plain(i18n.T(i18nk.BotMsgProgressYtdlpDownloading, map[string]any{
|
||||
"Count": len(task.URLs),
|
||||
})),
|
||||
styling.Plain(i18n.T(i18nk.BotMsgProgressSavePathPrefix, nil)),
|
||||
styling.Code(fmt.Sprintf("[%s]:%s", task.Storage.Name(), task.StorPath)),
|
||||
styling.Plain("\n\n"),
|
||||
styling.Plain(status),
|
||||
); err != nil {
|
||||
log.FromContext(ctx).Errorf("Failed to build entities: %s", err)
|
||||
return
|
||||
}
|
||||
text, entities := entityBuilder.Complete()
|
||||
req := &tg.MessagesEditMessageRequest{
|
||||
ID: p.msgID,
|
||||
}
|
||||
req.SetMessage(text)
|
||||
req.SetEntities(entities)
|
||||
req.SetReplyMarkup(&tg.ReplyInlineMarkup{
|
||||
Rows: []tg.KeyboardButtonRow{
|
||||
{
|
||||
Buttons: []tg.KeyboardButtonClass{
|
||||
tgutil.BuildCancelButton(task.TaskID()),
|
||||
},
|
||||
},
|
||||
}},
|
||||
)
|
||||
ext := tgutil.ExtFromContext(ctx)
|
||||
if ext != nil {
|
||||
ext.EditMessage(p.chatID, req)
|
||||
}
|
||||
}
|
||||
|
||||
// OnDone implements ProgressTracker.
|
||||
func (p *Progress) OnDone(ctx context.Context, task *Task, err error) {
|
||||
logger := log.FromContext(ctx)
|
||||
if err != nil {
|
||||
if errors.Is(err, context.Canceled) {
|
||||
logger.Infof("yt-dlp task %s was canceled", task.TaskID())
|
||||
ext := tgutil.ExtFromContext(ctx)
|
||||
if ext != nil {
|
||||
ext.EditMessage(p.chatID, &tg.MessagesEditMessageRequest{
|
||||
ID: p.msgID,
|
||||
Message: i18n.T(i18nk.BotMsgProgressTaskCanceledWithId, map[string]any{
|
||||
"TaskID": task.TaskID(),
|
||||
}),
|
||||
})
|
||||
}
|
||||
} else {
|
||||
logger.Errorf("yt-dlp task %s failed: %s", task.TaskID(), err)
|
||||
ext := tgutil.ExtFromContext(ctx)
|
||||
if ext != nil {
|
||||
ext.EditMessage(p.chatID, &tg.MessagesEditMessageRequest{
|
||||
ID: p.msgID,
|
||||
Message: i18n.T(i18nk.BotMsgProgressTaskFailedWithError, map[string]any{
|
||||
"Error": err.Error(),
|
||||
}),
|
||||
})
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
logger.Infof("yt-dlp task %s completed successfully", task.TaskID())
|
||||
|
||||
entityBuilder := entity.Builder{}
|
||||
if err := styling.Perform(&entityBuilder,
|
||||
styling.Plain(i18n.T(i18nk.BotMsgProgressYtdlpDone, map[string]any{
|
||||
"Count": len(task.URLs),
|
||||
})),
|
||||
styling.Plain(i18n.T(i18nk.BotMsgProgressSavePathPrefix, nil)),
|
||||
styling.Code(fmt.Sprintf("[%s]:%s", task.Storage.Name(), task.StorPath)),
|
||||
); err != nil {
|
||||
logger.Errorf("Failed to build entities: %s", err)
|
||||
return
|
||||
}
|
||||
text, entities := entityBuilder.Complete()
|
||||
req := &tg.MessagesEditMessageRequest{
|
||||
ID: p.msgID,
|
||||
}
|
||||
req.SetMessage(text)
|
||||
req.SetEntities(entities)
|
||||
|
||||
ext := tgutil.ExtFromContext(ctx)
|
||||
if ext != nil {
|
||||
ext.EditMessage(p.chatID, req)
|
||||
}
|
||||
}
|
||||
|
||||
var _ ProgressTracker = (*Progress)(nil)
|
||||
|
||||
func NewProgress(msgID int, userID int64) ProgressTracker {
|
||||
return &Progress{
|
||||
msgID: msgID,
|
||||
chatID: userID,
|
||||
minUpdateInterval: 2 * time.Second,
|
||||
}
|
||||
}
|
||||
61
core/tasks/ytdlp/task.go
Normal file
61
core/tasks/ytdlp/task.go
Normal file
@@ -0,0 +1,61 @@
|
||||
package ytdlp
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
|
||||
"github.com/krau/SaveAny-Bot/core"
|
||||
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
|
||||
"github.com/krau/SaveAny-Bot/storage"
|
||||
)
|
||||
|
||||
var _ core.Executable = (*Task)(nil)
|
||||
|
||||
type Task struct {
|
||||
ID string
|
||||
ctx context.Context
|
||||
URLs []string
|
||||
Flags []string
|
||||
Storage storage.Storage
|
||||
StorPath string
|
||||
Progress ProgressTracker
|
||||
}
|
||||
|
||||
// Title implements core.Executable.
|
||||
func (t *Task) Title() string {
|
||||
urlCount := len(t.URLs)
|
||||
if urlCount == 1 {
|
||||
return fmt.Sprintf("[%s](%s->%s:%s)", t.Type(), t.URLs[0], t.Storage.Name(), t.StorPath)
|
||||
}
|
||||
return fmt.Sprintf("[%s](%d URLs->%s:%s)", t.Type(), urlCount, t.Storage.Name(), t.StorPath)
|
||||
}
|
||||
|
||||
// Type implements core.Executable.
|
||||
func (t *Task) Type() tasktype.TaskType {
|
||||
return tasktype.TaskTypeYtdlp
|
||||
}
|
||||
|
||||
// TaskID implements core.Executable.
|
||||
func (t *Task) TaskID() string {
|
||||
return t.ID
|
||||
}
|
||||
|
||||
func NewTask(
|
||||
id string,
|
||||
ctx context.Context,
|
||||
urls []string,
|
||||
flags []string,
|
||||
stor storage.Storage,
|
||||
storPath string,
|
||||
progressTracker ProgressTracker,
|
||||
) *Task {
|
||||
return &Task{
|
||||
ID: id,
|
||||
ctx: ctx,
|
||||
URLs: urls,
|
||||
Flags: flags,
|
||||
Storage: stor,
|
||||
StorPath: storPath,
|
||||
Progress: progressTracker,
|
||||
}
|
||||
}
|
||||
114
core/tasks/ytdlp/task_test.go
Normal file
114
core/tasks/ytdlp/task_test.go
Normal file
@@ -0,0 +1,114 @@
|
||||
package ytdlp
|
||||
|
||||
import (
|
||||
"context"
|
||||
"io"
|
||||
"testing"
|
||||
|
||||
storcfg "github.com/krau/SaveAny-Bot/config/storage"
|
||||
storenum "github.com/krau/SaveAny-Bot/pkg/enums/storage"
|
||||
)
|
||||
|
||||
// MockStorage is a simple mock for testing
|
||||
type MockStorage struct{}
|
||||
|
||||
func (m *MockStorage) Init(ctx context.Context, cfg storcfg.StorageConfig) error { return nil }
|
||||
func (m *MockStorage) Type() storenum.StorageType { return "mock" }
|
||||
func (m *MockStorage) Name() string { return "test-storage" }
|
||||
func (m *MockStorage) JoinStoragePath(p string) string { return "test-path" }
|
||||
func (m *MockStorage) Save(ctx context.Context, reader io.Reader, path string) error { return nil }
|
||||
func (m *MockStorage) Exists(ctx context.Context, path string) bool { return false }
|
||||
|
||||
func TestNewTask(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
urls := []string{"https://example.com/video"}
|
||||
flags := []string{"--format", "best"}
|
||||
stor := &MockStorage{}
|
||||
storPath := "test-path"
|
||||
|
||||
task := NewTask("test-id", ctx, urls, flags, stor, storPath, nil)
|
||||
|
||||
if task == nil {
|
||||
t.Fatal("NewTask returned nil")
|
||||
}
|
||||
|
||||
if task.ID != "test-id" {
|
||||
t.Errorf("Expected task ID 'test-id', got '%s'", task.ID)
|
||||
}
|
||||
|
||||
if len(task.URLs) != 1 || task.URLs[0] != "https://example.com/video" {
|
||||
t.Errorf("Expected URLs to contain 'https://example.com/video', got %v", task.URLs)
|
||||
}
|
||||
|
||||
if len(task.Flags) != 2 || task.Flags[0] != "--format" || task.Flags[1] != "best" {
|
||||
t.Errorf("Expected flags to contain '--format' and 'best', got %v", task.Flags)
|
||||
}
|
||||
|
||||
if task.Storage.Name() != "test-storage" {
|
||||
t.Errorf("Expected storage name 'test-storage', got '%s'", task.Storage.Name())
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewTaskWithoutFlags(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
urls := []string{"https://example.com/video1", "https://example.com/video2"}
|
||||
var flags []string // No flags
|
||||
stor := &MockStorage{}
|
||||
storPath := "test-path"
|
||||
|
||||
task := NewTask("test-id-2", ctx, urls, flags, stor, storPath, nil)
|
||||
|
||||
if task == nil {
|
||||
t.Fatal("NewTask returned nil")
|
||||
}
|
||||
|
||||
if len(task.URLs) != 2 {
|
||||
t.Errorf("Expected 2 URLs, got %d", len(task.URLs))
|
||||
}
|
||||
|
||||
if len(task.Flags) != 0 {
|
||||
t.Errorf("Expected 0 flags, got %d", len(task.Flags))
|
||||
}
|
||||
}
|
||||
|
||||
func TestTaskTitle(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
stor := &MockStorage{}
|
||||
|
||||
// Test with single URL
|
||||
task1 := NewTask("id1", ctx, []string{"https://example.com/video"}, nil, stor, "path", nil)
|
||||
title1 := task1.Title()
|
||||
if title1 == "" {
|
||||
t.Error("Task title should not be empty")
|
||||
}
|
||||
|
||||
// Test with multiple URLs
|
||||
task2 := NewTask("id2", ctx, []string{"https://example.com/v1", "https://example.com/v2"}, nil, stor, "path", nil)
|
||||
title2 := task2.Title()
|
||||
if title2 == "" {
|
||||
t.Error("Task title should not be empty")
|
||||
}
|
||||
}
|
||||
|
||||
func TestTaskType(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
stor := &MockStorage{}
|
||||
task := NewTask("id", ctx, []string{"https://example.com"}, nil, stor, "path", nil)
|
||||
|
||||
taskType := task.Type()
|
||||
if taskType.String() != "ytdlp" {
|
||||
t.Errorf("Expected task type 'ytdlp', got '%s'", taskType.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestTaskID(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
stor := &MockStorage{}
|
||||
expectedID := "test-task-id-123"
|
||||
|
||||
task := NewTask(expectedID, ctx, []string{"https://example.com"}, nil, stor, "path", nil)
|
||||
|
||||
if task.TaskID() != expectedID {
|
||||
t.Errorf("Expected task ID '%s', got '%s'", expectedID, task.TaskID())
|
||||
}
|
||||
}
|
||||
@@ -49,4 +49,4 @@ func GetUserByID(ctx context.Context, id uint) (*User, error) {
|
||||
Preload(clause.Associations).
|
||||
Where("id = ?", id).First(&user).Error
|
||||
return &user, err
|
||||
}
|
||||
}
|
||||
|
||||
@@ -20,6 +20,9 @@ Save Any Bot is a tool that allows you to save files from Telegram to various st
|
||||
- Multi-user
|
||||
- Automatic organization based on storage rules
|
||||
- Watch specific chats and automatically save messages, with filters
|
||||
- Transfer files between different storage backends
|
||||
- Integrate with yt-dlp to download and save media from 1000+ websites
|
||||
- Aria2 integration to download files from URLs/magnets and save to storages
|
||||
- Write JS parser plugins to save files from almost any website
|
||||
- Supports multiple storage backends:
|
||||
- Alist
|
||||
|
||||
@@ -30,6 +30,7 @@ base_path = "./downloads"
|
||||
|
||||
### Global Configuration
|
||||
|
||||
- `lang`: The language used by the Bot, default is `zh-CN` (Simplified Chinese). `en` is used for English.
|
||||
- `stream`: Whether to enable Stream mode, default is `false`. When enabled, the Bot will stream files directly to storage endpoints (if supported), without downloading them locally.
|
||||
{{< hint warning >}}
|
||||
Stream mode is very useful for deployment environments with limited disk space, but it also has some drawbacks:
|
||||
@@ -47,6 +48,7 @@ Stream mode is very useful for deployment environments with limited disk space,
|
||||
- `proxy`: Global proxy configuration. After setting this, all network connections inside the program will try to use this proxy. Optional.
|
||||
|
||||
```toml
|
||||
lang = "en"
|
||||
stream = false
|
||||
workers = 3
|
||||
threads = 4
|
||||
@@ -62,7 +64,7 @@ proxy = "socks5://127.0.0.1:7890"
|
||||
- `rpc_retry`: Number of retries for RPC requests, default is 5.
|
||||
- `proxy`: Proxy configuration, optional.
|
||||
- `enable`: Whether to enable the proxy.
|
||||
- `url`: Proxy address, only supports `socks5://`
|
||||
- `url`: Proxy address
|
||||
- `userbot`: Userbot configuration, optional.
|
||||
- `enable`: Enable userbot integration. Requires logging in with a user account; you should use your own API ID & Hash when enabling this.
|
||||
- `session`: Path to the userbot session file, default is `data/usersession.db`.
|
||||
@@ -90,6 +92,27 @@ enable = false
|
||||
session = "data/usersession.db"
|
||||
```
|
||||
|
||||
### Aria2 Configuration
|
||||
|
||||
Aria2 is a powerful download manager that supports HTTP/HTTPS, FTP, BitTorrent, and other protocols. When enabled, the bot can use the `/aria2dl` command to download files via Aria2.
|
||||
|
||||
- `enable`: Whether to enable Aria2 support, default is `false`
|
||||
- `url`: Aria2 RPC address, typically `http://localhost:6800/jsonrpc`
|
||||
- `secret`: Aria2 RPC secret, if you configured `rpc-secret` in Aria2, you need to fill it in here
|
||||
- `remove_after_transfer`: Whether to remove local files downloaded by Aria2 after transfer, default is `true`
|
||||
|
||||
{{< hint info >}}
|
||||
Aria2 needs to be installed and running separately. You can refer to the [Aria2 official documentation](https://aria2.github.io/) to learn how to install and configure Aria2.
|
||||
{{< /hint >}}
|
||||
|
||||
```toml
|
||||
[aria2]
|
||||
enable = true
|
||||
url = "http://localhost:6800/jsonrpc"
|
||||
secret = "your-rpc-secret"
|
||||
remove_after_transfer = true
|
||||
```
|
||||
|
||||
### Storage Endpoints List
|
||||
|
||||
The storage endpoints list is used to define the storage locations supported by the Bot. Each storage endpoint needs to specify a name, type, and related configuration, using the double bracket syntax `[[storages]]`.
|
||||
@@ -103,6 +126,7 @@ Each storage endpoint requires at least the following fields:
|
||||
- `alist`: Alist
|
||||
- `webdav`: WebDAV
|
||||
- `s3`: aws S3 and other S3 compatible services
|
||||
- `rclone`: Uses rclone to implement uploads
|
||||
- `telegram`: Upload to Telegram
|
||||
|
||||
Example, this is a configuration that includes local storage and webdav storage:
|
||||
|
||||
@@ -80,4 +80,60 @@ chat_id = "123456789" # Telegram chat ID, the bot will send files to this chat
|
||||
force_file = false # Force sending as file, default is false
|
||||
skip_large = false # Skip large files, default is false. If enabled, files exceeding Telegram's limit will not be uploaded.
|
||||
spilt_size_mb = 2000 # Split size in MB, default is 2000 MB (2 GB). Files larger than this will be split into multiple parts (zip format). Ignored when skip_large is true.
|
||||
```
|
||||
|
||||
## Rclone
|
||||
|
||||
`type=rclone`
|
||||
|
||||
Supports multiple cloud storage services through the [rclone](https://rclone.org/) command-line tool. You need to install rclone and configure remote storage first.
|
||||
|
||||
```toml
|
||||
# Remote name configured in rclone, can be any remote defined in rclone.conf
|
||||
remote = "mydrive"
|
||||
# Base path in the remote storage, all files will be stored under this path
|
||||
base_path = "/telegram"
|
||||
# Path to rclone config file, optional, leave empty to use default path (~/.config/rclone/rclone.conf)
|
||||
config_path = ""
|
||||
# Additional flags to pass to rclone commands, optional
|
||||
flags = ["--transfers", "4", "--checkers", "8"]
|
||||
```
|
||||
|
||||
### Configuring rclone Remote
|
||||
|
||||
First, you need to configure an rclone remote. Run `rclone config` for interactive configuration, or directly edit the `rclone.conf` file.
|
||||
|
||||
rclone supports many cloud storage services, including but not limited to:
|
||||
- Google Drive
|
||||
- Dropbox
|
||||
- OneDrive
|
||||
- Amazon S3 and compatible services
|
||||
- SFTP
|
||||
- FTP
|
||||
- For more services, please refer to the [rclone official documentation](https://rclone.org/overview/)
|
||||
|
||||
### Usage Examples
|
||||
|
||||
After configuring Google Drive, you can configure the storage like this:
|
||||
|
||||
```toml
|
||||
[[storages]]
|
||||
name = "GoogleDrive"
|
||||
type = "rclone"
|
||||
enable = true
|
||||
remote = "gdrive"
|
||||
base_path = "/SaveAnyBot"
|
||||
```
|
||||
|
||||
If using a custom rclone config file:
|
||||
|
||||
```toml
|
||||
[[storages]]
|
||||
name = "MyRemote"
|
||||
type = "rclone"
|
||||
enable = true
|
||||
remote = "myremote"
|
||||
base_path = "/backup"
|
||||
config_path = "/path/to/rclone.conf"
|
||||
flags = ["--progress"]
|
||||
```
|
||||
@@ -4,7 +4,7 @@ title: "Installation and Updates"
|
||||
|
||||
# Installation and Updates
|
||||
|
||||
## Deploy from Pre-compiled Files (Recommended)
|
||||
## Deploy from Pre-compiled Binary (Recommended)
|
||||
|
||||
Download the binary file for your platform from the [Release](https://github.com/krau/SaveAny-Bot/releases) page.
|
||||
|
||||
@@ -17,7 +17,7 @@ chmod +x saveany-bot
|
||||
./saveany-bot
|
||||
```
|
||||
|
||||
### Process Monitoring
|
||||
### Daemon
|
||||
|
||||
{{< tabs "daemon" >}}
|
||||
{{< tab "systemd (Regular Linux)" >}}
|
||||
|
||||
@@ -13,112 +13,4 @@ To use the bot's Telegram file saving feature, you need to send or forward the f
|
||||
|
||||
1. File or media messages, such as images, videos, documents, etc.
|
||||
2. Telegram message links, for example: `https://t.me/acherkrau/1097`. **Even if the channel prohibits forwarding and saving, the bot can still download its files.**
|
||||
3. Telegra.ph article links. The bot will download all images in the article.
|
||||
|
||||
## Silent Mode (silent)
|
||||
|
||||
Use the `/silent` command to toggle silent mode.
|
||||
|
||||
By default, silent mode is off, and the bot will ask you for the save location of each file.
|
||||
|
||||
When silent mode is enabled, the bot will save files directly to the default location without confirmation.
|
||||
|
||||
Before enabling silent mode, you need to set the default save location using the `/storage` command.
|
||||
|
||||
|
||||
## Storage Rules
|
||||
|
||||
Storage rules allow you to define redirection rules when the bot uploads files to storage, so that saved files are automatically organized.
|
||||
|
||||
See: <a href="https://github.com/krau/SaveAny-Bot/issues/28" target="_blank">#28</a>
|
||||
|
||||
Currently supported rule types:
|
||||
|
||||
1. FILENAME-REGEX
|
||||
2. MESSAGE-REGEX
|
||||
3. IS-ALBUM
|
||||
|
||||
Basic syntax for adding rules:
|
||||
|
||||
"RuleType RuleContent StorageName Path"
|
||||
|
||||
Pay attention to spaces; the bot can only parse correctly formatted syntax. Below is an example of a valid rule command:
|
||||
|
||||
```
|
||||
/rule add FILENAME-REGEX (?i)\.(mp4|mkv|ts|avi|flv)$ MyAlist /videos
|
||||
```
|
||||
|
||||
In addition, if `CHOSEN` is used as the storage name in the rule, it means files will be stored under the path of the storage you selected by clicking the inline button.
|
||||
|
||||
Rule types:
|
||||
|
||||
### FILENAME-REGEX
|
||||
|
||||
Matches based on filename regex. The rule content must be a valid regular expression, such as:
|
||||
|
||||
```
|
||||
FILENAME-REGEX (?i)\.(mp4|mkv|ts|avi|flv)$ MyAlist /videos
|
||||
```
|
||||
|
||||
This means files with extensions mp4, mkv, ts, avi, flv will be saved to the `/videos` directory in the storage named `MyAlist` (also affected by the `base_path` in the configuration file).
|
||||
|
||||
### MESSAGE-REGEX
|
||||
|
||||
Similar to the above, but matches based on the text content of the message itself.
|
||||
|
||||
### IS-ALBUM
|
||||
|
||||
Matches album messages (media groups). Rule content can only be `true` or `false`.
|
||||
|
||||
If the path in the rule uses `NEW-FOR-ALBUM`, the bot will create a new folder for each media group and store all files of that group there. See: https://github.com/krau/SaveAny-Bot/issues/87
|
||||
|
||||
For example:
|
||||
|
||||
```
|
||||
IS-ALBUM true MyWebdav NEW-FOR-ALBUM
|
||||
```
|
||||
|
||||
This will save media-group messages to the storage named `MyWebdav`, creating a new folder (generated from the first file) for each album.
|
||||
|
||||
## Watch Chats
|
||||
|
||||
{{< hint warning >}}
|
||||
This feature requires enabling UserBot integration.
|
||||
{{< /hint >}}
|
||||
|
||||
You can watch messages in a specific chat and automatically save them to the default storage, following storage rules. You can also add filters so that only matching messages are saved.
|
||||
|
||||
Watch a chat:
|
||||
|
||||
```
|
||||
/watch <chat_id/username> [filter]
|
||||
```
|
||||
|
||||
Stop watching:
|
||||
|
||||
```
|
||||
/unwatch <chat_id/username>
|
||||
```
|
||||
|
||||
Filter types:
|
||||
|
||||
### msgre
|
||||
|
||||
Regex-match the message text. For example:
|
||||
|
||||
```
|
||||
/watch 12345678 msgre:.*hello.*
|
||||
```
|
||||
|
||||
This will watch the chat with ID `12345678`, and only save messages whose text contains `hello`.
|
||||
|
||||
## Save Files Outside Telegram
|
||||
|
||||
Besides files on Telegram, the bot can also save files from other websites via JavaScript plugins or built-in parsers.
|
||||
|
||||
> See the [Contributing Parsers](../contribute) document for details.
|
||||
|
||||
Just send links that match the requirements of a parser to the bot. Currently built-in parsers include:
|
||||
|
||||
- Twitter
|
||||
- Kemono
|
||||
3. Telegra.ph article links. The bot will download all images in the article.
|
||||
442
docs/content/en/usage/api.md
Normal file
442
docs/content/en/usage/api.md
Normal file
@@ -0,0 +1,442 @@
|
||||
---
|
||||
title: "HTTP API"
|
||||
weight: 20
|
||||
---
|
||||
|
||||
# HTTP API
|
||||
|
||||
SaveAny-Bot provides an HTTP API that allows you to programmatically create download/transfer tasks, query task status, cancel tasks, and more — without going through Telegram.
|
||||
|
||||
## Enabling the API
|
||||
|
||||
Add or modify the following section in `config.toml`:
|
||||
|
||||
```toml
|
||||
[api]
|
||||
enable = true
|
||||
host = "0.0.0.0" # Bind address, default 0.0.0.0
|
||||
port = 8080 # Listen port, default 8080
|
||||
token = "your-token" # Auth token — strongly recommended
|
||||
```
|
||||
|
||||
You can also override these settings with environment variables (prefix `SAVEANY_`):
|
||||
|
||||
| Environment Variable | Config Key |
|
||||
|---|---|
|
||||
| `SAVEANY_API_ENABLE` | `api.enable` |
|
||||
| `SAVEANY_API_HOST` | `api.host` |
|
||||
| `SAVEANY_API_PORT` | `api.port` |
|
||||
| `SAVEANY_API_TOKEN` | `api.token` |
|
||||
|
||||
{{< hint warning >}}
|
||||
If `token` is empty, the API server will be accessible **without any authentication**, which is a security risk.
|
||||
{{< /hint >}}
|
||||
|
||||
## Authentication
|
||||
|
||||
When `token` is configured, all API requests must include a Bearer token in the HTTP header:
|
||||
|
||||
```
|
||||
Authorization: Bearer <your-token>
|
||||
```
|
||||
|
||||
On authentication failure, the server returns `401`:
|
||||
|
||||
```json
|
||||
{ "error": "unauthorized", "message": "invalid token" }
|
||||
```
|
||||
|
||||
## Error Response Format
|
||||
|
||||
All errors use a consistent JSON format:
|
||||
|
||||
```json
|
||||
{
|
||||
"error": "error_code",
|
||||
"message": "human readable description"
|
||||
}
|
||||
```
|
||||
|
||||
Common error codes:
|
||||
|
||||
| Error Code | HTTP Status | Meaning |
|
||||
|---|---|---|
|
||||
| `unauthorized` | 401 | Authentication failed |
|
||||
| `method_not_allowed` | 405 | Wrong HTTP method |
|
||||
| `invalid_request` | 400 | Malformed request body or parameters |
|
||||
| `task_creation_failed` | 400 | Failed to create task |
|
||||
| `task_not_found` | 404 | Task ID does not exist |
|
||||
| `cancel_failed` | 500 | Failed to cancel task |
|
||||
| `internal_error` | 500 | Internal server error |
|
||||
|
||||
---
|
||||
|
||||
## Endpoints
|
||||
|
||||
### GET /health — Health Check
|
||||
|
||||
No authentication required.
|
||||
|
||||
**Response `200 OK`:**
|
||||
|
||||
```json
|
||||
{ "status": "ok" }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### GET /api/v1/storages — List Storages
|
||||
|
||||
Returns all currently loaded storage backends.
|
||||
|
||||
**Response `200 OK`:**
|
||||
|
||||
```json
|
||||
{
|
||||
"storages": [
|
||||
{ "name": "local", "type": "local" },
|
||||
{ "name": "MyMinio", "type": "s3" }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### GET /api/v1/task-types — List Supported Task Types
|
||||
|
||||
**Response `200 OK`:**
|
||||
|
||||
```json
|
||||
{
|
||||
"types": [
|
||||
"directlinks",
|
||||
"ytdlp",
|
||||
"aria2",
|
||||
"parseditem",
|
||||
"tgfiles",
|
||||
"tphpics",
|
||||
"transfer"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### POST /api/v1/tasks — Create Task
|
||||
|
||||
**Request headers:**
|
||||
|
||||
```
|
||||
Content-Type: application/json
|
||||
Authorization: Bearer <token>
|
||||
```
|
||||
|
||||
**Request body:**
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "<task_type>",
|
||||
"storage": "<storage_name>",
|
||||
"path": "<subpath>",
|
||||
"webhook": "<callback_url>",
|
||||
"params": { }
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
|---|---|---|---|
|
||||
| `type` | string | Yes | Task type — see below |
|
||||
| `storage` | string | Yes | Target storage name, must match a name in your config |
|
||||
| `path` | string | No | Subdirectory path within the storage |
|
||||
| `webhook` | string | No | Callback URL invoked when the task reaches a terminal state |
|
||||
| `params` | object | Yes | Type-specific parameters — see below |
|
||||
|
||||
**Response `201 Created`:**
|
||||
|
||||
```json
|
||||
{
|
||||
"task_id": "abc123xyz",
|
||||
"type": "directlinks",
|
||||
"status": "queued",
|
||||
"created_at": "2026-03-11T10:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
#### Task Types and params
|
||||
|
||||
##### directlinks — Direct URL Download
|
||||
|
||||
Download one or more files from direct HTTP/HTTPS URLs.
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "directlinks",
|
||||
"storage": "local",
|
||||
"path": "downloads",
|
||||
"params": {
|
||||
"urls": [
|
||||
"https://example.com/file.zip",
|
||||
"https://example.com/other.zip"
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| params field | Type | Required | Description |
|
||||
|---|---|---|---|
|
||||
| `urls` | []string | Yes | List of download URLs, at least 1 |
|
||||
|
||||
##### ytdlp — yt-dlp Media Download
|
||||
|
||||
{{< hint warning >}}
|
||||
Requires yt-dlp to be installed on the system.
|
||||
{{< /hint >}}
|
||||
|
||||
Download videos or audio via yt-dlp, supporting YouTube, Bilibili, and 1000+ other sites.
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "ytdlp",
|
||||
"storage": "local",
|
||||
"path": "videos",
|
||||
"params": {
|
||||
"urls": ["https://www.youtube.com/watch?v=xxx"],
|
||||
"flags": ["--extract-audio", "--audio-format", "mp3"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| params field | Type | Required | Description |
|
||||
|---|---|---|---|
|
||||
| `urls` | []string | Yes | List of media URLs, at least 1 |
|
||||
| `flags` | []string | No | Extra yt-dlp command-line flags |
|
||||
|
||||
##### aria2 — Aria2 Download
|
||||
|
||||
{{< hint warning >}}
|
||||
Requires Aria2 to be enabled and configured (RPC) in the config file.
|
||||
{{< /hint >}}
|
||||
|
||||
Download files via the Aria2 download manager, supporting HTTP/HTTPS, FTP, BitTorrent (magnet links, torrent files), and more.
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "aria2",
|
||||
"storage": "local",
|
||||
"path": "downloads",
|
||||
"params": {
|
||||
"urls": ["magnet:?xt=urn:btih:..."],
|
||||
"options": { "split": "4" }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| params field | Type | Required | Description |
|
||||
|---|---|---|---|
|
||||
| `urls` | []string | Yes | List of download URIs, at least 1 |
|
||||
| `options` | map[string]string | No | Aria2 download options |
|
||||
|
||||
##### parseditem — Parser Plugin Download
|
||||
|
||||
Hand a URL off to a registered JS plugin or built-in parser for processing and downloading.
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "parseditem",
|
||||
"storage": "local",
|
||||
"path": "parsed",
|
||||
"params": {
|
||||
"url": "https://some-site.com/page"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| params field | Type | Required | Description |
|
||||
|---|---|---|---|
|
||||
| `url` | string | Yes | The URL to parse |
|
||||
|
||||
Returns `400 task_creation_failed` if no parser is able to handle the URL.
|
||||
|
||||
##### tgfiles — Telegram Message File Download
|
||||
|
||||
Download files from Telegram messages via message links. Supported link formats:
|
||||
|
||||
- `https://t.me/username/123` — public channel or group
|
||||
- `https://t.me/c/123456789/123` — private channel by numeric ID
|
||||
- `https://t.me/c/123456789/111/456` — topic message (thread ID / message ID)
|
||||
- `https://t.me/username/111/456` — topic under a username-based chat
|
||||
|
||||
If the message is part of a media group (album), all files in the group are downloaded by default. Append `?single` to the link to force downloading only the single specified message.
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "tgfiles",
|
||||
"storage": "local",
|
||||
"path": "telegram",
|
||||
"params": {
|
||||
"message_links": [
|
||||
"https://t.me/username/123",
|
||||
"https://t.me/c/1234567890/456"
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| params field | Type | Required | Description |
|
||||
|---|---|---|---|
|
||||
| `message_links` | []string | Yes | List of Telegram message links, at least 1 |
|
||||
|
||||
##### tphpics — Telegraph Article Images
|
||||
|
||||
Download all images from a Telegra.ph article.
|
||||
|
||||
Supported URL prefixes: `https://telegra.ph/`, `http://telegra.ph/`, `https://telegraph.co/`, `http://telegraph.co/`
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "tphpics",
|
||||
"storage": "local",
|
||||
"path": "telegraph",
|
||||
"params": {
|
||||
"telegraph_url": "https://telegra.ph/Some-Article-01-01"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| params field | Type | Required | Description |
|
||||
|---|---|---|---|
|
||||
| `telegraph_url` | string | Yes | URL of the Telegra.ph article |
|
||||
|
||||
##### transfer — Storage-to-Storage Transfer
|
||||
|
||||
Transfer files directly between two storage backends without going through Telegram. The source storage must support both listing and reading.
|
||||
|
||||
{{< hint info >}}
|
||||
For `transfer` tasks, the top-level `storage` field is still required for validation, but the actual storages used are determined by `source_storage` and `target_storage` inside `params`.
|
||||
{{< /hint >}}
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "transfer",
|
||||
"storage": "local",
|
||||
"params": {
|
||||
"source_storage": "MyS3",
|
||||
"source_path": "backups/",
|
||||
"target_storage": "LocalDisk",
|
||||
"target_path": "restored/"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| params field | Type | Required | Description |
|
||||
|---|---|---|---|
|
||||
| `source_storage` | string | Yes | Source storage name |
|
||||
| `source_path` | string | Yes | Path within the source storage; must contain at least one file |
|
||||
| `target_storage` | string | Yes | Target storage name |
|
||||
| `target_path` | string | Yes | Destination path within the target storage |
|
||||
|
||||
---
|
||||
|
||||
### GET /api/v1/tasks — List All Tasks
|
||||
|
||||
Returns all tasks created via the API. Task records are stored in memory only and are cleared on restart.
|
||||
|
||||
**Response `200 OK`:**
|
||||
|
||||
```json
|
||||
{
|
||||
"tasks": [
|
||||
{
|
||||
"task_id": "abc123xyz",
|
||||
"type": "directlinks",
|
||||
"status": "running",
|
||||
"title": "file.zip",
|
||||
"storage": "local",
|
||||
"path": "downloads",
|
||||
"error": "",
|
||||
"created_at": "2026-03-11T10:00:00Z",
|
||||
"updated_at": "2026-03-11T10:00:05Z",
|
||||
"progress": {
|
||||
"total_bytes": 10485760,
|
||||
"downloaded_bytes": 5242880,
|
||||
"percent": 50.0
|
||||
}
|
||||
}
|
||||
],
|
||||
"total": 1
|
||||
}
|
||||
```
|
||||
|
||||
The `progress` field is only included when `total_bytes > 0`. The `error` field is only included when non-empty.
|
||||
|
||||
---
|
||||
|
||||
### GET /api/v1/tasks/{task_id} — Get Task
|
||||
|
||||
**Path parameter:** `task_id` — the ID returned when the task was created.
|
||||
|
||||
**Response `200 OK`:** Same structure as a single task object from the list above.
|
||||
|
||||
**Error responses:**
|
||||
- `400 invalid_request` — no task ID in path
|
||||
- `404 task_not_found` — task does not exist
|
||||
|
||||
---
|
||||
|
||||
### DELETE /api/v1/tasks/{task_id} — Cancel Task
|
||||
|
||||
**Path parameter:** `task_id`
|
||||
|
||||
**Response `200 OK`:**
|
||||
|
||||
```json
|
||||
{ "message": "task cancelled successfully" }
|
||||
```
|
||||
|
||||
**Error responses:**
|
||||
- `400 invalid_request` — no task ID in path
|
||||
- `404 task_not_found` — task does not exist
|
||||
- `500 cancel_failed` — cancellation failed
|
||||
|
||||
---
|
||||
|
||||
## Task Statuses
|
||||
|
||||
| Status | Meaning |
|
||||
|---|---|
|
||||
| `queued` | Task is queued and waiting to run |
|
||||
| `running` | Task is currently executing |
|
||||
| `completed` | Task finished successfully |
|
||||
| `failed` | Task encountered an error |
|
||||
| `cancelled` | Task was cancelled via the DELETE endpoint |
|
||||
|
||||
---
|
||||
|
||||
## Webhook Callbacks
|
||||
|
||||
When a `webhook` URL is provided in the create request, SaveAny-Bot sends a `POST` request to that URL when the task reaches a terminal state (`completed`, `failed`, or `cancelled`).
|
||||
|
||||
**Callback request headers:**
|
||||
|
||||
```
|
||||
Content-Type: application/json
|
||||
User-Agent: SaveAny-Bot/1.0
|
||||
```
|
||||
|
||||
**Callback request body:**
|
||||
|
||||
```json
|
||||
{
|
||||
"task_id": "abc123xyz",
|
||||
"type": "directlinks",
|
||||
"status": "completed",
|
||||
"storage": "local",
|
||||
"path": "downloads",
|
||||
"completed_at": "2026-03-11T10:01:00Z",
|
||||
"error": ""
|
||||
}
|
||||
```
|
||||
|
||||
`completed_at` is only present when status is `completed` or `failed`. `error` is only present when non-empty.
|
||||
|
||||
**Retry policy:** Up to 3 attempts, with delays of 1s, 2s, and 3s between retries. Each request has a 30-second timeout.
|
||||
41
docs/content/en/usage/aria2.md
Normal file
41
docs/content/en/usage/aria2.md
Normal file
@@ -0,0 +1,41 @@
|
||||
---
|
||||
title: "Aria2 Download"
|
||||
weight: 6
|
||||
---
|
||||
|
||||
# Aria2 Download
|
||||
|
||||
{{< hint warning >}}
|
||||
This feature requires enabling Aria2 in the configuration file and configuring the RPC connection.
|
||||
{{< /hint >}}
|
||||
|
||||
Use the `/aria2dl` command to download files via the Aria2 download manager, supporting HTTP/HTTPS, FTP, BitTorrent, and other protocols.
|
||||
|
||||
```bash
|
||||
/aria2dl <uri1> [uri2] [uri3] ...
|
||||
```
|
||||
|
||||
Examples:
|
||||
|
||||
```bash
|
||||
# Download HTTP link
|
||||
/aria2dl https://example.com/file.zip
|
||||
|
||||
# Download magnet link
|
||||
/aria2dl magnet:?xt=urn:btih:...
|
||||
|
||||
# Download torrent file (need to upload .torrent file first)
|
||||
/aria2dl https://example.com/file.torrent
|
||||
```
|
||||
|
||||
Configure Aria2:
|
||||
|
||||
Add to `config.toml`:
|
||||
|
||||
```toml
|
||||
[aria2]
|
||||
enable = true
|
||||
url = "http://localhost:6800/jsonrpc"
|
||||
secret = "your-rpc-secret" # If rpc-secret is configured
|
||||
remove_after_transfer = true # Remove local files after transfer
|
||||
```
|
||||
21
docs/content/en/usage/directlinks.md
Normal file
21
docs/content/en/usage/directlinks.md
Normal file
@@ -0,0 +1,21 @@
|
||||
---
|
||||
title: "Direct Download Links"
|
||||
weight: 5
|
||||
---
|
||||
|
||||
# Direct Download Links
|
||||
|
||||
Use the `/dl` command to directly download one or more HTTP/HTTPS files to storage.
|
||||
|
||||
```bash
|
||||
/dl <url1> [url2] [url3] ...
|
||||
```
|
||||
|
||||
Examples:
|
||||
|
||||
```bash
|
||||
/dl https://example.com/file.zip
|
||||
/dl https://example.com/file1.zip https://example.com/file2.zip
|
||||
```
|
||||
|
||||
The bot will validate the link format and then ask you to select the target storage location.
|
||||
15
docs/content/en/usage/parsers.md
Normal file
15
docs/content/en/usage/parsers.md
Normal file
@@ -0,0 +1,15 @@
|
||||
---
|
||||
title: "Save Files Outside Telegram"
|
||||
weight: 9
|
||||
---
|
||||
|
||||
# Save Files Outside Telegram
|
||||
|
||||
Besides files on Telegram, the bot can also save files from other websites via JavaScript plugins or built-in parsers.
|
||||
|
||||
> See the [Contributing Parsers](../contribute) document for details.
|
||||
|
||||
Just send links that match the requirements of a parser to the bot. Currently built-in parsers include:
|
||||
|
||||
- Twitter
|
||||
- Kemono
|
||||
58
docs/content/en/usage/rules.md
Normal file
58
docs/content/en/usage/rules.md
Normal file
@@ -0,0 +1,58 @@
|
||||
---
|
||||
title: "Storage Rules"
|
||||
weight: 3
|
||||
---
|
||||
|
||||
# Storage Rules
|
||||
|
||||
Storage rules allow you to define redirection rules when the bot uploads files to storage, so that saved files are automatically organized.
|
||||
|
||||
See: <a href="https://github.com/krau/SaveAny-Bot/issues/28" target="_blank">#28</a>
|
||||
|
||||
Currently supported rule types:
|
||||
|
||||
1. FILENAME-REGEX
|
||||
2. MESSAGE-REGEX
|
||||
3. IS-ALBUM
|
||||
|
||||
Basic syntax for adding rules:
|
||||
|
||||
"RuleType RuleContent StorageName Path"
|
||||
|
||||
Pay attention to spaces; the bot can only parse correctly formatted syntax. Below is an example of a valid rule command:
|
||||
|
||||
```
|
||||
/rule add FILENAME-REGEX (?i)\.(mp4|mkv|ts|avi|flv)$ MyAlist /videos
|
||||
```
|
||||
|
||||
In addition, if `CHOSEN` is used as the storage name in the rule, it means files will be stored under the path of the storage you selected by clicking the inline button.
|
||||
|
||||
Rule types:
|
||||
|
||||
## FILENAME-REGEX
|
||||
|
||||
Matches based on filename regex. The rule content must be a valid regular expression, such as:
|
||||
|
||||
```
|
||||
FILENAME-REGEX (?i)\.(mp4|mkv|ts|avi|flv)$ MyAlist /videos
|
||||
```
|
||||
|
||||
This means files with extensions mp4, mkv, ts, avi, flv will be saved to the `/videos` directory in the storage named `MyAlist` (also affected by the `base_path` in the configuration file).
|
||||
|
||||
## MESSAGE-REGEX
|
||||
|
||||
Similar to the above, but matches based on the text content of the message itself.
|
||||
|
||||
## IS-ALBUM
|
||||
|
||||
Matches album messages (media groups). Rule content can only be `true` or `false`.
|
||||
|
||||
If the path in the rule uses `NEW-FOR-ALBUM`, the bot will create a new folder for each media group and store all files of that group there. See: https://github.com/krau/SaveAny-Bot/issues/87
|
||||
|
||||
For example:
|
||||
|
||||
```
|
||||
IS-ALBUM true MyWebdav NEW-FOR-ALBUM
|
||||
```
|
||||
|
||||
This will save media-group messages to the storage named `MyWebdav`, creating a new folder (generated from the first file) for each album.
|
||||
14
docs/content/en/usage/silent.md
Normal file
14
docs/content/en/usage/silent.md
Normal file
@@ -0,0 +1,14 @@
|
||||
---
|
||||
title: "Silent Mode"
|
||||
weight: 2
|
||||
---
|
||||
|
||||
# Silent Mode (silent)
|
||||
|
||||
Use the `/silent` command to toggle silent mode.
|
||||
|
||||
By default, silent mode is off, and the bot will ask you for the save location of each file.
|
||||
|
||||
When silent mode is enabled, the bot will save files directly to the default location without confirmation.
|
||||
|
||||
Before enabling silent mode, you need to set the default save location using the `/storage` command.
|
||||
50
docs/content/en/usage/transfer.md
Normal file
50
docs/content/en/usage/transfer.md
Normal file
@@ -0,0 +1,50 @@
|
||||
---
|
||||
title: "Storage Transfer"
|
||||
weight: 8
|
||||
---
|
||||
|
||||
# Storage Transfer
|
||||
|
||||
Use the `/transfer` command to transfer files directly between different storages without going through Telegram.
|
||||
|
||||
```bash
|
||||
/transfer <source_storage>:/<source_path> [filter]
|
||||
```
|
||||
|
||||
Parameters:
|
||||
|
||||
- `source_storage`: Source storage name
|
||||
- `source_path`: Source path
|
||||
- `filter`: Optional regex filter to transfer only matching files
|
||||
|
||||
Examples:
|
||||
|
||||
```bash
|
||||
# Transfer entire directory
|
||||
/transfer local1:/downloads
|
||||
|
||||
# Transfer files from specified path
|
||||
/transfer alist1:/media/photos
|
||||
|
||||
# Transfer only mp4 files
|
||||
/transfer webdav1:/videos ".*\.mp4$"
|
||||
|
||||
# Transfer image files
|
||||
/transfer local1:/pictures "(?i)\.(jpg|png|gif)$"
|
||||
```
|
||||
|
||||
The bot will:
|
||||
|
||||
1. List all files in the source path
|
||||
2. Apply the filter (if provided)
|
||||
3. Display file count and total size
|
||||
4. Ask you to select the target storage
|
||||
5. Ask you to select the target directory (if configured for that storage)
|
||||
6. Start the transfer task
|
||||
|
||||
Notes:
|
||||
|
||||
- Source storage must support listing and reading
|
||||
- Target storage must support writing
|
||||
- Real-time progress is displayed during transfer
|
||||
- Transfer tasks can be cancelled
|
||||
36
docs/content/en/usage/watch.md
Normal file
36
docs/content/en/usage/watch.md
Normal file
@@ -0,0 +1,36 @@
|
||||
---
|
||||
title: "Watch Chats"
|
||||
weight: 4
|
||||
---
|
||||
|
||||
# Watch Chats
|
||||
|
||||
{{< hint warning >}}
|
||||
This feature requires enabling UserBot integration.
|
||||
{{< /hint >}}
|
||||
|
||||
You can watch messages in a specific chat and automatically save them to the default storage, following storage rules. You can also add filters so that only matching messages are saved.
|
||||
|
||||
Watch a chat:
|
||||
|
||||
```
|
||||
/watch <chat_id/username> [filter]
|
||||
```
|
||||
|
||||
Stop watching:
|
||||
|
||||
```
|
||||
/unwatch <chat_id/username>
|
||||
```
|
||||
|
||||
Filter types:
|
||||
|
||||
## msgre
|
||||
|
||||
Regex-match the message text. For example:
|
||||
|
||||
```
|
||||
/watch 12345678 msgre:.*hello.*
|
||||
```
|
||||
|
||||
This will watch the chat with ID `12345678`, and only save messages whose text contains `hello`.
|
||||
40
docs/content/en/usage/ytdlp.md
Normal file
40
docs/content/en/usage/ytdlp.md
Normal file
@@ -0,0 +1,40 @@
|
||||
---
|
||||
title: "yt-dlp Video Download"
|
||||
weight: 7
|
||||
---
|
||||
|
||||
# yt-dlp Video Download
|
||||
|
||||
{{< hint warning >}}
|
||||
This feature requires the yt-dlp command-line tool installed on your system.
|
||||
{{< /hint >}}
|
||||
|
||||
Use the `/ytdlp` command to download videos and audio from supported video websites, including YouTube, Bilibili, Twitter, and 1000+ other sites.
|
||||
|
||||
```bash
|
||||
/ytdlp <url1> [url2] [flags...]
|
||||
```
|
||||
|
||||
Examples:
|
||||
|
||||
```bash
|
||||
# Basic download
|
||||
/ytdlp https://www.youtube.com/watch?v=dQw4w9WgXcQ
|
||||
|
||||
# Download multiple videos
|
||||
/ytdlp https://www.youtube.com/watch?v=video1 https://www.youtube.com/watch?v=video2
|
||||
|
||||
# Use custom parameters
|
||||
/ytdlp https://www.youtube.com/watch?v=dQw4w9WgXcQ -f best
|
||||
/ytdlp https://www.youtube.com/watch?v=dQw4w9WgXcQ --extract-audio --audio-format mp3
|
||||
```
|
||||
|
||||
Common parameters:
|
||||
|
||||
- `-f <format>`: Specify download format (e.g., `best`, `worst`, `bestvideo+bestaudio`)
|
||||
- `--extract-audio`: Extract audio
|
||||
- `--audio-format <format>`: Audio format (e.g., `mp3`, `m4a`, `wav`)
|
||||
- `--write-sub`: Download subtitles
|
||||
- `--write-thumbnail`: Download thumbnail
|
||||
|
||||
For more parameters, see [yt-dlp documentation](https://github.com/yt-dlp/yt-dlp#usage-and-options).
|
||||
@@ -20,12 +20,16 @@ title: 介绍
|
||||
- 多用户使用
|
||||
- 基于存储规则的自动整理
|
||||
- 监听并自动转存指定聊天的消息, 支持过滤
|
||||
- 在不同存储端之间转存文件
|
||||
- 集成 yt-dlp, 从所支持的网站下载并转存媒体文件
|
||||
- 集成 Aria2, 支持直链/磁力下载和转存
|
||||
- 使用 js 编写解析器插件以转存任意网站的文件
|
||||
- 存储端支持:
|
||||
- Alist
|
||||
- S3
|
||||
- WebDAV
|
||||
- 本地磁盘
|
||||
- Rclone (通过命令行调用)
|
||||
- Telegram (重传回指定聊天)
|
||||
|
||||
## [贡献者](https://github.com/krau/SaveAny-Bot/graphs/contributors)
|
||||
|
||||
@@ -62,7 +62,7 @@ proxy = "socks5://127.0.0.1:7890"
|
||||
- `rpc_retry`: RPC 请求重试次数, 默认为 5.
|
||||
- `proxy`: 代理配置, 可选.
|
||||
- `enable`: 是否启用代理.
|
||||
- `url`: 代理地址, 只支持 `socks5://`
|
||||
- `url`: 代理地址
|
||||
- `userbot`: userbot 配置, 可选.
|
||||
- `enable`: 启用 userbot 集成, 需要登录用户账号, 此时请务必使用自己的 api id & hash.
|
||||
- `session`: userbot 会话文件路径, 默认为 `data/usersession.db`.
|
||||
@@ -90,6 +90,27 @@ enable = false
|
||||
session = "data/usersession.db"
|
||||
```
|
||||
|
||||
### Aria2 配置
|
||||
|
||||
Aria2 是一个强大的下载管理器,支持 HTTP/HTTPS、FTP、BitTorrent 等多种协议。启用后,Bot 可以使用 `/aria2dl` 命令通过 Aria2 下载文件。
|
||||
|
||||
- `enable`: 是否启用 Aria2 支持,默认为 `false`
|
||||
- `url`: Aria2 RPC 地址,通常为 `http://localhost:6800/jsonrpc`
|
||||
- `secret`: Aria2 RPC 密钥,如果你在 Aria2 中配置了 `rpc-secret`,需要在此填写
|
||||
- `remove_after_transfer`: 转存完成后是否删除 Aria2 下载的本地文件,默认为 `true`
|
||||
|
||||
{{< hint info >}}
|
||||
Aria2 需要单独安装和运行。你可以参考 [Aria2 官方文档](https://aria2.github.io/) 了解如何安装和配置 Aria2。
|
||||
{{< /hint >}}
|
||||
|
||||
```toml
|
||||
[aria2]
|
||||
enable = true
|
||||
url = "http://localhost:6800/jsonrpc"
|
||||
secret = "your-rpc-secret"
|
||||
remove_after_transfer = true
|
||||
```
|
||||
|
||||
### 存储端列表
|
||||
|
||||
存储端列表用于定义 Bot 支持的存储位置, 每个存储端需要指定名称、类型和相关配置, 使用双中括号语法 `[[storages]]` 定义.
|
||||
@@ -103,6 +124,7 @@ session = "data/usersession.db"
|
||||
- `alist`: Alist
|
||||
- `webdav`: WebDAV
|
||||
- `s3`: aws S3 及其他兼容 S3 的服务
|
||||
- `rclone`: 调用 rclone 实现上传
|
||||
- `telegram`: 上传到 Telegram
|
||||
|
||||
示例, 这是一个包含本地存储和 webdav 存储的配置:
|
||||
|
||||
@@ -86,4 +86,60 @@ skip_large = false
|
||||
# 超过该大小的文件将被分割成多个部分上传.(使用 zip 格式)
|
||||
# 当 skip_large 启用时, 该选项无效.
|
||||
spilt_size_mb = 2000
|
||||
```
|
||||
|
||||
## Rclone
|
||||
|
||||
`type=rclone`
|
||||
|
||||
通过 [rclone](https://rclone.org/) 命令行工具支持多种云存储服务. 需要先安装 rclone 并配置好远程存储.
|
||||
|
||||
```toml
|
||||
# rclone 配置的远程名称, 可以是任何在 rclone.conf 中配置的远程
|
||||
remote = "mydrive"
|
||||
# 在远程存储中的基础路径, 所有文件将存储在此路径下
|
||||
base_path = "/telegram"
|
||||
# rclone 配置文件的路径, 可选, 留空使用默认路径 (~/.config/rclone/rclone.conf)
|
||||
config_path = ""
|
||||
# 传递给 rclone 命令的额外参数, 可选
|
||||
flags = ["--transfers", "4", "--checkers", "8"]
|
||||
```
|
||||
|
||||
### 配置 rclone 远程
|
||||
|
||||
首先需要配置 rclone 远程, 运行 `rclone config` 命令进行交互式配置, 或直接编辑 `rclone.conf` 文件.
|
||||
|
||||
rclone 支持多种云存储服务, 包括但不限于:
|
||||
- Google Drive
|
||||
- Dropbox
|
||||
- OneDrive
|
||||
- Amazon S3 及兼容服务
|
||||
- SFTP
|
||||
- FTP
|
||||
- 更多服务请参考 [rclone 官方文档](https://rclone.org/overview/)
|
||||
|
||||
### 使用示例
|
||||
|
||||
配置 Google Drive 后, 可以这样配置存储:
|
||||
|
||||
```toml
|
||||
[[storages]]
|
||||
name = "GoogleDrive"
|
||||
type = "rclone"
|
||||
enable = true
|
||||
remote = "gdrive"
|
||||
base_path = "/SaveAnyBot"
|
||||
```
|
||||
|
||||
如果使用自定义的 rclone 配置文件:
|
||||
|
||||
```toml
|
||||
[[storages]]
|
||||
name = "MyRemote"
|
||||
type = "rclone"
|
||||
enable = true
|
||||
remote = "myremote"
|
||||
base_path = "/backup"
|
||||
config_path = "/path/to/rclone.conf"
|
||||
flags = ["--progress"]
|
||||
```
|
||||
@@ -13,112 +13,4 @@ weight: 10
|
||||
|
||||
1. 文件或媒体消息, 如图片, 视频, 文档等
|
||||
2. Telegram 消息链接, 例如: `https://t.me/acherkrau/1097`. **即使频道禁止了转发和保存, Bot 依然可以下载其文件.**
|
||||
3. Telegra.ph 的文章链接, Bot 将下载其中的所有图片
|
||||
|
||||
## 静默模式 (silent)
|
||||
|
||||
使用 `/silent` 命令可以开关静默模式.
|
||||
|
||||
默认情况下不开启静默模式, Bot 会询问你每个文件的保存位置.
|
||||
|
||||
开启静默模式后, Bot 会直接保存文件到默认位置, 无需确认.
|
||||
|
||||
在开启静默模式之前, 需要使用 `/storage` 命令设置默认保存位置.
|
||||
|
||||
## 存储规则
|
||||
|
||||
允许你为 Bot 在上传文件到存储时设置一些重定向规则, 用于自动整理所保存的文件.
|
||||
|
||||
见: <a href="https://github.com/krau/SaveAny-Bot/issues/28" target="_blank">#28</a>
|
||||
|
||||
目前支持的规则类型:
|
||||
|
||||
1. FILENAME-REGEX
|
||||
2. MESSAGE-REGEX
|
||||
3. IS-ALBUM
|
||||
|
||||
添加规则的基本语法:
|
||||
|
||||
"规则类型 规则内容 存储名 路径"
|
||||
|
||||
注意空格的使用, 语法正确 bot 才能解析, 以下是一条合法的添加规则命令:
|
||||
|
||||
```
|
||||
/rule add FILENAME-REGEX (?i)\.(mp4|mkv|ts|avi|flv)$ MyAlist /视频
|
||||
```
|
||||
|
||||
此外, 规则中的存储名若使用 "CHOSEN" , 则表示存储到点击按钮选择的存储端的路径下
|
||||
|
||||
规则类型:
|
||||
|
||||
### FILENAME-REGEX
|
||||
|
||||
根据文件名正则匹配, 规则内容要求为一个合法的正则表达式, 如
|
||||
|
||||
```
|
||||
FILENAME-REGEX (?i)\.(mp4|mkv|ts|avi|flv)$ MyAlist /视频
|
||||
```
|
||||
|
||||
表示将文件名后缀为 mp4,mkv,ts,avi,flv 的文件放到名为 MyAlist 存储下的 /视频 目录内 (同时受配置文件中的 `base_path` 影响)
|
||||
|
||||
### MESSAGE-REGEX
|
||||
|
||||
同上, 但是是根据消息本身的文本内容正则匹配
|
||||
|
||||
### IS-ALBUM
|
||||
|
||||
匹配相册消息 (media group), 规则内容只能为 `true` 或 `false`.
|
||||
|
||||
规则中的路径若使用 "NEW-FOR-ALBUM" , 则表示为该组消息新建一个文件夹来存储它们. 见: https://github.com/krau/SaveAny-Bot/issues/87
|
||||
|
||||
例如:
|
||||
|
||||
```
|
||||
IS-ALBUM true MyWebdav NEW-FOR-ALBUM
|
||||
```
|
||||
|
||||
这将会把以 media group 形式发送的消息保存到名为 MyWebdav 的存储下, 并为每个相册新建一个文件夹(由第一个文件生成)来存储它们.
|
||||
|
||||
|
||||
## 监听聊天
|
||||
|
||||
{{< hint warning >}}
|
||||
该功能需开启 UserBot 集成.
|
||||
{{< /hint >}}
|
||||
|
||||
监听指定聊天的消息, 并自动保存到默认存储中, 遵从存储规则, 并且可以设置过滤器来只保存匹配的消息.
|
||||
|
||||
监听聊天:
|
||||
|
||||
```
|
||||
/watch <chat_id/username> [filter]
|
||||
```
|
||||
|
||||
取消监听:
|
||||
|
||||
```
|
||||
/unwatch <chat_id/username>
|
||||
```
|
||||
|
||||
过滤器类型:
|
||||
|
||||
### msgre
|
||||
|
||||
正则匹配消息文本, 例如:
|
||||
|
||||
```
|
||||
/watch 12345678 msgre:.*hello.*
|
||||
```
|
||||
|
||||
这将会监听 ID 为 12345678 的聊天, 并且只保存消息文本中包含 "hello" 的消息.
|
||||
|
||||
## 转存 Telegram 之外的文件
|
||||
|
||||
除了 Telegram 上的文件, Bot 还可通过 JavaScript 插件或内置解析器来支持转存其他网站的文件.
|
||||
|
||||
> 查看[贡献解析器](../contribute)文档了解详情
|
||||
|
||||
只需向 Bot 发送符合解析器要求的链接即可使用, 当前内置的解析器:
|
||||
|
||||
- Twitter
|
||||
- Kemono
|
||||
3. Telegra.ph 的文章链接, Bot 将下载其中的所有图片
|
||||
442
docs/content/zh/usage/api.md
Normal file
442
docs/content/zh/usage/api.md
Normal file
@@ -0,0 +1,442 @@
|
||||
---
|
||||
title: "HTTP API"
|
||||
weight: 20
|
||||
---
|
||||
|
||||
# HTTP API
|
||||
|
||||
SaveAny-Bot 提供了一套 HTTP API,允许你通过程序化方式创建下载/转存任务、查询任务状态、取消任务等,无需通过 Telegram 操作。
|
||||
|
||||
## 启用 API
|
||||
|
||||
在 `config.toml` 中添加或修改以下配置:
|
||||
|
||||
```toml
|
||||
[api]
|
||||
enable = true
|
||||
host = "0.0.0.0" # 监听地址,默认 0.0.0.0
|
||||
port = 8080 # 监听端口,默认 8080
|
||||
token = "your-token" # 鉴权 Token,强烈建议设置
|
||||
```
|
||||
|
||||
也可通过环境变量覆盖(前缀 `SAVEANY_`):
|
||||
|
||||
| 环境变量 | 对应配置项 |
|
||||
|---|---|
|
||||
| `SAVEANY_API_ENABLE` | `api.enable` |
|
||||
| `SAVEANY_API_HOST` | `api.host` |
|
||||
| `SAVEANY_API_PORT` | `api.port` |
|
||||
| `SAVEANY_API_TOKEN` | `api.token` |
|
||||
|
||||
{{< hint warning >}}
|
||||
若 `token` 为空,API 服务将**不进行任何鉴权**即可访问,存在安全风险。
|
||||
{{< /hint >}}
|
||||
|
||||
## 鉴权
|
||||
|
||||
当配置了 `token` 时,所有 API 请求均需在 HTTP 请求头中携带 Bearer Token:
|
||||
|
||||
```
|
||||
Authorization: Bearer <your-token>
|
||||
```
|
||||
|
||||
鉴权失败时返回 `401`:
|
||||
|
||||
```json
|
||||
{ "error": "unauthorized", "message": "invalid token" }
|
||||
```
|
||||
|
||||
## 错误响应格式
|
||||
|
||||
所有错误均使用统一的 JSON 格式:
|
||||
|
||||
```json
|
||||
{
|
||||
"error": "error_code",
|
||||
"message": "错误说明"
|
||||
}
|
||||
```
|
||||
|
||||
常见错误码:
|
||||
|
||||
| 错误码 | HTTP 状态 | 含义 |
|
||||
|---|---|---|
|
||||
| `unauthorized` | 401 | 鉴权失败 |
|
||||
| `method_not_allowed` | 405 | HTTP 方法不正确 |
|
||||
| `invalid_request` | 400 | 请求体/参数非法 |
|
||||
| `task_creation_failed` | 400 | 任务创建失败 |
|
||||
| `task_not_found` | 404 | 任务 ID 不存在 |
|
||||
| `cancel_failed` | 500 | 取消任务失败 |
|
||||
| `internal_error` | 500 | 服务器内部错误 |
|
||||
|
||||
---
|
||||
|
||||
## 接口列表
|
||||
|
||||
### GET /health — 健康检查
|
||||
|
||||
无需鉴权。
|
||||
|
||||
**响应 `200 OK`:**
|
||||
|
||||
```json
|
||||
{ "status": "ok" }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### GET /api/v1/storages — 列出存储
|
||||
|
||||
返回当前所有已加载的存储后端。
|
||||
|
||||
**响应 `200 OK`:**
|
||||
|
||||
```json
|
||||
{
|
||||
"storages": [
|
||||
{ "name": "local", "type": "local" },
|
||||
{ "name": "MyMinio", "type": "s3" }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### GET /api/v1/task-types — 列出支持的任务类型
|
||||
|
||||
**响应 `200 OK`:**
|
||||
|
||||
```json
|
||||
{
|
||||
"types": [
|
||||
"directlinks",
|
||||
"ytdlp",
|
||||
"aria2",
|
||||
"parseditem",
|
||||
"tgfiles",
|
||||
"tphpics",
|
||||
"transfer"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### POST /api/v1/tasks — 创建任务
|
||||
|
||||
**请求头:**
|
||||
|
||||
```
|
||||
Content-Type: application/json
|
||||
Authorization: Bearer <token>
|
||||
```
|
||||
|
||||
**请求体:**
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "<任务类型>",
|
||||
"storage": "<存储名>",
|
||||
"path": "<子目录>",
|
||||
"webhook": "<回调URL>",
|
||||
"params": { }
|
||||
}
|
||||
```
|
||||
|
||||
| 字段 | 类型 | 必填 | 说明 |
|
||||
|---|---|---|---|
|
||||
| `type` | string | 是 | 任务类型,见下文 |
|
||||
| `storage` | string | 是 | 目标存储名,须与配置中的存储名一致 |
|
||||
| `path` | string | 否 | 存储内的子目录路径 |
|
||||
| `webhook` | string | 否 | 任务完成/失败时的回调地址 |
|
||||
| `params` | object | 是 | 各任务类型的专属参数,见下文 |
|
||||
|
||||
**响应 `201 Created`:**
|
||||
|
||||
```json
|
||||
{
|
||||
"task_id": "abc123xyz",
|
||||
"type": "directlinks",
|
||||
"status": "queued",
|
||||
"created_at": "2026-03-11T10:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
#### 任务类型与 params
|
||||
|
||||
##### directlinks — 直接下载链接
|
||||
|
||||
下载一个或多个 HTTP/HTTPS 直链文件。
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "directlinks",
|
||||
"storage": "local",
|
||||
"path": "downloads",
|
||||
"params": {
|
||||
"urls": [
|
||||
"https://example.com/file.zip",
|
||||
"https://example.com/other.zip"
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| params 字段 | 类型 | 必填 | 说明 |
|
||||
|---|---|---|---|
|
||||
| `urls` | []string | 是 | 下载地址列表,至少 1 条 |
|
||||
|
||||
##### ytdlp — yt-dlp 视频下载
|
||||
|
||||
{{< hint warning >}}
|
||||
需要在系统中安装 yt-dlp。
|
||||
{{< /hint >}}
|
||||
|
||||
通过 yt-dlp 下载视频/音频,支持 YouTube、Bilibili 等 1000+ 网站。
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "ytdlp",
|
||||
"storage": "local",
|
||||
"path": "videos",
|
||||
"params": {
|
||||
"urls": ["https://www.youtube.com/watch?v=xxx"],
|
||||
"flags": ["--extract-audio", "--audio-format", "mp3"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| params 字段 | 类型 | 必填 | 说明 |
|
||||
|---|---|---|---|
|
||||
| `urls` | []string | 是 | 媒体链接列表,至少 1 条 |
|
||||
| `flags` | []string | 否 | 额外的 yt-dlp 命令行参数 |
|
||||
|
||||
##### aria2 — Aria2 下载
|
||||
|
||||
{{< hint warning >}}
|
||||
需要在配置文件中启用并配置 Aria2 RPC。
|
||||
{{< /hint >}}
|
||||
|
||||
通过 Aria2 下载管理器下载文件,支持 HTTP/HTTPS、FTP、BitTorrent(磁力链接、种子)等协议。
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "aria2",
|
||||
"storage": "local",
|
||||
"path": "downloads",
|
||||
"params": {
|
||||
"urls": ["magnet:?xt=urn:btih:..."],
|
||||
"options": { "split": "4" }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| params 字段 | 类型 | 必填 | 说明 |
|
||||
|---|---|---|---|
|
||||
| `urls` | []string | 是 | 下载地址列表,至少 1 条 |
|
||||
| `options` | map[string]string | 否 | Aria2 下载选项 |
|
||||
|
||||
##### parseditem — 解析器下载
|
||||
|
||||
将 URL 交由已注册的 JS 插件或内置解析器处理后下载。
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "parseditem",
|
||||
"storage": "local",
|
||||
"path": "parsed",
|
||||
"params": {
|
||||
"url": "https://some-site.com/page"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| params 字段 | 类型 | 必填 | 说明 |
|
||||
|---|---|---|---|
|
||||
| `url` | string | 是 | 待解析的页面 URL |
|
||||
|
||||
若没有任何解析器能处理该 URL,则返回 `400 task_creation_failed`。
|
||||
|
||||
##### tgfiles — Telegram 消息文件下载
|
||||
|
||||
通过 Telegram 消息链接下载文件。支持以下链接格式:
|
||||
|
||||
- `https://t.me/username/123` — 公开频道/群组
|
||||
- `https://t.me/c/123456789/123` — 私有频道(数字 ID)
|
||||
- `https://t.me/c/123456789/111/456` — 话题消息
|
||||
- `https://t.me/username/111/456` — 用户名频道下的话题消息
|
||||
|
||||
若消息属于媒体组(相册),默认下载整组文件。在链接末尾追加 `?single` 可强制只下载单条消息的文件。
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "tgfiles",
|
||||
"storage": "local",
|
||||
"path": "telegram",
|
||||
"params": {
|
||||
"message_links": [
|
||||
"https://t.me/username/123",
|
||||
"https://t.me/c/1234567890/456"
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| params 字段 | 类型 | 必填 | 说明 |
|
||||
|---|---|---|---|
|
||||
| `message_links` | []string | 是 | Telegram 消息链接列表,至少 1 条 |
|
||||
|
||||
##### tphpics — Telegraph 文章图片下载
|
||||
|
||||
下载 Telegra.ph 文章中的所有图片。
|
||||
|
||||
支持的链接前缀:`https://telegra.ph/`、`http://telegra.ph/`、`https://telegraph.co/`、`http://telegraph.co/`
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "tphpics",
|
||||
"storage": "local",
|
||||
"path": "telegraph",
|
||||
"params": {
|
||||
"telegraph_url": "https://telegra.ph/Some-Article-01-01"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| params 字段 | 类型 | 必填 | 说明 |
|
||||
|---|---|---|---|
|
||||
| `telegraph_url` | string | 是 | Telegra.ph 文章 URL |
|
||||
|
||||
##### transfer — 存储间文件传输
|
||||
|
||||
在两个存储后端之间直接传输文件,无需经过 Telegram。源存储须支持列举(list)和读取(read)操作。
|
||||
|
||||
{{< hint info >}}
|
||||
`transfer` 任务中,顶层的 `storage` 字段仍然必须填写(用于通过参数校验),但实际使用的存储由 `params` 中的 `source_storage` 和 `target_storage` 决定。
|
||||
{{< /hint >}}
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "transfer",
|
||||
"storage": "local",
|
||||
"params": {
|
||||
"source_storage": "MyS3",
|
||||
"source_path": "backups/",
|
||||
"target_storage": "LocalDisk",
|
||||
"target_path": "restored/"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| params 字段 | 类型 | 必填 | 说明 |
|
||||
|---|---|---|---|
|
||||
| `source_storage` | string | 是 | 源存储名 |
|
||||
| `source_path` | string | 是 | 源存储中的路径,须包含至少一个文件 |
|
||||
| `target_storage` | string | 是 | 目标存储名 |
|
||||
| `target_path` | string | 是 | 目标存储中的路径 |
|
||||
|
||||
---
|
||||
|
||||
### GET /api/v1/tasks — 列出所有任务
|
||||
|
||||
返回所有 API 创建的任务(仅在内存中保留,重启后清空)。
|
||||
|
||||
**响应 `200 OK`:**
|
||||
|
||||
```json
|
||||
{
|
||||
"tasks": [
|
||||
{
|
||||
"task_id": "abc123xyz",
|
||||
"type": "directlinks",
|
||||
"status": "running",
|
||||
"title": "file.zip",
|
||||
"storage": "local",
|
||||
"path": "downloads",
|
||||
"error": "",
|
||||
"created_at": "2026-03-11T10:00:00Z",
|
||||
"updated_at": "2026-03-11T10:00:05Z",
|
||||
"progress": {
|
||||
"total_bytes": 10485760,
|
||||
"downloaded_bytes": 5242880,
|
||||
"percent": 50.0
|
||||
}
|
||||
}
|
||||
],
|
||||
"total": 1
|
||||
}
|
||||
```
|
||||
|
||||
`progress` 字段仅在 `total_bytes > 0` 时出现。`error` 字段仅在有错误时出现。
|
||||
|
||||
---
|
||||
|
||||
### GET /api/v1/tasks/{task_id} — 查询任务
|
||||
|
||||
**路径参数:** `task_id` — 创建任务时返回的 ID。
|
||||
|
||||
**响应 `200 OK`:** 同上列表中的单个任务对象。
|
||||
|
||||
**错误响应:**
|
||||
- `400 invalid_request` — 路径中未提供 task_id
|
||||
- `404 task_not_found` — 任务不存在
|
||||
|
||||
---
|
||||
|
||||
### DELETE /api/v1/tasks/{task_id} — 取消任务
|
||||
|
||||
**路径参数:** `task_id`
|
||||
|
||||
**响应 `200 OK`:**
|
||||
|
||||
```json
|
||||
{ "message": "task cancelled successfully" }
|
||||
```
|
||||
|
||||
**错误响应:**
|
||||
- `400 invalid_request` — 路径中未提供 task_id
|
||||
- `404 task_not_found` — 任务不存在
|
||||
- `500 cancel_failed` — 取消操作失败
|
||||
|
||||
---
|
||||
|
||||
## 任务状态
|
||||
|
||||
| 状态值 | 含义 |
|
||||
|---|---|
|
||||
| `queued` | 已入队,等待执行 |
|
||||
| `running` | 正在执行 |
|
||||
| `completed` | 已成功完成 |
|
||||
| `failed` | 执行失败 |
|
||||
| `cancelled` | 已通过 DELETE 接口取消 |
|
||||
|
||||
---
|
||||
|
||||
## Webhook 回调
|
||||
|
||||
创建任务时可设置 `webhook` 字段。当任务进入终态(`completed`、`failed`、`cancelled`)时,Bot 会向该地址发送一个 `POST` 请求。
|
||||
|
||||
**回调请求头:**
|
||||
|
||||
```
|
||||
Content-Type: application/json
|
||||
User-Agent: SaveAny-Bot/1.0
|
||||
```
|
||||
|
||||
**回调请求体:**
|
||||
|
||||
```json
|
||||
{
|
||||
"task_id": "abc123xyz",
|
||||
"type": "directlinks",
|
||||
"status": "completed",
|
||||
"storage": "local",
|
||||
"path": "downloads",
|
||||
"completed_at": "2026-03-11T10:01:00Z",
|
||||
"error": ""
|
||||
}
|
||||
```
|
||||
|
||||
`completed_at` 仅在状态为 `completed` 或 `failed` 时出现。`error` 仅在有错误时出现。
|
||||
|
||||
**重试机制:** 最多重试 3 次,重试间隔依次为 1 秒、2 秒、3 秒。每次请求超时为 30 秒。
|
||||
41
docs/content/zh/usage/aria2.md
Normal file
41
docs/content/zh/usage/aria2.md
Normal file
@@ -0,0 +1,41 @@
|
||||
---
|
||||
title: "Aria2 下载"
|
||||
weight: 6
|
||||
---
|
||||
|
||||
# Aria2 下载
|
||||
|
||||
{{< hint warning >}}
|
||||
该功能需要在配置文件中启用 Aria2 并配置 RPC 连接.
|
||||
{{< /hint >}}
|
||||
|
||||
使用 `/aria2dl` 命令可以通过 Aria2 下载管理器下载文件, 支持 HTTP/HTTPS、FTP、BitTorrent 等多种协议.
|
||||
|
||||
```bash
|
||||
/aria2dl <uri1> [uri2] [uri3] ...
|
||||
```
|
||||
|
||||
示例:
|
||||
|
||||
```bash
|
||||
# 下载 HTTP 链接
|
||||
/aria2dl https://example.com/file.zip
|
||||
|
||||
# 下载磁力链接
|
||||
/aria2dl magnet:?xt=urn:btih:...
|
||||
|
||||
# 下载种子文件 (需要先上传 .torrent 文件)
|
||||
/aria2dl https://example.com/file.torrent
|
||||
```
|
||||
|
||||
配置 Aria2:
|
||||
|
||||
在 `config.toml` 中添加:
|
||||
|
||||
```toml
|
||||
[aria2]
|
||||
enable = true
|
||||
url = "http://localhost:6800/jsonrpc"
|
||||
secret = "your-rpc-secret" # 如果配置了 rpc-secret
|
||||
remove_after_transfer = true # 转存完成后删除本地文件
|
||||
```
|
||||
21
docs/content/zh/usage/directlinks.md
Normal file
21
docs/content/zh/usage/directlinks.md
Normal file
@@ -0,0 +1,21 @@
|
||||
---
|
||||
title: "直接下载链接"
|
||||
weight: 5
|
||||
---
|
||||
|
||||
# 直接下载链接
|
||||
|
||||
使用 `/dl` 命令可以直接下载一个或多个 HTTP/HTTPS 链接的文件到存储中.
|
||||
|
||||
```bash
|
||||
/dl <url1> [url2] [url3] ...
|
||||
```
|
||||
|
||||
示例:
|
||||
|
||||
```bash
|
||||
/dl https://example.com/file.zip
|
||||
/dl https://example.com/file1.zip https://example.com/file2.zip
|
||||
```
|
||||
|
||||
Bot 会验证链接格式, 然后让你选择目标存储位置.
|
||||
15
docs/content/zh/usage/parsers.md
Normal file
15
docs/content/zh/usage/parsers.md
Normal file
@@ -0,0 +1,15 @@
|
||||
---
|
||||
title: "转存 Telegram 之外的文件"
|
||||
weight: 9
|
||||
---
|
||||
|
||||
# 转存 Telegram 之外的文件
|
||||
|
||||
除了 Telegram 上的文件, Bot 还可通过 JavaScript 插件或内置解析器来支持转存其他网站的文件.
|
||||
|
||||
> 查看[贡献解析器](../contribute)文档了解详情
|
||||
|
||||
只需向 Bot 发送符合解析器要求的链接即可使用, 当前内置的解析器:
|
||||
|
||||
- Twitter
|
||||
- Kemono
|
||||
58
docs/content/zh/usage/rules.md
Normal file
58
docs/content/zh/usage/rules.md
Normal file
@@ -0,0 +1,58 @@
|
||||
---
|
||||
title: "存储规则"
|
||||
weight: 3
|
||||
---
|
||||
|
||||
# 存储规则
|
||||
|
||||
允许你为 Bot 在上传文件到存储时设置一些重定向规则, 用于自动整理所保存的文件.
|
||||
|
||||
见: <a href="https://github.com/krau/SaveAny-Bot/issues/28" target="_blank">#28</a>
|
||||
|
||||
目前支持的规则类型:
|
||||
|
||||
1. FILENAME-REGEX
|
||||
2. MESSAGE-REGEX
|
||||
3. IS-ALBUM
|
||||
|
||||
添加规则的基本语法:
|
||||
|
||||
"规则类型 规则内容 存储名 路径"
|
||||
|
||||
注意空格的使用, 语法正确 bot 才能解析, 以下是一条合法的添加规则命令:
|
||||
|
||||
```
|
||||
/rule add FILENAME-REGEX (?i)\.(mp4|mkv|ts|avi|flv)$ MyAlist /视频
|
||||
```
|
||||
|
||||
此外, 规则中的存储名若使用 "CHOSEN" , 则表示存储到点击按钮选择的存储端的路径下
|
||||
|
||||
规则类型:
|
||||
|
||||
## FILENAME-REGEX
|
||||
|
||||
根据文件名正则匹配, 规则内容要求为一个合法的正则表达式, 如
|
||||
|
||||
```
|
||||
FILENAME-REGEX (?i)\.(mp4|mkv|ts|avi|flv)$ MyAlist /视频
|
||||
```
|
||||
|
||||
表示将文件名后缀为 mp4,mkv,ts,avi,flv 的文件放到名为 MyAlist 存储下的 /视频 目录内 (同时受配置文件中的 `base_path` 影响)
|
||||
|
||||
## MESSAGE-REGEX
|
||||
|
||||
同上, 但是是根据消息本身的文本内容正则匹配
|
||||
|
||||
## IS-ALBUM
|
||||
|
||||
匹配相册消息 (media group), 规则内容只能为 `true` 或 `false`.
|
||||
|
||||
规则中的路径若使用 "NEW-FOR-ALBUM" , 则表示为该组消息新建一个文件夹来存储它们. 见: https://github.com/krau/SaveAny-Bot/issues/87
|
||||
|
||||
例如:
|
||||
|
||||
```
|
||||
IS-ALBUM true MyWebdav NEW-FOR-ALBUM
|
||||
```
|
||||
|
||||
这将会把以 media group 形式发送的消息保存到名为 MyWebdav 的存储下, 并为每个相册新建一个文件夹(由第一个文件生成)来存储它们.
|
||||
14
docs/content/zh/usage/silent.md
Normal file
14
docs/content/zh/usage/silent.md
Normal file
@@ -0,0 +1,14 @@
|
||||
---
|
||||
title: "静默模式"
|
||||
weight: 2
|
||||
---
|
||||
|
||||
# 静默模式 (silent)
|
||||
|
||||
使用 `/silent` 命令可以开关静默模式.
|
||||
|
||||
默认情况下不开启静默模式, Bot 会询问你每个文件的保存位置.
|
||||
|
||||
开启静默模式后, Bot 会直接保存文件到默认位置, 无需确认.
|
||||
|
||||
在开启静默模式之前, 需要使用 `/storage` 命令设置默认保存位置.
|
||||
50
docs/content/zh/usage/transfer.md
Normal file
50
docs/content/zh/usage/transfer.md
Normal file
@@ -0,0 +1,50 @@
|
||||
---
|
||||
title: "存储间传输"
|
||||
weight: 8
|
||||
---
|
||||
|
||||
# 存储间传输
|
||||
|
||||
使用 `/transfer` 命令可以在不同存储之间直接传输文件, 无需经过 Telegram.
|
||||
|
||||
```bash
|
||||
/transfer <source_storage>:/<source_path> [filter]
|
||||
```
|
||||
|
||||
参数说明:
|
||||
|
||||
- `source_storage`: 源存储名称
|
||||
- `source_path`: 源路径
|
||||
- `filter`: 可选的正则表达式过滤器, 只传输匹配的文件
|
||||
|
||||
示例:
|
||||
|
||||
```bash
|
||||
# 传输整个目录
|
||||
/transfer local1:/downloads
|
||||
|
||||
# 传输指定路径的文件
|
||||
/transfer alist1:/media/photos
|
||||
|
||||
# 只传输 mp4 文件
|
||||
/transfer webdav1:/videos ".*\.mp4$"
|
||||
|
||||
# 传输图片文件
|
||||
/transfer local1:/pictures "(?i)\.(jpg|png|gif)$"
|
||||
```
|
||||
|
||||
Bot 会:
|
||||
|
||||
1. 列出源路径下的所有文件
|
||||
2. 应用过滤器 (如果提供)
|
||||
3. 显示文件数量和总大小
|
||||
4. 让你选择目标存储
|
||||
5. 让你选择目标目录 (如果该存储配置了目录)
|
||||
6. 开始传输任务
|
||||
|
||||
注意:
|
||||
|
||||
- 源存储必须支持列举和读取功能
|
||||
- 目标存储必须支持写入功能
|
||||
- 传输过程显示实时进度
|
||||
- 支持取消正在进行的传输任务
|
||||
36
docs/content/zh/usage/watch.md
Normal file
36
docs/content/zh/usage/watch.md
Normal file
@@ -0,0 +1,36 @@
|
||||
---
|
||||
title: "监听聊天"
|
||||
weight: 4
|
||||
---
|
||||
|
||||
# 监听聊天
|
||||
|
||||
{{< hint warning >}}
|
||||
该功能需开启 UserBot 集成.
|
||||
{{< /hint >}}
|
||||
|
||||
监听指定聊天的消息, 并自动保存到默认存储中, 遵从存储规则, 并且可以设置过滤器来只保存匹配的消息.
|
||||
|
||||
监听聊天:
|
||||
|
||||
```
|
||||
/watch <chat_id/username> [filter]
|
||||
```
|
||||
|
||||
取消监听:
|
||||
|
||||
```
|
||||
/unwatch <chat_id/username>
|
||||
```
|
||||
|
||||
过滤器类型:
|
||||
|
||||
## msgre
|
||||
|
||||
正则匹配消息文本, 例如:
|
||||
|
||||
```
|
||||
/watch 12345678 msgre:.*hello.*
|
||||
```
|
||||
|
||||
这将会监听 ID 为 12345678 的聊天, 并且只保存消息文本中包含 "hello" 的消息.
|
||||
40
docs/content/zh/usage/ytdlp.md
Normal file
40
docs/content/zh/usage/ytdlp.md
Normal file
@@ -0,0 +1,40 @@
|
||||
---
|
||||
title: "yt-dlp 视频下载"
|
||||
weight: 7
|
||||
---
|
||||
|
||||
# yt-dlp 视频下载
|
||||
|
||||
{{< hint warning >}}
|
||||
该功能需要在系统中安装 yt-dlp 命令行工具.
|
||||
{{< /hint >}}
|
||||
|
||||
使用 `/ytdlp` 命令可以下载支持的视频网站的视频和音频, 支持 YouTube、Bilibili、Twitter 等 1000+ 个网站.
|
||||
|
||||
```bash
|
||||
/ytdlp <url1> [url2] [flags...]
|
||||
```
|
||||
|
||||
示例:
|
||||
|
||||
```bash
|
||||
# 基本下载
|
||||
/ytdlp https://www.youtube.com/watch?v=dQw4w9WgXcQ
|
||||
|
||||
# 下载多个视频
|
||||
/ytdlp https://www.youtube.com/watch?v=video1 https://www.youtube.com/watch?v=video2
|
||||
|
||||
# 使用自定义参数
|
||||
/ytdlp https://www.youtube.com/watch?v=dQw4w9WgXcQ -f best
|
||||
/ytdlp https://www.youtube.com/watch?v=dQw4w9WgXcQ --extract-audio --audio-format mp3
|
||||
```
|
||||
|
||||
常用参数:
|
||||
|
||||
- `-f <format>`: 指定下载格式 (如 `best`, `worst`, `bestvideo+bestaudio`)
|
||||
- `--extract-audio`: 提取音频
|
||||
- `--audio-format <format>`: 音频格式 (如 `mp3`, `m4a`, `wav`)
|
||||
- `--write-sub`: 下载字幕
|
||||
- `--write-thumbnail`: 下载缩略图
|
||||
|
||||
更多参数请参考 [yt-dlp 文档](https://github.com/yt-dlp/yt-dlp#usage-and-options).
|
||||
92
go.mod
92
go.mod
@@ -1,47 +1,51 @@
|
||||
module github.com/krau/SaveAny-Bot
|
||||
|
||||
go 1.24.0
|
||||
go 1.25.0
|
||||
|
||||
require (
|
||||
github.com/blang/semver v3.5.1+incompatible
|
||||
github.com/celestix/gotgproto v1.0.0-beta22
|
||||
github.com/cenkalti/backoff/v4 v4.3.0
|
||||
github.com/charmbracelet/bubbles v0.21.0
|
||||
github.com/charmbracelet/bubbles v1.0.0
|
||||
github.com/charmbracelet/bubbletea v1.3.10
|
||||
github.com/charmbracelet/lipgloss v1.1.0
|
||||
github.com/charmbracelet/log v0.4.2
|
||||
github.com/dustin/go-humanize v1.0.1
|
||||
github.com/gabriel-vasile/mimetype v1.4.10
|
||||
github.com/goccy/go-yaml v1.18.0
|
||||
github.com/gabriel-vasile/mimetype v1.4.13
|
||||
github.com/goccy/go-yaml v1.19.2
|
||||
github.com/gotd/contrib v0.21.1
|
||||
github.com/gotd/td v0.136.0
|
||||
github.com/gotd/td v0.140.0
|
||||
github.com/johannesboyne/gofakes3 v0.0.0-20250916175020-ebf3e50324d3
|
||||
github.com/krau/ffmpeg-go v0.6.0
|
||||
github.com/minio/minio-go/v7 v7.0.95
|
||||
github.com/playwright-community/playwright-go v0.5200.1
|
||||
github.com/lrstanley/go-ytdlp v1.3.2
|
||||
github.com/minio/minio-go/v7 v7.0.98
|
||||
github.com/playwright-community/playwright-go v0.5700.1
|
||||
github.com/rs/xid v1.6.0
|
||||
github.com/spf13/cobra v1.10.1
|
||||
github.com/spf13/cobra v1.10.2
|
||||
github.com/spf13/viper v1.21.0
|
||||
github.com/unvgo/ghselfupdate v1.0.0
|
||||
github.com/unvgo/ghselfupdate v1.0.1
|
||||
github.com/yapingcat/gomedia v0.0.0-20240906162731-17feea57090c
|
||||
golang.org/x/net v0.47.0
|
||||
golang.org/x/term v0.37.0
|
||||
golang.org/x/net v0.51.0
|
||||
golang.org/x/term v0.40.0
|
||||
golang.org/x/time v0.14.0
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/AnimeKaizoku/cacher v1.0.3 // indirect
|
||||
github.com/ProtonMail/go-crypto v1.4.0 // indirect
|
||||
github.com/aws/smithy-go v1.24.0 // indirect
|
||||
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
|
||||
github.com/cespare/xxhash/v2 v2.3.0 // indirect
|
||||
github.com/charmbracelet/colorprofile v0.3.2 // indirect
|
||||
github.com/charmbracelet/colorprofile v0.4.2 // indirect
|
||||
github.com/charmbracelet/harmonica v0.2.0 // indirect
|
||||
github.com/charmbracelet/x/ansi v0.10.2 // indirect
|
||||
github.com/charmbracelet/x/cellbuf v0.0.13 // indirect
|
||||
github.com/charmbracelet/x/term v0.2.1 // indirect
|
||||
github.com/clipperhouse/uax29/v2 v2.2.0 // indirect
|
||||
github.com/charmbracelet/x/ansi v0.11.6 // indirect
|
||||
github.com/charmbracelet/x/cellbuf v0.0.15 // indirect
|
||||
github.com/charmbracelet/x/term v0.2.2 // indirect
|
||||
github.com/clipperhouse/displaywidth v0.11.0 // indirect
|
||||
github.com/clipperhouse/uax29/v2 v2.7.0 // indirect
|
||||
github.com/cloudflare/circl v1.6.3 // indirect
|
||||
github.com/coder/websocket v1.8.14 // indirect
|
||||
github.com/deckarep/golang-set/v2 v2.7.0 // indirect
|
||||
github.com/deckarep/golang-set/v2 v2.8.0 // indirect
|
||||
github.com/dlclark/regexp2 v1.11.5 // indirect
|
||||
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f // indirect
|
||||
github.com/fatih/color v1.18.0 // indirect
|
||||
@@ -56,11 +60,10 @@ require (
|
||||
github.com/go-logfmt/logfmt v0.6.1 // indirect
|
||||
github.com/go-sourcemap/sourcemap v2.1.4+incompatible // indirect
|
||||
github.com/go-stack/stack v1.8.1 // indirect
|
||||
github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
|
||||
github.com/goccy/go-json v0.10.5 // indirect
|
||||
github.com/go-viper/mapstructure/v2 v2.5.0 // indirect
|
||||
github.com/google/go-github/v30 v30.1.0 // indirect
|
||||
github.com/google/go-querystring v1.1.0 // indirect
|
||||
github.com/google/pprof v0.0.0-20251007162407-5df77e3f7d1d // indirect
|
||||
github.com/google/go-querystring v1.2.0 // indirect
|
||||
github.com/google/pprof v0.0.0-20260302011040-a15ffb7f9dcc // indirect
|
||||
github.com/google/uuid v1.6.0 // indirect
|
||||
github.com/gotd/ige v0.2.2 // indirect
|
||||
github.com/gotd/neo v0.1.5 // indirect
|
||||
@@ -68,11 +71,12 @@ require (
|
||||
github.com/jinzhu/inflection v1.0.0 // indirect
|
||||
github.com/jinzhu/now v1.1.5 // indirect
|
||||
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
|
||||
github.com/klauspost/crc32 v1.3.0 // indirect
|
||||
github.com/lucasb-eyer/go-colorful v1.3.0 // indirect
|
||||
github.com/mattn/go-colorable v0.1.14 // indirect
|
||||
github.com/mattn/go-isatty v0.0.20 // indirect
|
||||
github.com/mattn/go-localereader v0.0.1 // indirect
|
||||
github.com/mattn/go-runewidth v0.0.19 // indirect
|
||||
github.com/mattn/go-runewidth v0.0.20 // indirect
|
||||
github.com/minio/crc64nvme v1.1.1 // indirect
|
||||
github.com/minio/md5-simd v1.1.2 // indirect
|
||||
github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 // indirect
|
||||
@@ -80,7 +84,7 @@ require (
|
||||
github.com/muesli/termenv v0.16.0 // indirect
|
||||
github.com/ncruces/go-strftime v1.0.0 // indirect
|
||||
github.com/ncruces/julianday v1.0.0 // indirect
|
||||
github.com/ogen-go/ogen v1.16.0 // indirect
|
||||
github.com/ogen-go/ogen v1.20.1 // indirect
|
||||
github.com/philhofer/fwd v1.2.0 // indirect
|
||||
github.com/pkg/errors v0.9.1 // indirect
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
|
||||
@@ -88,39 +92,39 @@ require (
|
||||
github.com/ryszard/goskiplist v0.0.0-20150312221310-2dfbae5fcf46 // indirect
|
||||
github.com/segmentio/asm v1.2.1 // indirect
|
||||
github.com/shopspring/decimal v1.4.0 // indirect
|
||||
github.com/tetratelabs/wazero v1.10.1 // indirect
|
||||
github.com/tinylib/msgp v1.4.0 // indirect
|
||||
github.com/tetratelabs/wazero v1.11.0 // indirect
|
||||
github.com/tinylib/msgp v1.6.3 // indirect
|
||||
github.com/ulikunitz/xz v0.5.15 // indirect
|
||||
go.opentelemetry.io/otel v1.38.0 // indirect
|
||||
go.opentelemetry.io/otel/metric v1.38.0 // indirect
|
||||
go.opentelemetry.io/otel/trace v1.38.0 // indirect
|
||||
go.opentelemetry.io/otel v1.41.0 // indirect
|
||||
go.opentelemetry.io/otel/metric v1.41.0 // indirect
|
||||
go.opentelemetry.io/otel/trace v1.41.0 // indirect
|
||||
go.shabbyrobe.org/gocovmerge v0.0.0-20230507111327-fa4f82cfbf4d // indirect
|
||||
go.uber.org/atomic v1.11.0 // indirect
|
||||
go.uber.org/zap v1.27.1 // indirect
|
||||
go.yaml.in/yaml/v3 v3.0.4 // indirect
|
||||
golang.org/x/crypto v0.45.0 // indirect
|
||||
golang.org/x/mod v0.30.0 // indirect
|
||||
golang.org/x/tools v0.39.0 // indirect
|
||||
golang.org/x/crypto v0.48.0 // indirect
|
||||
golang.org/x/mod v0.33.0 // indirect
|
||||
golang.org/x/tools v0.42.0 // indirect
|
||||
gopkg.in/yaml.v2 v2.4.0 // indirect
|
||||
modernc.org/libc v1.66.10 // indirect
|
||||
modernc.org/libc v1.69.0 // indirect
|
||||
modernc.org/mathutil v1.7.1 // indirect
|
||||
modernc.org/memory v1.11.0 // indirect
|
||||
modernc.org/sqlite v1.39.1 // indirect
|
||||
modernc.org/sqlite v1.46.1 // indirect
|
||||
rsc.io/qr v0.2.0 // indirect
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/dgraph-io/ristretto/v2 v2.3.0
|
||||
github.com/dop251/goja v0.0.0-20251008123653-cf18d89f3cf6
|
||||
github.com/duke-git/lancet/v2 v2.3.7
|
||||
github.com/dgraph-io/ristretto/v2 v2.4.0
|
||||
github.com/dop251/goja v0.0.0-20260226184354-913bd86fb70c
|
||||
github.com/duke-git/lancet/v2 v2.3.8
|
||||
github.com/fsnotify/fsnotify v1.9.0 // indirect
|
||||
github.com/glebarez/sqlite v1.11.0
|
||||
github.com/inconshreveable/mousetrap v1.1.0 // indirect
|
||||
github.com/klauspost/compress v1.18.2 // indirect
|
||||
github.com/klauspost/compress v1.18.4 // indirect
|
||||
github.com/mitchellh/mapstructure v1.5.0
|
||||
github.com/ncruces/go-sqlite3 v0.30.1
|
||||
github.com/ncruces/go-sqlite3/gormlite v0.30.1
|
||||
github.com/nicksnyder/go-i18n/v2 v2.6.0
|
||||
github.com/ncruces/go-sqlite3 v0.30.5
|
||||
github.com/ncruces/go-sqlite3/gormlite v0.30.2
|
||||
github.com/nicksnyder/go-i18n/v2 v2.6.1
|
||||
github.com/pelletier/go-toml/v2 v2.2.4 // indirect
|
||||
github.com/sagikazarmark/locafero v0.12.0 // indirect
|
||||
github.com/spf13/afero v1.15.0 // indirect
|
||||
@@ -129,9 +133,9 @@ require (
|
||||
github.com/subosito/gotenv v1.6.0 // indirect
|
||||
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
|
||||
go.uber.org/multierr v1.11.0 // indirect
|
||||
golang.org/x/exp v0.0.0-20251023183803-a4bb9ffd2546 // indirect
|
||||
golang.org/x/sync v0.18.0
|
||||
golang.org/x/sys v0.38.0 // indirect
|
||||
golang.org/x/text v0.31.0
|
||||
golang.org/x/exp v0.0.0-20260218203240-3dfff04db8fa // indirect
|
||||
golang.org/x/sync v0.19.0
|
||||
golang.org/x/sys v0.41.0 // indirect
|
||||
golang.org/x/text v0.34.0
|
||||
gorm.io/gorm v1.31.1
|
||||
)
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user