Compare commits

..

13 Commits

Author SHA1 Message Date
copilot-swe-agent[bot]
ef1ecf960f Fix album (NEW-FOR-ALBUM) rule handling and proper filename logic
Co-authored-by: krau <71133316+krau@users.noreply.github.com>
2026-01-19 07:32:52 +00:00
copilot-swe-agent[bot]
e1b4087801 Add storage rule support to API task creation
Co-authored-by: krau <71133316+krau@users.noreply.github.com>
2026-01-19 07:17:35 +00:00
copilot-swe-agent[bot]
6896bdc852 Fix performance issues, add media group support, and improve filename handling
Co-authored-by: krau <71133316+krau@users.noreply.github.com>
2026-01-19 06:14:48 +00:00
copilot-swe-agent[bot]
3a6402a71b Add validation to require API token when API is enabled
Co-authored-by: krau <71133316+krau@users.noreply.github.com>
2026-01-19 05:49:12 +00:00
copilot-swe-agent[bot]
173a5e3733 Remove IP whitelist mechanism, keep only token authentication
Co-authored-by: krau <71133316+krau@users.noreply.github.com>
2026-01-19 05:42:14 +00:00
copilot-swe-agent[bot]
20a5e317ae Address PR feedback: remove redundant files, format code, add progress tracking
Co-authored-by: krau <71133316+krau@users.noreply.github.com>
2026-01-19 05:27:12 +00:00
copilot-swe-agent[bot]
127901fd24 Final improvements: better user ID validation, safer IP handling, context-aware logging
Co-authored-by: krau <71133316+krau@users.noreply.github.com>
2026-01-19 04:52:43 +00:00
copilot-swe-agent[bot]
30c165033e Address all security review comments: sanitize remaining error messages and handle errors properly
Co-authored-by: krau <71133316+krau@users.noreply.github.com>
2026-01-19 04:50:48 +00:00
copilot-swe-agent[bot]
9dcb5201e1 Fix security issues: sanitize error messages and fix test port
Co-authored-by: krau <71133316+krau@users.noreply.github.com>
2026-01-19 04:49:03 +00:00
copilot-swe-agent[bot]
7b0142ef82 Add API tests and test script
Co-authored-by: krau <71133316+krau@users.noreply.github.com>
2026-01-19 04:46:08 +00:00
copilot-swe-agent[bot]
2f6b2470a4 Add API documentation in English and Chinese
Co-authored-by: krau <71133316+krau@users.noreply.github.com>
2026-01-19 04:44:20 +00:00
copilot-swe-agent[bot]
ac10c32215 Add HTTP API server for file downloads from Telegram links
Co-authored-by: krau <71133316+krau@users.noreply.github.com>
2026-01-19 04:42:55 +00:00
copilot-swe-agent[bot]
7def7f5b28 Initial plan 2026-01-19 04:32:43 +00:00
145 changed files with 1711 additions and 8397 deletions

3
.gitignore vendored
View File

@@ -11,5 +11,4 @@ temp/
playwright/
testplugins/
*.exe
tmp-*
saveany-bot
tmp-*

View File

@@ -26,7 +26,7 @@ RUN --mount=type=cache,target=/root/.cache/go-build \
FROM alpine:latest
RUN apk add --no-cache curl ffmpeg yt-dlp
RUN apk add --no-cache curl ffmpeg
WORKDIR /app

View File

@@ -7,8 +7,6 @@ ARG BuildTime="Unknown"
WORKDIR /app
RUN apk add --no-cache ca-certificates
COPY go.mod go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod \
go mod download
@@ -33,9 +31,5 @@ FROM scratch
WORKDIR /app
COPY --from=builder /app/saveany-bot .
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
ENV SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
ENV SSL_CERT_DIR=/etc/ssl/certs
ENTRYPOINT ["/app/saveany-bot"]

View File

@@ -26,16 +26,12 @@
- Multi-user support
- Auto organize files based on storage rules
- Watch specified chats and auto-save messages, with filters
- Transfer files between different storage backends
- Integrate with yt-dlp to download and save media from 1000+ websites
- Aria2 integration to download files from URLs/magnets and save to storages
- Write JS parser plugins to save files from almost any website
- Storage backends:
- Alist
- S3
- WebDAV
- Local filesystem
- Rclone (via command line)
- Telegram (re-upload to specified chats)
## 📦 Quick Start

View File

@@ -24,16 +24,12 @@
- 多用户使用
- 基于存储规则的自动整理
- 监听并自动转存指定聊天的消息, 支持过滤
- 在不同存储端之间转存文件
- 集成 yt-dlp, 从所支持的网站下载并转存媒体文件
- 集成 Aria2, 支持直链/磁力下载和转存
- 使用 js 编写解析器插件以转存任意网站的文件
- 存储端支持:
- Alist
- S3
- WebDAV
- 本地磁盘
- Rclone
- Telegram (重传回指定聊天)
## 快速开始

View File

@@ -1,48 +0,0 @@
package api
import (
"context"
"crypto/subtle"
"net/http"
"strings"
"github.com/krau/SaveAny-Bot/config"
)
// tokenContextKey 用于在 context 中存储 token
type tokenContextKey struct{}
// AuthMiddleware 返回认证中间件
func AuthMiddleware() func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
cfg := config.C().API
// 从请求头获取 token
authHeader := r.Header.Get("Authorization")
if authHeader == "" {
WriteError(w, http.StatusUnauthorized, "unauthorized", "missing authorization header")
return
}
// 提取 Bearer token
parts := strings.SplitN(authHeader, " ", 2)
if len(parts) != 2 || strings.ToLower(parts[0]) != "bearer" {
WriteError(w, http.StatusUnauthorized, "unauthorized", "invalid authorization header format")
return
}
token := parts[1]
// 验证 token
if subtle.ConstantTimeCompare([]byte(token), []byte(cfg.Token)) != 1 {
WriteError(w, http.StatusUnauthorized, "unauthorized", "invalid token")
return
}
// 将 token 添加到 context
ctx := context.WithValue(r.Context(), tokenContextKey{}, token)
next.ServeHTTP(w, r.WithContext(ctx))
})
}
}

View File

@@ -1,382 +0,0 @@
package api
import (
"context"
"encoding/json"
"fmt"
"time"
"github.com/krau/SaveAny-Bot/config"
"github.com/krau/SaveAny-Bot/core"
"github.com/krau/SaveAny-Bot/core/tasks/aria2dl"
"github.com/krau/SaveAny-Bot/core/tasks/batchtfile"
"github.com/krau/SaveAny-Bot/core/tasks/directlinks"
"github.com/krau/SaveAny-Bot/core/tasks/parsed"
tphtask "github.com/krau/SaveAny-Bot/core/tasks/telegraph"
"github.com/krau/SaveAny-Bot/core/tasks/tfile"
"github.com/krau/SaveAny-Bot/core/tasks/transfer"
"github.com/krau/SaveAny-Bot/core/tasks/ytdlp"
"github.com/krau/SaveAny-Bot/parsers/parsers"
"github.com/krau/SaveAny-Bot/pkg/aria2"
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
"github.com/krau/SaveAny-Bot/pkg/parser"
"github.com/krau/SaveAny-Bot/pkg/taskevent"
"github.com/krau/SaveAny-Bot/pkg/telegraph"
"github.com/krau/SaveAny-Bot/storage"
"github.com/rs/xid"
)
// TaskFactory 任务工厂
type TaskFactory struct {
ctx context.Context
}
// NewTaskFactory 创建任务工厂
func NewTaskFactory(ctx context.Context) *TaskFactory {
return &TaskFactory{ctx: ctx}
}
// CreateTask 创建任务
func (f *TaskFactory) CreateTask(req *CreateTaskRequest) (*CreateTaskResponse, error) {
// 验证存储
stor, ok := storage.Storages[req.Storage]
if !ok {
return nil, fmt.Errorf("storage not found: %s", req.Storage)
}
taskID := xid.New().String()
createdAt := time.Now()
switch req.Type {
case tasktype.TaskTypeDirectlinks:
return f.createDirectLinksTask(taskID, createdAt, req, stor)
case tasktype.TaskTypeYtdlp:
return f.createYTDLPTask(taskID, createdAt, req, stor)
case tasktype.TaskTypeAria2:
return f.createAria2Task(taskID, createdAt, req, stor)
case tasktype.TaskTypeParseditem:
return f.createParsedTask(taskID, createdAt, req, stor)
case tasktype.TaskTypeTgfiles:
return f.createTGFilesTask(taskID, createdAt, req, stor)
case tasktype.TaskTypeTphpics:
return f.createTPHPicsTask(taskID, createdAt, req, stor)
case tasktype.TaskTypeTransfer:
return f.createTransferTask(taskID, createdAt, req)
default:
return nil, fmt.Errorf("unsupported task type: %s", req.Type)
}
}
func (f *TaskFactory) registerAndEnqueueTask(task core.Executable, taskType tasktype.TaskType, storageName, path, webhook string) error {
taskID := task.TaskID()
info := RegisterTask(taskID, string(taskType), storageName, path, task.Title(), webhook)
// Inject the progress sink into the context so the task's Emit calls update
// the API store (and fire the webhook on terminal states) without the task
// knowing about the API.
taskCtx := taskevent.WithSink(f.ctx, info)
err := core.AddTask(taskCtx, task)
if err != nil {
DeleteTask(taskID)
return fmt.Errorf("failed to add task: %w", err)
}
return nil
}
// createDirectLinksTask 创建直链下载任务
func (f *TaskFactory) createDirectLinksTask(taskID string, createdAt time.Time, req *CreateTaskRequest, stor storage.Storage) (*CreateTaskResponse, error) {
var params DirectLinksParams
if err := json.Unmarshal(req.Params, &params); err != nil {
return nil, fmt.Errorf("invalid params: %w", err)
}
if len(params.URLs) == 0 {
return nil, fmt.Errorf("no URLs provided")
}
task := directlinks.NewTask(taskID, f.ctx, params.URLs, stor, req.Path, nil)
err := f.registerAndEnqueueTask(task, tasktype.TaskTypeDirectlinks, req.Storage, req.Path, req.Webhook)
if err != nil {
return nil, err
}
return &CreateTaskResponse{
TaskID: taskID,
Type: tasktype.TaskTypeDirectlinks,
Status: TaskStatusQueued,
CreatedAt: createdAt,
}, nil
}
// createYTDLPTask 创建 yt-dlp 任务
func (f *TaskFactory) createYTDLPTask(taskID string, createdAt time.Time, req *CreateTaskRequest, stor storage.Storage) (*CreateTaskResponse, error) {
var params YTDLPParams
if err := json.Unmarshal(req.Params, &params); err != nil {
return nil, fmt.Errorf("invalid params: %w", err)
}
if len(params.URLs) == 0 {
return nil, fmt.Errorf("no URLs provided")
}
task := ytdlp.NewTask(taskID, f.ctx, params.URLs, params.Flags, stor, req.Path, nil)
err := f.registerAndEnqueueTask(task, tasktype.TaskTypeYtdlp, req.Storage, req.Path, req.Webhook)
if err != nil {
return nil, err
}
return &CreateTaskResponse{
TaskID: taskID,
Type: tasktype.TaskTypeYtdlp,
Status: TaskStatusQueued,
CreatedAt: createdAt,
}, nil
}
// createAria2Task 创建 Aria2 任务
func (f *TaskFactory) createAria2Task(taskID string, createdAt time.Time, req *CreateTaskRequest, stor storage.Storage) (*CreateTaskResponse, error) {
var params Aria2Params
if err := json.Unmarshal(req.Params, &params); err != nil {
return nil, fmt.Errorf("invalid params: %w", err)
}
if len(params.URLs) == 0 {
return nil, fmt.Errorf("no URLs provided")
}
// 检查 Aria2 是否启用
cfg := config.C().Aria2
if !cfg.Enable {
return nil, fmt.Errorf("aria2 is not enabled")
}
aria2Client, err := aria2.NewClient(cfg.Url, cfg.Secret)
if err != nil {
return nil, fmt.Errorf("failed to create aria2 client: %w", err)
}
// 添加下载任务到 Aria2
gid, err := aria2Client.AddURI(f.ctx, params.URLs, nil)
if err != nil {
return nil, fmt.Errorf("failed to add aria2 task: %w", err)
}
task := aria2dl.NewTask(taskID, f.ctx, gid, params.URLs, aria2Client, stor, req.Path, nil)
err = f.registerAndEnqueueTask(task, tasktype.TaskTypeAria2, req.Storage, req.Path, req.Webhook)
if err != nil {
return nil, err
}
return &CreateTaskResponse{
TaskID: taskID,
Type: tasktype.TaskTypeAria2,
Status: TaskStatusQueued,
CreatedAt: createdAt,
}, nil
}
// createParsedTask 创建解析任务
func (f *TaskFactory) createParsedTask(taskID string, createdAt time.Time, req *CreateTaskRequest, stor storage.Storage) (*CreateTaskResponse, error) {
var params ParsedParams
if err := json.Unmarshal(req.Params, &params); err != nil {
return nil, fmt.Errorf("invalid params: %w", err)
}
if params.URL == "" {
return nil, fmt.Errorf("no URL provided")
}
// 查找合适的解析器
var p parser.Parser
for _, parserItem := range parsers.Get() {
if parserItem.CanHandle(params.URL) {
p = parserItem
break
}
}
if p == nil {
return nil, fmt.Errorf("no parser found for URL: %s", params.URL)
}
// 解析 URL
item, err := p.Parse(f.ctx, params.URL)
if err != nil {
return nil, fmt.Errorf("failed to parse URL: %w", err)
}
task := parsed.NewTask(taskID, f.ctx, stor, req.Path, item, nil)
err = f.registerAndEnqueueTask(task, tasktype.TaskTypeParseditem, req.Storage, req.Path, req.Webhook)
if err != nil {
return nil, err
}
return &CreateTaskResponse{
TaskID: taskID,
Type: tasktype.TaskTypeParseditem,
Status: TaskStatusQueued,
CreatedAt: createdAt,
}, nil
}
// createTGFilesTask 创建 Telegram 文件下载任务
func (f *TaskFactory) createTGFilesTask(taskID string, createdAt time.Time, req *CreateTaskRequest, stor storage.Storage) (*CreateTaskResponse, error) {
var params TGFilesParams
if err := json.Unmarshal(req.Params, &params); err != nil {
return nil, fmt.Errorf("invalid params: %w", err)
}
if len(params.MessageLinks) == 0 {
return nil, fmt.Errorf("no message links provided")
}
// 提取文件
files, err := ExtractFilesFromLinks(f.ctx, params.MessageLinks)
if err != nil {
return nil, fmt.Errorf("failed to extract files: %w", err)
}
if len(files) == 0 {
return nil, fmt.Errorf("no files found in provided links")
}
var task core.Executable
if len(files) == 1 {
// 单个文件任务
tfileTask, err := tfile.NewTGFileTask(taskID, f.ctx, files[0], stor, req.Path, nil)
if err != nil {
return nil, fmt.Errorf("failed to create tfile task: %w", err)
}
task = tfileTask
} else {
// 批量文件任务
elems := make([]batchtfile.TaskElement, 0, len(files))
for _, file := range files {
elem, err := batchtfile.NewTaskElement(stor, req.Path, file)
if err != nil {
return nil, fmt.Errorf("failed to create task element: %w", err)
}
elems = append(elems, *elem)
}
task = batchtfile.NewBatchTGFileTask(taskID, f.ctx, elems, nil, true)
}
err = f.registerAndEnqueueTask(task, tasktype.TaskTypeTgfiles, req.Storage, req.Path, req.Webhook)
if err != nil {
return nil, err
}
return &CreateTaskResponse{
TaskID: taskID,
Type: tasktype.TaskTypeTgfiles,
Status: TaskStatusQueued,
CreatedAt: createdAt,
}, nil
}
// createTPHPicsTask 创建 Telegraph 图片下载任务
func (f *TaskFactory) createTPHPicsTask(taskID string, createdAt time.Time, req *CreateTaskRequest, stor storage.Storage) (*CreateTaskResponse, error) {
var params TPHPicsParams
if err := json.Unmarshal(req.Params, &params); err != nil {
return nil, fmt.Errorf("invalid params: %w", err)
}
if params.TelegraphURL == "" {
return nil, fmt.Errorf("no telegraph URL provided")
}
// 提取图片
pics, phPath, err := ExtractTelegraphImages(f.ctx, params.TelegraphURL)
if err != nil {
return nil, fmt.Errorf("failed to extract telegraph images: %w", err)
}
if len(pics) == 0 {
return nil, fmt.Errorf("no images found in telegraph page")
}
client := telegraph.NewClient()
task := tphtask.NewTask(taskID, f.ctx, phPath, pics, stor, req.Path, client, nil)
err = f.registerAndEnqueueTask(task, tasktype.TaskTypeTphpics, req.Storage, req.Path, req.Webhook)
if err != nil {
return nil, err
}
return &CreateTaskResponse{
TaskID: taskID,
Type: tasktype.TaskTypeTphpics,
Status: TaskStatusQueued,
CreatedAt: createdAt,
}, nil
}
// createTransferTask 创建存储间传输任务
func (f *TaskFactory) createTransferTask(taskID string, createdAt time.Time, req *CreateTaskRequest) (*CreateTaskResponse, error) {
var params TransferParams
if err := json.Unmarshal(req.Params, &params); err != nil {
return nil, fmt.Errorf("invalid params: %w", err)
}
// 验证源存储和目标存储
sourceStor, ok := storage.Storages[params.SourceStorage]
if !ok {
return nil, fmt.Errorf("source storage not found: %s", params.SourceStorage)
}
targetStor, ok := storage.Storages[params.TargetStorage]
if !ok {
return nil, fmt.Errorf("target storage not found: %s", params.TargetStorage)
}
// 检查源存储是否可读
sourceReadable, ok := sourceStor.(storage.StorageReadable)
if !ok {
return nil, fmt.Errorf("source storage does not support reading: %s", params.SourceStorage)
}
// 检查源存储是否可列
sourceListable, ok := sourceStor.(storage.StorageListable)
if !ok {
return nil, fmt.Errorf("source storage does not support listing: %s", params.SourceStorage)
}
// 列出源文件
files, err := sourceListable.ListFiles(f.ctx, params.SourcePath)
if err != nil {
return nil, fmt.Errorf("failed to list source files: %w", err)
}
if len(files) == 0 {
return nil, fmt.Errorf("no files found at source path: %s", params.SourcePath)
}
// 创建传输元素
elems := make([]transfer.TaskElement, 0, len(files))
for _, file := range files {
elem := transfer.NewTaskElement(sourceReadable, file, targetStor, params.TargetPath)
elems = append(elems, *elem)
}
task := transfer.NewTransferTask(taskID, f.ctx, elems, nil, true)
err = f.registerAndEnqueueTask(task, tasktype.TaskTypeTransfer, params.TargetStorage, params.TargetPath, req.Webhook)
if err != nil {
return nil, err
}
return &CreateTaskResponse{
TaskID: taskID,
Type: tasktype.TaskTypeTransfer,
Status: TaskStatusQueued,
CreatedAt: createdAt,
}, nil
}

View File

@@ -1,241 +1,629 @@
package api
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"math"
"net/http"
"net/url"
"path"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/celestix/gotgproto/ext"
"github.com/charmbracelet/log"
"github.com/gotd/td/tg"
"github.com/krau/SaveAny-Bot/client/bot"
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/ruleutil"
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
"github.com/krau/SaveAny-Bot/config"
"github.com/krau/SaveAny-Bot/core"
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
tftask "github.com/krau/SaveAny-Bot/core/tasks/tfile"
"github.com/krau/SaveAny-Bot/database"
"github.com/krau/SaveAny-Bot/pkg/queue"
"github.com/krau/SaveAny-Bot/pkg/tfile"
"github.com/krau/SaveAny-Bot/storage"
"github.com/rs/xid"
)
// Handlers 处理器结构体
type Handlers struct {
factory *TaskFactory
// Request/Response types
type CreateTaskRequest struct {
TelegramURL string `json:"telegram_url"`
StorageName string `json:"storage_name,omitempty"`
DirPath string `json:"dir_path,omitempty"`
UserID int64 `json:"user_id"`
}
// NewHandlers 创建处理器
func NewHandlers(factory *TaskFactory) *Handlers {
return &Handlers{factory: factory}
type CreateTaskResponse struct {
TaskID string `json:"task_id"`
Message string `json:"message"`
}
// CreateTaskHandler 创建任务处理器
func (h *Handlers) CreateTaskHandler(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
WriteError(w, http.StatusMethodNotAllowed, "method_not_allowed", "only POST method is allowed")
return
}
type TaskStatusResponse struct {
TaskID string `json:"task_id"`
Status string `json:"status"` // queued, running, completed, failed, canceled
Title string `json:"title"`
CreatedAt time.Time `json:"created_at"`
Error string `json:"error,omitempty"`
Downloaded int64 `json:"downloaded,omitempty"` // Bytes downloaded
Total int64 `json:"total,omitempty"` // Total bytes
ProgressPct float64 `json:"progress_pct,omitempty"` // Progress percentage (0-100)
}
type ListTasksResponse struct {
Queued []TaskInfo `json:"queued"`
Running []TaskInfo `json:"running"`
}
type TaskInfo struct {
ID string `json:"id"`
Title string `json:"title"`
}
type ErrorResponse struct {
Error string `json:"error"`
}
// Task tracking
var (
taskStatuses = make(map[string]*taskStatus)
taskStatusesMu sync.RWMutex
)
type taskStatus struct {
ID string
Status string
Title string
CreatedAt time.Time
Error string
Downloaded atomic.Int64 // Use atomic for lock-free updates
Total atomic.Int64 // Use atomic for lock-free updates
ProgressPct uint64 // Store as uint64 bits of float64 for atomic access
}
func handleHealth(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
json.NewEncoder(w).Encode(map[string]string{"status": "ok"})
}
func handleCreateTask(w http.ResponseWriter, r *http.Request) {
var req CreateTaskRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
WriteError(w, http.StatusBadRequest, "invalid_request", "failed to decode request body: "+err.Error())
respondError(w, "invalid request body", http.StatusBadRequest)
return
}
// 验证请求
if req.Type == "" {
WriteError(w, http.StatusBadRequest, "invalid_request", "task type is required")
// Validate request
if req.TelegramURL == "" {
respondError(w, "telegram_url is required", http.StatusBadRequest)
return
}
if req.UserID <= 0 {
respondError(w, "user_id is required and must be positive", http.StatusBadRequest)
return
}
if req.Storage == "" {
WriteError(w, http.StatusBadRequest, "invalid_request", "storage is required")
return
}
logger := log.FromContext(r.Context()).WithPrefix("api")
// 创建任务
resp, err := h.factory.CreateTask(&req)
// Get user from database
userDB, err := database.GetUserByChatID(r.Context(), req.UserID)
if err != nil {
WriteError(w, http.StatusBadRequest, "task_creation_failed", err.Error())
logger.Errorf("Failed to get user: %v", err)
respondError(w, "user not found", http.StatusBadRequest)
return
}
WriteJSON(w, http.StatusCreated, resp)
}
// Get storage
var stor storage.Storage
if req.StorageName != "" {
stor, err = storage.GetStorageByUserIDAndName(r.Context(), req.UserID, req.StorageName)
if err != nil {
logger.Errorf("Failed to get storage: %v", err)
respondError(w, "storage not found", http.StatusBadRequest)
return
}
} else {
// Use first available storage for the user
storages := storage.GetUserStorages(r.Context(), req.UserID)
if len(storages) == 0 {
respondError(w, "no storage available for user", http.StatusBadRequest)
return
}
stor = storages[0]
}
// ListTasksHandler 列出任务处理器
func (h *Handlers) ListTasksHandler(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodGet {
WriteError(w, http.StatusMethodNotAllowed, "method_not_allowed", "only GET method is allowed")
// Parse Telegram URL
botCtx := bot.ExtContext()
if botCtx == nil {
respondError(w, "bot not initialized", http.StatusInternalServerError)
return
}
tasks := GetAllTasks()
response := make([]TaskInfoResponse, 0, len(tasks))
for _, task := range tasks {
info := convertTaskProgressToResponse(task)
response = append(response, info)
}
WriteJSON(w, http.StatusOK, TasksListResponse{
Tasks: response,
Total: len(response),
})
}
// GetTaskHandler 获取单个任务处理器
func (h *Handlers) GetTaskHandler(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodGet {
WriteError(w, http.StatusMethodNotAllowed, "method_not_allowed", "only GET method is allowed")
linkUrl, err := url.Parse(req.TelegramURL)
if err != nil {
logger.Errorf("Failed to parse URL: %v", err)
respondError(w, "invalid telegram URL format", http.StatusBadRequest)
return
}
taskID := extractTaskIDFromPath(r.URL.Path)
if taskID == "" {
WriteError(w, http.StatusBadRequest, "invalid_request", "task ID is required")
chatID, msgID, err := tgutil.ParseMessageLink(botCtx, req.TelegramURL)
if err != nil {
logger.Errorf("Failed to parse Telegram URL: %v", err)
respondError(w, "invalid telegram URL format", http.StatusBadRequest)
return
}
task, ok := GetTask(taskID)
// Get message from Telegram
msg, err := tgutil.GetMessageByID(botCtx, chatID, msgID)
if err != nil {
logger.Errorf("Failed to get message: %v", err)
respondError(w, "failed to retrieve message", http.StatusBadRequest)
return
}
// Check if message has media
media, ok := msg.GetMedia()
if !ok {
WriteError(w, http.StatusNotFound, "task_not_found", "task not found: "+taskID)
respondError(w, "message has no media", http.StatusBadRequest)
return
}
resp := convertTaskProgressToResponse(task)
WriteJSON(w, http.StatusOK, resp)
}
// Collect files - handle both single and grouped messages
files := make([]tfile.TGFileMessage, 0)
// CancelTaskHandler 取消任务处理器
func (h *Handlers) CancelTaskHandler(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodDelete {
WriteError(w, http.StatusMethodNotAllowed, "method_not_allowed", "only DELETE method is allowed")
// Check for grouped messages (media group)
groupID, isGroup := msg.GetGroupedID()
if isGroup && groupID != 0 && !linkUrl.Query().Has("single") {
// Handle media group
gmsgs, err := tgutil.GetGroupedMessages(botCtx, chatID, msg)
if err != nil {
logger.Errorf("Failed to get grouped messages: %v", err)
// Fall back to single message
file, err := createTGFileWithMedia(botCtx, msg, media, userDB)
if err != nil {
logger.Errorf("Failed to create TGFile: %v", err)
respondError(w, "invalid message format", http.StatusBadRequest)
return
}
files = append(files, file)
} else {
// Process all messages in the group
for _, gmsg := range gmsgs {
if gmsg.Media == nil {
continue
}
gMedia, ok := gmsg.GetMedia()
if !ok {
continue
}
file, err := createTGFileWithMedia(botCtx, gmsg, gMedia, userDB)
if err != nil {
logger.Warnf("Failed to create TGFile for grouped message: %v", err)
continue
}
files = append(files, file)
}
}
} else {
// Single message
file, err := createTGFileWithMedia(botCtx, msg, media, userDB)
if err != nil {
logger.Errorf("Failed to create TGFile: %v", err)
respondError(w, "invalid message format", http.StatusBadRequest)
return
}
files = append(files, file)
}
if len(files) == 0 {
respondError(w, "no savable files found", http.StatusBadRequest)
return
}
taskID := extractTaskIDFromPath(r.URL.Path)
if taskID == "" {
WriteError(w, http.StatusBadRequest, "invalid_request", "task ID is required")
return
// Create tasks for all files with proper album handling
taskIDs := make([]string, 0, len(files))
baseDirPath := req.DirPath
if baseDirPath == "" {
baseDirPath = "/"
}
task, ok := GetTask(taskID)
if !ok {
WriteError(w, http.StatusNotFound, "task_not_found", "task not found: "+taskID)
return
// Create context with bot extension
injectCtx := tgutil.ExtWithContext(r.Context(), botCtx)
// Apply storage rules if enabled for the user
useRule := userDB.ApplyRule && userDB.Rules != nil
// Helper to apply rules to a file
applyRule := func(file tfile.TGFileMessage) (storage.Storage, ruleutil.MatchedDirPath) {
fileStor := stor
dirPath := ruleutil.MatchedDirPath(baseDirPath)
if useRule {
matched, matchedStorName, matchedDirPath := ruleutil.ApplyRule(injectCtx, userDB.Rules, ruleutil.NewInput(file))
if matched {
// Rule matched, apply overrides
if matchedDirPath != "" {
dirPath = matchedDirPath
}
if matchedStorName.Usable() {
var err error
fileStor, err = storage.GetStorageByUserIDAndName(injectCtx, userDB.ChatID, matchedStorName.String())
if err != nil {
logger.Errorf("Failed to get storage from rule: %v", err)
// Fall back to original storage
fileStor = stor
}
}
}
}
return fileStor, dirPath
}
// Cancel the task; the terminal status is set via the task event stream.
if err := core.CancelTask(r.Context(), taskID); err != nil {
WriteError(w, http.StatusInternalServerError, "cancel_failed", "failed to cancel task: "+err.Error())
return
// Separate files into regular and album files
type albumFile struct {
file tfile.TGFileMessage
storage storage.Storage
dirPath ruleutil.MatchedDirPath
}
albumFiles := make(map[int64][]albumFile)
for _, tgFile := range files {
fileStor, dirPath := applyRule(tgFile)
// Check if this needs album handling (NEW-FOR-ALBUM rule)
if dirPath.NeedNewForAlbum() {
groupId, isGroup := tgFile.Message().GetGroupedID()
if !isGroup || groupId == 0 {
logger.Warnf("File %s has NEW-FOR-ALBUM rule but is not in a group, treating as regular file", tgFile.Name())
// Treat as regular file with base dir path
storagePath := fileStor.JoinStoragePath(path.Join(baseDirPath, tgFile.Name()))
taskID := xid.New().String()
task, err := tftask.NewTGFileTask(taskID, injectCtx, tgFile, fileStor, storagePath, &apiProgressTracker{
taskID: taskID,
})
if err != nil {
logger.Errorf("Failed to create task: %v", err)
respondError(w, "failed to create task", http.StatusInternalServerError)
return
}
trackTask(taskID, task.Title(), "queued")
if err := core.AddTask(injectCtx, task); err != nil {
logger.Errorf("Failed to add task: %v", err)
updateTaskStatus(taskID, "failed", err.Error())
respondError(w, "failed to add task to queue", http.StatusInternalServerError)
return
}
taskIDs = append(taskIDs, taskID)
continue
}
// Group by album ID
if _, ok := albumFiles[groupId]; !ok {
albumFiles[groupId] = make([]albumFile, 0)
}
albumFiles[groupId] = append(albumFiles[groupId], albumFile{
file: tgFile,
storage: fileStor,
dirPath: dirPath,
})
} else {
// Regular file - create task immediately
storagePath := fileStor.JoinStoragePath(path.Join(dirPath.String(), tgFile.Name()))
taskID := xid.New().String()
task, err := tftask.NewTGFileTask(taskID, injectCtx, tgFile, fileStor, storagePath, &apiProgressTracker{
taskID: taskID,
})
if err != nil {
logger.Errorf("Failed to create task: %v", err)
respondError(w, "failed to create task", http.StatusInternalServerError)
return
}
trackTask(taskID, task.Title(), "queued")
if err := core.AddTask(injectCtx, task); err != nil {
logger.Errorf("Failed to add task: %v", err)
updateTaskStatus(taskID, "failed", err.Error())
respondError(w, "failed to add task to queue", http.StatusInternalServerError)
return
}
taskIDs = append(taskIDs, taskID)
}
}
task.UpdateStatus(TaskStatusCancelled)
WriteJSON(w, http.StatusOK, map[string]string{"message": "task cancelled successfully"})
}
// Handle album files - group them into subdirectories
for _, afiles := range albumFiles {
if len(afiles) <= 1 {
// Single file in album, treat as regular
for _, af := range afiles {
storagePath := af.storage.JoinStoragePath(path.Join(baseDirPath, af.file.Name()))
taskID := xid.New().String()
// ListStoragesHandler 列出存储处理器
func (h *Handlers) ListStoragesHandler(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodGet {
WriteError(w, http.StatusMethodNotAllowed, "method_not_allowed", "only GET method is allowed")
return
task, err := tftask.NewTGFileTask(taskID, injectCtx, af.file, af.storage, storagePath, &apiProgressTracker{
taskID: taskID,
})
if err != nil {
logger.Errorf("Failed to create task: %v", err)
respondError(w, "failed to create task", http.StatusInternalServerError)
return
}
trackTask(taskID, task.Title(), "queued")
if err := core.AddTask(injectCtx, task); err != nil {
logger.Errorf("Failed to add task: %v", err)
updateTaskStatus(taskID, "failed", err.Error())
respondError(w, "failed to add task to queue", http.StatusInternalServerError)
return
}
taskIDs = append(taskIDs, taskID)
}
continue
}
// Multiple files in album - create subdirectory named after first file
// Remove extension from first file's name to use as directory name
albumDir := strings.TrimSuffix(path.Base(afiles[0].file.Name()), path.Ext(afiles[0].file.Name()))
albumStor := afiles[0].storage
for _, af := range afiles {
// All files go into the album subdirectory
storagePath := albumStor.JoinStoragePath(path.Join(baseDirPath, albumDir, af.file.Name()))
taskID := xid.New().String()
task, err := tftask.NewTGFileTask(taskID, injectCtx, af.file, albumStor, storagePath, &apiProgressTracker{
taskID: taskID,
})
if err != nil {
logger.Errorf("Failed to create task for album file: %v", err)
respondError(w, "failed to create task", http.StatusInternalServerError)
return
}
trackTask(taskID, task.Title(), "queued")
if err := core.AddTask(injectCtx, task); err != nil {
logger.Errorf("Failed to add task: %v", err)
updateTaskStatus(taskID, "failed", err.Error())
respondError(w, "failed to add task to queue", http.StatusInternalServerError)
return
}
taskIDs = append(taskIDs, taskID)
}
}
storages := make([]StorageInfo, 0, len(storage.Storages))
for name, stor := range storage.Storages {
storages = append(storages, StorageInfo{
Name: name,
Type: string(stor.Type()),
// Send success response
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusCreated)
// Return first task ID for single file, or all task IDs for media group
if len(taskIDs) == 1 {
json.NewEncoder(w).Encode(CreateTaskResponse{
TaskID: taskIDs[0],
Message: "task created successfully",
})
} else {
json.NewEncoder(w).Encode(map[string]interface{}{
"task_ids": taskIDs,
"message": fmt.Sprintf("%d tasks created successfully", len(taskIDs)),
})
}
WriteJSON(w, http.StatusOK, StoragesResponse{Storages: storages})
}
// GetTaskTypesHandler 获取支持的任务类型
func (h *Handlers) GetTaskTypesHandler(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodGet {
WriteError(w, http.StatusMethodNotAllowed, "method_not_allowed", "only GET method is allowed")
// createTGFileWithMedia creates a TGFile with proper filename handling using user's strategy
func createTGFileWithMedia(botCtx *ext.Context, msg *tg.Message, media tg.MessageMediaClass, userDB *database.User) (tfile.TGFileMessage, error) {
// Use the same filename generation logic as bot handlers
opts := []tfile.TGFileOption{tfile.WithNameIfEmpty(tgutil.GenFileNameFromMessage(*msg))}
return tfile.FromMediaMessage(media, botCtx.Raw, msg, opts...)
}
func handleGetTask(w http.ResponseWriter, r *http.Request) {
taskID := r.PathValue("id")
if taskID == "" {
respondError(w, "task_id is required", http.StatusBadRequest)
return
}
types := []tasktype.TaskType{
tasktype.TaskTypeDirectlinks,
tasktype.TaskTypeYtdlp,
tasktype.TaskTypeAria2,
tasktype.TaskTypeParseditem,
tasktype.TaskTypeTgfiles,
tasktype.TaskTypeTphpics,
tasktype.TaskTypeTransfer,
taskStatusesMu.RLock()
status, exists := taskStatuses[taskID]
taskStatusesMu.RUnlock()
if !exists {
respondError(w, "task not found", http.StatusNotFound)
return
}
WriteJSON(w, http.StatusOK, map[string]any{
"types": types,
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(TaskStatusResponse{
TaskID: status.ID,
Status: status.Status,
Title: status.Title,
CreatedAt: status.CreatedAt,
Error: status.Error,
Downloaded: status.Downloaded.Load(),
Total: status.Total.Load(),
ProgressPct: math.Float64frombits(atomic.LoadUint64((*uint64)(&status.ProgressPct))),
})
}
// HealthCheckHandler 健康检查处理器
func (h *Handlers) HealthCheckHandler(w http.ResponseWriter, r *http.Request) {
WriteJSON(w, http.StatusOK, map[string]string{
"status": "ok",
})
}
func handleListTasks(w http.ResponseWriter, r *http.Request) {
queued := core.GetQueuedTasks(r.Context())
running := core.GetRunningTasks(r.Context())
// extractTaskIDFromPath 从路径中提取任务 ID
// 路径格式: /api/v1/tasks/:id
func extractTaskIDFromPath(path string) string {
parts := strings.Split(strings.Trim(path, "/"), "/")
if len(parts) < 4 {
return ""
response := ListTasksResponse{
Queued: convertTaskInfos(queued),
Running: convertTaskInfos(running),
}
return parts[3]
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(response)
}
// convertTaskProgressToResponse renders a task's current state, computing
// percent and speed from the snapshot taken under the task's mutex.
func convertTaskProgressToResponse(task *TaskProgressInfo) TaskInfoResponse {
status, total, downloaded, totalFiles, downloadedFiles, startedAt, errMsg, updatedAt := task.snapshot()
func handleCancelTask(w http.ResponseWriter, r *http.Request) {
taskID := r.PathValue("id")
if taskID == "" {
respondError(w, "task_id is required", http.StatusBadRequest)
return
}
resp := TaskInfoResponse{
TaskID: task.TaskID,
Type: tasktype.TaskType(task.Type),
if err := core.CancelTask(r.Context(), taskID); err != nil {
log.FromContext(r.Context()).Errorf("Failed to cancel task %s: %v", taskID, err)
respondError(w, "failed to cancel task", http.StatusInternalServerError)
return
}
updateTaskStatus(taskID, "canceled", "")
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(map[string]string{"message": "task canceled"})
}
// Helper functions
func respondError(w http.ResponseWriter, message string, statusCode int) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(statusCode)
json.NewEncoder(w).Encode(ErrorResponse{Error: message})
}
func trackTask(taskID, title, status string) {
taskStatusesMu.Lock()
defer taskStatusesMu.Unlock()
taskStatuses[taskID] = &taskStatus{
ID: taskID,
Status: status,
Title: task.Title,
Storage: task.Storage,
Path: task.Path,
Error: errMsg,
CreatedAt: task.CreatedAt,
UpdatedAt: updatedAt,
Title: title,
CreatedAt: time.Now(),
}
}
var percent float64
var speedMBPS float64
if total > 0 {
percent = float64(downloaded) * 100 / float64(total)
} else if totalFiles > 0 {
percent = float64(downloadedFiles) * 100 / float64(totalFiles)
func updateTaskStatus(taskID, status, errorMsg string) {
taskStatusesMu.Lock()
defer taskStatusesMu.Unlock()
if ts, exists := taskStatuses[taskID]; exists {
ts.Status = status
ts.Error = errorMsg
}
if !startedAt.IsZero() {
elapsed := time.Since(startedAt).Seconds()
if elapsed > 0 {
speedMBPS = float64(downloaded) / elapsed / (1024 * 1024)
}
func convertTaskInfos(tasks []queue.TaskInfo) []TaskInfo {
result := make([]TaskInfo, len(tasks))
for i, t := range tasks {
result[i] = TaskInfo{
ID: t.ID,
Title: t.Title,
}
}
return result
}
if total > 0 || totalFiles > 0 {
resp.Progress = &TaskProgress{
TotalBytes: total,
DownloadedBytes: downloaded,
TotalFiles: totalFiles,
DownloadedFiles: downloadedFiles,
Percent: percent,
SpeedMBPS: speedMBPS,
// apiProgressTracker implements tftask.ProgressTracker for API tasks
type apiProgressTracker struct {
taskID string
}
func (a *apiProgressTracker) OnStart(ctx context.Context, info tftask.TaskInfo) {
updateTaskStatus(a.taskID, "running", "")
}
func (a *apiProgressTracker) OnProgress(ctx context.Context, info tftask.TaskInfo, downloaded int64, total int64) {
// Use atomic operations to avoid mutex locks for better performance
// OnProgress is called very frequently during downloads
taskStatusesMu.RLock()
ts, exists := taskStatuses[a.taskID]
taskStatusesMu.RUnlock()
if exists {
ts.Downloaded.Store(downloaded)
ts.Total.Store(total)
if total > 0 {
progressPct := float64(downloaded) / float64(total) * 100.0
atomic.StoreUint64((*uint64)(&ts.ProgressPct), math.Float64bits(progressPct))
}
}
return resp
}
// NotFoundHandler 404 处理器
func NotFoundHandler(w http.ResponseWriter, r *http.Request) {
WriteError(w, http.StatusNotFound, "not_found", "endpoint not found: "+r.URL.Path)
func (a *apiProgressTracker) OnDone(ctx context.Context, info tftask.TaskInfo, err error) {
if err != nil {
updateTaskStatus(a.taskID, "failed", err.Error())
sendWebhook(a.taskID, "failed", err.Error())
} else {
updateTaskStatus(a.taskID, "completed", "")
sendWebhook(a.taskID, "completed", "")
}
}
// MethodNotAllowedHandler 405 处理器
func MethodNotAllowedHandler(w http.ResponseWriter, r *http.Request) {
WriteError(w, http.StatusMethodNotAllowed, "method_not_allowed", "method not allowed: "+r.Method)
// sendWebhook sends a callback to the configured webhook URL
func sendWebhook(taskID, status, errorMsg string) {
cfg := config.C()
if cfg.API.WebhookURL == "" {
return
}
taskStatusesMu.RLock()
ts, exists := taskStatuses[taskID]
taskStatusesMu.RUnlock()
if !exists {
return
}
logger := log.WithPrefix("webhook")
payload := TaskStatusResponse{
TaskID: ts.ID,
Status: status,
Title: ts.Title,
CreatedAt: ts.CreatedAt,
Error: errorMsg,
Downloaded: ts.Downloaded.Load(),
Total: ts.Total.Load(),
ProgressPct: math.Float64frombits(atomic.LoadUint64((*uint64)(&ts.ProgressPct))),
}
body, err := json.Marshal(payload)
if err != nil {
logger.Errorf("Failed to marshal webhook payload: %v", err)
return
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
req, err := http.NewRequestWithContext(ctx, "POST", cfg.API.WebhookURL, bytes.NewReader(body))
if err != nil {
logger.Errorf("Failed to create webhook request: %v", err)
return
}
req.Header.Set("Content-Type", "application/json")
if cfg.API.Token != "" {
req.Header.Set("Authorization", "Bearer "+cfg.API.Token)
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
logger.Errorf("Failed to send webhook: %v", err)
return
}
defer resp.Body.Close()
if resp.StatusCode >= 400 {
body, err := io.ReadAll(resp.Body)
if err != nil {
logger.Errorf("Webhook returned error status %d, failed to read response body: %v", resp.StatusCode, err)
} else {
logger.Errorf("Webhook returned error status %d: %s", resp.StatusCode, string(body))
}
}
}

View File

@@ -1,695 +0,0 @@
package api
import (
"bytes"
"context"
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
"time"
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
"github.com/krau/SaveAny-Bot/pkg/taskevent"
)
// setupTestServer creates a test server with handlers
func setupTestServer(t *testing.T) (*Handlers, *TaskFactory) {
factory := NewTaskFactory(t.Context())
handlers := NewHandlers(factory)
return handlers, factory
}
// TestCreateTaskHandler tests the create task endpoint
func TestCreateTaskHandler(t *testing.T) {
handlers, _ := setupTestServer(t)
tests := []struct {
name string
method string
body any
wantStatus int
wantErr bool
}{
{
name: "Method not allowed",
method: http.MethodGet,
body: nil,
wantStatus: http.StatusMethodNotAllowed,
wantErr: true,
},
{
name: "Invalid JSON body",
method: http.MethodPost,
body: "invalid json",
wantStatus: http.StatusBadRequest,
wantErr: true,
},
{
name: "Empty request body",
method: http.MethodPost,
body: CreateTaskRequest{},
// Will fail validation for missing type
wantStatus: http.StatusBadRequest,
wantErr: true,
},
{
name: "Missing type",
method: http.MethodPost,
body: CreateTaskRequest{
Storage: "test-storage",
Path: "downloads",
},
wantStatus: http.StatusBadRequest,
wantErr: true,
},
{
name: "Missing storage",
method: http.MethodPost,
body: CreateTaskRequest{
Type: tasktype.TaskTypeDirectlinks,
Path: "downloads",
},
wantStatus: http.StatusBadRequest,
wantErr: true,
},
{
name: "Storage not found",
method: http.MethodPost,
body: CreateTaskRequest{
Type: tasktype.TaskTypeDirectlinks,
Storage: "non-existent-storage",
Path: "downloads",
Params: json.RawMessage(`{"urls":["https://example.com/file.zip"]}`),
},
wantStatus: http.StatusBadRequest,
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var bodyBytes []byte
var err error
if tt.body != nil {
switch v := tt.body.(type) {
case string:
bodyBytes = []byte(v)
default:
bodyBytes, err = json.Marshal(tt.body)
if err != nil {
t.Fatalf("failed to marshal body: %v", err)
}
}
}
req := httptest.NewRequest(tt.method, "/api/v1/tasks", bytes.NewReader(bodyBytes))
if tt.body != nil {
req.Header.Set("Content-Type", "application/json")
}
rr := httptest.NewRecorder()
handlers.CreateTaskHandler(rr, req)
if rr.Code != tt.wantStatus {
t.Errorf("expected status %d, got %d", tt.wantStatus, rr.Code)
}
if tt.wantErr {
var errResp ErrorResponse
if err := json.Unmarshal(rr.Body.Bytes(), &errResp); err != nil {
t.Errorf("expected error response, got: %s", rr.Body.String())
}
}
})
}
}
// TestListTasksHandler tests the list tasks endpoint
func TestListTasksHandler(t *testing.T) {
handlers, _ := setupTestServer(t)
tests := []struct {
name string
method string
wantStatus int
}{
{
name: "Method not allowed",
method: http.MethodPost,
wantStatus: http.StatusMethodNotAllowed,
},
{
name: "Success",
method: http.MethodGet,
wantStatus: http.StatusOK,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
req := httptest.NewRequest(tt.method, "/api/v1/tasks", nil)
rr := httptest.NewRecorder()
handlers.ListTasksHandler(rr, req)
if rr.Code != tt.wantStatus {
t.Errorf("expected status %d, got %d", tt.wantStatus, rr.Code)
}
if tt.method == http.MethodGet {
var resp TasksListResponse
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
t.Errorf("failed to unmarshal response: %v", err)
}
if resp.Tasks == nil {
t.Error("expected non-nil tasks array")
}
}
})
}
}
// TestGetTaskHandler tests the get task endpoint
func TestGetTaskHandler(t *testing.T) {
handlers, _ := setupTestServer(t)
// Register a test task
testTaskID := "test-get-task"
RegisterTask(testTaskID, "directlinks", "local", "downloads", "Test", "")
defer DeleteTask(testTaskID)
tests := []struct {
name string
method string
path string
wantStatus int
wantFound bool
}{
{
name: "Method not allowed",
method: http.MethodPost,
path: "/api/v1/tasks/test-id",
wantStatus: http.StatusMethodNotAllowed,
},
{
name: "Missing task ID",
method: http.MethodGet,
path: "/api/v1/tasks",
wantStatus: http.StatusBadRequest,
},
{
name: "Task not found",
method: http.MethodGet,
path: "/api/v1/tasks/non-existent-task",
wantStatus: http.StatusNotFound,
},
{
name: "Task found",
method: http.MethodGet,
path: "/api/v1/tasks/" + testTaskID,
wantStatus: http.StatusOK,
wantFound: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
req := httptest.NewRequest(tt.method, tt.path, nil)
rr := httptest.NewRecorder()
handlers.GetTaskHandler(rr, req)
if rr.Code != tt.wantStatus {
t.Errorf("expected status %d, got %d", tt.wantStatus, rr.Code)
}
if tt.wantFound {
var resp TaskInfoResponse
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
t.Errorf("failed to unmarshal response: %v", err)
}
if resp.TaskID != testTaskID {
t.Errorf("expected task ID %s, got %s", testTaskID, resp.TaskID)
}
}
})
}
}
// TestCancelTaskHandler tests the cancel task endpoint
func TestCancelTaskHandler(t *testing.T) {
handlers, _ := setupTestServer(t)
// Register a test task
testTaskID := "test-cancel-task"
RegisterTask(testTaskID, "directlinks", "local", "downloads", "Test", "")
defer DeleteTask(testTaskID)
tests := []struct {
name string
method string
path string
wantStatus int
skipCore bool // Skip if core is not initialized
}{
{
name: "Method not allowed",
method: http.MethodGet,
path: "/api/v1/tasks/test-id",
wantStatus: http.StatusMethodNotAllowed,
},
{
name: "Missing task ID",
method: http.MethodDelete,
path: "/api/v1/tasks",
wantStatus: http.StatusBadRequest,
},
{
name: "Task not found",
method: http.MethodDelete,
path: "/api/v1/tasks/non-existent-task",
wantStatus: http.StatusNotFound,
},
{
name: "Cancel task",
method: http.MethodDelete,
path: "/api/v1/tasks/" + testTaskID,
wantStatus: http.StatusOK,
skipCore: true, // Requires initialized core task queue
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if tt.skipCore {
t.Skip("Skipping test: requires initialized core")
}
req := httptest.NewRequest(tt.method, tt.path, nil)
rr := httptest.NewRecorder()
handlers.CancelTaskHandler(rr, req)
if rr.Code != tt.wantStatus {
t.Errorf("expected status %d, got %d", tt.wantStatus, rr.Code)
}
})
}
}
// TestListStoragesHandler tests the list storages endpoint
func TestListStoragesHandler(t *testing.T) {
handlers, _ := setupTestServer(t)
tests := []struct {
name string
method string
wantStatus int
}{
{
name: "Method not allowed",
method: http.MethodPost,
wantStatus: http.StatusMethodNotAllowed,
},
{
name: "Success",
method: http.MethodGet,
wantStatus: http.StatusOK,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
req := httptest.NewRequest(tt.method, "/api/v1/storages", nil)
rr := httptest.NewRecorder()
handlers.ListStoragesHandler(rr, req)
if rr.Code != tt.wantStatus {
t.Errorf("expected status %d, got %d", tt.wantStatus, rr.Code)
}
if tt.method == http.MethodGet {
var resp StoragesResponse
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
t.Errorf("failed to unmarshal response: %v", err)
}
if resp.Storages == nil {
t.Error("expected non-nil storages array")
}
}
})
}
}
// TestConcurrentProgressStore tests concurrent access to progress store
func TestConcurrentProgressStore(t *testing.T) {
// Clear store before test
t.Cleanup(func() {
tasks := GetAllTasks()
for _, task := range tasks {
if strings.HasPrefix(task.TaskID, "concurrent-test-") {
DeleteTask(task.TaskID)
}
}
})
var wg sync.WaitGroup
numGoroutines := 100
// Concurrent registrations
for i := range numGoroutines {
wg.Add(1)
go func(id int) {
defer wg.Done()
taskID := fmt.Sprintf("concurrent-test-%d", id)
RegisterTask(taskID, "directlinks", "local", "downloads", "Test", "")
}(i)
}
// Concurrent reads
for i := range numGoroutines {
wg.Add(1)
go func(id int) {
defer wg.Done()
taskID := fmt.Sprintf("concurrent-test-%d", id)
GetTask(taskID)
}(i)
}
// Concurrent updates
for i := range numGoroutines {
wg.Add(1)
go func(id int) {
defer wg.Done()
taskID := fmt.Sprintf("concurrent-test-%d", id)
info, ok := GetTask(taskID)
if ok {
info.UpdateStatus(TaskStatusRunning)
}
}(i)
}
wg.Wait()
// Verify all tasks exist
for i := range numGoroutines {
taskID := fmt.Sprintf("concurrent-test-%d", i)
if _, ok := GetTask(taskID); !ok {
t.Errorf("task %s not found after concurrent operations", taskID)
}
}
}
// TestProgressTrackerConcurrentUpdates tests concurrent progress updates
func TestProgressTrackerConcurrentUpdates(t *testing.T) {
info := RegisterTask("concurrent-progress", "directlinks", "local", "downloads", "Test", "")
info.Emit(taskevent.Event{TaskID: "concurrent-progress", Phase: taskevent.PhaseStart, TotalBytes: 10000})
var wg sync.WaitGroup
numGoroutines := 50
updatesPerGoroutine := 100
// Concurrent progress updates via the Sink interface
for i := range numGoroutines {
wg.Add(1)
go func(id int) {
defer wg.Done()
for j := range updatesPerGoroutine {
info.Emit(taskevent.Event{
TaskID: "concurrent-progress",
Phase: taskevent.PhaseProgress,
DownloadedBytes: int64(id*updatesPerGoroutine + j),
TotalBytes: 10000,
})
}
}(i)
}
wg.Wait()
status, _, downloaded, _, _, _, _, _ := info.snapshot()
if status != TaskStatusRunning {
t.Errorf("expected status Running after concurrent updates, got %s", status)
}
if downloaded <= 0 {
t.Errorf("expected downloaded bytes > 0 after concurrent updates, got %d", downloaded)
}
}
// TestTaskFactoryValidation tests TaskFactory parameter validation
func TestTaskFactoryValidation(t *testing.T) {
factory := NewTaskFactory(context.Background())
tests := []struct {
name string
request *CreateTaskRequest
wantErr bool
errMsg string
}{
{
name: "Storage not found",
request: &CreateTaskRequest{
Type: tasktype.TaskTypeDirectlinks,
Storage: "non-existent",
Path: "downloads",
Params: json.RawMessage(`{"urls":["https://example.com/file.zip"]}`),
},
wantErr: true,
errMsg: "storage not found",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
_, err := factory.CreateTask(tt.request)
if (err != nil) != tt.wantErr {
t.Errorf("CreateTask() error = %v, wantErr %v", err, tt.wantErr)
return
}
if err != nil && tt.errMsg != "" {
if !strings.Contains(strings.ToLower(err.Error()), tt.errMsg) {
t.Errorf("error message %q does not contain %q", err.Error(), tt.errMsg)
}
}
})
}
}
// TestEdgeCases tests various edge cases
func TestEdgeCases(t *testing.T) {
tests := []struct {
name string
fn func(t *testing.T)
}{
{
name: "Empty request body",
fn: func(t *testing.T) {
handlers, _ := setupTestServer(t)
req := httptest.NewRequest(http.MethodPost, "/api/v1/tasks", nil)
rr := httptest.NewRecorder()
handlers.CreateTaskHandler(rr, req)
if rr.Code != http.StatusBadRequest {
t.Errorf("expected %d, got %d", http.StatusBadRequest, rr.Code)
}
},
},
{
name: "Very long task ID in path",
fn: func(t *testing.T) {
handlers, _ := setupTestServer(t)
longID := strings.Repeat("a", 1000)
req := httptest.NewRequest(http.MethodGet, "/api/v1/tasks/"+longID, nil)
rr := httptest.NewRecorder()
handlers.GetTaskHandler(rr, req)
if rr.Code != http.StatusNotFound {
t.Errorf("expected %d, got %d", http.StatusNotFound, rr.Code)
}
},
},
{
name: "Path with special characters",
fn: func(t *testing.T) {
path := "/api/v1/tasks/test%20id/with/slashes"
got := extractTaskIDFromPath(path)
expected := "test%20id"
if got != expected {
t.Errorf("expected %q, got %q", expected, got)
}
},
},
{
name: "Double slashes in path",
fn: func(t *testing.T) {
path := "/api/v1/tasks//task-id"
got := extractTaskIDFromPath(path)
expected := ""
if got != expected {
t.Errorf("expected %q, got %q", expected, got)
}
},
},
{
name: "Progress tracker with empty webhook",
fn: func(t *testing.T) {
info := RegisterTask("test-empty-webhook", "type", "storage", "path", "title", "")
if info.Webhook != "" {
t.Error("expected empty webhook")
}
},
},
}
for _, tt := range tests {
t.Run(tt.name, tt.fn)
}
}
// TestHealthCheckHandler tests the health check endpoint
func TestHealthCheckHandler(t *testing.T) {
handlers, _ := setupTestServer(t)
req := httptest.NewRequest(http.MethodGet, "/health", nil)
rr := httptest.NewRecorder()
handlers.HealthCheckHandler(rr, req)
if rr.Code != http.StatusOK {
t.Errorf("expected status %d, got %d", http.StatusOK, rr.Code)
}
var resp map[string]string
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
t.Fatalf("failed to unmarshal response: %v", err)
}
if resp["status"] != "ok" {
t.Errorf("expected status 'ok', got %q", resp["status"])
}
}
// TestGetTaskTypesHandler tests the task types endpoint
func TestGetTaskTypesHandler(t *testing.T) {
handlers, _ := setupTestServer(t)
tests := []struct {
name string
method string
wantStatus int
}{
{
name: "Method not allowed",
method: http.MethodPost,
wantStatus: http.StatusMethodNotAllowed,
},
{
name: "Success",
method: http.MethodGet,
wantStatus: http.StatusOK,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
req := httptest.NewRequest(tt.method, "/api/v1/task-types", nil)
rr := httptest.NewRecorder()
handlers.GetTaskTypesHandler(rr, req)
if rr.Code != tt.wantStatus {
t.Errorf("expected status %d, got %d", tt.wantStatus, rr.Code)
}
if tt.method == http.MethodGet {
var resp map[string]any
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
t.Errorf("failed to unmarshal response: %v", err)
}
if _, ok := resp["types"]; !ok {
t.Error("expected 'types' field in response")
}
}
})
}
}
// TestNotFoundHandler tests the 404 handler
func TestNotFoundHandler(t *testing.T) {
req := httptest.NewRequest(http.MethodGet, "/non-existent-path", nil)
rr := httptest.NewRecorder()
NotFoundHandler(rr, req)
if rr.Code != http.StatusNotFound {
t.Errorf("expected status %d, got %d", http.StatusNotFound, rr.Code)
}
var resp ErrorResponse
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
t.Fatalf("failed to unmarshal response: %v", err)
}
if resp.Error != "not_found" {
t.Errorf("expected error 'not_found', got %q", resp.Error)
}
}
// TestMethodNotAllowedHandler tests the 405 handler
func TestMethodNotAllowedHandler(t *testing.T) {
req := httptest.NewRequest(http.MethodPost, "/api/v1/tasks", nil)
rr := httptest.NewRecorder()
MethodNotAllowedHandler(rr, req)
if rr.Code != http.StatusMethodNotAllowed {
t.Errorf("expected status %d, got %d", http.StatusMethodNotAllowed, rr.Code)
}
var resp ErrorResponse
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
t.Fatalf("failed to unmarshal response: %v", err)
}
if resp.Error != "method_not_allowed" {
t.Errorf("expected error 'method_not_allowed', got %q", resp.Error)
}
}
// TestTaskProgressInfoTimeUpdate tests that timestamps are updated correctly
func TestTaskProgressInfoTimeUpdate(t *testing.T) {
info := RegisterTask("time-test", "directlinks", "local", "downloads", "Test", "")
defer DeleteTask("time-test")
originalTime := info.UpdatedAt
time.Sleep(10 * time.Millisecond) // Ensure time difference
info.UpdateStatus(TaskStatusRunning)
if !info.UpdatedAt.After(originalTime) {
t.Error("expected UpdatedAt to be updated")
}
}
// TestWebhookPayloadWithNilCompletedAt tests webhook payload with nil completed_at
func TestWebhookPayloadWithNilCompletedAt(t *testing.T) {
payload := WebhookPayload{
TaskID: "test-id",
Type: "directlinks",
Status: TaskStatusRunning,
Storage: "local",
Path: "downloads/file.zip",
CompletedAt: nil,
Error: "",
}
data, err := json.Marshal(payload)
if err != nil {
t.Fatalf("failed to marshal: %v", err)
}
var decoded map[string]any
if err := json.Unmarshal(data, &decoded); err != nil {
t.Fatalf("failed to unmarshal: %v", err)
}
// completed_at should be omitted when nil
if _, ok := decoded["completed_at"]; ok {
t.Error("expected completed_at to be omitted when nil")
}
}

66
api/middleware.go Normal file
View File

@@ -0,0 +1,66 @@
package api
import (
"net/http"
"strings"
"time"
"github.com/charmbracelet/log"
"github.com/krau/SaveAny-Bot/config"
)
// authMiddleware validates API token
func authMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Skip auth for health check
if r.URL.Path == "/health" {
next.ServeHTTP(w, r)
return
}
cfg := config.C()
// Check token if configured
if cfg.API.Token != "" {
authHeader := r.Header.Get("Authorization")
if authHeader == "" {
http.Error(w, `{"error":"unauthorized: missing token"}`, http.StatusUnauthorized)
return
}
token := strings.TrimPrefix(authHeader, "Bearer ")
if token != cfg.API.Token {
http.Error(w, `{"error":"unauthorized: invalid token"}`, http.StatusUnauthorized)
return
}
}
next.ServeHTTP(w, r)
})
}
// loggingMiddleware logs HTTP requests
func loggingMiddleware(logger *log.Logger) func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
start := time.Now()
// Wrap response writer to capture status code
wrapper := &responseWriter{ResponseWriter: w, statusCode: http.StatusOK}
next.ServeHTTP(wrapper, r)
logger.Infof("%s %s %d %s", r.Method, r.URL.Path, wrapper.statusCode, time.Since(start))
})
}
}
type responseWriter struct {
http.ResponseWriter
statusCode int
}
func (rw *responseWriter) WriteHeader(code int) {
rw.statusCode = code
rw.ResponseWriter.WriteHeader(code)
}

47
api/middleware_test.go Normal file
View File

@@ -0,0 +1,47 @@
package api
import (
"net/http"
"net/http/httptest"
"testing"
)
func TestAuthMiddleware_NoAuth(t *testing.T) {
// Create a test handler
handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
w.Write([]byte("OK"))
})
// Create request
req := httptest.NewRequest("GET", "/api/v1/tasks", nil)
rec := httptest.NewRecorder()
// Apply middleware
authMiddleware(handler).ServeHTTP(rec, req)
// When no token is configured, request should succeed or be unauthorized
if rec.Code != http.StatusOK && rec.Code != http.StatusUnauthorized {
t.Errorf("Expected status 200 or 401, got %d", rec.Code)
}
}
func TestAuthMiddleware_HealthCheck(t *testing.T) {
// Create a test handler
handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
w.Write([]byte("OK"))
})
// Create request to health endpoint
req := httptest.NewRequest("GET", "/health", nil)
rec := httptest.NewRecorder()
// Apply middleware
authMiddleware(handler).ServeHTTP(rec, req)
// Health check should always work without auth
if rec.Code != http.StatusOK {
t.Errorf("Expected status 200 for health check, got %d", rec.Code)
}
}

View File

@@ -1,216 +0,0 @@
package api
import (
"sync"
"time"
"github.com/krau/SaveAny-Bot/pkg/taskevent"
)
// TaskProgressInfo stores the progress of an API-submitted task. All fields are
// guarded by mu. It implements taskevent.Sink so the task layer can update it
// without knowing about the API.
type TaskProgressInfo struct {
mu sync.Mutex
TaskID string
Type string
Status TaskStatus
Title string
TotalBytes int64
DownloadedBytes int64
TotalFiles int
DownloadedFiles int
Storage string
Path string
Error string
CreatedAt time.Time
UpdatedAt time.Time
StartedAt time.Time
Webhook string
webhookNotified bool
}
// progressStore holds all API tasks. Entries are removed a fixed duration after
// they reach a terminal state to bound memory usage.
type progressStore struct {
mu sync.RWMutex
tasks map[string]*TaskProgressInfo
retention time.Duration
}
var store = &progressStore{
tasks: make(map[string]*TaskProgressInfo),
retention: 24 * time.Hour,
}
// RegisterTask registers a new API task and returns its progress info.
func RegisterTask(taskID, taskType, storage, path, title, webhook string) *TaskProgressInfo {
info := &TaskProgressInfo{
TaskID: taskID,
Type: taskType,
Status: TaskStatusQueued,
Title: title,
Storage: storage,
Path: path,
CreatedAt: time.Now(),
UpdatedAt: time.Now(),
Webhook: webhook,
}
store.mu.Lock()
store.tasks[taskID] = info
store.mu.Unlock()
return info
}
// GetTask returns the progress info for a task.
func GetTask(taskID string) (*TaskProgressInfo, bool) {
store.mu.RLock()
defer store.mu.RUnlock()
info, ok := store.tasks[taskID]
return info, ok
}
// GetAllTasks returns all tracked tasks.
func GetAllTasks() []*TaskProgressInfo {
store.mu.RLock()
defer store.mu.RUnlock()
tasks := make([]*TaskProgressInfo, 0, len(store.tasks))
for _, info := range store.tasks {
tasks = append(tasks, info)
}
return tasks
}
// DeleteTask removes a task record.
func DeleteTask(taskID string) {
store.mu.Lock()
defer store.mu.Unlock()
delete(store.tasks, taskID)
}
// CleanupExpired removes tasks that reached a terminal state more than the
// store's retention duration ago. It is safe to call periodically.
func CleanupExpired() {
now := time.Now()
store.mu.Lock()
defer store.mu.Unlock()
for id, info := range store.tasks {
info.mu.Lock()
terminal := info.Status == TaskStatusCompleted || info.Status == TaskStatusFailed || info.Status == TaskStatusCancelled
stale := terminal && now.Sub(info.UpdatedAt) > store.retention
info.mu.Unlock()
if stale {
delete(store.tasks, id)
}
}
}
// StartCleanupLoop runs CleanupExpired on a fixed interval until ctx is done.
// It should be started once during API server initialization.
func StartCleanupLoop(ctx interface{ Done() <-chan struct{} }) {
go func() {
ticker := time.NewTicker(10 * time.Minute)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
CleanupExpired()
}
}
}()
}
// UpdateStatus sets the task status.
func (t *TaskProgressInfo) UpdateStatus(status TaskStatus) {
t.mu.Lock()
t.Status = status
t.UpdatedAt = time.Now()
if status == TaskStatusRunning && t.StartedAt.IsZero() {
t.StartedAt = t.UpdatedAt
}
t.mu.Unlock()
}
// SetError marks the task failed with an error message.
func (t *TaskProgressInfo) SetError(err string) {
t.mu.Lock()
t.Error = err
t.Status = TaskStatusFailed
t.UpdatedAt = time.Now()
t.mu.Unlock()
}
// snapshot returns a point-in-time copy of the fields needed to render a
// response, so callers never touch the mutex directly.
func (t *TaskProgressInfo) snapshot() (status TaskStatus, total, downloaded int64, totalFiles, downloadedFiles int, startedAt time.Time, err string, updatedAt time.Time) {
t.mu.Lock()
defer t.mu.Unlock()
return t.Status, t.TotalBytes, t.DownloadedBytes, t.TotalFiles, t.DownloadedFiles, t.StartedAt, t.Error, t.UpdatedAt
}
// Emit implements taskevent.Sink. It translates task lifecycle events into
// status/progress updates and fires the webhook on terminal transitions.
func (t *TaskProgressInfo) Emit(e taskevent.Event) {
t.mu.Lock()
switch e.Phase {
case taskevent.PhaseStart:
t.Status = TaskStatusRunning
if t.StartedAt.IsZero() {
t.StartedAt = time.Now()
}
if e.TotalBytes > 0 {
t.TotalBytes = e.TotalBytes
}
case taskevent.PhaseProgress:
t.Status = TaskStatusRunning
if e.TotalBytes > 0 {
t.TotalBytes = e.TotalBytes
}
t.DownloadedBytes = e.DownloadedBytes
if e.TotalFiles > 0 {
t.TotalFiles = e.TotalFiles
}
if e.DownloadedFiles > 0 {
t.DownloadedFiles = e.DownloadedFiles
}
case taskevent.PhaseDone:
if e.Err != nil {
t.Status = TaskStatusFailed
t.Error = e.Err.Error()
} else {
t.Status = TaskStatusCompleted
}
}
t.UpdatedAt = time.Now()
notify := t.Webhook != "" && !t.webhookNotified && (t.Status == TaskStatusCompleted || t.Status == TaskStatusFailed)
if notify {
t.webhookNotified = true
}
t.mu.Unlock()
if notify {
payload := CreateWebhookPayload(t.TaskID, t.Type, t.Status, t.Storage, t.Path, e.Err)
SendWebhook(nil, payload)
}
}
// ProgressTracker is retained for compatibility but is no longer the primary
// progress path; taskevent drives updates now. These methods are safe no-ops
// when called on a nil receiver.
type ProgressTracker struct{}
func NewProgressTracker(taskID, taskType, storage, path, title, webhook string) *ProgressTracker {
return &ProgressTracker{}
}
func (p *ProgressTracker) OnStart(totalBytes int64, totalFiles int) {}
func (p *ProgressTracker) OnProgress(downloadedBytes int64, downloadedFiles int) {}
func (p *ProgressTracker) OnDone(err error) {}
func (p *ProgressTracker) GetInfo() *TaskProgressInfo { return nil }
func (p *ProgressTracker) UpdateProgressBytes(bytes int64) {}
func (p *ProgressTracker) UpdateProgressFiles(files int) {}

View File

@@ -10,161 +10,68 @@ import (
"github.com/krau/SaveAny-Bot/config"
)
// Server API 服务器
type Server struct {
httpServer *http.Server
factory *TaskFactory
}
var server *http.Server
// NewServer 创建新的 API 服务器
func NewServer(ctx context.Context) *Server {
cfg := config.C().API
// Init initializes and starts the HTTP API server
func Init(ctx context.Context) error {
cfg := config.C()
if !cfg.API.Enable {
return nil
}
factory := NewTaskFactory(ctx)
handlers := NewHandlers(factory)
// Validate that token is configured when API is enabled
if cfg.API.Token == "" {
return fmt.Errorf("API is enabled but token is not configured. Please set 'api.token' in your configuration file for security")
}
logger := log.FromContext(ctx).WithPrefix("api")
// 设置路由
mux := http.NewServeMux()
// 健康检查
mux.HandleFunc("/health", handlers.HealthCheckHandler)
// Register API routes
registerRoutes(mux)
// API v1 路由
mux.HandleFunc("/api/v1/tasks", func(w http.ResponseWriter, r *http.Request) {
switch r.Method {
case http.MethodGet:
handlers.ListTasksHandler(w, r)
case http.MethodPost:
handlers.CreateTaskHandler(w, r)
default:
MethodNotAllowedHandler(w, r)
}
})
mux.HandleFunc("/api/v1/tasks/", func(w http.ResponseWriter, r *http.Request) {
// 根据方法和路径分发
switch r.Method {
case http.MethodGet:
handlers.GetTaskHandler(w, r)
case http.MethodDelete:
handlers.CancelTaskHandler(w, r)
default:
MethodNotAllowedHandler(w, r)
}
})
mux.HandleFunc("/api/v1/storages", handlers.ListStoragesHandler)
mux.HandleFunc("/api/v1/task-types", handlers.GetTaskTypesHandler)
// Wrap with middleware
handler := loggingMiddleware(logger)(authMiddleware(mux))
// 404 处理
mux.HandleFunc("/", NotFoundHandler)
// Apply middleware chain.
var handler http.Handler = mux
// Apply auth middleware when a token is configured.
token := cfg.Token
if token != "" {
handler = AuthMiddleware()(handler)
server = &http.Server{
Addr: fmt.Sprintf(":%d", cfg.API.Port),
Handler: handler,
ReadTimeout: 15 * time.Second,
WriteTimeout: 15 * time.Second,
IdleTimeout: 60 * time.Second,
}
// Add logging middleware.
handler = loggingMiddleware(handler)
// Add recovery middleware.
handler = recoveryMiddleware(handler)
return &Server{
httpServer: &http.Server{
Addr: fmt.Sprintf("%s:%d", cfg.Host, cfg.Port),
Handler: handler,
ReadTimeout: 30 * time.Second,
WriteTimeout: 30 * time.Second,
IdleTimeout: 120 * time.Second,
},
factory: factory,
}
}
// Start 启动服务器
func (s *Server) Start(ctx context.Context) error {
logger := log.FromContext(ctx).With("module", "api")
logger.Infof("Starting API server on %s", s.httpServer.Addr)
// 在 goroutine 中启动服务器
go func() {
if err := s.httpServer.ListenAndServe(); err != nil && err != http.ErrServerClosed {
logger.Infof("Starting API server on port %d", cfg.API.Port)
if err := server.ListenAndServe(); err != nil && err != http.ErrServerClosed {
logger.Errorf("API server error: %v", err)
}
}()
// 监听 context 取消
// Graceful shutdown on context cancellation
go func() {
<-ctx.Done()
shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := s.httpServer.Shutdown(shutdownCtx); err != nil {
logger.Errorf("API server shutdown error: %v", err)
if err := server.Shutdown(shutdownCtx); err != nil {
logger.Errorf("Failed to shutdown API server: %v", err)
} else {
logger.Info("API server stopped")
}
}()
return nil
}
// loggingMiddleware 日志中间件
func loggingMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
start := time.Now()
func registerRoutes(mux *http.ServeMux) {
// Health check endpoint (no auth required)
mux.HandleFunc("/health", handleHealth)
// 包装 ResponseWriter 以获取状态码
wrapped := &responseWriter{ResponseWriter: w, statusCode: http.StatusOK}
next.ServeHTTP(wrapped, r)
log.Infof("%s %s %d %s", r.Method, r.URL.Path, wrapped.statusCode, time.Since(start))
})
}
// recoveryMiddleware 恢复中间件
func recoveryMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
defer func() {
if err := recover(); err != nil {
log.Errorf("Panic recovered: %v", err)
WriteError(w, http.StatusInternalServerError, "internal_error", "internal server error")
}
}()
next.ServeHTTP(w, r)
})
}
// responseWriter 包装 http.ResponseWriter 以捕获状态码
type responseWriter struct {
http.ResponseWriter
statusCode int
}
func (rw *responseWriter) WriteHeader(code int) {
rw.statusCode = code
rw.ResponseWriter.WriteHeader(code)
}
// Start initializes and starts the API server. It refuses to start without a
// token, since an open download proxy is a security risk.
func Start(ctx context.Context) error {
cfg := config.C().API
if !cfg.Enable {
return nil
}
if cfg.Token == "" {
return fmt.Errorf("API server is enabled but no token is set; refusing to start insecurely")
}
server := NewServer(ctx)
if err := server.Start(ctx); err != nil {
return err
}
StartCleanupLoop(ctx)
return nil
// API v1 endpoints
mux.HandleFunc("POST /api/v1/tasks", handleCreateTask)
mux.HandleFunc("GET /api/v1/tasks/{id}", handleGetTask)
mux.HandleFunc("GET /api/v1/tasks", handleListTasks)
mux.HandleFunc("DELETE /api/v1/tasks/{id}", handleCancelTask)
}

View File

@@ -1,272 +0,0 @@
package api
import (
"context"
"fmt"
"net/url"
"strconv"
"strings"
"github.com/celestix/gotgproto/ext"
"github.com/charmbracelet/log"
"github.com/gotd/td/tg"
"github.com/krau/SaveAny-Bot/client/bot"
userclient "github.com/krau/SaveAny-Bot/client/user"
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
"github.com/krau/SaveAny-Bot/pkg/tfile"
)
// MessageContext 保存消息和获取它所用的 context
type MessageContext struct {
Message *tg.Message
Client *ext.Context
}
// getClientContext 获取可用的客户端上下文
// 优先使用 Bot失败后回退到 Userbot
func getClientContext() (*ext.Context, error) {
// 首先尝试获取 Bot context
if botCtx := bot.ExtContext(); botCtx != nil {
return botCtx, nil
}
// 回退到 Userbot
if uc := userclient.GetCtx(); uc != nil {
return uc, nil
}
return nil, fmt.Errorf("no client available (bot and userbot are not initialized)")
}
// resolveChatID 解析聊天 ID
func resolveChatID(_ context.Context, idOrUsername string) (int64, error) {
// 如果是数字 ID
if id, err := strconv.ParseInt(idOrUsername, 10, 64); err == nil {
// 私有频道 ID 需要加上 -100 前缀
if id > 0 {
return -1000000000000 - id, nil
}
return id, nil
}
// 获取可用的客户端上下文
clientCtx, err := getClientContext()
if err != nil {
return 0, err
}
// 使用 tgutil 的 ParseChatID
return tgutil.ParseChatID(clientCtx, idOrUsername)
}
// ParseMessageLink 解析 Telegram 消息链接
// 支持格式:
// - https://t.me/username/123
// - https://t.me/c/123456789/123
// - https://t.me/c/123456789/111/456 (topic id)
// - https://t.me/username/123?comment=2 (评论)
func ParseMessageLink(ctx context.Context, link string) (int64, int, error) {
u, err := url.Parse(link)
if err != nil {
return 0, 0, fmt.Errorf("invalid URL: %w", err)
}
paths := strings.Split(strings.TrimPrefix(u.Path, "/"), "/")
if cmt := u.Query().Get("comment"); cmt != "" {
// 频道评论的消息链接
if len(paths) < 1 {
return 0, 0, fmt.Errorf("invalid message link format: %s", link)
}
// 简化处理:返回错误,提示不支持评论链接
return 0, 0, fmt.Errorf("comment links are not supported")
}
switch len(paths) {
case 2: // https://t.me/username/123
chatID, err := resolveChatID(ctx, paths[0])
if err != nil {
return 0, 0, fmt.Errorf("failed to resolve chat ID: %w", err)
}
msgID, err := strconv.Atoi(paths[1])
if err != nil {
return 0, 0, fmt.Errorf("failed to parse message ID: %w", err)
}
return chatID, msgID, nil
case 3:
// https://t.me/c/123456789/123
// https://t.me/username/123/456 , 123: topic id
chatPart, msgPart := paths[1], paths[2]
if paths[0] != "c" {
chatPart = paths[0]
}
chatID, err := resolveChatID(ctx, chatPart)
if err != nil {
return 0, 0, fmt.Errorf("failed to resolve chat ID: %w", err)
}
msgID, err := strconv.Atoi(msgPart)
if err != nil {
return 0, 0, fmt.Errorf("failed to parse message ID: %w", err)
}
return chatID, msgID, nil
case 4:
// https://t.me/c/123456789/111/456 111: topic id
if paths[0] != "c" {
return 0, 0, fmt.Errorf("invalid message link format: %s", link)
}
chatID, err := resolveChatID(ctx, paths[1])
if err != nil {
return 0, 0, fmt.Errorf("failed to resolve chat ID: %w", err)
}
msgID, err := strconv.Atoi(paths[3])
if err != nil {
return 0, 0, fmt.Errorf("failed to parse message ID: %w", err)
}
return chatID, msgID, nil
}
return 0, 0, fmt.Errorf("invalid message link format: %s", link)
}
// getMessageWithContext 通过 ID 获取消息,返回消息和使用的 context
// 确保消息获取和后续文件创建使用同一个 context
func getMessageWithContext(_ context.Context, chatID int64, msgID int) (*MessageContext, error) {
// 首先尝试使用 Bot
if botCtx := bot.ExtContext(); botCtx != nil {
msg, err := tgutil.GetMessageByID(botCtx, chatID, msgID)
if err == nil {
return &MessageContext{Message: msg, Client: botCtx}, nil
}
}
// 回退到 Userbot
uc := userclient.GetCtx()
if uc == nil {
return nil, fmt.Errorf("userbot not initialized and bot cannot access this message")
}
msg, err := tgutil.GetMessageByID(uc, chatID, msgID)
if err != nil {
return nil, err
}
return &MessageContext{Message: msg, Client: uc}, nil
}
// getGroupedMessagesWithContext 获取媒体组消息,返回消息列表和使用的 context
// 确保消息获取和后续文件创建使用同一个 context
func getGroupedMessagesWithContext(ctx *MessageContext, chatID int64) ([]*tg.Message, error) {
msg := ctx.Message
clientCtx := ctx.Client
groupID, ok := msg.GetGroupedID()
if !ok || groupID == 0 {
return []*tg.Message{msg}, nil
}
// 使用获取原始消息的同一个 client 获取媒体组
msgs, err := tgutil.GetGroupedMessages(clientCtx, chatID, msg)
if err != nil || len(msgs) == 0 {
// 如果获取失败,至少返回原始消息
return []*tg.Message{msg}, nil
}
return msgs, nil
}
// ExtractFilesFromLinks 从消息链接中提取文件
// 每个文件的处理流程:解析链接 -> 获取消息 -> 获取媒体组 -> 创建文件对象
// 对于单个文件,全程使用同一个 client context不会交叉
func ExtractFilesFromLinks(ctx context.Context, links []string) ([]tfile.TGFileMessage, error) {
logger := log.FromContext(ctx)
var files []tfile.TGFileMessage
for _, link := range links {
link = strings.TrimSpace(link)
if link == "" {
continue
}
// 验证链接格式
if !isValidMessageLink(link) {
logger.Errorf("Invalid message link format: %s", link)
continue
}
chatID, msgID, err := ParseMessageLink(ctx, link)
if err != nil {
logger.Errorf("Failed to parse message link %s: %v", link, err)
continue
}
// 解析链接 URL 检查是否有 single 参数
u, _ := url.Parse(link)
single := u != nil && u.Query().Has("single")
// 获取消息和使用的 contextBot 优先,失败回退 Userbot
msgCtx, err := getMessageWithContext(ctx, chatID, msgID)
if err != nil {
logger.Errorf("Failed to get message %d from chat %d: %v", msgID, chatID, err)
continue
}
msg := msgCtx.Message
clientCtx := msgCtx.Client
if msg.Media == nil {
logger.Warnf("Message %d has no media", msgID)
continue
}
media, ok := msg.GetMedia()
if !ok {
logger.Warnf("Failed to get media from message %d", msgID)
continue
}
// 检查是否是媒体组
groupID, isGroup := msg.GetGroupedID()
if isGroup && groupID != 0 && !single {
// 使用同一个 client context 获取媒体组
groupMsgs, err := getGroupedMessagesWithContext(msgCtx, chatID)
if err != nil {
logger.Errorf("Failed to get grouped messages: %v", err)
} else {
for _, gmsg := range groupMsgs {
if gmsg.Media == nil {
continue
}
gmedia, ok := gmsg.GetMedia()
if !ok {
continue
}
// 使用获取消息时使用的同一个 client context 创建文件
file, err := tfile.FromMediaMessage(gmedia, clientCtx.Raw, gmsg)
if err != nil {
logger.Errorf("Failed to create file from media: %v", err)
continue
}
files = append(files, file)
}
continue
}
}
// 单个文件 - 使用获取消息时使用的同一个 client context 创建文件
file, err := tfile.FromMediaMessage(media, clientCtx.Raw, msg)
if err != nil {
logger.Errorf("Failed to create file from media: %v", err)
continue
}
files = append(files, file)
}
if len(files) == 0 {
return nil, fmt.Errorf("no files found in provided links")
}
return files, nil
}
// isValidMessageLink 检查是否是有效的 Telegram 消息链接
func isValidMessageLink(link string) bool {
return strings.HasPrefix(link, "https://t.me/") || strings.HasPrefix(link, "http://t.me/")
}

View File

@@ -1,80 +0,0 @@
package api
import (
"context"
"fmt"
"net/url"
"strings"
"github.com/charmbracelet/log"
"github.com/krau/SaveAny-Bot/common/utils/tphutil"
"github.com/krau/SaveAny-Bot/pkg/telegraph"
)
// ExtractTelegraphImages 从 Telegraph URL 提取图片
func ExtractTelegraphImages(ctx context.Context, pageURL string) ([]string, string, error) {
logger := log.FromContext(ctx)
// 验证 URL 格式
if !isValidTelegraphURL(pageURL) {
return nil, "", fmt.Errorf("invalid telegraph URL format: %s", pageURL)
}
// 解析 URL 获取页面路径
pagepath, err := parseTelegraphPath(pageURL)
if err != nil {
return nil, "", err
}
logger.Debugf("Fetching telegraph page: %s", pagepath)
client := telegraph.NewClient()
page, err := client.GetPage(ctx, pagepath)
if err != nil {
return nil, "", fmt.Errorf("failed to get telegraph page: %w", err)
}
var imgs []string
for _, elem := range page.Content {
imgs = append(imgs, tphutil.GetNodeImages(elem)...)
}
if len(imgs) == 0 {
return nil, "", fmt.Errorf("no images found in telegraph page")
}
return imgs, pagepath, nil
}
// parseTelegraphPath 解析 Telegraph URL 获取页面路径
func parseTelegraphPath(pageURL string) (string, error) {
u, err := url.Parse(pageURL)
if err != nil {
return "", fmt.Errorf("invalid telegraph URL: %w", err)
}
if !strings.HasSuffix(u.Host, "telegra.ph") && !strings.HasSuffix(u.Host, "telegraph.co") {
return "", fmt.Errorf("invalid telegraph URL host: %s", u.Host)
}
paths := strings.Split(strings.TrimPrefix(u.Path, "/"), "/")
if len(paths) == 0 || paths[0] == "" {
return "", fmt.Errorf("invalid telegraph URL path: %s", u.Path)
}
pagepath := paths[len(paths)-1]
pagepath, err = url.PathUnescape(pagepath)
if err != nil {
return "", fmt.Errorf("failed to unescape telegraph path: %w", err)
}
return strings.TrimSpace(pagepath), nil
}
// isValidTelegraphURL 检查是否是有效的 Telegraph URL
func isValidTelegraphURL(url string) bool {
return strings.HasPrefix(url, "https://telegra.ph/") ||
strings.HasPrefix(url, "http://telegra.ph/") ||
strings.HasPrefix(url, "https://telegraph.co/") ||
strings.HasPrefix(url, "http://telegraph.co/")
}

View File

@@ -1,163 +0,0 @@
package api
import (
"encoding/json"
"net/http"
"time"
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
)
// TaskStatus 表示任务状态
type TaskStatus string
const (
TaskStatusQueued TaskStatus = "queued"
TaskStatusRunning TaskStatus = "running"
TaskStatusCompleted TaskStatus = "completed"
TaskStatusFailed TaskStatus = "failed"
TaskStatusCancelled TaskStatus = "cancelled"
)
// CreateTaskRequest 创建任务请求
type CreateTaskRequest struct {
Type tasktype.TaskType `json:"type"`
Storage string `json:"storage"`
Path string `json:"path"`
Webhook string `json:"webhook,omitempty"`
Params json.RawMessage `json:"params"`
}
// CreateTaskResponse 创建任务响应
type CreateTaskResponse struct {
TaskID string `json:"task_id"`
Type tasktype.TaskType `json:"type"`
Status TaskStatus `json:"status"`
CreatedAt time.Time `json:"created_at"`
}
// TaskProgress 任务进度
type TaskProgress struct {
TotalBytes int64 `json:"total_bytes,omitempty"`
DownloadedBytes int64 `json:"downloaded_bytes,omitempty"`
TotalFiles int `json:"total_files,omitempty"`
DownloadedFiles int `json:"downloaded_files,omitempty"`
Percent float64 `json:"percent,omitempty"`
SpeedMBPS float64 `json:"speed_mbps,omitempty"`
}
// TaskInfoResponse 任务信息响应
type TaskInfoResponse struct {
TaskID string `json:"task_id"`
Type tasktype.TaskType `json:"type"`
Status TaskStatus `json:"status"`
Title string `json:"title"`
Progress *TaskProgress `json:"progress,omitempty"`
Storage string `json:"storage"`
Path string `json:"path"`
Error string `json:"error,omitempty"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
// TasksListResponse 任务列表响应
type TasksListResponse struct {
Tasks []TaskInfoResponse `json:"tasks"`
Total int `json:"total"`
}
// StoragesResponse 存储列表响应
type StoragesResponse struct {
Storages []StorageInfo `json:"storages"`
}
// StorageInfo 存储信息
type StorageInfo struct {
Name string `json:"name"`
Type string `json:"type"`
}
// WebhookPayload Webhook 回调负载
type WebhookPayload struct {
TaskID string `json:"task_id"`
Type string `json:"type"`
Status TaskStatus `json:"status"`
Storage string `json:"storage"`
Path string `json:"path"`
CompletedAt *time.Time `json:"completed_at,omitempty"`
Error string `json:"error,omitempty"`
}
// ErrorResponse 错误响应
type ErrorResponse struct {
Error string `json:"error"`
Message string `json:"message,omitempty"`
}
// APIError API 错误
type APIError struct {
StatusCode int
ErrorCode string
Message string
}
func (e *APIError) Error() string {
return e.Message
}
// WriteJSON 写入 JSON 响应
func WriteJSON(w http.ResponseWriter, statusCode int, data any) error {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(statusCode)
return json.NewEncoder(w).Encode(data)
}
// WriteError 写入错误响应
func WriteError(w http.ResponseWriter, statusCode int, errCode, message string) error {
return WriteJSON(w, statusCode, ErrorResponse{
Error: errCode,
Message: message,
})
}
// Task 参数结构体
// DirectLinksParams directlinks 任务参数
type DirectLinksParams struct {
URLs []string `json:"urls"`
}
// YTDLPParams ytdlp 任务参数
type YTDLPParams struct {
URLs []string `json:"urls"`
Flags []string `json:"flags,omitempty"`
}
// Aria2Params aria2 任务参数
type Aria2Params struct {
URLs []string `json:"urls"`
Options map[string]string `json:"options,omitempty"`
}
// ParsedParams parsed 任务参数
type ParsedParams struct {
URL string `json:"url"`
}
// TransferParams transfer 任务参数
type TransferParams struct {
SourceStorage string `json:"source_storage"`
SourcePath string `json:"source_path"`
TargetStorage string `json:"target_storage"`
TargetPath string `json:"target_path"`
}
// TGFilesParams tgfiles 任务参数
type TGFilesParams struct {
MessageLinks []string `json:"message_links"`
}
// TPHPicsParams tphpics 任务参数
type TPHPicsParams struct {
TelegraphURL string `json:"telegraph_url"`
}

View File

@@ -1,99 +0,0 @@
package api
import (
"bytes"
"context"
"encoding/json"
"net/http"
"time"
"github.com/charmbracelet/log"
)
// webhookClient Webhook 客户端
var webhookClient = &http.Client{
Timeout: 30 * time.Second,
}
// SendWebhook 发送 Webhook 回调
func SendWebhook(ctx context.Context, payload *WebhookPayload) {
if payload == nil || payload.TaskID == "" {
return
}
// 获取任务信息以获取 webhook URL
info, ok := GetTask(payload.TaskID)
if !ok || info.Webhook == "" {
return
}
webhookURL := info.Webhook
// Async send with retries.
go func() {
var logger *log.Logger
if ctx != nil {
logger = log.FromContext(ctx).With("task_id", payload.TaskID)
} else {
logger = log.Default().With("task_id", payload.TaskID)
}
payloadBytes, err := json.Marshal(payload)
if err != nil {
logger.Errorf("Failed to marshal webhook payload: %v", err)
return
}
// 重试 3 次
for i := range 3 {
req, err := http.NewRequestWithContext(context.Background(), http.MethodPost, webhookURL, bytes.NewBuffer(payloadBytes))
if err != nil {
logger.Errorf("Failed to create webhook request: %v", err)
return
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("User-Agent", "SaveAny-Bot/1.0")
resp, err := webhookClient.Do(req)
if err != nil {
logger.Warnf("Webhook request failed (attempt %d/3): %v", i+1, err)
time.Sleep(time.Second * time.Duration(i+1))
continue
}
resp.Body.Close()
if resp.StatusCode >= 200 && resp.StatusCode < 300 {
logger.Debugf("Webhook sent successfully: %s", webhookURL)
return
}
logger.Warnf("Webhook returned non-2xx status (attempt %d/3): %d", i+1, resp.StatusCode)
time.Sleep(time.Second * time.Duration(i+1))
}
logger.Errorf("Failed to send webhook after 3 attempts")
}()
}
// CreateWebhookPayload creates a Webhook payload.
func CreateWebhookPayload(taskID string, taskType string, status TaskStatus, storage, path string, err error) *WebhookPayload {
payload := &WebhookPayload{
TaskID: taskID,
Type: taskType,
Status: status,
Storage: storage,
Path: path,
}
if status == TaskStatusCompleted || status == TaskStatusFailed {
now := time.Now()
payload.CompletedAt = &now
}
if err != nil {
payload.Error = err.Error()
}
return payload
}

View File

@@ -73,16 +73,14 @@ func handleAddCallback(ctx *ext.Context, update *ext.Update) error {
return dispatcher.EndGroups
}
dirPath = dir.Path
} else if data.SelectedDirPath != "" {
dirPath = data.SelectedDirPath
}
switch data.TaskType {
case tasktype.TaskTypeTgfiles:
if data.AsBatch {
return shortcut.CreateAndAddBatchTGFileTaskWithEdit(ctx, userID, selectedStorage, dirPath, data.Files, msgID, data.ConflictStrategy)
return shortcut.CreateAndAddBatchTGFileTaskWithEdit(ctx, userID, selectedStorage, dirPath, data.Files, msgID)
}
return shortcut.CreateAndAddTGFileTaskWithEdit(ctx, userID, selectedStorage, dirPath, data.Files[0], msgID, data.ConflictStrategy)
return shortcut.CreateAndAddTGFileTaskWithEdit(ctx, userID, selectedStorage, dirPath, data.Files[0], msgID)
case tasktype.TaskTypeTphpics:
return shortcut.CreateAndAddtelegraphWithEdit(ctx, userID, data.TphPageNode, data.TphDirPath, data.TphPics, selectedStorage, msgID)
case tasktype.TaskTypeParseditem:
@@ -102,9 +100,7 @@ func handleAddCallback(ctx *ext.Context, update *ext.Update) error {
}
shortcut.CreateAndAddAria2TaskWithEdit(ctx, selectedStorage, dirPath, data.Aria2URIs, client, msgID, userID)
case tasktype.TaskTypeYtdlp:
shortcut.CreateAndAddYtdlpTaskWithEdit(ctx, selectedStorage, dirPath, data.YtdlpURLs, data.YtdlpFlags, msgID, userID)
case tasktype.TaskTypeTransfer:
return handleTransferCallback(ctx, userID, selectedStorage, dirPath, data, msgID)
shortcut.CreateAndAddYtdlpTaskWithEdit(ctx, selectedStorage, dirPath, data.YtdlpURLs, msgID, userID)
default:
return fmt.Errorf("unexcept task type: %s", data.TaskType)
}

View File

@@ -8,7 +8,6 @@ import (
"github.com/celestix/gotgproto/dispatcher"
"github.com/celestix/gotgproto/ext"
"github.com/gotd/td/tg"
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/conflictutil"
"github.com/krau/SaveAny-Bot/common/i18n"
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
"github.com/krau/SaveAny-Bot/config"
@@ -27,10 +26,6 @@ func handleConfigCmd(ctx *ext.Context, update *ext.Update) error {
Text: i18n.T(i18nk.BotMsgConfigButtonFilenameStrategy),
Data: fmt.Appendf(nil, "%s %s", tcbdata.TypeConfig, "fnamest"),
},
&tg.KeyboardButtonCallback{
Text: i18n.T(i18nk.BotMsgConfigButtonConflictStrategy),
Data: fmt.Appendf(nil, "%s %s", tcbdata.TypeConfig, "conflictst"),
},
},
},
},
@@ -56,8 +51,6 @@ func handleConfigCallback(ctx *ext.Context, update *ext.Update) error {
switch args[1] {
case "fnamest":
return handleConfigFnameSTCallback(ctx, update)
case "conflictst":
return handleConfigConflictSTCallback(ctx, update)
default:
return invaildDataAnswer()
}
@@ -117,55 +110,6 @@ func handleConfigFnameSTCallback(ctx *ext.Context, update *ext.Update) error {
return dispatcher.EndGroups
}
func handleConfigConflictSTCallback(ctx *ext.Context, update *ext.Update) error {
userID := update.CallbackQuery.GetUserID()
user, err := database.GetUserByChatID(ctx, userID)
if err != nil {
return err
}
args := strings.Fields(string(update.CallbackQuery.Data))
if len(args) == 3 {
selected := args[2]
if !tcbdata.IsConflictStrategy(selected) {
return fmt.Errorf("invalid conflict strategy: %s", selected)
}
user.ConflictStrategy = selected
if err := database.UpdateUser(ctx, user); err != nil {
return err
}
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
ID: update.CallbackQuery.GetMsgID(),
Message: i18n.T(i18nk.BotMsgConfigInfoConflictStrategySet, map[string]any{
"Strategy": conflictutil.Display(selected),
}),
})
return dispatcher.EndGroups
}
opts := tcbdata.ConflictStrategyValues()
rows := make([]tg.KeyboardButtonRow, 0, len(opts))
for _, opt := range opts {
rows = append(rows, tg.KeyboardButtonRow{
Buttons: []tg.KeyboardButtonClass{
&tg.KeyboardButtonCallback{
Text: conflictutil.Display(opt),
Data: fmt.Appendf(nil, "%s %s %s", tcbdata.TypeConfig, "conflictst", opt),
},
},
})
}
markup := &tg.ReplyInlineMarkup{Rows: rows}
currentSt := conflictutil.EffectiveStrategy(user)
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
ID: update.CallbackQuery.GetMsgID(),
Message: i18n.T(i18nk.BotMsgConfigPromptSelectConflictStrategy, map[string]any{
"Strategy": conflictutil.Display(currentSt),
}),
ReplyMarkup: markup,
})
return dispatcher.EndGroups
}
func handleConfigFnameTmpl(ctx *ext.Context, update *ext.Update) error {
userID := update.GetUserChat().GetID()
user, err := database.GetUserByChatID(ctx, userID)

View File

@@ -84,7 +84,7 @@ func handleAria2DlCmd(ctx *ext.Context, update *ext.Update) error {
return nil
}
logger.Debug("Preparing aria2 download", "links", links)
// Initialize aria2 client to check connection
aria2ClientInitOnce.Do(func() {
aria2Client, aria2ClientInitErr = aria2.NewClient(config.C().Aria2.Url, config.C().Aria2.Secret)

View File

@@ -13,7 +13,7 @@ import (
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/shortcut"
"github.com/krau/SaveAny-Bot/common/i18n"
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
"github.com/krau/SaveAny-Bot/database"
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
"github.com/krau/SaveAny-Bot/config"
"github.com/krau/SaveAny-Bot/pkg/tcbdata"
"github.com/krau/SaveAny-Bot/pkg/tfile"
@@ -53,13 +53,9 @@ func handleGroupMediaMessage(ctx *ext.Context, update *ext.Update, message *tg.M
if !supported {
return dispatcher.EndGroups
}
userId := update.GetUserChat().GetID()
userDB, err := database.GetUserByChatID(ctx, userId)
if err != nil {
return err
}
tfOpts := mediautil.TfileOptions(ctx, userDB, message)
file, err := tfile.FromMediaMessage(media, ctx.Raw, message, tfOpts...)
file, err := tfile.FromMediaMessage(media, ctx.Raw, message, tfile.WithNameIfEmpty(
tgutil.GenFileNameFromMessage(*message),
))
if err != nil {
logger.Errorf("Failed to get file from media: %s", err)
return dispatcher.EndGroups
@@ -118,7 +114,7 @@ func processMediaGroup(ctx *ext.Context, update *ext.Update, groupID int64) {
if err != nil {
logger.Errorf("Failed to build storage selection keyboard: %s", err)
ctx.EditMessage(userId, &tg.MessagesEditMessageRequest{
ID: msg.ID,
ID: msg.ID,
Message: i18n.T(i18nk.BotMsgMediaGroupErrorBuildStorageSelectKeyboardFailed, map[string]any{
"Error": err.Error(),
}),

View File

@@ -31,7 +31,6 @@ var CommandHandlers = []DescCommandHandler{
{"dl", i18nk.BotMsgCmdDl, handleDlCmd},
{"aria2dl", i18nk.BotMsgCmdAria2dl, handleAria2DlCmd},
{"ytdlp", i18nk.BotMsgCmdYtdlp, handleYtdlpCmd},
{"transfer", i18nk.BotMsgCmdTransfer, handleTransferCmd},
{"task", i18nk.BotMsgCmdTask, handleTaskCmd},
{"cancel", i18nk.BotMsgCmdCancel, handleCancelCmd},
{"config", i18nk.BotMsgCmdConfig, handleConfigCmd},

View File

@@ -13,7 +13,6 @@ import (
"github.com/krau/SaveAny-Bot/common/i18n"
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
"github.com/krau/SaveAny-Bot/common/utils/strutil"
"github.com/krau/SaveAny-Bot/config"
"github.com/krau/SaveAny-Bot/database"
"github.com/krau/SaveAny-Bot/pkg/rule"
)
@@ -85,46 +84,6 @@ func handleRuleCmd(ctx *ext.Context, update *ext.Update) error {
return dispatcher.EndGroups
}
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgRuleInfoCreateRuleSuccess, nil)), nil)
case "preset":
// /rule preset <storage> [base_path]
if len(args) < 3 {
ctx.Reply(update, ext.ReplyTextStyledTextArray(msgelem.BuildRuleHelpStyling(user.ApplyRule, user.Rules)), nil)
return dispatcher.EndGroups
}
storageName := args[2]
if !config.C().HasStorage(user.ChatID, storageName) {
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgRuleErrorStorageNotFound, map[string]any{
"Storage": storageName,
})), nil)
return dispatcher.EndGroups
}
basePath := ""
if len(args) >= 4 {
basePath = args[3]
}
presets := rule.PresetCategories(basePath)
imported := 0
for _, p := range presets {
rd := &database.Rule{
Type: rule.FileNameRegex.String(),
Data: p.Regex,
StorageName: storageName,
DirPath: p.Dir,
UserID: user.ID,
}
if err := database.CreateRule(ctx, rd); err != nil {
logger.Errorf("failed to create preset rule %s: %s", p.Name, err)
continue
}
imported++
}
if imported == 0 {
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgRuleErrorCreateRuleFailed, nil)), nil)
return dispatcher.EndGroups
}
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgRuleInfoPresetImported, map[string]any{
"Count": imported,
})), nil)
case "del":
// /rule del <id>
if len(args) < 3 {

View File

@@ -38,7 +38,7 @@ func handleTaskCmd(ctx *ext.Context, update *ext.Update) error {
return dispatcher.EndGroups
}
ctx.Reply(update, ext.ReplyTextStyledTextArray([]styling.StyledTextOption{
styling.Plain(i18n.T(i18nk.BotMsgTasksCancelRequestedPrefix)),
styling.Plain(i18n.T(i18nk.BotMsgTasksCancelRequestedPrefix)),
styling.Code(taskID),
}), nil)
default:

View File

@@ -1,257 +0,0 @@
package handlers
import (
"fmt"
"regexp"
"strings"
"github.com/celestix/gotgproto/dispatcher"
"github.com/celestix/gotgproto/ext"
"github.com/charmbracelet/log"
"github.com/gotd/td/tg"
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/msgelem"
"github.com/krau/SaveAny-Bot/common/i18n"
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
"github.com/krau/SaveAny-Bot/common/utils/strutil"
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
"github.com/krau/SaveAny-Bot/core"
"github.com/krau/SaveAny-Bot/core/tasks/transfer"
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
"github.com/krau/SaveAny-Bot/pkg/storagetypes"
"github.com/krau/SaveAny-Bot/pkg/tcbdata"
"github.com/krau/SaveAny-Bot/storage"
"github.com/rs/xid"
)
func handleTransferCmd(ctx *ext.Context, update *ext.Update) error {
logger := log.FromContext(ctx)
args := strutil.ParseArgsRespectQuotes(update.EffectiveMessage.Text)
if len(args) < 2 {
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgTransferUsage, nil)), nil)
return dispatcher.EndGroups
}
// Parse source: storage_name:/path
sourceParts := strings.SplitN(args[1], ":", 2)
if len(sourceParts) != 2 {
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgTransferErrorInvalidSource, nil)), nil)
return dispatcher.EndGroups
}
sourceStorageName := sourceParts[0]
sourcePath := sourceParts[1]
userID := update.GetUserChat().GetID()
// Get source storage
sourceStorage, err := storage.GetStorageByUserIDAndName(ctx, userID, sourceStorageName)
if err != nil {
logger.Errorf("Failed to get source storage by user ID and name: %s", err)
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgTransferErrorStorageNotFound, map[string]any{
"StorageName": sourceStorageName,
"Error": err,
})), nil)
return dispatcher.EndGroups
}
// Check if source storage supports listing
listable, ok := sourceStorage.(storage.StorageListable)
if !ok {
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgTransferErrorStorageNotListable, map[string]any{
"StorageName": sourceStorageName,
})), nil)
return dispatcher.EndGroups
}
// Check if source storage supports reading
_, ok = sourceStorage.(storage.StorageReadable)
if !ok {
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgTransferErrorStorageNotReadable, map[string]any{
"StorageName": sourceStorageName,
})), nil)
return dispatcher.EndGroups
}
// Fetch file list
replied, err := ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgTransferInfoFetchingFiles, nil)), nil)
if err != nil {
logger.Errorf("Failed to reply: %s", err)
return dispatcher.EndGroups
}
files, err := listable.ListFiles(ctx, sourcePath)
if err != nil {
ctx.EditMessage(update.EffectiveChat().GetID(), &tg.MessagesEditMessageRequest{
ID: replied.ID,
Message: i18n.T(i18nk.BotMsgTransferErrorListFilesFailed, map[string]any{"Error": err}),
})
return dispatcher.EndGroups
}
// Optional filter
var filter *regexp.Regexp
if len(args) >= 3 {
filter, err = regexp.Compile(args[2])
if err != nil {
ctx.EditMessage(update.EffectiveChat().GetID(), &tg.MessagesEditMessageRequest{
ID: replied.ID,
Message: i18n.T(i18nk.BotMsgTransferErrorInvalidRegex, map[string]any{"Error": err}),
})
return dispatcher.EndGroups
}
}
// Filter files
filteredFiles := make([]storagetypes.FileInfo, 0)
for _, file := range files {
if file.IsDir {
continue
}
if filter != nil && !filter.MatchString(file.Name) {
continue
}
filteredFiles = append(filteredFiles, file)
}
if len(filteredFiles) == 0 {
ctx.EditMessage(update.EffectiveChat().GetID(), &tg.MessagesEditMessageRequest{
ID: replied.ID,
Message: i18n.T(i18nk.BotMsgTransferErrorNoFilesToTransfer, nil),
})
return dispatcher.EndGroups
}
// Prepare file paths for callback data
filePaths := make([]string, 0, len(filteredFiles))
var totalSize int64
for _, file := range filteredFiles {
filePaths = append(filePaths, file.Path)
totalSize += file.Size
}
// Build storage selection keyboard
markup, err := msgelem.BuildAddSelectStorageKeyboard(storage.GetUserStorages(ctx, userID), tcbdata.Add{
TaskType: tasktype.TaskTypeTransfer,
TransferSourceStorName: sourceStorageName,
TransferSourcePath: sourcePath,
TransferFiles: filePaths,
})
if err != nil {
logger.Errorf("Failed to build storage selection keyboard: %s", err)
ctx.EditMessage(update.EffectiveChat().GetID(), &tg.MessagesEditMessageRequest{
ID: replied.ID,
Message: i18n.T(i18nk.BotMsgTransferErrorBuildStorageSelectKeyboardFailed, map[string]any{"Error": err}),
})
return dispatcher.EndGroups
}
ctx.EditMessage(update.EffectiveChat().GetID(), &tg.MessagesEditMessageRequest{
ID: replied.ID,
Message: i18n.T(i18nk.BotMsgTransferInfoFilesSelectStorage, map[string]any{
"Count": len(filteredFiles),
"SizeMB": fmt.Sprintf("%.2f", float64(totalSize)/(1024*1024)),
}),
ReplyMarkup: markup,
})
return dispatcher.EndGroups
}
func handleTransferCallback(ctx *ext.Context, userID int64, targetStorage storage.Storage, dirPath string, data tcbdata.Add, msgID int) error {
logger := log.FromContext(ctx)
// Get source storage
sourceStorage, err := storage.GetStorageByUserIDAndName(ctx, userID, data.TransferSourceStorName)
if err != nil {
logger.Errorf("Failed to get source storage: %s", err)
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
ID: msgID,
Message: i18n.T(i18nk.BotMsgTransferErrorStorageNotFound, map[string]any{"StorageName": data.TransferSourceStorName, "Error": err}),
})
return dispatcher.EndGroups
}
// Check if source storage supports listing
listable, ok := sourceStorage.(storage.StorageListable)
if !ok {
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
ID: msgID,
Message: i18n.T(i18nk.BotMsgTransferErrorStorageNotListable, map[string]any{"StorageName": data.TransferSourceStorName}),
})
return dispatcher.EndGroups
}
// Re-fetch files to get FileInfo (since we only stored paths)
// This is necessary to get size and other metadata
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
ID: msgID,
Message: i18n.T(i18nk.BotMsgTransferInfoFetchingFiles, nil),
})
allFiles, err := listable.ListFiles(ctx, data.TransferSourcePath)
if err != nil {
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
ID: msgID,
Message: i18n.T(i18nk.BotMsgTransferErrorListFilesFailed, map[string]any{"Error": err}),
})
return dispatcher.EndGroups
}
// Create a map for quick lookup
fileMap := make(map[string]storagetypes.FileInfo)
for _, file := range allFiles {
fileMap[file.Path] = file
}
// Build task elements for the selected files
elems := make([]transfer.TaskElement, 0, len(data.TransferFiles))
var totalSize int64
for _, filePath := range data.TransferFiles {
fileInfo, ok := fileMap[filePath]
if !ok {
logger.Warnf("File not found in source storage: %s", filePath)
continue
}
elem := transfer.NewTaskElement(sourceStorage, fileInfo, targetStorage, dirPath)
elems = append(elems, *elem)
totalSize += fileInfo.Size
}
if len(elems) == 0 {
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
ID: msgID,
Message: i18n.T(i18nk.BotMsgTransferErrorNoFilesToTransfer, nil),
})
return dispatcher.EndGroups
}
// Create and add task
taskID := xid.New().String()
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
task := transfer.NewTransferTask(
taskID,
injectCtx,
elems,
transfer.NewProgressTracker(msgID, userID),
true, // IgnoreErrors
)
if err := core.AddTask(injectCtx, task); err != nil {
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
ID: msgID,
Message: i18n.T(i18nk.BotMsgTransferErrorAddTaskFailed, map[string]any{"Error": err}),
})
return dispatcher.EndGroups
}
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
ID: msgID,
Message: i18n.T(i18nk.BotMsgTransferInfoTaskAdded, map[string]any{
"Count": len(elems),
"SizeMB": fmt.Sprintf("%.2f", float64(totalSize)/(1024*1024)),
"TaskID": taskID,
}),
})
return dispatcher.EndGroups
}

View File

@@ -103,7 +103,7 @@ func handleUpdateCallback(ctx *ext.Context, u *ext.Update) error {
return err
}
ctx.EditMessage(u.GetUserChat().GetID(), &tg.MessagesEditMessageRequest{
ID: u.CallbackQuery.GetMsgID(),
ID: u.CallbackQuery.GetMsgID(),
Message: i18n.T(i18nk.BotMsgUpdateInfoUpgradingWithVersion, map[string]any{
"Current": config.Version,
}),
@@ -111,7 +111,7 @@ func handleUpdateCallback(ctx *ext.Context, u *ext.Update) error {
latest, err := ghselfupdate.UpdateSelf(currentV, config.GitRepo)
if err != nil {
ctx.EditMessage(u.GetUserChat().GetID(), &tg.MessagesEditMessageRequest{
ID: u.CallbackQuery.GetMsgID(),
ID: u.CallbackQuery.GetMsgID(),
Message: i18n.T(i18nk.BotMsgUpdateErrorUpgradeFailed, map[string]any{
"Error": err.Error(),
}),
@@ -119,7 +119,7 @@ func handleUpdateCallback(ctx *ext.Context, u *ext.Update) error {
return dispatcher.EndGroups
}
ctx.EditMessage(u.GetUserChat().GetID(), &tg.MessagesEditMessageRequest{
ID: u.CallbackQuery.GetMsgID(),
ID: u.CallbackQuery.GetMsgID(),
Message: i18n.T(i18nk.BotMsgUpdateInfoUpgradeSuccess, map[string]any{
"Version": latest.Version.String(),
}),

View File

@@ -1,55 +0,0 @@
package conflictutil
import (
"fmt"
"strings"
"github.com/krau/SaveAny-Bot/common/i18n"
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
"github.com/krau/SaveAny-Bot/database"
"github.com/krau/SaveAny-Bot/pkg/tcbdata"
)
const maxConflictLines = 10
func EffectiveStrategy(user *database.User) string {
if user != nil && tcbdata.IsConflictStrategy(user.ConflictStrategy) {
return user.ConflictStrategy
}
return tcbdata.ConflictStrategyRename
}
func ResolveStrategy(user *database.User, override string) string {
if tcbdata.IsConflictStrategy(override) {
return override
}
return EffectiveStrategy(user)
}
func Display(strategy string) string {
switch strategy {
case tcbdata.ConflictStrategyRename:
return i18n.T(i18nk.BotMsgConfigConflictStrategyRename, nil)
case tcbdata.ConflictStrategyAsk:
return i18n.T(i18nk.BotMsgConfigConflictStrategyAsk, nil)
case tcbdata.ConflictStrategyOverwrite:
return i18n.T(i18nk.BotMsgConfigConflictStrategyOverwrite, nil)
case tcbdata.ConflictStrategySkip:
return i18n.T(i18nk.BotMsgConfigConflictStrategySkip, nil)
default:
return strategy
}
}
func FormatPaths(conflicts []string) string {
if len(conflicts) <= maxConflictLines {
return strings.Join(conflicts, "\n")
}
return strings.Join(conflicts[:maxConflictLines], "\n") + "\n" + i18n.T(i18nk.BotMsgCommonPromptConflictMoreFiles, map[string]any{
"Count": len(conflicts) - maxConflictLines,
})
}
func FormatPath(storageName, storagePath string) string {
return fmt.Sprintf("[%s]:%s", storageName, storagePath)
}

View File

@@ -112,7 +112,7 @@ func BuildFilenameTemplateData(message *tg.Message) map[string]string {
}(),
MsgRaw: message.GetMessage(),
ChatID: func() string {
// 如果消息是频道的(从消息链接中fetch的) 直接使用其chat id,
// 如果消息是频道的(从消息链接中fetch的) 直接使用其chat id,
// 无论它是否是从其他来源转发的
if message.GetPost() {
peer := message.GetPeerID()

View File

@@ -24,8 +24,6 @@ func BuildRuleHelpStyling(enabled bool, rules []database.Rule) []styling.StyledT
styling.Plain(i18n.T(i18nk.BotMsgRuleHelpSwitchSuffix, nil)),
styling.Code("add"),
styling.Plain(i18n.T(i18nk.BotMsgRuleHelpAddSuffix, nil)),
styling.Code("preset"),
styling.Plain(i18n.T(i18nk.BotMsgRuleHelpPresetSuffix, nil)),
styling.Code("del"),
styling.Plain(i18n.T(i18nk.BotMsgRuleHelpDelSuffix, nil)),
styling.Plain(i18n.T(i18nk.BotMsgRuleHelpExistingRulesPrefix, nil)),

View File

@@ -38,8 +38,6 @@ func BuildAddSelectStorageKeyboard(stors []storage.Storage, adddata tcbdata.Add)
data := tcbdata.Add{
TaskType: taskType,
SelectedStorName: storage.Name(),
SelectedDirPath: adddata.SelectedDirPath,
ConflictStrategy: adddata.ConflictStrategy,
Files: adddata.Files,
AsBatch: len(adddata.Files) > 1,
@@ -52,13 +50,8 @@ func BuildAddSelectStorageKeyboard(stors []storage.Storage, adddata tcbdata.Add)
DirectLinks: adddata.DirectLinks,
Aria2URIs: adddata.Aria2URIs,
YtdlpURLs: adddata.YtdlpURLs,
YtdlpFlags: adddata.YtdlpFlags,
TransferSourceStorName: adddata.TransferSourceStorName,
TransferSourcePath: adddata.TransferSourcePath,
TransferFiles: adddata.TransferFiles,
Aria2URIs: adddata.Aria2URIs,
YtdlpURLs: adddata.YtdlpURLs,
}
dataid := xid.New().String()
err := cache.Set(dataid, data)
@@ -111,38 +104,6 @@ func BuildAddOneSelectStorageMessage(ctx context.Context, stors []storage.Storag
}, nil
}
func BuildConflictStrategyMarkup(adddata tcbdata.Add) (*tg.ReplyInlineMarkup, error) {
type option struct {
text string
strategy string
}
options := []option{
{text: i18n.T(i18nk.BotMsgCommonButtonConflictRename, nil), strategy: tcbdata.ConflictStrategyRename},
{text: i18n.T(i18nk.BotMsgCommonButtonConflictOverwrite, nil), strategy: tcbdata.ConflictStrategyOverwrite},
{text: i18n.T(i18nk.BotMsgCommonButtonConflictSkip, nil), strategy: tcbdata.ConflictStrategySkip},
}
buttons := make([]tg.KeyboardButtonClass, 0, len(options))
for _, opt := range options {
data := adddata
data.ConflictStrategy = opt.strategy
dataid := xid.New().String()
if err := cache.Set(dataid, data); err != nil {
return nil, err
}
buttons = append(buttons, &tg.KeyboardButtonCallback{
Text: opt.text,
Data: fmt.Appendf(nil, "%s %s", tcbdata.TypeAdd, dataid),
})
}
rows := make([]tg.KeyboardButtonRow, 0, len(buttons))
for _, button := range buttons {
rows = append(rows, tg.KeyboardButtonRow{
Buttons: []tg.KeyboardButtonClass{button},
})
}
return &tg.ReplyInlineMarkup{Rows: rows}, nil
}
// Builds the inline keyboard for setting default storage
func BuildSetDefaultStorageMarkup(
ctx context.Context,

View File

@@ -46,7 +46,7 @@ func CreateAndAddAria2TaskWithEdit(ctx *ext.Context, stor storage.Storage, dirPa
logger.Infof("Aria2 download added with GID: %s", gid)
// Create task with the GID
task := aria2dl.NewTask(xid.New().String(), injectCtx, gid, uris, aria2Client, stor, dirPath, aria2dl.NewProgress(msgID, userID))
task := aria2dl.NewTask(xid.New().String(), injectCtx, gid, uris, aria2Client, stor, stor.JoinStoragePath(dirPath), aria2dl.NewProgress(msgID, userID))
if err := core.AddTask(injectCtx, task); err != nil {
logger.Errorf("Failed to add task: %s", err)
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{

View File

@@ -16,7 +16,7 @@ import (
func CreateAndAddDirectTaskWithEdit(ctx *ext.Context, stor storage.Storage, dirPath string, links []string, msgID int, userID int64) error {
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
task := directlinks.NewTask(xid.New().String(), injectCtx, links, stor, dirPath, directlinks.NewProgress(msgID, userID))
task := directlinks.NewTask(xid.New().String(), injectCtx, links, stor, stor.JoinStoragePath(dirPath), directlinks.NewProgress(msgID, userID))
if err := core.AddTask(injectCtx, task); err != nil {
log.FromContext(ctx).Errorf("Failed to add task: %s", err)
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{

View File

@@ -18,11 +18,11 @@ import (
func CreateAndAddParsedTaskWithEdit(ctx *ext.Context, stor storage.Storage, dirPath string, item *parser.Item, msgID int, userID int64) error {
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
task := parsed.NewTask(xid.New().String(), injectCtx, stor, dirPath, item, parsed.NewProgress(msgID, userID))
task := parsed.NewTask(xid.New().String(), injectCtx, stor, stor.JoinStoragePath(dirPath), item, parsed.NewProgress(msgID, userID))
if err := core.AddTask(injectCtx, task); err != nil {
log.FromContext(ctx).Errorf("Failed to add task: %s", err)
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
ID: msgID,
ID: msgID,
Message: i18n.T(i18nk.BotMsgCommonErrorTaskAddFailed, map[string]any{
"Error": err.Error(),
}),

View File

@@ -8,7 +8,6 @@ import (
"github.com/celestix/gotgproto/ext"
"github.com/charmbracelet/log"
"github.com/gotd/td/tg"
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/conflictutil"
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/msgelem"
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/ruleutil"
"github.com/krau/SaveAny-Bot/common/i18n"
@@ -18,29 +17,25 @@ import (
"github.com/krau/SaveAny-Bot/core/tasks/batchtfile"
tftask "github.com/krau/SaveAny-Bot/core/tasks/tfile"
"github.com/krau/SaveAny-Bot/database"
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
"github.com/krau/SaveAny-Bot/pkg/tcbdata"
"github.com/krau/SaveAny-Bot/pkg/tfile"
"github.com/krau/SaveAny-Bot/storage"
"github.com/rs/xid"
)
// 创建一个 tfile.TGFileTask 并添加到任务队列中, 以编辑消息的方式反馈结果
func CreateAndAddTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor storage.Storage, dirPath string, file tfile.TGFileMessage, trackMsgID int, conflictStrategy ...string) error {
func CreateAndAddTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor storage.Storage, dirPath string, file tfile.TGFileMessage, trackMsgID int) error {
logger := log.FromContext(ctx)
strategy := selectedConflictStrategy(conflictStrategy)
user, err := database.GetUserByChatID(ctx, userID)
if err != nil {
logger.Errorf("Failed to get user by chat ID: %s", err)
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
ID: trackMsgID,
ID: trackMsgID,
Message: i18n.T(i18nk.BotMsgCommonErrorGetUserWithErrFailed, map[string]any{
"Error": err.Error(),
}),
})
return dispatcher.EndGroups
}
strategy = conflictutil.ResolveStrategy(user, strategy)
if user.ApplyRule && user.Rules != nil {
matched, matchedStorageName, matchedDirPath := ruleutil.ApplyRule(ctx, user.Rules, ruleutil.NewInput(file))
if !matched {
@@ -54,7 +49,7 @@ func CreateAndAddTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor storage
if err != nil {
logger.Errorf("Failed to get storage by user ID and name: %s", err)
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
ID: trackMsgID,
ID: trackMsgID,
Message: i18n.T(i18nk.BotMsgCommonErrorGetStorageFailed, map[string]any{
"Error": err.Error(),
}),
@@ -64,27 +59,8 @@ func CreateAndAddTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor storage
}
}
startCreateTask:
storagePath := path.Join(dirPath, file.Name())
if strategy == tcbdata.ConflictStrategyAsk || strategy == tcbdata.ConflictStrategySkip {
exists := stor.Exists(ctx, storagePath)
if exists && strategy == tcbdata.ConflictStrategyAsk {
return promptTGFileConflictStrategy(ctx, userID, stor.Name(), dirPath, []tfile.TGFileMessage{file}, false, []string{conflictutil.FormatPath(stor.Name(), storagePath)}, trackMsgID)
}
if exists {
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
ID: trackMsgID,
Message: i18n.T(i18nk.BotMsgCommonInfoAllConflictFilesSkipped, map[string]any{
"Skipped": file.Name(),
}),
ReplyMarkup: nil,
})
return dispatcher.EndGroups
}
}
storagePath := stor.JoinStoragePath(path.Join(dirPath, file.Name()))
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
if strategy == tcbdata.ConflictStrategyOverwrite {
injectCtx = storage.WithOverwrite(injectCtx)
}
taskid := xid.New().String()
task, err := tftask.NewTGFileTask(taskid, injectCtx, file, stor, storagePath,
tftask.NewProgressTrack(
@@ -93,7 +69,7 @@ startCreateTask:
if err != nil {
logger.Errorf("create task failed: %s", err)
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
ID: trackMsgID,
ID: trackMsgID,
Message: i18n.T(i18nk.BotMsgCommonErrorTaskCreateFailed, map[string]any{
"Error": err.Error(),
}),
@@ -103,7 +79,7 @@ startCreateTask:
if err := core.AddTask(injectCtx, task); err != nil {
logger.Errorf("add task failed: %s", err)
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
ID: trackMsgID,
ID: trackMsgID,
Message: i18n.T(i18nk.BotMsgCommonErrorTaskAddFailed, map[string]any{
"Error": err.Error(),
}),
@@ -121,21 +97,19 @@ startCreateTask:
}
// 创建一个 batchtfile.BatchTGFileTask 并添加到任务队列中, 以编辑消息的方式反馈结果
func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor storage.Storage, dirPath string, files []tfile.TGFileMessage, trackMsgID int, conflictStrategy ...string) error {
func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor storage.Storage, dirPath string, files []tfile.TGFileMessage, trackMsgID int) error {
logger := log.FromContext(ctx)
strategy := selectedConflictStrategy(conflictStrategy)
user, err := database.GetUserByChatID(ctx, userID)
if err != nil {
logger.Errorf("Failed to get user by chat ID: %s", err)
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
ID: trackMsgID,
ID: trackMsgID,
Message: i18n.T(i18nk.BotMsgCommonErrorGetUserWithErrFailed, map[string]any{
"Error": err.Error(),
}),
})
return dispatcher.EndGroups
}
strategy = conflictutil.ResolveStrategy(user, strategy)
useRule := user.ApplyRule && user.Rules != nil
@@ -154,24 +128,21 @@ func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor st
return storname, dirP
}
skipped := make([]string, 0)
conflicts := make([]string, 0)
elems := make([]batchtfile.TaskElement, 0, len(files))
type albumFile struct {
file tfile.TGFileMessage
storage storage.Storage
dirPath string
}
albumFiles := make(map[int64][]albumFile, 0)
for _, file := range files {
storName, matchedDirPath := applyRule(file)
storName, dirPath := applyRule(file)
fileStor := stor
if storName != stor.Name() && storName != "" {
fileStor, err = storage.GetStorageByUserIDAndName(ctx, user.ChatID, storName)
if err != nil {
logger.Errorf("Failed to get storage by user ID and name: %s", err)
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
ID: trackMsgID,
ID: trackMsgID,
Message: i18n.T(i18nk.BotMsgCommonErrorGetStorageFailed, map[string]any{
"Error": err.Error(),
}),
@@ -179,27 +150,16 @@ func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor st
return dispatcher.EndGroups
}
}
if !matchedDirPath.NeedNewForAlbum() {
storPath := path.Join(matchedDirPath.String(), file.Name())
if strategy == tcbdata.ConflictStrategyAsk || strategy == tcbdata.ConflictStrategySkip {
exists := fileStor.Exists(ctx, storPath)
if exists && strategy == tcbdata.ConflictStrategyAsk {
conflicts = append(conflicts, conflictutil.FormatPath(fileStor.Name(), storPath))
continue
}
if exists {
skipped = append(skipped, file.Name())
continue
}
}
if !dirPath.NeedNewForAlbum() {
storPath := fileStor.JoinStoragePath(path.Join(dirPath.String(), file.Name()))
elem, err := batchtfile.NewTaskElement(fileStor, storPath, file)
if err != nil {
logger.Errorf("Failed to create task element: %s", err)
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
ID: trackMsgID,
Message: i18n.T(i18nk.BotMsgCommonErrorTaskCreateFailed, map[string]any{
"Error": err.Error(),
}),
ID: trackMsgID,
Message: i18n.T(i18nk.BotMsgCommonErrorTaskCreateFailed, map[string]any{
"Error": err.Error(),
}),
})
return dispatcher.EndGroups
}
@@ -210,17 +170,12 @@ func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor st
logger.Warnf("File %s is not in a group, skipping album handling", file.Name())
continue
}
fileDirPath := matchedDirPath.String()
if matchedDirPath.NeedNewForAlbum() {
fileDirPath = dirPath
}
if _, ok := albumFiles[groupId]; !ok {
albumFiles[groupId] = make([]albumFile, 0)
}
albumFiles[groupId] = append(albumFiles[groupId], albumFile{
file: file,
storage: fileStor,
dirPath: fileDirPath,
})
}
}
@@ -233,23 +188,12 @@ func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor st
albumDir := strings.TrimSuffix(path.Base(afiles[0].file.Name()), path.Ext(afiles[0].file.Name()))
albumStor := afiles[0].storage
for _, af := range afiles {
afstorPath := path.Join(af.dirPath, albumDir, af.file.Name())
if strategy == tcbdata.ConflictStrategyAsk || strategy == tcbdata.ConflictStrategySkip {
exists := albumStor.Exists(ctx, afstorPath)
if exists && strategy == tcbdata.ConflictStrategyAsk {
conflicts = append(conflicts, conflictutil.FormatPath(albumStor.Name(), afstorPath))
continue
}
if exists {
skipped = append(skipped, af.file.Name())
continue
}
}
afstorPath := af.storage.JoinStoragePath(path.Join(dirPath, albumDir, af.file.Name()))
elem, err := batchtfile.NewTaskElement(albumStor, afstorPath, af.file)
if err != nil {
logger.Errorf("Failed to create task element for album file: %s", err)
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
ID: trackMsgID,
ID: trackMsgID,
Message: i18n.T(i18nk.BotMsgCommonErrorTaskCreateFailed, map[string]any{
"Error": err.Error(),
}),
@@ -260,30 +204,13 @@ func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor st
}
}
if strategy == tcbdata.ConflictStrategyAsk && len(conflicts) > 0 {
return promptTGFileConflictStrategy(ctx, userID, stor.Name(), dirPath, files, true, conflicts, trackMsgID)
}
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
if strategy == tcbdata.ConflictStrategyOverwrite {
injectCtx = storage.WithOverwrite(injectCtx)
}
if len(elems) == 0 {
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
ID: trackMsgID,
Message: i18n.T(i18nk.BotMsgCommonInfoAllConflictFilesSkipped, map[string]any{
"Skipped": strings.Join(skipped, "\n"),
}),
ReplyMarkup: nil,
})
return dispatcher.EndGroups
}
taskid := xid.New().String()
task := batchtfile.NewBatchTGFileTask(taskid, injectCtx, elems, batchtfile.NewProgressTrackerWithSkipped(trackMsgID, userID, skipped), true)
task := batchtfile.NewBatchTGFileTask(taskid, injectCtx, elems, batchtfile.NewProgressTracker(trackMsgID, userID), true)
if err := core.AddTask(injectCtx, task); err != nil {
logger.Errorf("Failed to add batch task: %s", err)
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
ID: trackMsgID,
ID: trackMsgID,
Message: i18n.T(i18nk.BotMsgCommonErrorTaskAddFailed, map[string]any{
"Error": err.Error(),
}),
@@ -292,47 +219,10 @@ func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor st
}
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
ID: trackMsgID,
Message: buildBatchAddedMessage(len(elems), skipped),
Message: i18n.T(i18nk.BotMsgCommonInfoBatchTasksAdded, map[string]any{
"Count": len(files),
}),
ReplyMarkup: nil,
})
return dispatcher.EndGroups
}
func promptTGFileConflictStrategy(ctx *ext.Context, userID int64, storageName, dirPath string, files []tfile.TGFileMessage, asBatch bool, conflicts []string, trackMsgID int) error {
markup, err := msgelem.BuildConflictStrategyMarkup(tcbdata.Add{
TaskType: tasktype.TaskTypeTgfiles,
SelectedStorName: storageName,
SettedDir: true,
SelectedDirPath: dirPath,
Files: files,
AsBatch: asBatch,
})
if err != nil {
return err
}
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
ID: trackMsgID,
Message: i18n.T(i18nk.BotMsgCommonPromptSelectConflictStrategy, map[string]any{"Files": conflictutil.FormatPaths(conflicts)}),
ReplyMarkup: markup,
})
return dispatcher.EndGroups
}
func selectedConflictStrategy(strategies []string) string {
if len(strategies) == 0 {
return ""
}
return strategies[0]
}
func buildBatchAddedMessage(count int, skipped []string) string {
if len(skipped) == 0 {
return i18n.T(i18nk.BotMsgCommonInfoBatchTasksAdded, map[string]any{
"Count": count,
})
}
return i18n.T(i18nk.BotMsgCommonInfoBatchTasksAddedWithSkipped, map[string]any{
"Count": count,
"Skipped": strings.Join(skipped, "\n"),
})
}

View File

@@ -25,21 +25,21 @@ func CreateAndAddtelegraphWithEdit(
pics []string,
stor storage.Storage,
trackMsgID int) error {
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
task := tphtask.NewTask(xid.New().String(),
injectCtx,
tphpage.Path,
pics,
stor,
dirPath,
stor.JoinStoragePath(dirPath),
tphutil.DefaultClient(),
tphtask.NewProgress(trackMsgID, userID),
)
if err := core.AddTask(injectCtx, task); err != nil {
log.FromContext(ctx).Errorf("Failed to add task: %s", err)
ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
ID: trackMsgID,
ID: trackMsgID,
Message: i18n.T(i18nk.BotMsgCommonErrorTaskAddFailed, map[string]any{
"Error": err.Error(),
}),

View File

@@ -15,7 +15,7 @@ import (
"github.com/krau/SaveAny-Bot/storage"
)
func CreateAndAddYtdlpTaskWithEdit(ctx *ext.Context, stor storage.Storage, dirPath string, urls []string, flags []string, msgID int, userID int64) error {
func CreateAndAddYtdlpTaskWithEdit(ctx *ext.Context, stor storage.Storage, dirPath string, urls []string, msgID int, userID int64) error {
logger := log.FromContext(ctx)
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
@@ -29,16 +29,15 @@ func CreateAndAddYtdlpTaskWithEdit(ctx *ext.Context, stor storage.Storage, dirPa
return dispatcher.EndGroups
}
logger.Infof("Creating yt-dlp task for %d URL(s) with %d flag(s)", len(urls), len(flags))
logger.Infof("Creating yt-dlp task for %d URL(s)", len(urls))
// Create yt-dlp task
task := ytdlp.NewTask(
xid.New().String(),
injectCtx,
urls,
flags,
stor,
dirPath,
stor.JoinStoragePath(dirPath),
ytdlp.NewProgress(msgID, userID),
)

View File

@@ -252,16 +252,6 @@ func listenMediaMessageEvent(ch chan userclient.MediaMessageEvent) {
logger.Errorf("Failed to get storage by user ID %d and name %s: %v", user.ChatID, user.DefaultStorage, err)
continue
}
// Resolve the default directory path from user.DefaultDir
var defaultDirPath string
if user.DefaultDir != 0 {
dir, err := database.GetDirByID(ctx, user.DefaultDir)
if err != nil {
logger.Warnf("Failed to get default dir for user %d: %v, using root", user.ChatID, err)
} else {
defaultDirPath = dir.Path
}
}
switch user.FilenameStrategy {
case fnamest.Message.String():
file.SetName(tgutil.GenFileNameFromMessage(*file.Message()))
@@ -296,14 +286,14 @@ func listenMediaMessageEvent(ch chan userclient.MediaMessageEvent) {
if needAlbumHandling {
// For media groups with NEW-FOR-ALBUM rule, collect all files of the same group
watchMediaGroupMgr.addFile(event.ChatID, user.ID, file, time.Duration(max(config.C().Telegram.MediaGroupTimeout, 1))*time.Second, func(files []tfile.TGFileMessage) {
processWatchMediaGroup(ctx, user, stor, defaultDirPath, files)
watchMediaGroupMgr.addFile(event.ChatID, user.ID, file, time.Duration(config.C().Telegram.MediaGroupTimeout)*time.Second, func(files []tfile.TGFileMessage) {
processWatchMediaGroup(ctx, user, stor, "", files)
})
continue
}
// Process single file or media group without album folder creation
dirPath := defaultDirPath
var dirPath string
if user.ApplyRule && user.Rules != nil {
matched, matchedStorageName, matchedDirPath := ruleutil.ApplyRule(ctx, user.Rules, ruleutil.NewInput(file))
if !matched {
@@ -319,7 +309,7 @@ func listenMediaMessageEvent(ch chan userclient.MediaMessageEvent) {
}
}
startCreateTask:
storagePath := path.Join(dirPath, file.Name())
storagePath := stor.JoinStoragePath(path.Join(dirPath, file.Name()))
injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
taskid := xid.New().String()
task, err := coretfile.NewTGFileTask(taskid, injectCtx, file, stor, storagePath, nil)
@@ -362,7 +352,6 @@ func processWatchMediaGroup(ctx *ext.Context, user *database.User, stor storage.
type albumFile struct {
file tfile.TGFileMessage
storage storage.Storage
dirPath string
}
albumFiles := make(map[int64][]albumFile)
@@ -385,11 +374,9 @@ func processWatchMediaGroup(ctx *ext.Context, user *database.User, stor storage.
continue
}
// Use the effective dirPath: if rule returns NEW-FOR-ALBUM sentinel, fall back to the
// base dirPath passed in (which is defaultDirPath from the caller).
effectiveDirPath := string(ruleDirPath)
if ruleDirPath.NeedNewForAlbum() {
effectiveDirPath = dirPath
if !ruleDirPath.NeedNewForAlbum() {
logger.Warnf("File %s does not need album folder, skipping", file.Name())
continue
}
if _, ok := albumFiles[groupId]; !ok {
@@ -398,7 +385,6 @@ func processWatchMediaGroup(ctx *ext.Context, user *database.User, stor storage.
albumFiles[groupId] = append(albumFiles[groupId], albumFile{
file: file,
storage: fileStor,
dirPath: effectiveDirPath,
})
}
@@ -417,7 +403,7 @@ func processWatchMediaGroup(ctx *ext.Context, user *database.User, stor storage.
logger.Infof("Creating album folder for group %d: %s with %d files", groupID, albumDir, len(afiles))
for _, af := range afiles {
afstorPath := path.Join(af.dirPath, albumDir, af.file.Name())
afstorPath := af.storage.JoinStoragePath(path.Join(dirPath, albumDir, af.file.Name()))
taskid := xid.New().String()
task, err := coretfile.NewTGFileTask(taskid, injectCtx, af.file, albumStor, afstorPath, nil)
if err != nil {

View File

@@ -7,6 +7,7 @@ import (
"github.com/celestix/gotgproto/dispatcher"
"github.com/celestix/gotgproto/ext"
"github.com/charmbracelet/log"
"github.com/duke-git/lancet/v2/slice"
"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/msgelem"
"github.com/krau/SaveAny-Bot/common/i18n"
@@ -24,59 +25,29 @@ func handleYtdlpCmd(ctx *ext.Context, update *ext.Update) error {
return dispatcher.EndGroups
}
// Separate URLs and flags from arguments
var urls []string
var flags []string
for i := 1; i < len(args); i++ {
arg := strings.TrimSpace(args[i])
if arg == "" {
continue
}
// Check if it's a flag (starts with - or --)
if strings.HasPrefix(arg, "-") {
flags = append(flags, arg)
// Check if the next argument might be a value for this flag
// Don't consume it if it starts with - or looks like a URL with scheme
if i+1 < len(args) {
nextArg := strings.TrimSpace(args[i+1])
if nextArg != "" && !strings.HasPrefix(nextArg, "-") {
// Check if it's clearly a URL (has ://)
// This handles common video URLs (http://, https://)
// For other yt-dlp inputs, users should ensure proper formatting
if strings.Contains(nextArg, "://") {
// It's a URL, don't consume it as a flag value
continue
}
// Otherwise, treat it as a flag value
flags = append(flags, nextArg)
i++ // Skip the next argument as it's been consumed
}
}
} else {
// Try to parse as URL
u, err := url.Parse(arg)
if err != nil || u.Scheme == "" || u.Host == "" {
logger.Warnf("Invalid URL: %s", arg)
continue
}
urls = append(urls, arg)
urls := args[1:]
// Validate and clean URLs
for i, link := range urls {
urls[i] = strings.TrimSpace(link)
u, err := url.Parse(link)
if err != nil || u.Scheme == "" || u.Host == "" {
logger.Warnf("Invalid URL: %s", link)
urls[i] = ""
}
}
urls = slice.Compact(urls)
if len(urls) == 0 {
ctx.Reply(update, ext.ReplyTextString(i18n.T(i18nk.BotMsgYtdlpErrorNoValidUrls)), nil)
return dispatcher.EndGroups
}
logger.Debugf("Preparing yt-dlp download for %d URL(s) with %d flag(s)", len(urls), len(flags))
logger.Debugf("Preparing yt-dlp download for %d URL(s)", len(urls))
// Build storage selection keyboard
markup, err := msgelem.BuildAddSelectStorageKeyboard(storage.GetUserStorages(ctx, update.GetUserChat().GetID()), tcbdata.Add{
TaskType: tasktype.TaskTypeYtdlp,
YtdlpURLs: urls,
YtdlpFlags: flags,
TaskType: tasktype.TaskTypeYtdlp,
YtdlpURLs: urls,
})
if err != nil {
return err

View File

@@ -1,129 +0,0 @@
package handlers
import (
"net/url"
"strings"
"testing"
)
// TestYtdlpArgumentParsing tests the URL and flag separation logic
func TestYtdlpArgumentParsing(t *testing.T) {
tests := []struct {
name string
input string
expectedURLs []string
expectedFlags []string
}{
{
name: "Single URL without flags",
input: "/ytdlp https://example.com/video",
expectedURLs: []string{"https://example.com/video"},
expectedFlags: []string{},
},
{
name: "Multiple URLs without flags",
input: "/ytdlp https://example.com/v1 https://example.com/v2",
expectedURLs: []string{"https://example.com/v1", "https://example.com/v2"},
expectedFlags: []string{},
},
{
name: "URL with format flag",
input: "/ytdlp --format best https://example.com/video",
expectedURLs: []string{"https://example.com/video"},
expectedFlags: []string{"--format", "best"},
},
{
name: "URL with extract-audio flag",
input: "/ytdlp --extract-audio --audio-format mp3 https://example.com/video",
expectedURLs: []string{"https://example.com/video"},
expectedFlags: []string{"--extract-audio", "--audio-format", "mp3"},
},
{
name: "Multiple URLs with flags",
input: "/ytdlp --format best https://example.com/v1 https://example.com/v2",
expectedURLs: []string{"https://example.com/v1", "https://example.com/v2"},
expectedFlags: []string{"--format", "best"},
},
{
name: "Flags mixed with URLs",
input: "/ytdlp https://example.com/v1 --format best https://example.com/v2",
expectedURLs: []string{"https://example.com/v1", "https://example.com/v2"},
expectedFlags: []string{"--format", "best"},
},
{
name: "Short flag",
input: "/ytdlp -f best https://example.com/video",
expectedURLs: []string{"https://example.com/video"},
expectedFlags: []string{"-f", "best"},
},
{
name: "Boolean flag",
input: "/ytdlp --extract-audio https://example.com/video",
expectedURLs: []string{"https://example.com/video"},
expectedFlags: []string{"--extract-audio"},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
args := strings.Split(tt.input, " ")
// Simulate the parsing logic from handleYtdlpCmd
var urls []string
var flags []string
for i := 1; i < len(args); i++ {
arg := strings.TrimSpace(args[i])
if arg == "" {
continue
}
// Check if it's a flag (starts with - or --)
if strings.HasPrefix(arg, "-") {
flags = append(flags, arg)
// Check if the next argument might be a value for this flag
if i+1 < len(args) {
nextArg := strings.TrimSpace(args[i+1])
if nextArg != "" && !strings.HasPrefix(nextArg, "-") {
// Check if it's clearly a URL (has ://)
if strings.Contains(nextArg, "://") {
// It's a URL, don't consume it as a flag value
continue
}
// Otherwise, treat it as a flag value
flags = append(flags, nextArg)
i++ // Skip the next argument as it's been consumed
}
}
} else {
// Try to parse as URL
u, err := url.Parse(arg)
if err != nil || u.Scheme == "" || u.Host == "" {
continue
}
urls = append(urls, arg)
}
}
// Verify URLs
if len(urls) != len(tt.expectedURLs) {
t.Errorf("Expected %d URLs, got %d", len(tt.expectedURLs), len(urls))
}
for i, expectedURL := range tt.expectedURLs {
if i >= len(urls) || urls[i] != expectedURL {
t.Errorf("Expected URL[%d] to be '%s', got '%s'", i, expectedURL, urls[i])
}
}
// Verify flags
if len(flags) != len(tt.expectedFlags) {
t.Errorf("Expected %d flags, got %d", len(tt.expectedFlags), len(flags))
}
for i, expectedFlag := range tt.expectedFlags {
if i >= len(flags) || flags[i] != expectedFlag {
t.Errorf("Expected flag[%d] to be '%s', got '%s'", i, expectedFlag, flags[i])
}
}
})
}
}

View File

@@ -15,7 +15,7 @@ import (
// https://github.com/iyear/tdl/blob/master/core/tclient/tclient.go
func NewDefaultMiddlewares(ctx context.Context, timeout time.Duration) []telegram.Middleware {
return []telegram.Middleware{
recovery.New(ctx, func() backoff.BackOff { return newBackoff(timeout) }),
recovery.New(ctx, newBackoff(timeout)),
retry.New(config.C().Telegram.RpcRetry),
floodwait.NewSimpleWaiter(),
}

View File

@@ -14,28 +14,19 @@ import (
)
type recovery struct {
ctx context.Context
newBackoff func() backoff.BackOff
ctx context.Context
backoff backoff.BackOff
}
// New returns a recovery middleware.
//
// newBackoff is a factory that must return a fresh backoff.BackOff on every call: backoff implementations in
// cenkalti/backoff/v4 (notably ExponentialBackOff) are not safe for concurrent
// use, and the Telegram client invokes RPCs from many goroutines in parallel.
//
// Sharing a single instance corrupts its internal counters, breaks the
// exponential interval, and defeats MaxElapsedTime - see issue #218.
func New(ctx context.Context, newBackoff func() backoff.BackOff) telegram.Middleware {
func New(ctx context.Context, backoff backoff.BackOff) telegram.Middleware {
return &recovery{
ctx: ctx,
newBackoff: newBackoff,
ctx: ctx,
backoff: backoff,
}
}
func (r *recovery) Handle(next tg.Invoker) telegram.InvokeFunc {
return func(ctx context.Context, input bin.Encoder, output bin.Decoder) error {
b := r.newBackoff()
return backoff.RetryNotify(func() error {
if err := next.Invoke(ctx, input, output); err != nil {
@@ -47,7 +38,7 @@ func (r *recovery) Handle(next tg.Invoker) telegram.InvokeFunc {
}
return nil
}, b, func(err error, duration time.Duration) {
}, r.backoff, func(err error, duration time.Duration) {
log.FromContext(ctx).Debug("Wait for connection recovery", "error", err, "duration", duration)
})
}

View File

@@ -5,7 +5,6 @@ import (
"fmt"
"github.com/krau/SaveAny-Bot/cmd/upload"
"github.com/krau/SaveAny-Bot/cmd/watch"
"github.com/krau/SaveAny-Bot/config"
"github.com/spf13/cobra"
)
@@ -19,7 +18,6 @@ var rootCmd = &cobra.Command{
func init() {
config.RegisterFlags(rootCmd)
upload.Register(rootCmd)
watch.Register(rootCmd)
}
func Execute(ctx context.Context) {

View File

@@ -2,9 +2,9 @@ package cmd
import (
"context"
"fmt"
"os"
"path/filepath"
"strings"
"time"
"slices"
@@ -27,27 +27,14 @@ import (
func Run(cmd *cobra.Command, _ []string) {
ctx, cancel := context.WithCancel(cmd.Context())
logger := log.NewWithOptions(os.Stdout, log.Options{
Level: log.InfoLevel,
Level: log.DebugLevel,
ReportTimestamp: true,
TimeFormat: time.TimeOnly,
ReportCaller: true,
})
log.SetDefault(logger)
ctx = log.WithContext(ctx, logger)
configFile := config.GetConfigFile(cmd)
if err := config.Init(ctx, configFile); err != nil {
logger.Fatal("Init failed", "error", err)
}
level, err := log.ParseLevel(strings.TrimSpace(config.C().Log.Level))
if err != nil {
logger.Warn("Invalid log level, fallback to debug", "level", config.C().Log.Level, "error", err)
level = log.DebugLevel
}
logger.SetLevel(level)
exitChan, err := initAll(ctx)
exitChan, err := initAll(ctx, cmd)
if err != nil {
logger.Fatal("Init failed", "error", err)
}
@@ -64,7 +51,11 @@ func Run(cmd *cobra.Command, _ []string) {
cleanCache()
}
func initAll(ctx context.Context) (<-chan struct{}, error) {
func initAll(ctx context.Context, cmd *cobra.Command) (<-chan struct{}, error) {
configFile := config.GetConfigFile(cmd)
if err := config.Init(ctx, configFile); err != nil {
return nil, fmt.Errorf("failed to load config: %w", err)
}
cache.Init()
logger := log.FromContext(ctx)
i18n.Init(config.C().Lang)
@@ -86,10 +77,11 @@ func initAll(ctx context.Context) (<-chan struct{}, error) {
logger.Fatal("User login failed", "error", err)
}
}
if err := api.Start(ctx); err != nil {
logger.Error("Failed to start API server", "error", err)
exitChan := bot.Init(ctx)
if err := api.Init(ctx); err != nil {
return nil, fmt.Errorf("failed to init API server: %w", err)
}
return bot.Init(ctx), nil
return exitChan, nil
}
func cleanCache() {

View File

@@ -90,7 +90,7 @@ func Upload(cmd *cobra.Command, args []string) error {
fileName := fileInfo.Name()
fileSize := fileInfo.Size()
uploadPath := path.Join(dirPath, fileName)
uploadPath := stor.JoinStoragePath(path.Join(dirPath, fileName))
ctx = context.WithValue(ctx, ctxkey.ContentLength, fileSize)
ctx = tgutil.ExtWithContext(ctx, bot.ExtContext())

View File

@@ -1,145 +0,0 @@
package watch
import (
"fmt"
"time"
"github.com/charmbracelet/log"
"github.com/krau/SaveAny-Bot/client/bot"
"github.com/krau/SaveAny-Bot/common/cache"
"github.com/krau/SaveAny-Bot/common/i18n"
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
"github.com/krau/SaveAny-Bot/config"
"github.com/krau/SaveAny-Bot/database"
stortype "github.com/krau/SaveAny-Bot/pkg/enums/storage"
"github.com/krau/SaveAny-Bot/storage"
"github.com/spf13/cobra"
)
var watchCmd = &cobra.Command{
Use: "watch",
Short: "watch a local directory and auto-upload changed files to storage",
Long: `Watch a local directory and automatically upload created or modified files
to the specified storage backend, preserving the relative directory structure.
Example:
saveany-bot watch -p /data/inbox -s mystorage -d backup --recursive`,
RunE: runWatch,
}
func Register(root *cobra.Command) {
flags := watchCmd.Flags()
flags.StringP("path", "p", "", "local directory path to watch")
watchCmd.MarkFlagRequired("path")
flags.StringP("storage", "s", "", "storage name to upload to")
watchCmd.MarkFlagRequired("storage")
flags.StringP("dir", "d", "", "storage dir to upload to, default is the base_path of the storage")
flags.BoolP("recursive", "r", false, "watch subdirectories recursively")
flags.Bool("overwrite", false, "overwrite existing files on storage instead of skipping")
flags.Bool("initial-scan", false, "upload existing files in the directory on startup")
flags.Duration("debounce", 2*time.Second, "wait time after the last change before uploading a file")
flags.Int("upload-workers", 0, "number of concurrent uploads, default is config.workers")
flags.Duration("retry-delay", 3*time.Second, "delay between upload retries")
root.AddCommand(watchCmd)
}
func runWatch(cmd *cobra.Command, _ []string) error {
watchPath, err := cmd.Flags().GetString("path")
if err != nil {
return err
}
storName, err := cmd.Flags().GetString("storage")
if err != nil {
return err
}
destDir, err := cmd.Flags().GetString("dir")
if err != nil {
return err
}
recursive, err := cmd.Flags().GetBool("recursive")
if err != nil {
return err
}
overwrite, err := cmd.Flags().GetBool("overwrite")
if err != nil {
return err
}
initialScan, err := cmd.Flags().GetBool("initial-scan")
if err != nil {
return err
}
debounce, err := cmd.Flags().GetDuration("debounce")
if err != nil {
return err
}
uploadWorkers, err := cmd.Flags().GetInt("upload-workers")
if err != nil {
return err
}
retryDelay, err := cmd.Flags().GetDuration("retry-delay")
if err != nil {
return err
}
ctx := cmd.Context()
logger := log.FromContext(ctx)
configFile := config.GetConfigFile(cmd)
if err := config.Init(ctx, configFile); err != nil {
return fmt.Errorf("failed to load config: %w", err)
}
i18n.Init(config.C().Lang)
cache.Init()
database.Init(ctx)
stor, err := storage.GetStorageByName(ctx, storName)
if err != nil {
return fmt.Errorf("failed to get storage %q: %w", storName, err)
}
// Telegram storage needs the bot client and its ext context injected into ctx.
if stor.Type() == stortype.Telegram {
bot.Init(ctx)
ctx = tgutil.ExtWithContext(ctx, bot.ExtContext())
}
if uploadWorkers < 1 {
uploadWorkers = config.C().Workers
}
uploader := NewUploader(ctx, UploaderOptions{
Storage: stor,
DestDir: destDir,
Overwrite: overwrite,
Workers: uploadWorkers,
Retry: config.C().Retry,
RetryDelay: retryDelay,
})
watcher, err := NewWatcher(ctx, WatcherOptions{
Root: watchPath,
Recursive: recursive,
Debounce: debounce,
Uploader: uploader,
})
if err != nil {
uploader.Close()
return fmt.Errorf("failed to create watcher: %w", err)
}
if initialScan {
watcher.ScanExisting(ctx)
}
logger.Infof("watch started: %s -> storage %q dir %q", watchPath, storName, destDir)
// Run blocks until ctx is cancelled (e.g. SIGINT).
runErr := watcher.Run(ctx)
// Wait for in-flight uploads to finish before exiting.
logger.Info("waiting for in-flight uploads to finish...")
uploader.Close()
logger.Info("watch stopped")
return runErr
}

View File

@@ -1,227 +0,0 @@
package watch
import (
"context"
"fmt"
"os"
"path"
"path/filepath"
"sync"
"time"
"github.com/charmbracelet/log"
"github.com/krau/SaveAny-Bot/pkg/enums/ctxkey"
"github.com/krau/SaveAny-Bot/storage"
)
type uploadJob struct {
// localPath is the absolute path of the local file.
localPath string
// relPath is relative to the watch root, used to preserve directory structure on storage.
relPath string
}
// Uploader uploads local files to the target storage via a worker pool.
// If a file changes while being uploaded, it is re-uploaded once after the
// current upload finishes, instead of being queued multiple times.
type Uploader struct {
stor storage.Storage
destDir string
overwrite bool
retry int
retryDelay time.Duration
logger *log.Logger
jobs chan uploadJob
wg sync.WaitGroup
mu sync.Mutex
// inflight maps in-progress (or queued) file paths. A true value means the
// file changed during upload and must be re-queued once done.
inflight map[string]bool
}
type UploaderOptions struct {
Storage storage.Storage
DestDir string
Overwrite bool
Workers int
Retry int
RetryDelay time.Duration
QueueSize int
}
// NewUploader creates and starts an Uploader. The caller must call Close when done.
func NewUploader(ctx context.Context, opts UploaderOptions) *Uploader {
if opts.Workers < 1 {
opts.Workers = 1
}
if opts.Retry < 1 {
opts.Retry = 1
}
if opts.RetryDelay <= 0 {
opts.RetryDelay = 3 * time.Second
}
if opts.QueueSize < opts.Workers {
opts.QueueSize = opts.Workers * 64
}
u := &Uploader{
stor: opts.Storage,
destDir: opts.DestDir,
overwrite: opts.Overwrite,
retry: opts.Retry,
retryDelay: opts.RetryDelay,
logger: log.FromContext(ctx).WithPrefix("uploader"),
jobs: make(chan uploadJob, opts.QueueSize),
inflight: make(map[string]bool),
}
for i := 0; i < opts.Workers; i++ {
u.wg.Add(1)
go u.worker(ctx)
}
return u
}
// Submit enqueues an upload job. If the same file is already in flight, it is
// marked for re-upload instead of being queued again. Returns false if ctx is
// cancelled before the job can be enqueued.
func (u *Uploader) Submit(ctx context.Context, job uploadJob) bool {
u.mu.Lock()
if _, ok := u.inflight[job.localPath]; ok {
u.inflight[job.localPath] = true
u.mu.Unlock()
u.logger.Debugf("file %s already in flight, marked for re-upload", job.localPath)
return true
}
u.inflight[job.localPath] = false
u.mu.Unlock()
select {
case u.jobs <- job:
return true
case <-ctx.Done():
u.mu.Lock()
delete(u.inflight, job.localPath)
u.mu.Unlock()
return false
}
}
func (u *Uploader) worker(ctx context.Context) {
defer u.wg.Done()
for {
select {
case <-ctx.Done():
return
case job, ok := <-u.jobs:
if !ok {
return
}
u.process(ctx, job)
}
}
}
func (u *Uploader) process(ctx context.Context, job uploadJob) {
if err := u.uploadWithRetry(ctx, job); err != nil {
if ctx.Err() != nil {
u.clearInflight(job.localPath)
return
}
u.logger.Errorf("failed to upload %s after %d attempt(s): %v", job.localPath, u.retry, err)
}
// Re-queue if the file changed again while it was being uploaded.
u.mu.Lock()
needReupload := u.inflight[job.localPath]
if needReupload {
u.inflight[job.localPath] = false
} else {
delete(u.inflight, job.localPath)
}
u.mu.Unlock()
if needReupload {
select {
case u.jobs <- job:
u.logger.Debugf("re-queued %s due to changes during upload", job.localPath)
case <-ctx.Done():
u.clearInflight(job.localPath)
}
}
}
func (u *Uploader) clearInflight(localPath string) {
u.mu.Lock()
delete(u.inflight, localPath)
u.mu.Unlock()
}
func (u *Uploader) uploadWithRetry(ctx context.Context, job uploadJob) error {
var lastErr error
for attempt := 1; attempt <= u.retry; attempt++ {
if ctx.Err() != nil {
return ctx.Err()
}
err := u.upload(ctx, job)
if err == nil {
return nil
}
if ctx.Err() != nil {
return ctx.Err()
}
lastErr = err
u.logger.Warnf("upload %s failed (attempt %d/%d): %v", job.localPath, attempt, u.retry, err)
if attempt < u.retry {
select {
case <-time.After(u.retryDelay):
case <-ctx.Done():
return ctx.Err()
}
}
}
return lastErr
}
func (u *Uploader) upload(ctx context.Context, job uploadJob) error {
file, err := os.Open(filepath.Clean(job.localPath))
if err != nil {
return fmt.Errorf("failed to open file: %w", err)
}
defer file.Close()
info, err := file.Stat()
if err != nil {
return fmt.Errorf("failed to stat file: %w", err)
}
if info.IsDir() {
return fmt.Errorf("path is a directory, not a file")
}
// Keep the relative directory structure on the storage side.
storagePath := path.Join(u.destDir, filepath.ToSlash(job.relPath))
uploadCtx := context.WithValue(ctx, ctxkey.ContentLength, info.Size())
if u.overwrite {
uploadCtx = storage.WithOverwrite(uploadCtx)
} else if u.stor.Exists(uploadCtx, storagePath) {
u.logger.Infof("skip existing file: %s", storagePath)
return nil
}
u.logger.Infof("uploading %s -> %s (%d bytes)", job.localPath, storagePath, info.Size())
if err := u.stor.Save(uploadCtx, file, storagePath); err != nil {
return fmt.Errorf("failed to save to storage: %w", err)
}
u.logger.Infof("uploaded %s", storagePath)
return nil
}
// Close stops accepting jobs and waits for in-flight uploads to finish.
func (u *Uploader) Close() {
close(u.jobs)
u.wg.Wait()
}

View File

@@ -1,269 +0,0 @@
package watch
import (
"context"
"fmt"
"io/fs"
"os"
"path/filepath"
"sync"
"time"
"github.com/charmbracelet/log"
"github.com/fsnotify/fsnotify"
)
// Watcher watches a local directory and submits stable files to the Uploader.
//
// Write-completion detection: fsnotify emits Write events throughout a write.
// Watcher debounces per file and only uploads once the file size stays
// unchanged across a debounce window, avoiding uploads of partial files.
type Watcher struct {
root string
recursive bool
debounce time.Duration
uploader *Uploader
logger *log.Logger
fsw *fsnotify.Watcher
mu sync.Mutex
pending map[string]*time.Timer
// lastSize is the last observed file size, used to detect a stable write.
lastSize map[string]int64
}
type WatcherOptions struct {
Root string
Recursive bool
Debounce time.Duration
Uploader *Uploader
}
// NewWatcher creates a Watcher.
func NewWatcher(ctx context.Context, opts WatcherOptions) (*Watcher, error) {
if opts.Debounce <= 0 {
opts.Debounce = 2 * time.Second
}
root, err := filepath.Abs(opts.Root)
if err != nil {
return nil, fmt.Errorf("failed to resolve root path: %w", err)
}
info, err := os.Stat(root)
if err != nil {
return nil, fmt.Errorf("failed to stat root path: %w", err)
}
if !info.IsDir() {
return nil, fmt.Errorf("watch path must be a directory: %s", root)
}
fsw, err := fsnotify.NewWatcher()
if err != nil {
return nil, fmt.Errorf("failed to create fsnotify watcher: %w", err)
}
w := &Watcher{
root: root,
recursive: opts.Recursive,
debounce: opts.Debounce,
uploader: opts.Uploader,
logger: log.FromContext(ctx).WithPrefix("watcher"),
fsw: fsw,
pending: make(map[string]*time.Timer),
lastSize: make(map[string]int64),
}
return w, nil
}
// Run starts watching and blocks until ctx is cancelled.
func (w *Watcher) Run(ctx context.Context) error {
if err := w.addDir(w.root); err != nil {
w.fsw.Close()
return fmt.Errorf("failed to watch root: %w", err)
}
w.logger.Infof("watching %s (recursive=%v, debounce=%s)", w.root, w.recursive, w.debounce)
defer w.cleanup()
for {
select {
case <-ctx.Done():
w.logger.Info("stopping watcher")
return nil
case event, ok := <-w.fsw.Events:
if !ok {
return nil
}
w.handleEvent(ctx, event)
case err, ok := <-w.fsw.Errors:
if !ok {
return nil
}
w.logger.Errorf("watch error: %v", err)
}
}
}
func (w *Watcher) handleEvent(ctx context.Context, event fsnotify.Event) {
// Remove/Rename: cancel any pending upload for this path.
if event.Has(fsnotify.Remove) || event.Has(fsnotify.Rename) {
w.cancelPending(event.Name)
return
}
if !event.Has(fsnotify.Create) && !event.Has(fsnotify.Write) {
return
}
info, err := os.Stat(event.Name)
if err != nil {
// File may have been removed or moved; ignore.
return
}
if info.IsDir() {
// New directory: watch it recursively and scan files already inside.
if event.Has(fsnotify.Create) && w.recursive {
if err := w.addDir(event.Name); err != nil {
w.logger.Errorf("failed to watch new dir %s: %v", event.Name, err)
}
w.scanExisting(ctx, event.Name)
}
return
}
w.scheduleUpload(ctx, event.Name)
}
// scheduleUpload schedules a debounced upload for a file.
func (w *Watcher) scheduleUpload(ctx context.Context, file string) {
w.mu.Lock()
defer w.mu.Unlock()
if t, ok := w.pending[file]; ok {
t.Stop()
}
w.pending[file] = time.AfterFunc(w.debounce, func() {
w.maybeUpload(ctx, file)
})
}
// maybeUpload submits the upload once the debounce window passes and the file
// size is stable; otherwise it waits another window.
func (w *Watcher) maybeUpload(ctx context.Context, file string) {
if ctx.Err() != nil {
return
}
info, err := os.Stat(file)
if err != nil {
w.cancelPending(file)
return
}
if info.IsDir() {
w.cancelPending(file)
return
}
w.mu.Lock()
prevSize, seen := w.lastSize[file]
curSize := info.Size()
if !seen || prevSize != curSize {
// Size still changing: likely still being written, wait another window.
w.lastSize[file] = curSize
w.pending[file] = time.AfterFunc(w.debounce, func() {
w.maybeUpload(ctx, file)
})
w.mu.Unlock()
return
}
// Size stable: treat write as complete.
delete(w.pending, file)
delete(w.lastSize, file)
w.mu.Unlock()
relPath, err := filepath.Rel(w.root, file)
if err != nil {
w.logger.Errorf("failed to compute relative path for %s: %v", file, err)
return
}
w.uploader.Submit(ctx, uploadJob{localPath: file, relPath: relPath})
}
func (w *Watcher) cancelPending(file string) {
w.mu.Lock()
defer w.mu.Unlock()
if t, ok := w.pending[file]; ok {
t.Stop()
delete(w.pending, file)
}
delete(w.lastSize, file)
}
// addDir adds a directory to the watch list, recursively when enabled.
func (w *Watcher) addDir(dir string) error {
if !w.recursive {
return w.fsw.Add(dir)
}
return filepath.WalkDir(dir, func(p string, d fs.DirEntry, err error) error {
if err != nil {
w.logger.Warnf("skip path %s: %v", p, err)
return nil
}
if d.IsDir() {
if addErr := w.fsw.Add(p); addErr != nil {
w.logger.Warnf("failed to watch dir %s: %v", p, addErr)
} else {
w.logger.Debugf("watching dir %s", p)
}
}
return nil
})
}
// scanExisting submits files already present under dir (initial sync and new-dir backfill).
func (w *Watcher) scanExisting(ctx context.Context, dir string) {
walkFn := func(p string, d fs.DirEntry, err error) error {
if err != nil {
w.logger.Warnf("skip path %s: %v", p, err)
return nil
}
if ctx.Err() != nil {
return ctx.Err()
}
if d.IsDir() {
if !w.recursive && p != dir {
return fs.SkipDir
}
return nil
}
relPath, relErr := filepath.Rel(w.root, p)
if relErr != nil {
w.logger.Errorf("failed to compute relative path for %s: %v", p, relErr)
return nil
}
w.uploader.Submit(ctx, uploadJob{localPath: p, relPath: relPath})
return nil
}
if err := filepath.WalkDir(dir, walkFn); err != nil && ctx.Err() == nil {
w.logger.Errorf("failed to scan dir %s: %v", dir, err)
}
}
// ScanExisting triggers a one-time scan and upload of existing files under the watch root.
func (w *Watcher) ScanExisting(ctx context.Context) {
w.logger.Info("scanning existing files for initial sync")
w.scanExisting(ctx, w.root)
}
func (w *Watcher) cleanup() {
w.mu.Lock()
for _, t := range w.pending {
t.Stop()
}
w.pending = make(map[string]*time.Timer)
w.lastSize = make(map[string]int64)
w.mu.Unlock()
w.fsw.Close()
}

View File

@@ -21,7 +21,6 @@ const (
BotMsgCmdDl Key = "bot.msg.cmd.dl"
BotMsgCmdFnametmpl Key = "bot.msg.cmd.fnametmpl"
BotMsgCmdHelp Key = "bot.msg.cmd.help"
BotMsgCmdImport Key = "bot.msg.cmd.import"
BotMsgCmdLswatch Key = "bot.msg.cmd.lswatch"
BotMsgCmdParser Key = "bot.msg.cmd.parser"
BotMsgCmdRule Key = "bot.msg.cmd.rule"
@@ -31,14 +30,10 @@ const (
BotMsgCmdStorage Key = "bot.msg.cmd.storage"
BotMsgCmdSyncpeers Key = "bot.msg.cmd.syncpeers"
BotMsgCmdTask Key = "bot.msg.cmd.task"
BotMsgCmdTransfer Key = "bot.msg.cmd.transfer"
BotMsgCmdUnwatch Key = "bot.msg.cmd.unwatch"
BotMsgCmdUpdate Key = "bot.msg.cmd.update"
BotMsgCmdWatch Key = "bot.msg.cmd.watch"
BotMsgCmdYtdlp Key = "bot.msg.cmd.ytdlp"
BotMsgCommonButtonConflictOverwrite Key = "bot.msg.common.button_conflict_overwrite"
BotMsgCommonButtonConflictRename Key = "bot.msg.common.button_conflict_rename"
BotMsgCommonButtonConflictSkip Key = "bot.msg.common.button_conflict_skip"
BotMsgCommonCancelButtonText Key = "bot.msg.common.cancel_button_text"
BotMsgCommonErrorBuildDirSelectKeyboardFailed Key = "bot.msg.common.error_build_dir_select_keyboard_failed"
BotMsgCommonErrorBuildStorageSelectKeyboardFailed Key = "bot.msg.common.error_build_storage_select_keyboard_failed"
@@ -66,10 +61,7 @@ const (
BotMsgCommonErrorTaskAddFailed Key = "bot.msg.common.error_task_add_failed"
BotMsgCommonErrorTaskCreateFailed Key = "bot.msg.common.error_task_create_failed"
BotMsgCommonErrorUpdateUserInfoFailed Key = "bot.msg.common.error_update_user_info_failed"
BotMsgCommonInfoAllConflictFilesSkipped Key = "bot.msg.common.info_all_conflict_files_skipped"
BotMsgCommonInfoBatchTasksAdded Key = "bot.msg.common.info_batch_tasks_added"
BotMsgCommonInfoBatchTasksAddedWithSkipped Key = "bot.msg.common.info_batch_tasks_added_with_skipped"
BotMsgCommonInfoConflictFilesSkipped Key = "bot.msg.common.info_conflict_files_skipped"
BotMsgCommonInfoDefaultStorageSet Key = "bot.msg.common.info_default_storage_set"
BotMsgCommonInfoDefaultStorageWithDirSet Key = "bot.msg.common.info_default_storage_with_dir_set"
BotMsgCommonInfoFetchingFileInfo Key = "bot.msg.common.info_fetching_file_info"
@@ -79,25 +71,16 @@ const (
BotMsgCommonInfoSilentModeOff Key = "bot.msg.common.info_silent_mode_off"
BotMsgCommonInfoSilentModeOn Key = "bot.msg.common.info_silent_mode_on"
BotMsgCommonInfoTaskAdded Key = "bot.msg.common.info_task_added"
BotMsgCommonPromptConflictMoreFiles Key = "bot.msg.common.prompt_conflict_more_files"
BotMsgCommonPromptSelectConflictStrategy Key = "bot.msg.common.prompt_select_conflict_strategy"
BotMsgCommonPromptSelectDefaultDir Key = "bot.msg.common.prompt_select_default_dir"
BotMsgCommonPromptSelectDefaultStorage Key = "bot.msg.common.prompt_select_default_storage"
BotMsgCommonPromptSelectDir Key = "bot.msg.common.prompt_select_dir"
BotMsgConfigButtonConflictStrategy Key = "bot.msg.config.button_conflict_strategy"
BotMsgConfigButtonFilenameStrategy Key = "bot.msg.config.button_filename_strategy"
BotMsgConfigConflictStrategyAsk Key = "bot.msg.config.conflict_strategy_ask"
BotMsgConfigConflictStrategyOverwrite Key = "bot.msg.config.conflict_strategy_overwrite"
BotMsgConfigConflictStrategyRename Key = "bot.msg.config.conflict_strategy_rename"
BotMsgConfigConflictStrategySkip Key = "bot.msg.config.conflict_strategy_skip"
BotMsgConfigErrorInvalidCallbackData Key = "bot.msg.config.error_invalid_callback_data"
BotMsgConfigErrorInvalidTemplate Key = "bot.msg.config.error_invalid_template"
BotMsgConfigFnametmplHelp Key = "bot.msg.config.fnametmpl_help"
BotMsgConfigInfoConflictStrategySet Key = "bot.msg.config.info_conflict_strategy_set"
BotMsgConfigInfoCurrentTemplatePrefix Key = "bot.msg.config.info_current_template_prefix"
BotMsgConfigInfoFilenameStrategySet Key = "bot.msg.config.info_filename_strategy_set"
BotMsgConfigInfoTemplateUpdated Key = "bot.msg.config.info_template_updated"
BotMsgConfigPromptSelectConflictStrategy Key = "bot.msg.config.prompt_select_conflict_strategy"
BotMsgConfigPromptSelectFilenameStrategy Key = "bot.msg.config.prompt_select_filename_strategy"
BotMsgConfigPromptSelectOption Key = "bot.msg.config.prompt_select_option"
BotMsgDirButtonDefault Key = "bot.msg.dir.button_default"
@@ -178,20 +161,6 @@ const (
BotMsgProgressTelegraphProgressPrefix Key = "bot.msg.progress.telegraph_progress_prefix"
BotMsgProgressTelegraphStartPrefix Key = "bot.msg.progress.telegraph_start_prefix"
BotMsgProgressTotalSizePrefix Key = "bot.msg.progress.total_size_prefix"
BotMsgProgressTransferAvgSpeedPrefix Key = "bot.msg.progress.transfer_avg_speed_prefix"
BotMsgProgressTransferElapsedTimePrefix Key = "bot.msg.progress.transfer_elapsed_time_prefix"
BotMsgProgressTransferFailedFilesPrefix Key = "bot.msg.progress.transfer_failed_files_prefix"
BotMsgProgressTransferFailedPrefix Key = "bot.msg.progress.transfer_failed_prefix"
BotMsgProgressTransferProcessingMore Key = "bot.msg.progress.transfer_processing_more"
BotMsgProgressTransferProcessingPrefix Key = "bot.msg.progress.transfer_processing_prefix"
BotMsgProgressTransferProgressPrefix Key = "bot.msg.progress.transfer_progress_prefix"
BotMsgProgressTransferRemainingTimePrefix Key = "bot.msg.progress.transfer_remaining_time_prefix"
BotMsgProgressTransferSpeedPrefix Key = "bot.msg.progress.transfer_speed_prefix"
BotMsgProgressTransferStartPrefix Key = "bot.msg.progress.transfer_start_prefix"
BotMsgProgressTransferSuccessPrefix Key = "bot.msg.progress.transfer_success_prefix"
BotMsgProgressTransferTotalFilesPrefix Key = "bot.msg.progress.transfer_total_files_prefix"
BotMsgProgressTransferTotalSizePrefix Key = "bot.msg.progress.transfer_total_size_prefix"
BotMsgProgressTransferUploadedPrefix Key = "bot.msg.progress.transfer_uploaded_prefix"
BotMsgProgressYtdlpDone Key = "bot.msg.progress.ytdlp_done"
BotMsgProgressYtdlpDownloading Key = "bot.msg.progress.ytdlp_downloading"
BotMsgProgressYtdlpStart Key = "bot.msg.progress.ytdlp_start"
@@ -200,7 +169,6 @@ const (
BotMsgRuleErrorGetUserRulesFailed Key = "bot.msg.rule.error_get_user_rules_failed"
BotMsgRuleErrorInvalidRuleId Key = "bot.msg.rule.error_invalid_rule_id"
BotMsgRuleErrorInvalidRuleType Key = "bot.msg.rule.error_invalid_rule_type"
BotMsgRuleErrorStorageNotFound Key = "bot.msg.rule.error_storage_not_found"
BotMsgRuleErrorUpdateUserFailed Key = "bot.msg.rule.error_update_user_failed"
BotMsgRuleHelpAddSuffix Key = "bot.msg.rule.help_add_suffix"
BotMsgRuleHelpAvailableOps Key = "bot.msg.rule.help_available_ops"
@@ -208,16 +176,13 @@ const (
BotMsgRuleHelpCurrentModeEnabled Key = "bot.msg.rule.help_current_mode_enabled"
BotMsgRuleHelpDelSuffix Key = "bot.msg.rule.help_del_suffix"
BotMsgRuleHelpExistingRulesPrefix Key = "bot.msg.rule.help_existing_rules_prefix"
BotMsgRuleHelpPresetSuffix Key = "bot.msg.rule.help_preset_suffix"
BotMsgRuleHelpSwitchSuffix Key = "bot.msg.rule.help_switch_suffix"
BotMsgRuleHelpUsage Key = "bot.msg.rule.help_usage"
BotMsgRuleInfoCreateRuleSuccess Key = "bot.msg.rule.info_create_rule_success"
BotMsgRuleInfoDeleteRuleSuccess Key = "bot.msg.rule.info_delete_rule_success"
BotMsgRuleInfoPresetImported Key = "bot.msg.rule.info_preset_imported"
BotMsgRuleInfoRuleModeDisabled Key = "bot.msg.rule.info_rule_mode_disabled"
BotMsgRuleInfoRuleModeEnabled Key = "bot.msg.rule.info_rule_mode_enabled"
BotMsgRulePromptProvideRuleId Key = "bot.msg.rule.prompt_provide_rule_id"
BotMsgRulePromptProvideStorageName Key = "bot.msg.rule.prompt_provide_storage_name"
BotMsgSaveErrorInvalidIdOrUsername Key = "bot.msg.save.error_invalid_id_or_username"
BotMsgSaveHelpText Key = "bot.msg.save_help_text"
BotMsgStorageInfoFilenamePrefix Key = "bot.msg.storage.info_filename_prefix"
@@ -251,22 +216,6 @@ const (
BotMsgTelegraphInfoPicCountPrefix Key = "bot.msg.telegraph.info_pic_count_prefix"
BotMsgTelegraphInfoPromptSelectStorage Key = "bot.msg.telegraph.info_prompt_select_storage"
BotMsgTelegraphInfoTitlePrefix Key = "bot.msg.telegraph.info_title_prefix"
BotMsgTransferErrorAddTaskFailed Key = "bot.msg.transfer.error_add_task_failed"
BotMsgTransferErrorBuildStorageSelectKeyboardFailed Key = "bot.msg.transfer.error_build_storage_select_keyboard_failed"
BotMsgTransferErrorInvalidRegex Key = "bot.msg.transfer.error_invalid_regex"
BotMsgTransferErrorInvalidSource Key = "bot.msg.transfer.error_invalid_source"
BotMsgTransferErrorInvalidTarget Key = "bot.msg.transfer.error_invalid_target"
BotMsgTransferErrorListFilesFailed Key = "bot.msg.transfer.error_list_files_failed"
BotMsgTransferErrorNoFilesToTransfer Key = "bot.msg.transfer.error_no_files_to_transfer"
BotMsgTransferErrorStorageNotFound Key = "bot.msg.transfer.error_storage_not_found"
BotMsgTransferErrorStorageNotListable Key = "bot.msg.transfer.error_storage_not_listable"
BotMsgTransferErrorStorageNotReadable Key = "bot.msg.transfer.error_storage_not_readable"
BotMsgTransferErrorTargetNotFound Key = "bot.msg.transfer.error_target_not_found"
BotMsgTransferInfoFetchingFiles Key = "bot.msg.transfer.info_fetching_files"
BotMsgTransferInfoFilesSelectStorage Key = "bot.msg.transfer.info_files_select_storage"
BotMsgTransferInfoTaskAdded Key = "bot.msg.transfer.info_task_added"
BotMsgTransferStartStats Key = "bot.msg.transfer.start_stats"
BotMsgTransferUsage Key = "bot.msg.transfer.usage"
BotMsgUpdateButtonUpgrade Key = "bot.msg.update.button_upgrade"
BotMsgUpdateErrorCheckLatestFailed Key = "bot.msg.update.error_check_latest_failed"
BotMsgUpdateErrorNoReleaseFound Key = "bot.msg.update.error_no_release_found"

View File

@@ -29,7 +29,6 @@ bot:
/silent - Toggle silent mode
/storage - Set default storage
/save [custom filename] - Save file
/import <storage_name> <dir_path> [channel_id] [filter] - Import files from storage to Telegram
/dir - Manage storage directories
/rule - Manage rules
/config - Modify configuration
@@ -53,8 +52,6 @@ bot:
dl: "Download files from given links"
aria2dl: "Download files using Aria2"
ytdlp: "Download video/audio using yt-dlp"
import: "Import files from storage to Telegram"
transfer: "Transfer files between storages"
task: "Manage task queue"
cancel: "Cancel task"
watch: "Watch chats (UserBot)"
@@ -112,17 +109,9 @@ bot:
error_task_add_failed: "Failed to add task: {{.Error}}"
info_task_added: "Task added"
info_batch_tasks_added: "Batch tasks added, total {{.Count}} files"
info_batch_tasks_added_with_skipped: "Batch tasks added, total {{.Count}} files\nSkipped conflicting files:\n{{.Skipped}}"
info_all_conflict_files_skipped: "All conflicting files were skipped:\n{{.Skipped}}"
info_conflict_files_skipped: "Skipped conflicting files:\n{{.Skipped}}"
error_task_create_failed: "Failed to create task: {{.Error}}"
error_get_dir_failed: "Failed to get directory: {{.Error}}"
prompt_select_dir: "Please select a directory to store to"
prompt_select_conflict_strategy: "Files with the same name already exist. Please select a save strategy:\n{{.Files}}"
prompt_conflict_more_files: "...and {{.Count}} more files"
button_conflict_rename: "Rename"
button_conflict_overwrite: "Overwrite"
button_conflict_skip: "Skip"
prompt_select_default_dir: "Please select a default directory to save to"
info_default_storage_set: "Default storage set to: {{.Name}}"
info_default_storage_with_dir_set: "Default storage set to: {{.Name}}:/{{.Dir}}"
@@ -196,11 +185,7 @@ bot:
help_switch_suffix: " - Toggle rule mode\n"
help_add_suffix: " <type> <data> <storage_name> <path> - Add rule\n"
help_del_suffix: " <rule_id> - Delete rule\n"
help_preset_suffix: " <storage_name> [base_path] - Import built-in filetype rules (video/image/audio/document/archive)\n"
help_existing_rules_prefix: "\nCurrent rules:\n"
prompt_provide_storage_name: "Please provide a storage name"
error_storage_not_found: "Storage not found: {{.Storage}}"
info_preset_imported: "Imported {{.Count}} built-in classification rules into storage {{.Storage}}"
dir:
error_get_user_dirs_failed: "Failed to get user directories"
error_get_user_failed: "Failed to get user"
@@ -278,17 +263,10 @@ bot:
config:
prompt_select_option: "Please select an option to configure"
button_filename_strategy: "Filename strategy"
button_conflict_strategy: "Duplicate file strategy"
error_invalid_callback_data: "Invalid callback data"
error_invalid_template: "Invalid template, please check syntax\n{{.Error}}"
info_filename_strategy_set: "Filename strategy set to: {{.Strategy}}"
info_conflict_strategy_set: "Duplicate file strategy set to: {{.Strategy}}"
prompt_select_filename_strategy: "Please select filename strategy, current strategy: {{.Strategy}}"
prompt_select_conflict_strategy: "Please select duplicate file strategy, current strategy: {{.Strategy}}"
conflict_strategy_rename: "Always rename"
conflict_strategy_ask: "Ask every time"
conflict_strategy_overwrite: "Always overwrite"
conflict_strategy_skip: "Always skip"
fnametmpl_help: |-
Use this command to set filename template, for example:
/fnametmpl Image_{{"{{.msgid}}"}}_{{"{{.msgdate}}"}}.jpg
@@ -311,33 +289,11 @@ bot:
error_no_valid_links: "No valid links to download"
info_files_select_storage: "Total {{.Count}} files, please select storage"
ytdlp:
usage: "Usage: /ytdlp [OPTIONS] <URL1> [URL2] ...\nExamples:\n /ytdlp https://example.com/video\n /ytdlp --format best https://example.com/video\n /ytdlp --extract-audio --audio-format mp3 https://example.com/video"
usage: "Usage: /ytdlp <URL1> <URL2> ..."
error_no_valid_urls: "No valid URLs"
info_urls_select_storage: "Found {{.Count}} links, please select storage"
info_downloading: "Downloading via yt-dlp..."
error_download_failed: "yt-dlp download failed: {{.Error}}"
transfer:
usage: |
Usage: /transfer <source_storage>:/<source_path> [filter]
Examples:
/transfer local1:/downloads
/transfer alist1:/media/photos
/transfer webdav1:/files ".*\.mp4$"
error_invalid_source: "Invalid source path format, should be: storage_name:/path"
error_invalid_target: "Invalid target path format, should be: storage_name:/path"
error_storage_not_found: "Storage '{{.StorageName}}' not found or access denied: {{.Error}}"
error_storage_not_listable: "Storage '{{.StorageName}}' does not support listing files"
error_storage_not_readable: "Storage '{{.StorageName}}' does not support reading files"
error_target_not_found: "Target storage '{{.StorageName}}' not found or access denied: {{.Error}}"
info_fetching_files: "Fetching file list..."
error_list_files_failed: "Failed to list files: {{.Error}}"
error_invalid_regex: "Invalid regular expression: {{.Error}}"
error_no_files_to_transfer: "No files to transfer in directory"
error_add_task_failed: "Failed to add task: {{.Error}}"
info_task_added: "Added {{.Count}} files to transfer queue\nTotal size: {{.SizeMB}} MB\nTask ID: {{.TaskID}}"
start_stats: "Total files: {{.Count}}\nTotal size: {{.SizeMB}} MB"
info_files_select_storage: "Total {{.Count}} files ({{.SizeMB}} MB), please select target storage"
error_build_storage_select_keyboard_failed: "Failed to build storage selection keyboard: {{.Error}}"
cancel:
usage: "Usage: /cancel <task_id>"
error_cancel_failed: "Failed to cancel task: {{.Error}}"
@@ -386,20 +342,6 @@ bot:
ytdlp_done: "yt-dlp download completed and transferred ({{.Count}} files)\n"
downloaded_prefix: "\nDownloaded: "
current_speed_prefix: "\nCurrent speed: "
transfer_start_prefix: "Transfering: "
transfer_progress_prefix: "Transfer progress: "
transfer_uploaded_prefix: "\nUploaded: "
transfer_speed_prefix: "\nSpeed: "
transfer_remaining_time_prefix: "\nRemaining time: "
transfer_processing_prefix: "\nProcessing:\n"
transfer_processing_more: "...and {{.Count}} more files\n"
transfer_failed_prefix: "Transfer failed\n"
transfer_success_prefix: "Transfer completed\n"
transfer_total_files_prefix: "\nTotal files: "
transfer_total_size_prefix: "\nTotal size: "
transfer_elapsed_time_prefix: "\nElapsed time: "
transfer_avg_speed_prefix: "\nAverage speed: "
transfer_failed_files_prefix: "\nFailed files: "
syncpeers:
start: "Starting to sync peers..."
done: "Peer sync completed, total {{.Count}} chats synced"

View File

@@ -30,7 +30,6 @@ bot:
/storage - 设置默认存储位置
/save [自定义文件名] - 保存文件
/dl <链接1> <链接2> ... - 下载给定链接的文件
/import <存储名> <目录路径> [频道ID] [过滤器] - 从存储端导入文件到 Telegram
/dir - 管理存储目录
/rule - 管理规则
/config - 修改配置
@@ -54,8 +53,6 @@ bot:
dl: "下载给定链接的文件"
aria2dl: "使用 Aria2 下载给定链接的文件"
ytdlp: "使用 yt-dlp 下载视频/音频"
import: "从存储端导入文件到 Telegram"
transfer: "在存储端之间传输文件"
task: "管理任务队列"
cancel: "取消任务"
watch: "监听聊天(UserBot)"
@@ -113,17 +110,9 @@ bot:
error_task_add_failed: "任务添加失败: {{.Error}}"
info_task_added: "任务已添加"
info_batch_tasks_added: "已添加批量任务, 共 {{.Count}} 个文件"
info_batch_tasks_added_with_skipped: "已添加批量任务, 共 {{.Count}} 个文件\n已跳过同名文件:\n{{.Skipped}}"
info_all_conflict_files_skipped: "全部同名文件已跳过:\n{{.Skipped}}"
info_conflict_files_skipped: "已跳过同名文件:\n{{.Skipped}}"
error_task_create_failed: "任务创建失败: {{.Error}}"
error_get_dir_failed: "获取目录失败: {{.Error}}"
prompt_select_dir: "请选择要存储到的目录"
prompt_select_conflict_strategy: "检测到同名文件, 请选择保存策略:\n{{.Files}}"
prompt_conflict_more_files: "...还有 {{.Count}} 个文件"
button_conflict_rename: "重命名"
button_conflict_overwrite: "覆盖"
button_conflict_skip: "跳过"
prompt_select_default_dir: "请选择要保存到的默认文件夹"
info_default_storage_set: "已将默认存储位置设为: {{.Name}}"
info_default_storage_with_dir_set: "已将默认存储位置设为: {{.Name}}:/{{.Dir}}"
@@ -197,11 +186,7 @@ bot:
help_switch_suffix: " - 开关规则模式\n"
help_add_suffix: " <类型> <数据> <存储名> <路径> - 添加规则\n"
help_del_suffix: " <规则ID> - 删除规则\n"
help_preset_suffix: " <存储名> [基础路径] - 导入内置文件类型分类规则(视频/图片/音频/文档/压缩包)\n"
help_existing_rules_prefix: "\n当前已添加的规则:\n"
prompt_provide_storage_name: "请提供存储名称"
error_storage_not_found: "未找到存储: {{.Storage}}"
info_preset_imported: "已导入 {{.Count}} 条内置分类规则到存储 {{.Storage}}"
dir:
error_get_user_dirs_failed: "获取用户文件夹失败"
error_get_user_failed: "获取用户失败"
@@ -279,17 +264,10 @@ bot:
config:
prompt_select_option: "请选择要配置的选项"
button_filename_strategy: "文件名策略"
button_conflict_strategy: "重名文件保存策略"
error_invalid_callback_data: "无效的回调数据"
error_invalid_template: "无效的模板, 请检查语法\n{{.Error}}"
info_filename_strategy_set: "已将文件名策略设置为: {{.Strategy}}"
info_conflict_strategy_set: "已将重名文件保存策略设置为: {{.Strategy}}"
prompt_select_filename_strategy: "请选择文件名策略, 当前策略: {{.Strategy}}"
prompt_select_conflict_strategy: "请选择重名文件保存策略, 当前策略: {{.Strategy}}"
conflict_strategy_rename: "始终重命名"
conflict_strategy_ask: "每次询问"
conflict_strategy_overwrite: "始终覆盖"
conflict_strategy_skip: "始终跳过"
fnametmpl_help: |-
使用该命令设置文件名模板, 示例:
/fnametmpl 图片_{{"{{.msgid}}"}}_{{"{{.msgdate}}"}}.jpg
@@ -312,33 +290,11 @@ bot:
error_no_valid_links: "没有有效的链接可供下载"
info_files_select_storage: "共 {{.Count}} 个文件, 请选择存储位置"
ytdlp:
usage: "用法: /ytdlp [选项] <URL1> [URL2] ...\n示例:\n /ytdlp https://example.com/video\n /ytdlp --format best https://example.com/video\n /ytdlp --extract-audio --audio-format mp3 https://example.com/video"
usage: "用法: /ytdlp <URL1> <URL2> ..."
error_no_valid_urls: "没有有效的 URL"
info_urls_select_storage: "共 {{.Count}} 个链接, 请选择存储位置"
info_downloading: "正在通过 yt-dlp 下载..."
error_download_failed: "yt-dlp 下载失败: {{.Error}}"
transfer:
usage: |
用法: /transfer <source_storage>:/<source_path> [filter]
示例:
/transfer local1:/downloads
/transfer alist1:/media/photos
/transfer webdav1:/files ".*\.mp4$"
error_invalid_source: "源路径格式无效,应为: storage_name:/path"
error_invalid_target: "目标路径格式无效,应为: storage_name:/path"
error_storage_not_found: "存储端 '{{.StorageName}}' 不存在或您无权访问: {{.Error}}"
error_storage_not_listable: "存储端 '{{.StorageName}}' 不支持列举文件功能"
error_storage_not_readable: "存储端 '{{.StorageName}}' 不支持读取文件功能"
error_target_not_found: "目标存储端 '{{.StorageName}}' 不存在或您无权访问: {{.Error}}"
info_fetching_files: "正在获取文件列表..."
error_list_files_failed: "获取文件列表失败: {{.Error}}"
error_invalid_regex: "正则表达式无效: {{.Error}}"
error_no_files_to_transfer: "目录中没有可传输的文件"
error_add_task_failed: "添加任务失败: {{.Error}}"
info_task_added: "已添加 {{.Count}} 个文件到传输队列\n总大小: {{.SizeMB}} MB\n任务 ID: {{.TaskID}}"
start_stats: "总文件数: {{.Count}}\n总大小: {{.SizeMB}} MB"
info_files_select_storage: "共 {{.Count}} 个文件 (总大小: {{.SizeMB}} MB),请选择目标存储位置"
error_build_storage_select_keyboard_failed: "构建存储选择键盘失败: {{.Error}}"
cancel:
usage: "用法: /cancel <task_id>"
error_cancel_failed: "取消任务失败: {{.Error}}"
@@ -387,20 +343,6 @@ bot:
ytdlp_done: "yt-dlp 下载完成并已转存 ({{.Count}} 个文件)\n"
downloaded_prefix: "\n已下载: "
current_speed_prefix: "\n当前速度: "
transfer_start_prefix: "正在转存: "
transfer_progress_prefix: "转存进度: "
transfer_uploaded_prefix: "\n已上传: "
transfer_speed_prefix: "\n速度: "
transfer_remaining_time_prefix: "\n剩余时间: "
transfer_processing_prefix: "\n正在处理:\n"
transfer_processing_more: "...和其他 {{.Count}} 个文件\n"
transfer_failed_prefix: "转存失败\n"
transfer_success_prefix: "转存完成\n"
transfer_total_files_prefix: "\n总文件数: "
transfer_total_size_prefix: "\n总大小: "
transfer_elapsed_time_prefix: "\n耗时: "
transfer_avg_speed_prefix: "\n平均速度: "
transfer_failed_files_prefix: "\n失败文件数: "
syncpeers:
start: "正在同步对话列表..."
success: "对话列表同步完成, 共同步 {{.Count}} 个对话"
@@ -411,4 +353,4 @@ bot:
info_adding_aria2_download: "正在添加 Aria2 下载任务..."
error_adding_aria2_download: "添加 Aria2 下载任务失败: {{.Error}}"
info_aria2_download_added: "Aria2 下载任务已添加, GID: {{.GID}}"
info_select_storage: "请选择存储位置, 选择后将添加到 Aria2 下载队列"
info_select_storage: "请选择存储位置, 选择后将添加到 Aria2 下载队列"

View File

@@ -1,9 +1,6 @@
package dlutil
import (
"fmt"
"time"
)
import "time"
var threadsLevels = []struct {
threads int
@@ -34,23 +31,3 @@ func GetSpeed(downloaded int64, startTime time.Time) float64 {
}
return float64(downloaded) / elapsed
}
// FormatSize formats a byte size as a human-readable string
func FormatSize(bytes int64) string {
const (
KB = 1024
MB = KB * 1024
GB = MB * 1024
)
switch {
case bytes >= GB:
return fmt.Sprintf("%.2f GB", float64(bytes)/float64(GB))
case bytes >= MB:
return fmt.Sprintf("%.2f MB", float64(bytes)/float64(MB))
case bytes >= KB:
return fmt.Sprintf("%.2f KB", float64(bytes)/float64(KB))
default:
return fmt.Sprintf("%d B", bytes)
}
}

View File

@@ -48,4 +48,4 @@ func NewProgressWriter(
wr: wr,
onWrite: onWrite,
}
}
}

View File

@@ -5,10 +5,6 @@ retry = 3 # 下载失败重试次数
threads = 4 # 单个任务下载使用的最大线程数
stream = false # 使用流式传输模式, 建议仅在硬盘空间十分有限时使用.
[log]
# 日志级别, 可选: debug, info, warn, error, fatal
level = "debug"
[telegram]
# Bot Token
# 更换 Bot Token 后请删除会话数据库文件 (默认路径为 data/session.db )
@@ -33,27 +29,16 @@ secret = ""
# 转存完成后删除 Aria2 下载的本地文件
remove_after_transfer = true
# yt-dlp 视频下载配置
[ytdlp]
# 默认下载的最高视频清晰度 (按高度限制), 如 1080, 720, 480; 0 表示不限制 (下载最佳画质)
# 仅在使用 /ytdlp 命令且未手动指定任何参数时生效
max_height = 1080
# 直接指定 yt-dlp format 选择表达式, 留空则使用 max_height
# 设置后优先级高于 max_height, 例如: "bv*[height<=720]+ba/b"
format = ""
# 下载后转封装的视频容器格式, 留空则不转封装. 默认 mp4
recode = "mp4"
# HTTP API 配置
[api]
# 启用 HTTP API
# 启用 HTTP API 服务
enable = false
# 监听地址
host = "0.0.0.0"
# 监听端口
# API 服务监听端口
port = 8080
# 认证 Token (必需)
# API 访问令牌 (留空则不验证)
token = ""
# 任务完成回调 Webhook URL (留空则不回调)
webhook_url = ""
# 存储列表
[[storages]]
@@ -88,4 +73,4 @@ blacklist = true
[[users]]
id = 123456
storages = ["本机1"]
blacklist = false # 使用白名单模式,此时,用户 123456 仅可使用标识名为 '本地1' 的存储
blacklist = false # 使用白名单模式,此时,用户 123456 仅可使用标识名为 '本地1' 的存储

View File

@@ -17,7 +17,6 @@ func RegisterFlags(cmd *cobra.Command) {
flags.Bool("stream", false, "enable stream mode")
flags.Bool("no-clean-cache", false, "do not clean cache on exit")
flags.String("proxy", "", "proxy URL (http, https, socks5, socks5h)")
flags.String("log-level", "", "log level (trace/debug, info, warn, error, fatal)")
// Telegram 配置
flags.String("telegram-token", "", "telegram bot token")
@@ -55,7 +54,6 @@ func bindFlags(cmd *cobra.Command) {
viper.BindPFlag("stream", flags.Lookup("stream"))
viper.BindPFlag("no_clean_cache", flags.Lookup("no-clean-cache"))
viper.BindPFlag("proxy", flags.Lookup("proxy"))
viper.BindPFlag("log.level", flags.Lookup("log-level"))
// Telegram
viper.BindPFlag("telegram.token", flags.Lookup("telegram-token"))

View File

@@ -1,5 +0,0 @@
package config
type logConfig struct {
Level string `toml:"level" mapstructure:"level" json:"level"`
}

View File

@@ -16,7 +16,6 @@ var storageFactories = map[storenum.StorageType]func(cfg *BaseConfig) (StorageCo
storenum.Minio: createStorageConfig(&MinioStorageConfig{}),
storenum.S3: createStorageConfig(&S3StorageConfig{}),
storenum.Telegram: createStorageConfig(&TelegramStorageConfig{}),
storenum.Rclone: createStorageConfig(&RcloneStorageConfig{}),
}
func createStorageConfig(configType StorageConfig) func(cfg *BaseConfig) (StorageConfig, error) {

View File

@@ -1,33 +0,0 @@
package storage
import (
"fmt"
storenum "github.com/krau/SaveAny-Bot/pkg/enums/storage"
)
type RcloneStorageConfig struct {
BaseConfig
// The name of the remote as defined in rclone config
Remote string `toml:"remote" mapstructure:"remote" json:"remote"`
BasePath string `toml:"base_path" mapstructure:"base_path" json:"base_path"`
// The path to the rclone config file, if not using the default
ConfigPath string `toml:"config_path" mapstructure:"config_path" json:"config_path"`
// Additional flags to pass to rclone commands
Flags []string `toml:"flags" mapstructure:"flags" json:"flags"`
}
func (r *RcloneStorageConfig) Validate() error {
if r.Remote == "" {
return fmt.Errorf("remote is required for rclone storage")
}
return nil
}
func (r *RcloneStorageConfig) GetType() storenum.StorageType {
return storenum.Rclone
}
func (r *RcloneStorageConfig) GetName() string {
return r.Name
}

View File

@@ -23,9 +23,7 @@ type Config struct {
Threads int `toml:"threads" mapstructure:"threads" json:"threads"`
Stream bool `toml:"stream" mapstructure:"stream" json:"stream"`
Proxy string `toml:"proxy" mapstructure:"proxy" json:"proxy"`
Log logConfig `toml:"log" mapstructure:"log" json:"log"`
Aria2 aria2Config `toml:"aria2" mapstructure:"aria2" json:"aria2"`
API apiConfig `toml:"api" mapstructure:"api" json:"api"`
Cache cacheConfig `toml:"cache" mapstructure:"cache" json:"cache"`
Users []userConfig `toml:"users" mapstructure:"users" json:"users"`
@@ -35,7 +33,7 @@ type Config struct {
Storages []storage.StorageConfig `toml:"-" mapstructure:"-" json:"storages"`
Parser parserConfig `toml:"parser" mapstructure:"parser" json:"parser"`
Hook hookConfig `toml:"hook" mapstructure:"hook" json:"hook"`
Ytdlp YtdlpConfig `toml:"ytdlp" mapstructure:"ytdlp" json:"ytdlp"`
API apiConfig `toml:"api" mapstructure:"api" json:"api"`
}
type aria2Config struct {
@@ -46,10 +44,10 @@ type aria2Config struct {
}
type apiConfig struct {
Enable bool `toml:"enable" mapstructure:"enable" json:"enable"`
Host string `toml:"host" mapstructure:"host" json:"host"`
Port int `toml:"port" mapstructure:"port" json:"port"`
Token string `toml:"token" mapstructure:"token" json:"token"`
Enable bool `toml:"enable" mapstructure:"enable" json:"enable"`
Port int `toml:"port" mapstructure:"port" json:"port"`
Token string `toml:"token" mapstructure:"token" json:"token"`
WebhookURL string `toml:"webhook_url" mapstructure:"webhook_url" json:"webhook_url"`
}
var cfg = &Config{}
@@ -102,11 +100,10 @@ func Init(ctx context.Context, configFile ...string) error {
defaultConfigs := map[string]any{
// 基础配置
"lang": "zh-Hans",
"workers": 3,
"retry": 3,
"threads": 4,
"log.level": "debug",
"lang": "zh-Hans",
"workers": 3,
"retry": 3,
"threads": 4,
// 缓存配置
"cache.ttl": 86400,
@@ -128,19 +125,22 @@ func Init(ctx context.Context, configFile ...string) error {
"db.session": "data/session.db",
// API
"api.enable": false,
"api.host": "0.0.0.0",
"api.port": 8080,
"api.token": "",
// yt-dlp
"ytdlp.recode": "mp4",
"api.enable": false,
"api.port": 8080,
"api.token": "",
"api.webhook_url": "",
}
for key, value := range defaultConfigs {
viper.SetDefault(key, value)
}
if err := viper.SafeWriteConfigAs("config.toml"); err != nil {
if _, ok := err.(viper.ConfigFileAlreadyExistsError); !ok {
return fmt.Errorf("error saving default config: %w", err)
}
}
if err := viper.ReadInConfig(); err != nil {
fmt.Println("Error reading config file, ", err)
return err

View File

@@ -1,13 +0,0 @@
package config
type YtdlpConfig struct {
// MaxHeight limits the video resolution by height in pixels (e.g. 1080, 720).
// 0 means no limit (best available). Ignored when Format is set.
MaxHeight int `toml:"max_height" mapstructure:"max_height" json:"max_height"`
// Format is a raw yt-dlp format selector (-f). When set, it takes precedence
// over MaxHeight and gives the user full control.
Format string `toml:"format" mapstructure:"format" json:"format"`
// Recode is the target video container yt-dlp recodes into (e.g. mp4).
// Empty disables recoding.
Recode string `toml:"recode" mapstructure:"recode" json:"recode"`
}

View File

@@ -8,7 +8,6 @@ import (
"github.com/krau/SaveAny-Bot/config"
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
"github.com/krau/SaveAny-Bot/pkg/queue"
"github.com/krau/SaveAny-Bot/pkg/taskevent"
)
var queueInstance *queue.TaskQueue[Executable]
@@ -31,14 +30,11 @@ func worker(ctx context.Context, qe *queue.TaskQueue[Executable], semaphore chan
break // queue closed and empty
}
exe := qtask.Data
taskCtx := qtask.Context()
logger.Infof("Processing task: %s", exe.TaskID())
taskevent.Emit(taskCtx, taskevent.Event{TaskID: exe.TaskID(), Phase: taskevent.PhaseStart})
if err := ExecCommandString(taskCtx, execHooks.TaskBeforeStart); err != nil {
if err := ExecCommandString(qtask.Context(), execHooks.TaskBeforeStart); err != nil {
logger.Errorf("Failed to execute before start hook for task %s: %v", exe.TaskID(), err)
}
err = exe.Execute(taskCtx)
if err != nil {
if err := exe.Execute(qtask.Context()); err != nil {
if errors.Is(err, context.Canceled) {
logger.Infof("Task %s was canceled", exe.TaskID())
if err := ExecCommandString(ctx, execHooks.TaskCancel); err != nil {
@@ -56,7 +52,6 @@ func worker(ctx context.Context, qe *queue.TaskQueue[Executable], semaphore chan
logger.Errorf("Failed to execute success hook for task %s: %v", exe.TaskID(), err)
}
}
taskevent.Emit(taskCtx, taskevent.Event{TaskID: exe.TaskID(), Phase: taskevent.PhaseDone, Err: err})
qe.Done(qtask.ID)
<-semaphore
}

View File

@@ -6,14 +6,12 @@ import (
"fmt"
"os"
"path/filepath"
"strconv"
"time"
"github.com/charmbracelet/log"
"github.com/krau/SaveAny-Bot/config"
"github.com/krau/SaveAny-Bot/pkg/aria2"
"github.com/krau/SaveAny-Bot/pkg/enums/ctxkey"
"github.com/krau/SaveAny-Bot/pkg/taskevent"
)
// Execute implements core.Executable.
@@ -79,12 +77,6 @@ func (t *Task) waitForDownload(ctx context.Context) error {
if t.Progress != nil {
t.Progress.OnProgress(ctx, t, status)
}
taskevent.Emit(ctx, taskevent.Event{
TaskID: t.ID,
Phase: taskevent.PhaseProgress,
TotalBytes: parseInt64(status.TotalLength),
DownloadedBytes: parseInt64(status.CompletedLength),
})
// Check if download is complete
if status.IsDownloadComplete() {
@@ -256,16 +248,3 @@ func (t *Task) cancelAria2Download() {
logger.Debugf("Failed to remove download result for %s: %v", t.GID, err)
}
}
// parseInt64 parses an aria2 status string (decimal bytes) into int64,
// returning 0 on failure so it can be used directly in progress events.
func parseInt64(s string) int64 {
if s == "" {
return 0
}
n, err := strconv.ParseInt(s, 10, 64)
if err != nil {
return 0
}
return n
}

View File

@@ -14,7 +14,6 @@ import (
"github.com/krau/SaveAny-Bot/common/utils/ioutil"
"github.com/krau/SaveAny-Bot/config"
"github.com/krau/SaveAny-Bot/pkg/enums/ctxkey"
"github.com/krau/SaveAny-Bot/pkg/taskevent"
"golang.org/x/sync/errgroup"
)
@@ -63,14 +62,8 @@ func (t *Task) processElement(ctx context.Context, elem TaskElement) error {
return elem.Storage.Save(uploadCtx, pr, elem.Path)
})
wr := ioutil.NewProgressWriter(pw, func(n int) {
downloaded := t.downloaded.Add(int64(n))
t.downloaded.Add(int64(n))
t.Progress.OnProgress(ctx, t)
taskevent.Emit(ctx, taskevent.Event{
TaskID: t.ID,
Phase: taskevent.PhaseProgress,
TotalBytes: t.totalSize,
DownloadedBytes: downloaded,
})
})
errg.Go(func() error {
defer pw.Close()
@@ -99,14 +92,8 @@ func (t *Task) processElement(ctx context.Context, elem TaskElement) error {
}
}()
wrAt := ioutil.NewProgressWriterAt(localFile, func(n int) {
downloaded := t.downloaded.Add(int64(n))
t.downloaded.Add(int64(n))
t.Progress.OnProgress(ctx, t)
taskevent.Emit(ctx, taskevent.Event{
TaskID: t.ID,
Phase: taskevent.PhaseProgress,
TotalBytes: t.totalSize,
DownloadedBytes: downloaded,
})
})
_, err = tdler.NewDownloader(elem.File).Parallel(ctx, wrAt)
if err != nil {

View File

@@ -5,7 +5,6 @@ import (
"errors"
"fmt"
"strconv"
"strings"
"sync/atomic"
"time"
@@ -31,7 +30,6 @@ type Progress struct {
ChatID int64
start time.Time
lastUpdatePercent atomic.Int32
skippedFiles []string
}
func (p *Progress) OnStart(ctx context.Context, info TaskInfo) {
@@ -153,14 +151,6 @@ func (p *Progress) OnDone(ctx context.Context, info TaskInfo, err error) {
styling.Code(strconv.Itoa(info.Count())),
styling.Plain(i18n.T(i18nk.BotMsgProgressTotalSizePrefix, nil)),
styling.Code(fmt.Sprintf("%.2f MB", float64(info.TotalSize())/(1024*1024))),
func() styling.StyledTextOption {
if len(p.skippedFiles) == 0 {
return styling.Plain("")
}
return styling.Plain("\n\n" + i18n.T(i18nk.BotMsgCommonInfoConflictFilesSkipped, map[string]any{
"Skipped": strings.Join(p.skippedFiles, "\n"),
}))
}(),
)
}
@@ -183,13 +173,8 @@ func (p *Progress) OnDone(ctx context.Context, info TaskInfo, err error) {
}
func NewProgressTracker(messageID int, chatID int64) ProgressTracker {
return NewProgressTrackerWithSkipped(messageID, chatID, nil)
}
func NewProgressTrackerWithSkipped(messageID int, chatID int64, skippedFiles []string) ProgressTracker {
return &Progress{
MessageID: messageID,
ChatID: chatID,
skippedFiles: skippedFiles,
MessageID: messageID,
ChatID: chatID,
}
}

View File

@@ -15,7 +15,6 @@ import (
"github.com/krau/SaveAny-Bot/common/utils/ioutil"
"github.com/krau/SaveAny-Bot/config"
"github.com/krau/SaveAny-Bot/pkg/enums/ctxkey"
"github.com/krau/SaveAny-Bot/pkg/taskevent"
"golang.org/x/sync/errgroup"
)
@@ -46,17 +45,9 @@ func (t *Task) Execute(ctx context.Context) error {
fetchedTotalBytes.Add(resp.ContentLength)
file.Size = resp.ContentLength
if name := resp.Header.Get("Content-Disposition"); name != "" {
// Set file name
filename := parseFilename(name)
if filename != "" {
file.Name = filename
}
}
// extract filename from URL if Content-Disposition is empty or invalid
if file.Name == "" {
file.Name = parseFilenameFromURL(file.URL)
}
if file.Name == "" {
return fmt.Errorf("failed to determine filename for %s: Content-Disposition header is empty and URL does not contain a valid filename", file.URL)
file.Name = filename
}
return nil
@@ -144,16 +135,10 @@ func (t *Task) processLink(ctx context.Context, file *File) error {
}
}()
wr := ioutil.NewProgressWriter(cacheFile, func(n int) {
downloaded := t.downloadedBytes.Add(int64(n))
t.downloadedBytes.Add(int64(n))
if t.Progress != nil {
t.Progress.OnProgress(ctx, t)
}
taskevent.Emit(ctx, taskevent.Event{
TaskID: t.ID,
Phase: taskevent.PhaseProgress,
TotalBytes: t.totalBytes,
DownloadedBytes: downloaded,
})
})
copyResultCh := make(chan error, 1)

View File

@@ -76,9 +76,6 @@ func (t *Task) StorageName() string {
// StoragePath implements TaskInfo.
func (t *Task) StoragePath() string {
if len(t.files) == 1 {
return t.StorPath + "/" + t.files[0].Name
}
return t.StorPath
}

View File

@@ -144,41 +144,6 @@ func tryDecodeGBK(s string) string {
return ""
}
// parseFilenameFromURL extracts filename from URL path
// This is used as a fallback when Content-Disposition is not available
func parseFilenameFromURL(rawURL string) string {
parsed, err := url.Parse(rawURL)
if err != nil {
return ""
}
// Get the path part and extract the last segment
path := parsed.Path
if path == "" {
return ""
}
// URL decode the path first
decodedPath, err := url.PathUnescape(path)
if err != nil {
decodedPath = path
}
// Get the last segment of the path
lastSlash := strings.LastIndex(decodedPath, "/")
if lastSlash == -1 {
return decodedPath
}
filename := decodedPath[lastSlash+1:]
// Remove query string if somehow still present
if idx := strings.Index(filename, "?"); idx != -1 {
filename = filename[:idx]
}
return filename
}
// parseFilenameFallback manually parses filename= when mime.ParseMediaType fails
func parseFilenameFallback(cd string) string {
// Look for filename= (case-insensitive)

View File

@@ -1,73 +0,0 @@
package directlinks
import (
"testing"
)
func TestParseFilenameFromURL(t *testing.T) {
tests := []struct {
name string
url string
expected string
}{
{
name: "simple filename",
url: "https://example.com/files/document.pdf",
expected: "document.pdf",
},
{
name: "filename with encoded characters",
url: "https://example.com/files/%E6%B5%8B%E8%AF%95.zip",
expected: "测试.zip",
},
{
name: "filename with query string in URL",
url: "https://example.com/files/image.png?token=abc123",
expected: "image.png",
},
{
name: "nested path",
url: "https://example.com/a/b/c/file.txt",
expected: "file.txt",
},
{
name: "URL with port",
url: "https://example.com:8080/downloads/archive.tar.gz",
expected: "archive.tar.gz",
},
{
name: "empty path",
url: "https://example.com",
expected: "",
},
{
name: "root path only",
url: "https://example.com/",
expected: "",
},
{
name: "filename with spaces encoded",
url: "https://example.com/my%20file%20name.pdf",
expected: "my file name.pdf",
},
{
name: "complex encoded filename",
url: "https://example.com/downloads/%E4%B8%AD%E6%96%87%E6%96%87%E4%BB%B6.docx",
expected: "中文文件.docx",
},
{
name: "invalid URL",
url: "://invalid-url",
expected: "",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := parseFilenameFromURL(tt.url)
if result != tt.expected {
t.Errorf("parseFilenameFromURL(%q) = %q, want %q", tt.url, result, tt.expected)
}
})
}
}

View File

@@ -16,7 +16,6 @@ import (
"github.com/krau/SaveAny-Bot/config"
"github.com/krau/SaveAny-Bot/pkg/enums/ctxkey"
"github.com/krau/SaveAny-Bot/pkg/parser"
"github.com/krau/SaveAny-Bot/pkg/taskevent"
"golang.org/x/sync/errgroup"
)
@@ -108,16 +107,10 @@ func (t *Task) processResource(ctx context.Context, resource parser.Resource) er
}
}()
wr := ioutil.NewProgressWriter(cacheFile, func(n int) {
downloaded := t.downloadedBytes.Add(int64(n))
t.downloadedBytes.Add(int64(n))
if t.progress != nil {
t.progress.OnProgress(ctx, t)
}
taskevent.Emit(ctx, taskevent.Event{
TaskID: t.ID,
Phase: taskevent.PhaseProgress,
TotalBytes: t.totalBytes,
DownloadedBytes: downloaded,
})
})
copyResultCh := make(chan error, 1)

View File

@@ -11,7 +11,6 @@ import (
"github.com/duke-git/lancet/v2/retry"
"github.com/krau/SaveAny-Bot/common/utils/fsutil"
"github.com/krau/SaveAny-Bot/config"
"github.com/krau/SaveAny-Bot/pkg/taskevent"
"golang.org/x/sync/errgroup"
)
@@ -28,14 +27,8 @@ func (t *Task) Execute(ctx context.Context) error {
logger.Errorf("Error processing picture %s: %v", pic, err)
return fmt.Errorf("failed to process picture %s: %w", pic, err)
}
downloaded := t.downloaded.Add(1)
t.downloaded.Add(1)
t.progress.OnProgress(gctx, t)
taskevent.Emit(gctx, taskevent.Event{
TaskID: t.ID,
Phase: taskevent.PhaseProgress,
TotalFiles: t.totalpics,
DownloadedFiles: int(downloaded),
})
return nil
})
}

View File

@@ -4,8 +4,6 @@ import (
"context"
"io"
"sync/atomic"
"github.com/krau/SaveAny-Bot/pkg/taskevent"
)
type ProgressWriterAt struct {
@@ -22,16 +20,9 @@ func (w *ProgressWriterAt) WriteAt(p []byte, off int64) (int, error) {
if err != nil {
return 0, err
}
downloaded := w.downloaded.Add(int64(at))
if w.progress != nil {
w.progress.OnProgress(w.ctx, w.info, downloaded, w.total)
w.progress.OnProgress(w.ctx, w.info, w.downloaded.Add(int64(at)), w.total)
}
taskevent.Emit(w.ctx, taskevent.Event{
TaskID: w.info.TaskID(),
Phase: taskevent.PhaseProgress,
TotalBytes: w.total,
DownloadedBytes: downloaded,
})
return at, nil
}
@@ -65,16 +56,9 @@ func (w *ProgressWriter) Write(p []byte) (int, error) {
if err != nil {
return 0, err
}
downloaded := w.downloaded.Add(int64(at))
if w.progress != nil {
w.progress.OnProgress(w.ctx, w.info, downloaded, w.total)
w.progress.OnProgress(w.ctx, w.info, w.downloaded.Add(int64(at)), w.total)
}
taskevent.Emit(w.ctx, taskevent.Event{
TaskID: w.info.TaskID(),
Phase: taskevent.PhaseProgress,
TotalBytes: w.total,
DownloadedBytes: downloaded,
})
return at, nil
}

View File

@@ -1,149 +0,0 @@
package transfer
import (
"context"
"fmt"
"io"
"os"
"path"
"path/filepath"
"github.com/charmbracelet/log"
"github.com/krau/SaveAny-Bot/config"
"github.com/krau/SaveAny-Bot/pkg/enums/ctxkey"
"github.com/krau/SaveAny-Bot/pkg/taskevent"
"github.com/krau/SaveAny-Bot/storage"
"golang.org/x/sync/errgroup"
)
// Execute implements core.Executable.
func (t *Task) Execute(ctx context.Context) error {
logger := log.FromContext(ctx).WithPrefix(fmt.Sprintf("transfer[%s]", t.ID))
logger.Info("Starting transfer task")
t.Progress.OnStart(ctx, t)
workers := config.C().Workers
eg, gctx := errgroup.WithContext(ctx)
eg.SetLimit(workers)
for _, elem := range t.elems {
eg.Go(func() error {
t.processingMu.RLock()
if t.processing[elem.ID] != nil {
t.processingMu.RUnlock()
return fmt.Errorf("element with ID %s is already being processed", elem.ID)
}
t.processingMu.RUnlock()
t.processingMu.Lock()
t.processing[elem.ID] = &elem
t.processingMu.Unlock()
defer func() {
t.processingMu.Lock()
delete(t.processing, elem.ID)
t.processingMu.Unlock()
}()
err := t.processElement(gctx, elem)
if err != nil && !t.IgnoreErrors {
return err
}
if err != nil {
t.processingMu.Lock()
t.failed[elem.ID] = err
t.processingMu.Unlock()
logger.Errorf("Failed to process file %s: %v", elem.FileInfo.Name, err)
}
return nil
})
}
err := eg.Wait()
if err != nil {
logger.Errorf("Error during transfer processing: %v", err)
} else {
logger.Info("Transfer task completed successfully")
}
t.Progress.OnDone(ctx, t, err)
return err
}
func (t *Task) processElement(ctx context.Context, elem TaskElement) error {
logger := log.FromContext(ctx).WithPrefix(fmt.Sprintf("file[%s]", elem.FileInfo.Name))
// Check whether the source storage supports reading
readableStorage, ok := elem.SourceStorage.(storage.StorageReadable)
if !ok {
return fmt.Errorf("source storage %s does not support reading", elem.SourceStorage.Name())
}
logger.Info("Opening file from source storage")
reader, size, err := readableStorage.OpenFile(ctx, elem.SourcePath)
if err != nil {
return fmt.Errorf("failed to open file: %w", err)
}
defer reader.Close()
// Build target storage path: /target_path/filename
storagePath := path.Join(elem.TargetPath, elem.FileInfo.Name)
// Inject file size into context
ctx = context.WithValue(ctx, ctxkey.ContentLength, size)
if config.C().Stream {
if err := elem.TargetStorage.Save(ctx, reader, storagePath); err != nil {
return fmt.Errorf("failed to upload file to storage: %w", err)
}
} else {
logger.Info("Downloading to temporary file for ReadSeeker support")
tempFile, err := t.downloadToTemp(reader, elem.FileInfo.Name)
if err != nil {
return fmt.Errorf("failed to download to temp: %w", err)
}
defer os.Remove(tempFile.Name())
defer tempFile.Close()
if _, err := tempFile.Seek(0, io.SeekStart); err != nil {
return fmt.Errorf("failed to seek temp file: %w", err)
}
logger.Infof("Uploading file to storage (size: %d bytes)", size)
if err := elem.TargetStorage.Save(ctx, tempFile, storagePath); err != nil {
return fmt.Errorf("failed to upload file to storage: %w", err)
}
}
t.uploaded.Add(size)
t.Progress.OnProgress(ctx, t)
taskevent.Emit(ctx, taskevent.Event{
TaskID: t.ID,
Phase: taskevent.PhaseProgress,
TotalBytes: t.totalSize,
DownloadedBytes: t.uploaded.Load(),
})
logger.Info("File uploaded successfully")
return nil
}
func (t *Task) downloadToTemp(reader io.Reader, filename string) (*os.File, error) {
tempDir := config.C().Temp.BasePath
if tempDir == "" {
tempDir = os.TempDir()
}
tempFile, err := os.CreateTemp(tempDir, filepath.Base(filename)+"-*.tmp")
if err != nil {
return nil, fmt.Errorf("failed to create temp file: %w", err)
}
if _, err := io.Copy(tempFile, reader); err != nil {
tempFile.Close()
os.Remove(tempFile.Name())
return nil, fmt.Errorf("failed to copy to temp file: %w", err)
}
return tempFile, nil
}

View File

@@ -1,247 +0,0 @@
package transfer
import (
"context"
"fmt"
"strings"
"sync/atomic"
"time"
"github.com/charmbracelet/log"
"github.com/gotd/td/telegram/message/entity"
"github.com/gotd/td/telegram/message/styling"
"github.com/gotd/td/tg"
"github.com/krau/SaveAny-Bot/common/i18n"
"github.com/krau/SaveAny-Bot/common/i18n/i18nk"
"github.com/krau/SaveAny-Bot/common/utils/dlutil"
"github.com/krau/SaveAny-Bot/common/utils/tgutil"
)
type ProgressTracker interface {
OnStart(ctx context.Context, info TaskInfo)
OnProgress(ctx context.Context, info TaskInfo)
OnDone(ctx context.Context, info TaskInfo, err error)
}
type Progress struct {
MessageID int
ChatID int64
start time.Time
lastUpdatePercent atomic.Int32
}
func NewProgressTracker(messageID int, chatID int64) ProgressTracker {
return &Progress{
MessageID: messageID,
ChatID: chatID,
}
}
func (p *Progress) OnStart(ctx context.Context, info TaskInfo) {
p.start = time.Now()
p.lastUpdatePercent.Store(0)
log.FromContext(ctx).Debugf("Transfer task progress tracking started for message %d in chat %d", p.MessageID, p.ChatID)
sizeMB := float64(info.TotalSize()) / (1024 * 1024)
statsText := i18n.T(i18nk.BotMsgTransferStartStats, map[string]any{
"SizeMB": fmt.Sprintf("%.2f", sizeMB),
"Count": info.Count(),
})
entityBuilder := entity.Builder{}
if err := styling.Perform(&entityBuilder,
styling.Plain(i18n.T(i18nk.BotMsgProgressTransferStartPrefix, nil)),
styling.Code(statsText),
); err != nil {
log.FromContext(ctx).Errorf("Failed to build entities: %s", err)
return
}
text, entities := entityBuilder.Complete()
req := &tg.MessagesEditMessageRequest{
ID: p.MessageID,
}
req.SetMessage(text)
req.SetEntities(entities)
req.SetReplyMarkup(&tg.ReplyInlineMarkup{
Rows: []tg.KeyboardButtonRow{
{
Buttons: []tg.KeyboardButtonClass{
tgutil.BuildCancelButton(info.TaskID()),
},
},
},
})
ext := tgutil.ExtFromContext(ctx)
if ext != nil {
_, err := ext.EditMessage(p.ChatID, req)
if err != nil {
log.FromContext(ctx).Errorf("Failed to send progress start message: %s", err)
}
}
}
func (p *Progress) OnProgress(ctx context.Context, info TaskInfo) {
if !shouldUpdateProgress(info.TotalSize(), info.Uploaded(), int(p.lastUpdatePercent.Load())) {
return
}
percent := int((info.Uploaded() * 100) / info.TotalSize())
if p.lastUpdatePercent.Load() == int32(percent) {
return
}
p.lastUpdatePercent.Store(int32(percent))
log.FromContext(ctx).Debugf("Progress update: %s, %d/%d", info.TaskID(), info.Uploaded(), info.TotalSize())
entityBuilder := entity.Builder{}
var progressText strings.Builder
progressText.WriteString(i18n.T(i18nk.BotMsgProgressTransferProgressPrefix, nil))
fmt.Fprintf(&progressText, "%d%%", percent)
progressText.WriteString(i18n.T(i18nk.BotMsgProgressTransferUploadedPrefix, nil))
fmt.Fprintf(&progressText, "%.2f MB / %.2f MB",
float64(info.Uploaded())/(1024*1024),
float64(info.TotalSize())/(1024*1024))
if p.start.Unix() > 0 {
elapsed := time.Since(p.start)
speed := float64(info.Uploaded()) / elapsed.Seconds()
progressText.WriteString(i18n.T(i18nk.BotMsgProgressTransferSpeedPrefix, nil))
progressText.WriteString(dlutil.FormatSize(int64(speed)) + "/s")
if info.Uploaded() > 0 {
remaining := time.Duration(float64(info.TotalSize()-info.Uploaded()) / speed * float64(time.Second))
progressText.WriteString(i18n.T(i18nk.BotMsgProgressTransferRemainingTimePrefix, nil))
progressText.WriteString(formatDuration(remaining))
}
}
processing := info.Processing()
if len(processing) > 0 {
progressText.WriteString(i18n.T(i18nk.BotMsgProgressTransferProcessingPrefix, nil))
for i, elem := range processing {
if i >= 3 {
progressText.WriteString(i18n.T(i18nk.BotMsgProgressTransferProcessingMore, map[string]any{"Count": len(processing) - 3}))
break
}
fmt.Fprintf(&progressText, "- %s\n", elem.FileName())
}
}
if err := styling.Perform(&entityBuilder,
styling.Plain(progressText.String()),
); err != nil {
log.FromContext(ctx).Errorf("Failed to build entities: %s", err)
return
}
text, entities := entityBuilder.Complete()
req := &tg.MessagesEditMessageRequest{
ID: p.MessageID,
}
req.SetMessage(text)
req.SetEntities(entities)
req.SetReplyMarkup(&tg.ReplyInlineMarkup{
Rows: []tg.KeyboardButtonRow{
{
Buttons: []tg.KeyboardButtonClass{
tgutil.BuildCancelButton(info.TaskID()),
},
},
},
})
ext := tgutil.ExtFromContext(ctx)
if ext != nil {
ext.EditMessage(p.ChatID, req)
}
}
func (p *Progress) OnDone(ctx context.Context, info TaskInfo, err error) {
log.FromContext(ctx).Debugf("Transfer task progress tracking done for message %d in chat %d", p.MessageID, p.ChatID)
entityBuilder := entity.Builder{}
var resultText strings.Builder
if err != nil {
resultText.WriteString(i18n.T(i18nk.BotMsgProgressTransferFailedPrefix, nil))
resultText.WriteString(i18n.T(i18nk.BotMsgProgressErrorPrefix, nil))
fmt.Fprintf(&resultText, "%v\n", err)
} else {
resultText.WriteString(i18n.T(i18nk.BotMsgProgressTransferSuccessPrefix, nil))
}
elapsed := time.Since(p.start)
resultText.WriteString(i18n.T(i18nk.BotMsgProgressTransferTotalFilesPrefix, nil))
fmt.Fprintf(&resultText, "%d\n", info.Count())
resultText.WriteString(i18n.T(i18nk.BotMsgProgressTransferTotalSizePrefix, nil))
fmt.Fprintf(&resultText, "%.2f MB\n", float64(info.TotalSize())/(1024*1024))
resultText.WriteString(i18n.T(i18nk.BotMsgProgressTransferUploadedPrefix, nil))
fmt.Fprintf(&resultText, "%.2f MB\n", float64(info.Uploaded())/(1024*1024))
resultText.WriteString(i18n.T(i18nk.BotMsgProgressTransferElapsedTimePrefix, nil))
fmt.Fprintf(&resultText, "%s\n", formatDuration(elapsed))
if elapsed.Seconds() > 0 {
avgSpeed := float64(info.Uploaded()) / elapsed.Seconds()
resultText.WriteString(i18n.T(i18nk.BotMsgProgressTransferAvgSpeedPrefix, nil))
fmt.Fprintf(&resultText, "%s/s\n", dlutil.FormatSize(int64(avgSpeed)))
}
failedFiles := info.FailedFiles()
if len(failedFiles) > 0 {
resultText.WriteString(i18n.T(i18nk.BotMsgProgressTransferFailedFilesPrefix, nil))
fmt.Fprintf(&resultText, "%d\n", len(failedFiles))
for i, name := range failedFiles {
if i >= 5 {
resultText.WriteString(i18n.T(i18nk.BotMsgProgressTransferProcessingMore, map[string]any{"Count": len(failedFiles) - 5}))
break
}
fmt.Fprintf(&resultText, "- %s\n", name)
}
}
if err := styling.Perform(&entityBuilder,
styling.Plain(resultText.String()),
); err != nil {
log.FromContext(ctx).Errorf("Failed to build entities: %s", err)
return
}
text, entities := entityBuilder.Complete()
req := &tg.MessagesEditMessageRequest{
ID: p.MessageID,
}
req.SetMessage(text)
req.SetEntities(entities)
ext := tgutil.ExtFromContext(ctx)
if ext != nil {
ext.EditMessage(p.ChatID, req)
}
}
func shouldUpdateProgress(total, current int64, lastPercent int) bool {
if total == 0 {
return false
}
currentPercent := int((current * 100) / total)
return currentPercent > lastPercent && currentPercent%5 == 0
}
func formatDuration(d time.Duration) string {
d = d.Round(time.Second)
h := d / time.Hour
d -= h * time.Hour
m := d / time.Minute
d -= m * time.Minute
s := d / time.Second
if h > 0 {
return fmt.Sprintf("%dh%dm%ds", h, m, s)
}
if m > 0 {
return fmt.Sprintf("%dm%ds", m, s)
}
return fmt.Sprintf("%ds", s)
}

View File

@@ -1,97 +0,0 @@
package transfer
import (
"context"
"fmt"
"sync"
"sync/atomic"
"github.com/krau/SaveAny-Bot/core"
"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
"github.com/krau/SaveAny-Bot/pkg/storagetypes"
"github.com/krau/SaveAny-Bot/storage"
"github.com/rs/xid"
)
var _ core.Executable = (*Task)(nil)
type TaskElement struct {
ID string
SourceStorage storage.Storage
SourcePath string
FileInfo storagetypes.FileInfo
TargetStorage storage.Storage
TargetPath string
}
type Task struct {
ID string
ctx context.Context
elems []TaskElement
Progress ProgressTracker
IgnoreErrors bool
uploaded atomic.Int64
totalSize int64
processing map[string]TaskElementInfo
processingMu sync.RWMutex
failed map[string]error
}
// Title implements core.Executable.
func (t *Task) Title() string {
return fmt.Sprintf("[%s](%d files/%.2fMB)", t.Type(), len(t.elems), float64(t.totalSize)/(1024*1024))
}
// Type implements core.Executable.
func (t *Task) Type() tasktype.TaskType {
return tasktype.TaskTypeTransfer
}
// TaskID implements core.Executable.
func (t *Task) TaskID() string {
return t.ID
}
func NewTaskElement(
sourceStorage storage.Storage,
fileInfo storagetypes.FileInfo,
targetStorage storage.Storage,
targetPath string,
) *TaskElement {
id := xid.New().String()
return &TaskElement{
ID: id,
SourceStorage: sourceStorage,
SourcePath: fileInfo.Path,
FileInfo: fileInfo,
TargetStorage: targetStorage,
TargetPath: targetPath,
}
}
func NewTransferTask(
id string,
ctx context.Context,
elems []TaskElement,
progress ProgressTracker,
ignoreErrors bool,
) *Task {
task := &Task{
ID: id,
ctx: ctx,
elems: elems,
Progress: progress,
uploaded: atomic.Int64{},
totalSize: func() int64 {
var total int64
for _, elem := range elems {
total += elem.FileInfo.Size
}
return total
}(),
processing: make(map[string]TaskElementInfo),
IgnoreErrors: ignoreErrors,
failed: make(map[string]error),
}
return task
}

View File

@@ -1,73 +0,0 @@
package transfer
type TaskElementInfo interface {
FileName() string
FileSize() int64
GetSourcePath() string
SourceStorageName() string
}
func (e *TaskElement) FileName() string {
return e.FileInfo.Name
}
func (e *TaskElement) FileSize() int64 {
return e.FileInfo.Size
}
func (e *TaskElement) GetSourcePath() string {
return e.SourcePath
}
func (e *TaskElement) SourceStorageName() string {
return e.SourceStorage.Name()
}
type TaskInfo interface {
TaskID() string
TotalSize() int64
Uploaded() int64
Count() int
Processing() []TaskElementInfo
FailedFiles() []string
}
func (t *Task) TotalSize() int64 {
return t.totalSize
}
func (t *Task) Uploaded() int64 {
return t.uploaded.Load()
}
func (t *Task) Count() int {
return len(t.elems)
}
func (t *Task) Processing() []TaskElementInfo {
t.processingMu.RLock()
defer t.processingMu.RUnlock()
result := make([]TaskElementInfo, 0, len(t.processing))
for _, elem := range t.processing {
result = append(result, elem)
}
return result
}
func (t *Task) FailedFiles() []string {
t.processingMu.RLock()
defer t.processingMu.RUnlock()
result := make([]string, 0, len(t.failed))
for id := range t.failed {
// Find the element by ID
for _, elem := range t.elems {
if elem.ID == id {
result = append(result, elem.FileInfo.Name)
break
}
}
}
return result
}

View File

@@ -80,32 +80,22 @@ func (t *Task) Execute(ctx context.Context) error {
func (t *Task) downloadFiles(ctx context.Context, tempDir string) ([]string, error) {
logger := log.FromContext(ctx)
// Configure yt-dlp command with essential settings
// Always set output path to ensure files go to temp directory
// Configure yt-dlp command
cmd := ytdlp.New().
Output(filepath.Join(tempDir, "%(title)s.%(ext)s"))
// Apply config-based format/quality defaults only when the user passes no
// custom flags. Any user flag means they take full control of yt-dlp.
if len(t.Flags) == 0 {
cmd = applyFormatConfig(cmd, config.C().Ytdlp)
}
// Note: If custom flags are provided, users have full control over format/quality
// The output path is always set above to ensure downloads go to the correct directory
FormatSort("res,ext:mp4:m4a").
RecodeVideo("mp4").
Output(filepath.Join(tempDir, "%(title)s.%(ext)s")).
RestrictFilenames()
if t.Progress != nil {
t.Progress.OnProgress(ctx, t, "Downloading...")
}
// Execute download with URLs and custom flags
logger.Infof("Executing yt-dlp for %d URL(s) with %d custom flag(s)", len(t.URLs), len(t.Flags))
// Combine flags and URLs as arguments (flags first, then URLs)
// yt-dlp accepts: yt-dlp [OPTIONS] URL [URL...]
args := append(t.Flags, t.URLs...)
// Execute download with URLs as arguments
logger.Infof("Executing yt-dlp for %d URL(s)", len(t.URLs))
// Run with context for cancellation support
result, err := cmd.Run(ctx, args...)
result, err := cmd.Run(ctx, t.URLs...)
if err != nil {
// Check if context was canceled
if errors.Is(err, context.Canceled) {

View File

@@ -1,40 +0,0 @@
package ytdlp
import (
"strconv"
ytdlp "github.com/lrstanley/go-ytdlp"
"github.com/krau/SaveAny-Bot/config"
)
// buildFormatSelector translates a max height into a yt-dlp format selector.
// It prefers merging the best video+audio within the height limit, then falls
// back to a single muxed stream. An empty result means "no explicit selector".
func buildFormatSelector(maxHeight int) string {
if maxHeight <= 0 {
return ""
}
h := strconv.Itoa(maxHeight)
return "bv*[height<=" + h + "]+ba/b[height<=" + h + "]/b"
}
// applyFormatConfig configures format/quality on the yt-dlp command according to
// the ytdlp config. It is only meant to be called when the user did not supply
// any custom flags, so config-driven defaults never conflict with user input.
func applyFormatConfig(cmd *ytdlp.Command, cfg config.YtdlpConfig) *ytdlp.Command {
switch {
case cfg.Format != "":
cmd = cmd.Format(cfg.Format)
case cfg.MaxHeight > 0:
cmd = cmd.Format(buildFormatSelector(cfg.MaxHeight))
default:
// Preserve the original default: prefer highest resolution mp4/m4a.
cmd = cmd.FormatSort("res,ext:mp4:m4a")
}
if cfg.Recode != "" {
cmd = cmd.RecodeVideo(cfg.Recode)
}
cmd = cmd.RestrictFilenames()
return cmd
}

View File

@@ -1,23 +0,0 @@
package ytdlp
import "testing"
func TestBuildFormatSelector(t *testing.T) {
tests := []struct {
name string
maxHeight int
want string
}{
{"no limit", 0, ""},
{"negative", -1, ""},
{"1080p", 1080, "bv*[height<=1080]+ba/b[height<=1080]/b"},
{"720p", 720, "bv*[height<=720]+ba/b[height<=720]/b"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := buildFormatSelector(tt.maxHeight); got != tt.want {
t.Errorf("buildFormatSelector(%d) = %q, want %q", tt.maxHeight, got, tt.want)
}
})
}
}

View File

@@ -15,7 +15,6 @@ type Task struct {
ID string
ctx context.Context
URLs []string
Flags []string
Storage storage.Storage
StorPath string
Progress ProgressTracker
@@ -44,7 +43,6 @@ func NewTask(
id string,
ctx context.Context,
urls []string,
flags []string,
stor storage.Storage,
storPath string,
progressTracker ProgressTracker,
@@ -53,7 +51,6 @@ func NewTask(
ID: id,
ctx: ctx,
URLs: urls,
Flags: flags,
Storage: stor,
StorPath: storPath,
Progress: progressTracker,

View File

@@ -1,114 +0,0 @@
package ytdlp
import (
"context"
"io"
"testing"
storcfg "github.com/krau/SaveAny-Bot/config/storage"
storenum "github.com/krau/SaveAny-Bot/pkg/enums/storage"
)
// MockStorage is a simple mock for testing
type MockStorage struct{}
func (m *MockStorage) Init(ctx context.Context, cfg storcfg.StorageConfig) error { return nil }
func (m *MockStorage) Type() storenum.StorageType { return "mock" }
func (m *MockStorage) Name() string { return "test-storage" }
func (m *MockStorage) JoinStoragePath(p string) string { return "test-path" }
func (m *MockStorage) Save(ctx context.Context, reader io.Reader, path string) error { return nil }
func (m *MockStorage) Exists(ctx context.Context, path string) bool { return false }
func TestNewTask(t *testing.T) {
ctx := context.Background()
urls := []string{"https://example.com/video"}
flags := []string{"--format", "best"}
stor := &MockStorage{}
storPath := "test-path"
task := NewTask("test-id", ctx, urls, flags, stor, storPath, nil)
if task == nil {
t.Fatal("NewTask returned nil")
}
if task.ID != "test-id" {
t.Errorf("Expected task ID 'test-id', got '%s'", task.ID)
}
if len(task.URLs) != 1 || task.URLs[0] != "https://example.com/video" {
t.Errorf("Expected URLs to contain 'https://example.com/video', got %v", task.URLs)
}
if len(task.Flags) != 2 || task.Flags[0] != "--format" || task.Flags[1] != "best" {
t.Errorf("Expected flags to contain '--format' and 'best', got %v", task.Flags)
}
if task.Storage.Name() != "test-storage" {
t.Errorf("Expected storage name 'test-storage', got '%s'", task.Storage.Name())
}
}
func TestNewTaskWithoutFlags(t *testing.T) {
ctx := context.Background()
urls := []string{"https://example.com/video1", "https://example.com/video2"}
var flags []string // No flags
stor := &MockStorage{}
storPath := "test-path"
task := NewTask("test-id-2", ctx, urls, flags, stor, storPath, nil)
if task == nil {
t.Fatal("NewTask returned nil")
}
if len(task.URLs) != 2 {
t.Errorf("Expected 2 URLs, got %d", len(task.URLs))
}
if len(task.Flags) != 0 {
t.Errorf("Expected 0 flags, got %d", len(task.Flags))
}
}
func TestTaskTitle(t *testing.T) {
ctx := context.Background()
stor := &MockStorage{}
// Test with single URL
task1 := NewTask("id1", ctx, []string{"https://example.com/video"}, nil, stor, "path", nil)
title1 := task1.Title()
if title1 == "" {
t.Error("Task title should not be empty")
}
// Test with multiple URLs
task2 := NewTask("id2", ctx, []string{"https://example.com/v1", "https://example.com/v2"}, nil, stor, "path", nil)
title2 := task2.Title()
if title2 == "" {
t.Error("Task title should not be empty")
}
}
func TestTaskType(t *testing.T) {
ctx := context.Background()
stor := &MockStorage{}
task := NewTask("id", ctx, []string{"https://example.com"}, nil, stor, "path", nil)
taskType := task.Type()
if taskType.String() != "ytdlp" {
t.Errorf("Expected task type 'ytdlp', got '%s'", taskType.String())
}
}
func TestTaskID(t *testing.T) {
ctx := context.Background()
stor := &MockStorage{}
expectedID := "test-task-id-123"
task := NewTask(expectedID, ctx, []string{"https://example.com"}, nil, stor, "path", nil)
if task.TaskID() != expectedID {
t.Errorf("Expected task ID '%s', got '%s'", expectedID, task.TaskID())
}
}

View File

@@ -3,6 +3,7 @@
package database
import (
_ "github.com/ncruces/go-sqlite3/embed"
"github.com/ncruces/go-sqlite3/gormlite"
"gorm.io/gorm"
)

View File

@@ -16,7 +16,6 @@ type User struct {
WatchChats []WatchChat
FilenameStrategy string
FilenameTemplate string
ConflictStrategy string
}
type WatchChat struct {

View File

@@ -49,4 +49,4 @@ func GetUserByID(ctx context.Context, id uint) (*User, error) {
Preload(clause.Associations).
Where("id = ?", id).First(&user).Error
return &user, err
}
}

View File

@@ -20,9 +20,6 @@ Save Any Bot is a tool that allows you to save files from Telegram to various st
- Multi-user
- Automatic organization based on storage rules
- Watch specific chats and automatically save messages, with filters
- Transfer files between different storage backends
- Integrate with yt-dlp to download and save media from 1000+ websites
- Aria2 integration to download files from URLs/magnets and save to storages
- Write JS parser plugins to save files from almost any website
- Supports multiple storage backends:
- Alist

View File

@@ -92,72 +92,6 @@ enable = false
session = "data/usersession.db"
```
### Aria2 Configuration
Aria2 is a powerful download manager that supports HTTP/HTTPS, FTP, BitTorrent, and other protocols. When enabled, the bot can use the `/aria2dl` command to download files via Aria2.
- `enable`: Whether to enable Aria2 support, default is `false`
- `url`: Aria2 RPC address, typically `http://localhost:6800/jsonrpc`
- `secret`: Aria2 RPC secret, if you configured `rpc-secret` in Aria2, you need to fill it in here
- `remove_after_transfer`: Whether to remove local files downloaded by Aria2 after transfer, default is `true`
{{< hint info >}}
Aria2 needs to be installed and running separately. You can refer to the [Aria2 official documentation](https://aria2.github.io/) to learn how to install and configure Aria2.
{{< /hint >}}
```toml
[aria2]
enable = true
url = "http://localhost:6800/jsonrpc"
secret = "your-rpc-secret"
remove_after_transfer = true
```
### yt-dlp Configuration
Configures the behavior of the `/ytdlp` command and the `ytdlp` HTTP-API task type when no custom flags are passed.
- `max_height`: Default maximum video resolution by height in pixels (e.g. `1080`, `720`). `0` means no limit (best available). Ignored when `format` is set.
- `format`: A raw yt-dlp format selector (`-f`). When set, it takes precedence over `max_height` and gives you full control, e.g. `bv*[height<=720]+ba/b`.
- `recode`: The target video container yt-dlp recodes into after download (e.g. `mp4`). Leave empty to disable recoding.
{{< hint info >}}
These defaults only apply when using the `/ytdlp` command without passing any custom flags. Passing custom flags on the command (or `flags` in the API) overrides them.
{{< /hint >}}
```toml
[ytdlp]
max_height = 1080
format = "" # e.g. "bv*[height<=720]+ba/b"
recode = "mp4" # empty disables recoding
```
### HTTP API Configuration
When enabled, SaveAny-Bot exposes an HTTP API for creating/querying/canceling tasks programmatically. See [HTTP API](../../usage/api) for the full endpoint reference.
- `enable`: Whether to enable the HTTP API server, default is `false`.
- `host`: Bind address, default `0.0.0.0`.
- `port`: Listen port, default `8080`.
- `token`: Authentication token. **Strongly recommended** — if empty, the API is exposed without any authentication.
```toml
[api]
enable = false
host = "0.0.0.0"
port = 8080
token = "your-token"
```
### Log Configuration
- `level`: Log level. One of `debug`, `info`, `warn`, `error`, `fatal`. Default is `info`.
```toml
[log]
level = "info"
```
### Storage Endpoints List
The storage endpoints list is used to define the storage locations supported by the Bot. Each storage endpoint needs to specify a name, type, and related configuration, using the double bracket syntax `[[storages]]`.
@@ -171,7 +105,6 @@ Each storage endpoint requires at least the following fields:
- `alist`: Alist
- `webdav`: WebDAV
- `s3`: aws S3 and other S3 compatible services
- `rclone`: Uses rclone to implement uploads
- `telegram`: Upload to Telegram
Example, this is a configuration that includes local storage and webdav storage:

View File

@@ -80,60 +80,4 @@ chat_id = "123456789" # Telegram chat ID, the bot will send files to this chat
force_file = false # Force sending as file, default is false
skip_large = false # Skip large files, default is false. If enabled, files exceeding Telegram's limit will not be uploaded.
spilt_size_mb = 2000 # Split size in MB, default is 2000 MB (2 GB). Files larger than this will be split into multiple parts (zip format). Ignored when skip_large is true.
```
## Rclone
`type=rclone`
Supports multiple cloud storage services through the [rclone](https://rclone.org/) command-line tool. You need to install rclone and configure remote storage first.
```toml
# Remote name configured in rclone, can be any remote defined in rclone.conf
remote = "mydrive"
# Base path in the remote storage, all files will be stored under this path
base_path = "/telegram"
# Path to rclone config file, optional, leave empty to use default path (~/.config/rclone/rclone.conf)
config_path = ""
# Additional flags to pass to rclone commands, optional
flags = ["--transfers", "4", "--checkers", "8"]
```
### Configuring rclone Remote
First, you need to configure an rclone remote. Run `rclone config` for interactive configuration, or directly edit the `rclone.conf` file.
rclone supports many cloud storage services, including but not limited to:
- Google Drive
- Dropbox
- OneDrive
- Amazon S3 and compatible services
- SFTP
- FTP
- For more services, please refer to the [rclone official documentation](https://rclone.org/overview/)
### Usage Examples
After configuring Google Drive, you can configure the storage like this:
```toml
[[storages]]
name = "GoogleDrive"
type = "rclone"
enable = true
remote = "gdrive"
base_path = "/SaveAnyBot"
```
If using a custom rclone config file:
```toml
[[storages]]
name = "MyRemote"
type = "rclone"
enable = true
remote = "myremote"
base_path = "/backup"
config_path = "/path/to/rclone.conf"
flags = ["--progress"]
```

View File

@@ -13,4 +13,112 @@ To use the bot's Telegram file saving feature, you need to send or forward the f
1. File or media messages, such as images, videos, documents, etc.
2. Telegram message links, for example: `https://t.me/acherkrau/1097`. **Even if the channel prohibits forwarding and saving, the bot can still download its files.**
3. Telegra.ph article links. The bot will download all images in the article.
3. Telegra.ph article links. The bot will download all images in the article.
## Silent Mode (silent)
Use the `/silent` command to toggle silent mode.
By default, silent mode is off, and the bot will ask you for the save location of each file.
When silent mode is enabled, the bot will save files directly to the default location without confirmation.
Before enabling silent mode, you need to set the default save location using the `/storage` command.
## Storage Rules
Storage rules allow you to define redirection rules when the bot uploads files to storage, so that saved files are automatically organized.
See: <a href="https://github.com/krau/SaveAny-Bot/issues/28" target="_blank">#28</a>
Currently supported rule types:
1. FILENAME-REGEX
2. MESSAGE-REGEX
3. IS-ALBUM
Basic syntax for adding rules:
"RuleType RuleContent StorageName Path"
Pay attention to spaces; the bot can only parse correctly formatted syntax. Below is an example of a valid rule command:
```
/rule add FILENAME-REGEX (?i)\.(mp4|mkv|ts|avi|flv)$ MyAlist /videos
```
In addition, if `CHOSEN` is used as the storage name in the rule, it means files will be stored under the path of the storage you selected by clicking the inline button.
Rule types:
### FILENAME-REGEX
Matches based on filename regex. The rule content must be a valid regular expression, such as:
```
FILENAME-REGEX (?i)\.(mp4|mkv|ts|avi|flv)$ MyAlist /videos
```
This means files with extensions mp4, mkv, ts, avi, flv will be saved to the `/videos` directory in the storage named `MyAlist` (also affected by the `base_path` in the configuration file).
### MESSAGE-REGEX
Similar to the above, but matches based on the text content of the message itself.
### IS-ALBUM
Matches album messages (media groups). Rule content can only be `true` or `false`.
If the path in the rule uses `NEW-FOR-ALBUM`, the bot will create a new folder for each media group and store all files of that group there. See: https://github.com/krau/SaveAny-Bot/issues/87
For example:
```
IS-ALBUM true MyWebdav NEW-FOR-ALBUM
```
This will save media-group messages to the storage named `MyWebdav`, creating a new folder (generated from the first file) for each album.
## Watch Chats
{{< hint warning >}}
This feature requires enabling UserBot integration.
{{< /hint >}}
You can watch messages in a specific chat and automatically save them to the default storage, following storage rules. You can also add filters so that only matching messages are saved.
Watch a chat:
```
/watch <chat_id/username> [filter]
```
Stop watching:
```
/unwatch <chat_id/username>
```
Filter types:
### msgre
Regex-match the message text. For example:
```
/watch 12345678 msgre:.*hello.*
```
This will watch the chat with ID `12345678`, and only save messages whose text contains `hello`.
## Save Files Outside Telegram
Besides files on Telegram, the bot can also save files from other websites via JavaScript plugins or built-in parsers.
> See the [Contributing Parsers](../contribute) document for details.
Just send links that match the requirements of a parser to the bot. Currently built-in parsers include:
- Twitter
- Kemono

View File

@@ -1,442 +1,257 @@
---
title: "HTTP API"
weight: 20
weight: 4
---
# HTTP API
SaveAny-Bot provides an HTTP API that allows you to programmatically create download/transfer tasks, query task status, cancel tasks, and more — without going through Telegram.
SaveAny-Bot provides a RESTful HTTP API for programmatic file downloads from Telegram.
## Enabling the API
## Configuration
Add or modify the following section in `config.toml`:
Enable the API in your `config.toml`:
```toml
[api]
# Enable HTTP API service
enable = true
host = "0.0.0.0" # Bind address, default 0.0.0.0
port = 8080 # Listen port, default 8080
token = "your-token" # Auth token — strongly recommended
# API server listen port
port = 8080
# API access token (leave empty to disable authentication)
token = "your-secret-token-here"
# Task completion callback webhook URL (leave empty to disable)
webhook_url = "https://your-server.com/webhook"
```
You can also override these settings with environment variables (prefix `SAVEANY_`):
| Environment Variable | Config Key |
|---|---|
| `SAVEANY_API_ENABLE` | `api.enable` |
| `SAVEANY_API_HOST` | `api.host` |
| `SAVEANY_API_PORT` | `api.port` |
| `SAVEANY_API_TOKEN` | `api.token` |
{{< hint warning >}}
If `token` is empty, the API server will be accessible **without any authentication**, which is a security risk.
{{< /hint >}}
## Authentication
When `token` is configured, all API requests must include a Bearer token in the HTTP header:
If `token` is configured, all API requests (except `/health`) must include an `Authorization` header:
```
Authorization: Bearer <your-token>
Authorization: Bearer your-secret-token-here
```
On authentication failure, the server returns `401`:
```json
{ "error": "unauthorized", "message": "invalid token" }
```
## Error Response Format
All errors use a consistent JSON format:
```json
{
"error": "error_code",
"message": "human readable description"
}
```
Common error codes:
| Error Code | HTTP Status | Meaning |
|---|---|---|
| `unauthorized` | 401 | Authentication failed |
| `method_not_allowed` | 405 | Wrong HTTP method |
| `invalid_request` | 400 | Malformed request body or parameters |
| `task_creation_failed` | 400 | Failed to create task |
| `task_not_found` | 404 | Task ID does not exist |
| `cancel_failed` | 500 | Failed to cancel task |
| `internal_error` | 500 | Internal server error |
---
## Endpoints
### GET /health — Health Check
### Health Check
No authentication required.
Check if the API server is running.
**Response `200 OK`:**
```json
{ "status": "ok" }
**Request:**
```
GET /health
```
---
### GET /api/v1/storages — List Storages
Returns all currently loaded storage backends.
**Response `200 OK`:**
**Response:**
```json
{
"storages": [
{ "name": "local", "type": "local" },
{ "name": "MyMinio", "type": "s3" }
]
"status": "ok"
}
```
---
### Create Download Task
### GET /api/v1/task-types — List Supported Task Types
**Response `200 OK`:**
```json
{
"types": [
"directlinks",
"ytdlp",
"aria2",
"parseditem",
"tgfiles",
"tphpics",
"transfer"
]
}
```
---
### POST /api/v1/tasks — Create Task
**Request headers:**
Create a new file download task from a Telegram message link.
**Request:**
```
POST /api/v1/tasks
Content-Type: application/json
Authorization: Bearer <token>
```
Authorization: Bearer your-secret-token-here
**Request body:**
```json
{
"type": "<task_type>",
"storage": "<storage_name>",
"path": "<subpath>",
"webhook": "<callback_url>",
"params": { }
"telegram_url": "https://t.me/channel/123",
"user_id": 123456789,
"storage_name": "local1",
"dir_path": "/downloads"
}
```
| Field | Type | Required | Description |
|---|---|---|---|
| `type` | string | Yes | Task type — see below |
| `storage` | string | Yes | Target storage name, must match a name in your config |
| `path` | string | No | Subdirectory path within the storage |
| `webhook` | string | No | Callback URL invoked when the task reaches a terminal state |
| `params` | object | Yes | Type-specific parameters — see below |
**Response `201 Created`:**
**Request Parameters:**
- `telegram_url` (required): Telegram message link (e.g., `https://t.me/channel/123`)
- `user_id` (required): Telegram user ID (must be configured in `config.toml`)
- `storage_name` (optional): Storage name to use. If not specified, uses the first available storage for the user
- `dir_path` (optional): Directory path in storage. Default is `/`
**Response (201 Created):**
```json
{
"task_id": "abc123xyz",
"type": "directlinks",
"status": "queued",
"created_at": "2026-03-11T10:00:00Z"
"task_id": "c9h8t1234abcd",
"message": "task created successfully"
}
```
#### Task Types and params
##### directlinks — Direct URL Download
Download one or more files from direct HTTP/HTTPS URLs.
**Error Response (4xx/5xx):**
```json
{
"type": "directlinks",
"storage": "local",
"path": "downloads",
"params": {
"urls": [
"https://example.com/file.zip",
"https://example.com/other.zip"
]
}
"error": "error description"
}
```
| params field | Type | Required | Description |
|---|---|---|---|
| `urls` | []string | Yes | List of download URLs, at least 1 |
### Get Task Status
##### ytdlp — yt-dlp Media Download
Get the status of a specific task.
{{< hint warning >}}
Requires yt-dlp to be installed on the system.
{{< /hint >}}
Download videos or audio via yt-dlp, supporting YouTube, Bilibili, and 1000+ other sites.
**Request:**
```
GET /api/v1/tasks/{task_id}
Authorization: Bearer your-secret-token-here
```
**Response (200 OK):**
```json
{
"type": "ytdlp",
"storage": "local",
"path": "videos",
"params": {
"urls": ["https://www.youtube.com/watch?v=xxx"],
"flags": ["--extract-audio", "--audio-format", "mp3"]
}
"task_id": "c9h8t1234abcd",
"status": "completed",
"title": "[tgfiles](file.pdf->local1:/downloads/file.pdf)",
"created_at": "2024-01-19T04:30:00Z",
"error": ""
}
```
| params field | Type | Required | Description |
|---|---|---|---|
| `urls` | []string | Yes | List of media URLs, at least 1 |
| `flags` | []string | No | Extra yt-dlp command-line flags |
**Status Values:**
- `queued`: Task is waiting in queue
- `running`: Task is currently downloading
- `completed`: Task completed successfully
- `failed`: Task failed with error (see `error` field)
- `canceled`: Task was canceled
##### aria2 — Aria2 Download
### List All Tasks
{{< hint warning >}}
Requires Aria2 to be enabled and configured (RPC) in the config file.
{{< /hint >}}
List all queued and running tasks.
Download files via the Aria2 download manager, supporting HTTP/HTTPS, FTP, BitTorrent (magnet links, torrent files), and more.
```json
{
"type": "aria2",
"storage": "local",
"path": "downloads",
"params": {
"urls": ["magnet:?xt=urn:btih:..."],
"options": { "split": "4" }
}
}
**Request:**
```
GET /api/v1/tasks
Authorization: Bearer your-secret-token-here
```
| params field | Type | Required | Description |
|---|---|---|---|
| `urls` | []string | Yes | List of download URIs, at least 1 |
| `options` | map[string]string | No | Aria2 download options |
##### parseditem — Parser Plugin Download
Hand a URL off to a registered JS plugin or built-in parser for processing and downloading.
**Response (200 OK):**
```json
{
"type": "parseditem",
"storage": "local",
"path": "parsed",
"params": {
"url": "https://some-site.com/page"
}
}
```
| params field | Type | Required | Description |
|---|---|---|---|
| `url` | string | Yes | The URL to parse |
Returns `400 task_creation_failed` if no parser is able to handle the URL.
##### tgfiles — Telegram Message File Download
Download files from Telegram messages via message links. Supported link formats:
- `https://t.me/username/123` — public channel or group
- `https://t.me/c/123456789/123` — private channel by numeric ID
- `https://t.me/c/123456789/111/456` — topic message (thread ID / message ID)
- `https://t.me/username/111/456` — topic under a username-based chat
If the message is part of a media group (album), all files in the group are downloaded by default. Append `?single` to the link to force downloading only the single specified message.
```json
{
"type": "tgfiles",
"storage": "local",
"path": "telegram",
"params": {
"message_links": [
"https://t.me/username/123",
"https://t.me/c/1234567890/456"
]
}
}
```
| params field | Type | Required | Description |
|---|---|---|---|
| `message_links` | []string | Yes | List of Telegram message links, at least 1 |
##### tphpics — Telegraph Article Images
Download all images from a Telegra.ph article.
Supported URL prefixes: `https://telegra.ph/`, `http://telegra.ph/`, `https://telegraph.co/`, `http://telegraph.co/`
```json
{
"type": "tphpics",
"storage": "local",
"path": "telegraph",
"params": {
"telegraph_url": "https://telegra.ph/Some-Article-01-01"
}
}
```
| params field | Type | Required | Description |
|---|---|---|---|
| `telegraph_url` | string | Yes | URL of the Telegra.ph article |
##### transfer — Storage-to-Storage Transfer
Transfer files directly between two storage backends without going through Telegram. The source storage must support both listing and reading.
{{< hint info >}}
For `transfer` tasks, the top-level `storage` field is still required for validation, but the actual storages used are determined by `source_storage` and `target_storage` inside `params`.
{{< /hint >}}
```json
{
"type": "transfer",
"storage": "local",
"params": {
"source_storage": "MyS3",
"source_path": "backups/",
"target_storage": "LocalDisk",
"target_path": "restored/"
}
}
```
| params field | Type | Required | Description |
|---|---|---|---|
| `source_storage` | string | Yes | Source storage name |
| `source_path` | string | Yes | Path within the source storage; must contain at least one file |
| `target_storage` | string | Yes | Target storage name |
| `target_path` | string | Yes | Destination path within the target storage |
---
### GET /api/v1/tasks — List All Tasks
Returns all tasks created via the API. Task records are stored in memory only and are cleared on restart.
**Response `200 OK`:**
```json
{
"tasks": [
"queued": [
{
"task_id": "abc123xyz",
"type": "directlinks",
"status": "running",
"title": "file.zip",
"storage": "local",
"path": "downloads",
"error": "",
"created_at": "2026-03-11T10:00:00Z",
"updated_at": "2026-03-11T10:00:05Z",
"progress": {
"total_bytes": 10485760,
"downloaded_bytes": 5242880,
"percent": 50.0
}
"id": "c9h8t1234abcd",
"title": "[tgfiles](file1.pdf->local1:/downloads/file1.pdf)"
}
],
"total": 1
"running": [
{
"id": "d2k9u5678efgh",
"title": "[tgfiles](file2.pdf->local1:/downloads/file2.pdf)"
}
]
}
```
The `progress` field is only included when `total_bytes > 0`. The `error` field is only included when non-empty.
### Cancel Task
---
Cancel a running or queued task.
### GET /api/v1/tasks/{task_id} — Get Task
**Path parameter:** `task_id` — the ID returned when the task was created.
**Response `200 OK`:** Same structure as a single task object from the list above.
**Error responses:**
- `400 invalid_request` — no task ID in path
- `404 task_not_found` — task does not exist
---
### DELETE /api/v1/tasks/{task_id} — Cancel Task
**Path parameter:** `task_id`
**Response `200 OK`:**
```json
{ "message": "task cancelled successfully" }
**Request:**
```
DELETE /api/v1/tasks/{task_id}
Authorization: Bearer your-secret-token-here
```
**Error responses:**
- `400 invalid_request` — no task ID in path
- `404 task_not_found` — task does not exist
- `500 cancel_failed` — cancellation failed
---
## Task Statuses
| Status | Meaning |
|---|---|
| `queued` | Task is queued and waiting to run |
| `running` | Task is currently executing |
| `completed` | Task finished successfully |
| `failed` | Task encountered an error |
| `cancelled` | Task was cancelled via the DELETE endpoint |
---
## Webhook Callbacks
When a `webhook` URL is provided in the create request, SaveAny-Bot sends a `POST` request to that URL when the task reaches a terminal state (`completed`, `failed`, or `cancelled`).
**Callback request headers:**
```
Content-Type: application/json
User-Agent: SaveAny-Bot/1.0
```
**Callback request body:**
**Response (200 OK):**
```json
{
"task_id": "abc123xyz",
"type": "directlinks",
"status": "completed",
"storage": "local",
"path": "downloads",
"completed_at": "2026-03-11T10:01:00Z",
"error": ""
"message": "task canceled"
}
```
`completed_at` is only present when status is `completed` or `failed`. `error` is only present when non-empty.
## Webhook Callback
**Retry policy:** Up to 3 attempts, with delays of 1s, 2s, and 3s between retries. Each request has a 30-second timeout.
If `webhook_url` is configured, the API will send a POST request to the webhook URL when a task completes or fails.
**Webhook Request:**
```
POST {webhook_url}
Content-Type: application/json
Authorization: Bearer your-secret-token-here
{
"task_id": "c9h8t1234abcd",
"status": "completed",
"title": "[tgfiles](file.pdf->local1:/downloads/file.pdf)",
"created_at": "2024-01-19T04:30:00Z",
"error": ""
}
```
## Example Usage
### Using cURL
**Create a download task:**
```bash
curl -X POST http://localhost:8080/api/v1/tasks \
-H "Content-Type: application/json" \
-H "Authorization: Bearer your-secret-token-here" \
-d '{
"telegram_url": "https://t.me/channel/123",
"user_id": 123456789,
"storage_name": "local1",
"dir_path": "/downloads"
}'
```
**Get task status:**
```bash
curl http://localhost:8080/api/v1/tasks/c9h8t1234abcd \
-H "Authorization: Bearer your-secret-token-here"
```
**List all tasks:**
```bash
curl http://localhost:8080/api/v1/tasks \
-H "Authorization: Bearer your-secret-token-here"
```
**Cancel a task:**
```bash
curl -X DELETE http://localhost:8080/api/v1/tasks/c9h8t1234abcd \
-H "Authorization: Bearer your-secret-token-here"
```
### Using Python
```python
import requests
API_URL = "http://localhost:8080"
TOKEN = "your-secret-token-here"
HEADERS = {
"Authorization": f"Bearer {TOKEN}",
"Content-Type": "application/json"
}
# Create a download task
response = requests.post(
f"{API_URL}/api/v1/tasks",
headers=HEADERS,
json={
"telegram_url": "https://t.me/channel/123",
"user_id": 123456789,
"storage_name": "local1",
"dir_path": "/downloads"
}
)
task_id = response.json()["task_id"]
# Get task status
response = requests.get(
f"{API_URL}/api/v1/tasks/{task_id}",
headers=HEADERS
)
status = response.json()
print(f"Task status: {status['status']}")
```
## Security Recommendations
1. **Always use a strong token** for production environments
2. **Use HTTPS** in production by placing the API behind a reverse proxy (e.g., Nginx, Caddy)
3. **Keep logs secure** as they may contain sensitive information
4. **Validate user permissions** - ensure `user_id` in requests corresponds to authorized users in your config

View File

@@ -1,41 +0,0 @@
---
title: "Aria2 Download"
weight: 6
---
# Aria2 Download
{{< hint warning >}}
This feature requires enabling Aria2 in the configuration file and configuring the RPC connection.
{{< /hint >}}
Use the `/aria2dl` command to download files via the Aria2 download manager, supporting HTTP/HTTPS, FTP, BitTorrent, and other protocols.
```bash
/aria2dl <uri1> [uri2] [uri3] ...
```
Examples:
```bash
# Download HTTP link
/aria2dl https://example.com/file.zip
# Download magnet link
/aria2dl magnet:?xt=urn:btih:...
# Download torrent file (need to upload .torrent file first)
/aria2dl https://example.com/file.torrent
```
Configure Aria2:
Add to `config.toml`:
```toml
[aria2]
enable = true
url = "http://localhost:6800/jsonrpc"
secret = "your-rpc-secret" # If rpc-secret is configured
remove_after_transfer = true # Remove local files after transfer
```

View File

@@ -1,90 +0,0 @@
---
title: "CLI Subcommands"
weight: 21
---
# CLI Subcommands
Besides running the Telegram bot with `./saveany-bot` (no subcommand), the binary exposes two helper subcommands for moving local files into a storage backend: `upload` (one-shot) and `watch` (continuous).
These subcommands load the same `config.toml` as the bot, initialize the database and caches, then perform their task. They do **not** start the Telegram bot itself, although storages of type `telegram` will spin up the bot client just for the upload.
## `upload` — Upload a Single File
```
saveany-bot upload -f <file> -s <storage> [-d <dir>] [--no-progress]
```
Flags:
| Flag | Required | Description |
|---|---|---|
| `-f, --file` | Yes | Path to the local file to upload |
| `-s, --storage` | Yes | Target storage name (must exist in `config.toml`) |
| `-d, --dir` | No | Destination directory within the storage. Defaults to the storage's `base_path` |
| `--no-progress` | No | Disable the terminal progress bar |
Examples:
```bash
# Upload a file to the default dir of storage "MyAlist"
./saveany-bot upload -f ./movie.mp4 -s MyAlist
# Upload into a specific subdirectory
./saveany-bot upload -f ./movie.mp4 -s MyAlist -d movies/2026
# Upload via Telegram storage without a progress bar
./saveany-bot upload -f ./photo.jpg -s MyChannel --no-progress
```
## `watch` — Watch a Directory and Auto-Upload
The `watch` subcommand continuously monitors a local directory and uploads created or modified files to a storage backend, preserving the relative directory structure from the watch root.
```
saveany-bot watch -p <path> -s <storage> [-d <dir>] [options]
```
Flags:
| Flag | Default | Description |
|---|---|---|
| `-p, --path` | *(required)* | Local directory to watch |
| `-s, --storage` | *(required)* | Target storage name |
| `-d, --dir` | storage's `base_path` | Destination directory within the storage |
| `-r, --recursive` | `false` | Watch subdirectories recursively |
| `--overwrite` | `false` | Overwrite existing files on the storage instead of skipping them |
| `--initial-scan` | `false` | Upload files already present in the directory on startup |
| `--debounce` | `2s` | How long to wait after the last write before uploading a file |
| `--upload-workers` | `config.workers` | Number of concurrent uploads |
| `--retry-delay` | `3s` | Delay between upload retries |
{{< hint info >}}
Write-completion detection: the watcher debounces per file and only uploads once the file size stays unchanged across the debounce window, so partial/write-in-progress files are not uploaded.
<br />
If a file changes while being uploaded, it is re-uploaded once after the current upload finishes (instead of being queued multiple times).
{{< /hint >}}
Examples:
```bash
# Watch ./inbox and upload new files to "MyAlist" recursively
./saveany-bot watch -p ./inbox -s MyAlist -r
# Watch with a custom destination dir and overwrite
./saveany-bot watch -p ./inbox -s MyAlist -d backup --overwrite
# On startup, also upload everything already in ./inbox
./saveany-bot watch -p ./inbox -s MyAlist --initial-scan
```
### Behavior notes
- Relative directory structure is preserved under the destination directory. A file written to `./inbox/sub/file.txt` with `--path ./inbox` is uploaded to `<dest_dir>/sub/file.txt`.
- `watch` runs until interrupted (e.g. `Ctrl-C` / `SIGINT`); in-flight uploads are drained before exit.
- Retries follow the global `retry` value from `config.toml`, with `--retry-delay` between attempts.
- Telegram-type storages will start the bot client automatically to perform uploads.
{{< hint warning >}}
`watch` is unrelated to the in-bot `/watch` command (which watches Telegram chats). This subcommand watches a **local filesystem directory** and uploads to a storage backend, independent of Telegram.
{{< /hint >}}

View File

@@ -1,75 +0,0 @@
---
title: "File Naming & Conflict Strategies"
weight: 11
---
# File Naming & Conflict Strategies
SaveAny-Bot lets you customize how saved files are named and how collisions with existing files are resolved, directly in Telegram via the `/config` and `/fnametmpl` commands.
## `/config` — User Configuration
The `/config` command opens an inline menu where you can change two per-user settings:
- **Filename strategy** — how the saved file is named
- **Duplicate file strategy** — what happens when a file with the same name already exists in the target storage
Settings are stored per user and apply to all of that user's subsequent save/transfer tasks.
### Filename strategy
| Option | Behavior |
|---|---|
| `Default` | Use the original media filename, or a generated name when no original filename is available |
| `Gen From Msg First` | Generate the filename from the message content (e.g. caption, text) and prefer that over the original filename |
| `Template` | Render the filename from a custom template you define with `/fnametmpl` |
### Duplicate file strategy
| Option | Behavior |
|---|---|
| `Always rename` (default) | Keep the existing file and save the new one with an alternate name |
| `Ask every time` | Prompt you with inline buttons each time a collision occurs |
| `Always overwrite` | Replace the existing file with the new one |
| `Always skip` | Do nothing for conflicting files |
{{< hint info >}}
The conflict strategy only kicks in for storage backends that can detect the existence of a file. Backends that do not support existence checks will fall back to overwriting.
{{< /hint >}}
## `/fnametmpl` — Custom Filename Template
When the filename strategy is set to `Template`, SaveAny-Bot renders each saved file's name using the template configured via `/fnametmpl`.
```
/fnametmpl [template]
```
- Running `/fnametmpl` without arguments shows your current template and the help text.
- Running it with a template string sets that template as your filename template.
The template uses Go [`text/template`](https://pkg.go.dev/text/template) syntax. The available variables are:
| Variable | Description |
|---|---|
| `{{.msgid}}` | Telegram message ID |
| `{{.msgtags}}` | Hashtags found in the message, joined with `_` |
| `{{.msggen}}` | Filename generated from the message |
| `{{.msgdate}}` | Message date, formatted `YYYY-MM-DD_HH-MM-SS` |
| `{{.msgraw}}` | Raw, unprocessed message text |
| `{{.origname}}` | The media's original filename (if any) |
| `{{.chatid}}` | Chat ID of the message |
Examples:
```
# Fixed prefix + message id + date
/fnametmpl Image_{{.msgid}}_{{.msgdate}}.jpg
# Use original name if available, otherwise a generated name
/fnametmpl {{.origname}}
```
{{< hint warning >}}
The template only takes effect when the filename strategy is set to `Template`. If template parsing fails, SaveAny-Bot falls back to the default filename naming logic.
{{< /hint >}}

View File

@@ -1,21 +0,0 @@
---
title: "Direct Download Links"
weight: 5
---
# Direct Download Links
Use the `/dl` command to directly download one or more HTTP/HTTPS files to storage.
```bash
/dl <url1> [url2] [url3] ...
```
Examples:
```bash
/dl https://example.com/file.zip
/dl https://example.com/file1.zip https://example.com/file2.zip
```
The bot will validate the link format and then ask you to select the target storage location.

View File

@@ -1,15 +0,0 @@
---
title: "Save Files Outside Telegram"
weight: 9
---
# Save Files Outside Telegram
Besides files on Telegram, the bot can also save files from other websites via JavaScript plugins or built-in parsers.
> See the [Contributing Parsers](../contribute) document for details.
Just send links that match the requirements of a parser to the bot. Currently built-in parsers include:
- Twitter
- Kemono

View File

@@ -1,97 +0,0 @@
---
title: "Storage Rules"
weight: 3
---
# Storage Rules
Storage rules allow you to define redirection rules when the bot uploads files to storage, so that saved files are automatically organized.
See: <a href="https://github.com/krau/SaveAny-Bot/issues/28" target="_blank">#28</a>
Currently supported rule types:
1. FILENAME-REGEX
2. MESSAGE-REGEX
3. IS-ALBUM
Basic syntax for adding rules:
"RuleType RuleContent StorageName Path"
Pay attention to spaces; the bot can only parse correctly formatted syntax. Below is an example of a valid rule command:
```
/rule add FILENAME-REGEX (?i)\.(mp4|mkv|ts|avi|flv)$ MyAlist /videos
```
In addition, if `CHOSEN` is used as the storage name in the rule, it means files will be stored under the path of the storage you selected by clicking the inline button.
You can also toggle whether rules are applied with `/rule switch`. When rule mode is off, all files go to the default storage.
## Preset Rules
Manually writing regex rules for common file types is tedious, so the bot ships a built-in set of preset categories (video, image, audio, document, archive) that you can import in one command:
```
/rule preset <storage> [base_path]
```
Parameters:
- `storage`: Target storage name (must exist and be accessible to you)
- `base_path`: Optional. Each preset category's subdirectory is created under this path. If omitted, the default category directory names are used directly.
Examples:
```
# Import preset rules into "MyAlist" with the default directory layout
/rule preset MyAlist
# Import preset rules with a custom base path "downloads/sorted"
/rule preset MyAlist downloads/sorted
```
This will create `FILENAME-REGEX` rules for each category, routing matched files to the corresponding subdirectory under `base_path`:
| Category | Matched extensions | Default directory |
|---|---|---|
| video | mp4, mkv, ts, avi, flv, mov, webm, wmv, rmvb, m2ts | `视频` |
| image | jpg, jpeg, png, gif, webp, bmp | `图片` |
| audio | mp3, flac, wav, aac, m4a, ogg | `音频` |
| document | pdf, doc, docx, xls, xlsx, ppt, pptx, txt, md, csv, epub, mobi, azw3, chm | `文档` |
| archive | zip, rar, 7z, tar, gz, bz2, xz, ... | `压缩包` |
{{< hint info >}}
Preset rules are regular `FILENAME-REGEX` rules once imported. You can view, edit, or delete them individually with `/rule` and `/rule del <id>` like any other rule.
{{< /hint >}}
Rule types:
## FILENAME-REGEX
Matches based on filename regex. The rule content must be a valid regular expression, such as:
```
FILENAME-REGEX (?i)\.(mp4|mkv|ts|avi|flv)$ MyAlist /videos
```
This means files with extensions mp4, mkv, ts, avi, flv will be saved to the `/videos` directory in the storage named `MyAlist` (also affected by the `base_path` in the configuration file).
## MESSAGE-REGEX
Similar to the above, but matches based on the text content of the message itself.
## IS-ALBUM
Matches album messages (media groups). Rule content can only be `true` or `false`.
If the path in the rule uses `NEW-FOR-ALBUM`, the bot will create a new folder for each media group and store all files of that group there. See: https://github.com/krau/SaveAny-Bot/issues/87
For example:
```
IS-ALBUM true MyWebdav NEW-FOR-ALBUM
```
This will save media-group messages to the storage named `MyWebdav`, creating a new folder (generated from the first file) for each album.

View File

@@ -1,14 +0,0 @@
---
title: "Silent Mode"
weight: 2
---
# Silent Mode (silent)
Use the `/silent` command to toggle silent mode.
By default, silent mode is off, and the bot will ask you for the save location of each file.
When silent mode is enabled, the bot will save files directly to the default location without confirmation.
Before enabling silent mode, you need to set the default save location using the `/storage` command.

View File

@@ -1,50 +0,0 @@
---
title: "Storage Transfer"
weight: 8
---
# Storage Transfer
Use the `/transfer` command to transfer files directly between different storages without going through Telegram.
```bash
/transfer <source_storage>:/<source_path> [filter]
```
Parameters:
- `source_storage`: Source storage name
- `source_path`: Source path
- `filter`: Optional regex filter to transfer only matching files
Examples:
```bash
# Transfer entire directory
/transfer local1:/downloads
# Transfer files from specified path
/transfer alist1:/media/photos
# Transfer only mp4 files
/transfer webdav1:/videos ".*\.mp4$"
# Transfer image files
/transfer local1:/pictures "(?i)\.(jpg|png|gif)$"
```
The bot will:
1. List all files in the source path
2. Apply the filter (if provided)
3. Display file count and total size
4. Ask you to select the target storage
5. Ask you to select the target directory (if configured for that storage)
6. Start the transfer task
Notes:
- Source storage must support listing and reading
- Target storage must support writing
- Real-time progress is displayed during transfer
- Transfer tasks can be cancelled

View File

@@ -1,36 +0,0 @@
---
title: "Watch Chats"
weight: 4
---
# Watch Chats
{{< hint warning >}}
This feature requires enabling UserBot integration.
{{< /hint >}}
You can watch messages in a specific chat and automatically save them to the default storage, following storage rules. You can also add filters so that only matching messages are saved.
Watch a chat:
```
/watch <chat_id/username> [filter]
```
Stop watching:
```
/unwatch <chat_id/username>
```
Filter types:
## msgre
Regex-match the message text. For example:
```
/watch 12345678 msgre:.*hello.*
```
This will watch the chat with ID `12345678`, and only save messages whose text contains `hello`.

View File

@@ -1,40 +0,0 @@
---
title: "yt-dlp Video Download"
weight: 7
---
# yt-dlp Video Download
{{< hint warning >}}
This feature requires the yt-dlp command-line tool installed on your system.
{{< /hint >}}
Use the `/ytdlp` command to download videos and audio from supported video websites, including YouTube, Bilibili, Twitter, and 1000+ other sites.
```bash
/ytdlp <url1> [url2] [flags...]
```
Examples:
```bash
# Basic download
/ytdlp https://www.youtube.com/watch?v=dQw4w9WgXcQ
# Download multiple videos
/ytdlp https://www.youtube.com/watch?v=video1 https://www.youtube.com/watch?v=video2
# Use custom parameters
/ytdlp https://www.youtube.com/watch?v=dQw4w9WgXcQ -f best
/ytdlp https://www.youtube.com/watch?v=dQw4w9WgXcQ --extract-audio --audio-format mp3
```
Common parameters:
- `-f <format>`: Specify download format (e.g., `best`, `worst`, `bestvideo+bestaudio`)
- `--extract-audio`: Extract audio
- `--audio-format <format>`: Audio format (e.g., `mp3`, `m4a`, `wav`)
- `--write-sub`: Download subtitles
- `--write-thumbnail`: Download thumbnail
For more parameters, see [yt-dlp documentation](https://github.com/yt-dlp/yt-dlp#usage-and-options).

View File

@@ -20,16 +20,12 @@ title: 介绍
- 多用户使用
- 基于存储规则的自动整理
- 监听并自动转存指定聊天的消息, 支持过滤
- 在不同存储端之间转存文件
- 集成 yt-dlp, 从所支持的网站下载并转存媒体文件
- 集成 Aria2, 支持直链/磁力下载和转存
- 使用 js 编写解析器插件以转存任意网站的文件
- 存储端支持:
- Alist
- S3
- WebDAV
- 本地磁盘
- Rclone (通过命令行调用)
- Telegram (重传回指定聊天)
## [贡献者](https://github.com/krau/SaveAny-Bot/graphs/contributors)

View File

@@ -30,7 +30,6 @@ base_path = "./downloads"
### 全局配置
- `lang`: Bot 使用的语言, 默认为 `zh-CN` (简体中文), 设为 `en` 则使用英语.
- `stream`: 是否启用 Stream 模式, 默认为 `false`. 启用后 Bot 将直接将文件流式传输到存储端(若存储端支持), 不需要下载到本地
{{< hint warning >}}
Stream 模式对于磁盘空间有限的部署环境十分有用, 但也有一些弊端:
@@ -48,7 +47,6 @@ Stream 模式对于磁盘空间有限的部署环境十分有用, 但也有一
- `proxy`: 全局代理配置, 配置后程序内一切网络连接将会尝试使用该代理, 可选.
```toml
lang = "zh-CN"
stream = false
workers = 3
threads = 4
@@ -92,72 +90,6 @@ enable = false
session = "data/usersession.db"
```
### Aria2 配置
Aria2 是一个强大的下载管理器,支持 HTTP/HTTPS、FTP、BitTorrent 等多种协议。启用后Bot 可以使用 `/aria2dl` 命令通过 Aria2 下载文件。
- `enable`: 是否启用 Aria2 支持,默认为 `false`
- `url`: Aria2 RPC 地址,通常为 `http://localhost:6800/jsonrpc`
- `secret`: Aria2 RPC 密钥,如果你在 Aria2 中配置了 `rpc-secret`,需要在此填写
- `remove_after_transfer`: 转存完成后是否删除 Aria2 下载的本地文件,默认为 `true`
{{< hint info >}}
Aria2 需要单独安装和运行。你可以参考 [Aria2 官方文档](https://aria2.github.io/) 了解如何安装和配置 Aria2。
{{< /hint >}}
```toml
[aria2]
enable = true
url = "http://localhost:6800/jsonrpc"
secret = "your-rpc-secret"
remove_after_transfer = true
```
### yt-dlp 配置
用于配置 `/ytdlp` 命令以及 HTTP API 中 `ytdlp` 任务类型在未传自定义参数时的默认行为.
- `max_height`: 默认下载的最高视频清晰度 (按高度限制), 如 `1080`, `720`, `480`; `0` 表示不限制 (下载最佳画质). 当设置了 `format` 时此项被忽略.
- `format`: 直接指定 yt-dlp format 选择表达式, 设置后优先级高于 `max_height`, 例如 `bv*[height<=720]+ba/b`.
- `recode`: 下载后转封装的视频容器格式 (如 `mp4`), 留空则不转封装.
{{< hint info >}}
这些默认值仅在使用 `/ytdlp` 命令且未传任何自定义参数时生效. 在命令上传递自定义参数 (或在 API 中传 `flags`) 会覆盖这些默认值.
{{< /hint >}}
```toml
[ytdlp]
max_height = 1080
format = "" # 例如 "bv*[height<=720]+ba/b"
recode = "mp4" # 留空则不转封装
```
### HTTP API 配置
启用后, SaveAny-Bot 会暴露一套 HTTP API, 用于以编程方式创建/查询/取消任务. 完整的接口说明见 [HTTP API](../../usage/api).
- `enable`: 是否启用 HTTP API 服务, 默认为 `false`.
- `host`: 监听地址, 默认 `0.0.0.0`.
- `port`: 监听端口, 默认 `8080`.
- `token`: 鉴权 Token, **强烈建议设置** — 若为空, API 将在无任何鉴权的情况下暴露.
```toml
[api]
enable = false
host = "0.0.0.0"
port = 8080
token = "your-token"
```
### 日志配置
- `level`: 日志级别, 可选 `debug`, `info`, `warn`, `error`, `fatal`. 默认为 `info`.
```toml
[log]
level = "info"
```
### 存储端列表
存储端列表用于定义 Bot 支持的存储位置, 每个存储端需要指定名称、类型和相关配置, 使用双中括号语法 `[[storages]]` 定义.
@@ -171,7 +103,6 @@ level = "info"
- `alist`: Alist
- `webdav`: WebDAV
- `s3`: aws S3 及其他兼容 S3 的服务
- `rclone`: 调用 rclone 实现上传
- `telegram`: 上传到 Telegram
示例, 这是一个包含本地存储和 webdav 存储的配置:

View File

@@ -86,60 +86,4 @@ skip_large = false
# 超过该大小的文件将被分割成多个部分上传.(使用 zip 格式)
# 当 skip_large 启用时, 该选项无效.
spilt_size_mb = 2000
```
## Rclone
`type=rclone`
通过 [rclone](https://rclone.org/) 命令行工具支持多种云存储服务. 需要先安装 rclone 并配置好远程存储.
```toml
# rclone 配置的远程名称, 可以是任何在 rclone.conf 中配置的远程
remote = "mydrive"
# 在远程存储中的基础路径, 所有文件将存储在此路径下
base_path = "/telegram"
# rclone 配置文件的路径, 可选, 留空使用默认路径 (~/.config/rclone/rclone.conf)
config_path = ""
# 传递给 rclone 命令的额外参数, 可选
flags = ["--transfers", "4", "--checkers", "8"]
```
### 配置 rclone 远程
首先需要配置 rclone 远程, 运行 `rclone config` 命令进行交互式配置, 或直接编辑 `rclone.conf` 文件.
rclone 支持多种云存储服务, 包括但不限于:
- Google Drive
- Dropbox
- OneDrive
- Amazon S3 及兼容服务
- SFTP
- FTP
- 更多服务请参考 [rclone 官方文档](https://rclone.org/overview/)
### 使用示例
配置 Google Drive 后, 可以这样配置存储:
```toml
[[storages]]
name = "GoogleDrive"
type = "rclone"
enable = true
remote = "gdrive"
base_path = "/SaveAnyBot"
```
如果使用自定义的 rclone 配置文件:
```toml
[[storages]]
name = "MyRemote"
type = "rclone"
enable = true
remote = "myremote"
base_path = "/backup"
config_path = "/path/to/rclone.conf"
flags = ["--progress"]
```

Some files were not shown because too many files have changed in this diff Show More