refactor: js api

* feat: WIP. add parser functionality and text message handling

* fix: use json to marshal js result

* feat: add metadata handling and version validation for jsParser

* refactor: rename parser package to parsers and restructure parser handling

* refactor: core code struct and impl parse task handle

* feat: impl parsed download

* fix: seek cache file when processing tph picture

* feat: implement parsed task handling and progress tracking

* feat: enhance task processing with concurrency control and progress tracking

* feat: add resource ID generation and improve resource processing handling

* feat: improve message formatting in parsed text and progress completion

* feat: add example js plugin

* feat: implement Twitter parser

* fix: twitter parse video json decode error

* feat: impl stream mode for parse task

.github/workflows/build-docker.yml

@@ -44,6 +44,12 @@ jobs:
           username: ${{ github.actor }}
           password: ${{ secrets.GITHUB_TOKEN }}
 
+      - name: Extract Dockerfile args
+        id: args
+        run: |
+          echo "git_commit=$(git rev-parse --short HEAD)" >> "$GITHUB_OUTPUT"
+          echo "build_time=$(git show -s --format=%cI)" >> "$GITHUB_OUTPUT"
+
       - name: Build and push Docker image
         id: build-and-push
         uses: docker/build-push-action@v6
@@ -59,5 +65,5 @@ jobs:
           cache-to: type=gha,mode=max
           build-args: |
             VERSION=${{ steps.meta.outputs.version }}
-            GitCommit=${{ github.sha }}
-            BuildTime=${{ fromJson(toJSON(github.event.repository.pushed_at)) }}
+            GitCommit=${{ steps.args.outputs.git_commit }}
+            BuildTime=${{ steps.args.outputs.build_time }}

.github/workflows/build-release.yml

@@ -63,9 +63,9 @@ jobs:
             README.md
           ldflags: >-
             -s -w
-            -X "github.com/krau/SaveAny-Bot/pkg/consts.Version=${{ env.VERSION }}"
-            -X "github.com/krau/SaveAny-Bot/pkg/consts.BuildTime=${{ format(github.event.repository.updated_at, 'yyyy-MM-dd HH:mm:ss') }}"
-            -X "github.com/krau/SaveAny-Bot/pkg/consts.GitCommit=${{ github.sha }}"
+            -X "github.com/krau/SaveAny-Bot/config.Version=${{ env.VERSION }}"
+            -X "github.com/krau/SaveAny-Bot/config.BuildTime=${{ format(github.event.repository.updated_at, 'yyyy-MM-dd HH:mm:ss') }}"
+            -X "github.com/krau/SaveAny-Bot/config.GitCommit=${{ github.sha }}"
           binary_name: saveany-bot
         env:
           VERSION: ${{ env.VERSION }}

.gitignore

@@ -7,4 +7,4 @@ session.*
 cache.db
 .vscode/
 temp/
-.hugo_build.lock
+.hugo_build.lock

Dockerfile

@@ -13,12 +13,14 @@ RUN --mount=type=cache,target=/go/pkg/mod \
 COPY . .
 RUN --mount=type=cache,target=/root/.cache/go-build \
     --mount=type=cache,target=/go/pkg \
-    CGO_ENABLED=0 \ 
-    go build -trimpath \ 
-    -ldflags "-s -w \ 
-    -X github.com/krau/SaveAny-Bot/common.Version=${VERSION} \
-    -X github.com/krau/SaveAny-Bot/common.GitCommit=${GitCommit} \
-    -X github.com/krau/SaveAny-Bot/common.BuildTime=${BuildTime}" \
+    CGO_ENABLED=0 \
+    go build -trimpath \
+    -ldflags=" \
+    -s -w \
+    -X 'github.com/krau/SaveAny-Bot/config.Version=${VERSION}' \
+    -X 'github.com/krau/SaveAny-Bot/config.GitCommit=${GitCommit}' \
+    -X 'github.com/krau/SaveAny-Bot/config.BuildTime=${BuildTime}' \
+    " \
     -o saveany-bot .
 
 FROM alpine:latest
@@ -33,4 +35,4 @@ COPY entrypoint.sh .
 RUN chmod +x /app/saveany-bot && \
     chmod +x /app/entrypoint.sh
 
-ENTRYPOINT ["/app/entrypoint.sh"]
+ENTRYPOINT ["/app/entrypoint.sh"]

README.md

@@ -4,28 +4,70 @@
 
 **简体中文** | [English](https://sabot.unv.app/en/)
 
-把 Telegram 上的文件转存到多种存储端.
+> **把 Telegram 上的文件转存到多种存储端.**
+
+[![Release Date](https://img.shields.io/github/release-date/krau/saveany-bot?label=release)](https://github.com/krau/saveany-bot/releases)
+[![tag](https://img.shields.io/github/v/tag/krau/saveany-bot.svg)](https://github.com/krau/saveany-bot/releases)
+[![Build Status](https://img.shields.io/github/actions/workflow/status/krau/saveany-bot/build-release.yml)](https://github.com/krau/saveany-bot/actions/workflows/build-release.yml)
+[![Stars](https://img.shields.io/github/stars/krau/saveany-bot?style=flat)](https://github.com/krau/saveany-bot/stargazers)
+[![Downloads](https://img.shields.io/github/downloads/krau/saveany-bot/total)](https://github.com/krau/saveany-bot/releases)
+[![Issues](https://img.shields.io/github/issues/krau/saveany-bot)](https://github.com/krau/saveany-bot/issues)
+[![Pull Requests](https://img.shields.io/github/issues-pr/krau/saveany-bot?label=pr)](https://github.com/krau/saveany-bot/pulls)
+[![License](https://img.shields.io/github/license/krau/saveany-bot)](./LICENSE)
 
 </div>
 
-## 部署
+## 🎯 Features
 
-请参考 [部署文档](https://sabot.unv.app/deployment/installation/)
-
-## Features
-
-- 支持文档/视频/图片/贴纸… 甚至还有 Telegraph
+- 支持文档/视频/图片/贴纸…甚至还有 [Telegraph](https://telegra.ph/)
 - 破解禁止保存的文件
 - 批量下载
 - 流式传输
-- 多用户
+- 多用户使用
 - 基于存储规则的自动整理
-- 支持多种存储端:
+- 监听并自动转存指定聊天的消息, 支持过滤
+- 使用 js 编写解析器插件以转存任意网站的文件
+- 存储端支持:
   - Alist
-  - Minio (S3 兼容)
+  - S3 (MinioSDK)
   - WebDAV
-  - Telegram (重传回指定聊天)
   - 本地磁盘
+  - Telegram (重传回指定聊天)
+
+## 📦 Quick Start
+
+创建文件 `config.toml` 并填入以下内容:
+
+```toml
+[telegram]
+token = "" # 你的 Bot Token, 在 @BotFather 获取
+[telegram.proxy]
+# 启用代理连接 telegram, 当前只支持 socks5
+enable = false
+url = "socks5://127.0.0.1:7890"
+
+[[storages]]
+name = "本地磁盘"
+type = "local"
+enable = true
+base_path = "./downloads"
+
+[[users]]
+id = 114514 # 你的 Telegram 账号 id
+storages = []
+blacklist = true
+```
+
+使用 Docker 运行 Save Any Bot:
+
+```bash
+docker run -d --name saveany-bot \
+    -v ./config.toml:/app/config.toml \
+    -v ./downloads:/app/downloads \
+    ghcr.io/krau/saveany-bot:latest
+```
+
+请 [**查看文档**](https://sabot.unv.app/) 以获取更多配置选项和使用方法.
 
 ## Sponsors
 
@@ -88,3 +130,9 @@
 - [gotgproto](https://github.com/celestix/gotgproto)
 - [tdl](https://github.com/iyear/tdl)
 - All the dependencies
+
+## Contact
+
+- [![Group](https://img.shields.io/badge/ProjectSaveAny-Group-blue)](https://t.me/ProjectSaveAny)
+- [![Discussion](https://img.shields.io/badge/Github-Discussion-white)](https://github.com/krau/saveany-bot/discussions)
+- [![PersonalChannel](https://img.shields.io/badge/Krau-PersonalChannel-cyan)](https://t.me/acherkrau)

client/bot/bot.go

@@ -19,16 +19,17 @@ import (
 	"golang.org/x/net/proxy"
 )
 
-func Init(ctx context.Context) {
+func Init(ctx context.Context) (<-chan struct{}) {
 	log.FromContext(ctx).Info("初始化 Bot...")
 	resultChan := make(chan struct {
 		client *gotgproto.Client
 		err    error
 	})
+	shouldRestart := make(chan struct{})
 	go func() {
 		var resolver dcs.Resolver
-		if config.Cfg.Telegram.Proxy.Enable && config.Cfg.Telegram.Proxy.URL != "" {
-			dialer, err := netutil.NewProxyDialer(config.Cfg.Telegram.Proxy.URL)
+		if config.C().Telegram.Proxy.Enable && config.C().Telegram.Proxy.URL != "" {
+			dialer, err := netutil.NewProxyDialer(config.C().Telegram.Proxy.URL)
 			if err != nil {
 				resultChan <- struct {
 					client *gotgproto.Client
@@ -43,19 +44,23 @@ func Init(ctx context.Context) {
 			resolver = dcs.DefaultResolver()
 		}
 		client, err := gotgproto.NewClient(
-			config.Cfg.Telegram.AppID,
-			config.Cfg.Telegram.AppHash,
-			gotgproto.ClientTypeBot(config.Cfg.Telegram.Token),
+			config.C().Telegram.AppID,
+			config.C().Telegram.AppHash,
+			gotgproto.ClientTypeBot(config.C().Telegram.Token),
 			&gotgproto.ClientOpts{
-				Session:          sessionMaker.SqlSession(gormlite.Open(config.Cfg.DB.Session)),
+				Session:          sessionMaker.SqlSession(gormlite.Open(config.C().DB.Session)),
 				DisableCopyright: true,
 				Middlewares:      middleware.NewDefaultMiddlewares(ctx, 5*time.Minute),
 				Resolver:         resolver,
 				Context:          ctx,
-				MaxRetries:       config.Cfg.Telegram.RpcRetry,
+				MaxRetries:       config.C().Telegram.RpcRetry,
 				AutoFetchReply:   true,
 				ErrorHandler: func(ctx *ext.Context, u *ext.Update, s string) error {
-					log.FromContext(ctx).Errorf("Unhandled error: %s", s)
+					if s == "SAVEANTBOT-RESTART" {
+						shouldRestart <- struct{}{}
+						return dispatcher.EndGroups
+					}
+					log.FromContext(ctx).Errorf("unhandled error: %s", s)
 					return dispatcher.EndGroups
 				},
 			},
@@ -70,17 +75,22 @@ func Init(ctx context.Context) {
 		client.API().BotsSetBotCommands(ctx, &tg.BotsSetBotCommandsRequest{
 			Scope: &tg.BotCommandScopeDefault{},
 		})
+		commands := []tg.BotCommand{
+			{Command: "start", Description: "开始使用"},
+			{Command: "help", Description: "显示帮助"},
+			{Command: "silent", Description: "开启/关闭静默模式"},
+			{Command: "storage", Description: "设置默认存储端"},
+			{Command: "save", Description: "保存文件"},
+			{Command: "dir", Description: "管理存储文件夹"},
+			{Command: "rule", Description: "管理规则"},
+		}
+		if config.C().Telegram.Userbot.Enable {
+			commands = append(commands, tg.BotCommand{Command: "watch", Description: "监听聊天"})
+			commands = append(commands, tg.BotCommand{Command: "unwatch", Description: "取消监听聊天"})
+		}
 		_, err = client.API().BotsSetBotCommands(ctx, &tg.BotsSetBotCommandsRequest{
-			Scope: &tg.BotCommandScopeDefault{},
-			Commands: []tg.BotCommand{
-				{Command: "start", Description: "开始使用"},
-				{Command: "help", Description: "显示帮助"},
-				{Command: "silent", Description: "开启/关闭静默模式"},
-				{Command: "storage", Description: "设置默认存储端"},
-				{Command: "save", Description: "保存所回复的文件"},
-				{Command: "dir", Description: "管理存储文件夹"},
-				{Command: "rule", Description: "管理规则"},
-			},
+			Scope:    &tg.BotCommandScopeDefault{},
+			Commands: commands,
 		})
 		resultChan <- struct {
 			client *gotgproto.Client
@@ -98,4 +108,5 @@ func Init(ctx context.Context) {
 		handlers.Register(result.client.Dispatcher)
 		log.FromContext(ctx).Info("Bot 初始化完成")
 	}
+	return shouldRestart
 }

client/bot/handlers/add_task.go

@@ -3,6 +3,7 @@ package handlers
 import (
 	"errors"
 	"fmt"
+	"path"
 	"strings"
 
 	"github.com/celestix/gotgproto/dispatcher"
@@ -11,6 +12,7 @@ import (
 	"github.com/gotd/td/tg"
 	"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/msgelem"
 	"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/shortcut"
+	"github.com/krau/SaveAny-Bot/common/utils/fsutil"
 	"github.com/krau/SaveAny-Bot/database"
 	"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
 	"github.com/krau/SaveAny-Bot/pkg/tcbdata"
@@ -72,7 +74,12 @@ func handleAddCallback(ctx *ext.Context, update *ext.Update) error {
 		}
 		return shortcut.CreateAndAddTGFileTaskWithEdit(ctx, userID, selectedStorage, dirPath, data.Files[0], msgID)
 	case tasktype.TaskTypeTphpics:
-		return shortcut.CreateAndAddTphTaskWithEdit(ctx, userID, data.TphPageNode, data.TphDirPath, data.TphPics, selectedStorage, msgID)
+		return shortcut.CreateAndAddtelegraphWithEdit(ctx, userID, data.TphPageNode, data.TphDirPath, data.TphPics, selectedStorage, msgID)
+	case tasktype.TaskTypeParseditem:
+		if len(data.ParsedItem.Resources) > 1 {
+			dirPath = path.Join(dirPath, fsutil.NormalizePathname(data.ParsedItem.Title))
+		}
+		shortcut.CreateAndAddParsedTaskWithEdit(ctx, selectedStorage, dirPath, data.ParsedItem, msgID, userID)
 	default:
 		log.FromContext(ctx).Errorf("Unsupported task type: %s", data.TaskType)
 	}

client/bot/handlers/config.go

@@ -0,0 +1,103 @@
+package handlers
+
+import (
+	"fmt"
+	"strings"
+
+	"github.com/celestix/gotgproto/dispatcher"
+	"github.com/celestix/gotgproto/ext"
+	"github.com/gotd/td/tg"
+	"github.com/krau/SaveAny-Bot/database"
+	"github.com/krau/SaveAny-Bot/pkg/enums/fnamest"
+	"github.com/krau/SaveAny-Bot/pkg/tcbdata"
+)
+
+func handleConfigCmd(ctx *ext.Context, update *ext.Update) error {
+	ctx.Reply(update, ext.ReplyTextString("请选择要配置的选项"), &ext.ReplyOpts{
+		Markup: &tg.ReplyInlineMarkup{
+			Rows: []tg.KeyboardButtonRow{
+				{
+					Buttons: []tg.KeyboardButtonClass{
+						&tg.KeyboardButtonCallback{
+							Text: "文件名策略",
+							Data: fmt.Appendf(nil, "%s %s", tcbdata.TypeConfig, "fnamest"),
+						},
+					},
+				},
+			},
+		},
+	})
+	return dispatcher.EndGroups
+}
+
+func handleConfigCallback(ctx *ext.Context, update *ext.Update) error {
+	args := strings.Fields(string(update.CallbackQuery.Data))
+	invaildDataAnswer := func() error {
+		ctx.AnswerCallback(&tg.MessagesSetBotCallbackAnswerRequest{
+			QueryID:   update.CallbackQuery.GetQueryID(),
+			Alert:     true,
+			Message:   "无效的回调数据",
+			CacheTime: 5,
+		})
+		return dispatcher.EndGroups
+	}
+	if len(args) < 2 {
+		return invaildDataAnswer()
+	}
+	switch args[1] {
+	case "fnamest":
+		return handleConfigFnameSTCallback(ctx, update)
+	default:
+		return invaildDataAnswer()
+	}
+}
+
+func handleConfigFnameSTCallback(ctx *ext.Context, update *ext.Update) error {
+	userID := update.CallbackQuery.GetUserID()
+	user, err := database.GetUserByChatID(ctx, userID)
+	if err != nil {
+		return err
+	}
+	args := strings.Fields(string(update.CallbackQuery.Data))
+	if len(args) == 3 {
+		selected := args[2]
+		st, err := fnamest.ParseFnameST(selected)
+		if err != nil {
+			return err
+		}
+		user.FilenameStrategy = st.String()
+		if err := database.UpdateUser(ctx, user); err != nil {
+			return err
+		}
+		ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
+			ID:      update.CallbackQuery.GetMsgID(),
+			Message: fmt.Sprintf("已将文件名策略设置为: %s", fnamest.FnameSTDisplay[st]),
+		})
+		return dispatcher.EndGroups
+	}
+	opts := fnamest.FnameSTValues()
+	buttons := make([]tg.KeyboardButtonClass, 0, len(opts))
+	for _, opt := range opts {
+		buttons = append(buttons, &tg.KeyboardButtonCallback{
+			Text: fnamest.FnameSTDisplay[opt],
+			Data: fmt.Appendf(nil, "%s %s %s", tcbdata.TypeConfig, "fnamest", opt),
+		})
+	}
+	markup := &tg.ReplyInlineMarkup{Rows: []tg.KeyboardButtonRow{
+		{Buttons: buttons},
+	}}
+	currentStStr := user.FilenameStrategy
+	if currentStStr == "" {
+		currentStStr = fnamest.Default.String()
+	}
+	currentSt, err := fnamest.ParseFnameST(currentStStr)
+	if err != nil {
+		currentSt = fnamest.Default
+	}
+	ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
+		ID:          update.CallbackQuery.GetMsgID(),
+		Message:     fmt.Sprintf("请选择文件名策略, 当前策略: %s", fnamest.FnameSTDisplay[currentSt]),
+		ReplyMarkup: markup,
+	})
+	return dispatcher.EndGroups
+}

client/bot/handlers/help.go

@@ -5,7 +5,7 @@ import (
 
 	"github.com/celestix/gotgproto/dispatcher"
 	"github.com/celestix/gotgproto/ext"
-	"github.com/krau/SaveAny-Bot/pkg/consts"
+	"github.com/krau/SaveAny-Bot/config"
 )
 
 func handleHelpCmd(ctx *ext.Context, update *ext.Update) error {
@@ -19,13 +19,17 @@ Save Any Bot - 转存你的 Telegram 文件
 /silent - 开关静默模式
 /storage - 设置默认存储位置
 /save [自定义文件名] - 保存文件
+/dir - 管理存储目录
+/rule - 管理规则
+/update - 检查更新并升级
 
-使用帮助: https://sabot.unv.app/usage/
+使用帮助: https://sabot.unv.app/usage
+反馈群组: https://t.me/ProjectSaveAny
 `
-	shortHash := consts.GitCommit
+	shortHash := config.GitCommit
 	if len(shortHash) > 7 {
 		shortHash = shortHash[:7]
 	}
-	ctx.Reply(update, ext.ReplyTextString(fmt.Sprintf(helpText, consts.Version, shortHash)), nil)
+	ctx.Reply(update, ext.ReplyTextString(fmt.Sprintf(helpText, config.Version, shortHash)), nil)
 	return dispatcher.EndGroups
 }

client/bot/handlers/media.go

@@ -13,6 +13,8 @@ import (
 	"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/msgelem"
 	"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/shortcut"
 	"github.com/krau/SaveAny-Bot/common/utils/tgutil"
+	"github.com/krau/SaveAny-Bot/database"
+	"github.com/krau/SaveAny-Bot/pkg/enums/fnamest"
 	"github.com/krau/SaveAny-Bot/pkg/tcbdata"
 	"github.com/krau/SaveAny-Bot/pkg/tfile"
 	"github.com/krau/SaveAny-Bot/storage"
@@ -26,12 +28,22 @@ func handleMediaMessage(ctx *ext.Context, update *ext.Update) error {
 		return handleGroupMediaMessage(ctx, update, message, groupID)
 	}
 	logger.Debugf("Got media: %s", message.Media.TypeName())
-
-	msg, file, err := shortcut.GetFileFromMessageWithReply(ctx, update, message)
+	userId := update.GetUserChat().GetID()
+	userDB, err := database.GetUserByChatID(ctx, userId)
 	if err != nil {
 		return err
 	}
-	userId := update.GetUserChat().GetID()
+	tfOpts := make([]tfile.TGFileOption, 0)
+	switch userDB.FilenameStrategy {
+	case fnamest.Message.String():
+		tfOpts = append(tfOpts, tfile.WithName(tgutil.GenFileNameFromMessage(*message)))
+	default:
+	}
+	msg, file, err := shortcut.GetFileFromMessageWithReply(ctx, update, message, tfOpts...)
+	if err != nil {
+		return err
+	}
+
 	stors := storage.GetUserStorages(ctx, userId)
 	req, err := msgelem.BuildAddOneSelectStorageMessage(ctx, stors, file, msg.ID)
 	if err != nil {
@@ -58,7 +70,17 @@ func handleSilentSaveMedia(ctx *ext.Context, update *ext.Update) error {
 	}
 	logger.Debugf("Got media: %s", message.Media.TypeName())
 	userID := update.GetUserChat().GetID()
-	msg, file, err := shortcut.GetFileFromMessageWithReply(ctx, update, message)
+	userDB, err := database.GetUserByChatID(ctx, userID)
+	if err != nil {
+		return err
+	}
+	tfOpts := make([]tfile.TGFileOption, 0)
+	switch userDB.FilenameStrategy {
+	case fnamest.Message.String():
+		tfOpts = append(tfOpts, tfile.WithName(tgutil.GenFileNameFromMessage(*message)))
+	default:
+	}
+	msg, file, err := shortcut.GetFileFromMessageWithReply(ctx, update, message, tfOpts...)
 	if err != nil {
 		return err
 	}

client/bot/handlers/middleware.go

@@ -11,7 +11,7 @@ import (
 
 func checkPermission(ctx *ext.Context, update *ext.Update) error {
 	userID := update.GetUserChat().GetID()
-	if !slice.Contain(config.Cfg.GetUsersID(), userID) {
+	if !slice.Contain(config.C().GetUsersID(), userID) {
 		const noPermissionText string = `
 您不在白名单中, 无法使用此 Bot.
 您可以部署自己的实例: https://github.com/krau/SaveAny-Bot

client/bot/handlers/parse.go

@@ -0,0 +1,115 @@
+// 处理任意文本消息, 用于通用地从外部源下载文件
+
+package handlers
+
+import (
+	"errors"
+
+	"github.com/celestix/gotgproto/dispatcher"
+	"github.com/celestix/gotgproto/ext"
+	"github.com/charmbracelet/log"
+	"github.com/gotd/td/tg"
+	"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/msgelem"
+	"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/shortcut"
+	"github.com/krau/SaveAny-Bot/common/utils/fsutil"
+	"github.com/krau/SaveAny-Bot/parsers"
+	"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
+	"github.com/krau/SaveAny-Bot/pkg/tcbdata"
+	"github.com/krau/SaveAny-Bot/storage"
+)
+
+func handleTextMessage(ctx *ext.Context, u *ext.Update) error {
+	logger := log.FromContext(ctx)
+	text := u.EffectiveMessage.Text
+	ok, pser := parsers.CanHandle(text)
+	if !ok {
+		return dispatcher.EndGroups
+	}
+	msg, err := ctx.Reply(u, ext.ReplyTextString("正在解析..."), nil)
+	if err != nil {
+		return err
+	}
+
+	item, err := pser.Parse(ctx, text)
+	if errors.Is(err, parsers.ErrNoParserFound) {
+		return dispatcher.EndGroups
+	}
+	if err != nil {
+		logger.Error("Failed to parse text", "error", err)
+		ctx.Reply(u, ext.ReplyTextString("Failed to parse text: "+err.Error()), nil)
+		return dispatcher.EndGroups
+	}
+	logger.Debug("Parsed item from text message", "title", item.Title, "url", item.URL)
+	userID := u.GetUserChat().GetID()
+	markup, err := msgelem.BuildAddSelectStorageKeyboard(storage.GetUserStorages(ctx, userID), tcbdata.Add{
+		TaskType:   tasktype.TaskTypeParseditem,
+		ParsedItem: item,
+	})
+	if err != nil {
+		logger.Errorf("Failed to build storage selection keyboard: %s", err)
+		ctx.Reply(u, ext.ReplyTextString("Failed to build storage selection keyboard: "+err.Error()), nil)
+		return dispatcher.EndGroups
+	}
+	text, entities, err := msgelem.BuildParsedTextEntity(*item)
+	if err != nil {
+		logger.Errorf("Failed to build parsed text entity: %s", err)
+		ctx.Reply(u, ext.ReplyTextString("Failed to build parsed text entity: "+err.Error()), nil)
+		return dispatcher.EndGroups
+	}
+	ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
+		Message:     text,
+		ReplyMarkup: markup,
+		Entities:    entities,
+		ID:          msg.ID,
+	})
+
+	return dispatcher.EndGroups
+}
+
+func handleSilentSaveText(ctx *ext.Context, u *ext.Update) error {
+	logger := log.FromContext(ctx)
+	stor := storage.FromContext(ctx)
+	if stor == nil {
+		logger.Warn("Context storage is nil")
+		ctx.Reply(u, ext.ReplyTextString("未找到存储"), nil)
+		return dispatcher.EndGroups
+	}
+	text := u.EffectiveMessage.Text
+	if text == "" {
+		return dispatcher.EndGroups
+	}
+	item, err := parsers.ParseWithContext(ctx, text)
+	if errors.Is(err, parsers.ErrNoParserFound) {
+		return dispatcher.EndGroups
+	}
+	if err != nil {
+		logger.Error("Failed to parse text", "error", err)
+		ctx.Reply(u, ext.ReplyTextString("Failed to parse text: "+err.Error()), nil)
+		return dispatcher.EndGroups
+	}
+	logger.Debug("Parsed item from text message", "title", item.Title, "url", item.URL)
+	userID := u.GetUserChat().GetID()
+	text, entities, err := msgelem.BuildParsedTextEntity(*item)
+	if err != nil {
+		logger.Errorf("Failed to build parsed text entity: %s", err)
+		ctx.Reply(u, ext.ReplyTextString("Failed to build parsed text entity: "+err.Error()), nil)
+		return dispatcher.EndGroups
+	}
+	msg, err := ctx.SendMessage(userID, &tg.MessagesSendMessageRequest{
+		Message:  text,
+		Entities: entities,
+		ReplyTo: &tg.InputReplyToMessage{
+			ReplyToMsgID:  u.EffectiveMessage.ID,
+			ReplyToPeerID: u.GetUserChat().AsInputPeer(),
+		},
+	})
+	if err != nil {
+		logger.Errorf("Failed to send message: %s", err)
+		return dispatcher.EndGroups
+	}
+	dirPath := ""
+	if len(item.Resources) > 1 {
+		dirPath = fsutil.NormalizePathname(item.Title)
+	}
+	return shortcut.CreateAndAddParsedTaskWithEdit(ctx, stor, dirPath, item, msg.ID, userID)
+}

client/bot/handlers/register.go

@@ -1,12 +1,26 @@
 package handlers
 
 import (
+	"path"
+	"regexp"
+	"strings"
+
 	"github.com/celestix/gotgproto/dispatcher"
 	"github.com/celestix/gotgproto/dispatcher/handlers"
 	"github.com/celestix/gotgproto/dispatcher/handlers/filters"
 	"github.com/celestix/gotgproto/ext"
+	"github.com/charmbracelet/log"
 	"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/re"
+	"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/ruleutil"
+	userclient "github.com/krau/SaveAny-Bot/client/user"
+	"github.com/krau/SaveAny-Bot/common/utils/tgutil"
+	"github.com/krau/SaveAny-Bot/config"
+	"github.com/krau/SaveAny-Bot/core"
+	"github.com/krau/SaveAny-Bot/core/tasks/tfile"
+	"github.com/krau/SaveAny-Bot/database"
 	"github.com/krau/SaveAny-Bot/pkg/tcbdata"
+	"github.com/krau/SaveAny-Bot/storage"
+	"github.com/rs/xid"
 )
 
 func Register(disp dispatcher.Dispatcher) {
@@ -23,10 +37,16 @@ func Register(disp dispatcher.Dispatcher) {
 	disp.AddHandler(handlers.NewCommand("storage", handleStorageCmd))
 	disp.AddHandler(handlers.NewCommand("dir", handleDirCmd))
 	disp.AddHandler(handlers.NewCommand("rule", handleRuleCmd))
+	disp.AddHandler(handlers.NewCommand("watch", handleWatchCmd))
+	disp.AddHandler(handlers.NewCommand("unwatch", handleUnwatchCmd))
 	disp.AddHandler(handlers.NewCommand("save", handleSilentMode(handleSaveCmd, handleSilentSaveReplied)))
+	disp.AddHandler(handlers.NewCommand("config", handleConfigCmd))
+	disp.AddHandler(handlers.NewCommand("update", handleUpdateCmd))
+	disp.AddHandler(handlers.NewCallbackQuery(filters.CallbackQuery.Prefix("update"), handleUpdateCallback))
 	disp.AddHandler(handlers.NewCallbackQuery(filters.CallbackQuery.Prefix(tcbdata.TypeAdd), handleAddCallback))
 	disp.AddHandler(handlers.NewCallbackQuery(filters.CallbackQuery.Prefix(tcbdata.TypeSetDefault), handleSetDefaultCallback))
-	disp.AddHandler(handlers.NewCallbackQuery(filters.CallbackQuery.Prefix("cancel"), handleCancelCallback))
+	disp.AddHandler(handlers.NewCallbackQuery(filters.CallbackQuery.Prefix(tcbdata.TypeCancel), handleCancelCallback))
+	disp.AddHandler(handlers.NewCallbackQuery(filters.CallbackQuery.Prefix(tcbdata.TypeConfig), handleConfigCallback))
 	linkRegexFilter, err := filters.Message.Regex(re.TgMessageLinkRegexString)
 	if err != nil {
 		panic("failed to create regex filter: " + err.Error())
@@ -38,4 +58,89 @@ func Register(disp dispatcher.Dispatcher) {
 	}
 	disp.AddHandler(handlers.NewMessage(telegraphUrlRegexFilter, handleSilentMode(handleTelegraphUrlMessage, handleSilentSaveTelegraph)))
 	disp.AddHandler(handlers.NewMessage(filters.Message.Media, handleSilentMode(handleMediaMessage, handleSilentSaveMedia)))
+	disp.AddHandler(handlers.NewMessage(filters.Message.Text, handleSilentMode(handleTextMessage, handleSilentSaveText)))
+
+	if config.C().Telegram.Userbot.Enable {
+		go listenMediaMessageEvent(userclient.GetMediaMessageCh())
+	}
+}
+
+func listenMediaMessageEvent(ch chan userclient.MediaMessageEvent) {
+	logger := log.FromContext(userclient.GetCtx())
+	for event := range ch {
+		logger.Debug("Received media message event", "chat_id", event.ChatID, "file_name", event.File.Name())
+		ctx := event.Ctx
+		file := event.File
+		chats, err := database.GetWatchChatsByChatID(ctx, event.ChatID)
+		if err != nil {
+			logger.Errorf("Failed to get watch chats for chat ID %d: %v", event.ChatID, err)
+			continue
+		}
+		msgText := event.File.Message().GetMessage()
+		for _, chat := range chats {
+			if chat.Filter != "" {
+				filter := strings.Split(chat.Filter, ":")
+				if len(filter) != 2 {
+					logger.Warnf("Invalid filter format in chat %d, skipping", chat.ChatID)
+					continue
+				}
+				filterType := filter[0]
+				filterData := filter[1]
+				switch filterType {
+				case "msgre": // [TODO] enums for filter types
+					if ok, err := regexp.MatchString(filterData, msgText); err != nil {
+						continue
+					} else if !ok {
+						continue
+					}
+				default:
+					logger.Warnf("Unsupported filter type %s in chat %d, skipping", filterType, chat.ChatID)
+					continue
+				}
+			}
+			user, err := database.GetUserByID(ctx, chat.UserID)
+			if err != nil {
+				logger.Errorf("Failed to get user by ID %d: %v", chat.UserID, err)
+				continue
+			}
+			if user.DefaultStorage == "" {
+				logger.Warnf("User %d has no default storage set, skipping media message handling", chat.UserID)
+				continue
+			}
+			stor, err := storage.GetStorageByUserIDAndName(ctx, user.ChatID, user.DefaultStorage)
+			if err != nil {
+				logger.Errorf("Failed to get storage by user ID %d and name %s: %v", user.ChatID, user.DefaultStorage, err)
+				continue
+			}
+			var dirPath string
+			if user.ApplyRule && user.Rules != nil {
+				matched, matchedStorageName, matchedDirPath := ruleutil.ApplyRule(ctx, user.Rules, ruleutil.NewInput(file))
+				if !matched {
+					goto startCreateTask
+				}
+				dirPath = matchedDirPath.String()
+				if matchedStorageName.IsUsable() {
+					stor, err = storage.GetStorageByUserIDAndName(ctx, user.ChatID, matchedStorageName.String())
+					if err != nil {
+						logger.Errorf("Failed to get storage by user ID and name: %s", err)
+						continue
+					}
+				}
+			}
+		startCreateTask:
+			storagePath := stor.JoinStoragePath(path.Join(dirPath, file.Name()))
+			injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
+			taskid := xid.New().String()
+			task, err := tfile.NewTGFileTask(taskid, injectCtx, file, stor, storagePath, nil)
+			if err != nil {
+				logger.Errorf("create task failed: %s", err)
+				continue
+			}
+			if err := core.AddTask(injectCtx, task); err != nil {
+				logger.Errorf("add task failed: %s", err)
+				continue
+			}
+			logger.Infof("Added media message task for user %d in chat %d: %s", chat.UserID, event.ChatID, file.Name())
+		}
+	}
 }

client/bot/handlers/rule.go

@@ -11,7 +11,7 @@ import (
 	"github.com/duke-git/lancet/v2/slice"
 	"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/msgelem"
 	"github.com/krau/SaveAny-Bot/database"
-	"github.com/krau/SaveAny-Bot/pkg/enums/rule"
+	"github.com/krau/SaveAny-Bot/pkg/rule"
 )
 
 func handleRuleCmd(ctx *ext.Context, update *ext.Update) error {

client/bot/handlers/save.go

@@ -7,7 +7,6 @@ import (
 
 	"github.com/celestix/gotgproto/dispatcher"
 	"github.com/celestix/gotgproto/ext"
-	"github.com/celestix/gotgproto/functions"
 	"github.com/charmbracelet/log"
 	"github.com/gotd/td/tg"
 	"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/mediautil"
@@ -158,9 +157,8 @@ func handleBatchSave(ctx *ext.Context, update *ext.Update, args []string) error
 			sb.Reset()
 			sb.WriteString(msg.GetMessage())
 			sb.WriteString(" ")
-			fn, _ := functions.GetMediaFileNameWithId(media)
+			fn, _ := tgutil.GetMediaFileName(media)
 			sb.WriteString(fn)
-			log.FromContext(ctx).Debugf("正在检查消息内容: %s", sb.String())
 			if !filter.MatchString(sb.String()) {
 				continue
 			}
@@ -194,5 +192,4 @@ func handleBatchSave(ctx *ext.Context, update *ext.Update, args []string) error
 		return dispatcher.EndGroups
 	}
 	return shortcut.CreateAndAddBatchTGFileTaskWithEdit(ctx, update.GetUserChat().GetID(), stor, "", files, replied.ID)
-
 }

client/bot/handlers/telegraph.go

@@ -71,6 +71,6 @@ func handleSilentSaveTelegraph(ctx *ext.Context, update *ext.Update) error {
 		return err
 	}
 	userID := update.GetUserChat().GetID()
-	return shortcut.CreateAndAddTphTaskWithEdit(ctx, userID, result.Page, result.TphDir, result.Pics, stor, msg.ID)
+	return shortcut.CreateAndAddtelegraphWithEdit(ctx, userID, result.Page, result.TphDir, result.Pics, stor, msg.ID)
 
 }

client/bot/handlers/update.go

@@ -0,0 +1,102 @@
+package handlers
+
+import (
+	"errors"
+	"fmt"
+	"regexp"
+	"strings"
+
+	"github.com/blang/semver"
+	"github.com/celestix/gotgproto/dispatcher"
+	"github.com/celestix/gotgproto/ext"
+	"github.com/gotd/td/telegram/message/html"
+	"github.com/gotd/td/tg"
+	"github.com/krau/SaveAny-Bot/config"
+	"github.com/rhysd/go-github-selfupdate/selfupdate"
+)
+
+func handleUpdateCmd(ctx *ext.Context, u *ext.Update) error {
+	currentV, err := semver.Parse(config.Version)
+	if err != nil {
+		ctx.Reply(u, ext.ReplyTextString(fmt.Sprintf("You are in dev or the version var failed to inject: %v", err)), nil)
+		return dispatcher.EndGroups
+	}
+	latest, ok, err := selfupdate.DetectLatest(config.GitRepo)
+	if err != nil {
+		ctx.Reply(u, ext.ReplyTextString(fmt.Sprintf("检测最新版本失败: %v", err)), nil)
+		return dispatcher.EndGroups
+	}
+	if !ok {
+		ctx.Reply(u, ext.ReplyTextString("没有找到版本信息"), nil)
+		return dispatcher.EndGroups
+	}
+	if latest.Version.LT(currentV) || latest.Version.Equals(currentV) {
+		ctx.Reply(u, ext.ReplyTextString(fmt.Sprintf("当前已经是最新版本: %s", config.Version)), nil)
+		return dispatcher.EndGroups
+	}
+	ctx.Sender.To(u.GetUserChat().AsInputPeer()).StyledText(ctx, html.String(nil, func() string {
+		md := latest.ReleaseNotes
+		md = regexp.MustCompile(`(?m)^###\s+&nbsp;&nbsp;&nbsp;(.+)$`).ReplaceAllString(md, "<b>$1</b>")
+		md = regexp.MustCompile(`(?m)^#####\s+&nbsp;&nbsp;&nbsp;&nbsp;(.+)$`).ReplaceAllString(md, "<i>$1</i>")
+
+		md = regexp.MustCompile(`(?m)^- `).ReplaceAllString(md, "• ")
+
+		md = regexp.MustCompile(`\[\((\w{6,})\)\]\((https?://[^\s)]+)\)`).ReplaceAllString(md, `(<a href="$2">$1</a>)`)
+
+		md = regexp.MustCompile(`\[(.+?)\]\((https?://[^\s)]+)\)`).ReplaceAllString(md, `<a href="$2">$1</a>`)
+
+		md = strings.ReplaceAll(md, "&nbsp;", " ")
+
+		return `<blockquote expandable>` + md + `</blockquote>`
+	}()))
+	text := fmt.Sprintf(`发现新版本: %s
+当前版本: %s
+
+文件大小: %.2f MB
+下载链接: %s
+发布时间: %s
+
+升级将重启 Bot , 是否升级?`, latest.Version, config.Version,
+		float64(latest.AssetByteSize)/(1024*1024), latest.AssetURL,
+		latest.PublishedAt.Format("2006-01-02 15:04:05"),
+	)
+	ctx.Reply(u, ext.ReplyTextString(text), &ext.ReplyOpts{
+		Markup: &tg.ReplyInlineMarkup{
+			Rows: []tg.KeyboardButtonRow{
+				{
+					Buttons: []tg.KeyboardButtonClass{
+						&tg.KeyboardButtonCallback{
+							Text: "升级",
+							Data: []byte("update"),
+						},
+					},
+				},
+			},
+		},
+	})
+	return dispatcher.EndGroups
+}
+
+func handleUpdateCallback(ctx *ext.Context, u *ext.Update) error {
+	currentV, err := semver.Parse(config.Version)
+	if err != nil {
+		return err
+	}
+	ctx.EditMessage(u.GetUserChat().GetID(), &tg.MessagesEditMessageRequest{
+		ID:      u.CallbackQuery.GetMsgID(),
+		Message: fmt.Sprintf("正在升级中, 当前版本: %s", config.Version),
+	})
+	latest, err := selfupdate.UpdateSelf(currentV, config.GitRepo)
+	if err != nil {
+		ctx.EditMessage(u.GetUserChat().GetID(), &tg.MessagesEditMessageRequest{
+			ID:      u.CallbackQuery.GetMsgID(),
+			Message: fmt.Sprintf("升级失败: %v", err),
+		})
+		return dispatcher.EndGroups
+	}
+	ctx.EditMessage(u.GetUserChat().GetID(), &tg.MessagesEditMessageRequest{
+		ID:      u.CallbackQuery.GetMsgID(),
+		Message: fmt.Sprintf("已升级至版本 %s\n若 Bot 未自动重启请手动启动", latest.Version),
+	})
+	return errors.New("SAVEANTBOT-RESTART")
+}

client/bot/handlers/utils/msgelem/parse.go

@@ -0,0 +1,39 @@
+package msgelem
+
+import (
+	"fmt"
+
+	"github.com/duke-git/lancet/v2/strutil"
+	"github.com/gotd/td/telegram/message/entity"
+	"github.com/gotd/td/telegram/message/styling"
+	"github.com/gotd/td/tg"
+	"github.com/krau/SaveAny-Bot/pkg/parser"
+)
+
+func BuildParsedTextEntity(item parser.Item) (string, []tg.MessageEntityClass, error) {
+	eb := entity.Builder{}
+	if err := styling.Perform(&eb,
+		styling.Bold(fmt.Sprintf("[%s]%s", item.Site, item.Title)),
+		styling.Plain("\n链接: "),
+		styling.Code(item.URL),
+		styling.Plain("\n作者: "),
+		styling.Code(item.Author),
+		styling.Plain("\n描述: "),
+		styling.Code(strutil.Ellipsis(item.Description, 233)),
+		styling.Plain("\n文件数量: "),
+		styling.Code(fmt.Sprintf("%d", len(item.Resources))),
+		styling.Plain("\n预计总大小: "),
+		styling.Code(fmt.Sprintf("%.2f MB", func() float64 {
+			var totalSize int64
+			for _, res := range item.Resources {
+				totalSize += res.Size
+			}
+			return float64(totalSize) / 1024 / 1024
+		}())),
+		styling.Plain("\n请选择存储位置"),
+	); err != nil {
+		return "", nil, fmt.Errorf("构建消息失败: %w", err)
+	}
+	text, entities := eb.Complete()
+	return text, entities, nil
+}

client/bot/handlers/utils/msgelem/save.go

@@ -10,6 +10,6 @@ const (
 
 	2. 设置默认存储后, 发送 /save <频道ID/用户名> <消息ID范围> 来批量保存文件. 遵从存储规则, 若未匹配到任何规则则使用默认存储.
 	示例:
-	/save @moreacg 114-514
+	/save @acherkrau 114-514
 	`
 )

client/bot/handlers/utils/msgelem/storage.go

@@ -24,6 +24,8 @@ func BuildAddSelectStorageKeyboard(stors []storage.Storage, adddata tcbdata.Add)
 			taskType = tasktype.TaskTypeTgfiles
 		} else if adddata.TphPageNode != nil {
 			taskType = tasktype.TaskTypeTphpics
+		} else if adddata.ParsedItem != nil {
+			taskType = tasktype.TaskTypeParseditem
 		} else {
 			return nil, fmt.Errorf("unknown task type: %s", taskType)
 		}
@@ -41,6 +43,8 @@ func BuildAddSelectStorageKeyboard(stors []storage.Storage, adddata tcbdata.Add)
 			TphPageNode: adddata.TphPageNode,
 			TphPics:     adddata.TphPics,
 			TphDirPath:  adddata.TphDirPath,
+
+			ParsedItem: adddata.ParsedItem,
 		}
 		dataid := xid.New().String()
 		err := cache.Set(dataid, data)

client/bot/handlers/utils/msgelem/watch.go

@@ -0,0 +1,19 @@
+package msgelem
+
+const (
+	WatchHelpText = `
+使用 /watch 命令监听一个聊天的消息, 并自动保存到默认存储中, 遵从存储规则.
+
+命令语法:
+/watch <chat_id> [filter]
+
+参数:
+- <chat_id>: 聊天的 ID 或用户名
+- [filter]: 可选, 格式为 过滤器类型:表达式 , 所有支持类型的过滤器请查看文档
+
+命令示例:
+/watch 2229835658 msgre:.*plana.*
+
+这将监听 ID 为 2229835658 的聊天, 并转存所有包含 "plana" 的媒体消息
+	`
+)

client/bot/handlers/utils/ruleutil/rule.go

@@ -7,8 +7,6 @@ import (
 
 	"github.com/charmbracelet/log"
 	"github.com/krau/SaveAny-Bot/database"
-	"github.com/krau/SaveAny-Bot/pkg/consts"
-	ruleenum "github.com/krau/SaveAny-Bot/pkg/enums/rule"
 	"github.com/krau/SaveAny-Bot/pkg/rule"
 	"github.com/krau/SaveAny-Bot/pkg/tfile"
 )
@@ -37,7 +35,7 @@ func (m matchedStorName) String() string {
 
 // can we use this storage name directly?
 func (m matchedStorName) IsUsable() bool {
-	return m != "" && m != consts.RuleStorNameChosen
+	return m != "" && m != rule.RuleStorNameChosen
 }
 
 type MatchedDirPath string
@@ -47,17 +45,17 @@ func (m MatchedDirPath) String() string {
 }
 
 func (m MatchedDirPath) NeedNewForAlbum() bool {
-	return m != "" && m == consts.RuleDirPathNewForAlbum
+	return m != "" && m == rule.RuleDirPathNewForAlbum
 }
 
-func ApplyRule(ctx context.Context, rules []database.Rule, inputs *ruleInput) (matchedStorageName matchedStorName, dirPath MatchedDirPath) {
+func ApplyRule(ctx context.Context, rules []database.Rule, inputs *ruleInput) (matched bool, matchedStorageName matchedStorName, dirPath MatchedDirPath) {
 	if inputs == nil || len(rules) == 0 {
-		return "", ""
+		return false, "", ""
 	}
 	logger := log.FromContext(ctx)
 	for _, ur := range rules {
 		switch ur.Type {
-		case ruleenum.FileNameRegex.String():
+		case rule.FileNameRegex.String():
 			ru, err := rule.NewRuleFileNameRegex(ur.StorageName, ur.DirPath, ur.Data)
 			if err != nil {
 				logger.Errorf("Failed to create rule: %s", err)
@@ -72,7 +70,7 @@ func ApplyRule(ctx context.Context, rules []database.Rule, inputs *ruleInput) (m
 				dirPath = MatchedDirPath(ru.StoragePath())
 				matchedStorageName = matchedStorName(ru.StorageName())
 			}
-		case ruleenum.MessageRegex.String():
+		case rule.MessageRegex.String():
 			ru, err := rule.NewRuleMessageRegex(ur.StorageName, ur.DirPath, ur.Data)
 			if err != nil {
 				logger.Errorf("Failed to create rule: %s", err)
@@ -87,7 +85,7 @@ func ApplyRule(ctx context.Context, rules []database.Rule, inputs *ruleInput) (m
 				dirPath = MatchedDirPath(ru.StoragePath())
 				matchedStorageName = matchedStorName(ru.StorageName())
 			}
-		case ruleenum.IsAlbum.String():
+		case rule.IsAlbum.String():
 			matchAlbum, err := convertor.ToBool(ur.Data)
 			if err != nil {
 				matchAlbum = false
@@ -108,5 +106,8 @@ func ApplyRule(ctx context.Context, rules []database.Rule, inputs *ruleInput) (m
 			}
 		}
 	}
-	return
+	if matchedStorageName != "" || dirPath != "" {
+		return true, matchedStorageName, dirPath
+	}
+	return false, "", ""
 }

client/bot/handlers/utils/shortcut/message.go

@@ -10,6 +10,7 @@ import (
 	"github.com/celestix/gotgproto/ext"
 	"github.com/celestix/gotgproto/types"
 	"github.com/charmbracelet/log"
+	"github.com/gotd/td/telegram/downloader"
 	"github.com/gotd/td/tg"
 	"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/mediautil"
 	"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/msgelem"
@@ -19,20 +20,21 @@ import (
 	"github.com/krau/SaveAny-Bot/common/utils/tgutil"
 	"github.com/krau/SaveAny-Bot/common/utils/tphutil"
 	"github.com/krau/SaveAny-Bot/config"
+	"github.com/krau/SaveAny-Bot/database"
+	"github.com/krau/SaveAny-Bot/pkg/enums/fnamest"
 	"github.com/krau/SaveAny-Bot/pkg/telegraph"
 	"github.com/krau/SaveAny-Bot/pkg/tfile"
 )
 
 // 获取消息中的文件并回复等待消息, 返回等待消息, 获取到的文件
-func GetFileFromMessageWithReply(ctx *ext.Context, update *ext.Update, message *tg.Message, tfileopts ...tfile.TGFileOptions) (replied *types.Message,
+func GetFileFromMessageWithReply(ctx *ext.Context, update *ext.Update, message *tg.Message, tfileopts ...tfile.TGFileOption) (replied *types.Message,
 	file tfile.TGFileMessage, err error,
 ) {
 	logger := log.FromContext(ctx)
 	media := message.Media
 	supported := mediautil.IsSupported(media)
 	if !supported {
-		ctx.Reply(update, ext.ReplyTextString("不支持的消息类型"), nil)
-		return nil, nil, dispatcher.EndGroups
+		return nil, nil, dispatcher.ContinueGroups
 	}
 
 	replied, err = ctx.Reply(update, ext.ReplyTextString("正在获取文件信息..."), nil)
@@ -40,7 +42,7 @@ func GetFileFromMessageWithReply(ctx *ext.Context, update *ext.Update, message *
 		logger.Errorf("Failed to reply: %s", err)
 		return nil, nil, dispatcher.EndGroups
 	}
-	options := []tfile.TGFileOptions{
+	options := []tfile.TGFileOption{
 		tfile.WithMessage(message),
 	}
 	if len(tfileopts) > 0 {
@@ -81,9 +83,14 @@ func GetFilesFromUpdateLinkMessageWithReplyEdit(ctx *ext.Context, update *ext.Up
 			logger.Errorf("failed to edit message: %s", err)
 		}
 	}
-
+	user, err := database.GetUserByChatID(ctx, update.GetUserChat().GetID())
+	if err != nil {
+		logger.Errorf("failed to get user from db: %s", err)
+		editReplied("获取用户信息失败: "+err.Error(), nil)
+		return nil, nil, nil, dispatcher.EndGroups
+	}
 	files = make([]tfile.TGFileMessage, 0, len(msgLinks))
-	addFile := func(client tfile.DlerClient, msg *tg.Message) {
+	addFile := func(client downloader.Client, msg *tg.Message) {
 		if msg == nil || msg.Media == nil {
 			logger.Warn("message is nil, skipping")
 			return
@@ -93,7 +100,14 @@ func GetFilesFromUpdateLinkMessageWithReplyEdit(ctx *ext.Context, update *ext.Up
 			logger.Debugf("message %d has no media", msg.GetID())
 			return
 		}
-		file, err := tfile.FromMediaMessage(media, client, msg, tfile.WithNameIfEmpty(tgutil.GenFileNameFromMessage(*msg)))
+		var opt tfile.TGFileOption
+		switch user.FilenameStrategy {
+		case fnamest.Message.String():
+			opt = tfile.WithName(tgutil.GenFileNameFromMessage(*msg))
+		default:
+			opt = tfile.WithNameIfEmpty(tgutil.GenFileNameFromMessage(*msg))
+		}
+		file, err := tfile.FromMediaMessage(media, client, msg, opt)
 		if err != nil {
 			logger.Errorf("failed to create file from media: %s", err)
 			return
@@ -102,7 +116,7 @@ func GetFilesFromUpdateLinkMessageWithReplyEdit(ctx *ext.Context, update *ext.Up
 	}
 
 	tctx := ctx
-	if config.Cfg.Telegram.Userbot.Enable {
+	if config.C().Telegram.Userbot.Enable {
 		tctx = uc.GetCtx()
 	}
 

client/bot/handlers/utils/shortcut/parsed.go

@@ -0,0 +1,35 @@
+package shortcut
+
+import (
+	"github.com/celestix/gotgproto/dispatcher"
+	"github.com/celestix/gotgproto/ext"
+	"github.com/charmbracelet/log"
+	"github.com/gotd/td/tg"
+	"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/msgelem"
+	"github.com/krau/SaveAny-Bot/common/utils/tgutil"
+	"github.com/krau/SaveAny-Bot/core"
+	"github.com/krau/SaveAny-Bot/core/tasks/parsed"
+	"github.com/krau/SaveAny-Bot/pkg/parser"
+	"github.com/krau/SaveAny-Bot/storage"
+	"github.com/rs/xid"
+)
+
+func CreateAndAddParsedTaskWithEdit(ctx *ext.Context, stor storage.Storage, dirPath string, item *parser.Item, msgID int, userID int64) error {
+	injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
+	task := parsed.NewTask(xid.New().String(), injectCtx, stor, stor.JoinStoragePath(dirPath), item, parsed.NewProgress(msgID, userID))
+	if err := core.AddTask(injectCtx, task); err != nil {
+		log.FromContext(ctx).Errorf("Failed to add task: %s", err)
+		ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
+			ID:      msgID,
+			Message: "任务添加失败: " + err.Error(),
+		})
+		return dispatcher.EndGroups
+	}
+	text, entities := msgelem.BuildTaskAddedEntities(ctx, item.Title, core.GetLength(ctx))
+	ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
+		ID:       msgID,
+		Message:  text,
+		Entities: entities,
+	})
+	return dispatcher.EndGroups
+}

client/bot/handlers/utils/shortcut/tftask.go

@@ -13,15 +13,15 @@ import (
 	"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/ruleutil"
 	"github.com/krau/SaveAny-Bot/common/utils/tgutil"
 	"github.com/krau/SaveAny-Bot/core"
-	"github.com/krau/SaveAny-Bot/core/batchtftask"
-	"github.com/krau/SaveAny-Bot/core/tftask"
+	"github.com/krau/SaveAny-Bot/core/tasks/batchtfile"
+	tftask "github.com/krau/SaveAny-Bot/core/tasks/tfile"
 	"github.com/krau/SaveAny-Bot/database"
 	"github.com/krau/SaveAny-Bot/pkg/tfile"
 	"github.com/krau/SaveAny-Bot/storage"
 	"github.com/rs/xid"
 )
 
-// 创建一个 tftask.TGFileTask 并添加到任务队列中, 以编辑消息的方式反馈结果
+// 创建一个 tfile.TGFileTask 并添加到任务队列中, 以编辑消息的方式反馈结果
 func CreateAndAddTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor storage.Storage, dirPath string, file tfile.TGFileMessage, trackMsgID int) error {
 	logger := log.FromContext(ctx)
 	user, err := database.GetUserByChatID(ctx, userID)
@@ -34,8 +34,13 @@ func CreateAndAddTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor storage
 		return dispatcher.EndGroups
 	}
 	if user.ApplyRule && user.Rules != nil {
-		matchedStorageName, matchedDirPath := ruleutil.ApplyRule(ctx, user.Rules, ruleutil.NewInput(file))
-		dirPath = matchedDirPath.String()
+		matched, matchedStorageName, matchedDirPath := ruleutil.ApplyRule(ctx, user.Rules, ruleutil.NewInput(file))
+		if !matched {
+			goto startCreateTask
+		}
+		if matchedDirPath != "" {
+			dirPath = matchedDirPath.String()
+		}
 		if matchedStorageName.IsUsable() {
 			stor, err = storage.GetStorageByUserIDAndName(ctx, user.ChatID, matchedStorageName.String())
 			if err != nil {
@@ -48,7 +53,7 @@ func CreateAndAddTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor storage
 			}
 		}
 	}
-
+startCreateTask:
 	storagePath := stor.JoinStoragePath(path.Join(dirPath, file.Name()))
 	injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
 	taskid := xid.New().String()
@@ -82,7 +87,7 @@ func CreateAndAddTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor storage
 	return dispatcher.EndGroups
 }
 
-// 创建一个 batchtftask.BatchTGFileTask 并添加到任务队列中, 以编辑消息的方式反馈结果
+// 创建一个 batchtfile.BatchTGFileTask 并添加到任务队列中, 以编辑消息的方式反馈结果
 func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor storage.Storage, dirPath string, files []tfile.TGFileMessage, trackMsgID int) error {
 	logger := log.FromContext(ctx)
 	user, err := database.GetUserByChatID(ctx, userID)
@@ -101,8 +106,10 @@ func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor st
 		if !useRule {
 			return stor.Name(), ruleutil.MatchedDirPath(dirPath)
 		}
-		storName, dirP := ruleutil.ApplyRule(ctx, user.Rules, ruleutil.NewInput(file))
-
+		matched, storName, dirP := ruleutil.ApplyRule(ctx, user.Rules, ruleutil.NewInput(file))
+		if !matched {
+			return stor.Name(), ruleutil.MatchedDirPath(dirPath)
+		}
 		storname := storName.String()
 		if !storName.IsUsable() {
 			storname = stor.Name()
@@ -110,7 +117,7 @@ func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor st
 		return storname, dirP
 	}
 
-	elems := make([]batchtftask.TaskElement, 0, len(files))
+	elems := make([]batchtfile.TaskElement, 0, len(files))
 	type albumFile struct {
 		file    tfile.TGFileMessage
 		storage storage.Storage
@@ -132,7 +139,7 @@ func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor st
 		}
 		if !dirPath.NeedNewForAlbum() {
 			storPath := fileStor.JoinStoragePath(path.Join(dirPath.String(), file.Name()))
-			elem, err := batchtftask.NewTaskElement(fileStor, storPath, file)
+			elem, err := batchtfile.NewTaskElement(fileStor, storPath, file)
 			if err != nil {
 				logger.Errorf("Failed to create task element: %s", err)
 				ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
@@ -167,7 +174,7 @@ func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor st
 		albumStor := afiles[0].storage
 		for _, af := range afiles {
 			afstorPath := af.storage.JoinStoragePath(path.Join(dirPath, albumDir, af.file.Name()))
-			elem, err := batchtftask.NewTaskElement(albumStor, afstorPath, af.file)
+			elem, err := batchtfile.NewTaskElement(albumStor, afstorPath, af.file)
 			if err != nil {
 				logger.Errorf("Failed to create task element for album file: %s", err)
 				ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{
@@ -182,7 +189,7 @@ func CreateAndAddBatchTGFileTaskWithEdit(ctx *ext.Context, userID int64, stor st
 
 	injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
 	taskid := xid.New().String()
-	task := batchtftask.NewBatchTGFileTask(taskid, injectCtx, elems, batchtftask.NewProgressTracker(trackMsgID, userID), true)
+	task := batchtfile.NewBatchTGFileTask(taskid, injectCtx, elems, batchtfile.NewProgressTracker(trackMsgID, userID), true)
 	if err := core.AddTask(injectCtx, task); err != nil {
 		logger.Errorf("Failed to add batch task: %s", err)
 		ctx.EditMessage(userID, &tg.MessagesEditMessageRequest{

client/bot/handlers/utils/shortcut/tphtask.go

@@ -9,19 +9,21 @@ import (
 	"github.com/krau/SaveAny-Bot/common/utils/tgutil"
 	"github.com/krau/SaveAny-Bot/common/utils/tphutil"
 	"github.com/krau/SaveAny-Bot/core"
-	"github.com/krau/SaveAny-Bot/core/tphtask"
+	tphtask "github.com/krau/SaveAny-Bot/core/tasks/telegraph"
 	"github.com/krau/SaveAny-Bot/pkg/telegraph"
 	"github.com/krau/SaveAny-Bot/storage"
 	"github.com/rs/xid"
 )
 
-func CreateAndAddTphTaskWithEdit(ctx *ext.Context,
+func CreateAndAddtelegraphWithEdit(
+	ctx *ext.Context,
 	userID int64,
 	tphpage *telegraph.Page,
 	dirPath string, // unescaped ph path for file storage
 	pics []string,
 	stor storage.Storage,
 	trackMsgID int) error {
+		
 	injectCtx := tgutil.ExtWithContext(ctx.Context, ctx)
 	task := tphtask.NewTask(xid.New().String(),
 		injectCtx,

client/bot/handlers/watch.go

@@ -0,0 +1,110 @@
+package handlers
+
+import (
+	"regexp"
+	"strings"
+
+	"github.com/celestix/gotgproto/dispatcher"
+	"github.com/celestix/gotgproto/ext"
+	"github.com/charmbracelet/log"
+	"github.com/krau/SaveAny-Bot/client/bot/handlers/utils/msgelem"
+	"github.com/krau/SaveAny-Bot/common/utils/tgutil"
+	"github.com/krau/SaveAny-Bot/database"
+)
+
+func handleWatchCmd(ctx *ext.Context, update *ext.Update) error {
+	logger := log.FromContext(ctx)
+	args := strings.Split(update.EffectiveMessage.Text, " ")
+	if len(args) < 2 {
+		ctx.Reply(update, ext.ReplyTextString(msgelem.WatchHelpText), nil)
+		return dispatcher.EndGroups
+	}
+	userChatID := update.GetUserChat().GetID()
+	user, err := database.GetUserByChatID(ctx, userChatID)
+	if err != nil {
+		logger.Errorf("获取用户失败: %s", err)
+		ctx.Reply(update, ext.ReplyTextString("获取用户失败"), nil)
+		return dispatcher.EndGroups
+	}
+	if user.DefaultStorage == "" {
+		ctx.Reply(update, ext.ReplyTextString("请先设置默认存储, 使用 /storage 命令"), nil)
+		return dispatcher.EndGroups
+	}
+	chatArg := args[1]
+	chatID, err := tgutil.ParseChatID(ctx, chatArg)
+	if err != nil {
+		ctx.Reply(update, ext.ReplyTextString("无效的ID或用户名: "+err.Error()), nil)
+		return dispatcher.EndGroups
+	}
+	watching, err := user.WatchingChat(ctx, chatID)
+	if err != nil {
+		logger.Errorf("Failed to check if user is watching chat %d: %s", chatID, err)
+		return dispatcher.EndGroups
+	}
+	if watching {
+		ctx.Reply(update, ext.ReplyTextString("已经在监听此聊天"), nil)
+		return dispatcher.EndGroups
+	}
+	filter := ""
+	if len(args) > 2 {
+		filterArg := strings.Join(args[2:], " ")
+		filterType := strings.Split(filterArg, ":")[0]
+		filterData := strings.Split(filterArg, ":")[1]
+		if filterType == "" || filterData == "" {
+			ctx.Reply(update, ext.ReplyTextString("过滤器格式错误, 请使用 <过滤器类型>:<表达式>"), nil)
+			return dispatcher.EndGroups
+		}
+		switch filterType {
+		case "msgre":
+			_, err := regexp.Compile(filterData)
+			if err != nil {
+				ctx.Reply(update, ext.ReplyTextString("正则表达式格式错误: "+err.Error()), nil)
+				return dispatcher.EndGroups
+			}
+			filter = filterType + ":" + filterData
+		default:
+			ctx.Reply(update, ext.ReplyTextString("不支持的过滤器类型, 请参阅文档"), nil)
+			return dispatcher.EndGroups
+		}
+	}
+	if err := user.WatchChat(ctx, database.WatchChat{
+		UserID: user.ID,
+		ChatID: chatID,
+		Filter: filter,
+	}); err != nil {
+		logger.Errorf("Failed to watch chat %d: %s", chatID, err)
+		ctx.Reply(update, ext.ReplyTextString("监听聊天失败: "+err.Error()), nil)
+		return dispatcher.EndGroups
+	}
+	ctx.Reply(update, ext.ReplyTextString("已开始监听聊天: "+chatArg), nil)
+	return dispatcher.EndGroups
+}
+
+func handleUnwatchCmd(ctx *ext.Context, update *ext.Update) error {
+	logger := log.FromContext(ctx)
+	args := strings.Split(update.EffectiveMessage.Text, " ")
+	if len(args) < 2 {
+		ctx.Reply(update, ext.ReplyTextString("请提供要取消监听的聊天ID或用户名"), nil)
+		return dispatcher.EndGroups
+	}
+	userChatID := update.GetUserChat().GetID()
+	user, err := database.GetUserByChatID(ctx, userChatID)
+	if err != nil {
+		logger.Errorf("获取用户失败: %s", err)
+		ctx.Reply(update, ext.ReplyTextString("获取用户失败"), nil)
+		return dispatcher.EndGroups
+	}
+	chatArg := args[1]
+	chatID, err := tgutil.ParseChatID(ctx, chatArg)
+	if err != nil {
+		ctx.Reply(update, ext.ReplyTextString("无效的ID或用户名: "+err.Error()), nil)
+		return dispatcher.EndGroups
+	}
+	if err := user.UnwatchChat(ctx, chatID); err != nil {
+		logger.Errorf("Failed to unwatch chat %d: %s", chatID, err)
+		ctx.Reply(update, ext.ReplyTextString("取消监听聊天失败: "+err.Error()), nil)
+		return dispatcher.EndGroups
+	}
+	ctx.Reply(update, ext.ReplyTextString("已取消监听聊天: "+chatArg), nil)
+	return dispatcher.EndGroups
+}

client/middleware/default.go

@@ -16,7 +16,7 @@ import (
 func NewDefaultMiddlewares(ctx context.Context, timeout time.Duration) []telegram.Middleware {
 	return []telegram.Middleware{
 		recovery.New(ctx, newBackoff(timeout)),
-		retry.New(config.Cfg.Telegram.RpcRetry),
+		retry.New(config.C().Telegram.RpcRetry),
 		floodwait.NewSimpleWaiter(),
 	}
 }

client/middleware/recovery/recovery.go

@@ -2,11 +2,11 @@ package recovery
 
 import (
 	"context"
+	"fmt"
 	"time"
 
 	"github.com/cenkalti/backoff/v4"
 	"github.com/charmbracelet/log"
-	"github.com/go-faster/errors"
 	"github.com/gotd/td/bin"
 	"github.com/gotd/td/telegram"
 	"github.com/gotd/td/tg"
@@ -31,7 +31,7 @@ func (r *recovery) Handle(next tg.Invoker) telegram.InvokeFunc {
 		return backoff.RetryNotify(func() error {
 			if err := next.Invoke(ctx, input, output); err != nil {
 				if r.shouldRecover(ctx, err) {
-					return errors.Wrap(err, "recover")
+					return fmt.Errorf("recovery: %w", err)
 				}
 
 				return backoff.Permanent(err)

client/user/userclient.go

@@ -6,13 +6,18 @@ import (
 
 	"github.com/celestix/gotgproto"
 	"github.com/celestix/gotgproto/dispatcher"
+	"github.com/celestix/gotgproto/dispatcher/handlers"
+	"github.com/celestix/gotgproto/dispatcher/handlers/filters"
 	"github.com/celestix/gotgproto/ext"
 	"github.com/celestix/gotgproto/sessionMaker"
+
 	"github.com/charmbracelet/log"
 	"github.com/gotd/td/telegram/dcs"
+	"github.com/gotd/td/tg"
 	"github.com/krau/SaveAny-Bot/client/middleware"
 	"github.com/krau/SaveAny-Bot/common/utils/netutil"
 	"github.com/krau/SaveAny-Bot/config"
+	"github.com/krau/SaveAny-Bot/database"
 	"github.com/ncruces/go-sqlite3/gormlite"
 	"golang.org/x/net/proxy"
 )
@@ -49,8 +54,8 @@ func Login(ctx context.Context) (*gotgproto.Client, error) {
 	})
 	go func() {
 		var resolver dcs.Resolver
-		if config.Cfg.Telegram.Proxy.Enable && config.Cfg.Telegram.Proxy.URL != "" {
-			dialer, err := netutil.NewProxyDialer(config.Cfg.Telegram.Proxy.URL)
+		if config.C().Telegram.Proxy.Enable && config.C().Telegram.Proxy.URL != "" {
+			dialer, err := netutil.NewProxyDialer(config.C().Telegram.Proxy.URL)
 			if err != nil {
 				res <- struct {
 					client *gotgproto.Client
@@ -65,16 +70,16 @@ func Login(ctx context.Context) (*gotgproto.Client, error) {
 			resolver = dcs.DefaultResolver()
 		}
 		tclient, err := gotgproto.NewClient(
-			config.Cfg.Telegram.AppID,
-			config.Cfg.Telegram.AppHash,
+			config.C().Telegram.AppID,
+			config.C().Telegram.AppHash,
 			gotgproto.ClientTypePhone(""),
 			&gotgproto.ClientOpts{
-				Session:          sessionMaker.SqlSession(gormlite.Open(config.Cfg.Telegram.Userbot.Session)),
+				Session:          sessionMaker.SqlSession(gormlite.Open(config.C().Telegram.Userbot.Session)),
 				AuthConversator:  &terminalAuthConversator{},
 				Context:          ctx,
 				DisableCopyright: true,
 				Resolver:         resolver,
-				MaxRetries:       config.Cfg.Telegram.RpcRetry,
+				MaxRetries:       config.C().Telegram.RpcRetry,
 				AutoFetchReply:   true,
 				Middlewares:      middleware.NewDefaultMiddlewares(ctx, 5*time.Minute),
 				ErrorHandler: func(ctx *ext.Context, u *ext.Update, s string) error {
@@ -106,6 +111,19 @@ func Login(ctx context.Context) (*gotgproto.Client, error) {
 			return nil, r.err
 		}
 		uc = r.client
+		uc.Dispatcher.AddHandler(handlers.NewMessage(filters.Message.Media, func(ctx *ext.Context, u *ext.Update) error {
+			switch u.UpdateClass.(type) {
+			case *tg.UpdateEditChannelMessage, *tg.UpdateEditMessage, *tg.UpdateDeleteChannelMessages, *tg.UpdateDeleteMessages:
+				return dispatcher.EndGroups
+			}
+			chatId := u.EffectiveChat().GetID()
+			watchChats, err := database.GetWatchChatsByChatID(ctx, chatId)
+			if err != nil || len(watchChats) == 0 {
+				return dispatcher.EndGroups
+			}
+			return dispatcher.ContinueGroups
+		}))
+		uc.Dispatcher.AddHandler(handlers.NewMessage(filters.Message.Media, handleMediaMessage))
 		log.FromContext(ctx).Infof("User client logged in successfully: %s", uc.Self.FirstName+" "+uc.Self.LastName)
 		return uc, nil
 	}

client/user/watch.go

@@ -0,0 +1,100 @@
+package user
+
+import (
+	"sync"
+	"time"
+
+	"github.com/celestix/gotgproto/dispatcher"
+	"github.com/celestix/gotgproto/ext"
+	"github.com/gotd/td/tg"
+	"github.com/krau/SaveAny-Bot/common/utils/tgutil"
+	"github.com/krau/SaveAny-Bot/pkg/tfile"
+)
+
+type MediaMessageEvent struct {
+	Ctx       *ext.Context
+	ChatID    int64 // from witch the media message was sent
+	MessageID int
+	File      tfile.TGFileMessage
+}
+
+type messageKey struct {
+	ChatID    int64
+	MessageID int
+}
+
+type MediaMessageHandler struct {
+	events   map[messageKey]MediaMessageEvent
+	timers   map[messageKey]*time.Timer
+	mu       sync.Mutex
+	debounce time.Duration
+}
+
+var (
+	mediaMessageCh      = make(chan MediaMessageEvent, 100)
+	mediaMessageHandler = &MediaMessageHandler{
+		events:   make(map[messageKey]MediaMessageEvent),
+		timers:   make(map[messageKey]*time.Timer),
+		debounce: 5 * time.Second,
+	}
+)
+
+func GetMediaMessageCh() chan MediaMessageEvent {
+	return mediaMessageCh
+}
+
+func sendMediaMessageEvent(event MediaMessageEvent) {
+	key := messageKey{ChatID: event.ChatID, MessageID: event.MessageID}
+
+	mediaMessageHandler.mu.Lock()
+	defer mediaMessageHandler.mu.Unlock()
+
+	if timer, exists := mediaMessageHandler.timers[key]; exists {
+		timer.Stop()
+	} else {
+		mediaMessageHandler.events[key] = event
+	}
+
+	mediaMessageHandler.timers[key] = time.AfterFunc(mediaMessageHandler.debounce, func() {
+		mediaMessageHandler.mu.Lock()
+		event := mediaMessageHandler.events[key]
+		delete(mediaMessageHandler.events, key)
+		delete(mediaMessageHandler.timers, key)
+		mediaMessageHandler.mu.Unlock()
+
+		mediaMessageCh <- event
+	})
+}
+
+func handleMediaMessage(ctx *ext.Context, update *ext.Update) error {
+	message := update.EffectiveMessage
+	media, ok := message.GetMedia()
+	if !ok || media == nil {
+		return dispatcher.EndGroups
+	}
+	support := func() bool {
+		switch media.(type) {
+		case *tg.MessageMediaDocument, *tg.MessageMediaPhoto:
+			return true
+		default:
+			return false
+		}
+	}()
+	if !support {
+		return dispatcher.EndGroups
+	}
+	file, err := tfile.FromMediaMessage(media, ctx.Raw, message.Message, tfile.WithNameIfEmpty(
+		tgutil.GenFileNameFromMessage(*message.Message),
+	))
+	if err != nil {
+		return err
+	}
+	chatId := update.EffectiveChat().GetID()
+	sendMediaMessageEvent(MediaMessageEvent{
+		Ctx:       ctx,
+		ChatID:    chatId,
+		MessageID: message.ID,
+		File:      file,
+	})
+	return dispatcher.EndGroups
+}

cmd/run.go

@@ -19,12 +19,13 @@ import (
 	"github.com/krau/SaveAny-Bot/config"
 	"github.com/krau/SaveAny-Bot/core"
 	"github.com/krau/SaveAny-Bot/database"
+	"github.com/krau/SaveAny-Bot/parsers"
 	"github.com/krau/SaveAny-Bot/storage"
 	"github.com/spf13/cobra"
 )
 
 func Run(cmd *cobra.Command, _ []string) {
-	ctx := cmd.Context()
+	ctx, cancel := context.WithCancel(cmd.Context())
 	logger := log.NewWithOptions(os.Stdout, log.Options{
 		Level:           log.DebugLevel,
 		ReportTimestamp: true,
@@ -33,7 +34,15 @@ func Run(cmd *cobra.Command, _ []string) {
 	})
 	ctx = log.WithContext(ctx, logger)
 
-	initAll(ctx)
+	exitChan, err := initAll(ctx)
+	if err != nil {
+		logger.Fatal("Failed to initialize", "error", err)
+	}
+	go func() {
+		<-exitChan
+		cancel()
+	}()
+
 	core.Run(ctx)
 
 	<-ctx.Done()
@@ -42,34 +51,43 @@ func Run(cmd *cobra.Command, _ []string) {
 	cleanCache()
 }
 
-func initAll(ctx context.Context) {
+func initAll(ctx context.Context) (<-chan struct{}, error) {
 	if err := config.Init(ctx); err != nil {
 		fmt.Println("Failed to load config:", err)
-		os.Exit(1)
+		return nil, err
 	}
 	cache.Init()
 	logger := log.FromContext(ctx)
-	i18n.Init(config.Cfg.Lang)
+	i18n.Init(config.C().Lang)
 	logger.Info(i18n.T(i18nk.Initing))
 	database.Init(ctx)
 	storage.LoadStorages(ctx)
-	if config.Cfg.Telegram.Userbot.Enable {
+	if config.C().Parser.PluginEnable {
+		for _, dir := range config.C().Parser.PluginDirs {
+			if err := parsers.LoadPlugins(ctx, dir); err != nil {
+				logger.Error("Failed to load parser plugins", "dir", dir, "error", err)
+			} else {
+				logger.Debug("Loaded parser plugins", "dir", dir)
+			}
+		}
+	}
+	if config.C().Telegram.Userbot.Enable {
 		_, err := userclient.Login(ctx)
 		if err != nil {
 			logger.Fatalf("User client login failed: %s", err)
 		}
 	}
-	bot.Init(ctx)
+	return bot.Init(ctx), nil
 }
 
 func cleanCache() {
-	if config.Cfg.NoCleanCache {
+	if config.C().NoCleanCache {
 		return
 	}
-	if config.Cfg.Temp.BasePath != "" && !config.Cfg.Stream {
-		if slices.Contains([]string{"/", ".", "\\", ".."}, filepath.Clean(config.Cfg.Temp.BasePath)) {
+	if config.C().Temp.BasePath != "" && !config.C().Stream {
+		if slices.Contains([]string{"/", ".", "\\", ".."}, filepath.Clean(config.C().Temp.BasePath)) {
 			log.Error(i18n.T(i18nk.InvalidCacheDir, map[string]any{
-				"Path": config.Cfg.Temp.BasePath,
+				"Path": config.C().Temp.BasePath,
 			}))
 			return
 		}
@@ -80,7 +98,7 @@ func cleanCache() {
 			}))
 			return
 		}
-		cachePath := filepath.Join(currentDir, config.Cfg.Temp.BasePath)
+		cachePath := filepath.Join(currentDir, config.C().Temp.BasePath)
 		cachePath, err = filepath.Abs(cachePath)
 		if err != nil {
 			log.Error(i18n.T(i18nk.GetCacheAbsPathFailed, map[string]any{

cmd/version.go

@@ -4,7 +4,7 @@ import (
 	"fmt"
 	"runtime"
 
-	"github.com/krau/SaveAny-Bot/pkg/consts"
+	"github.com/krau/SaveAny-Bot/config"
 	"github.com/rhysd/go-github-selfupdate/selfupdate"
 
 	"github.com/blang/semver"
@@ -16,7 +16,7 @@ var VersionCmd = &cobra.Command{
 	Aliases: []string{"v"},
 	Short:   "Print the version number of saveany-bot",
 	Run: func(cmd *cobra.Command, args []string) {
-		fmt.Printf("saveany-bot version: %s %s/%s\nBuildTime: %s, Commit: %s\n", consts.Version, runtime.GOOS, runtime.GOARCH, consts.BuildTime, consts.GitCommit)
+		fmt.Printf("saveany-bot version: %s %s/%s\nBuildTime: %s, Commit: %s\n", config.Version, runtime.GOOS, runtime.GOARCH, config.BuildTime, config.GitCommit)
 	},
 }
 
@@ -25,14 +25,14 @@ var upgradeCmd = &cobra.Command{
 	Aliases: []string{"up"},
 	Short:   "Upgrade saveany-bot to the latest version",
 	Run: func(cmd *cobra.Command, args []string) {
-		v := semver.MustParse(consts.Version)
-		latest, err := selfupdate.UpdateSelf(v, "krau/SaveAny-Bot")
+		v := semver.MustParse(config.Version)
+		latest, err := selfupdate.UpdateSelf(v, config.GitRepo)
 		if err != nil {
 			fmt.Println("Binary update failed:", err)
 			return
 		}
 		if latest.Version.Equals(v) {
-			fmt.Println("Current binary is the latest version", consts.Version)
+			fmt.Println("Current binary is the latest version", config.Version)
 		} else {
 			fmt.Println("Successfully updated to version", latest.Version)
 			fmt.Println("Release note:\n", latest.ReleaseNotes)

common/cache/ristretto.go

@@ -16,8 +16,8 @@ func Init() {
 		panic("cache already initialized")
 	}
 	c, err := ristretto.NewCache(&ristretto.Config[string, any]{
-		NumCounters: config.Cfg.Cache.NumCounters,
-		MaxCost:     config.Cfg.Cache.MaxCost,
+		NumCounters: config.C().Cache.NumCounters,
+		MaxCost:     config.C().Cache.MaxCost,
 		BufferItems: 64,
 		OnReject: func(item *ristretto.Item[any]) {
 			log.Warnf("Cache item rejected: key=%d, value=%v", item.Key, item.Value)
@@ -30,7 +30,7 @@ func Init() {
 }
 
 func Set(key string, value any) error {
-	ok := cache.SetWithTTL(key, value, 0, time.Duration(config.Cfg.Cache.TTL)*time.Second)
+	ok := cache.SetWithTTL(key, value, 0, time.Duration(config.C().Cache.TTL)*time.Second)
 	if !ok {
 		return fmt.Errorf("failed to set value in cache")
 	}

common/utils/fsutil/fs.go

@@ -3,6 +3,8 @@ package fsutil
 import (
 	"os"
 	"path/filepath"
+	"strings"
+	"unicode"
 
 	"github.com/gabriel-vasile/mimetype"
 )
@@ -55,3 +57,21 @@ func CreateFile(fp string) (*File, error) {
 	}
 	return &File{File: file}, nil
 }
+
+func NormalizePathname(s string) string {
+	specials := `\/:*?"<>|` + "\n\r\t"
+	var builder strings.Builder
+	for _, ch := range s {
+		if strings.ContainsRune(specials, ch) || unicode.IsControl(ch) {
+			builder.WriteRune('_')
+		} else {
+			builder.WriteRune(ch)
+		}
+	}
+
+	result := strings.TrimRightFunc(builder.String(), func(r rune) bool {
+		return r == '.' || r == '_' || unicode.IsSpace(r)
+	})
+
+	return result
+}

common/utils/fsutil/normalize_pathname_test.go

@@ -0,0 +1,46 @@
+package fsutil_test
+
+import (
+	"testing"
+
+	"github.com/krau/SaveAny-Bot/common/utils/fsutil"
+)
+
+func TestNormalizePathname(t *testing.T) {
+	tests := []struct {
+		input    string
+		expected string
+	}{
+		{
+			input:    "hello/world?.txt  ",
+			expected: "hello_world_.txt",
+		},
+		{
+			input:    "bad|name:\nfile\r.",
+			expected: "bad_name__file",
+		},
+		{
+			input:    "normal.txt",
+			expected: "normal.txt",
+		},
+		{
+			input:    "test....   ",
+			expected: "test",
+		},
+		{
+			input:    "abc<>def",
+			expected: "abc__def",
+		},
+		{
+			input:    "with\tcontrol",
+			expected: "with_control",
+		},
+	}
+
+	for _, tc := range tests {
+		got := fsutil.NormalizePathname(tc.input)
+		if got != tc.expected {
+			t.Errorf("NormalizePathname(%q) = %q; want %q", tc.input, got, tc.expected)
+		}
+	}
+}

common/utils/netutil/proxy.go

@@ -1,8 +1,15 @@
 package netutil
 
 import (
+	"context"
+	"fmt"
+	"net"
+	"net/http"
 	"net/url"
+	"sync"
 
+	"github.com/charmbracelet/log"
+	"github.com/krau/SaveAny-Bot/config"
 	"golang.org/x/net/proxy"
 )
 
@@ -13,3 +20,60 @@ func NewProxyDialer(proxyUrl string) (proxy.Dialer, error) {
 	}
 	return proxy.FromURL(url, proxy.Direct)
 }
+
+func NewProxyHTTPClient(proxyUrl string) (*http.Client, error) {
+	if proxyUrl == "" {
+		return &http.Client{
+			Transport: &http.Transport{
+				Proxy: http.ProxyFromEnvironment,
+			},
+		}, nil
+	}
+
+	u, err := url.Parse(proxyUrl)
+	if err != nil {
+		return nil, err
+	}
+
+	switch u.Scheme {
+	case "http", "https":
+		return &http.Client{
+			Transport: &http.Transport{
+				Proxy: http.ProxyURL(u),
+			},
+		}, nil
+	case "socks5":
+		dialer, err := proxy.SOCKS5("tcp", u.Host, nil, proxy.Direct)
+		if err != nil {
+			return nil, err
+		}
+
+		return &http.Client{
+			Transport: &http.Transport{
+				DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
+					return dialer.Dial(network, addr)
+				},
+			},
+		}, nil
+	default:
+		return nil, fmt.Errorf("unsupported proxy scheme: %s", u.Scheme)
+	}
+}
+
+var (
+	defaultProxyHttpClient         *http.Client
+	onceLoadDefaultProxyHttpClient sync.Once
+)
+
+func DefaultParserHTTPClient() *http.Client {
+	onceLoadDefaultProxyHttpClient.Do(func() {
+		client, err := NewProxyHTTPClient(config.C().Parser.Proxy)
+		if err != nil {
+			log.Warn("Failed to create default proxy HTTP client, using http.DefaultClient", "error", err)
+			defaultProxyHttpClient = http.DefaultClient
+		} else {
+			defaultProxyHttpClient = client
+		}
+	})
+	return defaultProxyHttpClient
+}

common/utils/tgutil/media.go

@@ -0,0 +1,40 @@
+package tgutil
+
+import (
+	"fmt"
+
+	"github.com/gabriel-vasile/mimetype"
+	"github.com/gotd/td/tg"
+)
+
+func GetMediaFileName(media tg.MessageMediaClass) (string, error) {
+	switch v := media.(type) {
+	case *tg.MessageMediaPhoto:
+		f, ok := v.Photo.AsNotEmpty()
+		if !ok {
+			return "", fmt.Errorf("unknown type media: %T", media)
+		}
+		return fmt.Sprintf("%d.png", f.ID), nil
+	case *tg.MessageMediaDocument:
+		f, ok := v.Document.AsNotEmpty()
+		if !ok {
+			return "", fmt.Errorf("unknown type media: %T", media)
+		}
+		fileName := ""
+		for _, attribute := range f.Attributes {
+			if name, ok := attribute.(*tg.DocumentAttributeFilename); ok {
+				fileName = name.GetFileName()
+				break
+			}
+		}
+		if fileName == "" {
+			mmt := mimetype.Lookup(f.GetMimeType())
+			if mmt != nil {
+				fileName = fmt.Sprintf("%d.%s", f.GetID(), mmt.Extension())
+			}
+		}
+		return fileName, nil
+	default:
+		return "", fmt.Errorf("unsupported type media: %T", media)
+	}
+}

common/utils/tgutil/message.go

@@ -4,9 +4,9 @@ import (
 	"fmt"
 	"strconv"
 	"strings"
+	"unicode"
 
 	"github.com/celestix/gotgproto/ext"
-	"github.com/celestix/gotgproto/functions"
 	"github.com/duke-git/lancet/v2/maputil"
 
 	"github.com/duke-git/lancet/v2/mathutil"
@@ -62,16 +62,12 @@ func GenFileNameFromMessage(message tg.Message) string {
 			return fmt.Sprintf("%s_%s", tagStr, strconv.Itoa(message.GetID()))
 		}
 		text = lcstrutil.Substring(strings.Map(func(r rune) rune {
-			if r < 0x20 || r == 0x7F {
-				return '_'
-			}
 			switch r {
-			// invalid characters
 			case '/', '\\',
 				':', '*', '?', '"', '<', '>', '|':
 				return '_'
-			// empty
-			case ' ', '\t', '\r', '\n':
+			}
+			if unicode.IsControl(r) || unicode.IsSpace(r) {
 				return '_'
 			}
 			if validator.IsPrintable(string(r)) {
@@ -86,7 +82,7 @@ func GenFileNameFromMessage(message tg.Message) string {
 	}()
 
 	if filename == "" {
-		mname, err := functions.GetMediaFileNameWithId(message.Media)
+		mname, err := GetMediaFileName(message.Media)
 		if err != nil {
 			filename = fmt.Sprintf("%d_%s", message.GetID(), xid.New().String())
 		} else {

common/utils/tgutil/resolve.go

@@ -86,7 +86,7 @@ func ParseMessageLink(ctx *ext.Context, link string) (int64, int, error) {
 		return chatID, msgID, nil
 	case 3:
 		// https://t.me/c/123456789/123
-		// https://t.me/acherkrau/123/456 , 456: message thread ID
+		// https://t.me/acherkrau/123/456 , 123: topic id
 		chatPart, msgPart := paths[1], paths[2]
 		if paths[0] != "c" {
 			chatPart = paths[0]

common/utils/tphutil/tph.go

@@ -13,8 +13,8 @@ func DefaultClient() *telegraph.Client {
 	if tphClient != nil {
 		return tphClient
 	}
-	if config.Cfg.Telegram.Proxy.Enable && config.Cfg.Telegram.Proxy.URL != "" {
-		proxyUrl := config.Cfg.Telegram.Proxy.URL
+	if config.C().Telegram.Proxy.Enable && config.C().Telegram.Proxy.URL != "" {
+		proxyUrl := config.C().Telegram.Proxy.URL
 		var err error
 		tphClient, err = telegraph.NewClientWithProxy(proxyUrl)
 		if err != nil {

config/parser.go

@@ -0,0 +1,15 @@
+package config
+
+type parserConfig struct {
+	PluginEnable bool                      `toml:"plugin_enable" mapstructure:"plugin_enable" json:"plugin_enable"`
+	PluginDirs   []string                  `toml:"plugin_dirs" mapstructure:"plugin_dirs" json:"plugin_dirs"`
+	Proxy        string                    `toml:"proxy" mapstructure:"proxy" json:"proxy"`
+	ParserCfgs   map[string]map[string]any `mapstructure:",remain"`
+}
+
+func (c Config) GetParserConfigByName(name string) map[string]any {
+	if c.Parser.ParserCfgs == nil {
+		return nil
+	}
+	return c.Parser.ParserCfgs[name]
+}

config/storage/telegram.go

@@ -9,6 +9,7 @@ import (
 type TelegramStorageConfig struct {
 	BaseConfig
 	ChatID    int64 `toml:"chat_id" mapstructure:"chat_id" json:"chat_id"`
+	ForceFile bool  `toml:"force_file" mapstructure:"force_file" json:"force_file"`
 	RateLimit int   `toml:"rate_limit" mapstructure:"rate_limit" json:"rate_limit"`
 	RateBurst int   `toml:"rate_burst" mapstructure:"rate_burst" json:"rate_burst"`
 }

config/tg.go

@@ -6,7 +6,7 @@ type telegramConfig struct {
 	AppHash  string        `toml:"app_hash" mapstructure:"app_hash" json:"app_hash"`
 	Proxy    tgProxyConfig `toml:"proxy" mapstructure:"proxy"`
 	RpcRetry int           `toml:"rpc_retry" mapstructure:"rpc_retry" json:"rpc_retry"`
-	Userbot  userbotConfig `toml:"userbot" mapstructure:"userbot" json:"userbot"` // [TODO]
+	Userbot  userbotConfig `toml:"userbot" mapstructure:"userbot" json:"userbot"`
 }
 
 type userbotConfig struct {

config/user.go

@@ -14,7 +14,7 @@ var userIDs []int64
 var storages []string
 var userStorages = make(map[int64][]string)
 
-func (c *Config) GetStorageNamesByUserID(userID int64) []string {
+func (c Config) GetStorageNamesByUserID(userID int64) []string {
 	us, ok := userStorages[userID]
 	if ok {
 		return us
@@ -22,11 +22,11 @@ func (c *Config) GetStorageNamesByUserID(userID int64) []string {
 	return nil
 }
 
-func (c *Config) GetUsersID() []int64 {
+func (c Config) GetUsersID() []int64 {
 	return userIDs
 }
 
-func (c *Config) HasStorage(userID int64, storageName string) bool {
+func (c Config) HasStorage(userID int64, storageName string) bool {
 	us, ok := userStorages[userID]
 	if !ok {
 		return false

config/version.go

@@ -0,0 +1,13 @@
+package config
+
+// inject version by '-X' flag
+// go build -ldflags "-X github.com/krau/SaveAny-Bot/config.Version=${{ env.VERSION }}"
+var (
+	Version   string = "dev"
+	BuildTime string = "unknown"
+	GitCommit string = "unknown"
+)
+
+const (
+	GitRepo = "krau/SaveAny-Bot"
+)

config/viper.go

@@ -4,7 +4,6 @@ import (
 	"context"
 	"errors"
 	"fmt"
-	"os"
 	"strings"
 
 	"github.com/duke-git/lancet/v2/slice"
@@ -28,10 +27,15 @@ type Config struct {
 	DB       dbConfig                `toml:"db" mapstructure:"db"`
 	Telegram telegramConfig          `toml:"telegram" mapstructure:"telegram"`
 	Storages []storage.StorageConfig `toml:"-" mapstructure:"-" json:"storages"`
+	Parser   parserConfig            `toml:"parser" mapstructure:"parser" json:"parser"`
 	Hook     hookConfig              `toml:"hook" mapstructure:"hook" json:"hook"`
 }
 
-var Cfg *Config = &Config{}
+var cfg = &Config{}
+
+func C() Config {
+	return *cfg
+}
 
 func (c Config) GetStorageByName(name string) storage.StorageConfig {
 	for _, storage := range c.Storages {
@@ -91,48 +95,48 @@ func Init(ctx context.Context) error {
 
 	if err := viper.ReadInConfig(); err != nil {
 		fmt.Println("Error reading config file, ", err)
-		os.Exit(1)
+		return err
 	}
 
-	if err := viper.Unmarshal(Cfg); err != nil {
+	if err := viper.Unmarshal(cfg); err != nil {
 		fmt.Println("Error unmarshalling config file, ", err)
-		os.Exit(1)
+		return err
 	}
 
 	storagesConfig, err := storage.LoadStorageConfigs(viper.GetViper())
 	if err != nil {
 		return fmt.Errorf("error loading storage configs: %w", err)
 	}
-	Cfg.Storages = storagesConfig
+	cfg.Storages = storagesConfig
 
 	storageNames := make(map[string]struct{})
-	for _, storage := range Cfg.Storages {
+	for _, storage := range cfg.Storages {
 		if _, ok := storageNames[storage.GetName()]; ok {
-			return errors.New(i18n.TWithoutInit(Cfg.Lang, i18nk.ConfigInvalidDuplicateStorageName, map[string]any{
+			return errors.New(i18n.TWithoutInit(cfg.Lang, i18nk.ConfigInvalidDuplicateStorageName, map[string]any{
 				"Name": storage.GetName(),
 			}))
 		}
 		storageNames[storage.GetName()] = struct{}{}
 	}
 
-	fmt.Println(i18n.TWithoutInit(Cfg.Lang, i18nk.LoadedStorages, map[string]any{
-		"Count": len(Cfg.Storages),
+	fmt.Println(i18n.TWithoutInit(cfg.Lang, i18nk.LoadedStorages, map[string]any{
+		"Count": len(cfg.Storages),
 	}))
-	for _, storage := range Cfg.Storages {
+	for _, storage := range cfg.Storages {
 		fmt.Printf("  - %s (%s)\n", storage.GetName(), storage.GetType())
 	}
 
-	if Cfg.Workers < 1 || Cfg.Retry < 1 {
-		return errors.New(i18n.TWithoutInit(Cfg.Lang, i18nk.ConfigInvalidWorkersOrRetry, map[string]any{
-			"Workers": Cfg.Workers,
-			"Retry":   Cfg.Retry,
+	if cfg.Workers < 1 || cfg.Retry < 1 {
+		return errors.New(i18n.TWithoutInit(cfg.Lang, i18nk.ConfigInvalidWorkersOrRetry, map[string]any{
+			"Workers": cfg.Workers,
+			"Retry":   cfg.Retry,
 		}))
 	}
 
-	for _, storage := range Cfg.Storages {
+	for _, storage := range cfg.Storages {
 		storages = append(storages, storage.GetName())
 	}
-	for _, user := range Cfg.Users {
+	for _, user := range cfg.Users {
 		userIDs = append(userIDs, user.ID)
 		if user.Blacklist {
 			userStorages[user.ID] = slice.Compact(slice.Difference(storages, user.Storages))
@@ -142,20 +146,3 @@ func Init(ctx context.Context) error {
 	}
 	return nil
 }
-
-func Set(key string, value any) {
-	viper.Set(key, value)
-}
-
-func ReloadConfig() error {
-	if err := viper.WriteConfig(); err != nil {
-		return err
-	}
-	if err := viper.ReadInConfig(); err != nil {
-		return err
-	}
-	if error := viper.Unmarshal(Cfg); error != nil {
-		return error
-	}
-	return nil
-}

core/core.go

@@ -20,7 +20,7 @@ type Exectable interface {
 
 func worker(ctx context.Context, qe *queue.TaskQueue[Exectable], semaphore chan struct{}) {
 	logger := log.FromContext(ctx)
-	execHooks := config.Cfg.Hook.Exec
+	execHooks := config.C().Hook.Exec
 	for {
 		semaphore <- struct{}{}
 		qtask, err := qe.Get()
@@ -58,11 +58,11 @@ func worker(ctx context.Context, qe *queue.TaskQueue[Exectable], semaphore chan
 
 func Run(ctx context.Context) {
 	log.FromContext(ctx).Info("Start processing tasks...")
-	semaphore := make(chan struct{}, config.Cfg.Workers)
+	semaphore := make(chan struct{}, config.C().Workers)
 	if queueInstance == nil {
 		queueInstance = queue.NewTaskQueue[Exectable]()
 	}
-	for range config.Cfg.Workers {
+	for range config.C().Workers {
 		go worker(ctx, queueInstance, semaphore)
 	}
 

core/tasks/batchtfile/execute.go

@@ -1,4 +1,4 @@
-package batchtftask
+package batchtfile
 
 import (
 	"context"
@@ -21,18 +21,23 @@ func (t *Task) Execute(ctx context.Context) error {
 	logger := log.FromContext(ctx).WithPrefix(fmt.Sprintf("batch_file[%s]", t.ID))
 	logger.Info("Starting batch file task")
 	t.Progress.OnStart(ctx, t)
-	workers := config.Cfg.Workers
+	workers := config.C().Workers
 	eg, gctx := errgroup.WithContext(ctx)
 	eg.SetLimit(workers)
 	for _, elem := range t.Elems {
-		elem := elem
 		eg.Go(func() error {
+			t.processingMu.RLock()
 			if t.processing[elem.ID] != nil {
 				return fmt.Errorf("element with ID %s is already being processed", elem.ID)
 			}
+			t.processingMu.RUnlock()
+			t.processingMu.Lock()
 			t.processing[elem.ID] = &elem
+			t.processingMu.Unlock()
 			defer func() {
+				t.processingMu.Lock()
 				delete(t.processing, elem.ID)
+				t.processingMu.Unlock()
 			}()
 			return t.processElement(gctx, elem)
 		})
@@ -119,6 +124,6 @@ func (t *Task) processElement(ctx context.Context, elem TaskElement) error {
 			return err
 		}
 		return nil
-	}, retry.Context(vctx), retry.RetryTimes(uint(config.Cfg.Retry)))
+	}, retry.Context(vctx), retry.RetryTimes(uint(config.C().Retry)))
 	return err
 }

core/tasks/batchtfile/progress.go

@@ -1,4 +1,4 @@
-package batchtftask
+package batchtfile
 
 import (
 	"context"

core/tasks/batchtfile/task.go

@@ -1,9 +1,10 @@
-package batchtftask
+package batchtfile
 
 import (
 	"context"
 	"fmt"
 	"path/filepath"
+	"sync"
 	"sync/atomic"
 
 	"github.com/krau/SaveAny-Bot/config"
@@ -31,7 +32,8 @@ type Task struct {
 	downloaded   atomic.Int64
 	totalSize    int64
 	processing   map[string]TaskElementInfo
-	failed       map[string]error // errors for each element
+	processingMu sync.RWMutex
+	failed       map[string]error // [TODO] errors for each element
 }
 
 func (t *Task) Type() tasktype.TaskType {
@@ -45,8 +47,8 @@ func NewTaskElement(
 ) (*TaskElement, error) {
 	id := xid.New().String()
 	_, ok := stor.(storage.StorageCannotStream)
-	if !config.Cfg.Stream || ok {
-		cachePath, err := filepath.Abs(filepath.Join(config.Cfg.Temp.BasePath, fmt.Sprintf("%s_%s", id, file.Name())))
+	if !config.C().Stream || ok {
+		cachePath, err := filepath.Abs(filepath.Join(config.C().Temp.BasePath, fmt.Sprintf("%s_%s", id, file.Name())))
 		if err != nil {
 			return nil, fmt.Errorf("failed to get absolute path for cache: %w", err)
 		}
@@ -89,6 +91,7 @@ func NewBatchTGFileTask(
 		}(),
 		processing:   make(map[string]TaskElementInfo),
 		IgnoreErrors: ignoreErrors,
+		processingMu: sync.RWMutex{},
 		failed:       make(map[string]error),
 	}
 	return task

core/tasks/batchtfile/taskinfo.go

@@ -1,4 +1,4 @@
-package batchtftask
+package batchtfile
 
 type TaskElementInfo interface {
 	FileName() string

core/tasks/batchtfile/utils.go

@@ -1,4 +1,4 @@
-package batchtftask
+package batchtfile
 
 var progressUpdatesLevels = []struct {
 	size        int64 // 文件大小阈值

core/tasks/parsed/execute.go

@@ -0,0 +1,139 @@
+package parsed
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"io"
+	"net/http"
+	"path"
+	"path/filepath"
+
+	"github.com/charmbracelet/log"
+	"github.com/duke-git/lancet/v2/retry"
+	"github.com/krau/SaveAny-Bot/common/utils/fsutil"
+	"github.com/krau/SaveAny-Bot/common/utils/ioutil"
+	"github.com/krau/SaveAny-Bot/config"
+	"github.com/krau/SaveAny-Bot/pkg/enums/ctxkey"
+	"github.com/krau/SaveAny-Bot/pkg/parser"
+	"golang.org/x/sync/errgroup"
+)
+
+func (t *Task) Execute(ctx context.Context) error {
+	logger := log.FromContext(ctx)
+	logger.Infof("Starting Parsed item task %s", t.item.Title)
+	if t.progress != nil {
+		t.progress.OnStart(ctx, t)
+	}
+	eg, gctx := errgroup.WithContext(ctx)
+	eg.SetLimit(config.C().Workers)
+	for _, resource := range t.item.Resources {
+		eg.Go(func() error {
+			t.processingMu.RLock()
+			if t.processing[resource.ID()] != nil {
+				return fmt.Errorf("resource %s is already being processed", resource.ID())
+			}
+			t.processingMu.RUnlock()
+			t.processingMu.Lock()
+			t.processing[resource.ID()] = &resource
+			t.processingMu.Unlock()
+			defer func() {
+				t.processingMu.Lock()
+				delete(t.processing, resource.URL)
+				t.processingMu.Unlock()
+			}()
+			err := t.processResource(gctx, resource)
+			t.downloaded.Add(1)
+			if errors.Is(err, context.Canceled) {
+				logger.Debug("Resource processing canceled")
+				return err
+			}
+			if err != nil {
+				logger.Errorf("Error processing resource %s: %v", resource.URL, err)
+				return fmt.Errorf("failed to process resource %s: %w", resource.URL, err)
+			}
+			return nil
+		})
+	}
+	err := eg.Wait()
+	if err != nil {
+		logger.Errorf("Error during Parsed item task execution: %v", err)
+	} else {
+		logger.Infof("Parsed item task %s completed successfully", t.item.Title)
+	}
+	if t.progress != nil {
+		t.progress.OnDone(ctx, t, err)
+	}
+	return err
+}
+
+func (t *Task) processResource(ctx context.Context, resource parser.Resource) error {
+	logger := log.FromContext(ctx)
+	err := retry.Retry(func() error {
+		req, err := http.NewRequestWithContext(ctx, http.MethodGet, resource.URL, nil)
+		if err != nil {
+			return err
+		}
+		if resource.Headers != nil {
+			for k, v := range resource.Headers {
+				req.Header.Set(k, v)
+			}
+		}
+		resp, err := t.httpClient.Do(req)
+		if err != nil {
+			return fmt.Errorf("failed to download resource %s: %w", resource.URL, err)
+		}
+		defer resp.Body.Close()
+		if resp.StatusCode != http.StatusOK {
+			return fmt.Errorf("failed to download resource %s: %s", resource.URL, resp.Status)
+		}
+		ctx = context.WithValue(ctx, ctxkey.ContentLength, func() int64 {
+			if resource.Size > 0 {
+				return resource.Size
+			}
+			return resp.ContentLength
+		}())
+		if t.stream {
+			return t.Stor.Save(ctx, resp.Body, path.Join(t.StorPath, resource.Filename))
+		}
+		cacheFile, err := fsutil.CreateFile(filepath.Join(config.C().Temp.BasePath,
+			fmt.Sprintf("resource_%s_%s", t.ID, resource.Filename)))
+		if err != nil {
+			return fmt.Errorf("failed to create cache file for resource %s: %w", resource.URL, err)
+		}
+		defer func() {
+			if err := cacheFile.CloseAndRemove(); err != nil {
+				logger.Errorf("Failed to close and remove cache file: %v", err)
+			}
+		}()
+		wr := ioutil.NewProgressWriter(cacheFile, func(n int) {
+			t.downloadedBytes.Add(int64(n))
+			if t.progress != nil {
+				t.progress.OnProgress(ctx, t)
+			}
+		})
+
+		copyResultCh := make(chan error, 1)
+		go func() {
+			_, err := io.Copy(wr, resp.Body)
+			copyResultCh <- err
+		}()
+		select {
+		case err := <-copyResultCh:
+			if err != nil {
+				return fmt.Errorf("failed to copy resource %s to cache file: %w", resource.URL, err)
+			}
+		case <-ctx.Done():
+			return ctx.Err()
+		}
+		_, err = cacheFile.Seek(0, 0)
+		if err != nil {
+			return fmt.Errorf("failed to seek cache file for resource %s: %w", resource.URL, err)
+		}
+		return t.Stor.Save(ctx, cacheFile, path.Join(t.StorPath, resource.Filename))
+	}, retry.Context(ctx), retry.RetryTimes(uint(config.C().Retry)))
+	if ctx.Err() != nil {
+		return ctx.Err()
+	}
+	return err
+}

core/tasks/parsed/progress.go

@@ -0,0 +1,209 @@
+package parsed
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"sync/atomic"
+	"time"
+
+	"github.com/charmbracelet/log"
+	"github.com/duke-git/lancet/v2/slice"
+	"github.com/gotd/td/telegram/message/entity"
+	"github.com/gotd/td/telegram/message/styling"
+	"github.com/gotd/td/tg"
+	"github.com/krau/SaveAny-Bot/common/utils/dlutil"
+	"github.com/krau/SaveAny-Bot/common/utils/tgutil"
+)
+
+var progressUpdatesLevels = []struct {
+	size        int64 // 文件大小阈值
+	stepPercent int   // 每多少 % 更新一次
+}{
+	{10 << 20, 100},
+	{50 << 20, 50},
+	{200 << 20, 20},
+	{500 << 20, 10},
+}
+
+func shouldUpdateProgress(total, downloaded int64, lastUpdatePercent int) bool {
+	if total <= 0 || downloaded <= 0 {
+		return false
+	}
+
+	percent := int((downloaded * 100) / total)
+	if percent <= lastUpdatePercent {
+		return false
+	}
+
+	step := progressUpdatesLevels[len(progressUpdatesLevels)-1].stepPercent
+	for _, lvl := range progressUpdatesLevels {
+		if total < lvl.size {
+			step = lvl.stepPercent
+			break
+		}
+	}
+
+	return percent >= lastUpdatePercent+step
+}
+
+type ProgressTracker interface {
+	OnStart(ctx context.Context, info TaskInfo)
+	OnProgress(ctx context.Context, info TaskInfo)
+	OnDone(ctx context.Context, info TaskInfo, err error)
+}
+
+type Progress struct {
+	MessageID         int
+	ChatID            int64
+	start             time.Time
+	lastUpdatePercent atomic.Int32
+}
+
+func (p *Progress) OnStart(ctx context.Context, info TaskInfo) {
+	logger := log.FromContext(ctx)
+	p.start = time.Now()
+	p.lastUpdatePercent.Store(0)
+	logger.Debugf("Parsed task progress tracking started for message %d in chat %d", p.MessageID, p.ChatID)
+	entityBuilder := entity.Builder{}
+	var entities []tg.MessageEntityClass
+	if err := styling.Perform(&entityBuilder,
+		styling.Plain(fmt.Sprintf("开始下载 %s 的资源\n总大小: ", info.Site())),
+		styling.Code(fmt.Sprintf("%.2f MB (%d个资源)", float64(info.TotalBytes())/(1024*1024), info.TotalResources())),
+	); err != nil {
+		log.FromContext(ctx).Errorf("Failed to build entities: %s", err)
+		return
+	}
+	text, entities := entityBuilder.Complete()
+	req := &tg.MessagesEditMessageRequest{
+		ID: p.MessageID,
+	}
+	req.SetMessage(text)
+	req.SetEntities(entities)
+	req.SetReplyMarkup(&tg.ReplyInlineMarkup{
+		Rows: []tg.KeyboardButtonRow{
+			{
+				Buttons: []tg.KeyboardButtonClass{
+					tgutil.BuildCancelButton(info.TaskID()),
+				},
+			},
+		}},
+	)
+	ext := tgutil.ExtFromContext(ctx)
+	if ext != nil {
+		ext.EditMessage(p.ChatID, req)
+		return
+	}
+}
+
+func (p *Progress) OnProgress(ctx context.Context, info TaskInfo) {
+	if !shouldUpdateProgress(info.TotalBytes(), info.DownloadedBytes(), int(p.lastUpdatePercent.Load())) {
+		return
+	}
+	percent := int((info.DownloadedBytes() * 100) / info.TotalBytes())
+	if p.lastUpdatePercent.Load() == int32(percent) {
+		return
+	}
+	p.lastUpdatePercent.Store(int32(percent))
+	log.FromContext(ctx).Debugf("Progress update: %s, %d/%d", info.TaskID(), info.DownloadedBytes(), info.TotalBytes())
+	entityBuilder := entity.Builder{}
+	var entities []tg.MessageEntityClass
+	if err := styling.Perform(&entityBuilder,
+		styling.Plain("正在下载\n总大小: "),
+		styling.Code(fmt.Sprintf("%.2f MB (%d个文件)", float64(info.TotalBytes())/(1024*1024), info.TotalResources())),
+		styling.Plain("\n正在处理:\n"),
+		func() styling.StyledTextOption {
+			var lines []string
+			for _, elem := range info.Processing() {
+				lines = append(lines, fmt.Sprintf("  - %s (%.2f MB)", elem.FileName(), float64(elem.FileSize())/(1024*1024)))
+			}
+			if len(lines) == 0 {
+				lines = append(lines, "  - 无")
+			}
+			return styling.Plain(slice.Join(lines, "\n"))
+		}(),
+		styling.Plain("\n平均速度: "),
+		styling.Bold(fmt.Sprintf("%.2f MB/s", dlutil.GetSpeed(info.DownloadedBytes(), p.start)/(1024*1024))),
+		styling.Plain("\n当前进度: "),
+		styling.Bold(fmt.Sprintf("%.2f%%", float64(info.DownloadedBytes())/float64(info.TotalBytes())*100)),
+	); err != nil {
+		log.FromContext(ctx).Errorf("Failed to build entities: %s", err)
+		return
+	}
+	text, entities := entityBuilder.Complete()
+	req := &tg.MessagesEditMessageRequest{
+		ID: p.MessageID,
+	}
+	req.SetMessage(text)
+	req.SetEntities(entities)
+	req.SetReplyMarkup(&tg.ReplyInlineMarkup{
+		Rows: []tg.KeyboardButtonRow{
+			{
+				Buttons: []tg.KeyboardButtonClass{
+					tgutil.BuildCancelButton(info.TaskID()),
+				},
+			},
+		}},
+	)
+	ext := tgutil.ExtFromContext(ctx)
+	if ext != nil {
+		ext.EditMessage(p.ChatID, req)
+		return
+	}
+}
+
+func (p *Progress) OnDone(ctx context.Context, info TaskInfo, err error) {
+	logger := log.FromContext(ctx)
+	if err != nil {
+		if errors.Is(err, context.Canceled) {
+			logger.Infof("Parsed task %s was canceled", info.TaskID())
+			ext := tgutil.ExtFromContext(ctx)
+			if ext != nil {
+				ext.EditMessage(p.ChatID, &tg.MessagesEditMessageRequest{
+					ID:      p.MessageID,
+					Message: fmt.Sprintf("处理已取消: %s", info.TaskID()),
+				})
+			}
+		} else {
+			logger.Errorf("Parsed task %s failed: %s", info.TaskID(), err)
+			ext := tgutil.ExtFromContext(ctx)
+			if ext != nil {
+				ext.EditMessage(p.ChatID, &tg.MessagesEditMessageRequest{
+					ID:      p.MessageID,
+					Message: fmt.Sprintf("处理失败: %s", err.Error()),
+				})
+			}
+		}
+		return
+	}
+	logger.Infof("Parsed task %s completed successfully", info.TaskID())
+
+	entityBuilder := entity.Builder{}
+	if err := styling.Perform(&entityBuilder,
+		styling.Plain("处理完成, 资源数量: "),
+		styling.Code(fmt.Sprintf("%d", info.TotalResources())),
+		styling.Plain("\n保存路径: "),
+		styling.Code(fmt.Sprintf("[%s]:%s", info.StorageName(), info.StoragePath())),
+	); err != nil {
+		logger.Errorf("Failed to build entities: %s", err)
+		return
+	}
+	text, entities := entityBuilder.Complete()
+	req := &tg.MessagesEditMessageRequest{
+		ID: p.MessageID,
+	}
+	req.SetMessage(text)
+	req.SetEntities(entities)
+
+	ext := tgutil.ExtFromContext(ctx)
+	if ext != nil {
+		ext.EditMessage(p.ChatID, req)
+	}
+}
+
+func NewProgress(messageID int, chatID int64) *Progress {
+	return &Progress{
+		MessageID: messageID,
+		ChatID:    chatID,
+	}
+}

core/tasks/parsed/task.go

@@ -0,0 +1,80 @@
+package parsed
+
+import (
+	"context"
+	"net/http"
+	"sync"
+	"sync/atomic"
+
+	"github.com/krau/SaveAny-Bot/common/utils/netutil"
+	"github.com/krau/SaveAny-Bot/config"
+	"github.com/krau/SaveAny-Bot/pkg/enums/tasktype"
+	"github.com/krau/SaveAny-Bot/pkg/parser"
+	"github.com/krau/SaveAny-Bot/storage"
+)
+
+type Task struct {
+	ID         string
+	Ctx        context.Context
+	Stor       storage.Storage
+	StorPath   string
+	item       *parser.Item
+	httpClient *http.Client
+	progress   ProgressTracker
+	stream     bool
+
+	totalResources  int64
+	downloaded      atomic.Int64 // downloaded resources count
+	totalBytes      int64        // total bytes to download
+	downloadedBytes atomic.Int64 // downloaded bytes count
+	processing      map[string]ResourceInfo
+	processingMu    sync.RWMutex
+	failed          map[string]error // [TODO] errors for each resource
+}
+
+func (t *Task) Type() tasktype.TaskType {
+	return tasktype.TaskTypeParseditem
+}
+
+func (t *Task) TaskID() string {
+	return t.ID
+}
+
+func NewTask(
+	id string,
+	ctx context.Context,
+	stor storage.Storage,
+	storPath string,
+	item *parser.Item,
+	progressTracker ProgressTracker,
+) *Task {
+	client := netutil.DefaultParserHTTPClient()
+	_, ok := stor.(storage.StorageCannotStream)
+	stream := config.C().Stream && !ok
+	return &Task{
+		ID:             id,
+		Ctx:            ctx,
+		Stor:           stor,
+		StorPath:       storPath,
+		item:           item,
+		totalResources: int64(len(item.Resources)),
+		downloaded:     atomic.Int64{},
+		totalBytes: func() int64 {
+			var total int64
+			for _, res := range item.Resources {
+				if res.Size < 0 {
+					continue // skip resources with unknown size
+				}
+				total += res.Size
+			}
+			return total
+		}(),
+		stream:          stream,
+		downloadedBytes: atomic.Int64{},
+		httpClient:      client,
+		progress:        progressTracker,
+		processing:      make(map[string]ResourceInfo),
+		processingMu:    sync.RWMutex{},
+		failed:          make(map[string]error),
+	}
+}

core/tasks/parsed/taskinfo.go

@@ -0,0 +1,51 @@
+package parsed
+
+type TaskInfo interface {
+	TaskID() string
+	Site() string
+	TotalResources() int64
+	Downloaded() int64
+	TotalBytes() int64
+	DownloadedBytes() int64
+	Processing() map[string]ResourceInfo
+	StorageName() string
+	StoragePath() string
+}
+
+func (t *Task) StoragePath() string {
+	return t.StorPath
+}
+func (t *Task) TotalResources() int64 {
+	return t.totalResources
+}
+
+func (t *Task) Downloaded() int64 {
+	return t.downloaded.Load()
+}
+
+func (t *Task) StorageName() string {
+	return t.Stor.Name()
+}
+
+func (t *Task) Site() string {
+	return t.item.Site
+}
+
+func (t *Task) TotalBytes() int64 {
+	return t.totalBytes
+}
+
+func (t *Task) DownloadedBytes() int64 {
+	return t.downloadedBytes.Load()
+}
+
+func (t *Task) Processing() map[string]ResourceInfo {
+	t.processingMu.RLock()
+	defer t.processingMu.RUnlock()
+	return t.processing
+}
+
+type ResourceInfo interface {
+	FileName() string
+	FileSize() int64
+}

core/tasks/telegraph/execute.go

@@ -1,4 +1,4 @@
-package tphtask
+package telegraph
 
 import (
 	"context"
@@ -20,10 +20,8 @@ func (t *Task) Execute(ctx context.Context) error {
 	logger.Infof("Starting Telegraph task %s", t.PhPath)
 	t.progress.OnStart(ctx, t)
 	eg, gctx := errgroup.WithContext(ctx)
-	eg.SetLimit(config.Cfg.Workers)
+	eg.SetLimit(config.C().Workers)
 	for i, pic := range t.Pics {
-		pic := pic
-		i := i
 		eg.Go(func() error {
 			err := t.processPic(gctx, pic, i)
 			if err != nil {
@@ -48,7 +46,7 @@ func (t *Task) Execute(ctx context.Context) error {
 func (t *Task) processPic(ctx context.Context, picUrl string, index int) error {
 	retryOpts := []retry.Option{
 		retry.Context(ctx),
-		retry.RetryTimes(uint(config.Cfg.Retry)),
+		retry.RetryTimes(uint(config.C().Retry)),
 	}
 	var lastErr error
 	err := retry.Retry(func() error {
@@ -61,7 +59,7 @@ func (t *Task) processPic(ctx context.Context, picUrl string, index int) error {
 		defer body.Close()
 		filename := fmt.Sprintf("%d%s", index+1, path.Ext(picUrl))
 		if t.cannotStream {
-			cacheFile, err := fsutil.CreateFile(filepath.Join(config.Cfg.Temp.BasePath,
+			cacheFile, err := fsutil.CreateFile(filepath.Join(config.C().Temp.BasePath,
 				fmt.Sprintf("tph_%s_%s", t.TaskID(), filename),
 			))
 			if err != nil {
@@ -79,6 +77,11 @@ func (t *Task) processPic(ctx context.Context, picUrl string, index int) error {
 				lastErr = fmt.Errorf("failed to copy picture %s to cache file: %w", filename, lastErr)
 				return lastErr
 			}
+			_, err = cacheFile.Seek(0, 0)
+			if err != nil {
+				lastErr = fmt.Errorf("failed to seek cache file for picture %s: %w", filename, err)
+				return lastErr
+			}
 			lastErr = t.Stor.Save(ctx, cacheFile, path.Join(t.StorPath, filename))
 		} else {
 			lastErr = t.Stor.Save(ctx, body, path.Join(t.StorPath, filename))

core/tasks/telegraph/progress.go

@@ -1,4 +1,4 @@
-package tphtask
+package telegraph
 
 import (
 	"context"

core/tasks/telegraph/task.go

@@ -1,4 +1,4 @@
-package tphtask
+package telegraph
 
 import (
 	"context"
@@ -39,7 +39,7 @@ func NewTask(
 	progress ProgressTracker,
 ) *Task {
 	_, cannotStream := stor.(storage.StorageCannotStream)
-	tphtask := &Task{
+	telegraph := &Task{
 		ID:           id,
 		Ctx:          ctx,
 		PhPath:       phPath,
@@ -52,5 +52,5 @@ func NewTask(
 		totalpics:    len(pics),
 		downloaded:   atomic.Int64{},
 	}
-	return tphtask
+	return telegraph
 }

core/tasks/telegraph/taskinfo.go

@@ -1,4 +1,4 @@
-package tphtask
+package telegraph
 
 type TaskInfo interface {
 	TaskID() string

core/tasks/telegraph/utils.go

@@ -1,4 +1,4 @@
-package tphtask
+package telegraph
 
 func shouldUpdateProgress(downloaded int64, total int64) bool {
 	if total <= 0 || downloaded <= 0 {

core/tasks/tfile/execute.go

@@ -1,4 +1,4 @@
-package tftask
+package tfile
 
 import (
 	"context"
@@ -16,7 +16,9 @@ import (
 
 func (t *Task) Execute(ctx context.Context) error {
 	logger := log.FromContext(ctx).WithPrefix(fmt.Sprintf("file[%s]", t.File.Name()))
-	t.Progress.OnStart(ctx, t)
+	if t.Progress != nil {
+		t.Progress.OnStart(ctx, t)
+	}
 	if t.stream {
 		return executeStream(ctx, t)
 	}
@@ -34,7 +36,9 @@ func (t *Task) Execute(ctx context.Context) error {
 	wrAt := newWriterAt(ctx, localFile, t.Progress, t)
 
 	defer func() {
-		t.Progress.OnDone(ctx, t, err)
+		if t.Progress != nil {
+			t.Progress.OnDone(ctx, t, err)
+		}
 	}()
 	_, err = tfile.NewDownloader(t.File).Parallel(ctx, wrAt)
 	if err != nil {
@@ -53,7 +57,7 @@ func (t *Task) Execute(ctx context.Context) error {
 		return fmt.Errorf("failed to get file stat: %w", err)
 	}
 	vctx := context.WithValue(ctx, ctxkey.ContentLength, fileStat.Size())
-	for i := range config.Cfg.Retry + 1 {
+	for i := range config.C().Retry + 1 {
 		if err = vctx.Err(); err != nil {
 			return fmt.Errorf("context canceled while saving file: %w", err)
 		}
@@ -64,7 +68,7 @@ func (t *Task) Execute(ctx context.Context) error {
 		}
 		defer file.Close()
 		if err = t.Storage.Save(vctx, file, t.Path); err != nil {
-			if i == config.Cfg.Retry {
+			if i == config.C().Retry {
 				return fmt.Errorf("failed to save file: %w", err)
 			}
 			logger.Errorf("Failed to save file: %s, retrying...", err)

core/tasks/tfile/progress.go

@@ -1,4 +1,4 @@
-package tftask
+package tfile
 
 import (
 	"context"

core/tasks/tfile/stream.go

@@ -1,4 +1,4 @@
-package tftask
+package tfile
 
 import (
 	"context"
@@ -32,7 +32,9 @@ func executeStream(ctx context.Context, task *Task) error {
 	})
 	var err error
 	defer func() {
-		task.Progress.OnDone(ctx, task, err)
+		if task.Progress != nil {
+			task.Progress.OnDone(ctx, task, err)
+		}
 	}()
 	if err = errg.Wait(); err != nil {
 		return err

core/tasks/tfile/taskinfo.go

@@ -1,4 +1,4 @@
-package tftask
+package tfile
 
 type TaskInfo interface {
 	TaskID() string

core/tasks/tfile/tftask.go

@@ -1,4 +1,4 @@
-package tftask
+package tfile
 
 import (
 	"context"
@@ -35,12 +35,12 @@ func NewTGFileTask(
 	progress ProgressTracker,
 ) (*Task, error) {
 	_, ok := stor.(storage.StorageCannotStream)
-	if !config.Cfg.Stream || ok {
-		cachePath, err := filepath.Abs(filepath.Join(config.Cfg.Temp.BasePath, fmt.Sprintf("%s_%s", id, file.Name())))
+	if !config.C().Stream || ok {
+		cachePath, err := filepath.Abs(filepath.Join(config.C().Temp.BasePath, fmt.Sprintf("%s_%s", id, file.Name())))
 		if err != nil {
 			return nil, fmt.Errorf("failed to get absolute path for cache: %w", err)
 		}
-		tftask := &Task{
+		tfile := &Task{
 			ID:        id,
 			Ctx:       ctx,
 			File:      file,
@@ -49,7 +49,7 @@ func NewTGFileTask(
 			Progress:  progress,
 			localPath: cachePath,
 		}
-		return tftask, nil
+		return tfile, nil
 	}
 	tfileTask := &Task{
 		ID:       id,

core/tasks/tfile/util.go

@@ -1,4 +1,4 @@
-package tftask
+package tfile
 
 var progressUpdatesLevels = []struct {
 	size        int64 // 文件大小阈值

core/tasks/tfile/writer.go

@@ -1,4 +1,4 @@
-package tftask
+package tfile
 
 import (
 	"context"
@@ -20,7 +20,9 @@ func (w *ProgressWriterAt) WriteAt(p []byte, off int64) (int, error) {
 	if err != nil {
 		return 0, err
 	}
-	w.progress.OnProgress(w.ctx, w.info, w.downloaded.Add(int64(at)), w.total)
+	if w.progress != nil {
+		w.progress.OnProgress(w.ctx, w.info, w.downloaded.Add(int64(at)), w.total)
+	}
 	return at, nil
 }
 
@@ -54,7 +56,9 @@ func (w *ProgressWriter) Write(p []byte) (int, error) {
 	if err != nil {
 		return 0, err
 	}
-	w.progress.OnProgress(w.ctx, w.info, w.downloaded.Add(int64(at)), w.total)
+	if w.progress != nil {
+		w.progress.OnProgress(w.ctx, w.info, w.downloaded.Add(int64(at)), w.total)
+	}
 	return at, nil
 }
 

database/chat.go

@@ -0,0 +1,39 @@
+package database
+
+import "context"
+
+func (user *User) WatchChat(ctx context.Context, chat WatchChat) error {
+	if len(user.WatchChats) == 0 {
+		user.WatchChats = make([]WatchChat, 0)
+	}
+
+	user.WatchChats = append(user.WatchChats, chat)
+	return db.WithContext(ctx).Save(user.WatchChats).Error
+}
+
+func (user *User) UnwatchChat(ctx context.Context, chatID int64) error {
+	var watchChat WatchChat
+	err := db.WithContext(ctx).Where("chat_id = ? AND user_id = ?", chatID, user.ID).First(&watchChat).Error
+	if err != nil {
+		return err
+	}
+	return db.WithContext(ctx).Unscoped().Delete(&watchChat).Error
+}
+
+func (user *User) WatchingChat(ctx context.Context, chatID int64) (bool, error) {
+	var count int64
+	err := db.WithContext(ctx).Model(&WatchChat{}).Where("chat_id = ? AND user_id = ?", chatID, user.ID).Count(&count).Error
+	if err != nil {
+		return false, err
+	}
+	return count > 0, nil
+}
+
+func GetWatchChatsByChatID(ctx context.Context, chatID int64) ([]*WatchChat, error) {
+	var watchChats []*WatchChat
+	err := db.WithContext(ctx).Where("chat_id = ?", chatID).Find(&watchChats).Error
+	if err != nil {
+		return nil, err
+	}
+	return watchChats, nil
+}

database/db.go

@@ -19,11 +19,11 @@ var db *gorm.DB
 
 func Init(ctx context.Context) {
 	logger := log.FromContext(ctx)
-	if err := os.MkdirAll(filepath.Dir(config.Cfg.DB.Path), 0755); err != nil {
+	if err := os.MkdirAll(filepath.Dir(config.C().DB.Path), 0755); err != nil {
 		logger.Fatal("Failed to create data directory: ", err)
 	}
 	var err error
-	db, err = gorm.Open(gormlite.Open(config.Cfg.DB.Path), &gorm.Config{
+	db, err = gorm.Open(gormlite.Open(config.C().DB.Path), &gorm.Config{
 		Logger: glogger.New(logger, glogger.Config{
 			Colorful:                  true,
 			SlowThreshold:             time.Second * 5,
@@ -37,7 +37,7 @@ func Init(ctx context.Context) {
 		logger.Fatal("Failed to open database: ", err)
 	}
 	logger.Debug("Database connected")
-	if err := db.AutoMigrate(&User{}, &Dir{}, &Rule{}); err != nil {
+	if err := db.AutoMigrate(&User{}, &Dir{}, &Rule{}, &WatchChat{}); err != nil {
 		logger.Fatal("迁移数据库失败, 如果您从旧版本升级, 建议手动删除数据库文件后重试: ", err)
 	}
 	if err := syncUsers(ctx); err != nil {
@@ -60,7 +60,7 @@ func syncUsers(ctx context.Context) error {
 	}
 
 	cfgUserMap := make(map[int64]struct{})
-	for _, u := range config.Cfg.Users {
+	for _, u := range config.C().Users {
 		cfgUserMap[u.ID] = struct{}{}
 	}
 

database/model.go

@@ -6,12 +6,21 @@ import (
 
 type User struct {
 	gorm.Model
-	ChatID         int64 `gorm:"uniqueIndex;not null"`
-	Silent         bool
-	DefaultStorage string
-	Dirs           []Dir
-	ApplyRule      bool
-	Rules          []Rule
+	ChatID           int64 `gorm:"uniqueIndex;not null"`
+	Silent           bool
+	DefaultStorage   string
+	Dirs             []Dir
+	ApplyRule        bool
+	Rules            []Rule
+	WatchChats       []WatchChat
+	FilenameStrategy string
+}
+
+type WatchChat struct {
+	gorm.Model
+	UserID uint // User's database ID (not chat ID)
+	ChatID int64
+	Filter string
 }
 
 type Dir struct {

database/user.go

@@ -1,6 +1,10 @@
 package database
 
-import "context"
+import (
+	"context"
+
+	"gorm.io/gorm/clause"
+)
 
 func CreateUser(ctx context.Context, chatID int64) error {
 	if _, err := GetUserByChatID(ctx, chatID); err == nil {
@@ -11,19 +15,16 @@ func CreateUser(ctx context.Context, chatID int64) error {
 
 func GetAllUsers(ctx context.Context) ([]User, error) {
 	var users []User
-	err := db.Preload("Dirs").
-		WithContext(ctx).
-		Preload("Rules").
+	err := db.WithContext(ctx).
+		Preload(clause.Associations).
 		Find(&users).Error
 	return users, err
 }
 
 func GetUserByChatID(ctx context.Context, chatID int64) (*User, error) {
 	var user User
-	err := db.
-		Preload("Dirs").
-		WithContext(ctx).
-		Preload("Rules").
+	err := db.WithContext(ctx).
+		Preload(clause.Associations).
 		Where("chat_id = ?", chatID).First(&user).Error
 	return &user, err
 }
@@ -36,5 +37,16 @@ func UpdateUser(ctx context.Context, user *User) error {
 }
 
 func DeleteUser(ctx context.Context, user *User) error {
-	return db.WithContext(ctx).Unscoped().Select("Dirs", "Rules").Delete(user).Error
+	return db.WithContext(ctx).
+		Unscoped().
+		Select(clause.Associations).
+		Delete(user).Error
 }
+
+func GetUserByID(ctx context.Context, id uint) (*User, error) {
+	var user User
+	err := db.WithContext(ctx).
+		Preload(clause.Associations).
+		Where("id = ?", id).First(&user).Error
+	return &user, err
+}

docs/content/en/deployment/installation.md

@@ -33,7 +33,7 @@ After=systemd-user-sessions.service
 Type=simple
 WorkingDirectory=/yourpath/
 ExecStart=/yourpath/saveany-bot
-Restart=on-failure
+Restart=always
 
 [Install]
 WantedBy=multi-user.target

docs/content/zh/_index.md

@@ -11,20 +11,22 @@ title: 介绍
 
 把 Telegram 上的文件转存到多种存储端.
 
-## 特性
+## 🎯 特性
 
-- 支持文档/视频/图片/贴纸... 甚至还有 Telegraph
+- 支持文档/视频/图片/贴纸…甚至还有 [Telegraph](https://telegra.ph/)
 - 破解禁止保存的文件
 - 批量下载
 - 流式传输
-- 多用户
+- 多用户使用
 - 基于存储规则的自动整理
-- 支持多种存储端:
-    - Alist
-    - Minio (S3 兼容)
-    - WebDAV
-    - Telegram (重传回指定聊天)
-    - 本地磁盘
+- 监听并自动转存指定聊天的消息, 支持过滤
+- 使用 js 编写解析器插件以转存任意网站的文件
+- 存储端支持:
+  - Alist
+  - S3 (MinioSDK)
+  - WebDAV
+  - 本地磁盘
+  - Telegram (重传回指定聊天)
 
 ## [贡献者](https://github.com/krau/SaveAny-Bot/graphs/contributors)
 

docs/content/zh/contribute/_index.md

@@ -5,10 +5,30 @@ weight: 20
 
 # 参与开发
 
+在开始之前, 请 Fork 本项目, 并克隆到本地, 并确保 Go 版本 >= 1.23. 
+
+以下是一些贡献代码的指南或建议, 你不必完全遵守, 但将有助于快速 review 并合并你的提交:
+
+- **新功能请先提交 Issue**, 以便讨论设计和实现细节, 并避免因与项目设计不符而被拒绝.
+- **使用现代开发工具**, 确保提交前格式化代码, 并保持风格一致.
+- **使用[语义化提交](https://www.conventionalcommits.org/zh-hans/v1.0.0/)**, 避免提交消息模糊或过于简单.
+
 ## 贡献新存储端
 
-1. Fork 本项目, 克隆到本地
-2. 在 `pkg/enums/storage/storages.go` 中添加新的存储端类型, 并运行代码生成
-3. 在 `config/storage` 目录下定义存储端配置, 并添加到 `config/storage/factory.go` 中
-4. 在 `storage` 目录下新建一个包, 编写存储端实现, 然后在 `storage/storage.go` 中导入并添加它
-5. 更新文档, 添加配置说明
+1. 在 `pkg/enums/storage/storages.go` 中添加新的存储端类型, 并运行代码生成
+2. 在 `config/storage` 目录下定义存储端配置, 并添加到 `config/storage/factory.go` 中
+3. 在 `storage` 目录下新建一个包, 编写存储端实现, 然后在 `storage/storage.go` 中导入并添加它
+4. 更新文档, 添加配置说明
+
+## 贡献新解析器
+
+你可以选择使用 Go 编写原生的解析器实现(推荐), 或是使用 JavaScript 以插件的方式实现.
+
+如果使用 Go 编写, 请:
+
+1. 在 `parsers` 目录下新建一个包, 编写解析器实现
+2. 在 `parsers/parser.go` 的 `init` 中注册解析器
+
+如果使用 JavaScript 编写, 请参考 `plugins/example_parser_basic.js` 的实现, 并在该文件夹下新建一个 js 文件, 实现你的解析逻辑.
+
+需要注意, `plugins` 目录下解析器默认不会被编译到二进制文件中, 用户需要手动下载它们并放到本地指定目录下以启用它们.

docs/content/zh/deployment/configuration/_index.md

@@ -61,9 +61,9 @@ Stream 模式对于磁盘空间有限的部署环境十分有用, 但也有一
 {{< hint warning >}}
 启用 userbot 集成后, bot 可以下载私密频道和群组的文件, 但具有无法避免的账号被封禁的风险.
 <br />
-并且, 由于上游依赖问题, 该功能不稳定, 会出现获取文件失败的情况.
+开启 userbot 集成后第一次启动 bot 时需要通过终端交互输入手机号, 2FA 和验证码.
 <br />
-开启 userbot 集成后第一次启动 bot 时需要通过终端交互输入手机号, 2FA 和验证码, 如果你使用 docker 部署, 请进入容器内执行相关操作.
+如果你使用 docker 部署, 请进入容器内执行相关操作.
 {{< /hint >}}
 
 ```toml
@@ -166,6 +166,18 @@ task_fail = "curl -X POST https://example.com/api/notify -d '任务失败'"
 task_cancel = "bash /path/to/cancel_script.sh"
 ```
 
+### 解析器
+
+解析器为 Bot 提供了处理非 Telegram 文件的能力, 例如从其他网站下载文件. 使用 `[parsers]` 配置.
+
+```toml
+[parsers]
+plugin_enable = true # 是否启用解析器插件
+plugin_dirs = ["./plugins"] # 插件目录, 可以是多个目录
+```
+
+上述两个配置项只用于控制以 JavaScript 编写的解析器插件, Bot 还有内置的使用 Go 实现的解析器, 目前默认开启.
+
 ### 杂项
 
 ```toml

docs/content/zh/deployment/installation.md

@@ -4,7 +4,7 @@ title: "安装与更新"
 
 # 安装与更新
 
-## 从预编译文件部署
+## 从预编译文件部署(推荐)
 
 在 [Release](https://github.com/krau/SaveAny-Bot/releases) 页面下载对应平台的二进制文件.
 
@@ -33,7 +33,7 @@ After=systemd-user-sessions.service
 Type=simple
 WorkingDirectory=/yourpath/
 ExecStart=/yourpath/saveany-bot
-Restart=on-failure
+Restart=always
 
 [Install]
 WantedBy=multi-user.target
@@ -131,13 +131,13 @@ docker run -d --name saveany-bot \
 
 ## 更新
 
-使用 `upgrade` 或 `up` 升级到最新版
+向 Bot 发送 `/update` 指令检查更新并升级, 或者使用 CLI 命令更新:
 
 ```bash
-./saveany-bot upgrade
+./saveany-bot up
 ```
 
-如果是 Docker 部署, 使用以下命令更新:
+如果是 Docker 部署, 还可以使用以下命令更新:
 
 ```bash
 docker pull ghcr.io/krau/saveany-bot:latest

docs/content/zh/usage/_index.md

@@ -9,12 +9,11 @@ weight: 10
 
 ## 转存文件
 
-Bot 接受两种消息: 文件和链接.
+要使用 Bot 的转存 Telegram 文件功能, 需要向 Bot 发送或转发以下类型的消息.
 
-对于链接, 目前支持以下类型的链接:
-
-1. Telegram 消息链接, 例如: `https://t.me/acherkrau/1097`. **即使频道禁止了转发和保存, Bot 依然可以下载其文件.**
-2. Telegra.ph 的文章链接, Bot 将下载其中的所有图片
+1. 文件或媒体消息, 如图片, 视频, 文档等
+2. Telegram 消息链接, 例如: `https://t.me/acherkrau/1097`. **即使频道禁止了转发和保存, Bot 依然可以下载其文件.**
+3. Telegra.ph 的文章链接, Bot 将下载其中的所有图片
 
 ## 静默模式 (silent)
 
@@ -50,7 +49,7 @@ Bot 接受两种消息: 文件和链接.
 
 此外, 规则中的存储名若使用 "CHOSEN" , 则表示存储到点击按钮选择的存储端的路径下
 
-规则介绍:
+规则类型:
 
 ### FILENAME-REGEX
 
@@ -78,4 +77,48 @@ FILENAME-REGEX (?i)\.(mp4|mkv|ts|avi|flv)$ MyAlist /视频
 IS-ALBUM true MyWebdav NEW-FOR-ALBUM
 ```
 
-这将会把以 media group 形式发送的消息保存到名为 MyWebdav 的存储下, 并为每个相册新建一个文件夹(由第一个文件生成)来存储它们.
+这将会把以 media group 形式发送的消息保存到名为 MyWebdav 的存储下, 并为每个相册新建一个文件夹(由第一个文件生成)来存储它们.
+
+
+## 监听聊天
+
+{{< hint warning >}}
+该功能需开启 UserBot 集成.
+{{< /hint >}}
+
+监听指定聊天的消息, 并自动保存到默认存储中, 遵从存储规则, 并且可以设置过滤器来只保存匹配的消息.
+
+监听聊天:
+
+```
+/watch <chat_id/username> [filter] 
+```
+
+取消监听:
+
+```
+/unwatch <chat_id/username>
+```
+
+过滤器类型:
+
+### msgre
+
+正则匹配消息文本, 例如:
+
+```
+/watch 12345678 msgre:.*hello.*
+```
+
+这将会监听 ID 为 12345678 的聊天, 并且只保存消息文本中包含 "hello" 的消息.
+
+## 转存 Telegram 之外的文件
+
+除了 Telegram 上的文件, Bot 还可通过 JavaScript 插件或内置解析器来支持转存其他网站的文件.
+
+> 查看[贡献解析器](../contribute)文档了解详情
+
+只需向 Bot 发送符合解析器要求的链接即可使用, 当前内置的解析器:
+
+- Twitter
+- Kemono

go.mod

@@ -10,7 +10,6 @@ require (
 	github.com/charmbracelet/log v0.4.2
 	github.com/fatih/color v1.18.0
 	github.com/gabriel-vasile/mimetype v1.4.9
-	github.com/go-faster/errors v0.7.1
 	github.com/gotd/contrib v0.21.0
 	github.com/gotd/td v0.129.0
 	github.com/minio/minio-go/v7 v7.0.95
@@ -42,15 +41,18 @@ require (
 	github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f // indirect
 	github.com/ghodss/yaml v1.0.0 // indirect
 	github.com/glebarez/go-sqlite v1.22.0 // indirect
+	github.com/go-faster/errors v0.7.1 // indirect
 	github.com/go-faster/jx v1.1.0 // indirect
 	github.com/go-faster/xor v1.0.0 // indirect
 	github.com/go-faster/yaml v0.4.6 // indirect
 	github.com/go-ini/ini v1.67.0 // indirect
 	github.com/go-logfmt/logfmt v0.6.0 // indirect
+	github.com/go-sourcemap/sourcemap v2.1.3+incompatible // indirect
 	github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
 	github.com/goccy/go-json v0.10.5 // indirect
 	github.com/google/go-github/v30 v30.1.0 // indirect
 	github.com/google/go-querystring v1.1.0 // indirect
+	github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e // indirect
 	github.com/google/uuid v1.6.0 // indirect
 	github.com/gotd/ige v0.2.2 // indirect
 	github.com/gotd/neo v0.1.5 // indirect
@@ -102,6 +104,7 @@ require (
 
 require (
 	github.com/dgraph-io/ristretto/v2 v2.2.0
+	github.com/dop251/goja v0.0.0-20250630131328-58d95d85e994
 	github.com/duke-git/lancet/v2 v2.3.7
 	github.com/fsnotify/fsnotify v1.9.0 // indirect
 	github.com/glebarez/sqlite v1.11.0 // indirect
@@ -128,4 +131,4 @@ require (
 	gorm.io/gorm v1.30.1
 )
 
-replace github.com/celestix/gotgproto v1.0.0-beta21 => github.com/krau/gotgproto v0.0.0-20250730080659-caaadb4b1f35
+replace github.com/celestix/gotgproto v1.0.0-beta21 => github.com/krau/gotgproto v0.0.0-20250815074212-7fbd56c33c00

go.sum

@@ -1,11 +1,11 @@
-github.com/AnimeKaizoku/cacher v1.0.3-0.20250508132714-ddc7471efeef h1:y8llZexgesBazUg/zdIlrZKHPFJ2zk9YxlLAJ5x4DpQ=
-github.com/AnimeKaizoku/cacher v1.0.3-0.20250508132714-ddc7471efeef/go.mod h1:jw0de/b0K6W7Y3T9rHCMGVKUf6oG7hENNcssxYcZTCc=
 github.com/AnimeKaizoku/cacher v1.0.3 h1:foNAmLfY/DXfA4yEy4uP6WK2Ni7JC+s3QhZv72Dn6zs=
 github.com/AnimeKaizoku/cacher v1.0.3/go.mod h1:jw0de/b0K6W7Y3T9rHCMGVKUf6oG7hENNcssxYcZTCc=
 github.com/BurntSushi/toml v1.5.0 h1:W5quZX/G/csjUnuI8SUYlsHs9M38FC7znL0lIO+DvMg=
 github.com/BurntSushi/toml v1.5.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
 github.com/MakeNowJust/heredoc v1.0.0 h1:cXCdzVdstXyiTqTvfqk9SDHpKNjxuom+DOlyEeQ4pzQ=
 github.com/MakeNowJust/heredoc v1.0.0/go.mod h1:mG5amYoWBHf8vpLOuehzbGGw0EHxpZZ6lCpQ4fNJ8LE=
+github.com/Masterminds/semver/v3 v3.2.1 h1:RN9w6+7QoMeJVGyfmbcgs28Br8cvmnucEXnY0rYXWg0=
+github.com/Masterminds/semver/v3 v3.2.1/go.mod h1:qvl/7zhW3nngYb5+80sSMF+FG2BjYrf8m9wsX0PNOMQ=
 github.com/atotto/clipboard v0.1.4 h1:EH0zSVneZPSuFR11BlR9YppQTVDbh5+16AmcJi4g1z4=
 github.com/atotto/clipboard v0.1.4/go.mod h1:ZY9tmq7sm5xIbd9bOK4onWV4S6X0u6GY7Vn0Yu86PYI=
 github.com/aymanbagabas/go-osc52/v2 v2.0.1 h1:HwpRHbFMcZLEVr42D4p7XBqjyuxQH5SMiErDT4WkJ2k=
@@ -22,8 +22,6 @@ github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UF
 github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
 github.com/charmbracelet/bubbles v0.21.0 h1:9TdC97SdRVg/1aaXNVWfFH3nnLAwOXr8Fn6u6mfQdFs=
 github.com/charmbracelet/bubbles v0.21.0/go.mod h1:HF+v6QUR4HkEpz62dx7ym2xc71/KBHg+zKwJtMw+qtg=
-github.com/charmbracelet/bubbletea v1.3.5 h1:JAMNLTbqMOhSwoELIr0qyP4VidFq72/6E9j7HHmRKQc=
-github.com/charmbracelet/bubbletea v1.3.5/go.mod h1:TkCnmH+aBd4LrXhXcqrKiYwRs7qyQx5rBgH5fVY3v54=
 github.com/charmbracelet/bubbletea v1.3.6 h1:VkHIxPJQeDt0aFJIsVxw8BQdh/F/L2KKZGsK6et5taU=
 github.com/charmbracelet/bubbletea v1.3.6/go.mod h1:oQD9VCRQFF8KplacJLo28/jofOI2ToOfGYeFgBBxHOc=
 github.com/charmbracelet/colorprofile v0.3.1 h1:k8dTHMd7fgw4bnFd7jXTLZrSU/CQrKnL3m+AxCzDz40=
@@ -44,8 +42,6 @@ github.com/charmbracelet/x/errors v0.0.0-20240508181413-e8d8b6e2de86 h1:JSt3B+U9
 github.com/charmbracelet/x/errors v0.0.0-20240508181413-e8d8b6e2de86/go.mod h1:2P0UgXMEa6TsToMSuFqKFQR+fZTO9CNGUNokkPatT/0=
 github.com/charmbracelet/x/exp/golden v0.0.0-20241011142426-46044092ad91 h1:payRxjMjKgx2PaCWLZ4p3ro9y97+TVLZNaRZgJwSVDQ=
 github.com/charmbracelet/x/exp/golden v0.0.0-20241011142426-46044092ad91/go.mod h1:wDlXFlCrmJ8J+swcL/MnGUuYnqgQdW9rhSD61oNMb6U=
-github.com/charmbracelet/x/exp/strings v0.0.0-20250629123816-066ae234febc h1:XFsX2G2Z1k1p9/52+7TYs2iYW//XCJXSD7xWlEeGvBM=
-github.com/charmbracelet/x/exp/strings v0.0.0-20250629123816-066ae234febc/go.mod h1:Rgw3/F+xlcUc5XygUtimVSxAqCOsqyvJjqF5UHRvc5k=
 github.com/charmbracelet/x/exp/strings v0.0.0-20250725211024-d60e1b0112b2 h1:mI6RFtm+NvDgzRhAL1GEFeOqaJkG+9gBvEnk55uJHKc=
 github.com/charmbracelet/x/exp/strings v0.0.0-20250725211024-d60e1b0112b2/go.mod h1:Rgw3/F+xlcUc5XygUtimVSxAqCOsqyvJjqF5UHRvc5k=
 github.com/charmbracelet/x/term v0.2.1 h1:AQeHeLZ1OqSXhrAWpYUtZyX1T3zVxfpZuEQMIQaGIAQ=
@@ -67,8 +63,8 @@ github.com/dgryski/go-farm v0.0.0-20240924180020-3414d57e47da h1:aIftn67I1fkbMa5
 github.com/dgryski/go-farm v0.0.0-20240924180020-3414d57e47da/go.mod h1:SqUrOPUnsFjfmXRMNPybcSiG0BgUW2AuFH8PAnS2iTw=
 github.com/dlclark/regexp2 v1.11.5 h1:Q/sSnsKerHeCkc/jSTNq1oCm7KiVgUMZRDUoRu0JQZQ=
 github.com/dlclark/regexp2 v1.11.5/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
-github.com/duke-git/lancet/v2 v2.3.6 h1:NKxSSh+dlgp37funvxLCf3xLBeUYa7VW1thYQP6j3Y8=
-github.com/duke-git/lancet/v2 v2.3.6/go.mod h1:zGa2R4xswg6EG9I6WnyubDbFO/+A/RROxIbXcwryTsc=
+github.com/dop251/goja v0.0.0-20250630131328-58d95d85e994 h1:aQYWswi+hRL2zJqGacdCZx32XjKYV8ApXFGntw79XAM=
+github.com/dop251/goja v0.0.0-20250630131328-58d95d85e994/go.mod h1:MxLav0peU43GgvwVgNbLAj1s/bSGboKkhuULvq/7hx4=
 github.com/duke-git/lancet/v2 v2.3.7 h1:nnNBA9KyoqwbPm4nFmEFVIbXeAmpqf6IDCH45+HHHNs=
 github.com/duke-git/lancet/v2 v2.3.7/go.mod h1:zGa2R4xswg6EG9I6WnyubDbFO/+A/RROxIbXcwryTsc=
 github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
@@ -107,8 +103,8 @@ github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
 github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
 github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
 github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
-github.com/go-viper/mapstructure/v2 v2.3.0 h1:27XbWsHIqhbdR5TIC911OfYvgSaW93HM+dX7970Q7jk=
-github.com/go-viper/mapstructure/v2 v2.3.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
+github.com/go-sourcemap/sourcemap v2.1.3+incompatible h1:W1iEw64niKVGogNgBN3ePyLFfuisuzeidWPMPWmECqU=
+github.com/go-sourcemap/sourcemap v2.1.3+incompatible/go.mod h1:F8jJfvm2KbVjc5NqelyYJmf/v5J0dwNLS2mL4sNA1Jg=
 github.com/go-viper/mapstructure/v2 v2.4.0 h1:EBsztssimR/CONLSZZ04E8qAkxNYq4Qp9LvH92wZUgs=
 github.com/go-viper/mapstructure/v2 v2.4.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
 github.com/goccy/go-json v0.10.5 h1:Fq85nIqj+gXn/S5ahsiTlK3TmC85qgirsdTP/+DeaC4=
@@ -133,8 +129,6 @@ github.com/gotd/ige v0.2.2 h1:XQ9dJZwBfDnOGSTxKXBGP4gMud3Qku2ekScRjDWWfEk=
 github.com/gotd/ige v0.2.2/go.mod h1:tuCRb+Y5Y3eNTo3ypIfNpQ4MFjrnONiL2jN2AKZXmb0=
 github.com/gotd/neo v0.1.5 h1:oj0iQfMbGClP8xI59x7fE/uHoTJD7NZH9oV1WNuPukQ=
 github.com/gotd/neo v0.1.5/go.mod h1:9A2a4bn9zL6FADufBdt7tZt+WMhvZoc5gWXihOPoiBQ=
-github.com/gotd/td v0.127.0 h1:81Gs9AM8zgA1PE1/rUHAZtY/aW3aTGbZAAQw/ztKO3E=
-github.com/gotd/td v0.127.0/go.mod h1:QsMlkwf9QmV5Oe+td8ykWHxPPxGU8l7Jb1M5oZ1B73Q=
 github.com/gotd/td v0.129.0 h1:8arlrzBK6qXjMCz1ltBVMCN/Nrc0negTq9mmIQnHyxA=
 github.com/gotd/td v0.129.0/go.mod h1:t9A85Tp/ujnYZwAgBM+hCoVAEagciAZxLBhoDsP7Yno=
 github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
@@ -149,8 +143,6 @@ github.com/jinzhu/now v1.1.5/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/
 github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=
 github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=
 github.com/klauspost/cpuid/v2 v2.0.1/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
-github.com/klauspost/cpuid/v2 v2.2.11 h1:0OwqZRYI2rFrjS4kvkDnqJkKHdHaRnCm68/DY4OxRzU=
-github.com/klauspost/cpuid/v2 v2.2.11/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
 github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
 github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
 github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
@@ -161,8 +153,8 @@ github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
 github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
 github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
 github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
-github.com/krau/gotgproto v0.0.0-20250730080659-caaadb4b1f35 h1:CVzeQNKRgkXm8DQX5bszxI73RcOVGTOFbMynhgjA2lQ=
-github.com/krau/gotgproto v0.0.0-20250730080659-caaadb4b1f35/go.mod h1:xjZlGA8ABRKkfGMmkHKyz520hK6pMfyE8yxpSTqohME=
+github.com/krau/gotgproto v0.0.0-20250815074212-7fbd56c33c00 h1:Evg8e3u5ZuqkqdwzrmiQZrTiFUas00Pw99hQK9PGX7A=
+github.com/krau/gotgproto v0.0.0-20250815074212-7fbd56c33c00/go.mod h1:xjZlGA8ABRKkfGMmkHKyz520hK6pMfyE8yxpSTqohME=
 github.com/lucasb-eyer/go-colorful v1.2.0 h1:1nnpGOrhyZZuNyfu1QjKiUICQ74+3FNCN69Aj6K7nkY=
 github.com/lucasb-eyer/go-colorful v1.2.0/go.mod h1:R4dSotOR9KMtayYi1e77YzuveK+i7ruzyGqttikkLy0=
 github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
@@ -173,14 +165,10 @@ github.com/mattn/go-localereader v0.0.1 h1:ygSAOl7ZXTx4RdPYinUpg6W99U8jWvWi9Ye2J
 github.com/mattn/go-localereader v0.0.1/go.mod h1:8fBrzywKY7BI3czFoHkuzRoWE9C+EiG4R1k4Cjx5p88=
 github.com/mattn/go-runewidth v0.0.16 h1:E5ScNMtiwvlvB5paMFdw9p4kSQzbXFikJ5SQO6TULQc=
 github.com/mattn/go-runewidth v0.0.16/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
-github.com/minio/crc64nvme v1.0.2 h1:6uO1UxGAD+kwqWWp7mBFsi5gAse66C4NXO8cmcVculg=
-github.com/minio/crc64nvme v1.0.2/go.mod h1:eVfm2fAzLlxMdUGc0EEBGSMmPwmXD5XiNRpnu9J3bvg=
 github.com/minio/crc64nvme v1.1.0 h1:e/tAguZ+4cw32D+IO/8GSf5UVr9y+3eJcxZI2WOO/7Q=
 github.com/minio/crc64nvme v1.1.0/go.mod h1:eVfm2fAzLlxMdUGc0EEBGSMmPwmXD5XiNRpnu9J3bvg=
 github.com/minio/md5-simd v1.1.2 h1:Gdi1DZK69+ZVMoNHRXJyNcxrMA4dSxoYHZSQbirFg34=
 github.com/minio/md5-simd v1.1.2/go.mod h1:MzdKDxYpY2BT9XQFocsiZf/NKVtR7nkE4RoEpN+20RM=
-github.com/minio/minio-go/v7 v7.0.94 h1:1ZoksIKPyaSt64AVOyaQvhDOgVC3MfZsWM6mZXRUGtM=
-github.com/minio/minio-go/v7 v7.0.94/go.mod h1:71t2CqDt3ThzESgZUlU1rBN54mksGGlkLcFgguDnnAc=
 github.com/minio/minio-go/v7 v7.0.95 h1:ywOUPg+PebTMTzn9VDsoFJy32ZuARN9zhB+K3IYEvYU=
 github.com/minio/minio-go/v7 v7.0.95/go.mod h1:wOOX3uxS334vImCNRVyIDdXX9OsXDm89ToynKgqUKlo=
 github.com/mitchellh/hashstructure/v2 v2.0.2 h1:vGKWl0YJqUNxE8d+h8f6NJLcCJrgbhC4NcD46KavDd4=
@@ -193,8 +181,6 @@ github.com/muesli/cancelreader v0.2.2 h1:3I4Kt4BQjOR54NavqnDogx/MIoWBFa0StPA8ELU
 github.com/muesli/cancelreader v0.2.2/go.mod h1:3XuTXfFS2VjM+HTLZY9Ak0l6eUKfijIfMUZ4EgX0QYo=
 github.com/muesli/termenv v0.16.0 h1:S5AlUN9dENB57rsbnkPyfdGuWIlkmzJjbFf0Tf5FWUc=
 github.com/muesli/termenv v0.16.0/go.mod h1:ZRfOIKPFDYQoDFF4Olj7/QJbW60Ol/kL1pU3VfY/Cnk=
-github.com/ncruces/go-sqlite3 v0.26.2 h1:5UkIBwdfMN2irpVI1dgi9TjTUlxNI06Rti1C8O7ZKVg=
-github.com/ncruces/go-sqlite3 v0.26.2/go.mod h1:XFTPtFIo1DmGCh+XVP8KGn9b/o2f+z0WZuT09x2N6eo=
 github.com/ncruces/go-sqlite3 v0.27.1 h1:suqlM7xhSyDVMV9RgX99MCPqt9mB6YOCzHZuiI36K34=
 github.com/ncruces/go-sqlite3 v0.27.1/go.mod h1:gpF5s+92aw2MbDmZK0ZOnCdFlpe11BH20CTspVqri0c=
 github.com/ncruces/go-sqlite3/gormlite v0.24.0 h1:81sHeq3CCdhjoqAB650n5wEdRlLO9VBvosArskcN3+c=
@@ -231,14 +217,10 @@ github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7
 github.com/rs/xid v1.6.0 h1:fV591PaemRlL6JfRxGDEPl69wICngIQ3shQtzfy2gxU=
 github.com/rs/xid v1.6.0/go.mod h1:7XoLgs4eV+QndskICGsho+ADou8ySMSjJKDIan90Nz0=
 github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
-github.com/sagikazarmark/locafero v0.9.0 h1:GbgQGNtTrEmddYDSAH9QLRyfAHY12md+8YFTqyMTC9k=
-github.com/sagikazarmark/locafero v0.9.0/go.mod h1:UBUyz37V+EdMS3hDF3QWIiVr/2dPrx49OMO0Bn0hJqk=
 github.com/sagikazarmark/locafero v0.10.0 h1:FM8Cv6j2KqIhM2ZK7HZjm4mpj9NBktLgowT1aN9q5Cc=
 github.com/sagikazarmark/locafero v0.10.0/go.mod h1:Ieo3EUsjifvQu4NZwV5sPd4dwvu0OCgEQV7vjc9yDjw=
 github.com/segmentio/asm v1.2.0 h1:9BQrFxC+YOHJlTlHGkTrFWf59nbL3XnCoFLTwDCI7ys=
 github.com/segmentio/asm v1.2.0/go.mod h1:BqMnlJP91P8d+4ibuonYZw9mfnzI9HfxselHZr5aAcs=
-github.com/sourcegraph/conc v0.3.0 h1:OQTbbt6P72L20UqAkXXuLOj79LfEanQ+YQFNpLA9ySo=
-github.com/sourcegraph/conc v0.3.0/go.mod h1:Sdozi7LEKbFPqYX2/J+iBAM6HpqSLTASQIKqDmF7Mt0=
 github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8 h1:+jumHNA0Wrelhe64i8F6HNlS8pkoyMv5sreGx2Ry5Rw=
 github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8/go.mod h1:3n1Cwaq1E1/1lhQhtRK2ts/ZwZEhjcQeJQ1RuC6Q/8U=
 github.com/spf13/afero v1.14.0 h1:9tH6MapGnn/j0eb0yIXiLjERO8RB6xIVZRDCX7PtqWA=
@@ -247,7 +229,6 @@ github.com/spf13/cast v1.9.2 h1:SsGfm7M8QOFtEzumm7UZrZdLLquNdzFYfIbEXntcFbE=
 github.com/spf13/cast v1.9.2/go.mod h1:jNfB8QC9IA6ZuY2ZjDp0KtFO2LZZlg4S/7bzP6qqeHo=
 github.com/spf13/cobra v1.9.1 h1:CXSaggrXdbHK9CF+8ywj8Amf7PBRmPCOJugH954Nnlo=
 github.com/spf13/cobra v1.9.1/go.mod h1:nDyEzZ8ogv936Cinf6g1RU9MRY64Ir93oCnqb9wxYW0=
-github.com/spf13/pflag v1.0.6 h1:jFzHGLGAlb3ruxLB8MhbI6A8+AQX/2eW4qeyNZXNp2o=
 github.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
 github.com/spf13/pflag v1.0.7 h1:vN6T9TfwStFPFM5XzjsvmzZkLuaLX+HS+0SeFLRgU6M=
 github.com/spf13/pflag v1.0.7/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
@@ -286,24 +267,16 @@ go.uber.org/zap v1.27.0 h1:aJMhYGrd5QSmlpLMr2MftRKl7t8J8PTZPA732ud/XR8=
 go.uber.org/zap v1.27.0/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=
 golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
 golang.org/x/crypto v0.0.0-20201221181555-eec23a3978ad/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
-golang.org/x/crypto v0.39.0 h1:SHs+kF4LP+f+p14esP5jAoDpHU8Gu/v9lFRK6IT5imM=
-golang.org/x/crypto v0.39.0/go.mod h1:L+Xg3Wf6HoL4Bn4238Z6ft6KfEpN0tJGo53AAPC632U=
 golang.org/x/crypto v0.40.0 h1:r4x+VvoG5Fm+eJcxMaY8CQM7Lb0l1lsmjGBQ6s8BfKM=
 golang.org/x/crypto v0.40.0/go.mod h1:Qr1vMER5WyS2dfPHAlsOj01wgLbsyWtFn/aY+5+ZdxY=
-golang.org/x/exp v0.0.0-20250620022241-b7579e27df2b h1:M2rDM6z3Fhozi9O7NWsxAkg/yqS/lQJ6PmkyIV3YP+o=
-golang.org/x/exp v0.0.0-20250620022241-b7579e27df2b/go.mod h1:3//PLf8L/X+8b4vuAfHzxeRUl04Adcb341+IGKfnqS8=
 golang.org/x/exp v0.0.0-20250718183923-645b1fa84792 h1:R9PFI6EUdfVKgwKjZef7QIwGcBKu86OEFpJ9nUEP2l4=
 golang.org/x/exp v0.0.0-20250718183923-645b1fa84792/go.mod h1:A+z0yzpGtvnG90cToK5n2tu8UJVP2XUATh+r+sfOOOc=
-golang.org/x/mod v0.25.0 h1:n7a+ZbQKQA/Ysbyb0/6IbB1H/X41mKgbhfv7AfG/44w=
-golang.org/x/mod v0.25.0/go.mod h1:IXM97Txy2VM4PJ3gI61r1YEk/gAj6zAHN3AdZt6S9Ww=
 golang.org/x/mod v0.26.0 h1:EGMPT//Ezu+ylkCijjPc+f4Aih7sZvaAr+O3EHBxvZg=
 golang.org/x/mod v0.26.0/go.mod h1:/j6NAhSk8iQ723BGAUyoAcn7SlD7s15Dp9Nd/SfeaFQ=
 golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
 golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
-golang.org/x/net v0.41.0 h1:vBTly1HeNPEn3wtREYfy4GZ/NECgw2Cnl+nK6Nz3uvw=
-golang.org/x/net v0.41.0/go.mod h1:B/K4NNqkfmg07DQYrbwvSluqCJOOXwUjeb/5lOisjbA=
 golang.org/x/net v0.42.0 h1:jzkYrhi3YQWD6MLBJcsklgQsoAcw89EcZbJw8Z614hs=
 golang.org/x/net v0.42.0/go.mod h1:FF1RA5d3u7nAYA4z2TkclSCKh68eSXtiFwcWQpPXdt8=
 golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
@@ -312,8 +285,6 @@ golang.org/x/oauth2 v0.30.0 h1:dnDm7JmhM45NNpd8FDDeLhK6FwqbOf4MLCM9zb1BOHI=
 golang.org/x/oauth2 v0.30.0/go.mod h1:B++QgG3ZKulg6sRPGD/mqlHQs5rB3Ml9erfeDY7xKlU=
 golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.15.0 h1:KWH3jNZsfyT6xfAfKiz6MRNmd46ByHDYaZ7KSkCtdW8=
-golang.org/x/sync v0.15.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
 golang.org/x/sync v0.16.0 h1:ycBJEhp9p4vXvUZNszeOq0kGTPghopOL8q0fq3vstxw=
 golang.org/x/sync v0.16.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
 golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
@@ -321,22 +292,16 @@ golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5h
 golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210809222454-d867a43fc93e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.33.0 h1:q3i8TbbEz+JRD9ywIRlyRAQbM0qF7hu24q3teo2hbuw=
-golang.org/x/sys v0.33.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
 golang.org/x/sys v0.34.0 h1:H5Y5sJ2L2JRdyv7ROF1he/lPdvFsd0mJHFw2ThKHxLA=
 golang.org/x/sys v0.34.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
 golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
 golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
-golang.org/x/text v0.26.0 h1:P42AVeLghgTYr4+xUnTRKDMqpar+PtX7KWuNQL21L8M=
-golang.org/x/text v0.26.0/go.mod h1:QK15LZJUUQVJxhz7wXgxSy/CJaTFjd0G+YLonydOVQA=
 golang.org/x/text v0.27.0 h1:4fGWRpyh641NLlecmyl4LOe6yDdfaYNrGb2zdfo4JV4=
 golang.org/x/text v0.27.0/go.mod h1:1D28KMCvyooCX9hBiosv5Tz/+YLxj0j7XhWjpSUF7CU=
 golang.org/x/time v0.12.0 h1:ScB/8o8olJvc+CQPWrK3fPZNfh7qgwCrY0zJmoEQLSE=
 golang.org/x/time v0.12.0/go.mod h1:CDIdPxbZBQxdj6cxyCIdrNogrJKMJ7pr37NYpMcMDSg=
 golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
-golang.org/x/tools v0.34.0 h1:qIpSLOxeCYGg9TrcJokLBG4KFA6d795g0xkBkiESGlo=
-golang.org/x/tools v0.34.0/go.mod h1:pAP9OwEaY1CAW3HOmg3hLZC5Z0CCmzjAF2UQMSqNARg=
 golang.org/x/tools v0.35.0 h1:mBffYraMEf7aa0sB+NuKnuCy8qI/9Bughn8dC2Gu5r0=
 golang.org/x/tools v0.35.0/go.mod h1:NKdj5HkL/73byiZSJjqJgKn3ep7KjFkBOkR/Hps3VPw=
 golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
@@ -353,23 +318,18 @@ gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
 gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
 gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
 gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
-gorm.io/gorm v1.30.0 h1:qbT5aPv1UH8gI99OsRlvDToLxW5zR7FzS9acZDOZcgs=
-gorm.io/gorm v1.30.0/go.mod h1:8Z33v652h4//uMA76KjeDH8mJXPm1QNCYrMeatR0DOE=
 gorm.io/gorm v1.30.1 h1:lSHg33jJTBxs2mgJRfRZeLDG+WZaHYCk3Wtfl6Ngzo4=
 gorm.io/gorm v1.30.1/go.mod h1:8Z33v652h4//uMA76KjeDH8mJXPm1QNCYrMeatR0DOE=
-modernc.org/cc/v4 v4.26.1 h1:+X5NtzVBn0KgsBCBe+xkDC7twLb/jNVj9FPgiwSQO3s=
-modernc.org/cc/v4 v4.26.1/go.mod h1:uVtb5OGqUKpoLWhqwNQo/8LwvoiEBLvZXIQ/SmO6mL0=
 modernc.org/cc/v4 v4.26.3 h1:yEN8dzrkRFnn4PUUKXLYIqVf2PJYAEjMTFjO3BDGc3I=
+modernc.org/cc/v4 v4.26.3/go.mod h1:uVtb5OGqUKpoLWhqwNQo/8LwvoiEBLvZXIQ/SmO6mL0=
 modernc.org/ccgo/v4 v4.28.0 h1:rjznn6WWehKq7dG4JtLRKxb52Ecv8OUGah8+Z/SfpNU=
 modernc.org/ccgo/v4 v4.28.0/go.mod h1:JygV3+9AV6SmPhDasu4JgquwU81XAKLd3OKTUDNOiKE=
-modernc.org/fileutil v1.3.3 h1:3qaU+7f7xxTUmvU1pJTZiDLAIoJVdUSSauJNHg9yXoA=
-modernc.org/fileutil v1.3.3/go.mod h1:HxmghZSZVAz/LXcMNwZPA/DRrQZEVP9VX0V4LQGQFOc=
+modernc.org/fileutil v1.3.8 h1:qtzNm7ED75pd1C7WgAGcK4edm4fvhtBsEiI/0NQ54YM=
+modernc.org/fileutil v1.3.8/go.mod h1:HxmghZSZVAz/LXcMNwZPA/DRrQZEVP9VX0V4LQGQFOc=
 modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI=
 modernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito=
-modernc.org/goabi0 v0.0.3 h1:y81b9r3asCh6Xtse6Nz85aYGB0cG3M3U6222yap1KWI=
-modernc.org/goabi0 v0.0.3/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
-modernc.org/libc v1.66.1 h1:4uQsntXbVyAgrV+j6NhKvDiUypoJL48BWQx6sy9y8ok=
-modernc.org/libc v1.66.1/go.mod h1:AiZxInURfEJx516LqEaFcrC+X38rt9G7+8ojIXQKHbo=
+modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=
+modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
 modernc.org/libc v1.66.6 h1:RyQpwAhM/19nXD8y3iejM/AjmKwY2TjxZTlUWTsWw2U=
 modernc.org/libc v1.66.6/go.mod h1:j8z0EYAuumoMQ3+cWXtmw6m+LYn3qm8dcZDFtFTSq+M=
 modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
@@ -380,8 +340,6 @@ modernc.org/opt v0.1.4 h1:2kNGMRiUjrp4LcaPuLY2PzUfqM/w9N23quVwhKt5Qm8=
 modernc.org/opt v0.1.4/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
 modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
 modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
-modernc.org/sqlite v1.38.0 h1:+4OrfPQ8pxHKuWG4md1JpR/EYAh3Md7TdejuuzE7EUI=
-modernc.org/sqlite v1.38.0/go.mod h1:1Bj+yES4SVvBZ4cBOpVZ6QgesMCKpJZDq0nxYzOpmNE=
 modernc.org/sqlite v1.38.2 h1:Aclu7+tgjgcQVShZqim41Bbw9Cho0y/7WzYptXqkEek=
 modernc.org/sqlite v1.38.2/go.mod h1:cPTJYSlgg3Sfg046yBShXENNtPrWrDX8bsbAQBzgQ5E=
 modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=

parsers/js.go

@@ -0,0 +1,130 @@
+package parsers
+
+import (
+	"context"
+	"encoding/json"
+	"fmt"
+	"os"
+	"path/filepath"
+
+	"github.com/charmbracelet/log"
+	"github.com/dop251/goja"
+	"github.com/krau/SaveAny-Bot/pkg/parser"
+)
+
+type jsParser struct {
+	meta  PluginMeta
+	vm    *goja.Runtime
+	reqCh chan jsParserReq
+}
+
+type jsParserReq struct {
+	method ParserMethod
+	url    string
+	respCh chan jsParserResp
+}
+
+type jsParserResp struct {
+	item *parser.Item
+	ok   bool
+	err  error
+}
+
+func (p *jsParser) CanHandle(url string) bool {
+	respCh := make(chan jsParserResp, 1)
+	p.reqCh <- jsParserReq{method: ParserMethodCanHandle, url: url, respCh: respCh}
+	resp := <-respCh
+	return resp.ok && resp.err == nil
+}
+
+func (p *jsParser) Parse(ctx context.Context, url string) (*parser.Item, error) {
+	respCh := make(chan jsParserResp, 1)
+	p.reqCh <- jsParserReq{method: ParserMethodParse, url: url, respCh: respCh}
+	select {
+	case resp := <-respCh:
+		return resp.item, resp.err
+	case <-ctx.Done():
+		return nil, ctx.Err()
+	}
+}
+
+func newJSParser(vm *goja.Runtime, canHandleFunc, parseFunc goja.Value, metadata PluginMeta) *jsParser {
+	p := &jsParser{
+		vm:    vm,
+		reqCh: make(chan jsParserReq, 10),
+		meta:  metadata,
+	}
+
+	go func() {
+		for req := range p.reqCh {
+			switch req.method {
+			case ParserMethodCanHandle:
+				fn, _ := goja.AssertFunction(canHandleFunc)
+				res, err := fn(goja.Undefined(), p.vm.ToValue(req.url))
+				if err != nil {
+					req.respCh <- jsParserResp{ok: false, err: err}
+					continue
+				}
+				req.respCh <- jsParserResp{ok: res.ToBoolean()}
+			case ParserMethodParse:
+				fn, _ := goja.AssertFunction(parseFunc)
+				result, err := fn(goja.Undefined(), p.vm.ToValue(req.url))
+				if err != nil {
+					req.respCh <- jsParserResp{err: err}
+					continue
+				}
+
+				var item parser.Item
+				if exported := result.Export(); exported != nil {
+					data, err := json.Marshal(exported)
+					if err != nil {
+						req.respCh <- jsParserResp{err: fmt.Errorf("failed to marshal result to JSON: %w", err)}
+						continue
+					}
+
+					if err := json.Unmarshal(data, &item); err != nil {
+						req.respCh <- jsParserResp{err: fmt.Errorf("failed to unmarshal JSON to Item: %w", err)}
+						continue
+					}
+				} else {
+					req.respCh <- jsParserResp{err: fmt.Errorf("JS function returned null or undefined")}
+					continue
+				}
+				req.respCh <- jsParserResp{item: &item}
+			}
+		}
+	}()
+
+	return p
+}
+
+func LoadPlugins(ctx context.Context, dir string) error {
+	entries, err := os.ReadDir(dir)
+	if err != nil {
+		return err
+	}
+
+	for _, e := range entries {
+		if filepath.Ext(e.Name()) != ".js" {
+			continue
+		}
+		scriptPath := filepath.Join(dir, e.Name())
+		code, err := os.ReadFile(scriptPath)
+		if err != nil {
+			return err
+		}
+
+		vm := goja.New()
+		vm.Set("registerParser", jsRegisterParser(vm))
+		// Inject some utils to vm
+		logger := log.FromContext(ctx).WithPrefix(fmt.Sprintf("[plugin|parser]/%s", e.Name()))
+		vm.Set("console", jsConsole(logger))
+		// http fetch funcs
+		vm.Set("ghttp", jsGhttp(vm))
+		
+		if _, err := vm.RunString(string(code)); err != nil {
+			return fmt.Errorf("error loading plugin %s: %w", e.Name(), err)
+		}
+	}
+	return nil
+}

parsers/js_api.go

@@ -0,0 +1,151 @@
+package parsers
+
+import (
+	"encoding/json"
+	"fmt"
+	"io"
+	"net/http"
+
+	"github.com/blang/semver"
+	"github.com/charmbracelet/log"
+	"github.com/dop251/goja"
+	"github.com/krau/SaveAny-Bot/common/utils/netutil"
+)
+
+func jsRegisterParser(vm *goja.Runtime) func(call goja.FunctionCall) goja.Value {
+	return func(call goja.FunctionCall) goja.Value {
+		jsObj := call.Argument(0)
+		if jsObj == nil || goja.IsUndefined(jsObj) || goja.IsNull(jsObj) {
+			panic("registerParser expects an object { canHandle, parse }")
+		}
+
+		obj := jsObj.ToObject(vm)
+		if obj == nil {
+			panic("registerParser: cannot convert argument to object")
+		}
+		metaValue := obj.Get("metadata")
+		if metaValue == nil || goja.IsUndefined(metaValue) {
+			panic("parser must provide metadata")
+		}
+		var metadata PluginMeta
+		if exported := metaValue.Export(); exported != nil {
+			data, err := json.Marshal(exported)
+			if err != nil {
+				panic(fmt.Sprintf("failed to marshal metadata to JSON: %v", err))
+			}
+			if err := json.Unmarshal(data, &metadata); err != nil {
+				panic(fmt.Sprintf("failed to unmarshal JSON to PluginMeta: %v", err))
+			}
+		} else {
+			panic("metadata cannot be null or undefined")
+		}
+
+		pluginV := semver.MustParse(metadata.Version)
+		if pluginV.LT(MinimumParserVersion) || pluginV.GT(LatestParserVersion) {
+			panic(fmt.Sprintf("parser version %s is not supported, must be between %s and %s", metadata.Version, MinimumParserVersion, LatestParserVersion))
+		}
+
+		handleFn := obj.Get("canHandle")
+		parseFn := obj.Get("parse")
+		if parseFn == nil || goja.IsUndefined(parseFn) {
+			panic("parser must provide a parse function")
+		}
+
+		parsers = append(parsers, newJSParser(vm, handleFn, parseFn, metadata))
+		return goja.Undefined()
+	}
+}
+
+var jsConsole = func(logger *log.Logger) map[string]any {
+	return map[string]any{
+		"log": func(args ...any) {
+			if len(args) == 0 {
+				return
+			}
+			if len(args) > 1 {
+				logger.Info(args[0], args[1:]...)
+			} else {
+				logger.Info(args[0])
+			}
+		},
+	}
+}
+
+var jsGhttp = func(vm *goja.Runtime) *goja.Object {
+	ghttp := vm.NewObject()
+	client := netutil.DefaultParserHTTPClient()
+	ghttp.Set("get", func(call goja.FunctionCall) goja.Value {
+		url := call.Argument(0).String()
+		resp, err := client.Get(url)
+		if err != nil {
+			return vm.ToValue(map[string]any{
+				"error": fmt.Sprintf("failed to fetch %s: %v", url, err),
+			})
+		}
+		defer resp.Body.Close()
+		if resp.StatusCode != http.StatusOK {
+			return vm.ToValue(map[string]any{
+				"error":  fmt.Sprintf("failed to fetch %s: %s", url, resp.Status),
+				"status": resp.StatusCode,
+			})
+		}
+		body, err := io.ReadAll(resp.Body)
+		if err != nil {
+			return vm.ToValue(map[string]any{
+				"error": fmt.Errorf("failed to read response body: %w", err).Error(),
+			})
+		}
+		return vm.ToValue(string(body))
+	})
+	ghttp.Set("getJSON", func(call goja.FunctionCall) goja.Value {
+		url := call.Argument(0).String()
+
+		resp, err := client.Get(url)
+		if err != nil {
+			return vm.ToValue(map[string]any{
+				"error": fmt.Sprintf("failed to fetch %s: %v", url, err),
+			})
+		}
+		defer resp.Body.Close()
+		if resp.StatusCode != http.StatusOK {
+			return vm.ToValue(map[string]any{
+				"error":  fmt.Sprintf("failed to fetch %s: %s", url, resp.Status),
+				"status": resp.StatusCode,
+			})
+		}
+		body, err := io.ReadAll(resp.Body)
+		if err != nil {
+			return vm.ToValue(map[string]any{
+				"error": fmt.Errorf("failed to read response body: %w", err).Error(),
+			})
+		}
+		var jsonData map[string]any
+		if err := json.Unmarshal(body, &jsonData); err != nil {
+			return vm.ToValue(map[string]any{
+				"error": fmt.Errorf("failed to unmarshal JSON: %w", err).Error(),
+			})
+		}
+		return vm.ToValue(map[string]any{
+			"data": jsonData,
+		})
+	})
+	ghttp.Set("head", func(call goja.FunctionCall) goja.Value {
+		url := call.Argument(0).String()
+		resp, err := client.Head(url)
+		if err != nil {
+			return vm.ToValue(map[string]any{
+				"error": fmt.Sprintf("failed to fetch %s: %v", url, err),
+			})
+		}
+		defer resp.Body.Close()
+		headers := make(map[string]string)
+		for k, v := range resp.Header {
+			headers[k] = v[0]
+		}
+		return vm.ToValue(map[string]any{
+			"status":  resp.StatusCode,
+			"headers": headers,
+		})
+	})
+	return ghttp
+}

parsers/kemono/download.go

@@ -0,0 +1,36 @@
+package kemono
+
+import (
+	"net/url"
+	"strings"
+)
+
+type DownloadInfo struct {
+	ServiceName string
+	UserID      string
+	PostID      string
+}
+
+func extractDownloadInfoFromURL(u string) *DownloadInfo {
+	if !strings.HasPrefix(u, "http://") && !strings.HasPrefix(u, "https://") {
+		u = "https://" + u
+	}
+	url, err := url.Parse(u)
+	if err != nil {
+		return nil
+	}
+	parts := strings.Split(strings.Trim(url.Path, "/"), "/")
+	if len(parts) == 3 {
+		return &DownloadInfo{
+			ServiceName: parts[0],
+			UserID:      parts[2],
+		}
+	} else if len(parts) == 5 && parts[3] == "post" {
+		return &DownloadInfo{
+			ServiceName: parts[0],
+			UserID:      parts[2],
+			PostID:      parts[4],
+		}
+	}
+	return nil
+}

parsers/kemono/kemono.go

@@ -0,0 +1,173 @@
+package kemono
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"net/http"
+	"net/url"
+	"path"
+	"strings"
+
+	"github.com/duke-git/lancet/v2/strutil"
+	"github.com/krau/SaveAny-Bot/common/utils/netutil"
+	"github.com/krau/SaveAny-Bot/pkg/parser"
+)
+
+type KemonoParser struct{}
+
+var (
+	kemonoDomains = []string{
+		"kemono.su",
+		"kemono.cr",
+	}
+	ErrFailedToExtractInfo = errors.New("failed to extract download info from URL")
+)
+
+const (
+	kemonoApiBase = "https://kemono.cr/api/v1"
+)
+
+func (k *KemonoParser) CanHandle(text string) bool {
+	text = strings.TrimPrefix(text, "https://")
+	text = strings.TrimPrefix(text, "http://")
+
+	var matchesDomain bool
+	for _, domain := range kemonoDomains {
+		if strings.Contains(text, domain) {
+			matchesDomain = true
+			break
+		}
+	}
+	if !matchesDomain {
+		return false
+	}
+
+	var path string
+	for _, domain := range kemonoDomains {
+		if idx := strings.Index(text, domain); idx != -1 {
+			remaining := text[idx+len(domain):]
+			if len(remaining) > 0 && remaining[0] == '/' {
+				path = remaining[1:]
+			}
+			break
+		}
+	}
+
+	if path == "" {
+		return false
+	}
+
+	parts := strings.Split(path, "/")
+	// servicename/user/id (user profile page)
+	// servicename/user/id/post/id (post page)
+	return len(parts) == 3 || (len(parts) == 5 && parts[3] == "post")
+}
+
+func (k *KemonoParser) Parse(ctx context.Context, u string) (*parser.Item, error) {
+	info := extractDownloadInfoFromURL(u)
+	if info == nil {
+		return nil, ErrFailedToExtractInfo
+	}
+	if info.PostID != "" {
+		return k.parseOne(ctx, info)
+	}
+	return k.parseUserPage(ctx, info)
+}
+
+func (k *KemonoParser) parseOne(ctx context.Context, info *DownloadInfo) (*parser.Item, error) {
+	client := netutil.DefaultParserHTTPClient()
+	endpoint := fmt.Sprintf("%s/%s/user/%s/post/%s", kemonoApiBase, info.ServiceName, info.UserID, info.PostID)
+	req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, nil)
+	if err != nil {
+		return nil, fmt.Errorf("failed to create request to Kemono API: %w", err)
+	}
+	req.Header.Set("Accept", "text/css")
+	resp, err := client.Do(req)
+	if err != nil {
+		return nil, fmt.Errorf("failed to fetch Kemono API: %w", err)
+	}
+	defer resp.Body.Close()
+	if resp.StatusCode != http.StatusOK {
+		return nil, fmt.Errorf("failed to fetch Kemono API, status code: %d", resp.StatusCode)
+	}
+	var postInfo PostInfo
+	if err := json.NewDecoder(resp.Body).Decode(&postInfo); err != nil {
+		return nil, fmt.Errorf("failed to decode Kemono API response: %w", err)
+	}
+	item := &parser.Item{
+		Site:        "kemono",
+		Title:       postInfo.Post.Title,
+		URL:         fmt.Sprintf("https://kemono.cr/%s/user/%s/post/%s", info.ServiceName, info.UserID, info.PostID),
+		Author:      postInfo.Post.User, // [TODO] request user profile
+		Description: postInfo.Post.Content,
+		Tags: func() []string {
+			if postInfo.Post.Tags != nil {
+				return *postInfo.Post.Tags
+			}
+			return nil
+		}(),
+	}
+	resources := make([]parser.Resource, 0)
+	for _, attachment := range postInfo.Attachments {
+		if attachment.Server == nil || attachment.Path == nil || attachment.Name == nil {
+			continue
+		}
+		var size int64
+		fileUrl := fmt.Sprintf("%s/data%s", *attachment.Server, *attachment.Path)
+		headReq, err := http.NewRequestWithContext(ctx, http.MethodHead, fileUrl, nil)
+		if err == nil {
+			resp, err := client.Do(headReq)
+			if err == nil {
+				size = resp.ContentLength
+				resp.Body.Close()
+			}
+		}
+		resources = append(resources, parser.Resource{
+			URL:      fmt.Sprintf("%s/data%s", *attachment.Server, *attachment.Path),
+			Filename: *attachment.Name,
+			Size:     size,
+		})
+	}
+	picCdnMap := make(map[string]string)
+	for _, preview := range postInfo.Previews {
+		if preview.Type == nil || *preview.Type != "thumbnail" {
+			continue
+		}
+		picCdnMap[*preview.Path] = *preview.Server
+	}
+	for _, attachment := range postInfo.Post.Attachments {
+		if !isImageExt(*attachment.Path) {
+			continue
+		}
+		picUrl, err := url.JoinPath(picCdnMap[*attachment.Path], "data", *attachment.Path)
+		if err != nil {
+			continue
+		}
+		var size int64
+		headReq, err := http.NewRequestWithContext(ctx, http.MethodHead, picUrl, nil)
+		if err == nil {
+			resp, err := client.Do(headReq)
+			if err == nil {
+				size = resp.ContentLength
+				resp.Body.Close()
+			}
+		}
+		resources = append(resources, parser.Resource{
+			URL:      picUrl,
+			Filename: *attachment.Name,
+			Size:     size,
+		})
+	}
+	item.Resources = resources
+	return item, nil
+}
+
+func (k *KemonoParser) parseUserPage(_ context.Context, _ *DownloadInfo) (*parser.Item, error) {
+	return nil, errors.New("kemono user page not implemented")
+}
+
+func isImageExt(attachmentPath string) bool {
+	return strutil.HasSuffixAny(path.Ext(strings.Split(attachmentPath, "?")[0]), []string{".jpg", ".jpeg", ".png", ".webp"})
+}

parsers/kemono/post_info.go

@@ -0,0 +1,62 @@
+// https://github.com/kemono-rs/kemono
+
+package kemono
+
+type PostInfo struct {
+	Post        Post             `json:"post"`
+	Attachments []AttachmentLike `json:"attachments"`
+	Previews    []AttachmentLike `json:"previews"`
+}
+
+type AttachmentLike struct {
+	Type   *string `json:"type,omitempty"`
+	Server *string `json:"server,omitempty"`
+	Name   *string `json:"name,omitempty"`
+	Path   *string `json:"path,omitempty"`
+}
+
+type Post struct {
+	ID          string           `json:"id"`
+	User        string           `json:"user"`
+	Service     string           `json:"service"`
+	Title       string           `json:"title"`
+	Content     string           `json:"content"`
+	Embed       Embed            `json:"embed"`
+	SharedFile  bool             `json:"shared_file"`
+	Added       *string          `json:"added,omitempty"`
+	Published   string           `json:"published"`
+	Edited      *string          `json:"edited,omitempty"`
+	File        File             `json:"file"`
+	Attachments []AttachmentLike `json:"attachments"`
+	Poll        *Poll            `json:"poll,omitempty"`
+	Captions    *string          `json:"captions,omitempty"`
+	Tags        *[]string        `json:"tags,omitempty"`
+	Next        *string          `json:"next,omitempty"`
+	Prev        *string          `json:"prev,omitempty"`
+}
+
+type File struct {
+	Name *string `json:"name,omitempty"`
+	Path *string `json:"path,omitempty"`
+}
+
+type Embed struct {
+	URL         *string `json:"url,omitempty"`
+	Subject     *string `json:"subject,omitempty"`
+	Description *string `json:"description,omitempty"`
+}
+
+type Poll struct {
+	Title          string   `json:"title"`
+	Choices        []Choice `json:"choices"`
+	ClosesAt       *string  `json:"closes_at,omitempty"`
+	CreatedAt      string   `json:"created_at"`
+	Description    *string  `json:"description,omitempty"`
+	AllowsMultiple bool     `json:"allows_multiple"`
+	TotalVotes     int64    `json:"total_votes"`
+}
+
+type Choice struct {
+	Text  string `json:"text"`
+	Votes int64  `json:"votes"`
+}

parsers/kemono/post_legacy.go

@@ -0,0 +1,16 @@
+package kemono
+
+type PostLegacy struct {
+	Props   Props    `json:"props"`
+	Results []Result `json:"results"`
+}
+
+type Props struct {
+	Count uint `json:"count"`
+	Limit uint `json:"limit"`
+}
+
+type Result struct {
+	ID    string `json:"id"`
+	Title string `json:"title"`
+}

parsers/kemono/user_profile.go

@@ -0,0 +1,8 @@
+package kemono
+
+type UserProfile struct {
+	ID       string  `json:"id"`
+	Name     string  `json:"name"`
+	Service  string  `json:"service"`
+	PublicID *string `json:"public_id,omitempty"`
+}

parsers/parser.go

@@ -0,0 +1,86 @@
+package parsers
+
+import (
+	"context"
+	"fmt"
+	"sync"
+
+	"github.com/krau/SaveAny-Bot/config"
+	"github.com/krau/SaveAny-Bot/parsers/kemono"
+	"github.com/krau/SaveAny-Bot/parsers/twitter"
+	"github.com/krau/SaveAny-Bot/pkg/parser"
+)
+
+var (
+	parsers       []parser.Parser
+	parsersMu     sync.Mutex
+	doConfig      sync.Once
+	configParsers = func() {
+		if len(parsers) == 0 {
+			return
+		}
+		for _, pser := range parsers {
+			if configurable, ok := pser.(parser.ConfigurableParser); ok {
+				cfg := config.C().GetParserConfigByName(configurable.Name())
+				if err := configurable.Configure(cfg); err != nil {
+					fmt.Printf("Error configuring parser %s: %v\n", configurable.Name(), err)
+				}
+			}
+		}
+	}
+)
+
+func AddParser(p ...parser.Parser) {
+	parsersMu.Lock()
+	defer parsersMu.Unlock()
+	parsers = append(parsers, p...)
+}
+
+func init() {
+	AddParser(new(twitter.TwitterParser), new(kemono.KemonoParser))
+}
+
+var (
+	ErrNoParserFound = fmt.Errorf("no parser found for the given URL")
+)
+
+func ParseWithContext(ctx context.Context, url string) (*parser.Item, error) {
+	doConfig.Do(configParsers)
+	ch := make(chan *parser.Item, 1)
+	errCh := make(chan error, 1)
+
+	go func() {
+		for _, pser := range parsers {
+			if !pser.CanHandle(url) {
+				continue
+			}
+			item, err := pser.Parse(ctx, url)
+			if err != nil {
+				errCh <- err
+				return
+			}
+			ch <- item
+			return
+		}
+		errCh <- ErrNoParserFound
+	}()
+
+	select {
+	case item := <-ch:
+		return item, nil
+	case err := <-errCh:
+		return nil, err
+	case <-ctx.Done():
+		return nil, ctx.Err()
+	}
+}
+
+func CanHandle(url string) (bool, parser.Parser) {
+	doConfig.Do(configParsers)
+	for _, pser := range parsers {
+		if pser.CanHandle(url) {
+			return true, pser
+		}
+	}
+	return false, nil
+}

parsers/plugin.go

@@ -0,0 +1,23 @@
+package parsers
+
+import "github.com/blang/semver"
+
+var (
+	LatestParserVersion  = semver.MustParse("1.0.0")
+	MinimumParserVersion = semver.MustParse("1.0.0")
+)
+
+type PluginMeta struct {
+	Name        string `json:"name"`
+	Version     string `json:"version"` // [TODO] 分版本解析, 但是我们现在只有 v1 所以先不写
+	Description string `json:"description"`
+	Author      string `json:"author"`
+}
+
+type ParserMethod uint
+
+const (
+	_ ParserMethod = iota
+	ParserMethodCanHandle
+	ParserMethodParse
+)

parsers/twitter/parser.go

@@ -0,0 +1,122 @@
+package twitter
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"net/http"
+	"path"
+	"regexp"
+	"strings"
+
+	"github.com/krau/SaveAny-Bot/common/utils/netutil"
+	"github.com/krau/SaveAny-Bot/pkg/parser"
+)
+
+type TwitterParser struct {
+	client    http.Client
+	apiDomain string
+}
+
+const (
+	fxTwitterApi = "api.fxtwitter.com"
+)
+
+var _ parser.ConfigurableParser = (*TwitterParser)(nil)
+
+var (
+	twitterSourceURLRegexp *regexp.Regexp = regexp.MustCompile(`(?:twitter|x)\.com/([^/]+)/status/(\d+)`)
+)
+
+func getTweetID(sourceURL string) string {
+	matches := twitterSourceURLRegexp.FindStringSubmatch(sourceURL)
+	if len(matches) < 3 {
+		return ""
+	}
+	return matches[2]
+}
+
+func (p *TwitterParser) Parse(ctx context.Context, u string) (*parser.Item, error) {
+	id := getTweetID(u)
+	if id == "" {
+		return nil, errors.New("invalid Twitter URL")
+	}
+	apiUrl := fmt.Sprintf("https://%s/_/status/%s", p.apiDomain, id)
+	req, err := http.NewRequestWithContext(ctx, http.MethodGet, apiUrl, nil)
+	if err != nil {
+		return nil, fmt.Errorf("failed to create request to Twitter API: %w", err)
+	}
+	resp, err := p.client.Do(req)
+	if err != nil {
+		return nil, fmt.Errorf("failed to fetch Twitter API: %w", err)
+	}
+	defer resp.Body.Close()
+	if resp.StatusCode != http.StatusOK {
+		return nil, fmt.Errorf("failed to fetch Twitter API, status code: %d", resp.StatusCode)
+	}
+	var fxResp FxTwitterApiResp
+	if err := json.NewDecoder(resp.Body).Decode(&fxResp); err != nil {
+		return nil, fmt.Errorf("failed to decode Twitter API response: %w", err)
+	}
+	if fxResp.Code != 200 {
+		return nil, fmt.Errorf("request twitter API error: %s", fxResp.Message)
+	}
+	if len(fxResp.Tweet.Media.All) == 0 {
+		return nil, errors.New("no media found in the tweet")
+	}
+	resources := make([]parser.Resource, 0, len(fxResp.Tweet.Media.All))
+	for _, media := range fxResp.Tweet.Media.All {
+		var size int64
+		resp, err := p.client.Head(media.URL)
+		if err == nil {
+			size = resp.ContentLength
+			resp.Body.Close()
+		}
+		resources = append(resources, parser.Resource{
+			URL:      media.URL,
+			Filename: path.Base(strings.Split(media.URL, "?")[0]),
+			Size:     size,
+		})
+	}
+	item := &parser.Item{
+		Site:        "Twitter",
+		Title:       fmt.Sprintf("Tweet/%s", id),
+		URL:         fxResp.Tweet.URL,
+		Description: fxResp.Tweet.Text,
+		Author:      fxResp.Tweet.Author.Name,
+		Tags:        make([]string, 0),
+		Extra:       make(map[string]any),
+		Resources:   resources,
+	}
+	return item, nil
+}
+
+func (p *TwitterParser) CanHandle(u string) bool {
+	return twitterSourceURLRegexp.MatchString(u)
+}
+
+func (p *TwitterParser) Name() string {
+	return "twitter"
+}
+
+func (p *TwitterParser) Configure(config map[string]any) error {
+	if config == nil {
+		p.apiDomain = fxTwitterApi
+		p.client = *netutil.DefaultParserHTTPClient()
+		return nil
+	}
+	if domain, ok := config["api_domain"].(string); ok && domain != "" {
+		p.apiDomain = domain
+	} else {
+		p.apiDomain = fxTwitterApi
+	}
+	if proxyUrl, ok := config["proxy"].(string); ok && proxyUrl != "" {
+		proxyClient, err := netutil.NewProxyHTTPClient(proxyUrl)
+		if err != nil {
+			return fmt.Errorf("failed to create proxy client: %w", err)
+		}
+		p.client = *proxyClient
+	}
+	return nil
+}

parsers/twitter/types.go

@@ -0,0 +1,122 @@
+package twitter
+
+// type AutoGenerated struct {
+// 	Code    int    `json:"code"`
+// 	Message string `json:"message"`
+// 	Tweet   struct {
+// 		URL     string `json:"url"`
+// 		ID      string `json:"id"`
+// 		Text    string `json:"text"`
+// 		RawText struct {
+// 			Text   string `json:"text"`
+// 			Facets []struct {
+// 				Type        string `json:"type"`
+// 				Indices     []int  `json:"indices"`
+// 				Original    string `json:"original"`
+// 				ID          string `json:"id,omitempty"`
+// 				Display     string `json:"display,omitempty"`
+// 				Replacement string `json:"replacement,omitempty"`
+// 			} `json:"facets"`
+// 		} `json:"raw_text"`
+// 		Author struct {
+// 			ID          string      `json:"id"`
+// 			Name        string      `json:"name"`
+// 			ScreenName  string      `json:"screen_name"`
+// 			AvatarURL   string      `json:"avatar_url"`
+// 			BannerURL   interface{} `json:"banner_url"`
+// 			Description string      `json:"description"`
+// 			Location    string      `json:"location"`
+// 			URL         string      `json:"url"`
+// 			Followers   int         `json:"followers"`
+// 			Following   int         `json:"following"`
+// 			Joined      string      `json:"joined"`
+// 			Likes       int         `json:"likes"`
+// 			MediaCount  int         `json:"media_count"`
+// 			Protected   bool        `json:"protected"`
+// 			Website     struct {
+// 				URL        string `json:"url"`
+// 				DisplayURL string `json:"display_url"`
+// 			} `json:"website"`
+// 			Tweets      int         `json:"tweets"`
+// 			AvatarColor interface{} `json:"avatar_color"`
+// 		} `json:"author"`
+// 		Replies           int         `json:"replies"`
+// 		Retweets          int         `json:"retweets"`
+// 		Likes             int         `json:"likes"`
+// 		Bookmarks         int         `json:"bookmarks"`
+// 		CreatedAt         string      `json:"created_at"`
+// 		CreatedTimestamp  int         `json:"created_timestamp"`
+// 		PossiblySensitive bool        `json:"possibly_sensitive"`
+// 		Views             int         `json:"views"`
+// 		IsNoteTweet       bool        `json:"is_note_tweet"`
+// 		CommunityNote     interface{} `json:"community_note"`
+// 		Lang              string      `json:"lang"`
+// 		ReplyingTo        interface{} `json:"replying_to"`
+// 		ReplyingToStatus  interface{} `json:"replying_to_status"`
+// 		Media             struct {
+// 			All []struct {
+// 				URL          string `json:"url"`
+// 				ThumbnailURL string `json:"thumbnail_url"`
+// 				Duration     int    `json:"duration"`
+// 				Width        int    `json:"width"`
+// 				Height       int    `json:"height"`
+// 				Format       string `json:"format"`
+// 				Type         string `json:"type"`
+// 				Variants     []struct {
+// 					Bitrate     int    `json:"bitrate"`
+// 					ContentType string `json:"content_type"`
+// 					URL         string `json:"url"`
+// 				} `json:"variants"`
+// 			} `json:"all"`
+// 			Photos []struct {
+// 				Type   string `json:"type"`
+// 				URL    string `json:"url"`
+// 				Width  int    `json:"width"`
+// 				Height int    `json:"height"`
+// 			} `json:"photos"`
+// 			Videos []struct {
+// 				URL          string `json:"url"`
+// 				ThumbnailURL string `json:"thumbnail_url"`
+// 				Duration     int    `json:"duration"`
+// 				Width        int    `json:"width"`
+// 				Height       int    `json:"height"`
+// 				Format       string `json:"format"`
+// 				Type         string `json:"type"`
+// 				Variants     []struct {
+// 					Bitrate     int    `json:"bitrate"`
+// 					ContentType string `json:"content_type"`
+// 					URL         string `json:"url"`
+// 				} `json:"variants"`
+// 			} `json:"videos"`
+// 		} `json:"media"`
+// 		Source      string      `json:"source"`
+// 		TwitterCard string      `json:"twitter_card"`
+// 		Color       interface{} `json:"color"`
+// 		Provider    string      `json:"provider"`
+// 	} `json:"tweet"`
+// }
+
+type FxTwitterApiResp struct {
+	Code    int    `json:"code"`
+	Message string `json:"message"`
+	Tweet   struct {
+		URL    string `json:"url"`
+		ID     string `json:"id"`
+		Text   string `json:"text"`
+		Author struct {
+			ID         string `json:"id"`
+			Name       string `json:"name"`
+			ScreenName string `json:"screen_name"`
+			Protected  bool   `json:"protected"`
+		} `json:"author"`
+		PossiblySensitive bool   `json:"possibly_sensitive"`
+		IsNoteTweet       bool   `json:"is_note_tweet"`
+		Lang              string `json:"lang"`
+		Media             struct {
+			All []struct {
+				URL  string `json:"url"`
+				Type string `json:"type"`
+			} `json:"all"`
+		} `json:"media"`
+	} `json:"tweet"`
+}

pkg/consts/tglimit/tglimit.go

@@ -7,4 +7,5 @@ import (
 const (
 	MaxPartSize       = 1024 * 1024
 	MaxUploadPartSize = uploader.MaximumPartSize
+	MaxPhotoSize      = 10 * 1024 * 1024
 )

pkg/consts/version.go

@@ -1,9 +0,0 @@
-package consts
-
-// inject version by '-X' flag
-// go build -ldflags "-X github.com/krau/SaveAny-Bot/pkg/consts.Version=${{ env.VERSION }}"
-var (
-	Version   string = "dev"
-	BuildTime string = "unknown"
-	GitCommit string = "unknown"
-)

pkg/enums/fnamest/filename_srategy.go

@@ -0,0 +1,14 @@
+package fnamest
+
+//go:generate go-enum --values --names --noprefix --flag --nocase
+
+// FnameST
+/* ENUM(
+default, message
+) */
+type FnameST string
+
+var FnameSTDisplay = map[FnameST]string{
+	Default: "默认",
+	Message: "优先从消息生成",
+}

pkg/enums/fnamest/filename_srategy_enum.go

@@ -0,0 +1,87 @@
+// Code generated by go-enum DO NOT EDIT.
+// Version: 0.6.1
+// Revision: a6f63bddde05aca4221df9c8e9e6d7d9674b1cb4
+// Build Date: 2025-03-18T23:42:14Z
+// Built By: goreleaser
+
+package fnamest
+
+import (
+	"fmt"
+	"strings"
+)
+
+const (
+	// Default is a FnameST of type default.
+	Default FnameST = "default"
+	// Message is a FnameST of type message.
+	Message FnameST = "message"
+)
+
+var ErrInvalidFnameST = fmt.Errorf("not a valid FnameST, try [%s]", strings.Join(_FnameSTNames, ", "))
+
+var _FnameSTNames = []string{
+	string(Default),
+	string(Message),
+}
+
+// FnameSTNames returns a list of possible string values of FnameST.
+func FnameSTNames() []string {
+	tmp := make([]string, len(_FnameSTNames))
+	copy(tmp, _FnameSTNames)
+	return tmp
+}
+
+// FnameSTValues returns a list of the values for FnameST
+func FnameSTValues() []FnameST {
+	return []FnameST{
+		Default,
+		Message,
+	}
+}
+
+// String implements the Stringer interface.
+func (x FnameST) String() string {
+	return string(x)
+}
+
+// IsValid provides a quick way to determine if the typed value is
+// part of the allowed enumerated values
+func (x FnameST) IsValid() bool {
+	_, err := ParseFnameST(string(x))
+	return err == nil
+}
+
+var _FnameSTValue = map[string]FnameST{
+	"default": Default,
+	"message": Message,
+}
+
+// ParseFnameST attempts to convert a string to a FnameST.
+func ParseFnameST(name string) (FnameST, error) {
+	if x, ok := _FnameSTValue[name]; ok {
+		return x, nil
+	}
+	// Case insensitive parse, do a separate lookup to prevent unnecessary cost of lowercasing a string if we don't need to.
+	if x, ok := _FnameSTValue[strings.ToLower(name)]; ok {
+		return x, nil
+	}
+	return FnameST(""), fmt.Errorf("%s is %w", name, ErrInvalidFnameST)
+}
+
+// Set implements the Golang flag.Value interface func.
+func (x *FnameST) Set(val string) error {
+	v, err := ParseFnameST(val)
+	*x = v
+	return err
+}
+
+// Get implements the Golang flag.Getter interface func.
+func (x *FnameST) Get() interface{} {
+	return *x
+}
+
+// Type implements the github.com/spf13/pFlag Value interface.
+func (x *FnameST) Type() string {
+	return "FnameST"
+}

pkg/enums/tasktype/tasktype.go

@@ -1,5 +1,5 @@
 package tasktype
 
 //go:generate go-enum --values --names --flag --nocase
-// ENUM(tgfiles,tphpics)
+// ENUM(tgfiles,tphpics,parseditem)
 type TaskType string

pkg/enums/tasktype/tasktype_enum.go

@@ -16,6 +16,8 @@ const (
 	TaskTypeTgfiles TaskType = "tgfiles"
 	// TaskTypeTphpics is a TaskType of type tphpics.
 	TaskTypeTphpics TaskType = "tphpics"
+	// TaskTypeParseditem is a TaskType of type parseditem.
+	TaskTypeParseditem TaskType = "parseditem"
 )
 
 var ErrInvalidTaskType = fmt.Errorf("not a valid TaskType, try [%s]", strings.Join(_TaskTypeNames, ", "))
@@ -23,6 +25,7 @@ var ErrInvalidTaskType = fmt.Errorf("not a valid TaskType, try [%s]", strings.Jo
 var _TaskTypeNames = []string{
 	string(TaskTypeTgfiles),
 	string(TaskTypeTphpics),
+	string(TaskTypeParseditem),
 }
 
 // TaskTypeNames returns a list of possible string values of TaskType.
@@ -37,6 +40,7 @@ func TaskTypeValues() []TaskType {
 	return []TaskType{
 		TaskTypeTgfiles,
 		TaskTypeTphpics,
+		TaskTypeParseditem,
 	}
 }
 
@@ -53,8 +57,9 @@ func (x TaskType) IsValid() bool {
 }
 
 var _TaskTypeValue = map[string]TaskType{
-	"tgfiles": TaskTypeTgfiles,
-	"tphpics": TaskTypeTphpics,
+	"tgfiles":    TaskTypeTgfiles,
+	"tphpics":    TaskTypeTphpics,
+	"parseditem": TaskTypeParseditem,
 }
 
 // ParseTaskType attempts to convert a string to a TaskType.

pkg/parser/parser.go

@@ -0,0 +1,70 @@
+package parser
+
+import (
+	"context"
+	"crypto/md5"
+	"fmt"
+)
+
+type Parser interface {
+	CanHandle(url string) bool
+	Parse(ctx context.Context, url string) (*Item, error)
+}
+
+type ConfigurableParser interface {
+	Parser
+	Configure(config map[string]any) error
+	Name() string
+}
+
+// Resource is a single downloadable resource with metadata.
+type Resource struct {
+	URL       string            `json:"url"`
+	Filename  string            `json:"filename"` // with ext
+	MimeType  string            `json:"mime_type"`
+	Extension string            `json:"extension"` // e.g. "mp4"
+	Size      int64             `json:"size"`    // 0 when unknown
+	Hash      map[string]string `json:"hash"`    // {"md5": "...", "sha256": "..."}
+	Headers   map[string]string `json:"headers"` // HTTP headers when downloading
+	Extra     map[string]any    `json:"extra"`
+}
+
+type Item struct {
+	Site        string         `json:"site"`
+	URL         string         `json:"url"` // original URL of the item
+	Title       string         `json:"title"`
+	Author      string         `json:"author"`
+	Description string         `json:"description"`
+	Tags        []string       `json:"tags"`
+	Resources   []Resource     `json:"resources"`
+	Extra       map[string]any `json:"extra"`
+}
+
+func (r *Resource) FileName() string {
+	return r.Filename
+}
+
+func (r *Resource) FileSize() int64 {
+	return r.Size
+}
+
+func (r *Resource) ID() string {
+	h := md5.New()
+	h.Write([]byte(r.URL))
+	h.Write([]byte(r.Filename))
+	h.Write([]byte(r.MimeType))
+	h.Write([]byte(r.Extension))
+	h.Write([]byte(fmt.Sprintf("%d", r.Size)))
+
+	for k, v := range r.Hash {
+		h.Write([]byte(k))
+		h.Write([]byte(v))
+	}
+
+	for k, v := range r.Headers {
+		h.Write([]byte(k))
+		h.Write([]byte(v))
+	}
+
+	return fmt.Sprintf("%x", h.Sum(nil))
+}

pkg/rule/consts.go

@@ -1,4 +1,4 @@
-package consts
+package rule
 
 const (
 	RuleStorNameChosen     = "CHOSEN"