refactor: quality overhaul (#234)

* fix: prevent queue deadlock after cancelling tasks

Get no longer recurses while holding the mutex.
Cancelled queued tasks leave the map, making their IDs reusable.
Closed empty queues return ErrQueueClosed.

* fix: init task queue lazily and close on shutdown

AddTask is safe before Run by initializing the queue once.
Close unblocks workers waiting in Get.

* fix: make resource fingerprints deterministic

Sort map keys before hashing so Resource.ID is stable.

* fix: harden per-item processing tracking in tasks

Use the resource fingerprint as the dedup key on insert and delete.
Check and set processing entries atomically instead of TOCTOU.
Count only successful downloads.

* fix: honor IgnoreErrors in batch tasks

Element failures no longer cancel sibling elements or stop later groups.

* fix: report streamed upload bytes in batch tasks

Stream downloads report their byte count as the upload total.

* fix: validate i18n key parity across locales

geni18n fails when a language file misses any key.
Align the syncpeers completion key between en and zh-Hans.
Translate three untranslated parse keys in zh-Hans.

* refactor: share progress throttling helpers

Move size-tiered and count-based throttling into progressutil.
Localize hardcoded Chinese progress strings.
Drop five duplicated implementations and dead local copies.

* refactor: share unique filename logic

Storage backends use fsutil.UniquePath instead of local loops.

* fix: sanitize local storage paths

Reject absolute paths and dot-dot escapes in Save.
Check close errors and wrap creation failures.

* fix: preserve webdav error causes

Wrap mkdir and write failures with %w and drop dead error values.

* fix: kill rclone subprocess on reader close

Prevent cat processes from hanging after the pipe is closed.

* fix: fail alist init instead of exiting

Replace log.Fatalf with wrapped errors so a bad alist cannot kill the bot.
Cancel token refresh with the init context and re-login on 401.

* fix: enforce telegram album limits and track saved paths

Use tglimit.MaxAlbumItems for album batching and video splitting.
Exists now reports previously saved paths instead of always false.

* fix: guard storage registry maps

Protect Storages and UserStorages with mutexes and expose read accessors.

* fix: harden JS parser plugin runtime

Validate semver instead of panicking and require canHandle.
Recover plugin worker panics and time out CanHandle calls.
Return a copy from the registry and sanitize install filenames.

* fix: guard kemono parser against nil fields

Skip sparse preview and attachment entries instead of panicking.

* refactor: remove commented-out dead code

Drop unused kemono legacy types and commented response structs.

* test: cover invalid plugin version rejection

* fix: show all queued tasks in /task list

Render up to ten tasks and append the truncation note once.

* fix: guard callback data parsing

Reject malformed callback payloads before indexing split parts.

* fix: isolate media groups per user

Key pending groups by chat, user and group id.

* fix: require permission for callback handlers

* fix: initialize userbot context once

Replace the racy lazy init with sync.OnceValue.

* fix: avoid leaking raw errors in /dir reply

* fix: notify users on invalid update version

* fix: fail fast when API listen fails

Bind synchronously and surface errors instead of logging them.

* fix: add timeouts and backoff to webhook delivery

* refactor: remove dead code from api and bot

Drop the empty ProgressTracker shim, unused token context key and a redundant SetBotCommands call.

* fix: load remote config without local lookup

Skip the local file search after reading a config URL and add a timeout.

* refactor: drop unused hook config

* docs: document parser plugin config

* ci: fix BuildTime formatting and align checkout

Actions format does not format dates; pass the raw timestamp.

* chore: ignore cache directory

* fix: make cache init idempotent

* fix: upload only downloaded batch elements

Failed elements keep partial cache files and never reach the backend.
Successfully downloaded siblings still upload when one element fails.

* fix: record telegram saved paths only after upload

Skip-large returns a sentinel so skipped files are not marked as saved.

* fix: deduplicate alist token refreshes

Guard token access with a mutex and merge concurrent logins.
Reuse a recent refresh to avoid login storms.

* test: cover concurrent alist 401 retry

Ten parallel uploads share a single re-login under -race.

* fix: count parsed resources in progress text

* fix: localize storage lookup errors in /dir

Use the shared i18n key and escape the dynamic error.

* fix: deduplicate concurrent storage initialization

singleflight merges first-time inits so side effects are not duplicated.

* fix: guard nil progress trackers in api-created tasks

Batch, telegraph and transfer tasks run without a Telegram tracker
when created through the API; their callbacks must not panic.

* fix: keep album order when filtering failed batch items

Download results are stored by original index so surviving
elements keep their source order.

* test: cover nil-tracker task execution

* fix: report partial failure in batch done message

IgnoreErrors runs with failed elements show success and failed
counts instead of claiming every file completed.

* fix: skip login refresh for token-only alist storage

401 responses surface the auth error instead of sending a
credential-less login; the refresh window never exceeds TokenExp.

* test: cover alist refresh semantics

Concurrent refresh uses username/password; token-only storage
never attempts a login on 401.

* fix: call tracker from notifyProgress instead of recursing

The helper called itself, overflowing the stack on any batch
task with a progress tracker.

* fix: propagate cancellation past IgnoreErrors

Cancelled tasks must not be reported as successful: only
ordinary element failures are ignored.

* test: cover notifyProgress tracker call

* test: cover cancellation with IgnoreErrors

* fix: refresh alist token after startup 401s

The init login no longer satisfies the dedup window, so an
early 401 triggers a real refresh; later 401s reuse it.
Clarify that streaming uploads cannot replay their body.

* test: exercise the alist 401 refresh path

Concurrent uploads now reject the init token, share one
refresh and retry with the new token; token-only stays inert.

* style: trim verbose comments

Drop process-style explanations; keep one-line behavior notes.

* style: gofmt test files

* fix: drop unbounded saved-path cache in telegram storage

Telegram cannot reliably query remote file existence, so the
cache answered a question it could not answer and grew without
bound. Exists returns false again, as before.

* style: format codes

api/auth.go

@@ -1,7 +1,6 @@
 package api
 
 import (
-	"context"
 	"crypto/subtle"
 	"net/http"
 	"strings"
@@ -9,9 +8,6 @@ import (
 	"github.com/krau/SaveAny-Bot/config"
 )
 
-// tokenContextKey 用于在 context 中存储 token
-type tokenContextKey struct{}
-
 // AuthMiddleware 返回认证中间件
 func AuthMiddleware() func(http.Handler) http.Handler {
 	return func(next http.Handler) http.Handler {
@@ -40,9 +36,7 @@ func AuthMiddleware() func(http.Handler) http.Handler {
 				return
 			}
 
-			// 将 token 添加到 context
-			ctx := context.WithValue(r.Context(), tokenContextKey{}, token)
-			next.ServeHTTP(w, r.WithContext(ctx))
+			next.ServeHTTP(w, r)
 		})
 	}
 }

api/factory.go

@@ -39,7 +39,7 @@ func NewTaskFactory(ctx context.Context) *TaskFactory {
 // CreateTask 创建任务
 func (f *TaskFactory) CreateTask(req *CreateTaskRequest) (*CreateTaskResponse, error) {
 	// 验证存储
-	stor, ok := storage.Storages[req.Storage]
+	stor, ok := storage.GetStorage(req.Storage)
 	if !ok {
 		return nil, fmt.Errorf("storage not found: %s", req.Storage)
 	}
@@ -327,12 +327,12 @@ func (f *TaskFactory) createTransferTask(taskID string, createdAt time.Time, req
 	}
 
 	// 验证源存储和目标存储
-	sourceStor, ok := storage.Storages[params.SourceStorage]
+	sourceStor, ok := storage.GetStorage(params.SourceStorage)
 	if !ok {
 		return nil, fmt.Errorf("source storage not found: %s", params.SourceStorage)
 	}
 
-	targetStor, ok := storage.Storages[params.TargetStorage]
+	targetStor, ok := storage.GetStorage(params.TargetStorage)
 	if !ok {
 		return nil, fmt.Errorf("target storage not found: %s", params.TargetStorage)
 	}

api/handlers.go

@@ -135,8 +135,9 @@ func (h *Handlers) ListStoragesHandler(w http.ResponseWriter, r *http.Request) {
 		return
 	}
 
-	storages := make([]StorageInfo, 0, len(storage.Storages))
-	for name, stor := range storage.Storages {
+	all := storage.AllStorages()
+	storages := make([]StorageInfo, 0, len(all))
+	for name, stor := range all {
 		storages = append(storages, StorageInfo{
 			Name: name,
 			Type: string(stor.Type()),

api/progress.go

@@ -198,19 +198,3 @@ func (t *TaskProgressInfo) Emit(e taskevent.Event) {
 		SendWebhook(nil, payload)
 	}
 }
-
-// ProgressTracker is retained for compatibility but is no longer the primary
-// progress path; taskevent drives updates now. These methods are safe no-ops
-// when called on a nil receiver.
-type ProgressTracker struct{}
-
-func NewProgressTracker(taskID, taskType, storage, path, title, webhook string) *ProgressTracker {
-	return &ProgressTracker{}
-}
-
-func (p *ProgressTracker) OnStart(totalBytes int64, totalFiles int)              {}
-func (p *ProgressTracker) OnProgress(downloadedBytes int64, downloadedFiles int) {}
-func (p *ProgressTracker) OnDone(err error)                                      {}
-func (p *ProgressTracker) GetInfo() *TaskProgressInfo                            { return nil }
-func (p *ProgressTracker) UpdateProgressBytes(bytes int64)                       {}
-func (p *ProgressTracker) UpdateProgressFiles(files int)                         {}

api/server.go

@@ -3,6 +3,7 @@ package api
 import (
 	"context"
 	"fmt"
+	"net"
 	"net/http"
 	"time"
 
@@ -90,9 +91,15 @@ func (s *Server) Start(ctx context.Context) error {
 
 	logger.Infof("Starting API server on %s", s.httpServer.Addr)
 
+	// Bind synchronously so listen failures are returned to the caller.
+	ln, err := net.Listen("tcp", s.httpServer.Addr)
+	if err != nil {
+		return fmt.Errorf("failed to listen on %s: %w", s.httpServer.Addr, err)
+	}
+
 	// 在 goroutine 中启动服务器
 	go func() {
-		if err := s.httpServer.ListenAndServe(); err != nil && err != http.ErrServerClosed {
+		if err := s.httpServer.Serve(ln); err != nil && err != http.ErrServerClosed {
 			logger.Errorf("API server error: %v", err)
 		}
 	}()

api/webhook.go

@@ -37,6 +37,9 @@ func SendWebhook(ctx context.Context, payload *WebhookPayload) {
 		} else {
 			logger = log.Default().With("task_id", payload.TaskID)
 		}
+		if ctx == nil {
+			ctx = context.Background()
+		}
 
 		payloadBytes, err := json.Marshal(payload)
 		if err != nil {
@@ -44,10 +47,15 @@ func SendWebhook(ctx context.Context, payload *WebhookPayload) {
 			return
 		}
 
-		// 重试 3 次
-		for i := range 3 {
-			req, err := http.NewRequestWithContext(context.Background(), http.MethodPost, webhookURL, bytes.NewBuffer(payloadBytes))
+		// 重试 3 次, 指数退避 (100ms/400ms/1.6s)
+		const maxAttempts = 3
+		const requestTimeout = 30 * time.Second
+		backoff := 100 * time.Millisecond
+		for i := range maxAttempts {
+			reqCtx, cancel := context.WithTimeout(ctx, requestTimeout)
+			req, err := http.NewRequestWithContext(reqCtx, http.MethodPost, webhookURL, bytes.NewBuffer(payloadBytes))
 			if err != nil {
+				cancel()
 				logger.Errorf("Failed to create webhook request: %v", err)
 				return
 			}
@@ -56,9 +64,13 @@ func SendWebhook(ctx context.Context, payload *WebhookPayload) {
 			req.Header.Set("User-Agent", "SaveAny-Bot/1.0")
 
 			resp, err := webhookClient.Do(req)
+			cancel()
 			if err != nil {
-				logger.Warnf("Webhook request failed (attempt %d/3): %v", i+1, err)
-				time.Sleep(time.Second * time.Duration(i+1))
+				logger.Warnf("Webhook request failed (attempt %d/%d): %v", i+1, maxAttempts, err)
+				if i < maxAttempts-1 {
+					time.Sleep(backoff)
+				}
+				backoff *= 4
 				continue
 			}
 			resp.Body.Close()
@@ -68,11 +80,14 @@ func SendWebhook(ctx context.Context, payload *WebhookPayload) {
 				return
 			}
 
-			logger.Warnf("Webhook returned non-2xx status (attempt %d/3): %d", i+1, resp.StatusCode)
-			time.Sleep(time.Second * time.Duration(i+1))
+			logger.Warnf("Webhook returned non-2xx status (attempt %d/%d): %d", i+1, maxAttempts, resp.StatusCode)
+			if i < maxAttempts-1 {
+				time.Sleep(backoff)
+			}
+			backoff *= 4
 		}
 
-		logger.Errorf("Failed to send webhook after 3 attempts")
+		logger.Errorf("Failed to send webhook after %d attempts", maxAttempts)
 	}()
 }