refactor: quality overhaul (#234)

* fix: prevent queue deadlock after cancelling tasks

Get no longer recurses while holding the mutex.
Cancelled queued tasks leave the map, making their IDs reusable.
Closed empty queues return ErrQueueClosed.

* fix: init task queue lazily and close on shutdown

AddTask is safe before Run by initializing the queue once.
Close unblocks workers waiting in Get.

* fix: make resource fingerprints deterministic

Sort map keys before hashing so Resource.ID is stable.

* fix: harden per-item processing tracking in tasks

Use the resource fingerprint as the dedup key on insert and delete.
Check and set processing entries atomically instead of TOCTOU.
Count only successful downloads.

* fix: honor IgnoreErrors in batch tasks

Element failures no longer cancel sibling elements or stop later groups.

* fix: report streamed upload bytes in batch tasks

Stream downloads report their byte count as the upload total.

* fix: validate i18n key parity across locales

geni18n fails when a language file misses any key.
Align the syncpeers completion key between en and zh-Hans.
Translate three untranslated parse keys in zh-Hans.

* refactor: share progress throttling helpers

Move size-tiered and count-based throttling into progressutil.
Localize hardcoded Chinese progress strings.
Drop five duplicated implementations and dead local copies.

* refactor: share unique filename logic

Storage backends use fsutil.UniquePath instead of local loops.

* fix: sanitize local storage paths

Reject absolute paths and dot-dot escapes in Save.
Check close errors and wrap creation failures.

* fix: preserve webdav error causes

Wrap mkdir and write failures with %w and drop dead error values.

* fix: kill rclone subprocess on reader close

Prevent cat processes from hanging after the pipe is closed.

* fix: fail alist init instead of exiting

Replace log.Fatalf with wrapped errors so a bad alist cannot kill the bot.
Cancel token refresh with the init context and re-login on 401.

* fix: enforce telegram album limits and track saved paths

Use tglimit.MaxAlbumItems for album batching and video splitting.
Exists now reports previously saved paths instead of always false.

* fix: guard storage registry maps

Protect Storages and UserStorages with mutexes and expose read accessors.

* fix: harden JS parser plugin runtime

Validate semver instead of panicking and require canHandle.
Recover plugin worker panics and time out CanHandle calls.
Return a copy from the registry and sanitize install filenames.

* fix: guard kemono parser against nil fields

Skip sparse preview and attachment entries instead of panicking.

* refactor: remove commented-out dead code

Drop unused kemono legacy types and commented response structs.

* test: cover invalid plugin version rejection

* fix: show all queued tasks in /task list

Render up to ten tasks and append the truncation note once.

* fix: guard callback data parsing

Reject malformed callback payloads before indexing split parts.

* fix: isolate media groups per user

Key pending groups by chat, user and group id.

* fix: require permission for callback handlers

* fix: initialize userbot context once

Replace the racy lazy init with sync.OnceValue.

* fix: avoid leaking raw errors in /dir reply

* fix: notify users on invalid update version

* fix: fail fast when API listen fails

Bind synchronously and surface errors instead of logging them.

* fix: add timeouts and backoff to webhook delivery

* refactor: remove dead code from api and bot

Drop the empty ProgressTracker shim, unused token context key and a redundant SetBotCommands call.

* fix: load remote config without local lookup

Skip the local file search after reading a config URL and add a timeout.

* refactor: drop unused hook config

* docs: document parser plugin config

* ci: fix BuildTime formatting and align checkout

Actions format does not format dates; pass the raw timestamp.

* chore: ignore cache directory

* fix: make cache init idempotent

* fix: upload only downloaded batch elements

Failed elements keep partial cache files and never reach the backend.
Successfully downloaded siblings still upload when one element fails.

* fix: record telegram saved paths only after upload

Skip-large returns a sentinel so skipped files are not marked as saved.

* fix: deduplicate alist token refreshes

Guard token access with a mutex and merge concurrent logins.
Reuse a recent refresh to avoid login storms.

* test: cover concurrent alist 401 retry

Ten parallel uploads share a single re-login under -race.

* fix: count parsed resources in progress text

* fix: localize storage lookup errors in /dir

Use the shared i18n key and escape the dynamic error.

* fix: deduplicate concurrent storage initialization

singleflight merges first-time inits so side effects are not duplicated.

* fix: guard nil progress trackers in api-created tasks

Batch, telegraph and transfer tasks run without a Telegram tracker
when created through the API; their callbacks must not panic.

* fix: keep album order when filtering failed batch items

Download results are stored by original index so surviving
elements keep their source order.

* test: cover nil-tracker task execution

* fix: report partial failure in batch done message

IgnoreErrors runs with failed elements show success and failed
counts instead of claiming every file completed.

* fix: skip login refresh for token-only alist storage

401 responses surface the auth error instead of sending a
credential-less login; the refresh window never exceeds TokenExp.

* test: cover alist refresh semantics

Concurrent refresh uses username/password; token-only storage
never attempts a login on 401.

* fix: call tracker from notifyProgress instead of recursing

The helper called itself, overflowing the stack on any batch
task with a progress tracker.

* fix: propagate cancellation past IgnoreErrors

Cancelled tasks must not be reported as successful: only
ordinary element failures are ignored.

* test: cover notifyProgress tracker call

* test: cover cancellation with IgnoreErrors

* fix: refresh alist token after startup 401s

The init login no longer satisfies the dedup window, so an
early 401 triggers a real refresh; later 401s reuse it.
Clarify that streaming uploads cannot replay their body.

* test: exercise the alist 401 refresh path

Concurrent uploads now reject the init token, share one
refresh and retry with the new token; token-only stays inert.

* style: trim verbose comments

Drop process-style explanations; keep one-line behavior notes.

* style: gofmt test files

* fix: drop unbounded saved-path cache in telegram storage

Telegram cannot reliably query remote file existence, so the
cache answered a question it could not answer and grew without
bound. Exists returns false again, as before.

* style: format codes

common/utils/fsutil/unique.go

@@ -0,0 +1,27 @@
+package fsutil
+
+import (
+	"fmt"
+	"path"
+	"strings"
+
+	"github.com/rs/xid"
+)
+
+// UniquePath returns a non-taken path under basePath: the name itself, then
+// numbered variants, then a random suffix.
+func UniquePath(basePath, name string, exists func(candidate string) bool, maxAttempts int) string {
+	candidate := path.Join(basePath, name)
+	if !exists(candidate) {
+		return candidate
+	}
+	ext := path.Ext(name)
+	stem := strings.TrimSuffix(name, ext)
+	for i := 1; i <= maxAttempts; i++ {
+		candidate = path.Join(basePath, fmt.Sprintf("%s_%d%s", stem, i, ext))
+		if !exists(candidate) {
+			return candidate
+		}
+	}
+	return path.Join(basePath, fmt.Sprintf("%s_%s%s", stem, xid.New().String(), ext))
+}

common/utils/progressutil/progress.go

@@ -0,0 +1,50 @@
+// Package progressutil provides shared progress-update throttling for task
+// progress trackers.
+package progressutil
+
+// updatesLevels picks the percent step by file size.
+var updatesLevels = []struct {
+	size        int64 // file size threshold
+	stepPercent int   // minimum percent step between updates
+}{
+	{10 << 20, 100},
+	{50 << 20, 20},
+	{200 << 20, 10},
+	{500 << 20, 5},
+}
+
+// ShouldUpdate reports whether a byte-based progress update should be shown.
+func ShouldUpdate(total, downloaded int64, lastUpdatePercent int) bool {
+	if total <= 0 || downloaded <= 0 {
+		return false
+	}
+
+	percent := int((downloaded * 100) / total)
+	if percent <= lastUpdatePercent {
+		return false
+	}
+
+	step := updatesLevels[len(updatesLevels)-1].stepPercent
+	for _, lvl := range updatesLevels {
+		if total < lvl.size {
+			step = lvl.stepPercent
+			break
+		}
+	}
+
+	return percent >= lastUpdatePercent+step
+}
+
+// ShouldUpdateCount reports whether a count-based progress update (e.g. files
+// downloaded so far) should be shown: every 10 units, or when finished.
+func ShouldUpdateCount(downloaded, total int64) bool {
+	if total <= 0 || downloaded <= 0 {
+		return false
+	}
+
+	const step = int64(10)
+	if downloaded < step {
+		return downloaded == total
+	}
+	return downloaded%step == 0 || downloaded == total
+}

common/utils/tgutil/message.go

@@ -194,97 +194,6 @@ func getMessagesRange(ctx *ext.Context, chatID int64, minId, maxId int) ([]*tg.M
 	return result, nil
 }
 
-// [TODO]
-// type MessageItem struct {
-// 	Message *tg.Message
-// 	Error   error
-// }
-
-// func IterMessages(ctx *ext.Context, chatID int64, minId, maxId int) (<-chan MessageItem, error) {
-// 	total := maxId - minId + 1
-// 	ch := make(chan MessageItem, 100)
-
-// 	go func() {
-// 		defer close(ch)
-// 		if !ctx.Self.Bot {
-// 			perr := ctx.PeerStorage.GetInputPeerById(chatID)
-// 			if perr == nil || perr.(*tg.InputPeerEmpty) != nil {
-// 				ch <- MessageItem{
-// 					Error: fmt.Errorf("peer not found: %d", chatID),
-// 				}
-// 				return
-// 			}
-
-// 			for i := 0; i < total; i += 100 {
-// 				start := minId + i
-// 				end := min(start+100, maxId)
-// 				msgs, err := ctx.Raw.MessagesGetHistory(ctx, &tg.MessagesGetHistoryRequest{
-// 					Peer:      perr,
-// 					OffsetID:  start,
-// 					AddOffset: start - end,
-// 					Limit:     100,
-// 				})
-// 				if err != nil {
-// 					ch <- MessageItem{
-// 						Error: fmt.Errorf("failed to get messages: %w", err),
-// 					}
-// 					return
-// 				}
-// 				var msgClass []tg.MessageClass
-// 				switch msgsv := msgs.(type) {
-// 				case *tg.MessagesMessages:
-// 					msgClass = msgsv.GetMessages()
-// 				case *tg.MessagesMessagesSlice:
-// 					msgClass = msgsv.GetMessages()
-// 				case *tg.MessagesChannelMessages:
-// 					msgClass = msgsv.GetMessages()
-// 				default:
-// 					ch <- MessageItem{
-// 						Error: fmt.Errorf("unsupported message type: %T", msgsv),
-// 					}
-// 					continue
-// 				}
-// 				for _, msg := range msgClass {
-// 					msg, ok := msg.AsNotEmpty()
-// 					if !ok {
-// 						continue
-// 					}
-// 					switch msg := msg.(type) {
-// 					case *tg.Message:
-// 						key := fmt.Sprintf("tgmsg:%d:%d:%d", ctx.Self.ID, chatID, msg.GetID())
-// 						cache.Set(key, msg)
-// 						ch <- MessageItem{
-// 							Message: msg,
-// 						}
-// 					}
-// 				}
-// 			}
-// 		} else {
-// 			for i := 0; i < total; i += 100 {
-// 				start := minId + i
-// 				end := min(start+100, maxId)
-// 				msgs, err := GetMessagesRange(ctx, chatID, start, end)
-// 				if err != nil {
-// 					ch <- MessageItem{
-// 						Error: fmt.Errorf("failed to get messages: %w", err),
-// 					}
-// 					return
-// 				}
-// 				for _, msg := range msgs {
-// 					if msg == nil {
-// 						continue
-// 					}
-// 					ch <- MessageItem{
-// 						Message: msg,
-// 					}
-// 				}
-// 			}
-// 		}
-// 	}()
-
-// 	return ch, nil
-// }
-
 func getMessageByID(ctx *ext.Context, chatID int64, msgID int) (*tg.Message, error) {
 	key := fmt.Sprintf("tgmsg:%d:%d:%d", ctx.Self.ID, chatID, msgID)
 	if msg, ok := cache.Get[*tg.Message](key); ok {