The #228 progress redesign dropped the total size that the old
batch progress message displayed. Add a TotalSize data field
(actual known sizes, formatted via dlutil.FormatSize) to the
batch status header template in both locales.
* fix: prevent queue deadlock after cancelling tasks
Get no longer recurses while holding the mutex.
Cancelled queued tasks leave the map, making their IDs reusable.
Closed empty queues return ErrQueueClosed.
* fix: init task queue lazily and close on shutdown
AddTask is safe before Run by initializing the queue once.
Close unblocks workers waiting in Get.
* fix: make resource fingerprints deterministic
Sort map keys before hashing so Resource.ID is stable.
* fix: harden per-item processing tracking in tasks
Use the resource fingerprint as the dedup key on insert and delete.
Check and set processing entries atomically instead of TOCTOU.
Count only successful downloads.
* fix: honor IgnoreErrors in batch tasks
Element failures no longer cancel sibling elements or stop later groups.
* fix: report streamed upload bytes in batch tasks
Stream downloads report their byte count as the upload total.
* fix: validate i18n key parity across locales
geni18n fails when a language file misses any key.
Align the syncpeers completion key between en and zh-Hans.
Translate three untranslated parse keys in zh-Hans.
* refactor: share progress throttling helpers
Move size-tiered and count-based throttling into progressutil.
Localize hardcoded Chinese progress strings.
Drop five duplicated implementations and dead local copies.
* refactor: share unique filename logic
Storage backends use fsutil.UniquePath instead of local loops.
* fix: sanitize local storage paths
Reject absolute paths and dot-dot escapes in Save.
Check close errors and wrap creation failures.
* fix: preserve webdav error causes
Wrap mkdir and write failures with %w and drop dead error values.
* fix: kill rclone subprocess on reader close
Prevent cat processes from hanging after the pipe is closed.
* fix: fail alist init instead of exiting
Replace log.Fatalf with wrapped errors so a bad alist cannot kill the bot.
Cancel token refresh with the init context and re-login on 401.
* fix: enforce telegram album limits and track saved paths
Use tglimit.MaxAlbumItems for album batching and video splitting.
Exists now reports previously saved paths instead of always false.
* fix: guard storage registry maps
Protect Storages and UserStorages with mutexes and expose read accessors.
* fix: harden JS parser plugin runtime
Validate semver instead of panicking and require canHandle.
Recover plugin worker panics and time out CanHandle calls.
Return a copy from the registry and sanitize install filenames.
* fix: guard kemono parser against nil fields
Skip sparse preview and attachment entries instead of panicking.
* refactor: remove commented-out dead code
Drop unused kemono legacy types and commented response structs.
* test: cover invalid plugin version rejection
* fix: show all queued tasks in /task list
Render up to ten tasks and append the truncation note once.
* fix: guard callback data parsing
Reject malformed callback payloads before indexing split parts.
* fix: isolate media groups per user
Key pending groups by chat, user and group id.
* fix: require permission for callback handlers
* fix: initialize userbot context once
Replace the racy lazy init with sync.OnceValue.
* fix: avoid leaking raw errors in /dir reply
* fix: notify users on invalid update version
* fix: fail fast when API listen fails
Bind synchronously and surface errors instead of logging them.
* fix: add timeouts and backoff to webhook delivery
* refactor: remove dead code from api and bot
Drop the empty ProgressTracker shim, unused token context key and a redundant SetBotCommands call.
* fix: load remote config without local lookup
Skip the local file search after reading a config URL and add a timeout.
* refactor: drop unused hook config
* docs: document parser plugin config
* ci: fix BuildTime formatting and align checkout
Actions format does not format dates; pass the raw timestamp.
* chore: ignore cache directory
* fix: make cache init idempotent
* fix: upload only downloaded batch elements
Failed elements keep partial cache files and never reach the backend.
Successfully downloaded siblings still upload when one element fails.
* fix: record telegram saved paths only after upload
Skip-large returns a sentinel so skipped files are not marked as saved.
* fix: deduplicate alist token refreshes
Guard token access with a mutex and merge concurrent logins.
Reuse a recent refresh to avoid login storms.
* test: cover concurrent alist 401 retry
Ten parallel uploads share a single re-login under -race.
* fix: count parsed resources in progress text
* fix: localize storage lookup errors in /dir
Use the shared i18n key and escape the dynamic error.
* fix: deduplicate concurrent storage initialization
singleflight merges first-time inits so side effects are not duplicated.
* fix: guard nil progress trackers in api-created tasks
Batch, telegraph and transfer tasks run without a Telegram tracker
when created through the API; their callbacks must not panic.
* fix: keep album order when filtering failed batch items
Download results are stored by original index so surviving
elements keep their source order.
* test: cover nil-tracker task execution
* fix: report partial failure in batch done message
IgnoreErrors runs with failed elements show success and failed
counts instead of claiming every file completed.
* fix: skip login refresh for token-only alist storage
401 responses surface the auth error instead of sending a
credential-less login; the refresh window never exceeds TokenExp.
* test: cover alist refresh semantics
Concurrent refresh uses username/password; token-only storage
never attempts a login on 401.
* fix: call tracker from notifyProgress instead of recursing
The helper called itself, overflowing the stack on any batch
task with a progress tracker.
* fix: propagate cancellation past IgnoreErrors
Cancelled tasks must not be reported as successful: only
ordinary element failures are ignored.
* test: cover notifyProgress tracker call
* test: cover cancellation with IgnoreErrors
* fix: refresh alist token after startup 401s
The init login no longer satisfies the dedup window, so an
early 401 triggers a real refresh; later 401s reuse it.
Clarify that streaming uploads cannot replay their body.
* test: exercise the alist 401 refresh path
Concurrent uploads now reject the init token, share one
refresh and retry with the new token; token-only stays inert.
* style: trim verbose comments
Drop process-style explanations; keep one-line behavior notes.
* style: gofmt test files
* fix: drop unbounded saved-path cache in telegram storage
Telegram cannot reliably query remote file existence, so the
cache answered a question it could not answer and grew without
bound. Exists returns false again, as before.
* style: format codes
* feat: show upload progress for file tasks
* feat: show upload progress for batch file tasks
* style: distinguish download and upload phases
* style: mark successful task completion
* fix: preserve storage save error context
* fix: serialize single-file progress updates
* fix: stabilize batch upload progress reporting
Correct batch upload totals, completion state, actual file sizes, and synchronized progress snapshots. Add regression coverage for concurrent updates and phase transitions.
* feat: show per-file transfer progress
Format single-file and batch download and upload states with Telegram entities, blockquotes, speeds, transferred sizes, progress bars, concise counters, and accurate confirmation handling. Cover upload retries plus final, error, and cancellation messages with regression tests.
* chore: remove transfer progress tests
* test: restore critical transfer progress coverage
* test: cover interleaved batch transfers
* fix: declare progress styles in locale templates
extractFrameAt() and getVideoMetadata() piped the media into ffmpeg/ffprobe
via pipe:0. A pipe is not seekable, so ffmpeg cannot decode non-faststart MP4s
whose moov atom sits at the END of the file (very common for yt-dlp / HLS-merged
downloads): the frame grab and probe silently fail, and the video is uploaded
with no thumbnail (and, for non-mp4 containers, no dimensions).
Hand ffmpeg/ffprobe a seekable file path instead. When the reader already is an
*os.File we use it directly; otherwise we spool to a temp file and clean it up.
Verified with an A/B upload of a 56 MB non-faststart clip: stock build produced
no thumbnail, patched build produced a correct thumbnail.
Co-authored-by: pennyucloud <valentino@pennyu.co.id>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Introduced new documentation for CLI subcommands: `upload` and `watch`, detailing their usage, flags, and examples.
- Added a section on file naming and conflict strategies, explaining how users can customize file names and handle duplicates via `/config` and `/fnametmpl` commands.
- Translated the new documentation into Chinese for broader accessibility.
* fix(api): update task route to handle GET and POST methods
Signed-off-by: Ilham Syahid S <ilhamsyahids@gmail.com>
* fix(api): implement ExecutableWrapper to manage task execution and status updates
Signed-off-by: Ilham Syahid S <ilhamsyahids@gmail.com>
* fix(api): refactor task registration and enqueueing into a separate method
Signed-off-by: Ilham Syahid S <ilhamsyahids@gmail.com>
---------
Signed-off-by: Ilham Syahid S <ilhamsyahids@gmail.com>
- Created new markdown files for Aria2 download, direct download links, parsers, storage rules, silent mode, storage transfer, chat watching, and yt-dlp video download.
- Included usage examples, configuration instructions, and detailed explanations for each feature.
- Translated documentation into Chinese for accessibility.
- Added Handlers struct and methods for task operations
- Implemented task progress tracking and storage
- Created server setup with middleware for logging and recovery
- Added support for Telegram file extraction and Telegraph image extraction
- Introduced webhook functionality for task status updates
- Defined request and response types for API interactions
* fix: update StoragePath method to return specific path for single file
* feat: add Rclone storage support with configuration and file operations
* docs: add Rclone support to documentation for configuration and usage
- Updated command usage in English and Simplified Chinese localization files to reflect changes in transfer command syntax.
- Removed batch import task implementation, replacing it with a new transfer task implementation.
- Introduced new task structure and progress tracking for file transfers.
- Updated task type enumeration to replace batch import with transfer.
- Added new fields in data structures to support transfer operations.
- Implemented file handling and progress reporting for the transfer task.
krau