Commit Graph
20 Commits
Author SHA1 Message Date
Krau c2f8ab3c01 refactor: quality overhaul (#234)
* fix: prevent queue deadlock after cancelling tasks

Get no longer recurses while holding the mutex.
Cancelled queued tasks leave the map, making their IDs reusable.
Closed empty queues return ErrQueueClosed.

* fix: init task queue lazily and close on shutdown

AddTask is safe before Run by initializing the queue once.
Close unblocks workers waiting in Get.

* fix: make resource fingerprints deterministic

Sort map keys before hashing so Resource.ID is stable.

* fix: harden per-item processing tracking in tasks

Use the resource fingerprint as the dedup key on insert and delete.
Check and set processing entries atomically instead of TOCTOU.
Count only successful downloads.

* fix: honor IgnoreErrors in batch tasks

Element failures no longer cancel sibling elements or stop later groups.

* fix: report streamed upload bytes in batch tasks

Stream downloads report their byte count as the upload total.

* fix: validate i18n key parity across locales

geni18n fails when a language file misses any key.
Align the syncpeers completion key between en and zh-Hans.
Translate three untranslated parse keys in zh-Hans.

* refactor: share progress throttling helpers

Move size-tiered and count-based throttling into progressutil.
Localize hardcoded Chinese progress strings.
Drop five duplicated implementations and dead local copies.

* refactor: share unique filename logic

Storage backends use fsutil.UniquePath instead of local loops.

* fix: sanitize local storage paths

Reject absolute paths and dot-dot escapes in Save.
Check close errors and wrap creation failures.

* fix: preserve webdav error causes

Wrap mkdir and write failures with %w and drop dead error values.

* fix: kill rclone subprocess on reader close

Prevent cat processes from hanging after the pipe is closed.

* fix: fail alist init instead of exiting

Replace log.Fatalf with wrapped errors so a bad alist cannot kill the bot.
Cancel token refresh with the init context and re-login on 401.

* fix: enforce telegram album limits and track saved paths

Use tglimit.MaxAlbumItems for album batching and video splitting.
Exists now reports previously saved paths instead of always false.

* fix: guard storage registry maps

Protect Storages and UserStorages with mutexes and expose read accessors.

* fix: harden JS parser plugin runtime

Validate semver instead of panicking and require canHandle.
Recover plugin worker panics and time out CanHandle calls.
Return a copy from the registry and sanitize install filenames.

* fix: guard kemono parser against nil fields

Skip sparse preview and attachment entries instead of panicking.

* refactor: remove commented-out dead code

Drop unused kemono legacy types and commented response structs.

* test: cover invalid plugin version rejection

* fix: show all queued tasks in /task list

Render up to ten tasks and append the truncation note once.

* fix: guard callback data parsing

Reject malformed callback payloads before indexing split parts.

* fix: isolate media groups per user

Key pending groups by chat, user and group id.

* fix: require permission for callback handlers

* fix: initialize userbot context once

Replace the racy lazy init with sync.OnceValue.

* fix: avoid leaking raw errors in /dir reply

* fix: notify users on invalid update version

* fix: fail fast when API listen fails

Bind synchronously and surface errors instead of logging them.

* fix: add timeouts and backoff to webhook delivery

* refactor: remove dead code from api and bot

Drop the empty ProgressTracker shim, unused token context key and a redundant SetBotCommands call.

* fix: load remote config without local lookup

Skip the local file search after reading a config URL and add a timeout.

* refactor: drop unused hook config

* docs: document parser plugin config

* ci: fix BuildTime formatting and align checkout

Actions format does not format dates; pass the raw timestamp.

* chore: ignore cache directory

* fix: make cache init idempotent

* fix: upload only downloaded batch elements

Failed elements keep partial cache files and never reach the backend.
Successfully downloaded siblings still upload when one element fails.

* fix: record telegram saved paths only after upload

Skip-large returns a sentinel so skipped files are not marked as saved.

* fix: deduplicate alist token refreshes

Guard token access with a mutex and merge concurrent logins.
Reuse a recent refresh to avoid login storms.

* test: cover concurrent alist 401 retry

Ten parallel uploads share a single re-login under -race.

* fix: count parsed resources in progress text

* fix: localize storage lookup errors in /dir

Use the shared i18n key and escape the dynamic error.

* fix: deduplicate concurrent storage initialization

singleflight merges first-time inits so side effects are not duplicated.

* fix: guard nil progress trackers in api-created tasks

Batch, telegraph and transfer tasks run without a Telegram tracker
when created through the API; their callbacks must not panic.

* fix: keep album order when filtering failed batch items

Download results are stored by original index so surviving
elements keep their source order.

* test: cover nil-tracker task execution

* fix: report partial failure in batch done message

IgnoreErrors runs with failed elements show success and failed
counts instead of claiming every file completed.

* fix: skip login refresh for token-only alist storage

401 responses surface the auth error instead of sending a
credential-less login; the refresh window never exceeds TokenExp.

* test: cover alist refresh semantics

Concurrent refresh uses username/password; token-only storage
never attempts a login on 401.

* fix: call tracker from notifyProgress instead of recursing

The helper called itself, overflowing the stack on any batch
task with a progress tracker.

* fix: propagate cancellation past IgnoreErrors

Cancelled tasks must not be reported as successful: only
ordinary element failures are ignored.

* test: cover notifyProgress tracker call

* test: cover cancellation with IgnoreErrors

* fix: refresh alist token after startup 401s

The init login no longer satisfies the dedup window, so an
early 401 triggers a real refresh; later 401s reuse it.
Clarify that streaming uploads cannot replay their body.

* test: exercise the alist 401 refresh path

Concurrent uploads now reject the init token, share one
refresh and retry with the new token; token-only stays inert.

* style: trim verbose comments

Drop process-style explanations; keep one-line behavior notes.

* style: gofmt test files

* fix: drop unbounded saved-path cache in telegram storage

Telegram cannot reliably query remote file existence, so the
cache answered a question it could not answer and grew without
bound. Exists returns false again, as before.

* style: format codes
2026-08-17 19:10:03 +08:00
Haopeng Huo 0e6ed66ef5 feat: show upload progress for batch file tasks (#228)
* feat: show upload progress for file tasks

* feat: show upload progress for batch file tasks

* style: distinguish download and upload phases

* style: mark successful task completion

* fix: preserve storage save error context

* fix: serialize single-file progress updates

* fix: stabilize batch upload progress reporting

Correct batch upload totals, completion state, actual file sizes, and synchronized progress snapshots. Add regression coverage for concurrent updates and phase transitions.

* feat: show per-file transfer progress

Format single-file and batch download and upload states with Telegram entities, blockquotes, speeds, transferred sizes, progress bars, concise counters, and accurate confirmation handling. Cover upload retries plus final, error, and cancellation messages with regression tests.

* chore: remove transfer progress tests

* test: restore critical transfer progress coverage

* test: cover interleaved batch transfers

* fix: declare progress styles in locale templates
2026-08-11 08:40:29 +08:00
krau 2bc460c609 feat: add preset rule import functionality and update related messages 2026-06-26 15:39:14 +08:00
Ryan 62e4a08e28 Merge pull request #204 from Rain-kl/feat/save-strategy
新增功能: 重名文件时选择保存策略 - 重命名,覆盖,跳过
2026-05-20 16:11:00 +08:00
krau dd0dea8cb5 feat!: Refactor batch import task to transfer task
- Updated command usage in English and Simplified Chinese localization files to reflect changes in transfer command syntax.
- Removed batch import task implementation, replacing it with a new transfer task implementation.
- Introduced new task structure and progress tracking for file transfers.
- Updated task type enumeration to replace batch import with transfer.
- Added new fields in data structures to support transfer operations.
- Implemented file handling and progress reporting for the transfer task.
2026-01-19 21:14:01 +08:00
krau 3d20fbd0fe feat: implement transfer command for file transfers between storages 2026-01-19 21:01:50 +08:00
KrauandCopilot f92c43b9c8 feat: support import files from storages to telegram (#183)
* feat: add import command and batch import functionality

- Implemented the `/import` command to allow users to import files from storage to Telegram.
- Added support for listing files in storage and filtering based on regex patterns.
- Created a batch import task to handle multiple file uploads concurrently.
- Introduced progress tracking for batch imports, providing real-time updates to users.
- Enhanced storage interfaces to support file listing and reading capabilities.
- Updated localization files for the new import command and its usage instructions.
- Added utility functions for file size formatting and speed calculation.
- Refactored Telegram storage handling to support reading from non-seekable streams.

* feat: add  i18n for import command

* feat: implement ListFiles and OpenFile methods for WebDAV and Alist storage

* Update common/i18n/locale/zh-Hans.yaml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update core/tasks/batchimport/progress.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update core/tasks/batchimport/execute.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update storage/alist/alist.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update common/i18n/locale/en.yaml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update pkg/storagetypes/fileinfo.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update common/i18n/locale/en.yaml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update core/tasks/batchimport/execute.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update storage/webdav/webdav.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update storage/telegram/telegram.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update core/tasks/batchimport/execute.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update storage/webdav/webdav.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: missing progress stats i18n

* refactor: use strutil to parse args

* chore: update generated code files for consistency

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-01-19 17:39:47 +08:00
krau 3ce00884a0 feat: add yt-dlp support for downloading video/audio and enhance related commands 2026-01-17 17:42:11 +08:00
krau b05d86509c feat: basic aria2 integration 2026-01-17 14:57:03 +08:00
krau 75e5fd10ea feat(aria2): add Aria2 download command and client integration 2026-01-03 17:40:55 +08:00
krau c8d8a2e0eb feat: add command to sync peer chats with userbot 2026-01-03 16:32:58 +08:00
krau c069d1edc7 fix: format the generated i18n keys 2025-12-21 13:03:50 +08:00
krau fd3b17a360 fix(i18n): add cancel button text 2025-12-21 12:59:14 +08:00
doomwithdon a3162eff89 Add BotMsgCancelCancelTask key for bot messages (#162) 2025-12-21 12:56:42 +08:00
krau 0b97d15b2a feat(i18n): Integrate internationalization for bot commands and add English locale support 2025-12-19 20:48:50 +08:00
krau 6db5633ca6 feat(i18n): Add progress tracking messages for various tasks and enhance error handling 2025-12-19 20:40:14 +08:00
krau 6e16dc6051 feat: Refactor bot handlers to utilize i18n for internationalization
- Updated silent.go to replace hardcoded strings with i18n keys for user feedback.
- Enhanced telegraph.go to use i18n for error messages and prompts.
- Modified update.go to implement i18n for version update notifications and errors.
- Refactored dir.go to utilize i18n for help messages related to directory operations.
- Updated parse.go to replace static text with i18n keys for parsed text entities.
- Enhanced rule.go to use i18n for rule management help messages.
- Refactored storage.go to implement i18n for storage selection prompts.
- Updated task.go to utilize i18n for task addition notifications.
- Added new i18n keys in keys.go for various messages.
- Updated Chinese localization in zh-Hans.yaml to reflect new i18n keys and messages.
2025-12-19 20:20:27 +08:00
krau 9ed2c425be feat(i18n): Enhance internationalization support and error handling messages
- Added i18n keys for various error and info messages related to task management, user handling, and storage operations.
- Updated error messages in the watch and task handling commands to use i18n for better localization.
- Refactored error handling in the database initialization and user synchronization processes to provide clearer messages.
- Improved logging messages for better clarity and consistency across the application.
- Added comprehensive documentation for AI collaboration and project structure in the repository.
2025-12-19 16:01:36 +08:00
krau 215e082028 feat: implement internationalization support and update help commands 2025-08-27 11:09:38 +08:00
Krau 900823cdb9 refactor: refactor task logic for better scalability (#76)
* refactor: a big refactor. wip

* refactor: port handle file

* refactor: place all handlers

* fix: task info nil pointer

* feat: enhance task progress tracking and context management

* feat: cancel task

* feat: stream mode

* feat: silent mode

* feat: dir cmd

* refactor: remove unused old file

* feat: rule cmd

* feat: handle silent mode

* feat: batch task

* fix: batch task progress and temp file cleanup

* refactor: update file creation and cleanup methods for better resource management

* feat: add save command with silent mode handling

* feat: message link

* feat: update message prompts to include file count in storage selection

* feat: slient save links

* refactor: reduce dup code

* feat: rule type

* feat: chose dir

* feat: refactor file handling and storage rules, improve error handling and logging

* feat: rule mode

* feat: telegraph pics

* fix: tphpics nil pointer and inaccurate dirpath

* feat: silent save telegraph

* feat: add suffix to avoid file overwrite

* feat: new storage telegram

* chore: tidy go mod
2025-06-15 23:57:49 +08:00