Compare commits

..
133 Commits
Author SHA1 Message Date
jxxghp 1d708870c9 fix: avoid blocking plugins on pre-existing pip errors 2026-07-21 08:25:03 +08:00
jxxghp 053e1b7562 fix: handle files in newly moved monitor directories 2026-07-21 08:22:46 +08:00
4ca3e40507 修复共享内存缓存的逐键 TTL (#6161)
MemoryBackend 原先为每个 region 保存一个 cachetools.TTLCache,导致首次写入的 TTL 成为后续所有 key 的统一生命周期。p115strmhelper 等动态 TTL 调用方因此可能把短期有效的 URL 缓存数天。

改用 TLRUCache 为每次写入独立计算过期时间,同时保留同步与异步 backend 共享 region、缓存值原样返回等既有行为。

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: jxxghp <jxxghp@gmail.com>
2026-07-21 08:10:20 +08:00
jxxghpandtraeagent 318d2ab7d7 feat: 修复单元测试失败 (#6165)
Co-authored-by: traeagent <traeagent@users.noreply.github.com>
2026-07-21 07:41:34 +08:00
Ekko 7c2390908a Fix/memory cache ttl (#6160) 2026-07-21 07:08:21 +08:00
InfinityPacer 5c2b503a74 fix(subscribe): protect runtime facts in public writes (#6157) 2026-07-20 16:44:49 +08:00
jxxghp 44fa202778 fix: map Bangumi movie subjects correctly (#6155) 2026-07-20 16:16:25 +08:00
jxxghp ed92be08af fix(transfer): support nested download categories (#6156) 2026-07-20 16:14:17 +08:00
jxxghp 9ed0704c5b fix: 重识别时清理旧下载器上下文 2026-07-20 15:35:56 +08:00
jxxghp e46b4e5ba0 fix: 重新整理时忽略旧下载哈希 (#6154) 2026-07-20 15:31:54 +08:00
jxxghp 87ad7988b2 Merge remote-tracking branch 'origin/v2' into v2 2026-07-20 15:19:49 +08:00
jxxghp 1382975b18 Centralize optional dependency stubs in test bootstrap 2026-07-20 14:53:36 +08:00
InfinityPacer d9a42c672a fix(subscribe): refine TV wash priority lifecycle (#6153) 2026-07-20 14:02:19 +08:00
jxxghp b042086efa test(indexer): anonymize TorrentLeech fixture 2026-07-20 13:49:02 +08:00
jxxghp 36fefa14e0 fix(indexer): repair TorrentLeech user data parsing 2026-07-20 13:46:25 +08:00
jxxghp 1332576c3f fix(download): accept legacy remote subscription paths (#6147) 2026-07-20 13:42:27 +08:00
jxxghp 4300af0e9c chore: update version to v2.14.6 2026-07-20 12:58:41 +08:00
jxxghp 405350c774 Preserve plugin diagnostics without degrading overall status 2026-07-20 10:15:15 +08:00
jxxghp d666134ed2 Route autonomous agent tasks through notification broadcasting 2026-07-20 08:54:32 +08:00
InfinityPacer 5588e37c6d fix: preserve season zero semantics (#6150) 2026-07-20 06:55:23 +08:00
InfinityPacer 2056aa0b2c fix(startup): keep systemd autostart services alive (#6149) 2026-07-20 06:54:09 +08:00
jxxghp b0ff3ae3c7 Fix Agent task relative-time validation (#6146) 2026-07-19 21:07:27 +08:00
jxxghp 31544629b4 Add autonomous Agent task scheduling 2026-07-19 20:19:06 +08:00
InfinityPacer 142393f2d3 fix(subscribe): reset manual episode tracking (#6144) 2026-07-19 17:36:54 +08:00
Xuanjie Xia 5cf79e0360 feat(site): 模拟登录支持勾选「记住我/保持登录」选项,获取长期会话 (#6138) 2026-07-18 19:24:18 +08:00
Xuanjie Xia f152a0381d fix(site): 模拟登录适配JS渲染(SPA)登录页 (#6137) 2026-07-18 16:47:24 +08:00
InfinityPacer 428c19b6ba fix(download): restore classification for configured roots (#6135) 2026-07-18 09:35:25 +08:00
jxxghp 8b5524a321 更新 version.py 2026-07-17 14:50:58 +08:00
jxxghp b972b46747 Merge remote-tracking branch 'origin/v2' into v2 2026-07-17 09:47:01 +08:00
jxxghp 0598fbdd75 fix(media): add support for custom words in media recognition 2026-07-17 09:46:55 +08:00
freeman 572299a45e feat(llm): 新增 Amazon Bedrock 提供商,支持 AK/SK 与 Bedrock API Key 双认证 (#6130) 2026-07-16 19:27:06 +08:00
kuke2733 229824a417 feat(subscribe): merge mediaserver library entries into files info (#6131) 2026-07-16 17:36:56 +08:00
DDSRem a0ee99aacc chore: bump moviepilot-rust to 0.2.3 (#6128) 2026-07-16 06:31:31 +08:00
InfinityPacer 92918ce380 ci(pr-agent): use shared review runner (#6127) 2026-07-16 06:24:52 +08:00
InfinityPacer a4335fe753 fix(lifecycle): harden application shutdown (#6125) 2026-07-16 06:24:31 +08:00
jxxghp 107ba37834 更新 version.py 2026-07-15 20:22:27 +08:00
jxxghp c27678ce06 fix(ugreen): send client id during login 2026-07-15 17:38:31 +08:00
InfinityPacer 7725342a80 fix(scheduler): refresh plugin jobs after reload (#6124) 2026-07-15 17:29:31 +08:00
InfinityPacer 893269f8c1 fix(modules): serialize configuration reload lifecycle (#6122) 2026-07-15 17:28:49 +08:00
InfinityPacer 00d46f3aab docs: clarify docstring punctuation style (#6121) 2026-07-15 16:01:47 +08:00
jxxghp 077241b6ed Merge remote-tracking branch 'origin/v2' into v2 2026-07-15 10:46:28 +08:00
jxxghp b24a07e388 fix: enhance response data structure in filtering rules with media info 2026-07-15 10:46:21 +08:00
InfinityPacer f814c271cc refactor(runtime): tighten resource cleanup and test isolation (#6116) 2026-07-14 16:03:29 +08:00
InfinityPacer e015c67689 chore(db): add driver error diagnostics (#6115) 2026-07-14 12:31:46 +08:00
qqcomeup 98b16bda8d 优化 Docker 启动完成日志 (#6112) 2026-07-14 12:31:07 +08:00
jxxghp b8233e1789 v2.14.3 2026-07-13 18:46:35 +08:00
InfinityPacer 83107bf447 ci(pr-agent): publish native code reviews (#6110) 2026-07-13 18:41:18 +08:00
jxxghp 3a2f90c567 fix(metainfo): improve regex for episode range recognition with end markers 2026-07-13 18:02:44 +08:00
DDSRem 4826e3301c chore: bump moviepilot-rust to 0.2.2 (#6109) 2026-07-13 18:00:50 +08:00
freeman 1855ba81ec fix(meta): 副标题识别 01-26Fin 等数字范围完结标记集数 (#6105) 2026-07-13 16:50:00 +08:00
jxxghp 96ef431efc feat: 支持豆瓣识别缓存管理 2026-07-13 12:33:56 +08:00
freeman 4f2935c85e fix(transfer): 种子未下载完成时不回写已整理标签 (#6106) 2026-07-13 12:05:01 +08:00
freeman 2a49495e27 fix(jellyfin): 媒体统计改为按用户视图逐库累计 (#5915) (#6104) 2026-07-13 11:51:16 +08:00
jxxghp b628bc7209 fix: 补齐识别缓存多语言响应 2026-07-13 09:58:11 +08:00
jxxghp 29068a5846 feat: 支持 TheMovieDb 识别缓存管理 2026-07-13 09:48:08 +08:00
jxxghp 51a7120c79 完善 qBittorrent 临时标签清理 (#6093) 2026-07-12 16:51:46 +08:00
jxxghp 476dfef7d9 修复 qBittorrent 重复任务临时标签残留 (#6093) 2026-07-12 16:47:12 +08:00
jxxghp bd5ddd6158 fix: remove standalone site collector download section from README 2026-07-12 16:38:53 +08:00
jxxghp a30a48b8f4 fix: allow publishing collector artifacts manually 2026-07-12 16:37:05 +08:00
jxxghp 8e60e5571b fix: support Windows collector console encoding 2026-07-12 16:30:30 +08:00
jxxghp 18c1ec4b82 feat: add standalone site adapter collector 2026-07-12 13:51:07 +08:00
InfinityPacer 30b932e07e fix(subscribe): preserve confirmed episode floor (#6102) 2026-07-12 07:18:29 +08:00
InfinityPacer 54be1143fc feat(plugin): sync federated assets during local development (#6100) 2026-07-11 21:59:53 +08:00
秋澪Akimio 13f27854fd fix: clear Rust parse options cache after updating custom identifiers (#6097) 2026-07-11 18:15:26 +08:00
InfinityPacer 770201c48c fix(plugin): exclude build dependencies from runtime copies (#6096) 2026-07-11 18:15:00 +08:00
Xuanjie Xia 685f044312 fix: 模拟登录时页面跳转导致 page.content() 竞态失败(未知错误) (#6091) 2026-07-10 12:44:36 +08:00
qqcomeup 8c0afac5d1 feat: support prompt-bound plugin input replies (#6087) 2026-07-09 12:52:22 +08:00
qqcomeup 099ef7d5bf fix: avoid blocking plugin release history refresh (#6084) 2026-07-08 12:51:24 +08:00
InfinityPacer f3ac69669c ci(pr-agent): simplify review workflow (#6082) 2026-07-08 12:49:19 +08:00
jxxghp eb4ecd990a fix: restore full test suite 2026-07-08 08:54:53 +08:00
jxxghp b51971ee7d feat: add agent MCP support 2026-07-08 08:44:33 +08:00
InfinityPacer 6f6ed998bb ci(pr-agent): align inline review workflow (#6079) 2026-07-08 07:04:39 +08:00
drdon1234 844407dc41 修复 qBittorrent 已完成但未做种任务识别 (#6076) 2026-07-08 07:01:03 +08:00
jxxghp c54605f8ce fix: support ugreen token_id login response 2026-07-07 20:15:19 +08:00
qqcomeup 0fbf05d72f fix: handle Telegram urllib3 header formatter compatibility (#6074) 2026-07-07 19:58:49 +08:00
jxxghp 09bb32f681 fix: cool down failed subscription resources 2026-07-07 17:07:13 +08:00
qqcomeup a37f118576 perf(docker): skip image path chown by default (#6071) 2026-07-07 16:25:43 +08:00
jxxghp e635bc8e04 chore: bump version to v2.14.2 2026-07-07 12:48:23 +08:00
jxxghp 8245124e82 support scheduler progress i18n (#6069) 2026-07-07 06:04:45 +08:00
qqcomeup 827ed8330c feat: add plugin-scoped text input sessions (#6068) 2026-07-07 05:15:18 +08:00
jxxghp 136c1baed3 fix: keep new subscribe state during guard 2026-07-06 23:10:25 +08:00
jxxghp 992031ef95 Revert "fix: trigger initial subscribe search"
This reverts commit 76803ae7a3.
2026-07-06 19:15:16 +08:00
jxxghp b16c50b03a feat: add backend i18n response support 2026-07-06 19:13:50 +08:00
jxxghp 76803ae7a3 fix: trigger initial subscribe search 2026-07-06 19:02:01 +08:00
jxxghp 56bda11947 fix python metainfo xxx title parsing 2026-07-06 12:36:06 +08:00
jxxghp 1b12d7664e fix: avoid cnsub matching file size unit 2026-07-06 11:58:03 +08:00
jxxghp db9960d9b9 refactor: enhance customization and streaming platform handling 2026-07-06 09:33:49 +08:00
InfinityPacer 2f0c1252da feat(subscribe): expose cached site candidates (#6062) 2026-07-06 06:54:19 +08:00
jxxghp 36d4434596 fix: allow manage users to access management APIs 2026-07-05 20:23:04 +08:00
jxxghp 93e907d032 fix: remove openlist directory via fs remove 2026-07-05 17:24:44 +08:00
jxxghp 132f27c1c6 fix: retry qbittorrent files after add 2026-07-05 17:15:07 +08:00
jxxghp b231ad415f fix title search filter rules 2026-07-05 17:02:19 +08:00
jxxghp 0f183ae08e test: fix download save path fixtures 2026-07-05 14:56:19 +08:00
jxxghp a71d3ea03f feat: support llm temperature override 2026-07-05 13:04:51 +08:00
InfinityPacer 7f82a9ea4d fix(security): scope subscriptions to owner (#6056) 2026-07-05 09:45:27 +08:00
InfinityPacer d977e4c48a fix(security): require signed subtitle downloads (#6055) 2026-07-05 09:43:58 +08:00
InfinityPacer 95b6adbeee fix(security): restrict download save paths (#6054) 2026-07-05 09:31:01 +08:00
InfinityPacer 964fee1106 fix(security): protect cookiecloud update uploads (#6053) 2026-07-05 09:28:07 +08:00
InfinityPacer 656473f3aa fix(security): require manage access for workflows (#6052) 2026-07-05 09:27:24 +08:00
InfinityPacer ab5995a609 fix(security): contain plugin zip extraction (#6051) 2026-07-05 09:26:09 +08:00
InfinityPacer 064e6535d5 fix(security): restrict agent log file access (#6050) 2026-07-05 09:25:25 +08:00
InfinityPacer cab2ac400a fix(security): rebind resource cookie on user changes (#6049) 2026-07-05 09:24:27 +08:00
InfinityPacer d14d401c86 docs(security): clarify api token admin contract (#6048) 2026-07-05 09:22:59 +08:00
Album 6c3c5e042d 修复手动整理按集数定位模板过滤 (#6043) 2026-07-03 07:55:59 +08:00
Copilot f3e5be37fd chore: bump backend and frontend versions (#6042) 2026-07-02 20:50:15 +08:00
jxxghp d8f7fa70af fix: normalize question marks in title matching 2026-07-02 08:51:16 +08:00
jxxghp 6916ee0988 修复电影合集整理识别错误 2026-07-02 08:06:44 +08:00
LeChristopher Blackwell 6fef533527 fix(security): honor X-Forwarded-Proto when setting resource_token cookie secure flag (#6038) 2026-07-01 22:00:22 +08:00
G0m3eandCursor c57985d553 fix(webpush): add WNS cache policy for Windows Edge push (#6034)
WNS rejects pywebpush default ttl=0 with 400 Bad Request unless X-WNS-Cache-Policy matches TTL; iOS/APNs endpoints are unaffected.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-01 10:45:24 +08:00
InfinityPacer ec07379a67 fix(subscribe): record movie completion priority (#6033) 2026-07-01 06:01:32 +08:00
InfinityPacer 2764742b86 fix(plugin): reset stopped plugin config and data (#6031) 2026-06-30 22:37:07 +08:00
jxxghp a0f613fa1e chore(version): update app and frontend versions to v2.14.0 2026-06-30 20:32:17 +08:00
jxxghp 73d5c95f4e Fix Telegram edit fallback 2026-06-29 22:48:13 +08:00
jxxghp 4d30dee74c feat(mediaserver): add item count to media library details across multiple servers 2026-06-29 21:37:58 +08:00
Pollo3470 302d8bbf5c fix(transfer): 修复订阅自定义识别词在整理时失效 (#6018) 2026-06-29 15:49:02 +08:00
jxxghp b646cbb4f6 Merge remote-tracking branch 'origin/v2' into v2 2026-06-29 12:55:19 +08:00
jxxghp dd73b97095 fix(base): improve permission error message for tool access 2026-06-29 12:55:14 +08:00
InfinityPacer 0cb0bac0e1 fix(llm): sanitize model endpoint errors (#6021) 2026-06-29 11:19:38 +08:00
jxxghp 9eb71c744b feat(mediaserver): add item count retrieval for various media servers 2026-06-29 10:45:10 +08:00
jxxghp 8bf826faa0 fix(dashboard): report application memory accurately 2026-06-29 07:37:03 +08:00
jxxghp df4e45c644 Merge remote-tracking branch 'origin/v2' into v2 2026-06-29 07:08:08 +08:00
jxxghp 494f809ef0 Refine transfer history batch actions 2026-06-29 07:07:33 +08:00
InfinityPacer a4f6e13881 fix(transfer): expose file meta in intercept event (#6019) 2026-06-29 06:44:40 +08:00
InfinityPacer 36fb82b7aa feat(subscribe): record episode download facts (#6015) 2026-06-29 06:43:51 +08:00
jxxghp 9b1bdb0cb2 feat(dashboard): add system summary endpoint and monthly media statistics 2026-06-28 17:49:09 +08:00
DDSRemandjxxghp 2a89bfd25c chore: bump moviepilot-rust to 0.1.14 (#6016)
Co-authored-by: jxxghp <51039935+jxxghp@users.noreply.github.com>
2026-06-28 17:31:00 +08:00
jxxghp d3d1c18316 chore: bump application and frontend versions to v2.13.16 2026-06-28 15:51:04 +08:00
jxxghp 027330f714 fix: disable notification history saving for agent messages 2026-06-28 15:25:52 +08:00
jxxghp 6fc1c672ad Merge remote-tracking branch 'origin/v2' into v2 2026-06-28 08:32:12 +08:00
InfinityPacer d383c9ffd1 feat(subscribe): add modified event payload contract (#6012) 2026-06-28 07:48:25 +08:00
jxxghp 27e1f634cb fix: add total episodes to transfer notifications 2026-06-27 22:45:17 +08:00
jxxghp a9197c434e Skip download dir selection for single matches 2026-06-27 22:29:55 +08:00
jxxghp 2a8708498c Simplify agent tool schema explanation test 2026-06-27 22:21:45 +08:00
291 changed files with 32342 additions and 2362 deletions
+15 -1
View File
@@ -7,11 +7,13 @@ body:
attributes:
value: |
请说明你希望添加的功能。
站点适配请求请先按 [站点适配采集说明](https://github.com/jxxghp/MoviePilot/blob/v2/docs/site-adapter-capture.md) 生成脱敏 ZIP,并在下方附加。Issue 及附件是公开内容,提交前必须解压预览四个文件。不要上传 Cookie、Authorization、通行密钥、会话字段或任何原始数据。
- type: input
id: version
attributes:
label: 当前程序版本
description: 目前使用的程序版本
description: 目前使用的程序版本;仅提供站点采集文件且未安装 MoviePilot 时填写“不适用”
validations:
required: true
- type: dropdown
@@ -22,6 +24,9 @@ body:
options:
- Docker
- Windows
- macOS
- Linux
- 仅提供站点采集文件
validations:
required: true
- type: dropdown
@@ -32,6 +37,7 @@ body:
options:
- 主程序
- 插件
- 站点适配
- 其他
validations:
required: true
@@ -43,6 +49,14 @@ body:
placeholder: "功能改进"
validations:
required: true
- type: textarea
id: site-adapter-capture
attributes:
label: 站点适配采集文件
description: 站点适配请求必须把采集器生成并人工预览确认过的脱敏 ZIP 拖到这里;Issue 附件公开,严禁附加 Cookie、原始 HTML、HAR 或浏览器网络归档。其他类型请填写“不适用”。
placeholder: "将 moviepilot-site-capture-*.zip 拖到这里;非站点适配填写:不适用"
validations:
required: true
- type: textarea
id: references
attributes:
+15 -86
View File
@@ -1,9 +1,8 @@
name: PR Agent
name: PR-Agent
on:
pull_request_target:
# PR-Agent 通过 base repo 上下文读取 PR diff 并发布 Review,不 checkout 或执行 PR 分支代码。
# pull_request_target 允许 fork PR 使用仓库 secrets,因此 workflow 只运行固定 digest 的 PR-Agent 容器。
# Fork 审查需要目标仓库凭据;该 job 仅通过 GitHub API 读取 PR 内容,不 checkout 或执行 PR 分支代码。
types:
- opened
- reopened
@@ -11,24 +10,17 @@ on:
- review_requested
- synchronize
issue_comment:
# 手动命令如 "/review"、"/describe"、"/improve" 和 "/ask ..." 只在 PR 评论中有意义。
# issue_comment 同时覆盖普通 issue,因此 job 里还会再判断是否属于 PR。
types:
- created
- edited
permissions:
# 读取仓库内容和 PR diff。
contents: read
# 更新 PR 描述、发布 PR Review 或修改 PR 相关元数据。
pull-requests: write
# PR 评论在 GitHub API 中属于 issue comments,手动命令和总结评论需要该权限。
issues: write
jobs:
pr-agent:
name: PR-Agent review and describe
# PR 事件自动处理;评论命令仅允许指定身份在 PR 下触发,避免任意评论消耗模型配额。
if: >-
github.event.sender.type != 'Bot' &&
(
@@ -36,88 +28,25 @@ jobs:
(
github.event_name == 'issue_comment' &&
github.event.issue.pull_request != null &&
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR", "CONTRIBUTOR", "FIRST_TIME_CONTRIBUTOR"]'), github.event.comment.author_association) &&
(
github.event.comment.body == '/review' ||
startsWith(github.event.comment.body, '/review ') ||
github.event.comment.body == '/describe' ||
startsWith(github.event.comment.body, '/describe ') ||
github.event.comment.body == '/improve' ||
startsWith(github.event.comment.body, '/improve ') ||
github.event.comment.body == '/ask' ||
startsWith(github.event.comment.body, '/ask ')
)
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR", "CONTRIBUTOR", "FIRST_TIME_CONTRIBUTOR"]'), github.event.comment.author_association)
)
)
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.event.issue.number }}
cancel-in-progress: ${{ github.event_name == 'pull_request_target' }}
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Run PR-Agent
id: pragent
# 使用版本号加 digest 固定容器构建,避免 tag 被重推后改变运行内容。
uses: docker://pragent/pr-agent:0.37.0-github_action@sha256:4ec7bac814050a1bc8c96ab2fab6b7b0f65df0049a5ec43f3fee1a0b551c28ca
- name: Run PR Review
uses: docker://ghcr.io/infinitypacer/pr-review-runner:latest
env:
# PR-Agent 使用该 token 读取 PR 元数据并发布评论。
GITHUB_TOKEN: ${{ github.token }}
# 仓库设置中添加的 SecretSettings -> Secrets and variables -> Actions。
# 该 key 只传给 PR-Agent 运行时,不写入仓库。
OPENAI_KEY: ${{ secrets.OPENAI_KEY }}
# 仓库设置中添加的 Secret。OpenAI 兼容服务通常需要填写以 "/v1" 结尾的 API 根地址。
OPENAI.API_BASE: ${{ secrets.OPENAI_API_BASE }}
# 模型、输出语言和大 diff 处理策略。
config.model: "gpt-5.5"
config.fallback_models: '["gpt-5.4"]'
config.reasoning_effort: "xhigh"
config.ai_timeout: "900"
config.response_language: "zh-CN"
config.large_patch_policy: "clip"
config.ignore_pr_title: '["^\\[Auto\\]", "^Auto"]'
config.ignore_pr_labels: '["skip pr-agent"]'
# pull_request_target 事件默认自动执行 /review 和 /describe/improve 保持手动触发。
github_action_config.auto_review: "true"
github_action_config.auto_describe: "true"
github_action_config.auto_improve: "false"
# 允许触发自动工具的 PR 动作。包含 synchronize,便于新 commit 推送后刷新结果。
github_action_config.pr_actions: '["opened", "reopened", "ready_for_review", "review_requested", "synchronize"]'
# 保留 action outputs,便于后续 workflow 编排或排查。
github_action_config.enable_output: "true"
# /describe 行为控制;与自动触发配置放在同一层,避免使用默认图表和标签策略。
pr_description.generate_ai_title: "false"
pr_description.publish_labels: "false"
pr_description.enable_pr_diagram: "false"
pr_description.collapsible_file_list: "adaptive"
pr_description.add_original_user_description: "true"
# /review 输出策略,聚焦维护者需要处理的风险和缺口。
pr_reviewer.extra_instructions: |
请用中文输出。
优先指出 P0/P1 风险,避免纠结纯格式问题。
重点检查安全、权限、状态一致性、异步/缓存、副作用和测试缺口。
pr_reviewer.num_max_findings: "5"
pr_reviewer.persistent_comment: "true"
pr_reviewer.publish_output_no_suggestions: "true"
pr_reviewer.require_tests_review: "true"
pr_reviewer.require_security_review: "true"
pr_reviewer.require_estimate_effort_to_review: "true"
pr_reviewer.require_can_be_split_review: "true"
pr_reviewer.require_todo_scan: "false"
pr_reviewer.enable_review_labels_effort: "false"
pr_reviewer.enable_review_labels_security: "true"
# /improve 和 /ask 的手动命令策略。
pr_code_suggestions.focus_only_on_problems: "true"
pr_code_suggestions.suggestions_score_threshold: "7"
pr_code_suggestions.commitable_code_suggestions: "false"
pr_questions.use_conversation_history: "true"
# 可选成本和噪音控制:
# github_action_config.auto_improve: "true"
# config.verbosity_level: "1"
# pr_reviewer.num_max_findings: "3"
PRR_AUTO_REVIEW_SCOPE: all
PRR_ALLOWED_ASSOCIATIONS: '["OWNER", "MEMBER", "COLLABORATOR", "CONTRIBUTOR", "FIRST_TIME_CONTRIBUTOR"]'
PRR_DISABLED_COMMANDS: '["/improve"]'
PRR_SKIP_LABEL: skip pr-agent
PRR_SKIP_TITLE_PATTERN: '^(?:\[Auto\]|Auto)'
config.response_language: zh-CN
@@ -0,0 +1,134 @@
name: Site Adapter Collector
on:
workflow_dispatch:
inputs:
release_tag:
description: Existing release tag to receive collector assets; leave empty for artifacts only
required: false
type: string
release:
types:
- published
permissions:
contents: read
jobs:
build:
name: Build ${{ matrix.platform_name }} collector
runs-on: ${{ matrix.runner }}
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
include:
- platform_name: Windows
platform_id: windows
runner: windows-latest
source_name: moviepilot-site-collector.exe
asset_name: moviepilot-site-collector-windows.exe
artifact_name: site-adapter-collector-windows
- platform_name: macOS
platform_id: macos
runner: macos-latest
source_name: moviepilot-site-collector
asset_name: MoviePilot-Site-Collector-macOS.zip
artifact_name: site-adapter-collector-macos
- platform_name: Linux
platform_id: linux
runner: ubuntu-latest
source_name: moviepilot-site-collector
asset_name: moviepilot-site-collector-linux
artifact_name: site-adapter-collector-linux
steps:
- name: Checkout code
uses: actions/checkout@v7
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: '3.12'
cache: pip
cache-dependency-path: scripts/site_adapter_collector_requirements.txt
- name: Install build dependencies
run: |
python -m pip install --upgrade pip setuptools wheel
pip install -r scripts/site_adapter_collector_requirements.txt
- name: Build single-file collector
run: |
pyinstaller --clean --noconfirm scripts/site_adapter_collector.spec
- name: Smoke-test collector
env:
SOURCE_NAME: ${{ matrix.source_name }}
run: |
python -c "import os, subprocess; from pathlib import Path; subprocess.run([str((Path('dist') / os.environ['SOURCE_NAME']).resolve()), '--help'], check=True)"
- name: Package macOS double-click archive
if: matrix.platform_id == 'macos'
shell: bash
env:
ASSET_NAME: ${{ matrix.asset_name }}
SOURCE_NAME: ${{ matrix.source_name }}
run: |
package_dir="dist/MoviePilot-Collector"
mkdir -p "$package_dir"
cp "dist/$SOURCE_NAME" "$package_dir/moviepilot-site-collector-macos"
cp scripts/start-site-adapter-collector.command "$package_dir/start-site-adapter-collector.command"
chmod +x "$package_dir/moviepilot-site-collector-macos"
chmod +x "$package_dir/start-site-adapter-collector.command"
cd dist
COPYFILE_DISABLE=1 zip -q -r -X "$ASSET_NAME" MoviePilot-Collector
- name: Rename Windows and Linux collector
if: matrix.platform_id != 'macos'
env:
ASSET_NAME: ${{ matrix.asset_name }}
SOURCE_NAME: ${{ matrix.source_name }}
run: |
python -c "import os; from pathlib import Path; (Path('dist') / os.environ['SOURCE_NAME']).replace(Path('dist') / os.environ['ASSET_NAME'])"
- name: Generate SHA-256 checksum
env:
ASSET_NAME: ${{ matrix.asset_name }}
run: |
python -c "import hashlib, os; from pathlib import Path; path = Path('dist') / os.environ['ASSET_NAME']; path.with_name(path.name + '.sha256').write_text(f'{hashlib.sha256(path.read_bytes()).hexdigest()} {path.name}\n', encoding='utf-8')"
- name: Upload collector artifact
uses: actions/upload-artifact@v7
with:
name: ${{ matrix.artifact_name }}
path: |
dist/${{ matrix.asset_name }}
dist/${{ matrix.asset_name }}.sha256
if-no-files-found: error
retention-days: 3
publish:
name: Upload collectors to release
if: github.event_name == 'release' || inputs.release_tag != ''
needs:
- build
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Download collector artifacts
uses: actions/download-artifact@v8
with:
pattern: site-adapter-collector-*
path: release-assets
merge-multiple: true
- name: Upload assets to published release
env:
GH_TOKEN: ${{ github.token }}
RELEASE_TAG: ${{ github.event.release.tag_name || inputs.release_tag }}
run: |
gh release view "$RELEASE_TAG" --repo "$GITHUB_REPOSITORY" >/dev/null
gh release upload "$RELEASE_TAG" release-assets/* --clobber --repo "$GITHUB_REPOSITORY"
+1
View File
@@ -37,6 +37,7 @@ coverage.json
htmlcov/
.vscode
venv
moviepilot-site-capture-*.zip
# Pylint
pylint-report.json
+1
View File
@@ -59,6 +59,7 @@ curl -fsSL https://raw.githubusercontent.com/jxxghp/MoviePilot/v2/scripts/bootst
- 文档规则入口:[docs/rules/README.md](docs/rules/README.md)
- 开发环境与本地源码运行:[docs/development-setup.md](docs/development-setup.md)
- 测试说明:[docs/testing.md](docs/testing.md)
- 新站点适配采集与 Feature Request 提交:[docs/site-adapter-capture.md](docs/site-adapter-capture.md)
- REST API 文档:https://api.movie-pilot.org
- 插件开发说明:https://wiki.movie-pilot.org/zh/plugindev
+1
View File
@@ -58,6 +58,7 @@ Before contributing, read the repository rules and local environment guide, keep
- Rule index: [docs/rules/README.md](docs/rules/README.md)
- Development setup and local source run: [docs/development-setup.md](docs/development-setup.md)
- Testing guide: [docs/testing.md](docs/testing.md)
- New site adapter capture and Feature Request submission: [docs/site-adapter-capture.md](docs/site-adapter-capture.md)
- REST API documentation: https://api.movie-pilot.org
- Plugin development guide: https://wiki.movie-pilot.org/zh/plugindev
+139 -15
View File
@@ -51,11 +51,14 @@ from app.agent.middleware.tool_selection import ToolSelectorMiddleware
from app.agent.middleware.usage import UsageMiddleware
from app.agent.prompt import prompt_manager
from app.agent.runtime import agent_runtime_manager
from app.agent.mcp import agent_mcp_manager
from app.agent.tools.factory import MoviePilotToolFactory
from app.agent.tools.impl.mcp import create_external_mcp_tools
from app.chain import ChainBase
from app.core.config import settings
from app.core.event import eventmanager
from app.db.agentchat_oper import AgentChatOper
from app.db.agenttask_oper import AgentTaskOper
from app.db.user_oper import UserOper
from app.log import logger
from app.schemas import AgentLLMProviderEventData, AgentTokensUsageEventData, Notification, NotificationType
@@ -1041,6 +1044,7 @@ class MoviePilotAgent:
settings.LLM_MAX_ITERATIONS,
self._public_runtime_config_signature(runtime_config),
agent_runtime_manager.current_signature(),
agent_mcp_manager.config_signature(),
)
def _get_cached_agent(
@@ -1097,6 +1101,39 @@ class MoviePilotAgent:
allow_message_tools=False,
)
async def _initialize_mcp_tools(self) -> List:
"""
初始化外部 MCP 工具列表。
"""
return await create_external_mcp_tools(
session_id=self.session_id,
user_id=self.user_id,
channel=self.channel,
source=self.source,
username=self.username,
stream_handler=self.stream_handler,
agent_context=self._tool_context,
)
async def _initialize_subagent_mcp_tools(self) -> List:
"""
初始化子代理可用的外部 MCP 工具列表。
"""
return await create_external_mcp_tools(
session_id=self.session_id,
user_id=self.user_id,
channel=self.channel,
source=self.source,
username=self.username,
stream_handler=None,
agent_context={
"user_reply_sent": False,
"reply_mode": None,
"should_dispatch_reply": False,
"is_admin": bool(self._tool_context.get("is_admin")),
},
)
async def _create_agent(self, streaming: bool = False):
"""
创建 LangGraph Agent(使用 create_agent + SummarizationMiddleware
@@ -1126,6 +1163,7 @@ class MoviePilotAgent:
# 工具列表
tools = self._initialize_tools()
tools.extend(await self._initialize_mcp_tools())
skills_middleware = SkillsMiddleware(
sources=[str(agent_runtime_manager.skills_dir)],
bundled_skills_dir=str(settings.ROOT_PATH / "skills"),
@@ -1142,9 +1180,11 @@ class MoviePilotAgent:
activity_log_tools = list(
getattr(activity_log_middleware, "tools", []) or []
)
subagent_tools = self._initialize_subagent_tools()
subagent_tools.extend(await self._initialize_subagent_mcp_tools())
subagent_middlewares, subagent_task_tools = create_subagent_middlewares(
model=non_streaming_model,
tools=self._initialize_subagent_tools(),
tools=subagent_tools,
stream_handler=self.stream_handler,
)
max_tools = settings.LLM_MAX_TOOLS
@@ -1588,22 +1628,24 @@ class MoviePilotAgent:
if not streaming_stopped:
await self.stream_handler.stop_streaming()
async def send_agent_message(self, message: str, title: str = ""):
async def send_agent_message(self, message: str, title: str = "") -> None:
"""
通过原渠道发送消息给用户
发送 Agent 消息;后台任务不绑定原渠道,交由通知链广播。
"""
broadcast = self.is_background
self._save_assistant_display_message_once(message)
await AgentChain().async_post_message(
Notification(
channel=self.channel,
source=self.source,
channel=None if broadcast else self.channel,
source=None if broadcast else self.source,
mtype=NotificationType.Agent,
userid=self.user_id,
username=self.username,
original_message_id=self.original_message_id,
original_chat_id=self.original_chat_id,
userid=None if broadcast else self.user_id,
username=self.username or (settings.SUPERUSER if broadcast else None),
original_message_id=None if broadcast else self.original_message_id,
original_chat_id=None if broadcast else self.original_chat_id,
title=title,
text=message,
save_history=False,
)
)
@@ -1960,12 +2002,11 @@ class AgentManager:
else:
agent = self.active_agents[session_id]
agent.user_id = task.user_id
if task.channel:
agent.channel = task.channel
if task.source:
agent.source = task.source
if task.username:
agent.username = task.username
# 每条队列任务都携带完整消息上下文,None 也必须覆盖,避免后台任务
# 复用会话 Agent 时继续沿用上一条入站消息的渠道。
agent.channel = task.channel
agent.source = task.source
agent.username = task.username
agent.original_message_id = task.original_message_id
agent.original_chat_id = task.original_chat_id
agent.reply_mode = task.reply_mode
@@ -2083,6 +2124,89 @@ class AgentManager:
await agent.cleanup()
memory_manager.clear_memory(session_id, user_id)
async def execute_scheduled_task(self, task_id: int) -> tuple[bool, str]:
"""
按持久化上下文唤醒 Agent 执行自主定时任务并向用户回传结果。
:param task_id: Agent 定时任务 ID
:return: 执行是否成功及结果摘要
"""
if not settings.AI_AGENT_ENABLE:
return False, "AI Agent 未启用"
oper = AgentTaskOper()
task = oper.get(task_id)
if not task or not task.enabled:
return False, "Agent 定时任务不存在或已停用"
if not oper.mark_running(task_id):
return False, "Agent 定时任务当前不可执行"
task_message = (
f"定时任务已按计划触发。请立即完成下面的任务,不要只确认收到,"
f"也不要重复创建同一个定时任务。\n\n"
f"任务名称:{task.name}\n"
f"任务内容:{task.content}\n\n"
"完成后请直接向用户报告本次执行结果;如果无法完成,请说明原因。"
)
success = True
result = ""
notification_username = task.username or settings.SUPERUSER
try:
result = await self.process_message(
session_id=task.session_id,
user_id=task.user_id,
message=task_message,
channel=None,
source=None,
username=notification_username,
original_chat_id=None,
reply_mode=ReplyMode.DISPATCH,
allow_message_tools=True,
wait_for_completion=True,
)
result_text = str(result or "").strip()
success = bool(result_text) and not result_text.startswith(
(AGENT_EXECUTION_ERROR_PREFIX, "处理消息时发生错误")
)
if not result_text:
result = "定时任务已执行,但 Agent 未返回结果"
await AgentChain().async_post_message(
Notification(
mtype=NotificationType.Agent,
username=notification_username,
title=f"定时任务:{task.name}",
text=result,
save_history=False,
)
)
except Exception as err:
success = False
result = f"Agent 定时任务执行失败:{str(err)}"
logger.error(f"Agent 定时任务 {task_id} 执行失败: {str(err)}")
await AgentChain().async_post_message(
Notification(
mtype=NotificationType.Agent,
username=notification_username,
title=f"定时任务执行失败:{task.name}",
text=result,
save_history=False,
)
)
finally:
current_task = oper.get(task_id)
oper.finish(
task_id=task_id,
success=success,
result=str(result or ""),
disable=bool(
current_task
and task.trigger_type == "date"
and current_task.trigger_type == task.trigger_type
and current_task.run_at == task.run_at
),
)
return success, str(result or "任务执行完成")
@staticmethod
def _build_heartbeat_prompt() -> str:
"""使用程序内置 System Tasks 定义构建心跳任务提示词。"""
+2
View File
@@ -536,6 +536,7 @@ class StreamingHandler:
original_chat_id=self._original_chat_id,
title=self._title,
text=current_text,
save_history=False,
),
)
if response and response.success and response.message_id:
@@ -581,6 +582,7 @@ class StreamingHandler:
original_chat_id=self._original_chat_id,
title=self._title,
text=current_text,
save_history=False,
),
)
if response and response.success and response.message_id:
+54 -17
View File
@@ -952,6 +952,7 @@ class LLMHelper:
base_url: str | None = None,
base_url_preset: str | None = None,
user_agent: str | None = None,
temperature: Optional[float] = None,
use_proxy: bool | None = None,
):
"""
@@ -967,6 +968,7 @@ class LLMHelper:
:param base_url: API Base URL。未显式传入时使用当前配置项 LLM_BASE_URL。
:param base_url_preset: Base URL 预设。未显式传入时使用当前配置项 LLM_BASE_URL_PRESET。
:param user_agent: OpenAI兼容接口请求 User-Agent。未显式传入时使用配置项 LLM_USER_AGENT。
:param temperature: LLM 温度参数。未显式传入时使用配置项 LLM_TEMPERATURE。
:param use_proxy: 是否为本次 LLM 调用使用系统代理。未显式传入时使用配置项 LLM_USE_PROXY。
:return: LLM实例
"""
@@ -978,6 +980,7 @@ class LLMHelper:
base_url_preset if base_url_preset is not None else settings.LLM_BASE_URL_PRESET
)
user_agent_value = user_agent if user_agent is not None else settings.LLM_USER_AGENT
temperature_value = temperature if temperature is not None else settings.LLM_TEMPERATURE
normalized_thinking_level = cls._resolve_thinking_level(
thinking_level=thinking_level,
)
@@ -1034,7 +1037,7 @@ class LLMHelper:
model=model_name,
api_key=runtime["api_key"],
retries=3,
temperature=settings.LLM_TEMPERATURE,
temperature=temperature_value,
streaming=streaming,
client_args=_build_google_client_args(llm_proxy),
**thinking_kwargs,
@@ -1048,13 +1051,36 @@ class LLMHelper:
api_key=runtime["api_key"],
api_base=runtime["base_url"],
max_retries=3,
temperature=settings.LLM_TEMPERATURE,
temperature=temperature_value,
streaming=streaming,
stream_usage=True,
http_client=_build_httpx_client(llm_proxy),
http_async_client=_build_httpx_client(llm_proxy, async_client=True),
**thinking_kwargs,
)
elif runtime["runtime"] == "bedrock":
from langchain_aws import ChatBedrockConverse
from app.agent.llm.provider import LLMProviderManager
aws_region = runtime.get("aws_region") or "us-east-1"
aws_auth = runtime.get("aws_auth") or {}
# Bearer 认证需要跳过 SigV4 签名并注入 Authorization 头,SigV4 认证
# 直接以 AK/SK 签名;两种方式统一由 provider 管理器构造 boto3 客户端。
bedrock_client = LLMProviderManager().create_bedrock_client(
"bedrock-runtime",
region=aws_region,
credentials=aws_auth,
base_url=runtime.get("base_url"),
use_proxy=use_proxy,
read_timeout=settings.LLM_TOOL_TIMEOUT,
)
model = ChatBedrockConverse(
model_id=model_name,
client=bedrock_client,
temperature=temperature_value,
disable_streaming=not streaming,
)
elif runtime["runtime"] in {"anthropic_compatible", "copilot_anthropic"}:
from langchain_anthropic import ChatAnthropic
@@ -1063,7 +1089,7 @@ class LLMHelper:
api_key=runtime["api_key"],
base_url=runtime["base_url"],
max_retries=3,
temperature=settings.LLM_TEMPERATURE,
temperature=temperature_value,
streaming=streaming,
stream_usage=True,
anthropic_proxy=llm_proxy,
@@ -1084,7 +1110,7 @@ class LLMHelper:
api_key=runtime["api_key"],
max_retries=3,
base_url=runtime.get("base_url"),
temperature=settings.LLM_TEMPERATURE,
temperature=temperature_value,
streaming=streaming,
stream_usage=True,
openai_proxy=llm_proxy,
@@ -1104,7 +1130,11 @@ class LLMHelper:
# 优先使用 provider / models.dev 目录中的上下文上限,减少用户手填成本。
model_profile = getattr(model, "profile", None)
if model_profile:
logger.debug(f"使用LLM模型: {model.model}Profile: {model.profile}")
# ChatBedrockConverse 等模型类没有 model 属性,模型名存放在 model_id。
logged_model_name = getattr(model, "model", None) or getattr(
model, "model_id", model_name
)
logger.debug(f"使用LLM模型: {logged_model_name}Profile: {model_profile}")
else:
model_record = runtime.get("model_record") or {}
model_metadata = runtime.get("model_metadata") or {}
@@ -1178,25 +1208,32 @@ class LLMHelper:
base_url: str | None = None,
base_url_preset: str | None = None,
user_agent: str | None = None,
temperature: Optional[float] = None,
use_proxy: bool | None = None,
) -> dict:
"""
使用当前已保存配置执行一次最小 LLM 调用。
使用当前配置或显式传入的临时配置执行一次最小 LLM 调用。
:param temperature: LLM 温度参数。未显式传入时沿用已保存配置。
"""
provider_name = provider if provider is not None else settings.LLM_PROVIDER
model_name = model if model is not None else settings.LLM_MODEL
start = time.perf_counter()
llm = await LLMHelper.get_llm(
streaming=False,
provider=provider_name,
model=model_name,
thinking_level=thinking_level,
api_key=api_key,
base_url=base_url,
base_url_preset=base_url_preset,
user_agent=user_agent,
use_proxy=use_proxy,
)
llm_kwargs = {
"streaming": False,
"provider": provider_name,
"model": model_name,
"thinking_level": thinking_level,
"api_key": api_key,
"base_url": base_url,
"base_url_preset": base_url_preset,
"user_agent": user_agent,
"use_proxy": use_proxy,
}
if temperature is not None:
llm_kwargs["temperature"] = temperature
llm = await LLMHelper.get_llm(**llm_kwargs)
try:
response = await asyncio.wait_for(llm.ainvoke(prompt), timeout=timeout)
except TimeoutError as err:
+488 -1
View File
@@ -7,13 +7,14 @@ import base64
import copy
import hashlib
import json
import re
import secrets
import threading
import time
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Dict, Optional, Tuple
from urllib.parse import urlencode
from urllib.parse import urlencode, urlsplit
import aiofiles
import httpx
@@ -106,6 +107,90 @@ class LLMProviderManager(metaclass=Singleton):
_MODELS_DEV_BUNDLED_PATH = Path(__file__).with_name("models.json")
_MODELS_DEV_CACHE_TTL = 7 * 24 * 60 * 60
_AUTH_SESSION_DONE_RETENTION = 300
_BEDROCK_DEFAULT_REGION = "us-east-1"
_BEDROCK_API_KEY_PREFIX = "bedrock-api-key-"
_BEDROCK_GPT_OSS_BASE_REGIONS = (
"ap-northeast-1",
"ap-south-1",
"ap-southeast-2",
"eu-central-1",
"eu-north-1",
"eu-west-1",
"eu-west-2",
"sa-east-1",
"us-east-1",
"us-east-2",
"us-west-2",
)
_BEDROCK_GPT_OSS_SAFEGUARD_REGIONS = (
"ap-northeast-1",
"ap-south-1",
"ap-southeast-2",
"eu-west-1",
"eu-west-2",
"sa-east-1",
"us-east-1",
"us-east-2",
"us-west-2",
)
_BEDROCK_ON_DEMAND_MODEL_REGIONS = {
"openai.gpt-oss-120b-1:0": _BEDROCK_GPT_OSS_BASE_REGIONS,
"openai.gpt-oss-20b-1:0": _BEDROCK_GPT_OSS_BASE_REGIONS,
"openai.gpt-oss-safeguard-120b": _BEDROCK_GPT_OSS_SAFEGUARD_REGIONS,
"openai.gpt-oss-safeguard-20b": _BEDROCK_GPT_OSS_SAFEGUARD_REGIONS,
"amazon.nova-lite-v1:0": (
"ap-northeast-1",
"ap-southeast-2",
"eu-west-2",
"us-east-1",
"us-gov-west-1",
),
"amazon.nova-micro-v1:0": (
"ap-southeast-2",
"eu-west-2",
"us-east-1",
"us-gov-west-1",
),
"amazon.nova-pro-v1:0": (
"ap-southeast-2",
"eu-west-2",
"us-east-1",
"us-gov-west-1",
),
"anthropic.claude-3-5-haiku-20241022-v1:0": (
"us-west-2",
),
"anthropic.claude-3-5-sonnet-20240620-v1:0": (
"ap-northeast-1",
"ap-northeast-2",
"ap-southeast-1",
"eu-central-1",
"eu-central-2",
"us-east-1",
"us-gov-west-1",
"us-west-2",
),
"anthropic.claude-3-5-sonnet-20241022-v2:0": (
"ap-southeast-2",
"us-west-2",
),
"anthropic.claude-3-7-sonnet-20250219-v1:0": (
"eu-west-2",
"us-gov-west-1",
),
"anthropic.claude-3-haiku-20240307-v1:0": (
"ap-northeast-1",
"ap-northeast-2",
"ap-south-1",
"ap-southeast-2",
"eu-central-1",
"eu-west-1",
"eu-west-3",
"us-east-1",
"us-gov-west-1",
"us-west-2",
),
}
_CHATGPT_CLIENT_ID = "app_EMoamEEZ73f0CkXaXp7hrann"
_CHATGPT_ISSUER = "https://auth.openai.com"
_CHATGPT_CODEX_BASE_URL = "https://chatgpt.com/backend-api/codex"
@@ -367,6 +452,50 @@ class LLMProviderManager(metaclass=Singleton):
api_key_hint="填写 Anthropic API Key。",
description="Anthropic Claude 官方端点。",
),
ProviderSpec(
id="amazon-bedrock",
name="Amazon Bedrock",
runtime="bedrock",
models_dev_provider_id="amazon-bedrock",
default_base_url="https://bedrock-runtime.us-east-1.amazonaws.com",
base_url_presets=(
url_preset(
id="bedrock-us-east-1",
label="美东(弗吉尼亚北部)us-east-1",
value="https://bedrock-runtime.us-east-1.amazonaws.com",
),
url_preset(
id="bedrock-us-west-2",
label="美西(俄勒冈)us-west-2",
value="https://bedrock-runtime.us-west-2.amazonaws.com",
),
url_preset(
id="bedrock-eu-central-1",
label="欧洲(法兰克福)eu-central-1",
value="https://bedrock-runtime.eu-central-1.amazonaws.com",
),
url_preset(
id="bedrock-ap-northeast-1",
label="亚太(东京)ap-northeast-1",
value="https://bedrock-runtime.ap-northeast-1.amazonaws.com",
),
url_preset(
id="bedrock-ap-southeast-1",
label="亚太(新加坡)ap-southeast-1",
value="https://bedrock-runtime.ap-southeast-1.amazonaws.com",
),
),
base_url_editable=True,
api_key_label="Bedrock API Key / AK:SK",
api_key_hint=(
"支持两种认证方式:填写 Amazon Bedrock API Keybedrock-api-key- 开头,"
"Bearer 认证);或填写 Access Key ID:Secret Access Key(可选追加 :Session Token"
"SigV4 认证)。Base URL 决定 AWS Region。"
),
model_list_strategy="bedrock",
description="Amazon Bedrock 托管模型服务,支持 Bedrock API Key 与 AK/SK 双认证。",
sort_order=35,
),
ProviderSpec(
id="deepseek",
name="DeepSeek",
@@ -1743,6 +1872,112 @@ class LLMProviderManager(metaclass=Singleton):
return normalized[:-3]
return normalized
@classmethod
def _extract_bedrock_region(cls, base_url: Optional[str]) -> str:
"""
从 Bedrock 运行时端点 URL 中提取 AWS Region
兼容标准端点、FIPS 端点与 PrivateLinkVPCE)端点等主机名形态,
从中识别 Region 段。
:param base_url: 形如 https://bedrock-runtime.us-east-1.amazonaws.com 的端点地址
:return: 提取到的 Region,无法识别时回退 us-east-1
"""
hostname = urlsplit((base_url or "").strip().lower()).hostname or ""
match = re.search(
r"(?:^|\.)(?:bedrock(?:-runtime)?(?:-fips)?)"
r"\.([a-z0-9-]+-\d+)(?:\.|$)",
hostname,
)
if match:
return match.group(1)
return cls._BEDROCK_DEFAULT_REGION
# Inference Profile 的地理前缀与可用 Region 的对应关系,用于降级目录按
# 当前 Region 过滤掉不可调用的 Profile 条目。
_BEDROCK_GEO_PREFIXES: dict[str, tuple[str, ...]] = {
"us": ("us-east-", "us-west-"),
"eu": ("eu-",),
"apac": ("ap-",),
"au": ("ap-southeast-2", "ap-southeast-4"),
"jp": ("ap-northeast-1", "ap-northeast-3"),
"ca": ("ca-",),
}
_BEDROCK_NON_COMMERCIAL_REGION_PREFIXES = (
"cn-",
"eu-isoe-",
"us-gov-",
"us-iso-",
"us-isob-",
"us-isof-",
)
@classmethod
def _bedrock_model_matches_region(cls, model_id: str, region: str) -> bool:
"""
判断目录中的模型 ID 在指定 Region 是否可调用
models.dev 目录同时收录裸模型 ID(直连调用)与带地理前缀的
Inference Profile IDus./eu./apac./global. 等)。带前缀的条目只在
对应地理分区和 AWS 分区的 Region 可用;global Profile 仅允许商业
AWS 分区。裸 ID 仅在明确记录的 ON_DEMAND Region 可用,未知条目
按不可直连处理。
:param model_id: 目录中的模型 ID
:param region: 当前 Base URL 对应的 AWS Region
:return: 该模型在当前 Region 可调用时返回 True
"""
prefix = model_id.split(".", 1)[0]
if prefix == "global":
return not region.startswith(cls._BEDROCK_NON_COMMERCIAL_REGION_PREFIXES)
region_prefixes = cls._BEDROCK_GEO_PREFIXES.get(prefix)
if region_prefixes is not None:
return (
not region.startswith(cls._BEDROCK_NON_COMMERCIAL_REGION_PREFIXES)
and region.startswith(region_prefixes)
)
on_demand_regions = cls._BEDROCK_ON_DEMAND_MODEL_REGIONS.get(model_id)
return on_demand_regions is not None and region in on_demand_regions
@classmethod
def _parse_bedrock_credentials(cls, api_key: Optional[str]) -> dict[str, Any]:
"""
解析 Bedrock 凭证字符串,识别 Bearer 与 SigV4 两种认证方式
- Bedrock API Keybedrock-api-key- 开头的长期 Key,或控制台生成的短期
Token)走 Bearer 认证;
- `AccessKeyId:SecretAccessKey` 或 `AccessKeyId:SecretAccessKey:SessionToken`
走 SigV4 认证,AWS Access Key ID 均以 "AKIA"/"ASIA" 开头。
:param api_key: 用户在 API Key 输入框填写的凭证内容
:return: 含 auth_scheme 及对应凭证字段的字典
"""
normalized = str(api_key or "").strip()
if not normalized:
raise LLMProviderAuthError(
"Amazon Bedrock 需要填写 Bedrock API Key 或 Access Key ID:Secret Access Key"
)
if not normalized.startswith(cls._BEDROCK_API_KEY_PREFIX):
parts = [part.strip() for part in normalized.split(":")]
if len(parts) in {2, 3} and all(parts):
credentials = {
"auth_scheme": "sigv4",
"access_key_id": parts[0],
"secret_access_key": parts[1],
}
if len(parts) == 3:
credentials["session_token"] = parts[2]
return credentials
if ":" in normalized:
raise LLMProviderAuthError(
"Amazon Bedrock AK/SK 凭证格式不正确,"
"请按 AccessKeyId:SecretAccessKey 或 "
"AccessKeyId:SecretAccessKey:SessionToken 填写"
)
return {"auth_scheme": "bearer", "bearer_token": normalized}
async def _list_models_from_google(
self,
api_key: str,
@@ -1857,6 +2092,235 @@ class LLMProviderManager(metaclass=Singleton):
)
return sorted(results, key=lambda item: item["name"].lower())
def _build_bedrock_boto3_config(
self,
use_proxy: Optional[bool] = None,
) -> Any:
"""
构造 Bedrock boto3 客户端配置,统一超时、重试与代理策略
:param use_proxy: 是否使用系统代理,None 时读取 LLM_USE_PROXY 配置
:return: botocore Config 实例
"""
from botocore.config import Config
should_use_proxy = settings.LLM_USE_PROXY if use_proxy is None else use_proxy
proxies = None
if should_use_proxy and settings.PROXY_HOST:
proxies = {"http": settings.PROXY_HOST, "https": settings.PROXY_HOST}
return Config(
connect_timeout=10,
read_timeout=60,
retries={"max_attempts": 3, "mode": "standard"},
proxies=proxies,
)
@staticmethod
def _bedrock_endpoint_url(
service_name: str, base_url: Optional[str]
) -> Optional[str]:
"""
解析应传给 boto3 客户端的自定义端点 URL
标准公有端点交由 boto3 按 Region 自行推导;用户填写 PrivateLink、
FIPS 等非标准端点时才显式透传,保证所选网络路径实际生效。
:param service_name: boto3 服务名(bedrock 或 bedrock-runtime
:param base_url: 用户配置的 Base URL
:return: 需要显式指定端点时返回 URL,否则返回 None
"""
normalized = (base_url or "").strip().rstrip("/")
if not normalized:
return None
if re.fullmatch(
rf"https://{service_name}\.[a-z0-9-]+\.amazonaws\.com",
normalized,
):
return None
return normalized
def create_bedrock_client(
self,
service_name: str,
region: str,
credentials: dict[str, Any],
base_url: Optional[str] = None,
use_proxy: Optional[bool] = None,
read_timeout: Optional[int] = None,
) -> Any:
"""
按解析后的凭证创建 Bedrock boto3 客户端,Bearer 方式注入 Authorization 头
:param service_name: boto3 服务名(bedrock 或 bedrock-runtime
:param region: AWS Region
:param credentials: `_parse_bedrock_credentials` 的解析结果
:param base_url: 用户配置的 Base URL,非标准端点(PrivateLink/FIPS 等)时透传给 boto3
:param use_proxy: 是否使用系统代理
:param read_timeout: 读取超时秒数,None 时使用默认值
:return: boto3 客户端实例
"""
import boto3
from botocore import UNSIGNED
config = self._build_bedrock_boto3_config(use_proxy)
if read_timeout:
config = config.merge(type(config)(read_timeout=read_timeout))
endpoint_kwargs: dict[str, Any] = {}
endpoint_url = self._bedrock_endpoint_url(service_name, base_url)
if endpoint_url:
endpoint_kwargs["endpoint_url"] = endpoint_url
if credentials["auth_scheme"] == "sigv4":
return boto3.client(
service_name,
region_name=region,
aws_access_key_id=credentials["access_key_id"],
aws_secret_access_key=credentials["secret_access_key"],
aws_session_token=credentials.get("session_token"),
config=config,
**endpoint_kwargs,
)
# Bearer 认证:以 UNSIGNED 跳过 SigV4 签名,再把 API Key 注入 Authorization 头。
bearer_token = credentials["bearer_token"]
config = config.merge(type(config)(signature_version=UNSIGNED))
client = boto3.client(
service_name,
region_name=region,
aws_access_key_id="unsigned",
aws_secret_access_key="unsigned",
config=config,
**endpoint_kwargs,
)
def _inject_bearer(request: Any, **_kwargs: Any) -> None:
request.headers["Authorization"] = f"Bearer {bearer_token}"
client.meta.events.register(
f"request-created.{service_name}",
_inject_bearer,
)
return client
async def _list_models_from_bedrock_fallback(
self,
region: str,
use_proxy: Optional[bool] = None,
) -> list[dict[str, Any]]:
"""
从 models.dev 目录筛选当前 Region 可调用的 Bedrock 模型
:param region: 当前 Base URL 对应的 AWS Region
:param use_proxy: 是否使用系统代理
:return: 过滤后的标准化模型记录列表
"""
models = await self._list_models_from_models_dev_only(
provider_id="amazon-bedrock",
use_proxy=use_proxy,
)
return [
model
for model in models
if self._bedrock_model_matches_region(model["id"], region)
]
async def _list_models_from_bedrock(
self,
api_key: str,
base_url: Optional[str],
use_proxy: Optional[bool] = None,
) -> list[dict[str, Any]]:
"""
从 Bedrock 控制面拉取模型目录,聚合跨区 Inference Profile 与直连模型
Bedrock 多数新模型仅允许通过 Inference Profileus./eu./apac./global. 前缀)
调用,因此优先列出 Profile,再补充支持 ON_DEMAND 直连的基础模型。
:param api_key: 用户填写的凭证内容(Bedrock API Key 或 AK/SK
:param base_url: Bedrock 运行时端点,决定 Region
:param use_proxy: 是否使用系统代理
:return: 标准化后的模型记录列表
"""
credentials = self._parse_bedrock_credentials(api_key)
region = self._extract_bedrock_region(base_url)
# runtime VPCE 无法安全推导对应的控制面 VPCE;FIPS 端点也不能绕回
# 公有非 FIPS 控制面,因此直接使用本地目录。
if self._bedrock_endpoint_url("bedrock-runtime", base_url):
return await self._list_models_from_bedrock_fallback(region, use_proxy)
client = self.create_bedrock_client(
"bedrock",
region=region,
credentials=credentials,
use_proxy=use_proxy,
)
def _fetch() -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
profiles: list[dict[str, Any]] = []
paginator = client.get_paginator("list_inference_profiles")
for page in paginator.paginate(typeEquals="SYSTEM_DEFINED"):
profiles.extend(page.get("inferenceProfileSummaries") or [])
foundation = client.list_foundation_models(
byOutputModality="TEXT",
byInferenceType="ON_DEMAND",
).get("modelSummaries") or []
return profiles, foundation
try:
profile_summaries, foundation_summaries = await asyncio.to_thread(_fetch)
except Exception as err:
# 部分 Bedrock API Key 的授权范围仅覆盖 bedrock-runtime 推理接口,
# 控制面查询被拒时降级到 models.dev 目录,保证仍能选择模型。
logger.warning(
f"获取 Amazon Bedrock 控制面模型列表失败,降级 models.dev 目录: {err}"
)
return await self._list_models_from_bedrock_fallback(region, use_proxy)
finally:
await asyncio.to_thread(client.close)
results: list[dict[str, Any]] = []
seen_ids: set[str] = set()
def _append_record(model_id: str, display_name: Optional[str]) -> None:
if not model_id or model_id in seen_ids:
return
seen_ids.add(model_id)
# Inference Profile 带区域前缀,models.dev 目录按基础模型 ID 收录,
# 去掉首个前缀段再查一次元数据。
metadata = self._cached_models_dev_model("amazon-bedrock", model_id)
if not metadata and "." in model_id:
metadata = self._cached_models_dev_model(
"amazon-bedrock",
model_id.split(".", 1)[1],
)
results.append(
self._normalize_model_record(
model_id=model_id,
display_name=display_name or (metadata or {}).get("name") or model_id,
metadata=metadata or {},
source="provider",
)
)
for profile in profile_summaries:
if (profile.get("status") or "ACTIVE") != "ACTIVE":
continue
_append_record(
str(profile.get("inferenceProfileId") or "").strip(),
profile.get("inferenceProfileName"),
)
# 控制面已按当前 Region 和 ON_DEMAND 筛选,不能复用仅面向
# models.dev 降级目录的静态白名单,否则 AWS 新增模型会被遗漏。
for summary in foundation_summaries:
lifecycle = (summary.get("modelLifecycle") or {}).get("status") or "ACTIVE"
if lifecycle != "ACTIVE":
continue
_append_record(
str(summary.get("modelId") or "").strip(),
summary.get("modelName"),
)
return sorted(results, key=lambda item: item["name"].lower())
@staticmethod
def _copilot_headers(
token: Optional[str] = None, include_auth: bool = True
@@ -2064,6 +2528,13 @@ class LLMProviderManager(metaclass=Singleton):
use_proxy=use_proxy,
)
if resolved_model_list_strategy == "bedrock":
return await self._list_models_from_bedrock(
api_key=runtime["api_key"],
base_url=runtime.get("base_url"),
use_proxy=use_proxy,
)
if resolved_model_list_strategy == "anthropic_compatible":
return await self._list_models_from_models_dev_only(
provider_id=provider_id,
@@ -2731,6 +3202,22 @@ class LLMProviderManager(metaclass=Singleton):
)
return result
if resolved_runtime == "bedrock":
effective_base_url = normalized_base_url or self._default_base_url_for_provider(
spec
)
credentials = self._parse_bedrock_credentials(normalized_api_key)
result.update(
{
"api_key": normalized_api_key,
"base_url": effective_base_url,
"aws_region": self._extract_bedrock_region(effective_base_url),
"aws_auth": credentials,
"auth_mode": "api_key",
}
)
return result
if resolved_runtime == "anthropic_compatible":
effective_base_url = normalized_base_url or self._default_base_url_for_provider(
spec
+600
View File
@@ -0,0 +1,600 @@
"""Agent 外部 MCP 客户端与配置管理。"""
from __future__ import annotations
import asyncio
import hashlib
import json
import os
import re
import uuid
from dataclasses import dataclass
from typing import Any, Optional
from urllib.parse import urljoin
from app.db.systemconfig_oper import SystemConfigOper
from app.log import logger
from app.schemas.agent import (
AgentMcpServerConfig,
AgentMcpServerTestResult,
AgentMcpServerToolInfo,
)
from app.schemas.types import SystemConfigKey
from app.utils.http import AsyncRequestUtils
MCP_PROTOCOL_VERSION = "2025-11-25"
MCP_CLIENT_NAME = "MoviePilot Agent"
DEFAULT_MCP_TIMEOUT = 30
MCP_TOOL_NAME_PATTERN = re.compile(r"[^a-zA-Z0-9_]+")
@dataclass(frozen=True)
class AgentMcpToolSpec:
"""已发现的外部 MCP 工具定义。"""
server: AgentMcpServerConfig
name: str
agent_tool_name: str
description: str
input_schema: dict[str, Any]
def _normalize_identifier(value: str, fallback: str = "mcp") -> str:
"""把服务器或工具名称转换为 Agent 工具可用的标识片段。"""
normalized = MCP_TOOL_NAME_PATTERN.sub("_", str(value or "").strip())
normalized = re.sub(r"_+", "_", normalized).strip("_").lower()
if not normalized:
normalized = fallback
if normalized[0].isdigit():
normalized = f"{fallback}_{normalized}"
return normalized[:64]
def _normalize_timeout(value: Any) -> int:
"""规范化 MCP 连接和调用超时时间。"""
try:
timeout = int(value or DEFAULT_MCP_TIMEOUT)
except (TypeError, ValueError):
timeout = DEFAULT_MCP_TIMEOUT
return min(max(timeout, 1), 600)
def _normalize_string_dict(value: Any) -> dict[str, str]:
"""规范化请求头和环境变量字典,移除空键。"""
if not isinstance(value, dict):
return {}
normalized: dict[str, str] = {}
for key, item in value.items():
normalized_key = str(key or "").strip()
if not normalized_key:
continue
normalized[normalized_key] = str(item or "")
return normalized
def _normalize_input_schema(value: Any) -> dict[str, Any]:
"""规范化 MCP 工具参数 Schema,保证至少是 object schema。"""
if not isinstance(value, dict):
return {"type": "object", "properties": {}, "required": []}
schema = dict(value)
schema.setdefault("type", "object")
schema.setdefault("properties", {})
schema.setdefault("required", [])
return schema
def _build_agent_tool_name(server: AgentMcpServerConfig, tool_name: str) -> str:
"""构造注入 Agent 的外部 MCP 工具名。"""
prefix = server.tool_prefix or f"mcp_{server.name or server.id}"
normalized_prefix = _normalize_identifier(prefix, fallback="mcp")
normalized_tool_name = _normalize_identifier(tool_name, fallback="tool")
if normalized_tool_name.startswith(f"{normalized_prefix}_"):
return normalized_tool_name
return f"{normalized_prefix}_{normalized_tool_name}"[:128]
def _jsonrpc_message(method: str, params: Optional[dict[str, Any]] = None, *, request_id: Optional[str] = None) -> dict:
"""构造 JSON-RPC 2.0 消息。"""
payload = {"jsonrpc": "2.0", "method": method}
if request_id is not None:
payload["id"] = request_id
if params is not None:
payload["params"] = params
return payload
def _raise_for_jsonrpc_error(payload: Any) -> None:
"""检查 JSON-RPC 响应错误并转换为运行时异常。"""
if isinstance(payload, dict) and payload.get("error"):
error = payload["error"]
if isinstance(error, dict):
message = error.get("message") or error
else:
message = error
raise RuntimeError(f"MCP JSON-RPC 错误: {message}")
def _extract_jsonrpc_result(payload: Any, request_id: str) -> Any:
"""从 JSON-RPC 响应中提取 result 字段。"""
if not isinstance(payload, dict):
raise RuntimeError("MCP 响应不是有效 JSON 对象")
if payload.get("id") != request_id:
raise RuntimeError("MCP 响应 ID 与请求不匹配")
_raise_for_jsonrpc_error(payload)
return payload.get("result")
async def _iter_sse_events(response) -> Any:
"""按 SSE 事件格式迭代响应流。"""
event_name = "message"
data_lines: list[str] = []
async for raw_line in response.aiter_lines():
line = raw_line.rstrip("\r")
if not line:
if data_lines:
yield {"event": event_name, "data": "\n".join(data_lines)}
event_name = "message"
data_lines = []
continue
if line.startswith(":"):
continue
field, _, value = line.partition(":")
if value.startswith(" "):
value = value[1:]
if field == "event":
event_name = value or "message"
elif field == "data":
data_lines.append(value)
if data_lines:
yield {"event": event_name, "data": "\n".join(data_lines)}
def _parse_sse_text_response(text: str, request_id: str) -> Any:
"""从非流式 SSE 文本响应中提取匹配请求的 JSON-RPC 结果。"""
event_name = "message"
data_lines: list[str] = []
for raw_line in str(text or "").splitlines():
line = raw_line.rstrip("\r")
if not line:
if data_lines:
payload = _load_sse_json_payload(event_name, "\n".join(data_lines))
if isinstance(payload, dict) and payload.get("id") == request_id:
return _extract_jsonrpc_result(payload, request_id)
event_name = "message"
data_lines = []
continue
if line.startswith(":"):
continue
field, _, value = line.partition(":")
if value.startswith(" "):
value = value[1:]
if field == "event":
event_name = value or "message"
elif field == "data":
data_lines.append(value)
if data_lines:
payload = _load_sse_json_payload(event_name, "\n".join(data_lines))
if isinstance(payload, dict) and payload.get("id") == request_id:
return _extract_jsonrpc_result(payload, request_id)
raise RuntimeError("MCP SSE 响应中未找到匹配请求")
def _load_sse_json_payload(event_name: str, data: str) -> Optional[dict]:
"""解析 SSE data 中的 JSON-RPC 消息。"""
if event_name not in {"message", "messages"}:
return None
try:
payload = json.loads(data)
except (TypeError, ValueError):
return None
return payload if isinstance(payload, dict) else None
class _StdioMcpSession:
"""stdio MCP 会话,按一次操作生命周期启动外部进程。"""
def __init__(self, server: AgentMcpServerConfig) -> None:
self.server = server
self.process: Optional[asyncio.subprocess.Process] = None
self.stderr_task: Optional[asyncio.Task] = None
async def __aenter__(self) -> "_StdioMcpSession":
"""启动 stdio MCP 子进程。"""
if not self.server.command:
raise RuntimeError("stdio MCP 服务器缺少启动命令")
env = os.environ.copy()
env.update(self.server.env or {})
self.process = await asyncio.create_subprocess_exec(
self.server.command,
*(self.server.args or []),
stdin=asyncio.subprocess.PIPE,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
env=env,
)
self.stderr_task = asyncio.create_task(self._drain_stderr())
return self
async def __aexit__(self, exc_type, exc, tb) -> None:
"""结束 stdio MCP 子进程。"""
if self.stderr_task:
self.stderr_task.cancel()
if not self.process:
return
if self.process.returncode is None:
self.process.terminate()
try:
await asyncio.wait_for(self.process.wait(), timeout=2)
except asyncio.TimeoutError:
self.process.kill()
await self.process.wait()
async def _drain_stderr(self) -> None:
"""持续读取子进程 stderr,避免缓冲区阻塞。"""
if not self.process or not self.process.stderr:
return
try:
while True:
line = await self.process.stderr.readline()
if not line:
break
logger.debug(f"MCP stdio[{self.server.name}] stderr: {line.decode(errors='replace').strip()}")
except asyncio.CancelledError:
return
async def notify(self, method: str, params: Optional[dict[str, Any]] = None) -> None:
"""发送不需要响应的 JSON-RPC 通知。"""
await self._write_json(_jsonrpc_message(method, params))
async def request(self, method: str, params: Optional[dict[str, Any]] = None) -> Any:
"""发送 JSON-RPC 请求并等待响应。"""
request_id = uuid.uuid4().hex
await self._write_json(_jsonrpc_message(method, params, request_id=request_id))
while True:
payload = await self._read_json()
if payload.get("id") == request_id:
return _extract_jsonrpc_result(payload, request_id)
async def _write_json(self, payload: dict) -> None:
"""写入一行 JSON-RPC 消息。"""
if not self.process or not self.process.stdin:
raise RuntimeError("stdio MCP 进程未启动")
data = json.dumps(payload, ensure_ascii=False, separators=(",", ":")) + "\n"
self.process.stdin.write(data.encode("utf-8"))
await self.process.stdin.drain()
async def _read_json(self) -> dict:
"""从 stdout 读取一行 JSON-RPC 消息。"""
if not self.process or not self.process.stdout:
raise RuntimeError("stdio MCP 进程未启动")
timeout = _normalize_timeout(self.server.timeout)
while True:
line = await asyncio.wait_for(self.process.stdout.readline(), timeout=timeout)
if not line:
raise RuntimeError("stdio MCP 进程已退出")
try:
payload = json.loads(line.decode("utf-8"))
except ValueError:
logger.debug(f"忽略非 JSON MCP stdout 行: {line!r}")
continue
if isinstance(payload, dict):
return payload
class _HttpMcpSession:
"""Streamable HTTP MCP 会话。"""
def __init__(self, server: AgentMcpServerConfig) -> None:
self.server = server
self.session_id: Optional[str] = None
async def __aenter__(self) -> "_HttpMcpSession":
"""进入 HTTP MCP 会话。"""
return self
async def __aexit__(self, exc_type, exc, tb) -> None:
"""退出 HTTP MCP 会话。"""
return None
async def notify(self, method: str, params: Optional[dict[str, Any]] = None) -> None:
"""发送不需要响应的 JSON-RPC 通知。"""
await self._post(_jsonrpc_message(method, params), expect_response=False)
async def request(self, method: str, params: Optional[dict[str, Any]] = None) -> Any:
"""发送 JSON-RPC 请求并等待响应。"""
request_id = uuid.uuid4().hex
return await self._post(
_jsonrpc_message(method, params, request_id=request_id),
expect_response=True,
request_id=request_id,
)
async def _post(
self,
payload: dict,
*,
expect_response: bool,
request_id: Optional[str] = None,
) -> Any:
"""向 Streamable HTTP MCP 服务发送一条 JSON-RPC 消息。"""
if not self.server.url:
raise RuntimeError("HTTP MCP 服务器缺少 URL")
headers = {
"Accept": "application/json, text/event-stream",
"Content-Type": "application/json",
**(self.server.headers or {}),
}
if self.session_id:
headers["Mcp-Session-Id"] = self.session_id
response = await AsyncRequestUtils(
headers=headers,
timeout=_normalize_timeout(self.server.timeout),
content_type="application/json",
accept_type="application/json, text/event-stream",
http2=False,
).post_res(self.server.url, json=payload, raise_exception=True)
try:
if not response:
raise RuntimeError("HTTP MCP 请求无响应")
response.raise_for_status()
session_id = response.headers.get("Mcp-Session-Id")
if session_id:
self.session_id = session_id
if not expect_response:
return None
content_type = response.headers.get("content-type", "").lower()
if "text/event-stream" in content_type:
return _parse_sse_text_response(response.text, request_id or "")
data = response.json()
return _extract_jsonrpc_result(data, request_id or "")
finally:
if response is not None:
await response.aclose()
class _SseMcpSession:
"""旧版 HTTP+SSE MCP 会话。"""
def __init__(self, server: AgentMcpServerConfig) -> None:
self.server = server
self.response = None
self.endpoint: Optional[str] = None
self._stream_manager = None
self._event_iterator = None
async def __aenter__(self) -> "_SseMcpSession":
"""打开 SSE 流并读取服务端回传的 POST endpoint。"""
if not self.server.url:
raise RuntimeError("SSE MCP 服务器缺少 URL")
self._stream_manager = AsyncRequestUtils(
headers={"Accept": "text/event-stream", **(self.server.headers or {})},
timeout=_normalize_timeout(self.server.timeout),
accept_type="text/event-stream",
http2=False,
).get_stream(self.server.url, raise_exception=True)
self.response = await self._stream_manager.__aenter__()
if not self.response:
raise RuntimeError("SSE MCP 连接无响应")
self.response.raise_for_status()
self._event_iterator = _iter_sse_events(self.response).__aiter__()
self.endpoint = await self._read_endpoint()
return self
async def __aexit__(self, exc_type, exc, tb) -> None:
"""关闭 SSE 流。"""
if self._stream_manager:
await self._stream_manager.__aexit__(exc_type, exc, tb)
async def notify(self, method: str, params: Optional[dict[str, Any]] = None) -> None:
"""发送不需要响应的 JSON-RPC 通知。"""
await self._post(_jsonrpc_message(method, params))
async def request(self, method: str, params: Optional[dict[str, Any]] = None) -> Any:
"""发送 JSON-RPC 请求并等待 SSE 流上的响应。"""
request_id = uuid.uuid4().hex
await self._post(_jsonrpc_message(method, params, request_id=request_id))
timeout = _normalize_timeout(self.server.timeout)
while True:
event = await asyncio.wait_for(self._event_iterator.__anext__(), timeout=timeout)
payload = _load_sse_json_payload(event.get("event", ""), event.get("data", ""))
if isinstance(payload, dict) and payload.get("id") == request_id:
return _extract_jsonrpc_result(payload, request_id)
async def _read_endpoint(self) -> str:
"""读取 SSE endpoint 事件中的 POST 地址。"""
timeout = _normalize_timeout(self.server.timeout)
while True:
event = await asyncio.wait_for(self._event_iterator.__anext__(), timeout=timeout)
if event.get("event") != "endpoint":
continue
endpoint = str(event.get("data") or "").strip()
if not endpoint:
continue
return urljoin(self.server.url, endpoint)
async def _post(self, payload: dict) -> None:
"""向 SSE 握手返回的 endpoint 发送 JSON-RPC 消息。"""
if not self.endpoint:
raise RuntimeError("SSE MCP endpoint 未初始化")
response = await AsyncRequestUtils(
headers={
"Accept": "application/json",
"Content-Type": "application/json",
**(self.server.headers or {}),
},
timeout=_normalize_timeout(self.server.timeout),
content_type="application/json",
accept_type="application/json",
http2=False,
).post_res(self.endpoint, json=payload, raise_exception=True)
try:
if not response:
raise RuntimeError("SSE MCP POST 请求无响应")
response.raise_for_status()
finally:
if response is not None:
await response.aclose()
async def _open_mcp_session(server: AgentMcpServerConfig):
"""根据配置创建对应的 MCP 传输会话。"""
transport = "http" if server.transport == "streamable_http" else server.transport
if transport == "stdio":
return _StdioMcpSession(server)
if transport == "sse":
return _SseMcpSession(server)
if transport == "http":
return _HttpMcpSession(server)
raise RuntimeError(f"不支持的 MCP 传输协议: {server.transport}")
class AgentMcpManager:
"""管理 Agent 外部 MCP 服务器配置、工具发现和工具调用。"""
def get_servers(self) -> list[AgentMcpServerConfig]:
"""读取已保存的外部 MCP 服务器配置。"""
raw_servers = SystemConfigOper().get(SystemConfigKey.AIAgentMcpServers) or []
if not isinstance(raw_servers, list):
return []
servers: list[AgentMcpServerConfig] = []
for raw_server in raw_servers:
try:
servers.append(self.normalize_server(raw_server))
except Exception as err:
logger.warning(f"忽略无效的 Agent MCP 配置: {err}")
return servers
async def save_servers(self, servers: list[AgentMcpServerConfig]) -> bool:
"""保存外部 MCP 服务器配置。"""
normalized_servers = [self.normalize_server(server).model_dump() for server in servers]
return await SystemConfigOper().async_set(
SystemConfigKey.AIAgentMcpServers,
normalized_servers or None,
)
def normalize_server(self, value: Any) -> AgentMcpServerConfig:
"""规范化单个 MCP 服务器配置。"""
if isinstance(value, AgentMcpServerConfig):
raw_server = value.model_dump()
elif isinstance(value, dict):
raw_server = dict(value)
else:
raise ValueError("MCP 服务器配置必须是对象")
raw_server["id"] = str(raw_server.get("id") or uuid.uuid4().hex[:12]).strip()
raw_server["name"] = str(raw_server.get("name") or raw_server["id"]).strip()
raw_server["transport"] = str(raw_server.get("transport") or "stdio").strip()
raw_server["description"] = str(raw_server.get("description") or "").strip() or None
raw_server["command"] = str(raw_server.get("command") or "").strip() or None
raw_server["args"] = [str(item) for item in raw_server.get("args") or []]
raw_server["env"] = _normalize_string_dict(raw_server.get("env"))
raw_server["url"] = str(raw_server.get("url") or "").strip() or None
raw_server["headers"] = _normalize_string_dict(raw_server.get("headers"))
raw_server["timeout"] = _normalize_timeout(raw_server.get("timeout"))
raw_server["tool_prefix"] = str(raw_server.get("tool_prefix") or "").strip() or None
raw_server["require_admin"] = bool(raw_server.get("require_admin", True))
return AgentMcpServerConfig.model_validate(raw_server)
def config_signature(self) -> str:
"""生成外部 MCP 配置签名,用于 Agent 图缓存失效。"""
payload = [server.model_dump() for server in self.get_servers()]
raw_text = json.dumps(payload, ensure_ascii=False, sort_keys=True, default=str)
return hashlib.sha256(raw_text.encode("utf-8")).hexdigest()
async def initialize_session(self, session) -> None:
"""完成 MCP initialize 和 initialized 通知流程。"""
await session.request(
"initialize",
{
"protocolVersion": MCP_PROTOCOL_VERSION,
"capabilities": {},
"clientInfo": {
"name": MCP_CLIENT_NAME,
"version": "1.0.0",
},
},
)
await session.notify("notifications/initialized")
async def list_server_tools(self, server: AgentMcpServerConfig) -> list[AgentMcpToolSpec]:
"""连接单个 MCP 服务器并读取工具列表。"""
normalized_server = self.normalize_server(server)
session_manager = await _open_mcp_session(normalized_server)
async with session_manager as session:
await self.initialize_session(session)
result = await session.request("tools/list")
tools_payload = result.get("tools", []) if isinstance(result, dict) else []
tool_specs: list[AgentMcpToolSpec] = []
for item in tools_payload:
if not isinstance(item, dict) or not item.get("name"):
continue
tool_name = str(item["name"])
tool_specs.append(
AgentMcpToolSpec(
server=normalized_server,
name=tool_name,
agent_tool_name=_build_agent_tool_name(normalized_server, tool_name),
description=str(item.get("description") or ""),
input_schema=_normalize_input_schema(item.get("inputSchema")),
)
)
return tool_specs
async def list_enabled_tool_specs(self) -> list[AgentMcpToolSpec]:
"""读取所有启用 MCP 服务器暴露的工具定义。"""
tool_specs: list[AgentMcpToolSpec] = []
seen_names: set[str] = set()
for server in self.get_servers():
if not server.enabled:
continue
try:
for spec in await self.list_server_tools(server):
if spec.agent_tool_name in seen_names:
logger.warning(f"跳过重复的 MCP Agent 工具名: {spec.agent_tool_name}")
continue
tool_specs.append(spec)
seen_names.add(spec.agent_tool_name)
except Exception as err:
logger.warning(f"读取 MCP 服务器 {server.name} 工具失败: {err}")
return tool_specs
async def call_server_tool(
self,
server: AgentMcpServerConfig,
tool_name: str,
arguments: Optional[dict[str, Any]] = None,
) -> Any:
"""调用单个 MCP 服务器上的指定工具。"""
normalized_server = self.normalize_server(server)
session_manager = await _open_mcp_session(normalized_server)
async with session_manager as session:
await self.initialize_session(session)
return await session.request(
"tools/call",
{
"name": tool_name,
"arguments": arguments or {},
},
)
async def test_server(self, server: AgentMcpServerConfig) -> AgentMcpServerTestResult:
"""测试 MCP 服务器连接并返回工具列表。"""
tool_specs = await self.list_server_tools(server)
tools = [
AgentMcpServerToolInfo(
name=spec.name,
agent_tool_name=spec.agent_tool_name,
description=spec.description,
input_schema=spec.input_schema,
)
for spec in tool_specs
]
return AgentMcpServerTestResult(
success=True,
message=f"连接成功,发现 {len(tools)} 个工具",
tools=tools,
tool_count=len(tools),
)
agent_mcp_manager = AgentMcpManager()
+9 -4
View File
@@ -204,9 +204,14 @@ You have a scheduled jobs system for user-requested delayed or recurring work.
{jobs_list}
Rules:
- Create jobs only when the user asks for delayed, recurring, reminder, or monitoring behavior.
- Do not create jobs for immediate one-time work or work already handled by MoviePilot schedulers.
- Each job lives in its own directory with a `JOB.md`; read the listed file before executing or updating an active job.
- For new delayed, recurring, reminder, or monitoring work, use the dedicated
`create_agent_task`, `query_agent_tasks`, `update_agent_task`, `run_agent_task`,
and `delete_agent_task` tools. These tools use integer task IDs. Do not create
or edit JOB.md files for new tasks.
- Use `query_schedulers` and `run_scheduler` only for MoviePilot system, plugin,
or workflow runtime services; never pass their string job IDs to Agent task tools.
- Do not create tasks for immediate one-time work or work already handled by MoviePilot schedulers.
- Entries listed above are legacy JOB.md tasks. Read their files only when a heartbeat asks you to execute them.
- During heartbeat checks, act only on `pending` or `in_progress` jobs, update status/last_run/logs, and leave recurring jobs `pending` after each run.
</jobs_system>
"""
@@ -230,7 +235,7 @@ class JobsMiddleware(AgentMiddleware[JobsState, ContextT, ResponseT]): # noqa
def _format_jobs_list(jobs: list[JobMetadata]) -> str:
"""格式化任务元数据列表用于系统提示词。"""
if not jobs:
return "(No active jobs. You can create jobs when users request periodic or scheduled tasks.)"
return "(No active legacy JOB.md tasks. Use create_agent_task for new scheduled work.)"
lines = []
for job in jobs:
+1
View File
@@ -24,6 +24,7 @@ You act as a proactive agent. Your goal is to fully resolve the user's media-rel
- Do not stop for approval on read-only operations.
- If the user has not explicitly requested an operation that changes system behavior, ask for confirmation before proceeding. This includes modifying system settings, updating plugin configuration, reloading plugins, running restart/stop/start commands, or triggering slash commands such as `/restart`.
- Always get explicit consent before destructive or high-impact actions such as starting downloads, deleting subscriptions, deleting download tasks or files, removing history, installing/uninstalling plugins, changing site authentication, changing scheduler or workflow execution state, restarting services, or stopping services.
- When the user explicitly asks for delayed, recurring, reminder, or monitoring work, use `create_agent_task` instead of promising to remember it or writing a JOB.md file. Use a `date` trigger with `delay_minutes` for requests such as "in 30 minutes", an exact `date` trigger for other single future runs, and a five-field `cron` trigger for recurring work. Manage existing autonomous tasks with `query_agent_tasks`, `update_agent_task`, `run_agent_task`, and `delete_agent_task`; these tools use integer `task_id` values. Use `query_schedulers` and `run_scheduler` only for MoviePilot system, plugin, or workflow runtime services, whose string `job_id` values must never be passed to autonomous-task tools.
- If the user explicitly requested the exact write action, perform the smallest correct change and then validate the result.
- If a requested action is ambiguous between read-only inspection and state change, inspect first and ask a short confirmation question before the state-changing step.
</confirmation_policy>
-2
View File
@@ -315,8 +315,6 @@ class PromptManager:
"项目根目录": settings.ROOT_PATH,
"配置目录": settings.CONFIG_PATH,
"临时目录": settings.TEMP_PATH,
"日志目录": settings.LOG_PATH,
"主日志文件": settings.LOG_PATH / "moviepilot.log",
}
return [f" - {label}: `{path}`" for label, path in paths.items()]
+21 -6
View File
@@ -425,7 +425,6 @@ class MoviePilotTool(BaseTool, metaclass=ABCMeta):
"""
roots = [
settings.CONFIG_PATH / "agent",
settings.LOG_PATH,
]
resolved_roots = []
for root in roots:
@@ -461,7 +460,7 @@ class MoviePilotTool(BaseTool, metaclass=ABCMeta):
allowed_text = "".join(str(root) for root in allowed_roots)
return (
resolved_path,
f"抱歉,普通用户只能{operation}Agent配置目录和日志目录内的文件或目录:{allowed_text}",
f"抱歉,普通用户只能{operation}Agent配置目录内的文件或目录:{allowed_text}",
)
async def _check_local_storage_access(
@@ -483,7 +482,7 @@ class MoviePilotTool(BaseTool, metaclass=ABCMeta):
return None, None
return (
None,
f"抱歉,普通用户只能{operation}本地配置目录、Agent记忆目录和日志目录,不能访问远程存储。",
f"抱歉,普通用户只能{operation}本地Agent配置目录,不能访问远程存储。",
)
return await self._check_local_file_access(path=path, operation=operation)
@@ -509,8 +508,8 @@ class MoviePilotTool(BaseTool, metaclass=ABCMeta):
return (
"抱歉,您没有执行此工具的权限。"
"只有渠道管理员或系统管理员才能执行工具操作。"
"如需执行工具,请联系渠道管理员将您的用户ID添加到渠道管理员列表中,"
"或联系系统管理员为您设置权限。"
"如需执行工具,请联系管理员将您的用户ID添加到渠道管理员列表中(设定 -> 通知 -> 对应渠道配置 -> 管理员名单)"
"或联系系统管理员为您设置管理员权限。"
)
async def _has_channel_admin_permission(self) -> bool:
@@ -621,7 +620,8 @@ class MoviePilotTool(BaseTool, metaclass=ABCMeta):
发送工具通知消息
WebAgent 渠道没有后端模块实例前端流式面板通过 Agent 上下文中的
回调直接接收通知其它渠道继续走统一消息链
回调直接接收通知无渠道的后台任务清空渠道侧定位信息后交由消息链广播
其它渠道继续走统一消息链
"""
callback = self._agent_context.get("notification_callback")
if (
@@ -631,6 +631,20 @@ class MoviePilotTool(BaseTool, metaclass=ABCMeta):
callback(notification)
return
if not self._channel or not self._source:
notification = notification.model_copy(
update={
"channel": None,
"source": None,
"userid": None,
"username": notification.username
or self._username
or settings.SUPERUSER,
"original_message_id": None,
"original_chat_id": None,
}
)
await ToolChain().async_post_message(notification)
async def send_tool_message(
@@ -649,5 +663,6 @@ class MoviePilotTool(BaseTool, metaclass=ABCMeta):
title=title,
text=message,
image=image,
save_history=False,
)
)
+12
View File
@@ -42,8 +42,13 @@ from app.agent.tools.impl.send_message import SendMessageTool
from app.agent.tools.impl.ask_user_choice import AskUserChoiceTool
from app.agent.tools.impl.send_local_file import SendLocalFileTool
from app.agent.tools.impl.send_voice_message import SendVoiceMessageTool
from app.agent.tools.impl.create_agent_task import CreateAgentTaskTool
from app.agent.tools.impl.delete_agent_task import DeleteAgentTaskTool
from app.agent.tools.impl.query_agent_tasks import QueryAgentTasksTool
from app.agent.tools.impl.query_schedulers import QuerySchedulersTool
from app.agent.tools.impl.run_agent_task import RunAgentTaskTool
from app.agent.tools.impl.run_scheduler import RunSchedulerTool
from app.agent.tools.impl.update_agent_task import UpdateAgentTaskTool
from app.agent.tools.impl.query_workflows import QueryWorkflowsTool
from app.agent.tools.impl.run_workflow import RunWorkflowTool
from app.agent.tools.impl.query_personas import QueryPersonasTool
@@ -141,6 +146,11 @@ class MoviePilotToolFactory:
QueryTransferHistoryTool,
TransferFileTool,
SendMessageTool,
CreateAgentTaskTool,
QueryAgentTasksTool,
UpdateAgentTaskTool,
RunAgentTaskTool,
DeleteAgentTaskTool,
QuerySchedulersTool,
RunSchedulerTool,
QueryWorkflowsTool,
@@ -181,6 +191,8 @@ class MoviePilotToolFactory:
"edit_file",
"execute_command",
"ask_user_choice",
"create_agent_task",
"query_agent_tasks",
)
@staticmethod
@@ -59,6 +59,10 @@ SYSTEMCONFIG_SETTING_METADATA = {
"group": "ai_agent",
"label": "AI 智能体配置",
},
SystemConfigKey.AIAgentMcpServers.value: {
"group": "ai_agent",
"label": "AI 智能体外部 MCP 服务器",
},
SystemConfigKey.CustomIdentifiers.value: {
"group": "custom_identifiers",
"label": "自定义识别词",
+11 -3
View File
@@ -15,7 +15,7 @@ from app.core.config import settings
from app.core.context import Context
from app.core.metainfo import MetaInfo
from app.db.site_oper import SiteOper
from app.helper.directory import DirectoryHelper
from app.helper.directory import DirectoryHelper, validate_download_save_path
from app.log import logger
from app.schemas import FileURI, TorrentInfo
from app.utils.crypto import HashUtils
@@ -183,8 +183,8 @@ class AddDownloadTasksTool(MoviePilotTool):
@staticmethod
def _resolve_direct_download_dir(save_path: Optional[str]) -> Optional[Path]:
"""解析直接下载使用的目录,优先使用 save_path,其次使用默认下载目录"""
if save_path:
return Path(save_path)
if save_path is not None:
return Path(validate_download_save_path(save_path))
download_dirs = DirectoryHelper().get_download_dirs()
if not download_dirs:
@@ -225,6 +225,8 @@ class AddDownloadTasksTool(MoviePilotTool):
merged_labels: Optional[str],
) -> tuple[Optional[str], Optional[str]]:
"""同步提交带上下文的下载任务,避免站点下载与下载器调用阻塞事件循环。"""
if save_path is not None:
save_path = validate_download_save_path(save_path)
return DownloadChain().download_single(
context=context,
downloader=downloader,
@@ -245,6 +247,12 @@ class AddDownloadTasksTool(MoviePilotTool):
if not torrent_inputs:
return "错误:torrent_url 不能为空。"
if save_path is not None:
try:
save_path = validate_download_save_path(save_path)
except ValueError as err:
return f"参数错误:save_path {str(err)}"
merged_labels = self._merge_labels_with_system_tag(labels)
success_count = 0
failed_messages = []
+1 -1
View File
@@ -97,7 +97,7 @@ class AddSubscribeTool(MoviePilotTool):
message += f" ({year})"
if media_type:
message += f" [{media_type}]"
if season:
if season is not None:
message += f"{season}"
elif media_type == "tv":
message += " 第1季(默认)"
+161
View File
@@ -0,0 +1,161 @@
import json
from datetime import datetime, timedelta
from typing import Literal, Optional, Type
import pytz
from pydantic import BaseModel, Field, model_validator
from app.agent.tools.base import MoviePilotTool
from app.agent.tools.tags import ToolTag
from app.core.config import settings
from app.db.agentchat_oper import AgentChatOper
from app.db.agenttask_oper import AgentTaskOper
from app.utils.timer import TimerUtils
class CreateAgentTaskInput(BaseModel):
"""创建 Agent 自主定时任务的输入参数。"""
name: str = Field(
...,
min_length=1,
max_length=100,
description="Short task name shown in task management and execution reports.",
)
content: str = Field(
...,
min_length=1,
max_length=10000,
description="Complete instructions that the agent must execute when the task fires.",
)
trigger_type: Literal["date", "cron"] = Field(
...,
description="Use 'date' for one exact future run or 'cron' for recurring work.",
)
trigger: Optional[str] = Field(
None,
min_length=1,
max_length=200,
description=(
"For date, an ISO 8601 local or timezone-aware time such as "
"2026-07-19 20:30:00; for cron, a standard five-field expression "
"(minute hour day month weekday). The MoviePilot system timezone is used."
),
)
delay_minutes: Optional[int] = Field(
None,
ge=1,
le=525600,
description=(
"For a one-time date task expressed as 'in N minutes', provide this instead "
"of trigger. MoviePilot calculates and persists the exact future run time."
),
)
@model_validator(mode="after")
def validate_trigger(self) -> "CreateAgentTaskInput":
"""校验任务触发配置并统一格式。"""
self.name = self.name.strip()
self.content = self.content.strip()
if not self.name or not self.content:
raise ValueError("name 和 content 不能只包含空白字符")
if self.trigger_type == "date":
if self.delay_minutes is not None:
# LangChain 会在 run() 前后各校验一次,延迟时间在持久化前统一计算。
self.trigger = None
return self
if self.trigger is None:
raise ValueError("date 任务必须提供 trigger 或 delay_minutes")
elif self.trigger is None or self.delay_minutes is not None:
raise ValueError("cron 任务必须提供 trigger,且不能提供 delay_minutes")
self.trigger_type, self.trigger = TimerUtils.normalize_schedule_trigger(
trigger_type=self.trigger_type,
trigger_value=self.trigger,
timezone_name=settings.TZ,
require_future=True,
)
return self
class CreateAgentTaskTool(MoviePilotTool):
"""创建可精确唤醒当前 Agent 会话的自主定时任务。"""
name: str = "create_agent_task"
tags: list[str] = [ToolTag.Write, ToolTag.AgentTask, ToolTag.Admin]
description: str = (
"Create a persistent autonomous agent task only when the user explicitly asks "
"for delayed, scheduled, recurring, reminder, or monitoring work. Use trigger_type "
"'date' with delay_minutes for requests such as 'check in 30 minutes', an exact "
"trigger time for other one-time work, and 'cron' for recurring schedules. When "
"fired, MoviePilot wakes the agent in this conversation, executes content, and "
"broadcasts user-facing messages through the configured notification channels."
)
args_schema: Type[BaseModel] = CreateAgentTaskInput
require_admin: bool = True
def get_tool_message(self, **kwargs: object) -> Optional[str]:
"""生成创建定时任务的提示消息。"""
return f"创建自主定时任务:{kwargs.get('name', '')}"
def _create_task(self, payload: CreateAgentTaskInput) -> dict:
"""持久化任务并立即注册到运行时调度器。"""
from app.scheduler import Scheduler
trigger_value = payload.trigger
if payload.trigger_type == "date" and payload.delay_minutes is not None:
timezone = pytz.timezone(settings.TZ)
trigger_value = (
datetime.now(timezone) + timedelta(minutes=payload.delay_minutes)
).isoformat(timespec="seconds")
_, trigger_value = TimerUtils.normalize_schedule_trigger(
trigger_type=payload.trigger_type,
trigger_value=trigger_value,
timezone_name=settings.TZ,
require_future=True,
)
chat = AgentChatOper().get(
session_id=self._session_id,
user_id=self._user_id,
)
task = AgentTaskOper().add(
name=payload.name.strip(),
content=payload.content.strip(),
trigger_type=payload.trigger_type,
cron_expression=trigger_value if payload.trigger_type == "cron" else None,
run_at=trigger_value if payload.trigger_type == "date" else None,
user_id=str(self._user_id),
username=self._username or (chat.username if chat else None),
session_id=str(self._session_id),
channel=self._channel or (chat.channel if chat else None),
source=self._source or (chat.source if chat else None),
original_chat_id=chat.original_chat_id if chat else None,
)
scheduler = Scheduler()
next_run_at = scheduler.update_agent_task_job(task.id)
return AgentTaskOper.to_dict(
task,
next_run_at=next_run_at,
timezone=settings.TZ,
)
async def run(
self,
name: str,
content: str,
trigger_type: str,
trigger: Optional[str] = None,
delay_minutes: Optional[int] = None,
**kwargs: object,
) -> str:
"""创建 Agent 自主定时任务。"""
if not settings.AI_AGENT_ENABLE:
return "AI Agent 未启用,无法创建自主定时任务"
payload = CreateAgentTaskInput(
name=name,
content=content,
trigger_type=trigger_type,
trigger=trigger,
delay_minutes=delay_minutes,
)
task = await self.run_blocking("db", self._create_task, payload)
return json.dumps(task, ensure_ascii=False, indent=2)
+50
View File
@@ -0,0 +1,50 @@
from typing import Optional, Type
from pydantic import BaseModel, Field
from app.agent.tools.base import MoviePilotTool
from app.agent.tools.tags import ToolTag
from app.db.agenttask_oper import AgentTaskOper
class DeleteAgentTaskInput(BaseModel):
"""删除 Agent 自主定时任务的输入参数。"""
task_id: int = Field(..., ge=1, description="ID of the task to permanently delete.")
class DeleteAgentTaskTool(MoviePilotTool):
"""永久删除 Agent 自主定时任务。"""
name: str = "delete_agent_task"
tags: list[str] = [ToolTag.Write, ToolTag.AgentTask, ToolTag.Admin]
description: str = (
"Permanently delete an autonomous agent task and remove its runtime schedule. "
"Use update_agent_task with enabled=false when the user only wants to pause it."
)
args_schema: Type[BaseModel] = DeleteAgentTaskInput
require_admin: bool = True
def get_tool_message(self, **kwargs: object) -> Optional[str]:
"""生成删除定时任务的提示消息。"""
return f"删除自主定时任务:{kwargs.get('task_id', '')}"
def _delete_task(self, task_id: int) -> bool:
"""删除当前用户的任务并移除运行时调度。"""
from app.scheduler import Scheduler
deleted = AgentTaskOper().delete(
task_id=task_id,
user_id=str(self._user_id),
)
if deleted:
Scheduler().remove_agent_task_job(task_id)
return deleted
async def run(self, task_id: int, **kwargs: object) -> str:
"""删除 Agent 自主定时任务。"""
payload = DeleteAgentTaskInput(task_id=task_id)
deleted = await self.run_blocking("db", self._delete_task, payload.task_id)
if not deleted:
return f"Agent 定时任务 {task_id} 不存在或不属于当前用户"
return f"Agent 定时任务 {task_id} 已删除"
+1 -1
View File
@@ -28,7 +28,7 @@ class EditFileTool(MoviePilotTool):
description: str = (
"Edit a local text file by replacing specific old text with new text. "
"Non-admin users can only edit files inside the MoviePilot Agent config "
"and log directories."
"directory."
)
args_schema: Type[BaseModel] = EditFileInput
+98
View File
@@ -0,0 +1,98 @@
"""外部 MCP 工具适配器。"""
import json
from typing import Any, Optional
from pydantic import PrivateAttr
from app.agent.mcp import AgentMcpToolSpec, agent_mcp_manager
from app.agent.tools.base import MoviePilotTool
from app.agent.tools.tags import ToolTag
class McpExternalTool(MoviePilotTool):
"""将外部 MCP 工具包装为 MoviePilot Agent 工具。"""
name: str = "mcp_external_tool"
tags: list[str] = [
ToolTag.Read,
ToolTag.Admin,
]
description: str = "Call an external MCP tool configured for MoviePilot Agent."
args_schema: dict[str, Any] = {"type": "object", "properties": {}, "required": []}
require_admin: bool = True
_spec: AgentMcpToolSpec = PrivateAttr()
def __init__(self, spec: AgentMcpToolSpec, session_id: str, user_id: str) -> None:
super().__init__(
session_id=session_id,
user_id=user_id,
name=spec.agent_tool_name,
description=spec.description
or f"Call external MCP tool {spec.name} on {spec.server.name}.",
args_schema=spec.input_schema,
require_admin=spec.server.require_admin,
)
self._spec = spec
def get_tool_message(self, **kwargs) -> Optional[str]:
"""根据 MCP 工具信息生成友好的提示消息。"""
return f"调用 MCP 工具: {self._spec.server.name}/{self._spec.name}"
async def run(self, **kwargs) -> str:
"""
调用外部 MCP 工具
:param kwargs: 传递给外部 MCP 工具的参数
:return: MCP 工具返回内容
"""
result = await agent_mcp_manager.call_server_tool(
server=self._spec.server,
tool_name=self._spec.name,
arguments=kwargs,
)
return self._format_mcp_result(result)
@staticmethod
def _format_mcp_result(result: Any) -> str:
"""将 MCP tools/call 返回结构转换为 Agent 可读文本。"""
if isinstance(result, dict):
content = result.get("content")
if isinstance(content, list):
parts = []
for item in content:
if not isinstance(item, dict):
continue
if item.get("type") == "text" and item.get("text") is not None:
parts.append(str(item["text"]))
elif item:
parts.append(json.dumps(item, ensure_ascii=False, default=str))
if parts:
return "\n".join(parts)
if result.get("isError"):
return json.dumps(result, ensure_ascii=False, indent=2, default=str)
if isinstance(result, str):
return result
return json.dumps(result, ensure_ascii=False, indent=2, default=str)
async def create_external_mcp_tools(
*,
session_id: str,
user_id: str,
channel: Optional[str] = None,
source: Optional[str] = None,
username: Optional[str] = None,
stream_handler=None,
agent_context: Optional[dict] = None,
) -> list[McpExternalTool]:
"""创建当前已启用的外部 MCP Agent 工具列表。"""
tools = []
for spec in await agent_mcp_manager.list_enabled_tool_specs():
tool = McpExternalTool(spec=spec, session_id=session_id, user_id=user_id)
tool.set_message_attr(channel=channel, source=source, username=username)
tool.set_stream_handler(stream_handler=stream_handler)
tool.set_agent_context(agent_context=agent_context)
tools.append(tool)
return tools
+88
View File
@@ -0,0 +1,88 @@
import json
from typing import Optional, Type
from pydantic import BaseModel, Field
from app.agent.tools.base import MoviePilotTool
from app.agent.tools.tags import ToolTag
from app.core.config import settings
from app.db.agenttask_oper import AgentTaskOper
class QueryAgentTasksInput(BaseModel):
"""查询 Agent 自主定时任务的输入参数。"""
task_id: Optional[int] = Field(
None,
ge=1,
description="Optional task ID. Omit it to list tasks owned by the current user.",
)
enabled: Optional[bool] = Field(
None,
description="Optional enabled-state filter used when listing tasks.",
)
class QueryAgentTasksTool(MoviePilotTool):
"""查询当前用户创建的 Agent 自主定时任务。"""
name: str = "query_agent_tasks"
tags: list[str] = [ToolTag.Read, ToolTag.AgentTask, ToolTag.Admin]
description: str = (
"Query persistent autonomous agent tasks owned by the current user, including "
"reminders, monitoring tasks, and recurring agent work. Returns the integer "
"task_id, instructions, trigger, enabled state, next run time, and latest result. "
"Do not use this for MoviePilot system, plugin, or workflow scheduler services."
)
args_schema: Type[BaseModel] = QueryAgentTasksInput
require_admin: bool = True
def get_tool_message(self, **kwargs: object) -> Optional[str]:
"""生成查询定时任务的提示消息。"""
task_id = kwargs.get("task_id")
return f"查询自主定时任务:{task_id}" if task_id else "查询自主定时任务"
def _query_tasks(
self,
task_id: Optional[int],
enabled: Optional[bool],
) -> list[dict]:
"""读取当前用户的任务及运行时下一次触发时间。"""
from app.scheduler import Scheduler
oper = AgentTaskOper()
if task_id:
task = oper.get(task_id=task_id, user_id=str(self._user_id))
tasks = [task] if task else []
else:
tasks = oper.list(user_id=str(self._user_id), enabled=enabled)
scheduler = Scheduler()
result = []
for task in tasks:
data = oper.to_dict(
task,
next_run_at=scheduler.get_agent_task_next_run(task.id),
timezone=settings.TZ,
)
result.append(data)
return result
async def run(
self,
task_id: Optional[int] = None,
enabled: Optional[bool] = None,
**kwargs: object,
) -> str:
"""查询 Agent 自主定时任务。"""
payload = QueryAgentTasksInput(task_id=task_id, enabled=enabled)
tasks = await self.run_blocking(
"db",
self._query_tasks,
payload.task_id,
payload.enabled,
)
return json.dumps(
{"total": len(tasks), "tasks": tasks},
ensure_ascii=False,
indent=2,
)
+4 -1
View File
@@ -44,7 +44,9 @@ class QueryDoctorReportTool(MoviePilotTool):
description: str = (
"Run MoviePilot Doctor in read-only mode and return a structured diagnostic report for troubleshooting. "
"Use this tool when analyzing startup failures, Docker/runtime issues, port conflicts, dependency problems, "
"database health, frontend assets, safe mode, or recent log error clues. This tool never applies fixes."
"database health, frontend assets, safe mode, or recent log error clues. Plugin-only log findings remain "
"visible with affects_report_status=false and do not downgrade the overall status. This tool never applies "
"fixes."
)
require_admin: bool = True
args_schema: Type[BaseModel] = QueryDoctorReportInput
@@ -73,6 +75,7 @@ class QueryDoctorReportTool(MoviePilotTool):
"title": item.get("title"),
"fixable": item.get("fixable"),
"fixed": item.get("fixed"),
"affects_report_status": item.get("affects_report_status", True),
}
for item in report.get("findings") or []
if isinstance(item, dict)
@@ -118,7 +118,7 @@ class QueryPopularSubscribesTool(MoviePilotTool):
# 处理标题
title = sub.get("name")
season = sub.get("season")
if season and int(season) > 1 and media.tmdb_id:
if season not in (None, "") and int(season) != 1 and media.tmdb_id:
# 小写数据转大写
season_str = cn2an.an2cn(season, "low")
title = f"{title}{season_str}"
+45 -23
View File
@@ -3,7 +3,7 @@
import json
from typing import Optional, Type
from pydantic import BaseModel, Field
from pydantic import BaseModel
from app.agent.tools.base import MoviePilotTool
from app.agent.tools.tags import ToolTag
@@ -11,47 +11,69 @@ from app.log import logger
class QuerySchedulersInput(BaseModel):
"""查询定时服务工具的输入参数模型"""
"""查询运行时定时服务的输入参数模型"""
class QuerySchedulersTool(MoviePilotTool):
"""查询系统、插件和工作流注册的运行时定时服务。"""
name: str = "query_schedulers"
tags: list[str] = [
ToolTag.Read,
ToolTag.Scheduler,
ToolTag.Admin,
]
description: str = "Query scheduled tasks and list all available scheduler jobs. Shows job status, next run time, and provider information."
description: str = (
"Query runtime scheduler services registered by MoviePilot system components, "
"plugins, and workflows. It excludes user-created autonomous agent tasks; use "
"query_agent_tasks for reminders, monitoring tasks, and other agent schedules."
)
args_schema: Type[BaseModel] = QuerySchedulersInput
require_admin: bool = True
def get_tool_message(self, **kwargs) -> Optional[str]:
"""生成友好的提示消息"""
return "查询定时服务"
def get_tool_message(self, **kwargs: object) -> Optional[str]:
"""生成查询运行时定时服务的提示消息"""
return "查询系统定时服务"
async def run(self, **kwargs) -> str:
async def run(self, **kwargs: object) -> str:
"""查询非 Agent 自主任务的运行时定时服务。"""
logger.info(f"执行工具: {self.name}")
try:
from app.scheduler import Scheduler
from app.scheduler import AGENT_TASK_JOB_PREFIX, Scheduler
scheduler = Scheduler()
schedulers = scheduler.list()
agent_task_prefix = f"{AGENT_TASK_JOB_PREFIX}-"
schedulers = [
scheduler_item
for scheduler_item in scheduler.list()
if not str(scheduler_item.id or "").startswith(agent_task_prefix)
]
if schedulers:
# 转换为字典列表以便JSON序列化
schedulers_list = []
for s in schedulers:
schedulers_list.append({
"id": s.id,
"name": s.name,
"provider": s.provider,
"status": s.status,
"next_run": s.next_run
})
schedulers_list = [
{
"id": scheduler_item.id,
"name": scheduler_item.name,
"provider": scheduler_item.provider,
"status": scheduler_item.status,
"next_run": scheduler_item.next_run,
}
for scheduler_item in schedulers
]
result_json = json.dumps(schedulers_list, ensure_ascii=False, indent=2)
# 限制最多30条结果
total_count = len(schedulers_list)
if total_count > 30:
limited_schedulers = schedulers_list[:30]
limited_json = json.dumps(limited_schedulers, ensure_ascii=False, indent=2)
return f"注意:查询结果共找到 {total_count} 条,为节省上下文空间,仅显示前 30 条结果。\n\n{limited_json}"
limited_json = json.dumps(
limited_schedulers,
ensure_ascii=False,
indent=2,
)
return (
f"注意:查询结果共找到 {total_count} 条,为节省上下文空间,"
f"仅显示前 30 条结果。\n\n{limited_json}"
)
return result_json
return "未找到定时服务"
return "未找到系统、插件或工作流定时服务"
except Exception as e:
logger.error(f"查询定时服务失败: {e}", exc_info=True)
return f"查询定时服务时发生错误: {str(e)}"
+78
View File
@@ -0,0 +1,78 @@
"""立即执行 Agent 自主定时任务工具。"""
from typing import Optional, Type
from pydantic import BaseModel, Field
from app.agent.tools.base import MoviePilotTool
from app.agent.tools.tags import ToolTag
from app.db.agenttask_oper import AgentTaskOper
class RunAgentTaskInput(BaseModel):
"""立即执行 Agent 自主定时任务的输入参数。"""
task_id: int = Field(
...,
ge=1,
description=(
"Integer autonomous task ID returned by query_agent_tasks. Do not pass a "
"runtime scheduler job_id such as agent-task-12."
),
)
class RunAgentTaskTool(MoviePilotTool):
"""将当前用户的 Agent 自主定时任务提交为立即执行。"""
name: str = "run_agent_task"
tags: list[str] = [ToolTag.Write, ToolTag.AgentTask, ToolTag.Admin]
description: str = (
"Queue an enabled autonomous agent task owned by the current user for immediate "
"execution. Use the integer task_id returned by query_agent_tasks. The task runs "
"after the current agent turn can finish and broadcasts its result through the "
"configured notification channels."
)
args_schema: Type[BaseModel] = RunAgentTaskInput
require_admin: bool = True
def get_tool_message(self, **kwargs: object) -> Optional[str]:
"""生成立即执行 Agent 任务的提示消息。"""
return f"立即执行自主定时任务:{kwargs.get('task_id', '')}"
def _get_task_state(self, task_id: int) -> tuple[str, Optional[str]]:
"""校验任务归属和状态,返回可执行性及任务名称。"""
task = AgentTaskOper().get(
task_id=task_id,
user_id=str(self._user_id),
)
if not task:
return "not_found", None
if not task.enabled:
return "disabled", task.name
if task.last_status == "running":
return "running", task.name
return "ready", task.name
async def run(self, task_id: int, **kwargs: object) -> str:
"""立即执行当前用户拥有且已启用的 Agent 自主定时任务。"""
from app.scheduler import Scheduler
payload = RunAgentTaskInput(task_id=task_id)
status, task_name = await self.run_blocking(
"db",
self._get_task_state,
payload.task_id,
)
if status == "not_found":
return f"Agent 定时任务 {task_id} 不存在或不属于当前用户"
if status == "disabled":
return f"Agent 定时任务 {task_id} 已暂停,请先恢复后再执行"
if status == "running":
return f"Agent 定时任务 {task_id} 正在执行,请勿重复触发"
if not Scheduler().start_agent_task(payload.task_id):
return f"Agent 定时任务 {task_id} 尚未注册到运行时调度器,无法立即执行"
return (
f"Agent 定时任务 {task_id} 已提交立即执行:{task_name}"
"执行完成后将通过已配置的通知渠道广播结果"
)
+22 -5
View File
@@ -14,23 +14,32 @@ class RunSchedulerInput(BaseModel):
job_id: str = Field(
...,
description="The ID of the scheduled job to run (can be obtained from query_schedulers tool)",
description=(
"Runtime scheduler job ID returned by query_schedulers. Do not pass an "
"autonomous agent task ID or an agent-task-* runtime ID."
),
)
class RunSchedulerTool(MoviePilotTool):
"""立即运行系统、插件或工作流注册的定时服务。"""
name: str = "run_scheduler"
tags: list[str] = [
ToolTag.Write,
ToolTag.Scheduler,
ToolTag.Admin,
]
description: str = "Manually trigger a scheduled task to run immediately. This will execute the specified scheduler job by its ID."
description: str = (
"Manually trigger a MoviePilot system, plugin, or workflow scheduler service by "
"the runtime job_id returned from query_schedulers. This tool does not run "
"user-created autonomous agent tasks; use run_agent_task with an integer task_id."
)
args_schema: Type[BaseModel] = RunSchedulerInput
require_admin: bool = True
def get_tool_message(self, **kwargs) -> Optional[str]:
"""根据运行参数生成友好的提示消息"""
def get_tool_message(self, **kwargs: object) -> Optional[str]:
"""根据运行参数生成友好的提示消息"""
job_id = kwargs.get("job_id", "")
return f"运行定时服务 (ID: {job_id})"
@@ -46,10 +55,18 @@ class RunSchedulerTool(MoviePilotTool):
return True, scheduler_item.name
return False, ""
async def run(self, job_id: str, **kwargs) -> str:
async def run(self, job_id: str, **kwargs: object) -> str:
"""立即运行非 Agent 自主任务的运行时定时服务。"""
logger.info(f"执行工具: {self.name}, 参数: job_id={job_id}")
try:
from app.scheduler import AGENT_TASK_JOB_PREFIX
if job_id.startswith(f"{AGENT_TASK_JOB_PREFIX}-"):
return (
"Agent 自主定时任务不能通过 run_scheduler 运行,"
"请使用 query_agent_tasks 查询整数 task_id 后调用 run_agent_task"
)
job_exists, job_name = await self.run_blocking(
"workflow", self._run_scheduler_sync, job_id
)
+1 -1
View File
@@ -43,7 +43,7 @@ class SearchMediaTool(MoviePilotTool):
message += f" ({year})"
if media_type:
message += f" [{media_type}]"
if season:
if season is not None:
message += f"{season}"
return message
+1
View File
@@ -105,6 +105,7 @@ class SendLocalFileTool(MoviePilotTool):
text=message,
file_path=str(resolved_path),
file_name=file_name or resolved_path.name,
save_history=False,
)
)
return "本地附件已发送"
+1
View File
@@ -100,6 +100,7 @@ class SendMessageTool(MoviePilotTool):
title=title,
text=text,
image=image_url,
save_history=False,
)
)
self._agent_context["user_reply_sent"] = True
@@ -96,6 +96,7 @@ class SendVoiceMessageTool(MoviePilotTool):
if voice_path and settings.AUDIO_OUTPUT_INCLUDE_TEXT
else None
),
save_history=False,
)
)
self._agent_context["user_reply_sent"] = True
+193
View File
@@ -0,0 +1,193 @@
import json
from datetime import datetime, timedelta
from typing import Literal, Optional, Type
import pytz
from pydantic import BaseModel, Field, model_validator
from app.agent.tools.base import MoviePilotTool
from app.agent.tools.tags import ToolTag
from app.core.config import settings
from app.db.agenttask_oper import AgentTaskOper
from app.utils.timer import TimerUtils
class UpdateAgentTaskInput(BaseModel):
"""更新 Agent 自主定时任务的输入参数。"""
task_id: int = Field(..., ge=1, description="ID of the task to update.")
name: Optional[str] = Field(None, min_length=1, max_length=100)
content: Optional[str] = Field(None, min_length=1, max_length=10000)
trigger_type: Optional[Literal["date", "cron"]] = Field(
None,
description="New trigger type. Must be provided together with trigger.",
)
trigger: Optional[str] = Field(
None,
min_length=1,
max_length=200,
description="New ISO 8601 date or five-field cron expression.",
)
delay_minutes: Optional[int] = Field(
None,
ge=1,
le=525600,
description=(
"For a one-time date task expressed as 'in N minutes', provide this instead "
"of trigger together with trigger_type='date'."
),
)
enabled: Optional[bool] = Field(
None,
description="Set false to pause the task or true to resume it.",
)
@model_validator(mode="after")
def validate_update(self) -> "UpdateAgentTaskInput":
"""校验更新内容和触发参数组合。"""
if self.name is not None:
self.name = self.name.strip()
if not self.name:
raise ValueError("name 不能只包含空白字符")
if self.content is not None:
self.content = self.content.strip()
if not self.content:
raise ValueError("content 不能只包含空白字符")
has_schedule_update = any(
value is not None
for value in (self.trigger_type, self.trigger, self.delay_minutes)
)
if has_schedule_update:
if self.trigger_type is None:
raise ValueError("修改触发配置时必须提供 trigger_type")
if self.trigger_type == "date":
if self.delay_minutes is not None:
# 保持校验幂等,具体绝对时间在更新调度前只计算一次。
self.trigger = None
elif self.trigger is None:
raise ValueError("date 任务必须提供 trigger 或 delay_minutes")
elif self.trigger is None or self.delay_minutes is not None:
raise ValueError("cron 任务必须提供 trigger,且不能提供 delay_minutes")
if all(
value is None
for value in (
self.name,
self.content,
self.trigger_type,
self.enabled,
)
):
raise ValueError("至少需要提供一个要更新的字段")
return self
class UpdateAgentTaskTool(MoviePilotTool):
"""修改、暂停或恢复 Agent 自主定时任务。"""
name: str = "update_agent_task"
tags: list[str] = [ToolTag.Write, ToolTag.AgentTask, ToolTag.Admin]
description: str = (
"Update an autonomous agent task's name, instructions, exact date or cron "
"trigger, relative delay_minutes, or enabled state. Use enabled=false to pause "
"and enabled=true to resume."
)
args_schema: Type[BaseModel] = UpdateAgentTaskInput
require_admin: bool = True
def get_tool_message(self, **kwargs: object) -> Optional[str]:
"""生成更新定时任务的提示消息。"""
return f"更新自主定时任务:{kwargs.get('task_id', '')}"
def _update_task(self, payload: UpdateAgentTaskInput) -> Optional[dict]:
"""更新当前用户的任务并刷新运行时调度。"""
from app.scheduler import Scheduler
oper = AgentTaskOper()
task = oper.get(task_id=payload.task_id, user_id=str(self._user_id))
if not task:
return None
if task.last_status == "running":
return {"error": f"Agent 定时任务 {payload.task_id} 正在执行,请稍后再修改"}
trigger_type = payload.trigger_type or task.trigger_type
trigger_value = payload.trigger
if trigger_type == "date" and payload.delay_minutes is not None:
timezone = pytz.timezone(settings.TZ)
trigger_value = (
datetime.now(timezone) + timedelta(minutes=payload.delay_minutes)
).isoformat(timespec="seconds")
if trigger_value is None:
trigger_value = (
task.cron_expression if trigger_type == "cron" else task.run_at
)
enabled = task.enabled if payload.enabled is None else payload.enabled
normalized_type, normalized_trigger = TimerUtils.normalize_schedule_trigger(
trigger_type=trigger_type,
trigger_value=trigger_value,
timezone_name=settings.TZ,
require_future=bool(enabled and trigger_type == "date"),
)
update_payload = {}
if payload.name is not None:
update_payload["name"] = payload.name.strip()
if payload.content is not None:
update_payload["content"] = payload.content.strip()
if payload.trigger_type is not None:
update_payload.update(
{
"trigger_type": normalized_type,
"cron_expression": (
normalized_trigger if normalized_type == "cron" else None
),
"run_at": normalized_trigger if normalized_type == "date" else None,
"last_status": "waiting",
"last_result": None,
}
)
if payload.enabled is not None:
update_payload["enabled"] = payload.enabled
if payload.enabled:
update_payload["last_status"] = "waiting"
oper.update(
task_id=payload.task_id,
payload=update_payload,
user_id=str(self._user_id),
)
scheduler = Scheduler()
next_run_at = scheduler.update_agent_task_job(payload.task_id)
updated_task = oper.get(task_id=payload.task_id, user_id=str(self._user_id))
return oper.to_dict(
updated_task,
next_run_at=next_run_at,
timezone=settings.TZ,
)
async def run(
self,
task_id: int,
name: Optional[str] = None,
content: Optional[str] = None,
trigger_type: Optional[str] = None,
trigger: Optional[str] = None,
delay_minutes: Optional[int] = None,
enabled: Optional[bool] = None,
**kwargs: object,
) -> str:
"""更新 Agent 自主定时任务。"""
payload = UpdateAgentTaskInput(
task_id=task_id,
name=name,
content=content,
trigger_type=trigger_type,
trigger=trigger,
delay_minutes=delay_minutes,
enabled=enabled,
)
task = await self.run_blocking("db", self._update_task, payload)
if not task:
return f"Agent 定时任务 {task_id} 不存在或不属于当前用户"
if task.get("error"):
return task["error"]
return json.dumps(task, ensure_ascii=False, indent=2)
@@ -7,6 +7,7 @@ from pydantic import BaseModel, Field
from app.agent.tools.base import MoviePilotTool
from app.agent.tools.tags import ToolTag
from app.core.metainfo import clear_rust_parse_options_cache
from app.db.systemconfig_oper import SystemConfigOper
from app.log import logger
from app.schemas.types import SystemConfigKey
@@ -85,6 +86,7 @@ class UpdateCustomIdentifiersTool(MoviePilotTool):
SystemConfigKey.CustomIdentifiers, value
)
if success:
clear_rust_parse_options_cache()
return json.dumps(
{
"success": True,
@@ -8,6 +8,7 @@ from pydantic import BaseModel, Field
from app.agent.tools.base import MoviePilotTool
from app.agent.tools.tags import ToolTag
from app.chain.download import DownloadChain
from app.helper.directory import validate_download_save_path
from app.log import logger
@@ -150,6 +151,18 @@ class UpdateDownloadTasksTool(MoviePilotTool):
],
}
if save_path is not None:
try:
save_path = validate_download_save_path(save_path)
except ValueError:
return {
"hash": hash_value,
"downloader": resolved_downloader,
"results": [
cls._build_result("save_path", False, "保存目录不在允许的下载目录范围内")
],
}
results = []
if tags:
tag_result = download_chain.set_torrents_tag(
+7 -5
View File
@@ -10,6 +10,7 @@ from app.agent.tools.tags import ToolTag
from app.core.event import eventmanager
from app.db.subscribe_oper import SubscribeOper
from app.log import logger
from app.schemas.event import SubscribeModifiedEventData
from app.schemas.types import EventType
@@ -261,13 +262,14 @@ class UpdateSubscribeTool(MoviePilotTool):
# 发送订阅调整事件
await eventmanager.async_send_event(
EventType.SubscribeModified,
{
"subscribe_id": subscribe_id,
"old_subscribe_info": old_subscribe_dict,
"subscribe_info": updated_subscribe.to_dict()
SubscribeModifiedEventData(
subscribe_id=subscribe_id,
old_subscribe_info=old_subscribe_dict,
subscribe_info=updated_subscribe.to_dict()
if updated_subscribe
else {},
},
scene="agent_update",
).to_dict(),
)
# 构建返回结果
+1 -1
View File
@@ -26,7 +26,7 @@ class WriteFileTool(MoviePilotTool):
]
description: str = (
"Write full content to a local text file. Non-admin users can only write "
"inside the MoviePilot Agent config and log directories."
"inside the MoviePilot Agent config directory."
)
args_schema: Type[BaseModel] = WriteFileInput
+1
View File
@@ -25,6 +25,7 @@ class ToolTag(str, Enum):
Plugin = "plugin"
Workflow = "workflow"
Scheduler = "scheduler"
AgentTask = "agent_task"
File = "file"
Directory = "directory"
Web = "web"
+111 -6
View File
@@ -20,6 +20,7 @@ from sqlalchemy.ext.asyncio import AsyncSession
from app import schemas
from app.agent import MoviePilotAgent, ReplyMode, StreamingHandler, agent_manager
from app.agent.llm.capability import AgentCapabilityManager
from app.agent.mcp import agent_mcp_manager
from app.chain.message import MessageChain
from app.chain.site import site_interaction_manager
from app.chain.skills import skills_interaction_manager
@@ -34,6 +35,7 @@ from app.db.models.agentchat import AgentChat
from app.db.user_oper import UserOper, get_current_active_user
from app.helper.agent import attach_web_agent_edit_queue, detach_web_agent_edit_queue
from app.helper.interaction import agent_interaction_manager, media_interaction_manager
from app.helper.locale import LocaleHelper
from app.log import logger
from app.schemas.types import EventType, MessageChannel
@@ -55,6 +57,78 @@ _WEB_AGENT_NOTICE_LISTENER_REGISTERED = False
_WEB_AGENT_BACKGROUND_TASKS: set[asyncio.Task] = set()
def _ensure_superuser(user: User) -> None:
"""校验当前用户是否为超级管理员。"""
if not getattr(user, "is_superuser", False):
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Forbidden")
@router.get("/mcp/servers", summary="查询 Agent MCP 服务器配置", response_model=schemas.Response)
async def list_agent_mcp_servers(
current_user: User = Depends(get_current_active_user),
) -> schemas.Response:
"""
查询 Agent 外部 MCP 服务器配置
"""
_ensure_superuser(current_user)
servers = agent_mcp_manager.get_servers()
enabled_count = len([server for server in servers if server.enabled])
return schemas.Response(
success=True,
data={
"servers": [server.model_dump() for server in servers],
"enabled_count": enabled_count,
"total_count": len(servers),
},
)
@router.post("/mcp/servers", summary="保存 Agent MCP 服务器配置", response_model=schemas.Response)
async def save_agent_mcp_servers(
request: schemas.AgentMcpServersSaveRequest,
current_user: User = Depends(get_current_active_user),
) -> schemas.Response:
"""
保存 Agent 外部 MCP 服务器配置
"""
_ensure_superuser(current_user)
success = await agent_mcp_manager.save_servers(request.servers)
return schemas.Response(
success=success,
message="保存MCP配置成功" if success else "保存MCP配置失败",
)
@router.post("/mcp/servers/test", summary="测试 Agent MCP 服务器", response_model=schemas.Response)
async def test_agent_mcp_server(
request: schemas.AgentMcpServerTestRequest,
current_user: User = Depends(get_current_active_user),
) -> schemas.Response:
"""
测试 Agent 外部 MCP 服务器连接并读取工具列表
"""
_ensure_superuser(current_user)
try:
result = await agent_mcp_manager.test_server(request.server)
return schemas.Response(
success=result.success,
message=result.message,
data=result.model_dump(),
)
except Exception as err:
logger.warning(f"测试 Agent MCP 服务器失败: {err}")
return schemas.Response(
success=False,
message=f"测试MCP服务器失败: {str(err)}",
data={
"success": False,
"message": str(err),
"tools": [],
"tool_count": 0,
},
)
class _WebAgentStreamingHandler(StreamingHandler):
"""
Web 前端专用流式处理器将工具提示和文本统一回调给 SSE
@@ -145,7 +219,9 @@ class _WebAgentMoviePilotAgent(MoviePilotAgent):
self.stream_handler = _WebAgentStreamingHandler(self._emit_output)
def _should_stream(self) -> bool:
"""Web 面板需要实时输出,即使 Web 渠道本身不支持消息编辑"""
"""Web 对话实时输出,复用会话执行后台任务时改用非流式广播"""
if self.is_background:
return False
return True
def set_notification_callback(
@@ -326,15 +402,25 @@ def _save_web_agent_display_snapshot(
logger.debug(f"保存WebAgent展示历史失败: {e}")
def _build_web_agent_sse(event_type: str, data: Optional[dict] = None) -> str:
def _build_web_agent_sse(
event_type: str,
data: Optional[dict] = None,
locale: Optional[str] = None,
) -> str:
"""
构建 Web Agent SSE 消息
:param event_type: 前端事件类型
:param data: 事件数据
:param locale: 当前请求语言
:return: 符合 SSE 格式的字符串
"""
payload = {"type": event_type, **(data or {})}
message = payload.get("message")
if event_type == "error" and isinstance(message, str):
payload["message_i18n"] = LocaleHelper.translate_text(
message, locale=locale
)
return f"data: {json.dumps(payload, ensure_ascii=False)}\n\n"
@@ -1597,6 +1683,7 @@ async def web_agent_stream(
:return: SSE 流式响应
"""
prompt = payload.text.strip()
locale = LocaleHelper.get_locale_from_request(request)
display_prompt = (payload.display_text or payload.text).strip()
is_traditional_message = (
_is_web_agent_traditional_message(prompt)
@@ -1610,6 +1697,7 @@ async def web_agent_stream(
_build_web_agent_sse(
"error",
{"message": denied_message},
locale=locale,
)
]),
media_type="text/event-stream",
@@ -1621,6 +1709,7 @@ async def web_agent_stream(
_build_web_agent_sse(
"error",
{"message": unknown_command_message},
locale=locale,
)
]),
media_type="text/event-stream",
@@ -1649,7 +1738,11 @@ async def web_agent_stream(
"""
生成传统消息链路的 WebAgent SSE 事件
"""
yield _build_web_agent_sse("start", {"session_id": session_id})
yield _build_web_agent_sse(
"start",
{"session_id": session_id},
locale=locale,
)
events = await _collect_web_agent_traditional_events(
text=prompt,
current_user=current_user,
@@ -1660,7 +1753,11 @@ async def web_agent_stream(
display_messages.append(assistant_message)
for event in events:
event_payload = copy.deepcopy(event)
yield _build_web_agent_sse(event_payload.pop("type"), event_payload)
yield _build_web_agent_sse(
event_payload.pop("type"),
event_payload,
locale=locale,
)
if await request.is_disconnected():
break
await run_in_threadpool(
@@ -1670,7 +1767,7 @@ async def web_agent_stream(
messages=display_messages,
client_session_id=payload.session_id or session_id,
)
yield _build_web_agent_sse("done", {})
yield _build_web_agent_sse("done", {}, locale=locale)
return StreamingResponse(
traditional_event_generator(),
@@ -1688,6 +1785,7 @@ async def web_agent_stream(
_build_web_agent_sse(
"error",
{"message": "智能助手未启用,请先在系统设置中开启。"},
locale=locale,
)
]),
media_type="text/event-stream",
@@ -1703,6 +1801,7 @@ async def web_agent_stream(
_build_web_agent_sse(
"error",
{"message": "语音识别失败,请稍后重试。"},
locale=locale,
)
]),
media_type="text/event-stream",
@@ -1713,6 +1812,7 @@ async def web_agent_stream(
_build_web_agent_sse(
"error",
{"message": "请输入要发送给智能助手的内容或选择附件。"},
locale=locale,
)
]),
media_type="text/event-stream",
@@ -1825,6 +1925,7 @@ async def web_agent_stream(
yield _build_web_agent_sse(
"start",
{"session_id": session_id},
locale=locale,
)
disconnected = False
while not global_vars.is_system_stopped:
@@ -1832,7 +1933,11 @@ async def web_agent_stream(
disconnected = True
break
event = await event_queue.get()
yield _build_web_agent_sse(event.pop("type"), event)
yield _build_web_agent_sse(
event.pop("type"),
event,
locale=locale,
)
if task.done() and event_queue.empty():
break
except asyncio.CancelledError:
+4
View File
@@ -39,6 +39,9 @@ def _anthropic_error_response(
def _check_auth(api_key: Optional[str]) -> Optional[JSONResponse]:
"""
Anthropic 兼容接口以 API_TOKEN 认证受信客户端认证通过即按管理员级 Agent 集成处理
"""
if not api_key or api_key != settings.API_TOKEN:
return _anthropic_error_response(
"invalid x-api-key",
@@ -122,6 +125,7 @@ async def messages(
session_seed = anthropic_version or "anthropic"
session_id = build_session_id(f"{session_seed}:{uuid.uuid4().hex}", SESSION_PREFIX)
# 兼容接口的 API_TOKEN 客户端按管理员级 MoviePilot Agent 集成处理。
agent = _CollectingMoviePilotAgent(
session_id=session_id,
user_id=session_id,
+69 -12
View File
@@ -18,7 +18,7 @@ from app.utils.system import SystemUtils
router = APIRouter()
def _build_statistic(name: Optional[str] = None) -> schemas.Statistic:
def _build_statistic(db: Session, name: Optional[str] = None) -> schemas.Statistic:
"""
构建媒体数量统计信息
"""
@@ -39,8 +39,14 @@ def _build_statistic(name: Optional[str] = None) -> schemas.Statistic:
if not has_episode_count:
# 所有媒体服务都未提供剧集统计时,返回 None 供前端展示“未获取”。
ret_statistic.episode_count = None
return ret_statistic
return schemas.Statistic()
else:
ret_statistic = schemas.Statistic()
movie_count_month, tv_count_month, episode_count_month = TransferHistory.monthly_media_statistics(db)
ret_statistic.movie_count_month = movie_count_month
ret_statistic.tv_count_month = tv_count_month
ret_statistic.episode_count_month = episode_count_month
return ret_statistic
def _build_storage() -> schemas.Storage:
@@ -84,22 +90,27 @@ def _build_downloader(name: Optional[str] = None) -> schemas.DownloaderInfo:
@router.get("/statistic", summary="媒体数量统计", response_model=schemas.Statistic)
def statistic(
name: Optional[str] = None, _: Any = Depends(get_current_active_superuser)
name: Optional[str] = None,
db: Session = Depends(get_db),
_: Any = Depends(get_current_active_superuser),
) -> Any:
"""
查询媒体数量统计信息
"""
return _build_statistic(name)
return _build_statistic(db, name)
@router.get(
"/statistic2", summary="媒体数量统计(API_TOKEN", response_model=schemas.Statistic
)
def statistic2(_: Annotated[str, Depends(verify_apitoken)]) -> Any:
def statistic2(
_: Annotated[str, Depends(verify_apitoken)],
db: Session = Depends(get_db),
) -> Any:
"""
查询媒体数量统计信息 API_TOKEN认证?token=xxx
"""
return _build_statistic()
return _build_statistic(db)
@router.get("/storage", summary="本地存储空间", response_model=schemas.Storage)
@@ -128,6 +139,14 @@ def processes(_: Any = Depends(get_current_active_superuser)) -> Any:
return SystemUtils.processes()
@router.get("/system", summary="系统摘要信息", response_model=schemas.DashboardSystemInfo)
def system_info(_: Any = Depends(get_current_active_superuser)) -> Any:
"""
查询仪表板系统摘要信息
"""
return SystemUtils.dashboard_system_info()
@router.get("/downloader", summary="下载器信息", response_model=schemas.DownloaderInfo)
def downloader(
name: Optional[str] = None, _: Any = Depends(get_current_active_superuser)
@@ -158,6 +177,23 @@ async def schedule(_: Any = Depends(get_current_active_superuser)) -> Any:
return Scheduler().list()
@router.get(
"/schedule/{job_id}/progress",
summary="后台服务进度",
response_model=schemas.Response,
)
async def schedule_progress(
job_id: str, _: Any = Depends(get_current_active_superuser)
) -> Any:
"""
查询指定后台服务的执行进度
"""
progress = Scheduler().get_progress(job_id)
if not progress:
return schemas.Response(success=False, message="后台服务不存在")
return schemas.Response(success=True, data=progress.model_dump())
@router.get(
"/schedule2",
summary="后台服务(API_TOKEN",
@@ -170,6 +206,23 @@ async def schedule2(_: Annotated[str, Depends(verify_apitoken)]) -> Any:
return Scheduler().list()
@router.get(
"/schedule2/{job_id}/progress",
summary="后台服务进度(API_TOKEN",
response_model=schemas.Response,
)
async def schedule_progress2(
job_id: str, _: Annotated[str, Depends(verify_apitoken)]
) -> Any:
"""
查询指定后台服务的执行进度 API_TOKEN认证?token=xxx
"""
progress = Scheduler().get_progress(job_id)
if not progress:
return schemas.Response(success=False, message="后台服务不存在")
return schemas.Response(success=True, data=progress.model_dump())
@router.get("/transfer", summary="文件整理统计", response_model=List[int])
async def transfer(
days: Optional[int] = 7,
@@ -199,22 +252,26 @@ def cpu2(_: Annotated[str, Depends(verify_apitoken)]) -> Any:
return SystemUtils.cpu_usage()
@router.get("/memory", summary="获取当前内存使用量和使用率", response_model=List[int])
@router.get(
"/memory",
summary="获取当前应用与系统内存信息",
response_model=schemas.DashboardMemoryInfo,
)
def memory(_: Any = Depends(get_current_active_superuser)) -> Any:
"""
获取当前内存使用率
获取当前应用与系统内存信息
"""
return SystemUtils.memory_usage()
@router.get(
"/memory2",
summary="获取当前内存使用量和使用率API_TOKEN",
response_model=List[int],
summary="获取当前应用与系统内存信息API_TOKEN",
response_model=schemas.DashboardMemoryInfo,
)
def memory2(_: Annotated[str, Depends(verify_apitoken)]) -> Any:
"""
获取当前内存使用率 API_TOKEN认证?token=xxx
获取当前应用与系统内存信息 API_TOKEN认证?token=xxx
"""
return SystemUtils.memory_usage()
+50
View File
@@ -6,11 +6,61 @@ from app import schemas
from app.chain.douban import DoubanChain
from app.core.context import MediaInfo
from app.core.security import verify_token
from app.db.models.user import User
from app.db.user_oper import get_current_active_superuser_async
from app.modules.douban.douban_cache import DoubanCache
from app.schemas import MediaType
router = APIRouter()
@router.get(
"/cache", summary="查询豆瓣识别缓存", response_model=schemas.Response
)
async def douban_recognition_cache(
_: User = Depends(get_current_active_superuser_async),
) -> schemas.Response:
"""查询可管理的豆瓣识别缓存。"""
cache_items = DoubanCache().list_items()
recognized_count = sum(1 for item in cache_items if item["douban_id"])
return schemas.Response(
success=True,
data={
"count": len(cache_items),
"recognized": recognized_count,
"unrecognized": len(cache_items) - recognized_count,
"data": cache_items,
},
)
@router.delete(
"/cache/{cache_key:path}",
summary="删除指定豆瓣识别缓存",
response_model=schemas.Response,
)
async def delete_douban_recognition_cache(
cache_key: str,
_: User = Depends(get_current_active_superuser_async),
) -> schemas.Response:
"""按缓存键删除单条豆瓣识别缓存。"""
deleted_item = DoubanCache().delete(cache_key)
if not deleted_item:
return schemas.Response(success=False, message="豆瓣识别缓存不存在")
return schemas.Response(success=True, message="豆瓣识别缓存删除成功")
@router.delete(
"/cache", summary="清空豆瓣识别缓存", response_model=schemas.Response
)
async def clear_douban_recognition_cache(
_: User = Depends(get_current_active_superuser_async),
) -> schemas.Response:
"""清空全部豆瓣识别缓存。"""
DoubanCache().clear()
return schemas.Response(success=True, message="豆瓣识别缓存清理完成")
@router.get(
"/person/{person_id}", summary="人物详情", response_model=schemas.MediaPerson
)
+31
View File
@@ -9,14 +9,41 @@ from app.core.context import MediaInfo, Context, SubtitleInfo, TorrentInfo
from app.core.metainfo import MetaInfo
from app.core.security import verify_token
from app.db.models.user import User
from app.db.site_oper import SiteOper
from app.db.systemconfig_oper import SystemConfigOper
from app.db.user_oper import get_current_active_user
from app.helper.directory import DirectoryHelper
from app.schemas.types import SystemConfigKey
from app.utils.security import SecurityUtils
router = APIRouter()
def _prepare_subtitle_download(subtitle: SubtitleInfo) -> tuple[bool, str]:
"""
校验字幕下载签名并用服务端站点配置覆盖请求凭据
"""
if subtitle.site is None:
return False, "字幕站点信息为空"
clean_url = SecurityUtils.verify_signed_url(
subtitle.enclosure,
purpose=SecurityUtils.subtitle_download_purpose(subtitle.site),
)
if not clean_url:
return False, "字幕下载链接签名无效"
site = SiteOper().get(subtitle.site)
if not site:
return False, "字幕站点信息不存在"
subtitle.enclosure = clean_url
subtitle.site_cookie = site.cookie
subtitle.site_ua = site.ua
subtitle.site_proxy = bool(site.proxy)
return True, ""
@router.get("/", summary="正在下载", response_model=List[schemas.DownloaderTorrent])
def current(
name: Optional[str] = None, _: schemas.TokenPayload = Depends(verify_token)
@@ -127,6 +154,10 @@ def download_subtitle(
"""
subtitle_info = SubtitleInfo()
subtitle_info.from_dict(subtitle_in.model_dump())
valid, message = _prepare_subtitle_download(subtitle_info)
if not valid:
return schemas.Response(success=False, message=message)
success, message, saved_files = DownloadChain().download_subtitle(
subtitle=subtitle_info,
tmdbid=tmdbid,
+5 -4
View File
@@ -22,8 +22,9 @@ from app.db.models import User
from app.db.models.downloadhistory import DownloadHistory, DownloadFiles
from app.db.models.transferhistory import TransferHistory
from app.db.user_oper import (
get_current_active_superuser_async,
get_current_active_manage_user,
get_current_active_superuser,
get_current_active_superuser_async,
)
from app.helper.progress import ProgressHelper
from app.schemas.types import EventType
@@ -223,7 +224,7 @@ def delete_transfer_history(
deletesrc: Optional[bool] = False,
deletedest: Optional[bool] = False,
db: Session = Depends(get_db),
_: User = Depends(get_current_active_superuser),
_: User = Depends(get_current_active_manage_user),
) -> Any:
"""
删除整理记录
@@ -264,7 +265,7 @@ def delete_transfer_history(
def ai_redo_transfer_history(
history_id: int,
db: Session = Depends(get_db),
_: User = Depends(get_current_active_superuser),
_: User = Depends(get_current_active_manage_user),
) -> Any:
"""
手动触发单条历史记录的 AI 重新整理并返回进度键
@@ -293,7 +294,7 @@ def ai_redo_transfer_history(
def batch_ai_redo_transfer_history(
payload: schemas.BatchTransferHistoryRedoRequest,
db: Session = Depends(get_db),
_: User = Depends(get_current_active_superuser),
_: User = Depends(get_current_active_manage_user),
) -> Any:
"""
手动触发多条历史记录的 AI 批量重新整理并返回进度键
+28 -17
View File
@@ -36,6 +36,7 @@ class LlmTestRequest(BaseModel):
base_url: Optional[str] = None
base_url_preset: Optional[str] = None
user_agent: Optional[str] = None
temperature: Optional[float] = None
use_proxy: Optional[bool] = None
@@ -48,7 +49,7 @@ class LlmProviderAuthStartRequest(BaseModel):
method: str
def _sanitize_llm_test_error(message: str, api_key: Optional[str] = None) -> str:
def _sanitize_llm_error(message: str, api_key: Optional[str] = None) -> str:
"""
清理错误信息中的敏感字段避免回显密钥
"""
@@ -70,11 +71,14 @@ def _sanitize_llm_test_error(message: str, api_key: Optional[str] = None) -> str
)
normalized_message = sanitized.lower().replace("_", "").replace(" ", "")
if "str" in normalized_message and "modeldump" in normalized_message:
if "str" in normalized_message and (
"modeldump" in normalized_message
or "setprivateattributes" in normalized_message
):
return (
"服务返回内容不是兼容的模型响应,"
"请检查基础地址是否填写为 API Base URL,不要填写网页地址或完整的 "
"chat/completions 路径"
"服务返回内容不是兼容的模型响应,请检查基础地址是否填写为 "
"API Base URL,如果服务要求 /v1 等版本路径,请包含在基础地址中,"
"不要填写网页地址或完整的 chat/completions 路径"
)
return sanitized
@@ -113,7 +117,10 @@ async def get_llm_models(
},
)
except Exception as err:
return schemas.Response(success=False, message=str(err))
return schemas.Response(
success=False,
message=_sanitize_llm_error(str(err), api_key),
)
@router.get("/providers", summary="获取LLM提供商目录", response_model=schemas.Response)
@@ -286,16 +293,20 @@ async def llm_test(
)
try:
result = await LLMHelper.test_current_settings(
provider=payload.provider,
model=payload.model,
thinking_level=payload.thinking_level,
api_key=payload.api_key,
base_url=payload.base_url,
base_url_preset=payload.base_url_preset,
user_agent=payload.user_agent,
use_proxy=payload.use_proxy,
)
test_kwargs = {
"provider": payload.provider,
"model": payload.model,
"thinking_level": payload.thinking_level,
"api_key": payload.api_key,
"base_url": payload.base_url,
"base_url_preset": payload.base_url_preset,
"user_agent": payload.user_agent,
"use_proxy": payload.use_proxy,
}
if payload.temperature is not None:
test_kwargs["temperature"] = payload.temperature
result = await LLMHelper.test_current_settings(**test_kwargs)
if not result.get("reply_preview"):
return schemas.Response(
success=False,
@@ -312,5 +323,5 @@ async def llm_test(
except Exception as err:
return schemas.Response(
success=False,
message=_sanitize_llm_test_error(str(err), payload.api_key),
message=_sanitize_llm_error(str(err), payload.api_key),
)
+1
View File
@@ -19,6 +19,7 @@ router = APIRouter()
# MCP 协议版本
MCP_PROTOCOL_VERSIONS = ["2025-11-25", "2025-06-18", "2024-11-05"]
MCP_PROTOCOL_VERSION = MCP_PROTOCOL_VERSIONS[0] # 默认使用最新版本
# MCP 经 API_TOKEN / X-API-KEY 认证后是管理员级集成入口;隐藏工具只收敛暴露面,不构成权限边界。
MCP_HIDDEN_TOOLS = {
"execute_command",
"search_web",
+8 -3
View File
@@ -26,13 +26,17 @@ router = APIRouter()
async def recognize(
title: str,
subtitle: Optional[str] = None,
custom_words: Optional[str] = None,
_: schemas.TokenPayload = Depends(verify_token),
) -> Any:
"""
根据标题副标题识别媒体信息
:param custom_words: 临时识别词每行一条规则传入时仅在本次识别中生效不会保存到系统配置
"""
# 识别媒体信息
metainfo = MetaInfo(title, subtitle)
# 识别媒体信息,传入临时识别词时优先于系统配置的识别词生效
metainfo = MetaInfo(
title, subtitle, custom_words=custom_words.split("\n") if custom_words else None
)
mediainfo = await MediaChain().async_recognize_by_meta(metainfo)
if mediainfo:
return Context(meta_info=metainfo, media_info=mediainfo).to_dict()
@@ -48,12 +52,13 @@ async def recognize2(
_: Annotated[str, Depends(verify_apitoken)],
title: str,
subtitle: Optional[str] = None,
custom_words: Optional[str] = None,
) -> Any:
"""
根据标题副标题识别媒体信息 API_TOKEN认证?token=xxx
"""
# 识别媒体信息
return await recognize(title, subtitle)
return await recognize(title, subtitle, custom_words)
@router.get(
+2 -1
View File
@@ -17,7 +17,7 @@ from app.db.message_oper import MessageOper
from app.db.systemconfig_oper import SystemConfigOper
from app.db.user_oper import get_current_active_superuser
from app.helper.service import ServiceConfigHelper
from app.helper.webpush import is_webpush_subscription_gone
from app.helper.webpush import is_webpush_subscription_gone, webpush_options_for_endpoint
from app.log import logger
from app.modules.wechat.WXBizMsgCrypt3 import WXBizMsgCrypt
from app.schemas.types import MessageChannel, SystemConfigKey
@@ -316,6 +316,7 @@ def send_notification(
data=json.dumps(payload.model_dump()),
vapid_private_key=settings.VAPID.get("privateKey"),
vapid_claims={"sub": settings.VAPID.get("subject")},
**webpush_options_for_endpoint(sub.get("endpoint")),
)
except WebPushException as err:
logger.error(f"WebPush发送失败: {str(err)}")
+5
View File
@@ -231,6 +231,9 @@ def _error_response(
def _check_auth(
credentials: Optional[HTTPAuthorizationCredentials],
) -> Optional[JSONResponse]:
"""
OpenAI 兼容接口以 API_TOKEN 认证受信客户端认证通过即按管理员级 Agent 集成处理
"""
if not credentials or credentials.scheme.lower() != "bearer":
return _error_response(
"Invalid bearer token.",
@@ -317,6 +320,7 @@ async def chat_completions(
session_id = build_session_id(session_key, SESSION_PREFIX)
username = str(payload.user or "openai-client")
# 兼容接口的 API_TOKEN 客户端按管理员级 MoviePilot Agent 集成处理。
agent = _CollectingMoviePilotAgent(
session_id=session_id,
user_id=session_key,
@@ -409,6 +413,7 @@ async def responses(
session_key = str(payload.user or uuid.uuid4())
session_id = build_session_id(session_key, SESSION_PREFIX)
# 兼容接口的 API_TOKEN 客户端按管理员级 MoviePilot Agent 集成处理。
agent = _CollectingMoviePilotAgent(
session_id=session_id,
user_id=session_key,
+84 -27
View File
@@ -1,3 +1,4 @@
import asyncio
import mimetypes
import shutil
from typing import Annotated, Any, List, Optional
@@ -39,6 +40,67 @@ PROTECTED_ROUTES = {"/api/v1/openapi.json", "/docs", "/docs/oauth2-redirect", "/
PLUGIN_PREFIX = f"{settings.API_V1_STR}/plugin"
router = APIRouter()
_plugin_release_refresh_tasks: set[asyncio.Task] = set()
async def _get_market_plugin_from_repo(
plugin_manager: PluginManager,
plugin_id: str,
repo_url: str,
force: bool,
) -> Optional[schemas.Plugin]:
"""
只读取指定插件仓库的市场元数据避免单插件详情触发全部市场刷新
"""
market_plugins = await plugin_manager.async_get_plugins_from_market(
repo_url, settings.VERSION_FLAG, force
)
market_plugin = next(
(
plugin
for plugin in market_plugins or []
if plugin.id == plugin_id
),
None,
)
if market_plugin or not settings.VERSION_FLAG:
return market_plugin
compatible_plugins = await plugin_manager.async_get_plugins_from_market(
repo_url, None, force
)
return next(
(
plugin
for plugin in compatible_plugins or []
if plugin.id == plugin_id
),
None,
)
async def _refresh_plugin_release_versions(plugin_id: str, repo_url: str) -> None:
"""
后台强制刷新 Release 缓存接口响应路径优先返回已有缓存
"""
try:
async with async_fresh(True):
await PluginHelper().async_get_plugin_release_versions(plugin_id, repo_url)
except Exception as e:
logger.warning(f"后台刷新插件 {plugin_id} Release 列表失败:{e}")
def _schedule_plugin_release_refresh(plugin_id: str, repo_url: str) -> None:
"""
保留后台任务引用避免任务被回收同时让 helper 负责同仓库强刷合并
"""
task = asyncio.create_task(_refresh_plugin_release_versions(plugin_id, repo_url))
_plugin_release_refresh_tasks.add(task)
def _discard_task(completed_task: asyncio.Task) -> None:
_plugin_release_refresh_tasks.discard(completed_task)
task.add_done_callback(_discard_task)
def register_plugin_api(plugin_id: Optional[str] = None):
@@ -239,6 +301,15 @@ async def _get_plugin_history_detail(
if local_repo_plugin:
return _merge_plugin_market_metadata(installed_plugin, local_repo_plugin)
if installed_plugin.repo_url:
market_plugin = await _get_market_plugin_from_repo(
plugin_manager, plugin_id, installed_plugin.repo_url, force
)
if not market_plugin:
logger.debug(f"插件 {plugin_id} 未从来源仓库获取到更新说明,返回本地插件信息")
return installed_plugin
return _merge_plugin_market_metadata(installed_plugin, market_plugin)
market_plugin = next(
(
plugin
@@ -359,30 +430,9 @@ async def plugin_releases(
}
plugin_manager = PluginManager()
market_plugins = await plugin_manager.async_get_plugins_from_market(
repo_url, settings.VERSION_FLAG, force
market_plugin = await _get_market_plugin_from_repo(
plugin_manager, plugin_id, repo_url, force
)
market_plugin = next(
(
plugin
for plugin in market_plugins or []
if plugin.id == plugin_id
),
None,
)
if not market_plugin and settings.VERSION_FLAG:
compatible_plugins = await plugin_manager.async_get_plugins_from_market(
repo_url, None, force
)
market_plugin = next(
(
plugin
for plugin in compatible_plugins or []
if plugin.id == plugin_id
),
None,
)
latest_version = market_plugin.plugin_version if market_plugin else None
current_version = plugin_manager.get_local_plugin_version(plugin_id)
if not getattr(market_plugin, "release", False):
@@ -393,8 +443,15 @@ async def plugin_releases(
"items": [],
}
async with async_fresh(force):
release_items = await PluginHelper().async_get_plugin_release_versions(plugin_id, repo_url)
plugin_helper = PluginHelper()
has_release_cache = (
await plugin_helper.async_has_plugin_release_cache(repo_url)
if force
else False
)
release_items = await plugin_helper.async_get_plugin_release_versions(plugin_id, repo_url)
if force and has_release_cache:
_schedule_plugin_release_refresh(plugin_id, repo_url)
items = []
for item in release_items:
version = item.get("version")
@@ -615,9 +672,9 @@ def reset_plugin(
# 事件处理器需要运行中插件完成补偿;补偿后先停止插件,避免删除数据时仍有任务读写旧状态。
plugin_manager.stop(plugin_id)
# 删除配置
plugin_manager.delete_plugin_config(plugin_id)
plugin_manager.delete_plugin_config(plugin_id, force=True)
# 删除插件所有数据
plugin_manager.delete_plugin_data(plugin_id)
plugin_manager.delete_plugin_data(plugin_id, force=True)
# 重新加载插件
reload_plugin(plugin_id)
return schemas.Response(success=True)
+104 -18
View File
@@ -12,9 +12,11 @@ from app.core.config import settings
from app.core.event import eventmanager
from app.core.metainfo import MetaInfo
from app.core.security import verify_resource_token, verify_token
from app.helper.locale import LocaleHelper
from app.log import logger
from app.schemas import MediaRecognizeConvertEventData
from app.schemas.types import MediaType, ChainEventType
from app.utils.security import SecurityUtils
router = APIRouter()
@@ -38,11 +40,75 @@ def _parse_media_type(mtype: Optional[str]) -> Optional[MediaType]:
return MediaType.from_agent(mtype) or MediaType(mtype)
def _sse_event(data: dict) -> str:
def _resolve_media_season(
explicit_season: Optional[int],
recognized_season: Optional[int],
) -> Optional[int]:
"""
合并显式季号与识别结果显式值优先且季 0 属于有效业务值
"""
return explicit_season if explicit_season is not None else recognized_season
def _sse_event(data: dict, locale: Optional[str] = None) -> str:
"""
转换为SSE事件
"""
return f"data: {json.dumps(data, ensure_ascii=False)}\n\n"
payload = data
message = payload.get("message")
text = payload.get("text")
if isinstance(message, str) or isinstance(text, str):
payload = data.copy()
if isinstance(message, str):
payload["message_i18n"] = LocaleHelper.translate_text(
message, locale=locale
)
if isinstance(text, str):
payload["text_i18n"] = LocaleHelper.translate_text(text, locale=locale)
return f"data: {json.dumps(payload, ensure_ascii=False)}\n\n"
def _serialize_signed_subtitle_result(subtitle: Any) -> dict:
"""
序列化字幕结果并签名下载链接签名用途绑定站点 ID
"""
data = subtitle.to_dict() if hasattr(subtitle, "to_dict") else dict(subtitle)
enclosure = data.get("enclosure")
if enclosure:
data["enclosure"] = SecurityUtils.sign_url(
enclosure,
purpose=SecurityUtils.subtitle_download_purpose(data.get("site")),
)
return data
def _serialize_signed_subtitle_results(subtitles: List[Any]) -> List[dict]:
"""
批量序列化字幕结果确保返回给客户端的下载链接均已签名
"""
return [_serialize_signed_subtitle_result(subtitle) for subtitle in subtitles]
def _sign_subtitle_search_event(event: dict) -> dict:
"""
签名字幕搜索流事件中的下载链接
"""
signed_event = dict(event)
if "items" in signed_event:
signed_event["items"] = _serialize_signed_subtitle_results(
signed_event.get("items") or []
)
return signed_event
async def _iter_signed_subtitle_search_events(
event_source: AsyncIterator[dict],
) -> AsyncIterator[dict]:
"""
输出仅包含签名字幕下载链接的搜索流事件
"""
async for event in event_source:
yield _sign_subtitle_search_event(event)
def _merge_append_event(pending_event: Optional[dict], event: dict) -> dict:
@@ -123,6 +189,7 @@ async def _stream_search_events(request: Request, event_source: AsyncIterator[di
"""
输出搜索SSE事件
"""
locale = LocaleHelper.get_locale_from_request(request)
try:
has_sent_final_replace = False
async for event in _iter_batched_search_events(event_source):
@@ -138,10 +205,13 @@ async def _stream_search_events(request: Request, event_source: AsyncIterator[di
and event.get("items")
):
event = {key: value for key, value in event.items() if key != "items"}
yield _sse_event(event)
yield _sse_event(event, locale=locale)
except Exception as err:
logger.error(f"渐进式搜索出错:{err}", exc_info=True)
yield _sse_event({"type": "error", "success": False, "message": str(err)})
yield _sse_event(
{"type": "error", "success": False, "message": str(err)},
locale=locale,
)
@router.get("/last", summary="查询搜索结果", response_model=List[schemas.Context])
@@ -168,7 +238,9 @@ async def search_latest_context(_: schemas.TokenPayload = Depends(verify_token))
success=True,
data={
"params": params,
"results": [result.to_dict() for result in results],
"results": _serialize_signed_subtitle_results(results)
if params.get("result_type") == "subtitle"
else [result.to_dict() for result in results],
},
)
@@ -236,8 +308,10 @@ async def search_by_id_stream(
doubanid=doubanid, mtype=media_type
)
if tmdbinfo:
if tmdbinfo.get("season") and not media_season:
media_season = tmdbinfo.get("season")
media_season = _resolve_media_season(
explicit_season=media_season,
recognized_season=tmdbinfo.get("season"),
)
torrents = search_chain.async_search_by_id_stream(
tmdbid=tmdbinfo.get("id"),
mtype=media_type,
@@ -342,7 +416,7 @@ async def search_by_id_stream(
meta.year = year
if media_type:
meta.type = media_type
if media_season:
if media_season is not None:
meta.type = MediaType.TV
meta.begin_season = media_season
mediainfo = await media_chain.async_recognize_by_meta(
@@ -443,8 +517,10 @@ async def search_by_id(
doubanid=doubanid, mtype=media_type
)
if tmdbinfo:
if tmdbinfo.get("season") and not media_season:
media_season = tmdbinfo.get("season")
media_season = _resolve_media_season(
explicit_season=media_season,
recognized_season=tmdbinfo.get("season"),
)
torrents = await search_chain.async_search_by_id(
tmdbid=tmdbinfo.get("id"),
mtype=media_type,
@@ -536,7 +612,7 @@ async def search_by_id(
meta.year = year
if media_type:
meta.type = media_type
if media_season:
if media_season is not None:
meta.type = MediaType.TV
meta.begin_season = media_season
mediainfo = await media_chain.async_recognize_by_meta(
@@ -625,7 +701,11 @@ async def search_subtitle_by_title_stream(
title=keyword, page=page, sites=_parse_site_list(sites), cache_local=True
)
return StreamingResponse(
_stream_search_events(request, event_source), media_type="text/event-stream"
_stream_search_events(
request,
_iter_signed_subtitle_search_events(event_source),
),
media_type="text/event-stream",
)
@@ -645,7 +725,7 @@ async def search_subtitle_by_title(
if not subtitles:
return schemas.Response(success=False, message="未搜索到任何字幕")
return schemas.Response(
success=True, data=[subtitle.to_dict() for subtitle in subtitles]
success=True, data=_serialize_signed_subtitle_results(subtitles)
)
@@ -704,8 +784,10 @@ async def _build_subtitle_search_source(
)
if not tmdbinfo:
return None, "未识别到TMDB媒体信息"
if tmdbinfo.get("season") and not media_season:
media_season = tmdbinfo.get("season")
media_season = _resolve_media_season(
explicit_season=media_season,
recognized_season=tmdbinfo.get("season"),
)
return call_search(tmdbid=tmdbinfo.get("id")), ""
return call_search(doubanid=doubanid), ""
@@ -747,7 +829,7 @@ async def _build_subtitle_search_source(
meta.year = year
if media_type:
meta.type = media_type
if media_season:
if media_season is not None:
meta.type = MediaType.TV
meta.begin_season = media_season
mediainfo = await media_chain.async_recognize_by_meta(
@@ -798,7 +880,11 @@ async def search_subtitle_by_id_stream(
yield event
return StreamingResponse(
_stream_search_events(request, event_source()), media_type="text/event-stream"
_stream_search_events(
request,
_iter_signed_subtitle_search_events(event_source()),
),
media_type="text/event-stream",
)
@@ -832,7 +918,7 @@ async def search_subtitle_by_id(
if not subtitles:
return schemas.Response(success=False, message="未搜索到任何字幕")
return schemas.Response(
success=True, data=[subtitle.to_dict() for subtitle in subtitles]
success=True, data=_serialize_signed_subtitle_results(subtitles)
)
+14 -12
View File
@@ -22,6 +22,8 @@ from app.db.models.siteuserdata import SiteUserData
from app.db.site_oper import SiteOper
from app.db.systemconfig_oper import SystemConfigOper
from app.db.user_oper import (
get_current_active_manage_user,
get_current_active_manage_user_async,
get_current_active_superuser,
get_current_active_superuser_async,
)
@@ -37,7 +39,7 @@ router = APIRouter()
@router.get("/", summary="所有站点", response_model=List[schemas.Site])
async def read_sites(
db: AsyncSession = Depends(get_async_db),
_: User = Depends(get_current_active_superuser),
_: User = Depends(get_current_active_manage_user_async),
) -> List[dict]:
"""
获取站点列表
@@ -50,7 +52,7 @@ async def add_site(
*,
db: AsyncSession = Depends(get_async_db),
site_in: schemas.Site,
_: User = Depends(get_current_active_superuser),
_: User = Depends(get_current_active_manage_user_async),
) -> Any:
"""
新增站点
@@ -89,7 +91,7 @@ async def update_site(
*,
db: AsyncSession = Depends(get_async_db),
site_in: schemas.Site,
_: User = Depends(get_current_active_superuser),
_: User = Depends(get_current_active_manage_user_async),
) -> Any:
"""
更新站点信息
@@ -150,7 +152,7 @@ def reset(
async def update_sites_priority(
priorities: List[dict],
db: AsyncSession = Depends(get_async_db),
_: User = Depends(get_current_active_superuser_async),
_: User = Depends(get_current_active_manage_user_async),
) -> Any:
"""
批量更新站点优先级
@@ -203,7 +205,7 @@ def update_cookie_by_body(
site_id: int,
site_cookie_update: schemas.SiteCookieUpdate,
db: Session = Depends(get_db),
_: User = Depends(get_current_active_superuser),
_: User = Depends(get_current_active_manage_user),
) -> Any:
"""
使用请求体中的用户密码更新站点Cookie
@@ -226,7 +228,7 @@ def update_cookie(
password: str,
code: Optional[str] = None,
db: Session = Depends(get_db),
_: User = Depends(get_current_active_superuser),
_: User = Depends(get_current_active_manage_user),
) -> Any:
"""
使用用户密码更新站点Cookie
@@ -246,7 +248,7 @@ def update_cookie(
def refresh_userdata(
site_id: int,
db: Session = Depends(get_db),
_: User = Depends(get_current_active_superuser),
_: User = Depends(get_current_active_manage_user),
) -> Any:
"""
刷新站点用户数据
@@ -273,7 +275,7 @@ def refresh_userdata(
)
async def read_userdata_latest(
db: AsyncSession = Depends(get_async_db),
_: User = Depends(get_current_active_superuser_async),
_: User = Depends(get_current_active_manage_user_async),
) -> Any:
"""
查询所有站点最新用户数据
@@ -291,7 +293,7 @@ async def read_userdata(
site_id: int,
workdate: Optional[str] = None,
db: AsyncSession = Depends(get_async_db),
_: User = Depends(get_current_active_superuser_async),
_: User = Depends(get_current_active_manage_user_async),
) -> Any:
"""
查询站点用户数据
@@ -395,7 +397,7 @@ async def site_resource(
cat: Optional[str] = None,
page: Optional[int] = 0,
db: AsyncSession = Depends(get_async_db),
_: User = Depends(get_current_active_superuser_async),
_: User = Depends(get_current_active_manage_user_async),
) -> Any:
"""
浏览站点资源
@@ -543,7 +545,7 @@ async def support_sites(_: User = Depends(get_current_active_superuser_async)):
async def read_site(
site_id: int,
db: AsyncSession = Depends(get_async_db),
_: User = Depends(get_current_active_superuser_async),
_: User = Depends(get_current_active_manage_user_async),
) -> Any:
"""
通过ID获取站点信息
@@ -561,7 +563,7 @@ async def read_site(
async def delete_site(
site_id: int,
db: AsyncSession = Depends(get_async_db),
_: User = Depends(get_current_active_superuser_async),
_: User = Depends(get_current_active_manage_user_async),
) -> Any:
"""
删除站点
+7 -6
View File
@@ -15,6 +15,7 @@ from app.core.config import settings
from app.core.security import verify_token
from app.db.models import User
from app.db.user_oper import (
get_current_active_manage_user,
get_current_active_superuser,
get_current_active_superuser_async,
)
@@ -91,7 +92,7 @@ def list_files(
fileitem: schemas.FileItem,
sort: Optional[str] = "updated_at",
keyword: Optional[str] = None,
_: User = Depends(get_current_active_superuser),
_: User = Depends(get_current_active_manage_user),
) -> Any:
"""
查询当前目录下所有目录和文件
@@ -117,7 +118,7 @@ def list_files(
def mkdir(
fileitem: schemas.FileItem,
name: str,
_: User = Depends(get_current_active_superuser),
_: User = Depends(get_current_active_manage_user),
) -> Any:
"""
创建目录
@@ -135,7 +136,7 @@ def mkdir(
@router.post("/delete", summary="删除文件或目录", response_model=schemas.Response)
def delete(
fileitem: schemas.FileItem, _: User = Depends(get_current_active_superuser)
fileitem: schemas.FileItem, _: User = Depends(get_current_active_manage_user)
) -> Any:
"""
删除文件或目录
@@ -150,7 +151,7 @@ def delete(
@router.post("/download", summary="下载文件")
def download(
fileitem: schemas.FileItem, _: User = Depends(get_current_active_superuser)
fileitem: schemas.FileItem, _: User = Depends(get_current_active_manage_user)
) -> Any:
"""
下载文件或目录
@@ -166,7 +167,7 @@ def download(
@router.post("/image", summary="预览图片")
def image(
fileitem: schemas.FileItem, _: User = Depends(get_current_active_superuser)
fileitem: schemas.FileItem, _: User = Depends(get_current_active_manage_user)
) -> Any:
"""
下载文件或目录
@@ -185,7 +186,7 @@ def rename(
fileitem: schemas.FileItem,
new_name: str,
recursive: Optional[bool] = False,
_: User = Depends(get_current_active_superuser),
_: User = Depends(get_current_active_manage_user),
) -> Any:
"""
重命名文件或目录
+184 -86
View File
@@ -17,10 +17,11 @@ from app.db.models.subscribe import Subscribe
from app.db.models.subscribehistory import SubscribeHistory
from app.db.models.user import User
from app.db.systemconfig_oper import SystemConfigOper
from app.db.user_oper import get_current_active_user_async
from app.db.user_oper import get_current_active_user, get_current_active_user_async
from app.helper.server import MoviePilotServerHelper
from app.log import logger
from app.scheduler import Scheduler
from app.schemas.event import SubscribeModifiedEventData
from app.schemas.types import MediaType, EventType, SystemConfigKey
router = APIRouter()
@@ -50,14 +51,69 @@ def build_subscribe_event_payload(subscribe: Subscribe) -> dict:
return {column.name: values.get(column.name) for column in subscribe.__table__.columns}
def can_access_subscribe(
subscribe: Subscribe | SubscribeHistory | None, current_user: User
) -> bool:
"""
判断当前用户是否可访问订阅及其历史记录
超级用户拥有全局订阅管理能力普通用户只能访问 username 精确匹配自己的订阅
username 表示无法归属的 legacy 订阅只能由超级用户管理
"""
if not subscribe:
return False
if current_user.is_superuser:
return True
username = subscribe.username
return bool(username) and username == current_user.name
async def get_accessible_subscribe(
db: AsyncSession, subscribe_id: int, current_user: User
) -> Subscribe | None:
"""
按订阅 ID 读取当前用户可访问的订阅行
"""
subscribe = await Subscribe.async_get(db, subscribe_id)
if can_access_subscribe(subscribe, current_user):
return subscribe
return None
def get_accessible_subscribe_sync(
db: Session, subscribe_id: int, current_user: User
) -> Subscribe | None:
"""
同步读取当前用户可访问的订阅行
"""
subscribe = Subscribe.get(db, subscribe_id)
if can_access_subscribe(subscribe, current_user):
return subscribe
return None
def select_accessible_subscribe(
subscribes: List[Subscribe], current_user: User
) -> Subscribe | None:
"""
从候选订阅中选择当前用户可访问的第一条记录
"""
for subscribe in subscribes or []:
if can_access_subscribe(subscribe, current_user):
return subscribe
return None
@router.get("/", summary="查询所有订阅", response_model=List[schemas.Subscribe])
async def read_subscribes(
db: AsyncSession = Depends(get_async_db),
_: schemas.TokenPayload = Depends(verify_token),
current_user: User = Depends(get_current_active_user_async),
) -> Any:
"""
查询所有订阅
"""
if not current_user.is_superuser:
return await Subscribe.async_list_by_username(db, current_user.name)
return await Subscribe.async_list(db)
@@ -68,7 +124,7 @@ async def list_subscribes(_: Annotated[str, Depends(verify_apitoken)]) -> Any:
"""
查询所有订阅 API_TOKEN认证?token=xxx
"""
return await read_subscribes()
return await Subscribe.async_list()
@router.post("/", summary="新增订阅", response_model=schemas.Response)
@@ -96,16 +152,14 @@ async def create_subscribe(
title = subscribe_in.name
else:
title = None
# 订阅用户
subscribe_in.username = current_user.name
# 转化为字典
subscribe_dict = subscribe_in.model_dump()
if subscribe_in.id:
subscribe_dict.pop("id", None)
# completed_episode 是响应派生字段,禁止写入持久层
subscribe_dict.pop("completed_episode", None)
subscribe_dict = subscribe_in.to_public_write_payload()
subscribe_dict["username"] = current_user.name
sid, message = await SubscribeChain().async_add(
mtype=mtype, title=title, exist_ok=True, **subscribe_dict
mtype=mtype,
title=title,
exist_ok=True,
owner_scope=not current_user.is_superuser,
**subscribe_dict,
)
return schemas.Response(success=bool(sid), message=message, data={"id": sid})
@@ -115,30 +169,22 @@ async def update_subscribe(
*,
subscribe_in: schemas.Subscribe,
db: AsyncSession = Depends(get_async_db),
_: schemas.TokenPayload = Depends(verify_token),
current_user: User = Depends(get_current_active_user_async),
) -> Any:
"""
更新订阅信息
"""
subscribe = await Subscribe.async_get(db, subscribe_in.id)
subscribe = await get_accessible_subscribe(db, subscribe_in.id, current_user)
if not subscribe:
return schemas.Response(success=False, message="订阅不存在")
# 避免更新缺失集数
old_subscribe_dict = subscribe.to_dict()
subscribe_dict = subscribe_in.model_dump()
if subscribe_in.episode_priority is None:
subscribe_dict.pop("episode_priority", None)
# completed_episode 是响应派生字段,禁止写入持久层
subscribe_dict.pop("completed_episode", None)
if not subscribe_in.lack_episode:
# 没有缺失集数时,缺失集数清空,避免更新为0
subscribe_dict.pop("lack_episode")
elif subscribe_in.total_episode:
# 总集数增加时,缺失集数也要增加
if subscribe_in.total_episode > (subscribe.total_episode or 0):
subscribe_dict["lack_episode"] = subscribe.lack_episode + (
subscribe_in.total_episode - (subscribe.total_episode or 0)
)
subscribe_dict = subscribe_in.to_public_write_payload()
subscribe_dict["username"] = subscribe.username
if subscribe_in.total_episode and subscribe_in.total_episode > (subscribe.total_episode or 0):
# 扩大目标范围时,新增加的集数尚无下载事实,应同步计入缺失集数。
subscribe_dict["lack_episode"] = (subscribe.lack_episode or 0) + (
subscribe_in.total_episode - (subscribe.total_episode or 0)
)
# 是否手动修改过总集数
if subscribe_in.total_episode != subscribe.total_episode:
subscribe_dict["manual_total_episode"] = 1
@@ -149,11 +195,12 @@ async def update_subscribe(
# 发送订阅调整事件
await eventmanager.async_send_event(
EventType.SubscribeModified,
{
"subscribe_id": subscribe_in.id,
"old_subscribe_info": old_subscribe_dict,
"subscribe_info": updated_subscribe.to_dict() if updated_subscribe else {},
},
SubscribeModifiedEventData(
subscribe_id=subscribe_in.id,
old_subscribe_info=old_subscribe_dict,
subscribe_info=updated_subscribe.to_dict() if updated_subscribe else {},
scene="update",
).to_dict(),
)
return schemas.Response(success=True)
@@ -163,12 +210,12 @@ async def update_subscribe_status(
subid: int,
state: str,
db: AsyncSession = Depends(get_async_db),
_: schemas.TokenPayload = Depends(verify_token),
current_user: User = Depends(get_current_active_user_async),
) -> Any:
"""
更新订阅状态
"""
subscribe = await Subscribe.async_get(db, subid)
subscribe = await get_accessible_subscribe(db, subid, current_user)
if not subscribe:
return schemas.Response(success=False, message="订阅不存在")
valid_states = ["R", "P", "S"]
@@ -181,11 +228,12 @@ async def update_subscribe_status(
# 发送订阅调整事件
await eventmanager.async_send_event(
EventType.SubscribeModified,
{
"subscribe_id": subid,
"old_subscribe_info": old_subscribe_dict,
"subscribe_info": updated_subscribe.to_dict() if updated_subscribe else {},
},
SubscribeModifiedEventData(
subscribe_id=subid,
old_subscribe_info=old_subscribe_dict,
subscribe_info=updated_subscribe.to_dict() if updated_subscribe else {},
scene="status",
).to_dict(),
)
return schemas.Response(success=True)
@@ -196,7 +244,7 @@ async def subscribe_mediaid(
season: Optional[int] = None,
title: Optional[str] = None,
db: AsyncSession = Depends(get_async_db),
_: schemas.TokenPayload = Depends(verify_token),
current_user: User = Depends(get_current_active_user_async),
) -> Any:
"""
根据 TMDBID/豆瓣ID/BangumiId 查询订阅 tmdb:/douban:
@@ -206,23 +254,27 @@ async def subscribe_mediaid(
tmdbid = mediaid[5:]
if not tmdbid or not str(tmdbid).isdigit():
return Subscribe()
result = await Subscribe.async_exists(db, tmdbid=int(tmdbid), season=season)
subscribes = await Subscribe.async_get_by_tmdbid(db, int(tmdbid), season)
result = select_accessible_subscribe(subscribes, current_user)
elif mediaid.startswith("douban:"):
doubanid = mediaid[7:]
if not doubanid:
return Subscribe()
result = await Subscribe.async_get_by_doubanid(db, doubanid)
subscribes = await Subscribe.async_list_by_doubanid(db, doubanid)
result = select_accessible_subscribe(subscribes, current_user)
if not result and title:
title_check = True
elif mediaid.startswith("bangumi:"):
bangumiid = mediaid[8:]
if not bangumiid or not str(bangumiid).isdigit():
return Subscribe()
result = await Subscribe.async_get_by_bangumiid(db, int(bangumiid))
subscribes = await Subscribe.async_list_by_bangumiid(db, int(bangumiid))
result = select_accessible_subscribe(subscribes, current_user)
if not result and title:
title_check = True
else:
result = await Subscribe.async_get_by_mediaid(db, mediaid)
subscribes = await Subscribe.async_list_by_mediaid(db, mediaid)
result = select_accessible_subscribe(subscribes, current_user)
if not result and title:
title_check = True
# 使用名称检查订阅
@@ -230,18 +282,23 @@ async def subscribe_mediaid(
meta = MetaInfo(title)
if season is not None:
meta.begin_season = season
result = await Subscribe.async_get_by_title(
subscribes = await Subscribe.async_list_by_title(
db, title=meta.name, season=meta.begin_season
)
result = select_accessible_subscribe(subscribes, current_user)
return result if result else Subscribe()
@router.get("/refresh", summary="刷新订阅", response_model=schemas.Response)
def refresh_subscribes(_: schemas.TokenPayload = Depends(verify_token)) -> Any:
def refresh_subscribes(
current_user: User = Depends(get_current_active_user),
) -> Any:
"""
刷新所有订阅
"""
if not current_user.is_superuser:
return schemas.Response(success=False, message="订阅不存在")
Scheduler().start("subscribe_refresh")
return schemas.Response(success=True)
@@ -250,12 +307,12 @@ def refresh_subscribes(_: schemas.TokenPayload = Depends(verify_token)) -> Any:
async def reset_subscribes(
subid: int,
db: AsyncSession = Depends(get_async_db),
_: schemas.TokenPayload = Depends(verify_token),
current_user: User = Depends(get_current_active_user_async),
) -> Any:
"""
重置订阅
"""
subscribe = await Subscribe.async_get(db, subid)
subscribe = await get_accessible_subscribe(db, subid, current_user)
if subscribe:
# 在更新之前获取旧数据
old_subscribe_dict = subscribe.to_dict()
@@ -267,6 +324,8 @@ async def reset_subscribes(
"lack_episode": subscribe.total_episode,
"current_priority": None,
"episode_priority": {},
# 重置代表放弃手动总集数,后续订阅检查重新按 TMDB 集数更新。
"manual_total_episode": 0,
"state": "R",
},
)
@@ -275,39 +334,57 @@ async def reset_subscribes(
# 发送订阅调整事件
await eventmanager.async_send_event(
EventType.SubscribeModified,
{
"subscribe_id": subid,
"old_subscribe_info": old_subscribe_dict,
"subscribe_info": updated_subscribe.to_dict()
SubscribeModifiedEventData(
subscribe_id=subid,
old_subscribe_info=old_subscribe_dict,
subscribe_info=updated_subscribe.to_dict()
if updated_subscribe
else {},
},
scene="reset",
).to_dict(),
)
return schemas.Response(success=True)
return schemas.Response(success=False, message="订阅不存在")
@router.get("/check", summary="刷新订阅 TMDB 信息", response_model=schemas.Response)
def check_subscribes(_: schemas.TokenPayload = Depends(verify_token)) -> Any:
def check_subscribes(
current_user: User = Depends(get_current_active_user),
) -> Any:
"""
刷新订阅 TMDB 信息
"""
if not current_user.is_superuser:
return schemas.Response(success=False, message="订阅不存在")
Scheduler().start("subscribe_tmdb")
return schemas.Response(success=True)
@router.get("/search", summary="搜索所有订阅", response_model=schemas.Response)
async def search_subscribes(
background_tasks: BackgroundTasks, _: schemas.TokenPayload = Depends(verify_token)
background_tasks: BackgroundTasks,
db: AsyncSession = Depends(get_async_db),
current_user: User = Depends(get_current_active_user_async),
) -> Any:
"""
搜索所有订阅
"""
background_tasks.add_task(
Scheduler().start,
job_id="subscribe_search",
**{"sid": None, "state": "R", "manual": True},
)
if current_user.is_superuser:
background_tasks.add_task(
Scheduler().start,
job_id="subscribe_search",
**{"sid": None, "state": "R", "manual": True},
)
else:
subscribes = await Subscribe.async_list_by_username(
db, current_user.name, state="R"
)
for subscribe in subscribes:
background_tasks.add_task(
Scheduler().start,
job_id="subscribe_search",
**{"sid": subscribe.id, "state": None, "manual": True},
)
return schemas.Response(success=True)
@@ -317,11 +394,15 @@ async def search_subscribes(
async def search_subscribe(
subscribe_id: int,
background_tasks: BackgroundTasks,
_: schemas.TokenPayload = Depends(verify_token),
db: AsyncSession = Depends(get_async_db),
current_user: User = Depends(get_current_active_user_async),
) -> Any:
"""
根据订阅编号搜索订阅
"""
subscribe = await get_accessible_subscribe(db, subscribe_id, current_user)
if not subscribe:
return schemas.Response(success=False, message="订阅不存在")
background_tasks.add_task(
Scheduler().start,
job_id="subscribe_search",
@@ -335,7 +416,7 @@ async def delete_subscribe_by_mediaid(
mediaid: str,
season: Optional[int] = None,
db: AsyncSession = Depends(get_async_db),
_: schemas.TokenPayload = Depends(verify_token),
current_user: User = Depends(get_current_active_user_async),
) -> Any:
"""
根据TMDBID或豆瓣ID删除订阅 tmdb:/douban:
@@ -351,15 +432,17 @@ async def delete_subscribe_by_mediaid(
doubanid = mediaid[7:]
if not doubanid:
return schemas.Response(success=False)
subscribe = await Subscribe.async_get_by_doubanid(db, doubanid)
if subscribe:
delete_subscribes.append(subscribe)
subscribes = await Subscribe.async_list_by_doubanid(db, doubanid)
delete_subscribes.extend(subscribes)
else:
subscribe = await Subscribe.async_get_by_mediaid(db, mediaid)
if subscribe:
delete_subscribes.append(subscribe)
subscribes = await Subscribe.async_list_by_mediaid(db, mediaid)
delete_subscribes.extend(subscribes)
delete_events = []
for subscribe in delete_subscribes:
for subscribe in [
subscribe
for subscribe in delete_subscribes
if can_access_subscribe(subscribe, current_user)
]:
subscribe_info = build_subscribe_event_payload(subscribe)
subscribe_id = subscribe_info.get("id")
if not subscribe_id:
@@ -460,14 +543,19 @@ async def subscribe_history(
page: Optional[int] = 1,
count: Optional[int] = 30,
db: AsyncSession = Depends(get_async_db),
_: schemas.TokenPayload = Depends(verify_token),
current_user: User = Depends(get_current_active_user_async),
) -> Any:
"""
查询电影/电视剧订阅历史
"""
histories = await SubscribeHistory.async_list_by_type(
db, mtype=mtype, page=page, count=count
)
if current_user.is_superuser:
histories = await SubscribeHistory.async_list_by_type(
db, mtype=mtype, page=page, count=count
)
else:
histories = await SubscribeHistory.async_list_by_type_and_username(
db, mtype=mtype, username=current_user.name, page=page, count=count
)
result = []
for history in histories:
history_item = schemas.Subscribe.model_validate(history, from_attributes=True)
@@ -484,12 +572,14 @@ async def subscribe_history(
async def delete_subscribe_history(
history_id: int,
db: AsyncSession = Depends(get_async_db),
_: schemas.TokenPayload = Depends(verify_token),
current_user: User = Depends(get_current_active_user_async),
) -> Any:
"""
删除订阅历史
"""
await SubscribeHistory.async_delete(db, history_id)
history = await SubscribeHistory.async_get(db, history_id)
if can_access_subscribe(history, current_user):
await SubscribeHistory.async_delete(db, history_id)
return schemas.Response(success=True)
@@ -534,7 +624,7 @@ async def popular_subscribes(
# 处理标题
title = sub.get("name")
season = sub.get("season")
if season and int(season) > 1 and media.tmdb_id:
if season not in (None, "") and int(season) != 1 and media.tmdb_id:
# 小写数据转大写
season_str = cn2an.an2cn(season, "low")
title = f"{title}{season_str}"
@@ -561,11 +651,13 @@ async def popular_subscribes(
async def user_subscribes(
username: str,
db: AsyncSession = Depends(get_async_db),
_: schemas.TokenPayload = Depends(verify_token),
current_user: User = Depends(get_current_active_user_async),
) -> Any:
"""
查询用户订阅
"""
if not current_user.is_superuser and username != current_user.name:
return []
return await Subscribe.async_list_by_username(db, username)
@@ -577,12 +669,12 @@ async def user_subscribes(
def subscribe_files(
subscribe_id: int,
db: Session = Depends(get_db),
_: schemas.TokenPayload = Depends(verify_token),
current_user: User = Depends(get_current_active_user),
) -> Any:
"""
订阅相关文件信息
"""
subscribe = Subscribe.get(db, subscribe_id)
subscribe = get_accessible_subscribe_sync(db, subscribe_id, current_user)
if subscribe:
return SubscribeChain().subscribe_files_info(subscribe)
return schemas.SubscrbieInfo()
@@ -590,11 +682,16 @@ def subscribe_files(
@router.post("/share", summary="分享订阅", response_model=schemas.Response)
async def subscribe_share(
sub: schemas.SubscribeShare, _: schemas.TokenPayload = Depends(verify_token)
sub: schemas.SubscribeShare,
db: AsyncSession = Depends(get_async_db),
current_user: User = Depends(get_current_active_user_async),
) -> Any:
"""
分享订阅
"""
subscribe = await get_accessible_subscribe(db, sub.subscribe_id, current_user)
if not subscribe:
return schemas.Response(success=False, message="订阅不存在")
state, errmsg = await MoviePilotServerHelper.async_sub_share(
subscribe_id=sub.subscribe_id,
share_title=sub.share_title,
@@ -724,26 +821,27 @@ async def subscribe_share_statistics(
async def read_subscribe(
subscribe_id: int,
db: AsyncSession = Depends(get_async_db),
_: schemas.TokenPayload = Depends(verify_token),
current_user: User = Depends(get_current_active_user_async),
) -> Any:
"""
根据订阅编号查询订阅信息
"""
if not subscribe_id:
return Subscribe()
return await Subscribe.async_get(db, subscribe_id)
subscribe = await get_accessible_subscribe(db, subscribe_id, current_user)
return subscribe if subscribe else Subscribe()
@router.delete("/{subscribe_id}", summary="删除订阅", response_model=schemas.Response)
async def delete_subscribe(
subscribe_id: int,
db: AsyncSession = Depends(get_async_db),
_: schemas.TokenPayload = Depends(verify_token),
current_user: User = Depends(get_current_active_user_async),
) -> Any:
"""
删除订阅信息
"""
subscribe = await Subscribe.async_get(db, subscribe_id)
subscribe = await get_accessible_subscribe(db, subscribe_id, current_user)
if subscribe:
# 在删除之前获取订阅信息
subscribe_info = build_subscribe_event_payload(subscribe)
+54 -23
View File
@@ -35,10 +35,11 @@ from app.db.user_oper import (
get_current_active_user_async,
)
from app.helper.image import ImageHelper
from app.helper.locale import LocaleHelper
from app.helper.message import MessageHelper
from app.helper.server import MoviePilotServerHelper
from app.helper.progress import ProgressHelper
from app.helper.rule import RuleHelper
from app.helper.server import MoviePilotServerHelper
from app.helper.system import SystemHelper
from app.log import logger
from app.scheduler import Scheduler
@@ -797,13 +798,14 @@ async def get_progress(
实时获取处理进度返回格式为SSE
"""
progress = ProgressHelper(process_type)
locale = LocaleHelper.get_current_locale()
async def event_generator():
try:
while not global_vars.is_system_stopped:
if await request.is_disconnected():
break
detail = progress.get()
detail = progress.get(locale=locale)
yield f"data: {json.dumps(detail)}\n\n"
await asyncio.sleep(0.5)
except asyncio.CancelledError:
@@ -1121,33 +1123,64 @@ def ruletest(
"""
过滤规则测试规则类型 1-订阅2-洗版3-搜索
"""
metainfo = MetaInfo(title=title, subtitle=subtitle)
torrent = schemas.TorrentInfo(
title=title,
description=subtitle,
)
# 查询规则组详情
rulegroup = RuleHelper().get_rule_group(rulegroup_name)
result_data = {
"title": title,
"subtitle": subtitle,
"rulegroup_name": rulegroup_name,
"rulegroup": rulegroup.model_dump() if rulegroup else None,
"meta_info": metainfo.to_dict(),
"media_info": None,
"torrent_info": torrent.model_dump(),
"priority": None,
"matched": False,
}
if not rulegroup:
return schemas.Response(
success=False, message=f"过滤规则组 {rulegroup_name} 不存在!"
success=False,
message=f"过滤规则组 {rulegroup_name} 不存在!",
data=result_data,
)
# 根据标题查询媒体信息
media_info = MediaChain().recognize_by_meta(
MetaInfo(title=title, subtitle=subtitle),
metainfo,
obtain_images=False,
)
result_data["media_info"] = media_info.to_dict() if media_info else None
if not media_info:
return schemas.Response(success=False, message="未识别到媒体信息!")
return schemas.Response(
success=False,
message="未识别到媒体信息!",
data=result_data,
)
# 过滤
result = SearchChain().filter_torrents(
rule_groups=[rulegroup.name], torrent_list=[torrent], mediainfo=media_info
)
if not result:
return schemas.Response(success=False, message="不符合过滤规则!")
return schemas.Response(
success=False,
message="不符合过滤规则!",
data=result_data,
)
result_data.update(
{
"matched": True,
"priority": 100 - result[0].pri_order + 1,
"torrent_info": result[0].model_dump(),
}
)
return schemas.Response(
success=True, data={"priority": 100 - result[0].pri_order + 1}
success=True,
data=result_data,
)
@@ -1271,13 +1304,20 @@ def modulelist(_: schemas.TokenPayload = Depends(verify_token)):
"""
查询已加载的模块ID列表
"""
modules = [
{
"id": k,
"name": v.get_name(),
}
for k, v in ModuleManager().get_modules().items()
]
modules = []
for module_id, module in ModuleManager().get_modules().items():
name = module.get_name()
modules.append(
{
"id": module_id,
"name": name,
"name_i18n": LocaleHelper.translate(
f"system.modules.{module_id}.name",
default=name,
),
"name_key": f"system.modules.{module_id}.name",
}
)
return schemas.Response(success=True, data={"modules": modules})
@@ -1299,12 +1339,7 @@ def restart_system(_: User = Depends(get_current_active_superuser)):
"""
if not SystemHelper.can_restart():
return schemas.Response(success=False, message="当前运行环境不支持重启操作!")
# 标识停止事件
global_vars.stop_system()
# 执行重启
ret, msg = SystemHelper.restart()
if not ret:
global_vars.resume_system()
return schemas.Response(success=ret, message=msg)
@@ -1322,11 +1357,7 @@ def upgrade_system(
if not SystemHelper.can_restart():
return schemas.Response(success=False, message="当前运行环境不支持升级操作!")
# 标识停止事件
global_vars.stop_system()
ret, msg = SystemHelper.upgrade(mode=mode or "release")
if not ret:
global_vars.resume_system()
return schemas.Response(success=ret, message=msg)
+50
View File
@@ -5,11 +5,61 @@ from fastapi import APIRouter, Depends
from app import schemas
from app.chain.tmdb import TmdbChain
from app.core.security import verify_token
from app.db.models.user import User
from app.db.user_oper import get_current_active_superuser_async
from app.modules.themoviedb.tmdb_cache import TmdbCache
from app.schemas.types import MediaType
router = APIRouter()
@router.get(
"/cache", summary="查询 TheMovieDb 识别缓存", response_model=schemas.Response
)
async def tmdb_recognition_cache(
_: User = Depends(get_current_active_superuser_async),
) -> schemas.Response:
"""查询可管理的 TheMovieDb 识别缓存。"""
cache_items = TmdbCache().list_items()
recognized_count = sum(1 for item in cache_items if item["tmdb_id"])
return schemas.Response(
success=True,
data={
"count": len(cache_items),
"recognized": recognized_count,
"unrecognized": len(cache_items) - recognized_count,
"data": cache_items,
},
)
@router.delete(
"/cache/{cache_key:path}",
summary="删除指定 TheMovieDb 识别缓存",
response_model=schemas.Response,
)
async def delete_tmdb_recognition_cache(
cache_key: str,
_: User = Depends(get_current_active_superuser_async),
) -> schemas.Response:
"""按缓存键删除单条 TheMovieDb 识别缓存。"""
deleted_item = TmdbCache().delete(cache_key)
if not deleted_item:
return schemas.Response(success=False, message="TheMovieDb 识别缓存不存在")
return schemas.Response(success=True, message="TheMovieDb 识别缓存删除成功")
@router.delete(
"/cache", summary="清空 TheMovieDb 识别缓存", response_model=schemas.Response
)
async def clear_tmdb_recognition_cache(
_: User = Depends(get_current_active_superuser_async),
) -> schemas.Response:
"""清空全部 TheMovieDb 识别缓存。"""
TmdbCache().clear()
return schemas.Response(success=True, message="TheMovieDb 识别缓存清理完成")
@router.get(
"/seasons/{tmdbid}", summary="TMDB所有季", response_model=List[schemas.TmdbSeason]
)
+14 -15
View File
@@ -6,14 +6,16 @@ from sqlalchemy.orm import Session
from app import schemas
from app.chain.media import MediaChain
from app.chain.storage import StorageChain
from app.chain.transfer import TransferChain
from app.core.config import settings, global_vars
from app.core.security import verify_token, verify_apitoken
from app.db import get_db
from app.db.models import User
from app.db.models.transferhistory import TransferHistory
from app.db.user_oper import get_current_active_superuser
from app.db.user_oper import (
get_current_active_manage_user,
get_current_active_superuser,
)
from app.helper.directory import DirectoryHelper
from app.log import logger
from app.schemas import (
@@ -183,7 +185,7 @@ def _get_manual_transfer_target_key(
def match_manual_transfer_target_path(
transer_item: ManualTransferItem,
db: Session = Depends(get_db),
_: User = Depends(get_current_active_superuser),
_: User = Depends(get_current_active_manage_user),
) -> Any:
"""
根据源文件匹配手动整理目的路径
@@ -243,7 +245,7 @@ def manual_transfer(
transer_item: ManualTransferItem,
background: Optional[bool] = False,
db: Session = Depends(get_db),
_: User = Depends(get_current_active_superuser),
_: User = Depends(get_current_active_manage_user),
) -> Any:
"""
手动转移文件或历史记录支持自定义剧集识别格式
@@ -256,6 +258,7 @@ def manual_transfer(
downloader = None
download_hash = None
src_fileitems: List[FileItem] = []
cleanup_dest_fileitem: Optional[FileItem] = None
target_path = Path(transer_item.target_path) if transer_item.target_path else None
if transer_item.logid:
# 查询历史记录
@@ -266,23 +269,17 @@ def manual_transfer(
)
# 强制转移
force = True
downloader = history.downloader
download_hash = history.download_hash
# 下载器与 Hash 是同一组下载上下文,重新识别时由当前文件路径重新匹配。
downloader = history.downloader if transer_item.from_history else None
download_hash = history.download_hash if transer_item.from_history else None
if history.status and ("move" in history.mode):
# 重新整理成功的转移,则使用成功的 dest 做 in_path
src_fileitems = [FileItem(**history.dest_fileitem)]
else:
# 源路径
src_fileitems = [FileItem(**history.src_fileitem)]
# 目的路径
if history.dest_fileitem and not transer_item.preview:
# 删除旧的已整理文件
dest_fileitem = FileItem(**history.dest_fileitem)
state = StorageChain().delete_media_file(dest_fileitem)
if not state:
return schemas.Response(
success=False, message=f"{dest_fileitem.path} 删除失败"
)
cleanup_dest_fileitem = FileItem(**history.dest_fileitem)
# 从历史数据获取信息
if transer_item.from_history:
@@ -427,6 +424,7 @@ def manual_transfer(
download_hash=download_hash,
preview=transer_item.preview,
sync_extra_files=False,
cleanup_dest_fileitem=cleanup_dest_fileitem,
)
if transer_item.preview:
if isinstance(errormsg, dict):
@@ -508,6 +506,7 @@ def manual_transfer(
download_hash=download_hash,
preview=transer_item.preview,
sync_extra_files=True,
cleanup_dest_fileitem=cleanup_dest_fileitem,
)
# 失败
if not state:
@@ -533,7 +532,7 @@ def manual_transfer(
)
def recommend_episode_format(
recommend_item: EpisodeFormatRecommendItem,
_: User = Depends(get_current_active_superuser),
_: User = Depends(get_current_active_manage_user),
) -> Any:
"""
根据目录样本推荐集数定位模板
+21 -18
View File
@@ -10,11 +10,14 @@ from app import schemas
from app.chain.workflow import WorkflowChain
from app.core.config import global_vars
from app.core.plugin import PluginManager
from app.core.security import verify_token
from app.workflow import WorkFlowManager
from app.db import get_async_db, get_db
from app.db.models import Workflow
from app.db.models import Workflow, User
from app.db.systemconfig_oper import SystemConfigOper
from app.db.user_oper import (
get_current_active_manage_user,
get_current_active_manage_user_async,
)
from app.db.workflow_oper import WorkflowOper
from app.helper.server import MoviePilotServerHelper
from app.scheduler import Scheduler
@@ -30,7 +33,7 @@ WORKFLOW_TRIGGER_MANUAL = "manual"
@router.get("/", summary="所有工作流", response_model=List[schemas.Workflow])
async def list_workflows(
db: AsyncSession = Depends(get_async_db),
_: schemas.TokenPayload = Depends(verify_token),
_: User = Depends(get_current_active_manage_user_async),
) -> Any:
"""
获取工作流列表
@@ -42,7 +45,7 @@ async def list_workflows(
async def create_workflow(
workflow: schemas.Workflow,
db: AsyncSession = Depends(get_async_db),
_: schemas.TokenPayload = Depends(verify_token),
_: User = Depends(get_current_active_manage_user_async),
) -> Any:
"""
创建工作流
@@ -62,7 +65,7 @@ async def create_workflow(
@router.get("/plugin/actions", summary="查询插件动作", response_model=List[dict])
def list_plugin_actions(
plugin_id: str = None, _: schemas.TokenPayload = Depends(verify_token)
plugin_id: str = None, _: User = Depends(get_current_active_manage_user)
) -> Any:
"""
获取所有动作
@@ -71,7 +74,7 @@ def list_plugin_actions(
@router.get("/actions", summary="所有动作", response_model=List[dict])
async def list_actions(_: schemas.TokenPayload = Depends(verify_token)) -> Any:
async def list_actions(_: User = Depends(get_current_active_manage_user_async)) -> Any:
"""
获取所有动作
"""
@@ -79,7 +82,7 @@ async def list_actions(_: schemas.TokenPayload = Depends(verify_token)) -> Any:
@router.get("/event_types", summary="获取所有事件类型", response_model=List[dict])
async def get_event_types(_: schemas.TokenPayload = Depends(verify_token)) -> Any:
async def get_event_types(_: User = Depends(get_current_active_manage_user_async)) -> Any:
"""
获取所有事件类型
"""
@@ -94,7 +97,7 @@ async def get_event_types(_: schemas.TokenPayload = Depends(verify_token)) -> An
@router.post("/share", summary="分享工作流", response_model=schemas.Response)
async def workflow_share(
workflow: schemas.WorkflowShare, _: schemas.TokenPayload = Depends(verify_token)
workflow: schemas.WorkflowShare, _: User = Depends(get_current_active_manage_user_async)
) -> Any:
"""
分享工作流
@@ -115,7 +118,7 @@ async def workflow_share(
@router.delete("/share/{share_id}", summary="删除分享", response_model=schemas.Response)
async def workflow_share_delete(
share_id: int, _: schemas.TokenPayload = Depends(verify_token)
share_id: int, _: User = Depends(get_current_active_manage_user_async)
) -> Any:
"""
删除分享
@@ -128,7 +131,7 @@ async def workflow_share_delete(
async def workflow_fork(
workflow: schemas.WorkflowShare,
db: AsyncSession = Depends(get_async_db),
_: schemas.User = Depends(verify_token),
_: User = Depends(get_current_active_manage_user_async),
) -> Any:
"""
复用工作流
@@ -194,7 +197,7 @@ async def workflow_shares(
name: Optional[str] = None,
page: Optional[int] = 1,
count: Optional[int] = 30,
_: schemas.TokenPayload = Depends(verify_token),
_: User = Depends(get_current_active_manage_user_async),
) -> Any:
"""
查询分享的工作流
@@ -208,7 +211,7 @@ async def workflow_shares(
def run_workflow(
workflow_id: int,
from_begin: Optional[bool] = True,
_: schemas.TokenPayload = Depends(verify_token),
_: User = Depends(get_current_active_manage_user),
) -> Any:
"""
执行工作流
@@ -225,7 +228,7 @@ def run_workflow(
def start_workflow(
workflow_id: int,
db: Session = Depends(get_db),
_: schemas.TokenPayload = Depends(verify_token),
_: User = Depends(get_current_active_manage_user),
) -> Any:
"""
启用工作流
@@ -259,7 +262,7 @@ def start_workflow(
def pause_workflow(
workflow_id: int,
db: Session = Depends(get_db),
_: schemas.TokenPayload = Depends(verify_token),
_: User = Depends(get_current_active_manage_user),
) -> Any:
"""
停用工作流
@@ -287,7 +290,7 @@ def pause_workflow(
async def reset_workflow(
workflow_id: int,
db: AsyncSession = Depends(get_async_db),
_: schemas.TokenPayload = Depends(verify_token),
_: User = Depends(get_current_active_manage_user_async),
) -> Any:
"""
重置工作流
@@ -308,7 +311,7 @@ async def reset_workflow(
async def get_workflow(
workflow_id: int,
db: AsyncSession = Depends(get_async_db),
_: schemas.TokenPayload = Depends(verify_token),
_: User = Depends(get_current_active_manage_user_async),
) -> Any:
"""
获取工作流详情
@@ -320,7 +323,7 @@ async def get_workflow(
def update_workflow(
workflow: schemas.Workflow,
db: Session = Depends(get_db),
_: schemas.TokenPayload = Depends(verify_token),
_: User = Depends(get_current_active_manage_user),
) -> Any:
"""
更新工作流
@@ -350,7 +353,7 @@ def update_workflow(
def delete_workflow(
workflow_id: int,
db: Session = Depends(get_db),
_: schemas.TokenPayload = Depends(verify_token),
_: User = Depends(get_current_active_manage_user),
) -> Any:
"""
删除工作流
+21 -2
View File
@@ -1,10 +1,11 @@
import gzip
import hmac
import json
from typing import Annotated, Callable, Any, Dict, Optional
import aiofiles
from anyio import Path as AsyncPath
from fastapi import APIRouter, Body, Depends, HTTPException, Path, Request, Response
from fastapi import APIRouter, Body, Depends, Header, HTTPException, Path, Request, Response
from fastapi.responses import PlainTextResponse
from fastapi.routing import APIRoute
@@ -44,6 +45,24 @@ async def verify_server_enabled():
return True
async def verify_update_auth(
x_cookiecloud_auth: Annotated[
Optional[str], Header(alias="X-CookieCloud-Auth")
] = None,
):
"""
校验CookieCloud上传接口的可选共享认证头
"""
expected_header = (settings.COOKIECLOUD_AUTH_HEADER or "").strip()
if not expected_header:
return True
provided_header = (x_cookiecloud_auth or "").strip()
if not hmac.compare_digest(provided_header, expected_header):
raise HTTPException(status_code=403, detail="CookieCloud认证失败")
return True
cookie_router = APIRouter(
route_class=GzipRoute,
tags=["servcookie"],
@@ -61,7 +80,7 @@ async def post_root():
return "Hello MoviePilot! COOKIECLOUD API ROOT = /cookiecloud"
@cookie_router.post("/update")
@cookie_router.post("/update", dependencies=[Depends(verify_update_auth)])
async def update_cookie(req: schemas.CookieData):
"""
上传Cookie数据
+363 -52
View File
@@ -1,11 +1,13 @@
import base64
import copy
import hashlib
import json
import re
import shutil
import time
from pathlib import Path
from typing import List, Optional, Tuple, Set, Dict, Union
from urllib.parse import parse_qs, urlparse
from app import schemas
from app.chain import ChainBase
@@ -16,9 +18,10 @@ from app.core.context import MediaInfo, SubtitleInfo, TorrentInfo, Context
from app.core.event import eventmanager, Event
from app.core.meta import MetaBase
from app.core.metainfo import MetaInfo
from app.db.downloadfailure_oper import DownloadFailureOper
from app.db.downloadhistory_oper import DownloadHistoryOper
from app.db.mediaserver_oper import MediaServerOper
from app.helper.directory import DirectoryHelper
from app.helper.directory import DirectoryHelper, validate_download_save_path
from app.helper.thread import ThreadHelper
from app.helper.torrent import TorrentHelper
from app.log import logger
@@ -31,6 +34,21 @@ from app.utils.string import StringUtils
from app.utils.system import SystemUtils
DOWNLOAD_FAILURE_RESOURCE_TTL_SECONDS = 24 * 60 * 60
DOWNLOAD_FAILURE_TRANSIENT_TTL_SECONDS = 60 * 60
DOWNLOAD_FAILURE_RESOURCE_ERROR_KEYWORDS = (
"无法读取种子文件",
"下载种子内容为空",
"无法获取下载地址",
"种子下载失败",
"torrent not found",
"not found",
"404",
"deleted",
"invalid torrent",
)
class DownloadChain(ChainBase):
"""
下载处理链
@@ -107,29 +125,69 @@ class DownloadChain(ChainBase):
def _resolve_media_download_dir(
media_info: MediaInfo,
save_path: Optional[str] = None,
) -> Union[str, Path]:
) -> Tuple[Optional[str], Optional[Path], str]:
"""
根据媒体信息解析下载目录
"""
storage = 'local'
if save_path:
return storage, Path(save_path)
if save_path is not None:
try:
validated_save_path = validate_download_save_path(save_path)
except ValueError as err:
logger.warn(str(err))
return None, None, str(err)
if re.match(r"^[A-Za-z]:/", validated_save_path):
target_dir = Path(validated_save_path)
else:
file_uri = FileURI.from_uri(validated_save_path)
storage = file_uri.storage or storage
target_dir = Path(file_uri.path)
dir_info = DirectoryHelper().get_download_dir_by_save_path(
media=media_info,
save_path=validated_save_path,
)
if dir_info:
target_dir = DownloadChain._append_download_classification(
root_path=target_dir,
dir_info=dir_info,
media_info=media_info,
)
return storage, target_dir, ""
dir_info = DirectoryHelper().get_dir(media_info, include_unsorted=True)
storage = dir_info.storage if dir_info else storage
if not dir_info:
logger.error(f"未找到下载目录:{media_info.type.value} {media_info.title_year}")
return None
return None, None, "未找到下载目录"
download_dir = DownloadChain._append_download_classification(
root_path=Path(dir_info.download_path),
dir_info=dir_info,
media_info=media_info,
)
return storage, download_dir, ""
@staticmethod
def _append_download_classification(
root_path: Path,
dir_info: schemas.TransferDirectoryConf,
media_info: MediaInfo,
) -> Path:
"""
按下载目录配置拼装媒体类型和类别子目录
:param root_path: 下载根目录
:param dir_info: 下载目录配置
:param media_info: 媒体信息
:return: 应传给存储或下载器的媒体下载目录
"""
download_dir = root_path
if not dir_info.media_type and dir_info.download_type_folder:
download_dir = Path(dir_info.download_path) / media_info.type.value
else:
download_dir = Path(dir_info.download_path)
download_dir = download_dir / media_info.type.value
if not dir_info.media_category and dir_info.download_category_folder and media_info.category:
download_dir = download_dir / media_info.category
return storage, download_dir
return download_dir
@staticmethod
def _upload_subtitle_file(
@@ -293,12 +351,12 @@ class DownloadChain(ChainBase):
if not mediainfo:
return False, "无法识别媒体信息", []
storage, target_dir = self._resolve_media_download_dir(
storage, target_dir, error_msg = self._resolve_media_download_dir(
media_info=mediainfo,
save_path=save_path,
)
if not target_dir:
return False, "未找到下载目录", []
return False, error_msg or "未找到下载目录", []
request = RequestUtils(
cookies=subtitle.site_cookie,
@@ -357,6 +415,186 @@ class DownloadChain(ChainBase):
except Exception as err:
logger.error(f"提交下载成功后处理后台任务失败:{str(err)}")
@staticmethod
def _is_subscribe_source(source: Optional[str]) -> bool:
"""
判断下载来源是否为订阅任务
"""
return bool(source and str(source).startswith("Subscribe|"))
@staticmethod
def _format_failure_episodes(meta: Optional[MetaBase]) -> Optional[str]:
"""
从识别元数据中格式化用于失败记录的集数
"""
if not meta:
return None
if getattr(meta, "episode", None):
return meta.episode
episode_list = getattr(meta, "episode_list", None)
if episode_list:
return StringUtils.format_ep(list(episode_list))
return None
@staticmethod
def _torrent_resource_key(torrent: Optional[TorrentInfo]) -> str:
"""
生成不保存敏感下载链接的种子资源键
"""
if not torrent:
return ""
for attr_name in ("torrent_id", "info_hash"):
value = getattr(torrent, attr_name, None)
if value:
return str(value)
for attr_name in ("page_url", "enclosure"):
url = getattr(torrent, attr_name, None)
if not url:
continue
match = re.search(r"\[(.*?)](.*)", str(url))
if match:
url = match.group(2)
parsed = urlparse(str(url))
params = parse_qs(parsed.query)
for param_name in ("id", "torrentid", "torrent_id", "tid", "hash"):
values = params.get(param_name)
if values:
return f"{parsed.netloc}:{param_name}={values[0]}"
if parsed.netloc and parsed.path:
return f"{parsed.netloc}{parsed.path}"
title = getattr(torrent, "title", "") or ""
size = getattr(torrent, "size", "") or ""
return f"title={title}|size={size}"
@classmethod
def _build_download_failure_fingerprint(cls, context: Context) -> Optional[str]:
"""
根据媒体和种子资源信息生成失败冷却指纹
"""
media = getattr(context, "media_info", None)
torrent = getattr(context, "torrent_info", None)
if not media or not torrent:
return None
media_type = getattr(getattr(media, "type", None), "value", getattr(media, "type", None))
media_key = (
getattr(media, "tmdb_id", None)
or getattr(media, "douban_id", None)
or getattr(media, "imdb_id", None)
or getattr(media, "tvdb_id", None)
or f"{getattr(media, 'title', '')}:{getattr(media, 'year', '')}"
)
meta = getattr(context, "meta_info", None)
site = getattr(torrent, "site", None) or getattr(torrent, "site_name", None)
meta_season = getattr(meta, "season", None)
media_season = getattr(media, "season", None)
season = meta_season if meta_season is not None else media_season
payload = {
"media_type": str(media_type or ""),
"media_key": str(media_key or ""),
"season": str(season) if season is not None else "",
"episodes": cls._format_failure_episodes(meta) or "",
"site": str(site or ""),
"resource": cls._torrent_resource_key(torrent),
}
if not payload["media_type"] or not payload["media_key"] or not payload["resource"]:
return None
raw_text = json.dumps(payload, ensure_ascii=False, sort_keys=True)
return hashlib.sha256(raw_text.encode("utf-8")).hexdigest()
@staticmethod
def _download_failure_ttl(error_msg: Optional[str]) -> int:
"""
按失败原因确定资源冷却时间
"""
error_text = str(error_msg or "").lower()
if any(keyword in error_text for keyword in DOWNLOAD_FAILURE_RESOURCE_ERROR_KEYWORDS):
return DOWNLOAD_FAILURE_RESOURCE_TTL_SECONDS
return DOWNLOAD_FAILURE_TRANSIENT_TTL_SECONDS
def _record_download_failure(
self,
context: Context,
error_msg: Optional[str],
downloader: Optional[str] = None,
source: Optional[str] = None,
episodes: Optional[Set[int]] = None,
) -> Optional[str]:
"""
记录资源级下载失败并返回本次失败指纹
"""
fingerprint = self._build_download_failure_fingerprint(context)
if not fingerprint:
return None
now_timestamp = time.time()
now_time = time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(now_timestamp))
next_retry_at = time.strftime(
"%Y-%m-%d %H:%M:%S",
time.localtime(now_timestamp + self._download_failure_ttl(error_msg)),
)
media = context.media_info
meta = context.meta_info
torrent = context.torrent_info
site = getattr(torrent, "site", None)
try:
DownloadFailureOper().record_failure(
fingerprint=fingerprint,
now_time=now_time,
next_retry_at=next_retry_at,
type=getattr(getattr(media, "type", None), "value", getattr(media, "type", None)),
title=getattr(media, "title", None),
year=getattr(media, "year", None),
tmdbid=getattr(media, "tmdb_id", None),
doubanid=getattr(media, "douban_id", None),
seasons=getattr(meta, "season", None),
episodes=StringUtils.format_ep(list(episodes)) if episodes else self._format_failure_episodes(meta),
site=site if isinstance(site, int) else None,
site_name=getattr(torrent, "site_name", None),
torrent_id=self._torrent_resource_key(torrent),
torrent_name=getattr(torrent, "title", None),
torrent_size=getattr(torrent, "size", None),
downloader=downloader,
source=str(source)[:1000] if source else None,
error_message=str(error_msg or "")[:1000],
)
except Exception as err:
logger.error(f"记录下载失败冷却失败:{str(err)}")
return fingerprint
def _active_download_failure_fingerprints(
self,
contexts: List[Context],
source: Optional[str],
) -> Set[str]:
"""
查询当前订阅候选中仍处于冷却期的失败指纹
"""
if not self._is_subscribe_source(source):
return set()
fingerprints = [
fingerprint
for fingerprint in [
self._build_download_failure_fingerprint(context)
for context in contexts or []
]
if fingerprint
]
if not fingerprints:
return set()
now_time = time.strftime("%Y-%m-%d %H:%M:%S", time.localtime())
try:
return set(
DownloadFailureOper()
.get_active_by_fingerprints(fingerprints=fingerprints, now_time=now_time)
.keys()
)
except Exception as err:
logger.error(f"查询下载失败冷却失败:{str(err)}")
return set()
def download_torrent(self, torrent: TorrentInfo,
channel: MessageChannel = None,
source: Optional[str] = None,
@@ -478,7 +716,8 @@ class DownloadChain(ChainBase):
userid: Union[str, int] = None,
username: Optional[str] = None,
label: Optional[str] = None,
return_detail: bool = False) -> Union[Optional[str], Tuple[Optional[str], Optional[str]]]:
return_detail: bool = False,
custom_words: Optional[str] = None) -> Union[Optional[str], Tuple[Optional[str], Optional[str]]]:
"""
下载及发送通知
:param context: 资源上下文
@@ -493,6 +732,7 @@ class DownloadChain(ChainBase):
:param username: 调用下载的用户名/插件名
:param label: 自定义标签
:param return_detail: 是否返回详细结果False 时返回下载任务 hash NoneTrue 时返回 (hash, error_msg)
:param custom_words: 下载来源如订阅的完整自定义识别词文本随下载记录存档供整理时原样复现识别
:return: return_detail=False 时返回下载任务 hash Nonereturn_detail=True 时返回 (hash, error_msg)
"""
_torrent = context.torrent_info
@@ -525,9 +765,16 @@ class DownloadChain(ChainBase):
f"Reason: {event_data.reason}")
return (None, "下载被事件取消") if return_detail else None
# 如果事件修改了下载路径,使用新路径
if event_data.options and event_data.options.get("save_path"):
if event_data.options and "save_path" in event_data.options:
save_path = event_data.options.get("save_path")
if save_path is not None:
try:
save_path = validate_download_save_path(save_path)
except ValueError as err:
logger.warn(str(err))
return (None, str(err)) if return_detail else None
# 补充完整的media数据
if not _media.genre_ids:
new_media = self.recognize_media(mtype=_media.type, tmdbid=_media.tmdb_id,
@@ -561,41 +808,29 @@ class DownloadChain(ChainBase):
torrent_content = cache_backend.get(torrent_file.as_posix(), region="torrents")
if not torrent_content:
self._record_download_failure(
context=context,
error_msg="下载种子内容为空",
downloader=downloader or _site_downloader,
source=source,
episodes=episodes,
)
return (None, "下载种子内容为空") if return_detail else None
# 获取种子文件的文件夹名和文件清单
_folder_name, _file_list = TorrentHelper().get_fileinfo_from_torrent_content(torrent_content)
storage = 'local'
# 下载目录
if save_path:
download_dir = Path(save_path)
else:
# 根据媒体信息查询下载目录配置
dir_info = DirectoryHelper().get_dir(_media, include_unsorted=True)
storage = dir_info.storage if dir_info else storage
# 拼装子目录
if dir_info:
# 一级目录
if not dir_info.media_type and dir_info.download_type_folder:
# 一级自动分类
download_dir = Path(dir_info.download_path) / _media.type.value
else:
# 一级不分类
download_dir = Path(dir_info.download_path)
# 二级目录
if not dir_info.media_category and dir_info.download_category_folder and _media and _media.category:
# 二级自动分类
download_dir = download_dir / _media.category
else:
# 未找到下载目录,且没有自定义下载目录
logger.error(f"未找到下载目录:{_media.type.value} {_media.title_year}")
storage, download_dir, error_msg = self._resolve_media_download_dir(
media_info=_media,
save_path=save_path,
)
if not download_dir:
if error_msg == "未找到下载目录":
self.messagehelper.put(f"{_media.type.value} {_media.title_year} 未找到下载目录!",
title="下载失败", role="system")
return (None, "未找到下载目录") if return_detail else None
fileURI = FileURI(storage=storage, path=download_dir.as_posix())
download_dir = Path(fileURI.uri)
return (None, error_msg or "未找到下载目录") if return_detail else None
file_uri = FileURI(storage=storage, path=download_dir.as_posix())
download_dir = Path(file_uri.uri)
# 添加下载
result: Optional[tuple] = self.download(content=torrent_content,
@@ -649,7 +884,8 @@ class DownloadChain(ChainBase):
date=time.strftime("%Y-%m-%d %H:%M:%S", time.localtime()),
media_category=_media.category,
episode_group=_media.episode_group,
note={"source": source}
note={"source": source},
custom_words=custom_words
)
# 登记下载文件
@@ -714,6 +950,13 @@ class DownloadChain(ChainBase):
# 下载失败
logger.error(f"{_media.title_year} 添加下载任务失败:"
f"{_torrent.title} - {_torrent.enclosure}{error_msg}")
self._record_download_failure(
context=context,
error_msg=error_msg,
downloader=_downloader or downloader or _site_downloader,
source=source,
episodes=episodes,
)
# 只发送给对应渠道和用户
self.post_message(Notification(
channel=channel,
@@ -738,7 +981,8 @@ class DownloadChain(ChainBase):
source: Optional[str] = None,
userid: Optional[str] = None,
username: Optional[str] = None,
downloader: Optional[str] = None
downloader: Optional[str] = None,
custom_words: Optional[str] = None
) -> Tuple[List[Context], Dict[Union[int, str], Dict[int, NotExistMediaInfo]]]:
"""
根据缺失数据自动种子列表中组合择优下载
@@ -750,6 +994,7 @@ class DownloadChain(ChainBase):
:param userid: 用户ID
:param username: 调用下载的用户名/插件名
:param downloader: 下载器
:param custom_words: 下载来源如订阅的完整自定义识别词文本随下载记录存档供整理时原样复现识别
:return: 已经下载的资源列表剩余未下载到的剧集 no_exists[tmdb_id/douban_id] = {season: NotExistMediaInfo}
"""
# 已下载的项目
@@ -877,6 +1122,28 @@ class DownloadChain(ChainBase):
# 仅排序,不提前按媒体控重;下载失败时需要继续尝试同组后续候选。
contexts = TorrentHelper().sort_torrents(contexts)
active_failure_fingerprints = self._active_download_failure_fingerprints(
contexts=contexts,
source=source,
)
def __is_context_in_failure_cooldown(_context: Context) -> bool:
"""
判断候选资源是否仍处于失败冷却期
"""
fingerprint = self._build_download_failure_fingerprint(_context)
if fingerprint and fingerprint in active_failure_fingerprints:
logger.info(f"{_context.torrent_info.title} 近期添加下载失败,暂时跳过该资源")
return True
return False
def __remember_context_failure(_context: Context) -> None:
"""
将本轮失败候选加入内存冷却集合避免同一批次重复尝试
"""
fingerprint = self._build_download_failure_fingerprint(_context)
if fingerprint:
active_failure_fingerprints.add(fingerprint)
# 如果是电影,直接下载
downloaded_movies = set()
@@ -884,17 +1151,21 @@ class DownloadChain(ChainBase):
if global_vars.is_system_stopped:
break
if context.media_info.type == MediaType.MOVIE:
if __is_context_in_failure_cooldown(context):
continue
movie_key = __get_movie_download_key(context)
if movie_key in downloaded_movies:
continue
logger.info(f"开始下载电影 {context.torrent_info.title} ...")
if self.download_single(context, save_path=save_path, channel=channel,
source=source, userid=userid, username=username,
downloader=downloader):
downloader=downloader, custom_words=custom_words):
# 下载成功
logger.info(f"{context.torrent_info.title} 添加下载成功")
downloaded_list.append(context)
downloaded_movies.add(movie_key)
else:
__remember_context_failure(context)
# 电视剧整季匹配
if no_exists:
@@ -909,7 +1180,7 @@ class DownloadChain(ChainBase):
if not tv.episodes:
if not need_seasons.get(need_mid):
need_seasons[need_mid] = []
need_seasons[need_mid].append(tv.season or 1)
need_seasons[need_mid].append(tv.season if tv.season is not None else 1)
logger.info(f"缺失整季:{need_seasons}")
# 查找整季包含的种子,只处理整季没集的种子或者是集数超过季的种子
for need_mid, need_season in need_seasons.items():
@@ -939,14 +1210,24 @@ class DownloadChain(ChainBase):
# 不重复添加
if context in downloaded_list:
continue
if __is_context_in_failure_cooldown(context):
continue
# 种子季是需要季或者子集
if set(torrent_season).issubset(set(need_season)):
complete_coverage_matched = False
if len(torrent_season) == 1:
# 只有一季的可能是命名错误,需要打开种子鉴别,只有实际集数大于等于总集数才下载
logger.info(f"开始下载种子 {torrent.title} ...")
content, _, torrent_files = self.download_torrent(torrent)
if not content:
logger.warn(f"{torrent.title} 种子下载失败!")
self._record_download_failure(
context=context,
error_msg="下载种子内容为空",
downloader=downloader,
source=source,
)
__remember_context_failure(context)
continue
if isinstance(content, str):
logger.warn(f"{meta.org_string} 下载地址是磁力链,无法确定种子文件集数")
@@ -965,7 +1246,13 @@ class DownloadChain(ChainBase):
required_episodes = __get_required_episodes(need_mid, torrent_season[0]) \
if __requires_complete_coverage(need_tv_info) else set()
need_total = __get_season_episodes(need_mid, torrent_season[0])
if required_episodes and not required_episodes.issubset(torrent_episodes_set):
complete_coverage_matched = bool(required_episodes) \
and required_episodes.issubset(torrent_episodes_set)
if complete_coverage_matched:
logger.info(
f"{meta.org_string} 解析文件集数已完整覆盖目标范围:"
f"{StringUtils.format_ep(sorted(required_episodes))}")
if required_episodes and not complete_coverage_matched:
missing_episodes = sorted(required_episodes.difference(torrent_episodes_set))
logger.info(
f"{meta.org_string} 解析文件集数未覆盖目标范围,"
@@ -986,7 +1273,8 @@ class DownloadChain(ChainBase):
source=source,
userid=userid,
username=username,
downloader=downloader
downloader=downloader,
custom_words=custom_words
)
else:
# 下载
@@ -994,10 +1282,13 @@ class DownloadChain(ChainBase):
download_id = self.download_single(context, save_path=save_path,
channel=channel, source=source,
userid=userid, username=username,
downloader=downloader)
downloader=downloader,
custom_words=custom_words)
if download_id:
# 下载成功
if complete_coverage_matched:
context.confirmed_full_coverage = True
logger.info(f"{torrent.title} 添加下载成功")
downloaded_list.append(context)
# 更新仍需季集
@@ -1008,6 +1299,8 @@ class DownloadChain(ChainBase):
if not need_season:
# 全部下载完成
break
else:
__remember_context_failure(context)
# 电视剧季内的集匹配
if no_exists:
logger.info(f"开始电视剧完整集匹配:{no_exists}")
@@ -1048,6 +1341,8 @@ class DownloadChain(ChainBase):
# 不重复添加
if context in downloaded_list:
continue
if __is_context_in_failure_cooldown(context):
continue
# 种子季
torrent_season = meta.season_list
# 只处理单季含集的种子
@@ -1076,9 +1371,12 @@ class DownloadChain(ChainBase):
download_id = self.download_single(context, save_path=save_path,
channel=channel, source=source,
userid=userid, username=username,
downloader=downloader)
downloader=downloader,
custom_words=custom_words)
if download_id:
# 下载成功
if __requires_complete_coverage(tv):
context.confirmed_full_coverage = True
logger.info(f"{meta.title} 添加下载成功")
downloaded_list.append(context)
# 更新仍需集数
@@ -1087,6 +1385,8 @@ class DownloadChain(ChainBase):
_sea=need_season,
_current=torrent_episodes)
logger.info(f"{need_season} 剩余需要集:{need_episodes}")
else:
__remember_context_failure(context)
# 仍然缺失的剧集,从整季中选择需要的集数文件下载,仅支持QB和TR
if no_exists:
@@ -1129,6 +1429,8 @@ class DownloadChain(ChainBase):
# 不重复添加
if context in downloaded_list:
continue
if __is_context_in_failure_cooldown(context):
continue
# 没有需要集后退出
if not need_episodes:
break
@@ -1147,6 +1449,13 @@ class DownloadChain(ChainBase):
content, _, torrent_files = self.download_torrent(torrent)
if not content:
logger.info(f"{torrent.title} 种子下载失败!")
self._record_download_failure(
context=context,
error_msg="下载种子内容为空",
downloader=downloader,
source=source,
)
__remember_context_failure(context)
continue
if isinstance(content, str):
logger.warn(f"{meta.org_string} 下载地址是磁力链,无法解析种子文件集数")
@@ -1171,9 +1480,11 @@ class DownloadChain(ChainBase):
source=source,
userid=userid,
username=username,
downloader=downloader
downloader=downloader,
custom_words=custom_words
)
if not download_id:
__remember_context_failure(context)
continue
# 下载成功
logger.info(f"{torrent.title} 添加下载成功")
+25 -13
View File
@@ -653,6 +653,16 @@ class MediaChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
self.obtain_images(mediainfo=mediainfo)
return mediainfo
@staticmethod
def _parse_recognize_event_number(value) -> Optional[int]:
"""
解析辅助识别返回的季集号兼容整数和数字字符串并保留数值 0
"""
if value is None:
return None
text = str(value).strip()
return int(text) if text.isdigit() else None
def recognize_help(
self,
title: str,
@@ -686,10 +696,8 @@ class MediaChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
title = str(event_data["name"]).split("/")[0].strip().replace(".", " ")
if event_data.get("year"):
year = str(event_data["year"]).split("/")[0].strip()
if event_data.get("season") and str(event_data["season"]).isdigit():
season_number = int(event_data["season"])
if event_data.get("episode") and str(event_data["episode"]).isdigit():
episode_number = int(event_data["episode"])
season_number = self._parse_recognize_event_number(event_data.get("season"))
episode_number = self._parse_recognize_event_number(event_data.get("episode"))
if not title:
return None
if title == "Unknown":
@@ -837,7 +845,7 @@ class MediaChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
tmdbinfo = self._match_tmdb_with_names(
meta_names=meta_names,
year=year,
mtype=MediaType.TV,
mtype=MediaInfo.get_bangumi_media_type(bangumiinfo),
season=meta.begin_season,
)
return tmdbinfo
@@ -877,7 +885,10 @@ class MediaChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
year = self._extract_year_from_bangumi(bangumiinfo)
# 使用名称识别豆瓣媒体信息
return self.match_doubaninfo(
name=meta.name, year=year, mtype=MediaType.TV, season=meta.begin_season
name=meta.name,
year=year,
mtype=MediaInfo.get_bangumi_media_type(bangumiinfo),
season=meta.begin_season,
)
return None
@@ -1635,10 +1646,8 @@ class MediaChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
title = str(event_data["name"]).split("/")[0].strip().replace(".", " ")
if event_data.get("year"):
year = str(event_data["year"]).split("/")[0].strip()
if event_data.get("season") and str(event_data["season"]).isdigit():
season_number = int(event_data["season"])
if event_data.get("episode") and str(event_data["episode"]).isdigit():
episode_number = int(event_data["episode"])
season_number = self._parse_recognize_event_number(event_data.get("season"))
episode_number = self._parse_recognize_event_number(event_data.get("episode"))
if not title:
return None
if title == "Unknown":
@@ -1654,7 +1663,7 @@ class MediaChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
org_meta.year = year
org_meta.begin_season = season_number
org_meta.begin_episode = episode_number
if org_meta.begin_season or org_meta.begin_episode:
if org_meta.begin_season is not None or org_meta.begin_episode is not None:
org_meta.type = MediaType.TV
# 重新识别
return await self.async_recognize_media(
@@ -1855,7 +1864,7 @@ class MediaChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
tmdbinfo = await self._async_match_tmdb_with_names(
meta_names=meta_names,
year=year,
mtype=MediaType.TV,
mtype=MediaInfo.get_bangumi_media_type(bangumiinfo),
season=meta.begin_season,
)
return tmdbinfo
@@ -1895,6 +1904,9 @@ class MediaChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
year = self._extract_year_from_bangumi(bangumiinfo)
# 使用名称识别豆瓣媒体信息
return await self.async_match_doubaninfo(
name=meta.name, year=year, mtype=MediaType.TV, season=meta.begin_season
name=meta.name,
year=year,
mtype=MediaInfo.get_bangumi_media_type(bangumiinfo),
season=meta.begin_season,
)
return None
+265 -11
View File
@@ -1,6 +1,6 @@
import threading
from datetime import datetime
from typing import List, Union, Optional, Generator, Any
from typing import Callable, Dict, List, Union, Optional, Generator, Any
from app.chain import ChainBase
from app.core.config import global_vars
@@ -109,6 +109,35 @@ class MediaServerChain(ChainBase):
yield from self.run_module("mediaserver_items", server=server, library_id=library_id,
start_index=start_index, limit=limit)
def items_count(self, server: str, library_id: Union[str, int]) -> Optional[int]:
"""
获取指定媒体库可同步的媒体条目总数
:param server: 媒体服务器名称
:param library_id: 媒体库ID
:return: 媒体条目总数无法获取时返回None
"""
return self.run_module(
"mediaserver_items_count",
server=server,
library_id=library_id,
)
def media_count(self, server: str) -> Optional[int]:
"""
获取指定媒体服务器可同步的电影和电视剧总数
:param server: 媒体服务器名称
:return: 电影和电视剧总数无法获取时返回None
"""
statistics = self.run_module("media_statistic", server=server)
if not statistics:
return None
return sum(
(statistic.movie_count or 0) + (statistic.tv_count or 0)
for statistic in statistics
)
def iteminfo(self, server: str, item_id: Union[str, int]) -> MediaServerItem:
"""
获取媒体服务器项目信息
@@ -181,6 +210,24 @@ class MediaServerChain(ChainBase):
"""
return self.run_module("mediaserver_play_url", server=server, item_id=item_id)
def get_season_episode_ids(self, server: str, item_id: Union[str, int],
season: int) -> Dict[int, str]:
"""
获取指定季的集号到媒体服务器条目 ID 映射
:param server: 媒体服务器名称
:param item_id: 剧集在媒体服务器中的条目 ID
:param season: 季号
:return: 集号到条目 ID 的映射无数据时返回空字典
"""
result = self.run_module(
"mediaserver_season_episode_ids",
server=server,
item_id=item_id,
season=season,
)
return result or {}
def get_image_cookies(
self, server: Optional[str], image_url: str
) -> Optional[str | dict]:
@@ -191,13 +238,17 @@ class MediaServerChain(ChainBase):
"mediaserver_image_cookies", server=server, image_url=image_url
)
def sync(self):
def sync(self, progress_callback: Optional[Callable[..., None]] = None) -> None:
"""
同步媒体库所有数据到本地数据库
:param progress_callback: 定时服务进度更新回调
"""
# 设置的媒体服务器
mediaservers = ServiceConfigHelper.get_mediaserver_configs()
if not mediaservers:
if progress_callback:
progress_callback(value=100, text="未配置媒体服务器,跳过同步")
return
with lock:
# 汇总统计
@@ -206,7 +257,86 @@ class MediaServerChain(ChainBase):
enabled_servers = [mediaserver.name for mediaserver in mediaservers
if mediaserver and mediaserver.enabled and mediaserver.name]
dboper.delete_excluded_servers(enabled_servers)
total_servers = len(enabled_servers)
if progress_callback:
progress_callback(
value=0,
text=f"开始同步媒体服务器,共 {total_servers} 个 ...",
data={"total": total_servers, "finished": 0},
)
if not total_servers:
if progress_callback:
progress_callback(value=100, text="没有已启用的媒体服务器")
return
server_sync_contexts = {}
global_media_total = 0
global_counts_available = True
for mediaserver in mediaservers:
if not mediaserver or not mediaserver.enabled:
continue
server_name = mediaserver.name
logger.info(f"正在统计媒体服务器 {server_name} 的待同步媒体数量")
libraries = self.librarys(server_name)
if not libraries:
server_sync_contexts[server_name] = None
continue
sync_libraries = mediaserver.sync_libraries or []
selected_libraries = []
for library in libraries:
if sync_libraries \
and "all" not in sync_libraries \
and str(library.id) not in sync_libraries:
logger.info(f"{library.name} 未在 {server_name} 同步媒体库列表中,跳过")
continue
selected_libraries.append(library)
library_media_counts = {
str(library.id): None for library in selected_libraries
}
sync_all_libraries = (
not sync_libraries or "all" in sync_libraries
)
server_media_count = (
self.media_count(server_name)
if sync_all_libraries else None
)
if server_media_count:
global_media_total += server_media_count
logger.info(
f"媒体服务器 {server_name}{server_media_count} 个媒体待同步"
)
else:
for library in selected_libraries:
media_count = self.items_count(
server=server_name,
library_id=library.id,
)
library_media_counts[str(library.id)] = media_count
if media_count is None:
global_counts_available = False
logger.warning(
f"未获取到 {server_name} 媒体库 {library.name} 的媒体总数,"
f"同步进度将按媒体库完成度计算"
)
else:
global_media_total += media_count
logger.info(
f"{server_name} 媒体库 {library.name}"
f"{media_count} 个媒体待同步"
)
server_sync_contexts[server_name] = (
selected_libraries,
library_media_counts,
)
if not global_counts_available:
global_media_total = None
# 遍历媒体服务器
server_index = 0
global_media_finished = 0
for mediaserver in mediaservers:
if not mediaserver:
continue
@@ -214,21 +344,55 @@ class MediaServerChain(ChainBase):
if not mediaserver.enabled:
logger.info(f"媒体服务器 {mediaserver.name} 未启用,跳过")
continue
server_index += 1
server_name = mediaserver.name
sync_libraries = mediaserver.sync_libraries or []
if progress_callback:
progress_value = (
global_media_finished / global_media_total * 100
if global_media_total else
(server_index - 1) / total_servers * 100
)
progress_callback(
value=progress_value,
text=(
f"正在同步媒体服务器"
f"{server_index}/{total_servers}{server_name} ..."
),
data={
"total": total_servers,
"finished": server_index - 1,
"current": server_name,
"media_total": global_media_total,
"media_finished": global_media_finished,
},
)
logger.info(f"开始同步媒体服务器 {server_name} 的数据 ...")
libraries = self.librarys(server_name)
if not libraries:
sync_context = server_sync_contexts.get(server_name)
if sync_context is None:
logger.info(f"没有获取到媒体服务器 {server_name} 的媒体库,跳过")
if progress_callback:
progress_value = (
global_media_finished / global_media_total * 100
if global_media_total else
server_index / total_servers * 100
)
progress_callback(
value=progress_value,
text=f"媒体服务器 {server_name} 无可同步媒体库",
data={
"total": total_servers,
"finished": server_index,
"media_total": global_media_total,
"media_finished": global_media_finished,
},
)
continue
sync_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S.%f")
for library in libraries:
if sync_libraries \
and "all" not in sync_libraries \
and str(library.id) not in sync_libraries:
logger.info(f"{library.name} 未在 {server_name} 同步媒体库列表中,跳过")
continue
selected_libraries, library_media_counts = sync_context
total_libraries = len(selected_libraries)
for library_index, library in enumerate(selected_libraries, start=1):
logger.info(f"正在同步 {server_name} 媒体库 {library.name} ...")
library_media_total = library_media_counts.get(str(library.id))
library_count = 0
for item in self.items(server=server_name, library_id=library.id):
if global_vars.is_system_stopped:
@@ -238,6 +402,7 @@ class MediaServerChain(ChainBase):
logger.debug(f"正在同步 {item.title} ...")
# 计数
library_count += 1
global_media_finished += 1
seasoninfo = {}
# 类型
item_type = "电视剧" if item.item_type in ["Series", "show"] else "电影"
@@ -252,9 +417,98 @@ class MediaServerChain(ChainBase):
item_dict["item_type"] = item_type
item_dict["lst_mod_date"] = sync_time
dboper.upsert(**item_dict)
if progress_callback:
if global_media_total:
progress_value = min(
global_media_finished / global_media_total,
1,
) * 100
else:
library_progress = (
min(library_count / library_media_total, 1)
if library_media_total else 0
)
server_progress = (
library_index - 1 + library_progress
) / total_libraries
progress_value = (
server_index - 1 + server_progress
) / total_servers * 100
progress_callback(
value=progress_value,
text=(
f"正在同步 {server_name} 媒体库 {library.name}"
f"{library_count}/{library_media_total}"
if library_media_total is not None
else f"正在同步 {server_name} 媒体库 {library.name}"
),
data={
"total": total_servers,
"finished": server_index - 1,
"current": server_name,
"library_total": total_libraries,
"library_finished": library_index - 1,
"current_library": library.name,
"library_media_total": library_media_total,
"library_media_finished": library_count,
"media_total": global_media_total,
"media_finished": global_media_finished,
},
)
logger.info(f"{server_name} 媒体库 {library.name} 同步完成,共同步数量:{library_count}")
# 总数累加
total_count += library_count
if progress_callback:
if global_media_total:
progress_value = min(
global_media_finished / global_media_total,
1,
) * 100
else:
server_progress = library_index / total_libraries
progress_value = (
server_index - 1 + server_progress
) / total_servers * 100
progress_callback(
value=progress_value,
text=(
f"{server_name} 媒体库"
f"{library_index}/{total_libraries}{library.name} 同步完成"
),
data={
"total": total_servers,
"finished": server_index - 1,
"current": server_name,
"library_total": total_libraries,
"library_finished": library_index,
"current_library": library.name,
"library_media_total": library_media_total,
"library_media_finished": library_count,
"media_total": global_media_total,
"media_finished": global_media_finished,
},
)
stale_count = dboper.delete_stale(server=server_name, sync_time=sync_time)
logger.info(f"媒体服务器 {server_name} 清理陈旧数据完成,删除数量:{stale_count}")
logger.info(f"媒体服务器 {server_name} 数据同步完成,总同步数量:{total_count}")
if progress_callback:
progress_value = (
min(global_media_finished / global_media_total, 1) * 100
if global_media_total else
server_index / total_servers * 100
)
progress_callback(
value=progress_value,
text=(
f"媒体服务器({server_index}/{total_servers}"
f"{server_name} 同步完成"
),
data={
"total": total_servers,
"finished": server_index,
"media_total": global_media_total,
"media_finished": global_media_finished,
},
)
if progress_callback:
progress_callback(value=100, text="媒体服务器同步完成")
+173 -7
View File
@@ -29,7 +29,12 @@ from app.db.models import TransferHistory
from app.db.transferhistory_oper import TransferHistoryOper
from app.db.user_oper import UserOper
from app.helper.directory import DirectoryHelper
from app.helper.interaction import agent_interaction_manager, media_interaction_manager, PendingMediaInteraction
from app.helper.interaction import (
agent_interaction_manager,
media_interaction_manager,
plugin_input_interaction_manager,
PendingMediaInteraction,
)
from app.helper.torrent import TorrentHelper
from app.log import logger
from app.schemas import CommingMessage, DownloadDirectory, FileURI, NotExistMediaInfo, Notification
@@ -136,9 +141,9 @@ class MessageChain(ChainBase):
logger.debug(f"未识别到消息内容::{body}{form}{args}")
return
# 获取原消息ID信息
original_message_id = info.message_id
original_chat_id = info.chat_id
reply_to_message_id = info.reply_to_message_id
# 处理消息
self.handle_message(
@@ -149,6 +154,7 @@ class MessageChain(ChainBase):
text=text,
original_message_id=original_message_id,
original_chat_id=original_chat_id,
reply_to_message_id=reply_to_message_id,
images=images,
audio_refs=audio_refs,
files=files,
@@ -160,12 +166,13 @@ class MessageChain(ChainBase):
source: str,
userid: Union[str, int],
username: str,
text: str,
text: Optional[str],
original_message_id: Optional[Union[str, int]] = None,
original_chat_id: Optional[str] = None,
images: Optional[List[CommingMessage.MessageImage]] = None,
audio_refs: Optional[List[str]] = None,
files: Optional[List[CommingMessage.MessageAttachment]] = None,
reply_to_message_id: Optional[Union[str, int]] = None,
) -> None:
"""
识别消息内容执行操作
@@ -201,6 +208,21 @@ class MessageChain(ChainBase):
)
return
if self._handle_plugin_input_interaction(
channel=channel,
source=source,
userid=userid,
username=username,
text=text,
original_chat_id=original_chat_id,
reply_to_message_id=reply_to_message_id,
images=images,
audio_refs=audio_refs,
files=files,
has_audio_input=has_audio_input,
):
return
is_agent_message = self._is_agent_message(
userid=userid,
text=text,
@@ -236,6 +258,7 @@ class MessageChain(ChainBase):
text=text,
original_message_id=original_message_id,
original_chat_id=original_chat_id,
reply_to_message_id=reply_to_message_id,
images=images,
audio_refs=audio_refs,
files=files,
@@ -259,7 +282,7 @@ class MessageChain(ChainBase):
source: str,
userid: Union[str, int],
username: str,
text: str,
text: Optional[str],
original_message_id: Optional[Union[str, int]] = None,
original_chat_id: Optional[str] = None,
images: Optional[List[CommingMessage.MessageImage]] = None,
@@ -267,6 +290,7 @@ class MessageChain(ChainBase):
files: Optional[List[CommingMessage.MessageAttachment]] = None,
has_audio_input: bool = False,
processing_status: Optional[_ProcessingStatus] = None,
reply_to_message_id: Optional[Union[str, int]] = None,
) -> bool:
"""执行实际消息路由,便于统一包裹处理中状态。"""
@@ -290,6 +314,21 @@ class MessageChain(ChainBase):
)
return False
if self._handle_plugin_input_interaction(
channel=channel,
source=source,
userid=userid,
username=username,
text=text,
original_chat_id=original_chat_id,
reply_to_message_id=reply_to_message_id,
images=images,
audio_refs=audio_refs,
files=files,
has_audio_input=has_audio_input,
):
return False
no_ai_requested, no_ai_text = self._strip_no_ai_prefix(text)
if no_ai_requested:
text = no_ai_text
@@ -411,10 +450,128 @@ class MessageChain(ChainBase):
"userid": userid,
"channel": channel,
"source": source,
"chat_id": original_chat_id,
"reply_to_message_id": reply_to_message_id,
},
)
return False
def _handle_plugin_input_interaction(
self,
channel: MessageChannel,
source: str,
userid: Union[str, int],
username: str,
text: str,
original_chat_id: Optional[Union[str, int]] = None,
images: Optional[List[CommingMessage.MessageImage]] = None,
audio_refs: Optional[List[str]] = None,
files: Optional[List[CommingMessage.MessageAttachment]] = None,
has_audio_input: bool = False,
reply_to_message_id: Optional[Union[str, int]] = None,
) -> bool:
"""
将插件输入会话中的下一条普通文本派发给指定插件
"""
if not text or not text.strip() or images or audio_refs or files or has_audio_input:
return False
if text.startswith("CALLBACK:"):
return False
is_cancel_text = text.strip().lower() in {"取消", "退出", "q", "quit", "exit"}
request, status = plugin_input_interaction_manager.consume_by_user(
userid,
channel,
source,
original_chat_id,
reply_to_message_id=reply_to_message_id,
bypass_reply_check=is_cancel_text,
)
if not request:
return False
if status == "expired":
self.eventmanager.send_event(
EventType.MessageAction,
{
"plugin_id": request.plugin_id,
"__mp_target_plugin_id": request.plugin_id,
"text": f"plugin_input_expired|{request.request_id}",
"userid": userid,
"channel": channel,
"source": source,
"username": username,
"chat_id": original_chat_id,
"reply_to_message_id": reply_to_message_id,
"prompt_id": request.prompt_id,
"input_session_id": request.request_id,
"expired": True,
"payload": request.payload,
},
)
self.post_message(
Notification(
channel=channel,
source=source,
userid=userid,
username=username,
title="插件输入已超时,请重新发起操作。",
save_history=False,
)
)
return not text.strip().startswith("/")
if is_cancel_text:
self.eventmanager.send_event(
EventType.MessageAction,
{
"plugin_id": request.plugin_id,
"__mp_target_plugin_id": request.plugin_id,
"text": f"plugin_input_cancel|{request.request_id}",
"userid": userid,
"channel": channel,
"source": source,
"username": username,
"chat_id": original_chat_id,
"reply_to_message_id": reply_to_message_id,
"prompt_id": request.prompt_id,
"input_session_id": request.request_id,
"cancelled": True,
"payload": request.payload,
},
)
self.post_message(
Notification(
channel=channel,
source=source,
userid=userid,
username=username,
title="已取消插件输入",
save_history=False,
)
)
return True
self.eventmanager.send_event(
EventType.MessageAction,
{
"plugin_id": request.plugin_id,
"__mp_target_plugin_id": request.plugin_id,
"text": f"plugin_input|{request.request_id}",
"input_text": text,
"userid": userid,
"channel": channel,
"source": source,
"username": username,
"chat_id": original_chat_id,
"reply_to_message_id": reply_to_message_id,
"prompt_id": request.prompt_id,
"input_session_id": request.request_id,
"payload": request.payload,
},
)
return True
@classmethod
def _strip_no_ai_prefix(cls, text: str) -> Tuple[bool, str]:
"""
@@ -1930,7 +2087,7 @@ class MediaInteractionChain(ChainBase):
mediakey = mediainfo.tmdb_id or mediainfo.douban_id
no_exists = {mediakey: {}}
if meta.begin_season:
if meta.begin_season is not None:
episodes = mediainfo.seasons.get(meta.begin_season)
if not episodes:
return {}
@@ -2635,6 +2792,8 @@ class MediaInteractionChain(ChainBase):
download_dirs = self._get_download_dirs(media_info)
if not download_dirs:
return False
if len(download_dirs) == 1 and not self._is_auto_download_dir(download_dirs[0]):
return False
request.pending_torrent_page = request.page
request.phase = "download-dir"
@@ -3252,6 +3411,11 @@ class MediaInteractionChain(ChainBase):
"""
获取可供消息交互选择的下载目录
"""
dir_infos = [
dir_info
for dir_info in DirectoryHelper().get_download_dirs()
if dir_info.download_path
]
download_dirs = [
DownloadDirectory(
name=dir_info.name,
@@ -3265,11 +3429,13 @@ class MediaInteractionChain(ChainBase):
media_type=dir_info.media_type,
media_category=dir_info.media_category,
)
for dir_info in DirectoryHelper().get_download_dirs()
if dir_info.download_path and cls._match_download_dir_media(dir_info, media_info)
for dir_info in dir_infos
if cls._match_download_dir_media(dir_info, media_info)
]
if not download_dirs:
return []
if len(download_dirs) == 1:
return download_dirs
return [cls._build_auto_download_dir(), *download_dirs]
@classmethod
+49 -5
View File
@@ -1,4 +1,4 @@
from typing import List, Optional
from typing import Callable, List, Optional
import pillow_avif # noqa 用于自动注册AVIF支持
@@ -27,11 +27,16 @@ class RecommendChain(ChainBase, metaclass=Singleton):
# 推荐缓存区域
recommend_cache_region = "recommend"
def refresh_recommend(self, manual: bool = False):
def refresh_recommend(
self,
manual: bool = False,
progress_callback: Optional[Callable[..., None]] = None,
) -> None:
"""
刷新推荐
:param manual: 手动触发
:param progress_callback: 定时服务进度更新回调
"""
logger.debug("Starting to refresh Recommend data.")
@@ -56,6 +61,14 @@ class RecommendChain(ChainBase, metaclass=Singleton):
recommends = []
# 记录哪些方法已完成
methods_finished = set()
total_requests = len(recommend_methods) * self.cache_max_pages
finished_requests = 0
if progress_callback:
progress_callback(
value=0,
text=f"开始刷新推荐缓存,共 {total_requests} 个数据分页 ...",
data={"total": total_requests, "finished": 0},
)
# 这里避免区间内连续调用相同来源,因此遍历方案为每页遍历所有推荐来源,再进行页数遍历
for page in range(1, self.cache_max_pages + 1):
for method in recommend_methods:
@@ -67,6 +80,21 @@ class RecommendChain(ChainBase, metaclass=Singleton):
# 手动触发的刷新,总是需要获取最新数据
with fresh(manual):
data = method(page=page)
finished_requests += 1
if progress_callback:
progress_callback(
value=finished_requests / total_requests * 90,
text=(
f"正在刷新推荐缓存"
f"{finished_requests}/{total_requests}..."
),
data={
"total": total_requests,
"finished": finished_requests,
"current": method.__name__,
"page": page,
},
)
if not data:
logger.debug("All recommendation methods have finished fetching data. Ending pagination early.")
methods_finished.add(method)
@@ -77,24 +105,40 @@ class RecommendChain(ChainBase, metaclass=Singleton):
break
# 缓存收集到的海报
self.__cache_posters(recommends)
if progress_callback:
progress_callback(value=90, text="推荐数据刷新完成,正在缓存海报 ...")
self.__cache_posters(recommends, progress_callback=progress_callback)
logger.debug("Recommend data refresh completed.")
if progress_callback:
progress_callback(value=100, text="推荐缓存刷新完成")
def __cache_posters(self, datas: List[dict]):
def __cache_posters(
self,
datas: List[dict],
progress_callback: Optional[Callable[..., None]] = None,
) -> None:
"""
提取 poster_path 并缓存图片
:param datas: 数据列表
:param progress_callback: 定时服务进度更新回调
"""
if not settings.GLOBAL_IMAGE_CACHE:
return
for data in datas:
total_num = len(datas)
for index, data in enumerate(datas, start=1):
if global_vars.is_system_stopped:
return
poster_path = data.get("poster_path")
if poster_path:
poster_url = poster_path.replace("original", "w500")
self.__fetch_and_save_image(poster_url)
if progress_callback:
progress_callback(
value=90 + (index / total_num * 10 if total_num else 10),
text=f"正在缓存推荐海报({index}/{total_num}...",
data={"poster_total": total_num, "poster_finished": index},
)
@staticmethod
def __fetch_and_save_image(url: str):
+41 -4
View File
@@ -203,7 +203,7 @@ class SearchChain(ChainBase):
"area": str(params.get("area") or ""),
"title": str(params.get("title") or ""),
"year": str(params.get("year") or ""),
"season": str(params.get("season") or ""),
"season": str(params["season"]) if params.get("season") is not None else "",
"episode": str(params.get("episode") or ""),
"sites": str(params.get("sites") or ""),
"result_type": str(params.get("result_type") or "torrent"),
@@ -530,7 +530,7 @@ class SearchChain(ChainBase):
def search_by_title(self, title: str, page: Optional[int] = 0,
sites: List[int] = None, cache_local: Optional[bool] = False) -> List[Context]:
"""
根据标题搜索资源不识别不过滤直接返回站点内容
根据标题搜索资源不识别媒体信息按默认搜索过滤规则返回站点内容
:param title: 标题为空时返回所有站点首页内容
:param page: 页码
:param sites: 站点ID列表
@@ -552,6 +552,10 @@ class SearchChain(ChainBase):
if not torrents:
logger.warn(f'{title} 未搜索到资源')
return []
torrents = self.__filter_title_search_torrents(torrents=torrents)
if not torrents:
logger.warn(f'{title} 没有符合过滤规则的资源')
return []
# 组装上下文
contexts = [
Context(
@@ -791,7 +795,7 @@ class SearchChain(ChainBase):
async def async_search_by_title(self, title: str, page: Optional[int] = 0,
sites: List[int] = None, cache_local: Optional[bool] = False) -> List[Context]:
"""
根据标题异步搜索资源不识别不过滤直接返回站点内容
根据标题异步搜索资源不识别媒体信息按默认搜索过滤规则返回站点内容
:param title: 标题为空时返回所有站点首页内容
:param page: 页码
:param sites: 站点ID列表
@@ -813,6 +817,10 @@ class SearchChain(ChainBase):
if not torrents:
logger.warn(f'{title} 未搜索到资源')
return []
torrents = await run_in_threadpool(self.__filter_title_search_torrents, torrents=torrents)
if not torrents:
logger.warn(f'{title} 没有符合过滤规则的资源')
return []
# 组装上下文
contexts = [
Context(
@@ -830,7 +838,7 @@ class SearchChain(ChainBase):
sites: List[int] = None,
cache_local: Optional[bool] = False) -> AsyncIterator[dict]:
"""
根据标题渐进式搜索资源不识别不过滤站点完成顺序返回结果
根据标题渐进式搜索资源不识别媒体信息默认搜索过滤规则返回结果
"""
if cache_local:
self.cancel_ai_recommend()
@@ -845,8 +853,14 @@ class SearchChain(ChainBase):
logger.info(f'开始渐进式浏览资源,站点:{sites} ...')
contexts: List[Context] = []
rule_groups: List[str] = SystemConfigOper().get(SystemConfigKey.SearchFilterRuleGroups) or []
async for event in self.__async_search_all_sites_stream(keyword=title, sites=sites, page=page):
result = event.pop("items", []) or []
result = await run_in_threadpool(
self.__filter_title_search_torrents,
torrents=result,
rule_groups=rule_groups,
)
batch_contexts = [
Context(
meta_info=MetaInfo(title=torrent.title, subtitle=torrent.description),
@@ -876,6 +890,29 @@ class SearchChain(ChainBase):
"total_items": len(contexts)
}
def __filter_title_search_torrents(self,
torrents: List[TorrentInfo],
rule_groups: Optional[List[str]] = None) -> List[TorrentInfo]:
"""
对标题搜索结果应用默认搜索过滤规则不执行媒体识别和标题精确匹配
"""
if not torrents:
return []
if rule_groups is None:
rule_groups = SystemConfigOper().get(SystemConfigKey.SearchFilterRuleGroups) or []
if not rule_groups:
return torrents
logger.info(f'开始过滤标题搜索结果,使用规则组:{rule_groups} ...')
filtered_torrents = self.filter_torrents(
rule_groups=rule_groups,
torrent_list=torrents,
mediainfo=None,
) or []
logger.info(f'标题搜索过滤完成,剩余 {len(filtered_torrents)} 个资源')
return filtered_torrents
async def async_search_by_id_stream(self, tmdbid: Optional[int] = None, doubanid: Optional[str] = None,
mtype: MediaType = None, area: Optional[str] = "title",
season: Optional[int] = None, sites: List[int] = None,
+70 -10
View File
@@ -1,7 +1,7 @@
import base64
import re
from datetime import datetime
from typing import List, Optional, Tuple, Union, Dict
from typing import Callable, List, Optional, Tuple, Union, Dict
from urllib.parse import urljoin
from app.helper.sites import SitesHelper # noqa
@@ -103,24 +103,54 @@ class SiteChain(ChainBase):
))
return userdata
def refresh_userdatas(self) -> Optional[Dict[str, SiteUserData]]:
def refresh_userdatas(
self,
progress_callback: Optional[Callable[..., None]] = None,
) -> Optional[Dict[str, SiteUserData]]:
"""
刷新所有站点的用户数据
:param progress_callback: 定时服务进度更新回调
"""
any_site_updated = False
result = {}
for site in SitesHelper().get_indexers():
sites = [site for site in SitesHelper().get_indexers() if site.get("is_active")]
total_num = len(sites)
if progress_callback:
progress_callback(
value=0,
text=f"开始刷新站点数据,共 {total_num} 个站点 ...",
data={"total": total_num, "finished": 0},
)
for index, site in enumerate(sites, start=1):
if global_vars.is_system_stopped:
return None
if site.get("is_active"):
userdata = self.refresh_userdata(site)
if userdata:
any_site_updated = True
result[site.get("name")] = userdata
if progress_callback:
progress_callback(
value=(index - 1) / total_num * 100 if total_num else 100,
text=f"正在刷新站点数据({index}/{total_num}{site.get('name')} ...",
data={
"total": total_num,
"finished": index - 1,
"current": site.get("id"),
},
)
userdata = self.refresh_userdata(site)
if userdata:
any_site_updated = True
result[site.get("name")] = userdata
if progress_callback:
progress_callback(
value=index / total_num * 100 if total_num else 100,
text=f"站点数据({index}/{total_num})刷新完成",
data={"total": total_num, "finished": index},
)
if any_site_updated:
eventmanager.send_event(EventType.SiteRefreshed, {
"site_id": "*"
})
if progress_callback:
progress_callback(value=100, text="站点数据刷新完成")
return result
@@ -323,9 +353,16 @@ class SiteChain(ChainBase):
del html
return favicon_url, None
def sync_cookies(self, manual=False) -> Tuple[bool, str]:
def sync_cookies(
self,
manual: bool = False,
progress_callback: Optional[Callable[..., None]] = None,
) -> Tuple[bool, str]:
"""
通过CookieCloud同步站点Cookie
:param manual: 是否手动同步
:param progress_callback: 定时服务进度更新回调
"""
def __indexer_domain(inx: dict, sub_domain: str) -> str:
@@ -340,9 +377,13 @@ class SiteChain(ChainBase):
return sub_domain
logger.info("开始同步CookieCloud站点 ...")
if progress_callback:
progress_callback(value=0, text="开始下载 CookieCloud 数据 ...")
cookies, msg = CookieCloudHelper().download()
if not cookies:
logger.error(f"CookieCloud同步失败:{msg}")
if progress_callback:
progress_callback(value=100, text=f"CookieCloud同步失败:{msg}")
if manual:
self.messagehelper.put(msg, title="CookieCloud同步失败", role="system")
return False, msg
@@ -353,11 +394,22 @@ class SiteChain(ChainBase):
siteshelper = SitesHelper()
siteoper = SiteOper()
rsshelper = RssHelper()
for domain, cookie in cookies.items():
total_num = len(cookies)
for index, (domain, cookie) in enumerate(cookies.items(), start=1):
# 检查系统是否停止
if global_vars.is_system_stopped:
logger.info("系统正在停止,中断CookieCloud同步")
return False, "系统正在停止,同步被中断"
if progress_callback:
progress_callback(
value=(index - 1) / total_num * 100 if total_num else 100,
text=f"正在同步 CookieCloud 站点({index}/{total_num}{domain} ...",
data={
"total": total_num,
"finished": index - 1,
"current": domain,
},
)
# 索引器信息
indexer = siteshelper.get_indexer(domain)
@@ -465,6 +517,12 @@ class SiteChain(ChainBase):
eventmanager.send_event(EventType.SiteUpdated, {
"domain": domain,
})
if progress_callback:
progress_callback(
value=index / total_num * 100 if total_num else 100,
text=f"CookieCloud 站点({index}/{total_num})同步完成",
data={"total": total_num, "finished": index},
)
# 处理完成
ret_msg = f"更新了{_update_count}个站点,新增了{_add_count}个站点"
if _fail_count > 0:
@@ -472,6 +530,8 @@ class SiteChain(ChainBase):
if manual:
self.messagehelper.put(ret_msg, title="CookieCloud同步成功", role="system")
logger.info(f"CookieCloud同步成功:{ret_msg}")
if progress_callback:
progress_callback(value=100, text=f"CookieCloud同步成功:{ret_msg}")
return True, ret_msg
@eventmanager.register(EventType.SiteUpdated)
+1028 -262
View File
File diff suppressed because it is too large Load Diff
-4
View File
@@ -38,8 +38,6 @@ class SystemChain(ChainBase):
"""
重启系统
"""
from app.core.config import global_vars
if channel and userid:
self.post_message(Notification(
channel=channel,
@@ -54,8 +52,6 @@ class SystemChain(ChainBase):
}, self._restart_file)
# 主动备份一次插件
self.backup_plugins()
# 设置停止标志,通知所有模块准备停止
global_vars.stop_system()
# 重启
SystemHelper.restart()
+273 -6
View File
@@ -1,6 +1,7 @@
import copy
import re
import traceback
from typing import Dict, List, Union, Optional
from typing import Callable, Dict, List, Union, Optional
from app.helper.sites import SitesHelper # noqa
@@ -92,6 +93,238 @@ class TorrentsChain(ChainBase):
return torrents_cache
def get_subscribe_cache_candidates(
self,
subscribe,
stype: Optional[str] = None,
allow_title_match: bool = False,
) -> List[Context]:
"""
按订阅身份读取 RSS/spider 缓存候选返回不会回写缓存的 Context 副本
主程序只提供缓存读取与轻量候选筛选不在这里判断站点证据能否扩展
订阅目标或放行完成标题兜底候选会显式标记为低置信来源
"""
results: List[Context] = []
for contexts in (self.get_torrents(stype=stype) or {}).values():
for context in contexts or []:
if not context:
continue
copied = copy.deepcopy(context)
if self._context_matches_subscribe(copied, subscribe):
results.append(copied)
continue
if allow_title_match and self._context_title_matches_subscribe(copied, subscribe):
self._mark_title_match_candidate(copied, subscribe)
results.append(copied)
return results
@classmethod
def _context_matches_subscribe(cls, context: Context, subscribe) -> bool:
"""
严格身份匹配候选自身识别出的媒体 ID 命中订阅且季信息不排除订阅季
"""
if not cls._context_media_type_matches(context, subscribe):
return False
if not cls._context_season_matches_subscribe(context, subscribe):
return False
subscribe_tmdbid = cls._normalize_id(getattr(subscribe, "tmdbid", None))
subscribe_doubanid = cls._normalize_id(getattr(subscribe, "doubanid", None))
context_tmdbids = cls._context_tmdb_ids(context)
context_doubanids = cls._context_douban_ids(context)
return bool(
subscribe_tmdbid and subscribe_tmdbid in context_tmdbids
or subscribe_doubanid and subscribe_doubanid in context_doubanids
)
@classmethod
def _context_title_matches_subscribe(cls, context: Context, subscribe) -> bool:
"""
标题兜底只服务诊断仅允许身份缺失候选按标题命中显式冲突 ID 不兜底
"""
if cls._context_has_media_identity(context):
return False
if not cls._context_media_type_matches(context, subscribe):
return False
if not cls._context_season_matches_subscribe(context, subscribe):
return False
subscribe_title = cls._normalize_title(getattr(subscribe, "name", None))
if not subscribe_title:
return False
meta_info = getattr(context, "meta_info", None)
torrent_info = getattr(context, "torrent_info", None)
candidate_titles = [
getattr(torrent_info, "title", None),
getattr(meta_info, "title", None),
getattr(meta_info, "name", None),
]
return any(
subscribe_title in candidate_title
for candidate_title in (cls._normalize_title(title) for title in candidate_titles)
if candidate_title
)
@staticmethod
def _mark_title_match_candidate(context: Context, subscribe) -> None:
"""
标记标题兜底候选避免下游把目标媒体回填误认为候选自身识别结果
"""
context.match_source = "title"
context.candidate_recognized = False
context.media_info_is_target = True
context.media_info = MediaInfo(
type=getattr(subscribe, "type", None),
title=getattr(subscribe, "name", None),
tmdb_id=getattr(subscribe, "tmdbid", None),
douban_id=getattr(subscribe, "doubanid", None),
season=getattr(subscribe, "season", None),
)
@classmethod
def _context_media_type_matches(cls, context: Context, subscribe) -> bool:
"""
类型已知且冲突时拒绝缺失类型不作为缓存候选过滤条件
"""
subscribe_type = cls._normalize_media_type(getattr(subscribe, "type", None))
media_info = getattr(context, "media_info", None)
meta_info = getattr(context, "meta_info", None)
context_types = {
cls._normalize_media_type(value)
for value in (
getattr(media_info, "type", None),
getattr(meta_info, "type", None),
)
}
context_types.discard(None)
return not subscribe_type or not context_types or all(
context_type == subscribe_type for context_type in context_types
)
@classmethod
def _context_season_matches_subscribe(cls, context: Context, subscribe) -> bool:
"""
资源季信息只要明确排除订阅季就拒绝跨季覆盖目标季留给插件诊断
"""
target_season = cls._normalize_int(getattr(subscribe, "season", None))
if target_season is None:
return True
meta_info = getattr(context, "meta_info", None)
explicit_meta_seasons = cls._context_meta_seasons(meta_info)
if explicit_meta_seasons:
return target_season in explicit_meta_seasons
media_info = getattr(context, "media_info", None)
media_season = cls._normalize_int(getattr(media_info, "season", None))
return media_season is None or target_season == media_season
@classmethod
def _context_meta_seasons(cls, meta_info) -> set[int]:
"""
提取标题解析出的显式季范围多季包以该范围为准
"""
meta_fields = vars(meta_info) if meta_info else {}
if "season_list" in meta_fields:
season_list = {
season
for season in (
cls._normalize_int(item)
for item in (meta_fields.get("season_list") or [])
)
if season is not None
}
if season_list:
return season_list
begin_season = cls._normalize_int(getattr(meta_info, "begin_season", None))
end_season = cls._normalize_int(getattr(meta_info, "end_season", None))
if begin_season is not None and end_season is not None:
start, end = sorted((begin_season, end_season))
return set(range(start, end + 1))
if begin_season is not None:
return {begin_season}
if end_season is not None:
return {end_season}
return set()
@staticmethod
def _context_has_media_identity(context: Context) -> bool:
"""
判断候选是否已经带有明确媒体 ID
"""
return bool(TorrentsChain._context_tmdb_ids(context) or TorrentsChain._context_douban_ids(context))
@staticmethod
def _context_tmdb_ids(context: Context) -> set[str]:
"""
提取候选已有 TMDB ID兼容 media_info 与标题显式标签
"""
media_info = getattr(context, "media_info", None)
meta_info = getattr(context, "meta_info", None)
return {
value for value in (
TorrentsChain._normalize_id(getattr(media_info, "tmdb_id", None)),
TorrentsChain._normalize_id(getattr(meta_info, "tmdbid", None)),
) if value
}
@staticmethod
def _context_douban_ids(context: Context) -> set[str]:
"""
提取候选已有豆瓣 ID兼容 media_info 与标题显式标签
"""
media_info = getattr(context, "media_info", None)
meta_info = getattr(context, "meta_info", None)
return {
value for value in (
TorrentsChain._normalize_id(getattr(media_info, "douban_id", None)),
TorrentsChain._normalize_id(getattr(meta_info, "doubanid", None)),
) if value
}
@staticmethod
def _normalize_id(value) -> Optional[str]:
"""
统一比较媒体 ID避免 int/string 形态差异影响缓存候选筛选
"""
if value is None:
return None
value = str(value).strip()
return value or None
@staticmethod
def _normalize_int(value) -> Optional[int]:
"""
将季号等动态字段转为 int无法解析时视为缺失
"""
if value is None:
return None
try:
return int(value)
except (TypeError, ValueError):
return None
@staticmethod
def _normalize_media_type(value) -> Optional[str]:
"""
统一 MediaType 枚举与字符串形态
"""
if isinstance(value, MediaType):
value = value.value
if value == MediaType.UNKNOWN.value:
return None
return value
@staticmethod
def _normalize_title(value) -> str:
"""
归一标题用于低置信标题兜底匹配
"""
return (StringUtils.clear_upper(value or "") or "").strip()
def clear_torrents(self):
"""
清理种子缓存数据
@@ -192,11 +425,17 @@ class TorrentsChain(ChainBase):
del rss_items
return ret_torrents
def refresh(self, stype: Optional[str] = None, sites: List[int] = None) -> Dict[str, List[Context]]:
def refresh(
self,
stype: Optional[str] = None,
sites: List[int] = None,
progress_callback: Optional[Callable[..., None]] = None,
) -> Dict[str, List[Context]]:
"""
刷新站点最新资源识别并缓存起来
:param stype: 强制指定缓存类型spider:爬虫缓存rss:rss缓存
:param sites: 强制指定站点ID列表为空则读取设置的订阅站点
:param progress_callback: 资源刷新进度更新回调
"""
def __is_no_cache_site(_domain: str) -> bool:
@@ -226,13 +465,34 @@ class TorrentsChain(ChainBase):
# 需要刷新的站点domain
domains = []
indexers = [
indexer for indexer in SitesHelper().get_indexers()
if not sites or indexer.get("id") in sites
]
total_indexers = len(indexers)
if progress_callback:
progress_callback(
value=0,
text=f"开始刷新站点资源,共 {total_indexers} 个站点 ...",
data={"total": total_indexers, "finished": 0},
)
# 遍历站点缓存资源
for indexer in SitesHelper().get_indexers():
for index, indexer in enumerate(indexers, start=1):
if global_vars.is_system_stopped:
break
# 未开启的站点不刷新
if sites and indexer.get("id") not in sites:
continue
if progress_callback:
progress_callback(
value=(index - 1) / total_indexers * 100 if total_indexers else 100,
text=(
f"正在刷新站点资源({index}/{total_indexers}"
f"{indexer.get('name')} ..."
),
data={
"total": total_indexers,
"finished": index - 1,
"current": indexer.get("id"),
},
)
domain = StringUtils.get_url_domain(indexer.get("domain"))
domains.append(domain)
if stype == "spider":
@@ -335,6 +595,13 @@ class TorrentsChain(ChainBase):
if sites and torrents_cache:
torrents_cache = {k: v for k, v in torrents_cache.items() if k in domains}
if progress_callback:
progress_callback(
value=100,
text="站点资源刷新完成",
data={"total": total_indexers, "finished": total_indexers},
)
return torrents_cache
@staticmethod
+261 -57
View File
@@ -40,6 +40,7 @@ from app.schemas import (
TransferQueue,
TransferJob,
TransferJobTask,
TmdbEpisode,
)
from app.schemas.exception import OperationInterrupted
from app.schemas.types import (
@@ -781,6 +782,7 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
}
def __init__(self):
"""初始化文件整理处理链。"""
super().__init__()
# 主要媒体文件后缀
self._media_exts = settings.RMT_MEDIAEXT
@@ -840,6 +842,7 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
logger.info("文件整理线程已停止")
def on_config_changed(self):
"""配置变更时重启文件整理线程。"""
self.__stop()
self.__init()
@@ -947,6 +950,7 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
mediainfo=task.mediainfo,
transferinfo=transferinfo,
season_episode=se_str,
episodes_info=task.episodes_info,
username=task.username,
)
@@ -1238,14 +1242,44 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
history_exists: bool = True,
):
"""
当同一种子的任务都已结束时回写下载器已整理标签
当同一种子的任务都已结束且种子已完成下载回写下载器已整理标签
"""
if (
history_exists
and download_hash
and self.jobview.is_torrent_done(download_hash)
not history_exists
or not download_hash
or not self.jobview.is_torrent_done(download_hash)
):
self.transfer_completed(hashs=download_hash, downloader=downloader)
return
# 作业视图只包含已登记的整理任务;多集种子部分文件先下载完成时,
# 剩余文件尚未产生任务,此时打已整理标签会使下载器轮询永久跳过
# 剩余文件(#6009),因此必须确认种子已整体下载完成。
if not self.__is_torrent_download_completed(download_hash, downloader):
logger.debug(
f"种子 {download_hash} 尚未下载完成或状态未知,暂不设置已整理标签"
)
return
if not self.jobview.is_torrent_done(download_hash):
logger.debug(
f"种子 {download_hash} 存在新登记的整理任务,暂不设置已整理标签"
)
return
self.transfer_completed(hashs=download_hash, downloader=downloader)
def __is_torrent_download_completed(
self, download_hash: str, downloader: Optional[str]
) -> bool:
"""
检查种子在下载器中是否已完成下载查询不到或查询失败时视为未完成
留待下载器定时轮询兜底避免误打已整理标签
"""
try:
torrents = self.list_torrents(hashs=download_hash, downloader=downloader)
if not torrents:
return False
return all((torrent.progress or 0) >= 100 for torrent in torrents)
except Exception as e:
logger.error(f"检查种子 {download_hash} 下载进度失败:{e}")
return False
def __send_metadata_scrape_event(
self, task: TransferTask, transferinfo: TransferInfo
@@ -1541,7 +1575,13 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
if download_history:
task.username = download_history.username
# 识别媒体信息
if download_history.tmdbid or download_history.doubanid:
history_year_conflict = self._is_movie_year_conflict(
task.meta, download_history
)
if (
(download_history.tmdbid or download_history.doubanid)
and not history_year_conflict
):
# 下载记录中已存在识别信息
mediainfo: Optional[MediaInfo] = self.recognize_media(
mtype=MediaType(download_history.type),
@@ -1554,6 +1594,18 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
# 更新自定义媒体类别
if download_history.media_category:
mediainfo.category = download_history.media_category
else:
if history_year_conflict:
logger.info(
f"{task.fileitem.name} 文件年份 {task.meta.year} 与下载记录年份 "
f"{download_history.year} 不一致,按文件名重新识别"
)
mediainfo = MediaChain().recognize_by_meta(
task.meta,
obtain_images=True,
)
if mediainfo and download_history.media_category:
mediainfo.category = download_history.media_category
else:
# 识别媒体信息
mediainfo = MediaChain().recognize_by_meta(
@@ -1915,9 +1967,11 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
sample_files.append(item)
return sample_files
def process(self) -> bool:
def process(self, progress_callback: Optional[Callable[..., None]] = None) -> bool:
"""
获取下载器中的种子列表并执行整理
:param progress_callback: 定时服务进度更新回调
"""
# 全局锁,避免定时服务重复
with downloader_lock:
@@ -1929,9 +1983,13 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
dir_info.monitor_type == "downloader" and dir_info.storage == "local"
for dir_info in download_dirs
):
if progress_callback:
progress_callback(value=100, text="未配置下载器监控目录,跳过整理")
return True
logger.info("开始整理下载器中已经完成下载的文件 ...")
if progress_callback:
progress_callback(value=0, text="正在查询已完成下载任务 ...")
# 从下载器获取种子列表
if torrents_list := self.list_torrents(status=TorrentStatus.TRANSFER):
@@ -1949,14 +2007,38 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
if not torrents:
logger.info("没有已完成下载但未整理的任务")
if progress_callback:
progress_callback(value=100, text="没有已完成下载但未整理的任务")
return False
logger.info(f"获取到 {len(torrents)} 个已完成的下载任务")
if progress_callback:
progress_callback(
value=0,
text=f"获取到 {len(torrents)} 个已完成下载任务",
data={"total": len(torrents), "finished": 0},
)
try:
for torrent in torrents:
total_num = len(torrents)
for index, torrent in enumerate(torrents, start=1):
if global_vars.is_system_stopped:
break
if progress_callback:
torrent_name = (
getattr(torrent, "title", None)
or getattr(torrent, "name", None)
or torrent.hash
)
progress_callback(
value=(index - 1) / total_num * 100,
text=f"正在整理下载任务({index}/{total_num}{torrent_name} ...",
data={
"total": total_num,
"finished": index - 1,
"current": torrent.hash,
},
)
# 文件路径
file_path = torrent.path
@@ -2020,9 +2102,15 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
extension=file_path.suffix.lstrip("."),
),
mediainfo=mediainfo,
downloader=torrent.downloader,
download_hash=torrent.hash,
)
downloader=torrent.downloader,
download_hash=torrent.hash,
)
if progress_callback:
progress_callback(
value=index / total_num * 100,
text=f"下载任务({index}/{total_num})整理处理完成",
data={"total": total_num, "finished": index},
)
finally:
torrents.clear()
@@ -2127,6 +2215,7 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
"""
shared_roots: set[str] = set()
media_type_dirs = {mtype.value for mtype in MediaType}
media_categories = None
for dir_info in DirectoryHelper().get_download_dirs():
if not dir_info.download_path:
@@ -2140,6 +2229,7 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
relative_parts = file_path.relative_to(download_root).parts
current_root = download_root
part_index = 0
media_type = dir_info.media_type
if (
not dir_info.media_type
@@ -2149,6 +2239,7 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
):
current_root = current_root / relative_parts[part_index]
shared_roots.add(current_root.as_posix())
media_type = relative_parts[part_index]
part_index += 1
if (
@@ -2156,8 +2247,32 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
and dir_info.download_category_folder
and len(relative_parts) > part_index
):
current_root = current_root / relative_parts[part_index]
shared_roots.add(current_root.as_posix())
category_root = current_root / relative_parts[part_index]
shared_roots.add(category_root.as_posix())
if media_categories is None:
media_categories = MediaChain().media_category() or {}
if media_type:
category_names = media_categories.get(media_type, [])
else:
category_names = {
category
for categories in media_categories.values()
for category in categories
}
category_paths = sorted(
(Path(category).parts for category in category_names if category),
key=len,
)
for category_parts in category_paths:
relative_category_parts = tuple(
relative_parts[part_index:part_index + len(category_parts)]
)
if relative_category_parts != category_parts:
continue
category_root = current_root
for category_part in category_parts:
category_root = category_root / category_part
shared_roots.add(category_root.as_posix())
return shared_roots
@@ -2266,6 +2381,31 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
return None
@staticmethod
def _is_movie_year_conflict(
file_meta: MetaBase, media: Union[DownloadHistory, MediaInfo]
) -> bool:
"""
判断文件名年份是否与已识别电影年份冲突
多电影合集只保存一条下载历史不能把合集首部电影的媒体 ID 套用到其它年份的文件
电视剧季包仍应继续复用同一条下载历史
"""
file_year = getattr(file_meta, "year", None)
media_year = getattr(media, "year", None)
if not file_meta or not media or not file_year or not media_year:
return False
media_type = getattr(media, "type", None)
if not isinstance(media_type, MediaType):
try:
media_type = MediaType(media_type)
except (TypeError, ValueError):
return False
return (
media_type == MediaType.MOVIE
and str(file_year) != str(media_year)
)
@staticmethod
def __optional_attr_equal(
source: MetaBase,
@@ -2401,6 +2541,32 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
self.__normalize_dir_path(Path(current_item.path).parent),
)
@staticmethod
def _get_subscribe_custom_words(
history_record: Optional[DownloadHistory],
) -> Optional[List[str]]:
"""
获取整理用自定义识别词优先使用下载时保存的快照无快照历史旧记录时再按来源实时反查订阅
快照优先可避免整理阶段因订阅季号漂移来源解析失败或订阅完成被删导致识别词丢失从而原样入库到偏移前的季集
"""
if not history_record:
return None
# 下载时保存的完整订阅识别词快照优先
if history_record.custom_words:
return history_record.custom_words.split("\n")
# 兜底:历史旧记录无快照时,按下载来源实时反查订阅
if not isinstance(history_record.note, dict):
return None
subscribe = SubscribeChain().get_subscribe_by_source(
history_record.note.get("source")
)
return (
subscribe.custom_words.split("\n")
if subscribe and subscribe.custom_words
else None
)
def do_transfer(
self,
fileitem: FileItem,
@@ -2423,6 +2589,7 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
manual: Optional[bool] = False,
preview: Optional[bool] = False,
sync_extra_files: Optional[bool] = False,
cleanup_dest_fileitem: Optional[FileItem] = None,
continue_callback: Callable = None,
) -> Tuple[bool, Union[str, dict]]:
"""
@@ -2447,6 +2614,7 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
:param manual: 是否手动整理
:param preview: 是否仅预览
:param sync_extra_files: 是否在整理主视频文件时同步整理同媒体附加文件
:param cleanup_dest_fileitem: 确认存在待整理任务后需要清理的旧目标文件
:param continue_callback: 继续处理回调
返回成功标识错误信息
"""
@@ -2456,9 +2624,8 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
if preview:
# 预览模式始终同步执行,避免进入异步队列
background = False
manual_single_file = bool(manual and fileitem and fileitem.type == "file")
# 自定义格式
has_episode_format_template = bool(epformat and epformat.format)
formaterHandler = (
FormatParser(
eformat=epformat.format,
@@ -2476,24 +2643,7 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
)
# 汇总错误信息
err_msgs: List[str] = []
def _get_subscribe_custom_words(
history_record: Optional[DownloadHistory],
) -> Optional[List[str]]:
"""
根据下载记录获取订阅自定义识别词
"""
if not history_record or not isinstance(history_record.note, dict):
return None
# 使用source动态获取订阅
subscribe = SubscribeChain().get_subscribe_by_source(
history_record.note.get("source")
)
return (
subscribe.custom_words.split("\n")
if subscribe and subscribe.custom_words
else None
)
matched_episode_format_template = False
def _build_file_meta(
source_path: Path,
@@ -2552,29 +2702,20 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
return current_meta
def _filter(item: FileItem, is_bluray_dir: bool) -> bool:
def _is_allowed_transfer_item(item: FileItem, is_bluray_dir: bool) -> bool:
"""
过滤文件项
判断候选文件项是否允许进入整理规划
:return: True 表示保留False 表示排除
"""
nonlocal matched_episode_format_template
if continue_callback and not continue_callback():
raise OperationInterrupted()
is_extra_file = self.__is_subtitle_file(item) or self.__is_audio_file(item)
# 手动单文件整理时,前端可能把同目录文件拆成多个根文件提交;
# 此时应优先信任用户显式选择的根文件,并允许附加文件进入后续同媒体匹配流程,
# 避免仅因模板未覆盖字幕/音轨后缀而被提前过滤。
should_bypass_epformat_match = (
(manual_single_file and item.path == fileitem.path)
or (sync_extra_files and is_extra_file)
)
# 有集自定义格式,过滤文件
if (
formaterHandler
and not should_bypass_epformat_match
and not formaterHandler.match(item.name)
):
return False
# 存在集数定位模板时,模板匹配结果作为手动整理的硬过滤条件。
if has_episode_format_template and formaterHandler:
if not formaterHandler.match(item.name):
return False
matched_episode_format_template = True
# 过滤后缀和大小(蓝光目录、附加文件不过滤)
if (
not is_bluray_dir
@@ -2601,6 +2742,32 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
return False
return True
def _keep_candidate_item(item: FileItem, is_bluray_dir: bool) -> bool:
"""
收集候选文件时仅检查中断状态不套用整理业务过滤
"""
if continue_callback and not continue_callback():
raise OperationInterrupted()
return True
def _collect_candidate_file_items() -> List[Tuple[FileItem, bool]]:
"""
收集来源下的候选文件项不在此阶段套用整理业务过滤
"""
return self.__get_trans_fileitems(fileitem, predicate=_keep_candidate_item)
def _filter_allowed_file_items(
candidates: List[Tuple[FileItem, bool]]
) -> List[Tuple[FileItem, bool]]:
"""
将候选文件项筛选为本轮允许整理的文件项
"""
return [
(candidate_item, candidate_bluray_dir)
for candidate_item, candidate_bluray_dir in candidates
if _is_allowed_transfer_item(candidate_item, candidate_bluray_dir)
]
def _build_main_meta(
main_fileitem: FileItem,
main_bluray_dir: bool,
@@ -2618,7 +2785,7 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
)
return _build_file_meta(
main_path,
custom_word_list=_get_subscribe_custom_words(main_download_history),
custom_word_list=self._get_subscribe_custom_words(main_download_history),
)
def _append_item(
@@ -2682,7 +2849,7 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
continue
if not (self.__is_subtitle_file(item) or self.__is_audio_file(item)):
continue
if not _filter(item, False):
if not _is_allowed_transfer_item(item, False):
continue
extra_items.append((item, False))
return main_fileitems, extra_items
@@ -2776,7 +2943,7 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
bluray_dir=main_bluray_dir,
download_hash=download_hash,
)
subscribe_custom_words = _get_subscribe_custom_words(
subscribe_custom_words = self._get_subscribe_custom_words(
main_download_history
)
main_meta = _build_file_meta(
@@ -2829,19 +2996,36 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
return planned_items, inherited_map
candidate_file_items: List[Tuple[FileItem, bool]] = []
try:
# 获取经过筛选后的待整理文件项列表
file_items = self.__get_trans_fileitems(fileitem, predicate=_filter)
candidate_file_items = _collect_candidate_file_items()
file_items = _filter_allowed_file_items(candidate_file_items)
except OperationInterrupted:
return False, f"{fileitem.name} 已取消"
finally:
candidate_file_items.clear()
if not file_items:
if has_episode_format_template and not matched_episode_format_template:
logger.info(f"{fileitem.path} 未匹配到集数定位模板,跳过整理")
if preview:
return True, {
"summary": {"total": 0, "success": 0, "failed": 0},
"items": [],
"message": "",
}
return True, ""
logger.warn(f"{fileitem.path} 没有找到可整理的媒体文件")
return False, f"{fileitem.name} 没有找到可整理的媒体文件"
file_items, inherited_meta_map = _plan_file_items(file_items)
planned_file_count = len(file_items)
if cleanup_dest_fileitem and planned_file_count and not preview:
state = StorageChain().delete_media_file(cleanup_dest_fileitem)
if not state:
return False, f"{cleanup_dest_fileitem.path} 删除失败"
if preview:
logger.info(f"正在预览 {planned_file_count} 个文件的整理路径...")
else:
@@ -2899,7 +3083,7 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
else:
file_meta = _build_file_meta(
file_path,
custom_word_list=_get_subscribe_custom_words(download_history),
custom_word_list=self._get_subscribe_custom_words(download_history),
)
else:
file_meta = _build_file_meta(file_path)
@@ -2918,11 +3102,19 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
_downloader = downloader
_download_hash = download_hash
# 自动整理预载的媒体信息来自整条下载历史;电影合集内文件年份冲突时逐文件识别。
task_mediainfo = mediainfo
if (
not manual
and self._is_movie_year_conflict(file_meta, task_mediainfo)
):
task_mediainfo = None
# 后台整理
transfer_task = TransferTask(
fileitem=file_item,
meta=file_meta,
mediainfo=mediainfo,
mediainfo=task_mediainfo,
target_directory=target_directory,
target_storage=target_storage,
target_path=target_path,
@@ -3297,6 +3489,7 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
download_hash: Optional[str] = None,
preview: Optional[bool] = False,
sync_extra_files: Optional[bool] = True,
cleanup_dest_fileitem: Optional[FileItem] = None,
) -> Tuple[bool, Union[str, dict]]:
"""
手动整理支持复杂条件带进度显示
@@ -3320,6 +3513,7 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
:param download_hash: 下载任务哈希
:param preview: 是否仅预览
:param sync_extra_files: 是否同步整理同媒体附加文件
:param cleanup_dest_fileitem: 确认存在待整理任务后需要清理的旧目标文件
"""
logger.info(f"手动整理:{fileitem.path} ...")
if tmdbid or doubanid:
@@ -3360,6 +3554,7 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
download_hash=download_hash,
preview=preview,
sync_extra_files=sync_extra_files,
cleanup_dest_fileitem=cleanup_dest_fileitem,
)
if not state:
return False, errmsg
@@ -3386,6 +3581,7 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
download_hash=download_hash,
preview=preview,
sync_extra_files=sync_extra_files,
cleanup_dest_fileitem=cleanup_dest_fileitem,
)
return state, errmsg
@@ -3395,10 +3591,17 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
mediainfo: MediaInfo,
transferinfo: TransferInfo,
season_episode: Optional[str] = None,
episodes_info: Optional[List[TmdbEpisode]] = None,
username: Optional[str] = None,
):
"""
发送入库成功的消息
:param meta: 文件元数据
:param mediainfo: 识别的媒体信息
:param transferinfo: 文件整理信息
:param season_episode: 已入库季集文本
:param episodes_info: 当前季的全部集信息
:param username: 用户名
"""
self.post_message(
Notification(
@@ -3412,6 +3615,7 @@ class TransferChain(ChainBase, ConfigReloadMixin, metaclass=Singleton):
mediainfo=mediainfo,
transferinfo=transferinfo,
season_episode=season_episode,
episodes_info=episodes_info,
username=username,
)
+31 -1
View File
@@ -1155,11 +1155,16 @@ class WorkflowChain(ChainBase):
self.process(workflow_id, from_begin=False)
@staticmethod
def process(workflow_id: int, from_begin: Optional[bool] = True) -> Tuple[bool, str]:
def process(
workflow_id: int,
from_begin: Optional[bool] = True,
progress_callback: Optional[Callable[..., None]] = None,
) -> Tuple[bool, str]:
"""
处理工作流
:param workflow_id: 工作流ID
:param from_begin: 是否从头开始默认为True
:param progress_callback: 定时服务进度更新回调
"""
workflowoper = WorkflowOper()
@@ -1173,6 +1178,23 @@ class WorkflowChain(ChainBase):
context=_serialize_workflow_context(context),
execution_state=_serialize_workflow_value(execution_state)
)
if progress_callback:
runtime = execution_state.get("runtime") or {}
finished_actions = int(runtime.get("finished_actions") or 0)
total_actions = len(workflow.actions)
progress_callback(
value=finished_actions / total_actions * 100,
text=(
f"工作流动作({finished_actions}/{total_actions}"
f"{action.name or action.type or action.id} "
f"{'执行完成' if completed else '执行中'}"
),
data={
"total": total_actions,
"finished": finished_actions,
"current": action.id,
},
)
# 重置工作流
if from_begin:
@@ -1191,6 +1213,12 @@ class WorkflowChain(ChainBase):
return False, "工作流无流程"
logger.info(f"开始执行工作流 {workflow.name},共 {len(workflow.actions)} 个动作 ...")
if progress_callback:
progress_callback(
value=0,
text=f"开始执行工作流 {workflow.name} ...",
data={"total": len(workflow.actions), "finished": 0},
)
workflowoper.start(workflow_id)
# 执行工作流
@@ -1207,6 +1235,8 @@ class WorkflowChain(ChainBase):
return False, executor.errmsg
logger.info(f"工作流 {workflow.name} 执行完成")
workflowoper.success(workflow_id)
if progress_callback:
progress_callback(value=100, text=f"工作流 {workflow.name} 执行完成")
return True, ""
@staticmethod
+1 -1
View File
@@ -988,7 +988,7 @@ def logs(lines: int, follow: bool, stdio: bool, frontend_log: bool) -> None:
@click.option("--fix", is_flag=True, help="执行白名单安全修复")
@click.option("--deep", is_flag=True, help="执行可能较慢的深度检查")
def doctor(json_output: bool, fix: bool, deep: bool) -> None:
"""离线诊断本地 MoviePilot 运行环境"""
"""离线诊断本地 MoviePilot 运行环境,插件日志告警不影响整体状态"""
from app.doctor import run_doctor
from app.doctor.formatters import format_json_report, format_text_report
+110 -38
View File
@@ -13,7 +13,7 @@ import aiofiles
import aioshutil
from anyio import Path as AsyncPath
from cachetools import LRUCache as MemoryLRUCache
from cachetools import TTLCache as MemoryTTLCache
from cachetools import TLRUCache as MemoryTLRUCache
from cachetools.keys import hashkey
from app.core.config import settings
@@ -357,15 +357,52 @@ class AsyncCacheBackend(CacheBackend):
pass
class _MemoryTLRUCache(MemoryTLRUCache):
"""
支持为每个 key 设置独立 TTL 的内存缓存
"""
def __init__(self, maxsize: int, ttl: int):
self.__ttl = ttl
self.__setting_ttls: Dict[str, int] = {}
super().__init__(maxsize=maxsize, ttu=self._get_expiration)
def _get_expiration(self, key: str, _value: Any, now: float) -> float:
return now + self.__setting_ttls.get(key, self.__ttl)
@property
def ttl(self) -> int:
"""
默认缓存存活时间单位秒
"""
return self.__ttl
def set(self, key: str, value: Any, ttl: int) -> None:
"""
使用指定 TTL 设置缓存值
"""
if ttl <= 0:
try:
del self[key]
except KeyError:
pass
return
self.__setting_ttls[key] = ttl
try:
super().__setitem__(key, value)
finally:
self.__setting_ttls.pop(key, None)
class MemoryBackend(CacheBackend):
"""
基于 `cachetools.TTLCache` 实现的缓存后端
基于 `cachetools.TLRUCache` 实现的缓存后端
"""
# 类变量 _region_caches 的互斥锁
_lock = threading.Lock()
# 存储各个 region 的缓存实例,region -> TTLCache
_region_caches: Dict[str, Union[MemoryTTLCache, MemoryLRUCache]] = {}
# 存储各个 region 的缓存实例,region -> TLRUCache/LRUCache
_region_caches: Dict[str, Union[_MemoryTLRUCache, MemoryLRUCache]] = {}
def __init__(self, cache_type: Literal['ttl', 'lru'] = 'ttl',
maxsize: Optional[int] = None, ttl: Optional[int] = None):
@@ -378,9 +415,9 @@ class MemoryBackend(CacheBackend):
"""
self.cache_type = cache_type
self.maxsize = maxsize or DEFAULT_CACHE_SIZE
self.ttl = ttl or DEFAULT_CACHE_TTL
self.ttl = DEFAULT_CACHE_TTL if ttl is None else ttl
def __get_region_cache(self, region: str) -> Optional[Union[MemoryTTLCache, MemoryLRUCache]]:
def __get_region_cache(self, region: str) -> Optional[Union[_MemoryTLRUCache, MemoryLRUCache]]:
"""
获取指定区域的缓存实例如果不存在则返回 None
"""
@@ -394,21 +431,29 @@ class MemoryBackend(CacheBackend):
:param key: 缓存的键
:param value: 缓存的值
:param ttl: 缓存的存活时间传入为永久缓存单位秒
:param ttl: 缓存的存活时间传入则使用 backend 默认值单位秒
:param region: 缓存的区
"""
ttl = ttl or self.ttl
maxsize = kwargs.get("maxsize", self.maxsize)
ttl = self.ttl if ttl is None else ttl
maxsize = kwargs.get("maxsize") or self.maxsize
region = self.get_region(region)
# 设置缓存值
with self._lock:
# 如果该 key 尚未有缓存实例,则创建一个新的 TTLCache 实例
region_cache = self._region_caches.setdefault(
region,
MemoryTTLCache(maxsize=maxsize, ttl=ttl) if self.cache_type == 'ttl'
else MemoryLRUCache(maxsize=maxsize)
)
region_cache[key] = value
region_cache = self._region_caches.get(region)
if region_cache is None:
region_cache = (
_MemoryTLRUCache(maxsize=maxsize, ttl=ttl) if self.cache_type == 'ttl'
else MemoryLRUCache(maxsize=maxsize)
)
self._region_caches[region] = region_cache
elif isinstance(region_cache, _MemoryTLRUCache) != (self.cache_type == 'ttl'):
raise ValueError(
f"Cache region {region!r} already uses a different cache type"
)
if isinstance(region_cache, _MemoryTLRUCache):
region_cache.set(key, value, ttl=ttl)
else:
region_cache[key] = value
def exists(self, key: str, region: Optional[str] = DEFAULT_CACHE_REGION) -> bool:
"""
@@ -458,19 +503,18 @@ class MemoryBackend(CacheBackend):
:param region: 缓存的区为None时清空所有区缓存
"""
if region:
# 清理指定缓存区
region_cache = self.__get_region_cache(region)
if region_cache:
with self._lock:
with self._lock:
if region:
# 清理指定缓存区
region_cache = self.__get_region_cache(region)
if region_cache is not None:
region_cache.clear()
logger.debug(f"Cleared cache for region: {region}")
else:
# 清除所有区域的缓存
for region_cache in self._region_caches.values():
with self._lock:
logger.debug(f"Cleared cache for region: {region}")
else:
# 清除所有区域的缓存
for region_cache in self._region_caches.values():
region_cache.clear()
logger.info("Cleared all cache")
logger.info("Cleared all cache")
def items(self, region: Optional[str] = DEFAULT_CACHE_REGION) -> Generator[Tuple[str, Any], None, None]:
"""
@@ -520,7 +564,7 @@ class AsyncMemoryBackend(AsyncCacheBackend):
:param key: 缓存的键
:param value: 缓存的值
:param ttl: 缓存的存活时间传入为永久缓存单位秒
:param ttl: 缓存的存活时间传入则使用 backend 默认值单位秒
:param region: 缓存的区
"""
return self._backend.set(key=key, value=value, ttl=ttl, region=region, **kwargs)
@@ -600,11 +644,14 @@ class RedisBackend(CacheBackend):
:param key: 缓存的键
:param value: 缓存的值
:param ttl: 缓存的存活时间未传入则为永久缓存单位秒
:param ttl: 缓存的存活时间未传入则使用 backend 默认值单位秒
:param region: 缓存的区
:param kwargs: kwargs
"""
ttl = ttl or self.ttl
ttl = self.ttl if ttl is None else ttl
if ttl is not None and ttl <= 0:
self.redis_helper.delete(key, region=region)
return
self.redis_helper.set(key, value, ttl=ttl, region=region, **kwargs)
def exists(self, key: str, region: Optional[str] = DEFAULT_CACHE_REGION) -> bool:
@@ -681,11 +728,14 @@ class AsyncRedisBackend(AsyncCacheBackend):
:param key: 缓存的键
:param value: 缓存的值
:param ttl: 缓存的存活时间未传入则为永久缓存单位秒
:param ttl: 缓存的存活时间未传入则使用 backend 默认值单位秒
:param region: 缓存的区
:param kwargs: kwargs
"""
ttl = ttl or self.ttl
ttl = self.ttl if ttl is None else ttl
if ttl is not None and ttl <= 0:
await self.redis_helper.delete(key, region=region)
return
await self.redis_helper.set(key, value, ttl=ttl, region=region, **kwargs)
async def exists(self, key: str, region: Optional[str] = DEFAULT_CACHE_REGION) -> bool:
@@ -1018,7 +1068,7 @@ def FileCache(base: Path = settings.TEMP_PATH, ttl: Optional[int] = None) -> Cac
"""
if settings.CACHE_BACKEND_TYPE == "redis":
# 如果使用 Redis,则设置缓存的存活时间为配置的天数转换为秒
return RedisBackend(ttl=ttl or settings.TEMP_FILE_DAYS * 24 * 3600)
return RedisBackend(ttl=ttl if ttl is not None else settings.TEMP_FILE_DAYS * 24 * 3600)
else:
# 如果使用文件系统,在停止服务时会自动清理过期文件
return FileBackend(base=base)
@@ -1030,7 +1080,7 @@ def AsyncFileCache(base: Path = settings.TEMP_PATH, ttl: Optional[int] = None) -
"""
if settings.CACHE_BACKEND_TYPE == "redis":
# 如果使用 Redis,则设置缓存的存活时间为配置的天数转换为秒
return AsyncRedisBackend(ttl=ttl or settings.TEMP_FILE_DAYS * 24 * 3600)
return AsyncRedisBackend(ttl=ttl if ttl is not None else settings.TEMP_FILE_DAYS * 24 * 3600)
else:
# 如果使用文件系统,在停止服务时会自动清理过期文件
return AsyncFileBackend(base=base)
@@ -1075,11 +1125,11 @@ def AsyncCache(cache_type: Literal['ttl', 'lru'] = 'ttl',
def cached(region: Optional[str] = None, maxsize: Optional[int] = 1024, ttl: Optional[int] = None,
skip_none: Optional[bool] = True, skip_empty: Optional[bool] = False, shared_key: Optional[str] = None):
"""
自定义缓存装饰器支持为每个 key 动态传递 maxsize ttl
自定义缓存装饰器支持配置缓存区域的 maxsize 和每个 key ttl
:param region: 缓存区域的标识符默认根据模块名函数名等自动生成标识
:param maxsize: 缓存区内的最大条目数
:param ttl: 缓存的存活时间单位秒未传入则为永久缓存单位秒
:param ttl: 缓存的存活时间单位秒未传入时使用 LRU 缓存
:param skip_none: 跳过 None 缓存默认为 True
:param skip_empty: 跳过空值缓存 None, [], {}, "", set()默认为 False
:param shared_key: 同步/异步函数共享缓存的键默认使用函数名异步函数名会标准化为同步格式如移除 `async_` 前缀
@@ -1186,7 +1236,7 @@ def cached(region: Optional[str] = None, maxsize: Optional[int] = 1024, ttl: Opt
if is_async:
# 异步函数使用异步缓存后端
cache_backend = AsyncCache(cache_type="ttl" if ttl else "lru", maxsize=maxsize, ttl=ttl)
cache_backend = AsyncCache(cache_type="ttl" if ttl is not None else "lru", maxsize=maxsize, ttl=ttl)
# 异步函数的缓存装饰器
@wraps(func)
async def async_wrapper(*args, **kwargs):
@@ -1214,12 +1264,23 @@ def cached(region: Optional[str] = None, maxsize: Optional[int] = 1024, ttl: Opt
"""
await cache_backend.clear(region=cache_region)
async def cache_exists(*args, **kwargs) -> bool:
"""
判断当前参数对应的有效缓存是否存在
"""
cache_key = __get_cache_key(args, kwargs)
cached_value = await cache_backend.get(cache_key, region=cache_region)
return should_cache(cached_value) and await async_is_valid_cache_value(
cache_key, cached_value, cache_region
)
async_wrapper.cache_region = cache_region
async_wrapper.cache_clear = cache_clear
async_wrapper.cache_exists = cache_exists
return async_wrapper
else:
# 同步函数使用同步缓存后端
cache_backend = Cache(cache_type="ttl" if ttl else "lru", maxsize=maxsize, ttl=ttl)
cache_backend = Cache(cache_type="ttl" if ttl is not None else "lru", maxsize=maxsize, ttl=ttl)
# 同步函数的缓存装饰器
@wraps(func)
def wrapper(*args, **kwargs):
@@ -1246,8 +1307,19 @@ def cached(region: Optional[str] = None, maxsize: Optional[int] = 1024, ttl: Opt
"""
cache_backend.clear(region=cache_region)
def cache_exists(*args, **kwargs) -> bool:
"""
判断当前参数对应的有效缓存是否存在
"""
cache_key = __get_cache_key(args, kwargs)
cached_value = cache_backend.get(cache_key, region=cache_region)
return should_cache(cached_value) and is_valid_cache_value(
cache_key, cached_value, cache_region
)
wrapper.cache_region = cache_region
wrapper.cache_clear = cache_clear
wrapper.cache_exists = cache_exists
return wrapper
return decorator
+2 -6
View File
@@ -377,6 +377,8 @@ class ConfigModel(BaseModel):
COOKIECLOUD_KEY: Optional[str] = None
# CookieCloud端对端加密密码
COOKIECLOUD_PASSWORD: Optional[str] = None
# CookieCloud本地上传接口的X-CookieCloud-Auth期望值,留空表示不校验
COOKIECLOUD_AUTH_HEADER: Optional[str] = None
# CookieCloud同步间隔(分钟)
COOKIECLOUD_INTERVAL: Optional[int] = 60 * 24
# CookieCloud同步黑名单,多个域名,分割
@@ -1209,12 +1211,6 @@ class GlobalVar(object):
"""
self.STOP_EVENT.set()
def resume_system(self):
"""
恢复系统运行标记
"""
self.STOP_EVENT.clear()
@property
def is_system_stopped(self):
"""
+28 -2
View File
@@ -9,9 +9,15 @@ from app.core.metainfo import MetaInfo
from app.schemas.types import MediaType
from app.utils.string import StringUtils
BANGUMI_MOVIE_PLATFORMS = frozenset({"movie", "电影", "剧场版"})
@dataclass
class TorrentInfo:
"""
种子搜索结果信息
"""
# 站点ID
site: int = None
# 站点名称
@@ -239,6 +245,10 @@ class SubtitleInfo:
@dataclass
class MediaInfo:
"""
统一媒体信息负责聚合各元数据源的标准字段
"""
# 内部标记:是否命中本地识别缓存,不参与序列化
recognize_cache_hit = False
# 来源:themoviedb、douban、bangumi
@@ -717,7 +727,20 @@ class MediaInfo:
elif type(current_value) is type(value):
setattr(self, key, value)
def set_bangumi_info(self, info: dict):
@staticmethod
def get_bangumi_media_type(info: dict) -> MediaType:
"""
根据Bangumi媒介平台获取标准媒体类型未知平台兼容回退为电视剧
:param info: Bangumi条目信息
:return: 标准媒体类型
"""
platform = str(info.get("platform") or "").strip().casefold()
if platform in BANGUMI_MOVIE_PLATFORMS:
return MediaType.MOVIE
return MediaType.TV
def set_bangumi_info(self, info: dict) -> None:
"""
初始化Bangumi信息
"""
@@ -731,7 +754,7 @@ class MediaInfo:
self.bangumi_id = info.get("id")
# 类型
if not self.type:
self.type = MediaType.TV
self.type = self.get_bangumi_media_type(info)
# 标题
if not self.title:
self.title = info.get("name_cn") or info.get("name")
@@ -919,6 +942,8 @@ class Context:
media_info_is_target: bool = False
# 调用方对本候选允许下载的剧集集合,None 表示不限制,空集合表示拒绝交付任何集。
allowed_episodes: Optional[Set[int]] = None
# 下载层确认候选资源覆盖完整目标范围,供订阅事实写入判断整包资源。
confirmed_full_coverage: bool = False
def to_dict(self):
"""
@@ -935,4 +960,5 @@ class Context:
"media_info_is_target": self.media_info_is_target,
# 保留 None / 空集 / 非空集 三态语义,避免下游误把"显式拒绝"当成"不限制"。
"allowed_episodes": sorted(self.allowed_episodes) if self.allowed_episodes is not None else None,
"confirmed_full_coverage": self.confirmed_full_coverage,
}
+36
View File
@@ -439,11 +439,19 @@ class EventManager(metaclass=Singleton):
if not handlers:
logger.debug(f"No handlers found for broadcast event: {event}")
return
target_plugin_id = None
if event.event_type == EventType.MessageAction and isinstance(event.event_data, dict):
target_plugin_id = event.event_data.get("__mp_target_plugin_id")
# 为每个处理器提供独立的事件实例,防止某个处理器对 event_data 的修改影响其他处理器
for handler_id, handler in handlers.items():
if target_plugin_id and not self.__should_dispatch_to_target_plugin(
handler, handler_id, str(target_plugin_id)
):
continue
# 仅浅拷贝顶层字典,避免不必要的深拷贝开销;这样可以隔离键级别的替换/赋值
if isinstance(event.event_data, dict):
event_data_copy = event.event_data.copy()
event_data_copy.pop("__mp_target_plugin_id", None)
else:
event_data_copy = event.event_data
isolated_event = Event(event_type=event.event_type,
@@ -459,6 +467,34 @@ class EventManager(metaclass=Singleton):
# 对于同步函数,在线程池中运行
self.__executor.submit(self.__safe_invoke_handler, handler, isolated_event)
@classmethod
def __should_dispatch_to_target_plugin(
cls,
handler: Callable,
handler_identifier: str,
target_plugin_id: str,
) -> bool:
"""
限定插件输入事件只投递给目标插件避免自由文本被其他插件观察到
"""
class_name, method_name = cls.__parse_handler_names(handler)
if class_name != target_plugin_id:
return False
identifier_parts = (handler_identifier or "").split(".")
if len(identifier_parts) < 2:
logger.debug(
"Target plugin dispatch skipped because handler identifier is invalid: "
f"target={target_plugin_id}, handler={handler_identifier}"
)
return False
if identifier_parts[-2:] != [class_name, method_name]:
logger.debug(
"Target plugin dispatch skipped because handler identifier does not match handler: "
f"target={target_plugin_id}, handler={handler_identifier}, parsed={class_name}.{method_name}"
)
return False
return True
def __safe_invoke_handler(self, handler: Callable, event: Event):
"""
调用处理器处理链式或广播事件
+9 -2
View File
@@ -17,7 +17,7 @@ class CustomizationMatcher(metaclass=Singleton):
self._customization_re_cache = {}
@staticmethod
def _normalize_customization(customization):
def normalize_customization(customization):
"""
规范化自定义占位符配置兼容历史字符串与列表两种保存格式
"""
@@ -27,6 +27,13 @@ class CustomizationMatcher(metaclass=Singleton):
return []
return list(filter(None, customization))
@staticmethod
def _normalize_customization(customization):
"""
兼容旧调用统一转到公开的自定义占位符规范化入口
"""
return CustomizationMatcher.normalize_customization(customization)
def match(self, title=None):
"""
:param title: 资源标题或文件名
@@ -35,7 +42,7 @@ class CustomizationMatcher(metaclass=Singleton):
if not title:
return ""
# 自定义占位符需要跟随系统配置实时生效,避免单例缓存导致保存后仍沿用旧规则。
customization = self._normalize_customization(
customization = self.normalize_customization(
self.systemconfig.get(SystemConfigKey.Customization)
)
if not customization:
+19 -14
View File
@@ -38,6 +38,14 @@ class MetaAnime(MetaBase):
_name_nostring_pattern = re.compile(_name_nostring_re, re.IGNORECASE)
_fps_pattern = re.compile(r"(%s)" % _fps_re, re.IGNORECASE)
@staticmethod
def _parse_season_number(value):
"""解析第三方动漫季号,仅接受整数或纯数字字符串并保留数值 0。"""
if value is None:
return None
text = str(value).strip()
return int(text) if text.isdigit() else None
def __init__(self, title: str, subtitle: str = None, isfile: bool = False):
super().__init__(title, subtitle, isfile)
if not title:
@@ -111,22 +119,19 @@ class MetaAnime(MetaBase):
# 季号
anime_season = anitopy_info.get("anime_season")
if isinstance(anime_season, list):
if len(anime_season) == 1:
begin_season = anime_season[0]
end_season = None
else:
begin_season = anime_season[0]
end_season = anime_season[-1]
elif anime_season:
begin_season = anime_season
end_season = None
seasons = [
season for item in anime_season
if (season := self._parse_season_number(item)) is not None
]
begin_season = seasons[0] if seasons else None
end_season = seasons[-1] if len(seasons) > 1 else None
else:
begin_season = None
begin_season = self._parse_season_number(anime_season)
end_season = None
if begin_season:
self.begin_season = int(begin_season)
if end_season and int(end_season) != self.begin_season:
self.end_season = int(end_season)
if begin_season is not None:
self.begin_season = begin_season
if end_season is not None and end_season != self.begin_season:
self.end_season = end_season
self.total_season = (self.end_season - self.begin_season) + 1
else:
self.total_season = 1
+37
View File
@@ -24,6 +24,13 @@ SUBTITLE_EPISODE_ALL_RE = re.compile(
r"([0-9一二三四五六七八九十百零]+)\s*集\s*全|[全共]\s*([0-9一二三四五六七八九十百零]+)\s*[集话話期幕]",
re.IGNORECASE,
)
# 结尾分支显式区分有无右方括号,避免可选括号回溯后绕过数字后缀边界
SUBTITLE_EPISODE_RANGE_FIN_RE = re.compile(
r"(?<!\d)\[?\s*(\d{1,4})\s*-\s*(\d{1,4})\s*"
r"(?:(?:Fin|End)(?![a-z0-9])|完结(?![\u4e00-\u9fff]))"
r"(?:\s*\](?!\d)|(?!\s*(?:\]\d|\d))\s*)",
re.IGNORECASE,
)
VIDEO_BIT_RE = re.compile(
r"(?<![A-Za-z0-9])(?P<bit>8|10|12|16)[\s._-]*bits?(?![A-Za-z0-9])",
re.IGNORECASE,
@@ -292,6 +299,36 @@ class MetaBase(object):
self.type = MediaType.TV
self._subtitle_flag = True
return
# 01-26Fin 等数字范围+完结标记
self.__init_episode_range_fin(title_text)
else:
# 副标题无中文季集标记时,仍识别 01-26Fin 等数字范围+完结标记
self.__init_episode_range_fin(title_text)
def __init_episode_range_fin(self, title_text: str):
"""
识别 01-26Fin / [01-38 END] "数字范围+完结标记"格式的集数信息
"""
episode_range_str = SUBTITLE_EPISODE_RANGE_FIN_RE.search(title_text)
if not episode_range_str:
return
try:
begin_episode = int(episode_range_str.group(1))
end_episode = int(episode_range_str.group(2))
except Exception as err:
logger.debug(f'识别集失败:{str(err)} - {traceback.format_exc()}')
return
if begin_episode < 1 or begin_episode > end_episode or end_episode >= 10000:
return
# 两个数字都落在常见年份区间时视为年份范围而非集数(如 2019-2020完结)
if begin_episode >= 1900 and end_episode <= 2155:
return
if self.begin_episode is None:
self.begin_episode = begin_episode
self.end_episode = end_episode
self.total_episode = end_episode
self.type = MediaType.TV
self._subtitle_flag = True
@property
def season(self) -> str:
+7 -7
View File
@@ -56,7 +56,7 @@ class MetaVideo(MetaBase):
_name_se_words = ['', '', '', '', '', '', '']
_name_movie_words = ['剧场版', '劇場版', '电影版', '電影版']
_name_nostring_re = r"^PTS|^JADE|^AOD|^CHC|^[A-Z]{1,4}TV[\-0-9UVHDK]*" \
r"|\d{1,2}th|\d{1,2}bit|IMAX|^3D|\s+3D|XXX|\s+DC$" \
r"|\d{1,2}th|\d{1,2}bit|IMAX|^3D|\s+3D|\s+DC$" \
r"|[第\s共]+[0-9一二三四五六七八九十\-\s]+季" \
r"|[第\s共]+[0-9一二三四五六七八九十百零\-\s]+[集话話]" \
r"|连载|日剧|美剧|电视剧|动画片|动漫|欧美|西德|日韩|超高清|高清|无水印|下载|蓝光|翡翠台|梦幻天堂·龙网|★?\d*月?新番" \
@@ -251,7 +251,7 @@ class MetaVideo(MetaBase):
if name.isdecimal() \
and int(name) < 1800 \
and not self.year \
and not self.begin_season \
and self.begin_season is None \
and not self.resource_pix \
and not self.resource_type \
and not self.audio_encode \
@@ -259,7 +259,7 @@ class MetaVideo(MetaBase):
if self.begin_episode is None:
self.begin_episode = int(name)
name = None
elif self.is_in_episode(int(name)) and not self.begin_season:
elif self.is_in_episode(int(name)) and self.begin_season is None:
name = None
return name
@@ -366,7 +366,7 @@ class MetaVideo(MetaBase):
if not self.name:
return
if not self.year \
and not self.begin_season \
and self.begin_season is None \
and not self.begin_episode \
and not self.resource_pix \
and not self.resource_type:
@@ -690,7 +690,7 @@ class MetaVideo(MetaBase):
if not self.year \
and not self.resource_pix \
and not self.resource_type \
and not self.begin_season \
and self.begin_season is None \
and not self.begin_episode:
return
re_res = self._video_encode_pattern.search(token)
@@ -738,7 +738,7 @@ class MetaVideo(MetaBase):
if not self.year \
and not self.resource_pix \
and not self.resource_type \
and not self.begin_season \
and self.begin_season is None \
and not self.begin_episode:
return
video_bit = self.extract_video_bit(token)
@@ -759,7 +759,7 @@ class MetaVideo(MetaBase):
if not self.year \
and not self.resource_pix \
and not self.resource_type \
and not self.begin_season \
and self.begin_season is None \
and not self.begin_episode:
return
re_res = self._audio_encode_pattern.search(token)
+13 -9
View File
@@ -89,6 +89,18 @@ class ReleaseGroupsMatcher(metaclass=Singleton):
self.systemconfig = SystemConfigOper()
self.__groups_re_cache = {}
def get_release_groups(self) -> str:
"""
返回内置与用户自定义制作组组成的匹配规则
"""
custom_release_groups = self.systemconfig.get(SystemConfigKey.CustomReleaseGroups)
if isinstance(custom_release_groups, list):
custom_release_groups = list(filter(None, custom_release_groups))
if custom_release_groups:
custom_release_groups_str = '|'.join(custom_release_groups)
return f"{self.__release_groups}|{custom_release_groups_str}"
return self.__release_groups
def __get_groups_re(self, groups: str):
"""
发布组规则通常很长按规则文本缓存编译结果避免每个标题都重复编译
@@ -108,15 +120,7 @@ class ReleaseGroupsMatcher(metaclass=Singleton):
if not title:
return ""
if not groups:
# 自定义组
custom_release_groups = self.systemconfig.get(SystemConfigKey.CustomReleaseGroups)
if isinstance(custom_release_groups, list):
custom_release_groups = list(filter(None, custom_release_groups))
if custom_release_groups:
custom_release_groups_str = '|'.join(custom_release_groups)
groups = f"{self.__release_groups}|{custom_release_groups_str}"
else:
groups = self.__release_groups
groups = self.get_release_groups()
title = f"{title} "
groups_re = self.__get_groups_re(groups)
unique_groups = []
+6
View File
@@ -297,6 +297,12 @@ class StreamingPlatforms(metaclass=Singleton):
if alias:
self._lookup_cache[alias.upper()] = canonical_name
def get_lookup_cache(self) -> dict:
"""
返回流媒体平台查询表副本供批量解析配置复用
"""
return dict(self._lookup_cache)
def get_streaming_platform_name(self, platform_code: str) -> Optional[str]:
"""
根据流媒体平台简称或全称获取标准名称
+31 -10
View File
@@ -1,3 +1,4 @@
import hashlib
from pathlib import Path
from functools import lru_cache
from typing import Tuple, List, Optional
@@ -40,6 +41,7 @@ _EMBY_TMDB_RE_LIST = (
re.compile(r'\{tmdbid[=\-](\d+)\}'),
re.compile(r'\{tmdb[=\-](\d+)\}'),
)
_RUST_PARSE_OPTIONS_CACHE_KEY = "_cache_key"
def _empty_metainfo() -> dict:
@@ -72,6 +74,28 @@ def _apply_range_total(metainfo: dict, begin_key: str, end_key: str, total_key:
metainfo[total_key] = 1
def _rust_parse_options_cache_key(options: dict) -> str:
"""
生成 Rust Meta 配置缓存键避免扩展层每次重新展开大配置
"""
digest = hashlib.blake2b(digest_size=16)
def update(value) -> None:
digest.update(repr(value).encode("utf-8"))
digest.update(b"\0")
streaming_platforms = options.get("streaming_platforms") or {}
update(tuple(options.get("custom_words") or []))
update(tuple(options.get("media_exts") or []))
update(options.get("release_groups") or "")
update(tuple(options.get("customization") or []))
update(tuple(sorted(
(str(key), str(value))
for key, value in streaming_platforms.items()
)))
return digest.hexdigest()
def _find_metainfo_python(title: str) -> Tuple[str, dict]:
"""
使用 Python 解析标题中的显式媒体标签作为 Rust 入口不可用时的兜底
@@ -209,24 +233,20 @@ def _rust_default_parse_options() -> dict:
from app.schemas.types import SystemConfigKey
systemconfig = SystemConfigOper()
custom_release_groups = systemconfig.get(SystemConfigKey.CustomReleaseGroups)
if isinstance(custom_release_groups, list):
custom_release_groups = list(filter(None, custom_release_groups))
release_matcher = ReleaseGroupsMatcher()
release_groups = release_matcher._ReleaseGroupsMatcher__release_groups
if custom_release_groups:
release_groups = f"{release_groups}|{'|'.join(custom_release_groups)}"
release_groups = ReleaseGroupsMatcher().get_release_groups()
customization = CustomizationMatcher._normalize_customization(
customization = CustomizationMatcher.normalize_customization(
systemconfig.get(SystemConfigKey.Customization)
)
return {
options = {
"custom_words": systemconfig.get(SystemConfigKey.CustomIdentifiers) or [],
"media_exts": settings.RMT_MEDIAEXT + settings.RMT_SUBEXT + settings.RMT_AUDIOEXT,
"release_groups": release_groups,
"customization": customization,
"streaming_platforms": StreamingPlatforms()._lookup_cache,
"streaming_platforms": StreamingPlatforms().get_lookup_cache(),
}
options[_RUST_PARSE_OPTIONS_CACHE_KEY] = _rust_parse_options_cache_key(options)
return options
@lru_cache(maxsize=256)
@@ -236,6 +256,7 @@ def _rust_custom_parse_options(custom_words: Tuple[str, ...]) -> dict:
"""
options = dict(_rust_default_parse_options())
options["custom_words"] = list(custom_words)
options[_RUST_PARSE_OPTIONS_CACHE_KEY] = _rust_parse_options_cache_key(options)
return options
+5 -6
View File
@@ -58,12 +58,11 @@ class ModuleManager(metaclass=Singleton):
"""
logger.info("正在停止所有模块...")
for module_id, module in self._running_modules.items():
if hasattr(module, "stop"):
try:
module.stop()
logger.debug(f"Moudle Stoped{module_id}")
except Exception as err:
logger.error(f"Stop Moudle Error{module_id}{str(err)} - {traceback.format_exc()}", exc_info=True)
try:
module.stop()
logger.debug(f"Moudle Stoped{module_id}")
except Exception as err:
logger.error(f"Stop Moudle Error{module_id}{str(err)} - {traceback.format_exc()}", exc_info=True)
logger.info("所有模块停止完成")
def reload(self):
+163 -42
View File
@@ -363,6 +363,20 @@ class PluginManager(ConfigReloadMixin, metaclass=Singleton):
logger.warn(f"检测到本地插件 {candidate.get('id')} 依赖文件变化,请重新安装本地插件以安装依赖")
continue
federated_change = self._get_federated_plugin_change(event_path)
if federated_change:
pid, candidate, remote_entry_ready = federated_change
# 运行目录由构建方直接写入;外部本地仓库只在入口完整时同步运行副本。
if candidate and remote_entry_ready:
if candidate.get("compatible") is False:
logger.info(
f"检测到本地插件 {pid} 联邦构建产物变化,"
f"但跳过同步:{candidate.get('skip_reason')}"
)
elif pid not in local_plugins_to_sync:
local_plugins_to_sync[pid] = (candidate, event_path, False)
continue
# 跳过非 .py 文件
if not event_path.name.endswith(".py"):
continue
@@ -385,13 +399,14 @@ class PluginManager(ConfigReloadMixin, metaclass=Singleton):
f"文件:{event_path},但跳过同步:{local_candidate.get('skip_reason')}"
)
continue
local_plugins_to_sync[local_candidate.get("id")] = (local_candidate, event_path)
local_plugins_to_sync[local_candidate.get("id")] = (local_candidate, event_path, True)
for pid, (candidate, event_path) in local_plugins_to_sync.items():
for pid, (candidate, event_path, should_reload) in local_plugins_to_sync.items():
package_version = candidate.get("package_version")
source_root = f"plugins.{package_version}" if package_version else "plugins"
logger.info(f"检测到本地插件 {pid} 文件变化,来源:{source_root},文件:{event_path}")
if self._sync_local_plugin_if_installed(pid, candidate):
change_name = "Python 文件" if should_reload else "联邦构建产物"
logger.info(f"检测到本地插件 {pid} {change_name}变化,来源:{source_root},文件:{event_path}")
if self._sync_local_plugin_if_installed(pid, candidate) and should_reload:
plugins_to_reload.add(pid)
# 触发重载
@@ -403,6 +418,71 @@ class PluginManager(ConfigReloadMixin, metaclass=Singleton):
except Exception as e:
logger.error(f"插件 {pid} 热重载失败: {e}", exc_info=True)
def _get_federated_plugin_change(
self,
event_path: Path,
) -> Optional[Tuple[str, Optional[dict], bool]]:
"""
识别运行态 Vue 插件声明目录内的构建产物变化
:return: 插件 ID本地仓库候选和联邦入口是否完整非联邦目录变化返回 None
"""
try:
event_path = event_path.resolve()
candidate = self._get_local_plugin_candidate_from_path(event_path)
if candidate:
pid = candidate.get("id")
plugin_dir = Path(candidate.get("path")).resolve()
else:
runtime_root = (settings.ROOT_PATH / "app" / "plugins").resolve()
if not event_path.is_relative_to(runtime_root):
return None
relative_parts = event_path.relative_to(runtime_root).parts
if not relative_parts:
return None
plugin_dir = runtime_root / relative_parts[0]
pid = next(
(
plugin_id
for plugin_id in self._running_plugins
if plugin_id.lower() == relative_parts[0].lower()
),
None,
)
if not pid:
return None
plugin = self._running_plugins.get(pid)
if not plugin:
return None
render_mode, dist_path = plugin.get_render_mode()
if render_mode != "vue" or not isinstance(dist_path, str) or not dist_path:
return None
relative_dist_path = Path(dist_path)
if relative_dist_path.is_absolute() or ".." in relative_dist_path.parts or "\\" in dist_path:
return None
plugin_dir = plugin_dir.resolve()
dist_dir = (plugin_dir / relative_dist_path).resolve()
if (
dist_dir == plugin_dir
or not dist_dir.is_relative_to(plugin_dir)
or not event_path.is_relative_to(dist_dir)
):
return None
remote_entry = dist_dir / "remoteEntry.js"
remote_entry_ready = (
remote_entry.is_file()
and remote_entry.resolve().is_relative_to(plugin_dir)
)
return pid, candidate, remote_entry_ready
except Exception as e:
logger.error(f"识别插件联邦构建产物变化时出错: {e}")
return None
@staticmethod
def _get_plugin_id_from_path(event_path: Path) -> Optional[str]:
"""
@@ -517,7 +597,7 @@ class PluginManager(ConfigReloadMixin, metaclass=Singleton):
source_dir,
dest_dir,
dirs_exist_ok=True,
ignore=shutil.ignore_patterns("__pycache__", "*.pyc", ".DS_Store")
ignore=shutil.ignore_patterns("__pycache__", "*.pyc", ".DS_Store", "node_modules")
)
PluginManager()._recent_local_sync[pid] = time.time()
logger.info(f"已同步本地插件 {pid}{source_dir} -> {dest_dir}")
@@ -723,21 +803,23 @@ class PluginManager(ConfigReloadMixin, metaclass=Singleton):
await SystemConfigOper().async_set(self._config_key % pid, conf)
return True
def delete_plugin_config(self, pid: str) -> bool:
def delete_plugin_config(self, pid: str, force: bool = False) -> bool:
"""
删除插件配置
:param pid: 插件ID
:param force: 插件停止后仍允许按插件 ID 删除持久化配置
"""
if not self._plugins.get(pid):
if not force and not self._plugins.get(pid):
return False
return SystemConfigOper().delete(self._config_key % pid)
def delete_plugin_data(self, pid: str) -> bool:
def delete_plugin_data(self, pid: str, force: bool = False) -> bool:
"""
删除插件数据
:param pid: 插件ID
:param force: 插件停止后仍允许按插件 ID 删除持久化数据
"""
if not self._plugins.get(pid):
if not force and not self._plugins.get(pid):
return False
PluginDataOper().del_data(pid)
return True
@@ -1624,60 +1706,99 @@ class PluginManager(ConfigReloadMixin, metaclass=Singleton):
return " ".join(normalized_labels) or None
return None
async def async_get_online_plugins(self, force: bool = False) -> List[schemas.Plugin]:
async def async_get_online_plugins(
self,
force: bool = False,
progress_callback: Optional[Callable[..., None]] = None,
) -> List[schemas.Plugin]:
"""
异步获取所有在线插件信息
:param force: 是否强制刷新忽略缓存
:param progress_callback: 定时服务进度更新回调
"""
if not settings.PLUGIN_MARKET:
if progress_callback:
progress_callback(value=100, text="未配置插件市场,跳过刷新")
return []
# 用于存储高于 v1 版本的插件(如 v2, v3 等)
async def fetch_market(
market: str,
package_version: Optional[str],
result_version: str,
task_index: int,
) -> Tuple[int, str, List[schemas.Plugin]]:
"""
获取单个市场版本的插件列表并保留结果分组
"""
plugins = await self.async_get_plugins_from_market(
market,
package_version,
force,
)
return task_index, result_version, plugins or []
higher_version_plugins = []
# 用于存储 v1 版本插件
base_version_plugins = []
# 使用异步并发获取线上插件
import asyncio
tasks = []
task_to_version = {}
for m in settings.PLUGIN_MARKET.split(","):
if not m:
for market in settings.PLUGIN_MARKET.split(","):
if not market:
continue
# 创建任务获取 v1 版本插件
base_task = asyncio.create_task(self.async_get_plugins_from_market(m, None, force))
tasks.append(base_task)
task_to_version[base_task] = "base_version"
# 创建任务获取高版本插件(如 v2、v3)
tasks.append(
asyncio.create_task(
fetch_market(market, None, "base_version", len(tasks))
)
)
if settings.VERSION_FLAG:
higher_version_task = asyncio.create_task(
self.async_get_plugins_from_market(m, settings.VERSION_FLAG, force))
tasks.append(higher_version_task)
task_to_version[higher_version_task] = "higher_version"
tasks.append(
asyncio.create_task(
fetch_market(
market,
settings.VERSION_FLAG,
"higher_version",
len(tasks),
)
)
)
# 并发执行所有任务
if tasks:
completed_tasks = await asyncio.gather(*tasks, return_exceptions=True)
for i, result in enumerate(completed_tasks):
task = tasks[i]
version = task_to_version[task]
# 检查是否有异常
if isinstance(result, Exception):
logger.error(f"获取插件市场数据失败:{str(result)}")
continue
plugins = result
total_tasks = len(tasks)
finished_tasks = 0
task_results = {}
if progress_callback:
progress_callback(
value=0,
text=f"开始刷新插件市场,共 {total_tasks} 个请求 ...",
data={"total": total_tasks, "finished": 0},
)
for completed_task in asyncio.as_completed(tasks):
try:
task_index, version, plugins = await completed_task
task_results[task_index] = (version, plugins)
except Exception as err:
logger.error(f"获取插件市场数据失败:{str(err)}")
finished_tasks += 1
if progress_callback:
progress_callback(
value=finished_tasks / total_tasks * 100,
text=(
f"插件市场请求"
f"{finished_tasks}/{total_tasks})处理完成"
),
data={"total": total_tasks, "finished": finished_tasks},
)
for task_index in sorted(task_results):
version, plugins = task_results[task_index]
if plugins:
if version == "higher_version":
higher_version_plugins.extend(plugins) # 收集高版本插件
higher_version_plugins.extend(plugins)
else:
base_version_plugins.extend(plugins) # 收集 v1 版本插件
base_version_plugins.extend(plugins)
result = self.process_plugins_list(higher_version_plugins, base_version_plugins)
logger.info(f"获取到 {len(result)} 个线上插件")
if progress_callback:
progress_callback(value=100, text="插件市场缓存刷新完成")
return result
async def async_get_plugins_from_market(self, market: str,
+23 -3
View File
@@ -169,6 +169,15 @@ def set_or_refresh_resource_token_cookie(
# 根据剩余时长提前刷新令牌
if remaining_time < timedelta(seconds=(settings.RESOURCE_ACCESS_TOKEN_EXPIRE_SECONDS / 3)):
raise jwt.ExpiredSignatureError
expected_claims = {
"sub": str(payload.sub),
"username": payload.username,
"super_user": payload.super_user,
"level": payload.level,
"purpose": "resource",
}
if any(decoded_token.get(claim) != value for claim, value in expected_claims.items()):
raise jwt.InvalidTokenError("资源令牌身份或权限上下文不匹配")
except jwt.PyJWTError:
logger.debug(f"Token error occurred. refreshing token")
except Exception as e:
@@ -188,12 +197,19 @@ def set_or_refresh_resource_token_cookie(
purpose="resource"
)
# 判断请求是否为 HTTPS:直连协议为 https,或经反向代理转发时携带 X-Forwarded-Proto: https。
# 无法确认为明文 HTTP 时按 fail-safe 默认设置 secure=True,避免代理终止 HTTPS 后以 HTTP 转发导致 Cookie 明文传输。
is_https = (
request.url.scheme == "https"
or request.headers.get("x-forwarded-proto", "").lower() == "https"
)
# 设置会话级别的 HttpOnly Cookie
response.set_cookie(
key=settings.PROJECT_NAME,
value=resource_token,
httponly=True,
secure=request.url.scheme == "https", # 根据当前请求协议设置 secure 属性
secure=is_https, # 根据当前请求协议(含反向代理转发标识)设置 secure 属性
samesite="lax" # 不同浏览器对 "Strict" 的处理可能不同,设置 SameSite 为 "Lax",以平衡安全性和兼容性
)
@@ -309,7 +325,9 @@ def __verify_key(key: str | None, expected_key: str, key_type: str) -> str:
def verify_apitoken(token: Annotated[str | None, Security(__get_api_token)]) -> str:
"""
使用 API Token 进行身份认证
使用 API Token 进行受信第三方集成认证
校验值来自 settings.API_TOKEN通过后只确认集成凭据有效不生成 per-user 权限上下文
:param token: API Token URL 查询参数中获取 token=xxx
:return: 返回校验通过的 API Token
"""
@@ -318,7 +336,9 @@ def verify_apitoken(token: Annotated[str | None, Security(__get_api_token)]) ->
def verify_apikey(apikey: Annotated[str | None, Security(__get_api_key)]) -> str:
"""
使用 API Key 进行身份认证
使用 API Key 形式进行受信第三方集成认证
请求字段名兼容 API Key实际校验值来自 settings.API_TOKEN不生成 per-user 权限上下文
:param apikey: API Key URL 查询参数中获取 apikey=xxx或请求头中获取 X-API-KEY=xxx
:return: 返回校验通过的 API Key
"""
+53 -1
View File
@@ -1,12 +1,60 @@
import asyncio
from typing import Any, Generator, List, Optional, Self, Tuple, AsyncGenerator, Union
from sqlalchemy import NullPool, QueuePool, and_, create_engine, inspect, text, select, delete, Column, Integer, \
from sqlalchemy import NullPool, QueuePool, and_, create_engine, event, inspect, text, select, delete, Column, Integer, \
Sequence, Identity
from sqlalchemy.engine import Engine as SQLAlchemyEngine, ExceptionContext
from sqlalchemy.ext.asyncio import create_async_engine, AsyncSession, async_sessionmaker
from sqlalchemy.orm import Session, as_declarative, declared_attr, scoped_session, sessionmaker
from app.core.config import settings
from app.log import logger
def _database_error_metadata(error: BaseException) -> Optional[dict[str, Any]]:
"""提取 SQLite 与 PostgreSQL 驱动提供的稳定错误分类字段。"""
metadata = {"error_type": type(error).__name__}
# DBAPI 驱动字段并不共享统一类型,动态读取可同时兼容 sqlite3、psycopg2 与 asyncpg。
sqlite_errorcode = getattr(error, "sqlite_errorcode", None)
sqlite_errorname = getattr(error, "sqlite_errorname", None)
if sqlite_errorcode is not None or sqlite_errorname:
if sqlite_errorcode is not None:
metadata["error_code"] = sqlite_errorcode
if sqlite_errorname:
metadata["error_name"] = sqlite_errorname
return metadata
sqlstate = getattr(error, "sqlstate", None) or getattr(error, "pgcode", None)
if not sqlstate:
sqlstate = getattr(getattr(error, "diag", None), "sqlstate", None)
if sqlstate:
metadata["sqlstate"] = sqlstate
return metadata
return None
def _log_database_error(exception_context: ExceptionContext) -> None:
"""记录非敏感驱动错误码,并保持 SQLAlchemy 原有异常传播。"""
metadata = _database_error_metadata(exception_context.original_exception)
if not metadata:
return
dialect = exception_context.dialect
fields = {
"database": dialect.name,
"driver": dialect.driver,
**metadata,
}
logger.error(
"数据库驱动异常:" + ", ".join(f"{key}={value}" for key, value in fields.items())
)
def _register_database_error_logging(engine: SQLAlchemyEngine) -> None:
"""为主程序 Engine 注册统一的底层驱动错误诊断。"""
event.listen(engine, "handle_error", _log_database_error)
def get_id_column():
@@ -71,6 +119,7 @@ def _get_sqlite_engine(is_async: bool = False):
# 创建数据库引擎
engine = create_engine(**_db_kwargs)
_register_database_error_logging(engine)
# 设置WAL模式
_journal_mode = "WAL" if settings.DB_WAL_ENABLE else "DELETE"
@@ -91,6 +140,7 @@ def _get_sqlite_engine(is_async: bool = False):
}
# 创建异步数据库引擎
async_engine = create_async_engine(**_db_kwargs)
_register_database_error_logging(async_engine.sync_engine)
# 设置WAL模式
_journal_mode = "WAL" if settings.DB_WAL_ENABLE else "DELETE"
@@ -146,6 +196,7 @@ def _get_postgresql_engine(is_async: bool = False):
# 创建数据库引擎
engine = create_engine(**_db_kwargs)
_register_database_error_logging(engine)
print(f"PostgreSQL database connected to {settings.DB_POSTGRESQL_TARGET}/{settings.DB_POSTGRESQL_DATABASE}")
return engine
@@ -163,6 +214,7 @@ def _get_postgresql_engine(is_async: bool = False):
}
# 创建异步数据库引擎
async_engine = create_async_engine(**_db_kwargs)
_register_database_error_logging(async_engine.sync_engine)
print(f"Async PostgreSQL database connected to {settings.DB_POSTGRESQL_TARGET}/{settings.DB_POSTGRESQL_DATABASE}")
return async_engine
+150
View File
@@ -0,0 +1,150 @@
from datetime import datetime
from typing import Optional
from app.db import DbOper
from app.db.models.agenttask import AgentTask
class AgentTaskOper(DbOper):
"""
Agent 自主定时任务管理
"""
@staticmethod
def _now() -> str:
"""生成当前数据库时间字符串。"""
return datetime.now().strftime("%Y-%m-%d %H:%M:%S")
def add(self, **kwargs: object) -> AgentTask:
"""
新增 Agent 定时任务
"""
now = self._now()
task_id = AgentTask.add_task(
self._db,
**kwargs,
enabled=True,
last_status="waiting",
run_count=0,
created_at=now,
updated_at=now,
)
return self.get(task_id)
def get(
self,
task_id: int,
user_id: Optional[str] = None,
) -> Optional[AgentTask]:
"""
查询单个 Agent 定时任务
"""
return AgentTask.get_for_user(self._db, task_id=task_id, user_id=user_id)
def list(
self,
user_id: Optional[str] = None,
enabled: Optional[bool] = None,
) -> list[AgentTask]:
"""
查询 Agent 定时任务列表
"""
return AgentTask.list_for_user(self._db, user_id=user_id, enabled=enabled)
def update(
self,
task_id: int,
payload: dict,
user_id: Optional[str] = None,
) -> bool:
"""
更新 Agent 定时任务
"""
normalized_payload = {
key: value
for key, value in payload.items()
if key in {
"name",
"content",
"trigger_type",
"cron_expression",
"run_at",
"enabled",
"last_status",
"last_result",
}
}
if not normalized_payload:
return False
normalized_payload["updated_at"] = self._now()
return AgentTask.update_task(
self._db,
task_id=task_id,
payload=normalized_payload,
user_id=user_id,
)
def delete(self, task_id: int, user_id: Optional[str] = None) -> bool:
"""
删除 Agent 定时任务
"""
return AgentTask.delete_task(
self._db,
task_id=task_id,
user_id=user_id,
)
def mark_running(self, task_id: int) -> bool:
"""
Agent 定时任务标记为运行中
"""
return AgentTask.mark_running(
self._db,
task_id=task_id,
run_at=self._now(),
)
def finish(
self,
task_id: int,
success: bool,
result: str,
disable: bool = False,
) -> bool:
"""
记录 Agent 定时任务执行结果
"""
return AgentTask.finish_task(
self._db,
task_id=task_id,
success=success,
result=(result or "")[:20000],
disable=disable,
)
@staticmethod
def to_dict(
task: AgentTask,
next_run_at: Optional[str] = None,
timezone: Optional[str] = None,
) -> dict:
"""
Agent 定时任务转换为工具可返回的结构
"""
return {
"id": task.id,
"name": task.name,
"content": task.content,
"trigger_type": task.trigger_type,
"cron_expression": task.cron_expression,
"run_at": task.run_at,
"timezone": timezone,
"enabled": bool(task.enabled),
"last_status": task.last_status,
"last_run_at": task.last_run_at,
"last_result": task.last_result,
"run_count": task.run_count or 0,
"next_run_at": next_run_at,
"created_at": task.created_at,
"updated_at": task.updated_at,
}
+61
View File
@@ -0,0 +1,61 @@
from typing import Dict, List, Optional
from app.db import DbOper
from app.db.models.downloadfailure import DownloadFailure
class DownloadFailureOper(DbOper):
"""
下载失败冷却记录管理
"""
def get_active_by_fingerprints(
self,
fingerprints: List[str],
now_time: str,
) -> Dict[str, DownloadFailure]:
"""
批量按指纹查询仍在冷却期的失败记录
"""
failures = DownloadFailure.get_active_by_fingerprints(
self._db,
fingerprints=fingerprints,
now_time=now_time,
)
return {
failure.fingerprint: failure
for failure in failures
if failure and failure.fingerprint
}
def record_failure(
self,
fingerprint: str,
now_time: str,
next_retry_at: str,
**kwargs: object,
) -> DownloadFailure:
"""
新增或更新资源失败记录
"""
return DownloadFailure.record_failure(
self._db,
fingerprint=fingerprint,
now_time=now_time,
next_retry_at=next_retry_at,
**kwargs,
)
def delete_expired(
self,
before_time: str,
limit: Optional[int] = 500,
) -> int:
"""
删除已过期较久的失败记录
"""
return DownloadFailure.delete_expired(
self._db,
before_time=before_time,
limit=limit,
)
+2
View File
@@ -1,4 +1,6 @@
from .agentchat import AgentChat
from .agenttask import AgentTask
from .downloadfailure import DownloadFailure
from .downloadhistory import DownloadHistory, DownloadFiles
from .mediaserver import MediaServerItem
from .message import Message
+170
View File
@@ -0,0 +1,170 @@
from datetime import datetime
from typing import Optional
from sqlalchemy import Boolean, Column, Index, Integer, String, Text
from sqlalchemy.orm import Session
from app.db import Base, db_query, db_update, get_id_column
class AgentTask(Base):
"""
Agent 自主定时任务表
"""
id = get_id_column()
# 任务名称
name = Column(String, nullable=False)
# 交给 Agent 执行的完整任务内容
content = Column(Text, nullable=False)
# 触发类型:date-单次触发,cron-周期触发
trigger_type = Column(String, nullable=False)
# 标准五段 cron 表达式
cron_expression = Column(String)
# 单次触发时间,使用带时区的 ISO 8601 格式
run_at = Column(String)
# 是否继续接受调度
enabled = Column(Boolean, nullable=False, default=True)
# 创建任务的用户与会话上下文
user_id = Column(String, nullable=False)
username = Column(String)
session_id = Column(String, nullable=False)
channel = Column(String)
source = Column(String)
original_chat_id = Column(String)
# 最近一次执行状态与结果
last_status = Column(String, nullable=False, default="waiting")
last_run_at = Column(String)
last_result = Column(Text)
run_count = Column(Integer, nullable=False, default=0)
created_at = Column(String, nullable=False)
updated_at = Column(String, nullable=False)
__table_args__ = (
Index("ix_agenttask_enabled", "enabled"),
Index("ix_agenttask_user_created", "user_id", "created_at", "id"),
)
@classmethod
@db_update
def add_task(cls, db: Session, **kwargs: object) -> int:
"""
新增 Agent 定时任务并返回任务 ID
"""
task = cls(**kwargs)
db.add(task)
db.flush()
return task.id
@classmethod
@db_query
def get_for_user(
cls,
db: Session,
task_id: int,
user_id: Optional[str] = None,
) -> Optional["AgentTask"]:
"""
按任务 ID 和可选用户 ID 查询 Agent 定时任务
"""
query = db.query(cls).filter(cls.id == task_id)
if user_id is not None:
query = query.filter(cls.user_id == user_id)
return query.first()
@classmethod
@db_query
def list_for_user(
cls,
db: Session,
user_id: Optional[str] = None,
enabled: Optional[bool] = None,
) -> list["AgentTask"]:
"""
按用户和启用状态查询 Agent 定时任务
"""
query = db.query(cls)
if user_id is not None:
query = query.filter(cls.user_id == user_id)
if enabled is not None:
query = query.filter(cls.enabled.is_(enabled))
return query.order_by(cls.created_at.desc(), cls.id.desc()).all()
@classmethod
@db_update
def update_task(
cls,
db: Session,
task_id: int,
payload: dict,
user_id: Optional[str] = None,
) -> bool:
"""
按任务 ID 和可选用户 ID 更新 Agent 定时任务
"""
query = db.query(cls).filter(cls.id == task_id)
if user_id is not None:
query = query.filter(cls.user_id == user_id)
return bool(query.update(payload))
@classmethod
@db_update
def delete_task(
cls,
db: Session,
task_id: int,
user_id: Optional[str] = None,
) -> bool:
"""
按任务 ID 和可选用户 ID 删除 Agent 定时任务
"""
query = db.query(cls).filter(cls.id == task_id)
if user_id is not None:
query = query.filter(cls.user_id == user_id)
return bool(query.delete())
@classmethod
@db_update
def mark_running(cls, db: Session, task_id: int, run_at: str) -> bool:
"""
将可执行任务标记为运行中
"""
updated_at = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
return bool(
db.query(cls)
.filter(
cls.id == task_id,
cls.enabled.is_(True),
cls.last_status != "running",
)
.update(
{
"last_status": "running",
"last_run_at": run_at,
"updated_at": updated_at,
}
)
)
@classmethod
@db_update
def finish_task(
cls,
db: Session,
task_id: int,
success: bool,
result: str,
disable: bool = False,
) -> bool:
"""
记录 Agent 定时任务执行结果并按需关闭单次任务
"""
payload = {
"last_status": "success" if success else "failed",
"last_result": result,
"run_count": cls.run_count + 1,
"updated_at": datetime.now().strftime("%Y-%m-%d %H:%M:%S"),
}
if disable:
payload["enabled"] = False
return bool(db.query(cls).filter(cls.id == task_id).update(payload))
+137
View File
@@ -0,0 +1,137 @@
from typing import List, Optional
from sqlalchemy import Column, Float, Index, Integer, String
from sqlalchemy.orm import Session
from app.db import Base, db_query, db_update, get_id_column
class DownloadFailure(Base):
"""
下载失败冷却记录
"""
id = get_id_column()
# 资源失败指纹
fingerprint = Column(String, nullable=False)
# 类型 电影/电视剧
type = Column(String)
# 标题
title = Column(String)
# 年份
year = Column(String)
# TMDBID
tmdbid = Column(Integer)
# 豆瓣ID
doubanid = Column(String)
# Sxx
seasons = Column(String)
# Exx
episodes = Column(String)
# 站点ID
site = Column(Integer)
# 站点名称
site_name = Column(String)
# 种子资源键
torrent_id = Column(String)
# 种子名称
torrent_name = Column(String)
# 种子大小
torrent_size = Column(Float)
# 下载器
downloader = Column(String)
# 下载来源
source = Column(String)
# 失败原因
error_message = Column(String)
# 重试次数
retry_count = Column(Integer, default=0)
# 首次失败时间
first_failed_at = Column(String)
# 最近失败时间
last_failed_at = Column(String)
# 下次允许重试时间
next_retry_at = Column(String)
__table_args__ = (
Index("ux_downloadfailure_fingerprint", "fingerprint", unique=True),
Index("ix_downloadfailure_next_retry_at", "next_retry_at"),
Index("ix_downloadfailure_media_site", "type", "tmdbid", "doubanid", "site"),
)
@classmethod
@db_query
def get_active_by_fingerprints(
cls,
db: Session,
fingerprints: List[str],
now_time: str,
) -> List["DownloadFailure"]:
"""
按指纹批量查询仍处于冷却期的失败记录
"""
normalized = list(dict.fromkeys([fingerprint for fingerprint in fingerprints if fingerprint]))
if not normalized:
return []
return (
db.query(cls)
.filter(cls.fingerprint.in_(normalized), cls.next_retry_at > now_time)
.all()
)
@classmethod
@db_update
def record_failure(
cls,
db: Session,
fingerprint: str,
now_time: str,
next_retry_at: str,
**kwargs: object,
) -> "DownloadFailure":
"""
新增或更新资源失败记录
"""
failure = db.query(cls).filter(cls.fingerprint == fingerprint).first()
payload = {
**kwargs,
"fingerprint": fingerprint,
"last_failed_at": now_time,
"next_retry_at": next_retry_at,
}
if failure:
payload["retry_count"] = (failure.retry_count or 0) + 1
for key, value in payload.items():
setattr(failure, key, value)
return failure
failure = cls(
**payload,
retry_count=1,
first_failed_at=now_time,
)
db.add(failure)
return failure
@classmethod
@db_update
def delete_expired(
cls,
db: Session,
before_time: str,
limit: Optional[int] = 500,
) -> int:
"""
分批清理已过期较久的失败冷却记录
"""
ids = [
row[0]
for row in db.query(cls.id)
.filter(cls.next_retry_at < before_time)
.order_by(cls.id.asc())
.limit(limit)
.all()
]
if not ids:
return 0
return db.query(cls).filter(cls.id.in_(ids)).delete(synchronize_session=False)
+89
View File
@@ -130,6 +130,46 @@ class Subscribe(Base):
return None
return result.scalars().first()
@classmethod
@db_query
def exists_by_username(cls, db: Session, username: str, tmdbid: Optional[int] = None,
doubanid: Optional[str] = None, season: Optional[int] = None):
"""
按订阅 owner 查询同一媒体的订阅行
"""
if not username:
return None
if tmdbid:
query = db.query(cls).filter(cls.username == username, cls.tmdbid == tmdbid)
if season is not None:
query = query.filter(cls.season == season)
return query.first()
elif doubanid:
return db.query(cls).filter(cls.username == username, cls.doubanid == doubanid).first()
return None
@classmethod
@async_db_query
async def async_exists_by_username(cls, db: AsyncSession, username: str, tmdbid: Optional[int] = None,
doubanid: Optional[str] = None, season: Optional[int] = None):
"""
异步按订阅 owner 查询同一媒体的订阅行
"""
if not username:
return None
if tmdbid:
query = select(cls).filter(cls.username == username, cls.tmdbid == tmdbid)
if season is not None:
query = query.filter(cls.season == season)
result = await db.execute(query)
elif doubanid:
result = await db.execute(
select(cls).filter(cls.username == username, cls.doubanid == doubanid)
)
else:
return None
return result.scalars().first()
@classmethod
@db_query
def get_by_state(cls, db: Session, state: str):
@@ -174,6 +214,22 @@ class Subscribe(Base):
)
return result.scalars().first()
@classmethod
@async_db_query
async def async_list_by_title(cls, db: AsyncSession, title: str, season: Optional[int] = None):
"""
异步按标题查询候选订阅列表
"""
if season is not None:
result = await db.execute(
select(cls).filter(cls.name == title, cls.season == season)
)
else:
result = await db.execute(
select(cls).filter(cls.name == title)
)
return result.scalars().all()
@classmethod
@db_query
def get_by_tmdbid(cls, db: Session, tmdbid: int, season: Optional[int] = None):
@@ -209,6 +265,17 @@ class Subscribe(Base):
)
return result.scalars().first()
@classmethod
@async_db_query
async def async_list_by_doubanid(cls, db: AsyncSession, doubanid: str):
"""
异步按豆瓣 ID 查询候选订阅列表
"""
result = await db.execute(
select(cls).filter(cls.doubanid == doubanid)
)
return result.scalars().all()
@classmethod
@db_query
def get_by_bangumiid(cls, db: Session, bangumiid: int):
@@ -222,6 +289,17 @@ class Subscribe(Base):
)
return result.scalars().first()
@classmethod
@async_db_query
async def async_list_by_bangumiid(cls, db: AsyncSession, bangumiid: int):
"""
异步按 Bangumi ID 查询候选订阅列表
"""
result = await db.execute(
select(cls).filter(cls.bangumiid == bangumiid)
)
return result.scalars().all()
@classmethod
@db_query
def get_by_mediaid(cls, db: Session, mediaid: str):
@@ -235,6 +313,17 @@ class Subscribe(Base):
)
return result.scalars().first()
@classmethod
@async_db_query
async def async_list_by_mediaid(cls, db: AsyncSession, mediaid: str):
"""
异步按自定义媒体 ID 查询候选订阅列表
"""
result = await db.execute(
select(cls).filter(cls.mediaid == mediaid)
)
return result.scalars().all()
@classmethod
@db_query
def get_by(cls, db: Session, type: str, season: Optional[str] = None,
+25
View File
@@ -102,6 +102,31 @@ class SubscribeHistory(Base):
)
return result.scalars().all()
@classmethod
@async_db_query
async def async_list_by_type_and_username(
cls,
db: AsyncSession,
mtype: str,
username: str,
page: Optional[int] = 1,
count: Optional[int] = 30
):
"""
按订阅 owner 查询指定类型的历史分页
"""
if not username:
return []
result = await db.execute(
select(cls).filter(
cls.type == mtype,
cls.username == username
).order_by(
cls.date.desc()
).offset((page - 1) * count).limit(count)
)
return result.scalars().all()
@classmethod
@db_query
def exists(cls, db: Session, tmdbid: Optional[int] = None, doubanid: Optional[str] = None,
+45
View File
@@ -1,3 +1,4 @@
import re
import time
from typing import Optional
@@ -6,6 +7,7 @@ from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import Session
from app.db import db_query, db_update, get_id_column, Base, async_db_query
from app.schemas.types import MediaType
def _text_like(column, pattern: str, wildcard: bool = False):
@@ -205,6 +207,49 @@ class TransferHistory(Base):
time.localtime(time.time() - 86400 * days))).subquery()
return db.query(sub_query.c.date, func.count(sub_query.c.id)).group_by(sub_query.c.date).all()
@classmethod
@db_query
def monthly_media_statistics(cls, db: Session):
"""
统计当月成功整理的电影电视剧和剧集数量
电影和电视剧按媒体身份去重剧集优先按历史记录中的集数字段计算
缺少集数时按单条成功整理记录计数
"""
month_prefix = time.strftime("%Y-%m-", time.localtime())
histories = db.query(cls).filter(
cls.status.is_(True),
cls.date.like(f"{month_prefix}%"),
cls.type.in_([MediaType.MOVIE.value, MediaType.TV.value]),
).all()
movie_identities = set()
tv_identities = set()
episode_count = 0
for history in histories:
identity = (history.tmdbid or 0, history.title or "", history.year or "")
if history.type == MediaType.MOVIE.value:
movie_identities.add(identity)
continue
tv_identities.add(identity)
episode_count += cls._history_episode_count(history)
return len(movie_identities), len(tv_identities), episode_count
@staticmethod
def _history_episode_count(history: "TransferHistory") -> int:
"""从单条整理历史中估算成功入库的剧集数量。"""
episode_numbers = [int(value) for value in re.findall(r"\d+", history.episodes or "")]
if len(episode_numbers) >= 2 and "-" in (history.episodes or ""):
return max(1, episode_numbers[-1] - episode_numbers[0] + 1)
if episode_numbers:
return len(set(episode_numbers))
if isinstance(history.files, list) and history.files:
return len(history.files)
return 1
@classmethod
@async_db_query
async def async_statistic(cls, db: AsyncSession, days: Optional[int] = 7):
+48 -16
View File
@@ -29,10 +29,19 @@ class SubscribeOper(DbOper):
"""
新增订阅
"""
subscribe = Subscribe.exists(self._db,
tmdbid=mediainfo.tmdb_id,
doubanid=mediainfo.douban_id,
season=kwargs.get('season'))
owner_scope = bool(kwargs.pop("owner_scope", False))
username = kwargs.get("username") if owner_scope else None
if username:
subscribe = Subscribe.exists_by_username(self._db,
username=username,
tmdbid=mediainfo.tmdb_id,
doubanid=mediainfo.douban_id,
season=kwargs.get('season'))
else:
subscribe = Subscribe.exists(self._db,
tmdbid=mediainfo.tmdb_id,
doubanid=mediainfo.douban_id,
season=kwargs.get('season'))
kwargs.update({
"name": mediainfo.title,
"year": mediainfo.year,
@@ -55,10 +64,17 @@ class SubscribeOper(DbOper):
subscribe = Subscribe(**kwargs)
subscribe.create(self._db)
# 查询订阅
subscribe = Subscribe.exists(self._db,
tmdbid=mediainfo.tmdb_id,
doubanid=mediainfo.douban_id,
season=kwargs.get('season'))
if username:
subscribe = Subscribe.exists_by_username(self._db,
username=username,
tmdbid=mediainfo.tmdb_id,
doubanid=mediainfo.douban_id,
season=kwargs.get('season'))
else:
subscribe = Subscribe.exists(self._db,
tmdbid=mediainfo.tmdb_id,
doubanid=mediainfo.douban_id,
season=kwargs.get('season'))
return subscribe.id, "新增订阅成功"
else:
return subscribe.id, "订阅已存在"
@@ -67,10 +83,19 @@ class SubscribeOper(DbOper):
"""
异步新增订阅
"""
subscribe = await Subscribe.async_exists(self._db,
tmdbid=mediainfo.tmdb_id,
doubanid=mediainfo.douban_id,
season=kwargs.get('season'))
owner_scope = bool(kwargs.pop("owner_scope", False))
username = kwargs.get("username") if owner_scope else None
if username:
subscribe = await Subscribe.async_exists_by_username(self._db,
username=username,
tmdbid=mediainfo.tmdb_id,
doubanid=mediainfo.douban_id,
season=kwargs.get('season'))
else:
subscribe = await Subscribe.async_exists(self._db,
tmdbid=mediainfo.tmdb_id,
doubanid=mediainfo.douban_id,
season=kwargs.get('season'))
kwargs.update({
"name": mediainfo.title,
"year": mediainfo.year,
@@ -93,10 +118,17 @@ class SubscribeOper(DbOper):
subscribe = Subscribe(**kwargs)
await subscribe.async_create(self._db)
# 查询订阅
subscribe = await Subscribe.async_exists(self._db,
tmdbid=mediainfo.tmdb_id,
doubanid=mediainfo.douban_id,
season=kwargs.get('season'))
if username:
subscribe = await Subscribe.async_exists_by_username(self._db,
username=username,
tmdbid=mediainfo.tmdb_id,
doubanid=mediainfo.douban_id,
season=kwargs.get('season'))
else:
subscribe = await Subscribe.async_exists(self._db,
tmdbid=mediainfo.tmdb_id,
doubanid=mediainfo.douban_id,
season=kwargs.get('season'))
return subscribe.id, "新增订阅成功"
else:
return subscribe.id, "订阅已存在"
+30
View File
@@ -58,6 +58,36 @@ async def get_current_active_user_async(
return current_user
def _ensure_manage_user(current_user: User) -> User:
"""
校验用户具备全局管理权限
"""
permissions = current_user.permissions or {}
if not current_user.is_superuser and not bool(permissions.get("manage")):
raise HTTPException(
status_code=400, detail="用户权限不足"
)
return current_user
def get_current_active_manage_user(
current_user: User = Depends(get_current_active_user),
) -> User:
"""
获取当前拥有管理权限的激活用户
"""
return _ensure_manage_user(current_user)
async def get_current_active_manage_user_async(
current_user: User = Depends(get_current_active_user_async),
) -> User:
"""
异步获取当前拥有管理权限的激活用户
"""
return _ensure_manage_user(current_user)
def get_current_active_superuser(
current_user: User = Depends(get_current_user),
) -> User:
+97 -17
View File
@@ -45,6 +45,14 @@ LOG_ERROR_PATTERNS = (
re.compile(r"加载插件.+出错"),
re.compile(r"数据库更新失败"),
)
LOG_RECORD_PATTERN = re.compile(
r"(?:【(?:DEBUG|INFO|WARNING|ERROR|CRITICAL)】|(?:DEBUG|INFO|WARNING|ERROR|CRITICAL):)"
)
CONSOLE_LOGGER_PATTERN = re.compile(r"\[([^\]]+)]")
PLUGIN_ERROR_PATTERNS = (
re.compile(r"(?:^|\s-\s)plugin\.py\s+-\s", re.IGNORECASE),
re.compile(r"插件.+(?:出错|失败|异常|错误)"),
)
SENSITIVE_PATTERNS = (
re.compile(r"(?i)(api[_-]?token|token|password|secret|cookie)(\s*[:=]\s*)[^\s&]+"),
re.compile(r"\bghp_[A-Za-z0-9]{20,}\b"),
@@ -92,10 +100,23 @@ class DoctorRunnerProtocol:
recommendation: str,
fixable: bool = False,
fixed: bool = False,
affects_report_status: bool = True,
context: Optional[dict[str, Any]] = None,
) -> DoctorFinding:
"""
添加诊断发现
:param finding_id: 诊断项稳定标识
:param severity: 诊断严重级别
:param status: 单项诊断状态
:param title: 诊断项标题
:param detail: 诊断详情
:param recommendation: 处理建议
:param fixable: 是否支持 Doctor 自动修复
:param fixed: 本次运行是否已修复
:param affects_report_status: 是否参与整体报告状态聚合
:param context: 可选结构化上下文
:return: 新增的诊断发现
"""
raise NotImplementedError
@@ -275,6 +296,7 @@ def _tail_lines(path: Path, max_lines: int = 120, max_bytes: int = 256 * 1024) -
def _find_error_lines(lines: list[str], max_matches: int = 12) -> list[str]:
"""从近期日志中提取错误关键词命中的行。"""
matches: list[str] = []
for line in lines:
if any(pattern.search(line) for pattern in LOG_ERROR_PATTERNS):
@@ -282,6 +304,39 @@ def _find_error_lines(lines: list[str], max_matches: int = 12) -> list[str]:
return matches[-max_matches:]
def _partition_error_lines(
lines: list[str],
plugin_logger_names: set[str],
max_matches: int = 12,
) -> tuple[list[str], list[str]]:
"""
将主日志错误线索拆分为核心错误和插件子系统错误
:param lines: 近期日志行
:param plugin_logger_names: 已发现的插件控制台 logger 名称
:param max_matches: 每类最多保留的错误行数
:return: 核心错误行和插件错误行
"""
core_matches: list[str] = []
plugin_matches: list[str] = []
plugin_context = False
for line in lines:
if LOG_RECORD_PATTERN.search(line):
logger_match = CONSOLE_LOGGER_PATTERN.search(line)
console_logger = logger_match.group(1).strip().lower() if logger_match else ""
plugin_context = (
console_logger in plugin_logger_names
or any(pattern.search(line) for pattern in PLUGIN_ERROR_PATTERNS)
)
if not any(pattern.search(line) for pattern in LOG_ERROR_PATTERNS):
continue
if plugin_context or any(pattern.search(line) for pattern in PLUGIN_ERROR_PATTERNS):
plugin_matches.append(line)
else:
core_matches.append(line)
return core_matches[-max_matches:], plugin_matches[-max_matches:]
def _frontend_dir() -> Path:
root_public = settings.ROOT_PATH / "public"
configured = Path(settings.FRONTEND_PATH)
@@ -687,14 +742,18 @@ def _check_frontend_assets(runner: DoctorRunnerProtocol) -> None:
def _check_logs(runner: DoctorRunnerProtocol) -> None:
"""扫描近期日志,并区分核心运行异常与插件扩展异常。"""
log_files = [
_backend_app_log_file(),
_backend_stdio_log_file(),
_frontend_stdio_log_file(),
]
plugin_log_dir = settings.LOG_PATH / "plugins"
plugin_logger_names: set[str] = set()
if plugin_log_dir.exists():
log_files.extend(sorted(plugin_log_dir.rglob("*.log"))[:20])
plugin_log_files = sorted(plugin_log_dir.rglob("*.log"))
plugin_logger_names = {path.stem.lower() for path in plugin_log_files}
log_files.extend(plugin_log_files[:20])
found_any = False
for path in log_files:
@@ -702,23 +761,44 @@ def _check_logs(runner: DoctorRunnerProtocol) -> None:
continue
found_any = True
lines = _tail_lines(path)
errors = _find_error_lines(lines)
if not errors:
is_plugin_log = plugin_log_dir in path.parents
if is_plugin_log:
scoped_errors = [(True, _find_error_lines(lines))]
else:
core_errors, plugin_errors = _partition_error_lines(
lines,
plugin_logger_names,
)
scoped_errors = [(False, core_errors), (True, plugin_errors)]
if not any(errors for _, errors in scoped_errors):
continue
is_plugin = plugin_log_dir in path.parents
runner.add(
finding_id=f"logs.{path.stem}.recent_errors",
severity=DoctorSeverity.Warn,
status=DoctorFindingStatus.Degraded,
title="最近日志存在插件异常" if is_plugin else "最近日志存在错误线索",
detail="\n".join(errors),
recommendation=(
"可使用安全模式启动后检查插件配置。"
if is_plugin
else "结合前后的启动日志定位异常;必要时执行 `moviepilot doctor --json` 交给 Agent 或 Issue 流程。"
),
context={"log_file": str(path), "matches": len(errors)},
)
has_core_errors = bool(scoped_errors[0][1]) if not is_plugin_log else False
for is_plugin_error, errors in scoped_errors:
if not errors:
continue
finding_suffix = (
"plugin_errors"
if is_plugin_error and has_core_errors
else "recent_errors"
)
runner.add(
finding_id=f"logs.{path.stem}.{finding_suffix}",
severity=DoctorSeverity.Warn,
status=DoctorFindingStatus.Degraded,
title="最近日志存在插件异常" if is_plugin_error else "最近日志存在错误线索",
detail="\n".join(errors),
recommendation=(
"可使用安全模式启动后检查插件配置。"
if is_plugin_error
else "结合前后的启动日志定位异常;必要时执行 `moviepilot doctor --json` 交给 Agent 或 Issue 流程。"
),
affects_report_status=not is_plugin_error,
context={
"log_file": str(path),
"matches": len(errors),
"component": "plugin" if is_plugin_error else "core",
},
)
if not found_any:
runner.add(
+2
View File
@@ -47,6 +47,8 @@ def _format_finding(finding: DoctorFinding) -> list[str]:
marker = finding.severity.value.upper()
if finding.fixed:
marker = "FIXED"
elif not finding.affects_report_status:
marker = f"{marker}/ADVISORY"
lines = [f"[{marker}] {finding.title}", f"ID: {finding.id}"]
if finding.detail:
lines.append(f"原因: {finding.detail}")

Some files were not shown because too many files have changed in this diff Show More