mirror of
https://github.com/jxxghp/MoviePilot.git
synced 2026-09-04 23:17:20 +08:00
Merge remote-tracking branch 'upstream/v3' into codex/chore/floating-build-toolchain
# Conflicts: # docs/architecture-optimization-checklist.md # docs/architecture-refactor-roadmap.md
This commit is contained in:
@@ -69,7 +69,7 @@ MoviePilot V3 已经形成较清晰的模块化单体:`foundation`、`domain`
|
||||
|
||||
| 指标 | 当前值 | 解释 |
|
||||
|---|---:|---|
|
||||
| 宿主 Python 模块 / 内部依赖边 | 853 / 6,979 | `dependency-baseline.json` 当前快照 |
|
||||
| 宿主 Python 模块 / 内部依赖边 | 856 / 7,006 | `dependency-baseline.json` 当前快照 |
|
||||
| 非平凡 SCC | 2 | 新增 Chain 包根环;另一个是隔离的 29 模块 TMDB 移植包环 |
|
||||
| 跨层 DB 边界债务 | 0 | Application、Chain、API、Agent、Runtime、Workflow 到 DB 的受控债务均为零 |
|
||||
| Model/Oper 事务债务 | 0 | 自建 Session、自动事务装饰器、直接 commit/rollback 等基线均为零 |
|
||||
@@ -78,8 +78,8 @@ MoviePilot V3 已经形成较清晰的模块化单体:`foundation`、`domain`
|
||||
| Python 源码量 | 约 271,400 行 | 60 个文件超过 1,000 行,14 个超过 2,000 行 |
|
||||
| 长方法 | 281 个超过 80 行 | 67 个超过 150 行,23 个超过 250 行;大量是私有方法 |
|
||||
| 全量 mypy 历史债务 | 11,808 / 596 文件 | strict frontier 当前覆盖 41 个文件,本批迁移路径的类型债务已清零 |
|
||||
| Ruff 历史诊断 | 868 | 低水位门禁通过,但规则集只覆盖 `E4/E7/E9/F/I` |
|
||||
| 覆盖率低水位 | Application 79.02%,Domain 79.29% | Chain、Runtime、Agent、Adapter、Startup 未进入包级覆盖率门禁 |
|
||||
| Ruff 历史诊断 | 840 | 低水位门禁通过,但规则集只覆盖 `E4/E7/E9/F/I` |
|
||||
| 覆盖率低水位 | Application 79.39%,Domain 79.29% | Chain、Runtime、Agent、Adapter、Startup 未进入包级覆盖率门禁 |
|
||||
|
||||
### 3.3 热点文件
|
||||
|
||||
@@ -101,7 +101,7 @@ MoviePilot V3 已经形成较清晰的模块化单体:`foundation`、`domain`
|
||||
|
||||
## 4. 优化清单总表
|
||||
|
||||
状态含义:`阻塞` 表示当前门禁已失败;`待执行` 表示尚未开始;`渐进` 表示应按触碰路径逐步收敛。
|
||||
状态含义:`阻塞` 表示当前门禁已失败;`待执行` 表示尚未开始;`执行中` 表示只完成部分纵切面;`已验证` 表示实现和本地合同已落地但未在本文中声明未知的提交或 CI;`渐进` 表示应按触碰路径逐步收敛。
|
||||
|
||||
| ID | 优先级 | 状态 | 事项 | 目标结果 |
|
||||
|---|---|---|---|---|
|
||||
@@ -110,10 +110,10 @@ MoviePilot V3 已经形成较清晰的模块化单体:`foundation`、`domain`
|
||||
| ARCH-102 | P1 | 已交付 | 将 Transfer pending 升级为真实 E3 状态机 | `e9de149db`、`a2e249f20` 已推送;Unit Tests `33092427327`、Pylint `33092427348` 全绿,崩溃结果未知时进入人工确认 |
|
||||
| ARCH-103 | P1 | 执行中 | 类型化 Chain/Agent 数据 Port 与 DTO | 宿主主路径不再注入无 Session Oper,不向入口泄漏 ORM |
|
||||
| ARCH-104 | P1 | 待执行 | 收口跨多次写入的业务事务 | 站点/规则引用清理可整体回滚或幂等恢复 |
|
||||
| ARCH-105 | P1 | 待执行 | 明确 post-commit 与 Outbox 完成语义 | “业务已提交、后置效果 pending”可被调用方正确识别 |
|
||||
| ARCH-105 | P1 | 已验证 | 明确 post-commit 与 Outbox 完成语义 | 业务提交、effect 完成/pending 可区分;stager/store 分离且 claim/settlement 受 fencing,外部 sink 仍承担 at-least-once 幂等边界 |
|
||||
| ARCH-106 | P1 | 待执行 | 让线程/队列/日志 writer 由 bootstrap/lifecycle 显式构造 | 导入或普通 Chain 构造不再启动进程资源 |
|
||||
| ARCH-107 | P1 | 待执行 | 消除 Chain SCC,强化循环门禁 | SCC 只剩精确豁免的 TMDB 移植包环 |
|
||||
| ARCH-108 | P1 | 待执行 | 决策并收口 Application/Chain 到 Adapter 与 HTTP 边界 | 依赖倒置有明确例外、低水位和迁移顺序 |
|
||||
| ARCH-108 | P1 | 执行中 | 决策并收口 Application/Chain 到 Adapter 与 HTTP 边界 | Passkey 缓存纵切面已验证,其余 Adapter/HTTP/DNS 债务继续按低水位迁移 |
|
||||
| ARCH-109 | P1 | 待执行 | 按用例拆分超大 Chain、Scheduler 和厚 API | 稳定 Facade 保留,决策/I/O/状态/生命周期各有 owner |
|
||||
| ARCH-110 | P1 | 待执行 | Module/Event Contract 分可信级执行 | 宿主 provider 严格,第三方插件仍兼容诊断 |
|
||||
| ARCH-111 | P1 | 待执行 | 升级复杂度、类型、覆盖率和并发原语门禁 | 高风险私有路径也进入只降不增的治理面 |
|
||||
@@ -268,14 +268,13 @@ MoviePilot V3 已经形成较清晰的模块化单体:`foundation`、`domain`
|
||||
|
||||
**问题与证据**
|
||||
|
||||
- `app/application/chain/data.py:14-29,134-176` 的 Oper factory 和 getter 基本都是 `Any`;
|
||||
`app/application/agentdata.py:91-120` 还通过 `__dict__.update()` 动态组装端口。
|
||||
- `app/startup/initializers/modules.py:848-863,896-910` 仍向生产 Chain/Agent 注入多个无 Session Oper。
|
||||
- `app/application/chain/data.py` 的 Site/Subscribe/TransferHistory factory 仍是 `Any`;
|
||||
`app/application/agentdata.py` 仍通过 `__dict__.update()` 动态组装未迁移端口。
|
||||
- `app/startup/initializers/modules.py` 仍向生产 Chain/Agent 注入部分无 Session Oper。
|
||||
- 无 Session Oper 会为单次调用独立创建事务;一个业务操作的“查询后更新”可能被拆成多个事务。
|
||||
- Workflow query 已在 S1-L2 迁入冻结 DTO 和 adapter-owned Session 投影;其余 Chain/Agent raw data port
|
||||
仍返回 `Any`/ORM,Subscription mutation 内部也消费 ORM,因而仍存在 Session 生命周期外
|
||||
detached/lazy-load 的潜在风险。公开 Subscription、Site、History QueryService 已经投影 DTO,
|
||||
属于完成项,不应重做。
|
||||
- Workflow、User 和 DownloadHistory 已迁入冻结 DTO 与 adapter-owned Session 投影;
|
||||
剩余 Chain/Agent raw data port 仍可返回 `Any`/ORM,Subscription mutation 内部也消费 ORM,
|
||||
因而尚有 Session 生命周期外 detached/lazy-load 风险。
|
||||
- S1-L2 由 `b4f873654`、`a01a35bcb` 交付;精确 head SHA 的 Unit Tests `33098869736` 与
|
||||
Pylint `33098869837` 全绿,Application 覆盖率低水位提升并固化至 `78.78%`。该证据只完成
|
||||
Workflow query 纵切面,不能替代 S1-L3 对其余 Chain/Agent raw data port 的清零。
|
||||
@@ -293,15 +292,20 @@ MoviePilot V3 已经形成较清晰的模块化单体:`foundation`、`domain`
|
||||
伪注入;Workflow 执行服务只在 Application owner 配置一次,不再重复注册到 `ChainDataPorts`。
|
||||
- [x] DownloadFailure/MediaServer 两个 registry 字段改用冻结 DTO 与 typed Repository factory;
|
||||
ORM 不越过短 Session,媒体库远端枚举期间不持有事务,旧 Oper/Compat 与公开 Chain ABI 保持不变。
|
||||
- [x] User Chain/Agent/认证查询改用冻结 `UserSnapshot`/`UserAuthSnapshot`;创建、更名和
|
||||
删除由请求级 UoW 原子提交,用户名唯一约束、最后一个启用超级管理员保护与
|
||||
UserConfig/PassKey 级联约束共同守住身份聚合。
|
||||
- [x] DownloadHistory 查询和写入改用冻结 DTO、typed Port 与短 Session adapter,删除在单一
|
||||
UoW 中处理;TransferHistory 仍是 raw port,History 整体仍在执行中。
|
||||
- [ ] `ChainDataPorts`/`AgentDataPorts` 可暂时保留为兼容聚合器,但字段必须显式、可类型检查。
|
||||
- [ ] 以一个业务纵切面迁移并验证后,再迁移下一组,禁止一次替换所有 Oper。
|
||||
- [ ] 增加 AST 门禁,禁止向 `ChainDataPorts`、`AgentDataPorts` 和新的 canonical use-case service
|
||||
注入裸 Oper;SystemConfig singleton、legacy transaction runner 等兼容边界使用精确 allowlist。
|
||||
|
||||
**首批建议**
|
||||
**后续顺序**
|
||||
|
||||
1. Workflow query DTO。
|
||||
2. Chain/Agent 的 Subscribe/History/User/Site raw port。
|
||||
1. 完成 TransferHistory,收口 History 剩余半边。
|
||||
2. Chain/Agent 的 Subscribe/Site raw port。
|
||||
3. Subscription mutation 与站点、规则组引用更新。
|
||||
4. Agent 数据能力。
|
||||
|
||||
@@ -350,28 +354,29 @@ MoviePilot V3 已经形成较清晰的模块化单体:`foundation`、`domain`
|
||||
|
||||
### ARCH-105 明确 post-commit 与 Outbox 完成语义
|
||||
|
||||
**问题与证据**
|
||||
**已实现事实与语义边界**
|
||||
|
||||
- `app/application/outbox.py:167-192` 在业务提交后直接执行 `after_commit()`、即时 publish 和完成标记;
|
||||
某一步抛错时,调用方可能收到失败,但业务行和 intent 已经提交。
|
||||
- 通用 `DurableEventCommand` commit 后没有先调用已经定义的 `claim_by_event_key()`;dispatcher 可在
|
||||
commit 与请求线程即时 publish 之间先 claim 并投递,随后请求线程再次 publish,形成双发窗口。
|
||||
`app/application/subscription/complete.py:111-145` 已提供先 claim 的正确参考。
|
||||
- 下载历史及事件 intent 已原子提交;通知在 commit 后同步执行,模块后处理和字幕再投进线程池。
|
||||
进程在提交与这些动作完成之间退出时,未持久化的动作不会自动恢复。
|
||||
- `SqlAlchemyOutboxRepository` 同时提供不提交的 `stage()` 和内部自提交的 dispatcher 方法,
|
||||
事务所有权没有由类型清楚表达。
|
||||
- `OutboxStager` 只在业务 Session 中 stage/flush;`OutboxDispatchStore` 的 claim、complete 和 retry
|
||||
每次使用独立短事务,业务与 dispatcher 的事务所有权已由类型分开。
|
||||
- 请求线程即时投递与 dispatcher 均先按稳定 event key 原子 claim;同一 lease 期间只有
|
||||
一个 owner,过期 owner 不能用旧 attempt 覆盖新 owner 的 complete/retry。
|
||||
- `PostCommitResult`/`PostCommitEffectError` 保留“业务已提交”事实,并逐项列出已完成和
|
||||
pending effect,后置效果失败不伪装成业务回滚。
|
||||
- lease/attempt fencing 只保护宿主的认领与结算。外部调用成功但 complete 落库前崩溃时,
|
||||
intent 仍会重放;因此交付承诺是 at-least-once。事件载荷和宿主 correlation context
|
||||
携带稳定 event key,支持幂等的消费者应使用它;旧通知插件保持原方法签名,不能宣称外部
|
||||
provider 已获得 exactly-once 或统一幂等能力。
|
||||
|
||||
**目标与步骤**
|
||||
|
||||
- [ ] Command 返回结构化结果:业务是否提交、哪些后置效果完成、哪些处于 pending。
|
||||
- [ ] commit 后先取得 delivery lease;未取得时跳过请求线程直投,确保与 dispatcher 排他。
|
||||
- [ ] 每个 post-commit effect 使用独立 intent 隔离和结算,避免效果 A 失败导致效果 B 被一起重放。
|
||||
- [x] Command 返回结构化结果:业务是否提交、哪些后置效果完成、哪些处于 pending。
|
||||
- [x] commit 后先取得 delivery lease;未取得时跳过请求线程直投,确保与 dispatcher 排他。
|
||||
- [x] 每个 post-commit effect 使用独立 intent 隔离和结算,避免效果 A 失败导致效果 B 被一起重放。
|
||||
单个外部效果仍是 at-least-once,必须有稳定幂等键和幂等 handler/消费者。
|
||||
- [ ] 按完成承诺、可重建性、外部不可逆性和业务重要性划分 E0-E3;用户可见性只是因素之一。
|
||||
- [ ] 拆分 `OutboxStager` 与 `OutboxDispatchStore`,避免业务 Session 调用自提交方法。
|
||||
- [ ] 在 commit、claim、publish、complete、通知和任务提交各断点注入异常/崩溃,验证声明等级与实际恢复一致。
|
||||
- [ ] 增加请求线程即时投递与 dispatcher 并发竞争测试,以及“外部调用成功、complete 前崩溃”的重放测试。
|
||||
- [x] 拆分 `OutboxStager` 与 `OutboxDispatchStore`,避免业务 Session 调用自提交方法。
|
||||
- [x] 在 commit、claim、publish、complete、通知和任务提交各断点注入异常/崩溃,验证声明等级与实际恢复一致。
|
||||
- [x] 增加请求线程即时投递与 dispatcher 并发竞争测试,以及“外部调用成功、complete 前崩溃”的重放测试。
|
||||
|
||||
```bash
|
||||
.venv/bin/python -m pytest \
|
||||
@@ -469,7 +474,8 @@ MoviePilot V3 已经形成较清晰的模块化单体:`foundation`、`domain`
|
||||
`security/passkey.py`、`backup.py`、`image.py`、`rss.py`、`security/cookie.py`。
|
||||
- `app/chain` 有 8 个文件、13 条直接 Adapter 导入,使用 `RequestUtils`、Browser、Cloudflare、
|
||||
CookieCloud、ServerHelper 等具体能力。
|
||||
- Passkey Application 服务直接判断 Redis 后端并调用 `RedisHelper.pop()`,安全策略识别了具体实现。
|
||||
- Passkey Application 已改为消费启动注入的 `PasskeyChallengeCache`,不再判断 Redis 或导入
|
||||
具体 cache adapter;Memory/Redis 均实现严格 `AtomicCacheBackend.store/consume`。
|
||||
- 审计时 LLM streaming、第三方 SDK、移植库和本地控制面没有精确例外表;S0-L2.4b 已建立
|
||||
66 条完整 egress identity 与 zero-growth policy,其中 11 条普通 HTTP/Session bridge 和 1 条
|
||||
Application DNS I/O 是清零债务;每条初始边另有独立指纹上界,不能靠同时刷新 baseline/policy
|
||||
@@ -479,7 +485,7 @@ MoviePilot V3 已经形成较清晰的模块化单体:`foundation`、`domain`
|
||||
|
||||
- [x] 建立 Application/Chain 原始 Adapter 直连事实与精确临时 policy,冻结新增、替换和陈旧条目。
|
||||
- [x] 建立全宿主 direct egress 事实;SDK/stream/vendor/local-control 例外精确到 bindings/uses 指纹。
|
||||
- [ ] 将 Passkey 原子领取提升为 runtime cache contract,由 Memory/Redis backend 分别实现。
|
||||
- [x] 将 Passkey 原子领取提升为 runtime cache contract,由 Memory/Redis backend 分别实现。
|
||||
- [ ] 为 Backup 定义 Application-owned artifact store Port,由 startup 注入文件系统实现。
|
||||
- [ ] 将 policy 中 11 条普通 HTTP/Session bridge 债务迁移到统一网络能力并把目标收缩为空。
|
||||
- [ ] 为 Application SSRF 校验注入 DNS 解析 Port,清除 `socket.getaddrinfo` 直接 I/O。
|
||||
@@ -611,7 +617,8 @@ MoviePilot V3 已经形成较清晰的模块化单体:`foundation`、`domain`
|
||||
|
||||
1. ARCH-102 设计并迁移 Transfer E3 状态机。
|
||||
2. 以站点/规则引用清理为 ARCH-103/104 的第一个 typed Port + UoW 纵切面。
|
||||
3. 完成 ARCH-105 的 post-commit 结构化结果与 Outbox 角色拆分。
|
||||
3. ARCH-105 已完成 post-commit 结构化结果、Outbox 角色拆分和 claim fencing;
|
||||
后续只继续清理独立 intent 与更完整的崩溃矩阵。
|
||||
|
||||
**退出条件**:故障注入覆盖清单列出的崩溃窗口;schema 有 migration;主路径不再 fail-open;调用方能区分
|
||||
业务提交与后置效果 pending。
|
||||
@@ -620,7 +627,7 @@ MoviePilot V3 已经形成较清晰的模块化单体:`foundation`、`domain`
|
||||
|
||||
1. ARCH-106 将日志、消息队列和主循环 gateway 纳入 lifecycle。
|
||||
2. ARCH-107 拆出 `chain/base.py` 并消除新增 SCC。
|
||||
3. ARCH-108 先迁移 Passkey、Backup,再按风险迁移外部调用。
|
||||
3. ARCH-108 的 Passkey 缓存边界已验证;下一纵切面是 Backup,之后按风险迁移外部调用。
|
||||
|
||||
**退出条件**:冷导入不启动线程;SCC 只剩 TMDB 精确豁免;Application/Chain 到 Adapter 的债务只降不增。
|
||||
|
||||
|
||||
@@ -704,8 +704,8 @@ flowchart LR
|
||||
|
||||
| 指标 | 当前值 |
|
||||
|---|---:|
|
||||
| Python 模块 | 853 |
|
||||
| 内部导入边 | 6,979 |
|
||||
| Python 模块 | 856 |
|
||||
| 内部导入边 | 7,006 |
|
||||
| 非平凡 SCC | 2(`ARCH-107` 临时 Chain 包根环;精确 containment 的 TMDB 移植包环) |
|
||||
| Direct egress | 66(12 条待迁移债务,54 条精确 containment) |
|
||||
| Module Contract V2 spec | 217(其中 215 个进入 `run_module` 观察面) |
|
||||
|
||||
@@ -103,14 +103,14 @@ canonical 主程序;兼容只经统一 Compat/SDK 门面提供。
|
||||
| S1-L3.2 Chain registry/DI | `ACTIVE` | S1-L3.1 | 显式类型化 factory,删除 PortProxy 与失效的双重注入,构造器注入真实控制调用 |
|
||||
| S1-L3.2.1 Registry hygiene | `DELIVERED` | S1-L3.1 | `ac7a20132`:删除零消费者 PortProxy/动态转发和 `ChainRuntimeContext.data_ports` 伪注入;Workflow 退出 Chain registry,只保留 Application owner 单一配置入口;Unit Tests `33120205586`、Pylint `33120205581` 全绿 |
|
||||
| S1-L3.3 DownloadFailure/MediaServer | `DELIVERED` | S1-L3.2 | `5fb62108a`:两组 raw factory 已替换为冻结 DTO/typed Port;失败冷却在 Session 内投影,媒体库查询只返回标量且每个 upsert/cleanup 独立短事务,远端枚举不持有 Session;旧 Oper 与插件可见 Chain ABI 保持不变;Unit Tests `33127544925`、Pylint `33127544927` 全绿,Application 覆盖率低水位提升至 `78.95%` |
|
||||
| S1-L3.4 User | `PLANNED` | S1-L3.3 | 认证、偏好与渠道绑定投影冻结快照,User Chain/Agent 不接收 ORM |
|
||||
| S1-L3.5 History | `PLANNED` | S1-L3.4 | Download/Transfer history 统一 typed query/mutation,删除下载历史双事务 fail-open |
|
||||
| S1-L3.4 User | `VERIFIED` | S1-L3.3 | User Chain/Agent/认证改用冻结 typed snapshot;创建、更名、删除与最后一个启用超级管理员保护归并到单 UoW;用户名唯一索引及 UserConfig/PassKey 级联迁移落地,UserConfig 在 commit 后持写锁重载数据库事实源并发布内存快照 |
|
||||
| S1-L3.5 History | `IN_PROGRESS` | S1-L3.4 | DownloadHistory 已迁入冻结 DTO、typed query/write Port 与短 Session adapter,下载历史删除不再拆成 fail-open 双事务;TransferHistory 半边尚未完成,本项不宣称整体交付 |
|
||||
| S1-L3.6 Site | `PLANNED` | S1-L3.5 | 复用 Site query/health,补齐同步 typed command,Session 内完成 DTO 投影 |
|
||||
| S1-L3.7 Subscription | `PLANNED` | S1-L3.6 | Chain/Workflow/interaction 全部消费 typed query/command;完成后进入 S1-L4 原子事务收口 |
|
||||
| S1-L3.8 Agent/Transfer locator gate | `PLANNED` | S1-L3.7 | 删除 AgentDataPorts 与 Chain locator 跨层泄漏,AST 门禁确认 canonical 无 raw getter/Oper/Any |
|
||||
| S1-L4 Subscription mutation UoW | `PLANNED` | S1-L3 | Subscription mutation 不跨 Session 传 ORM,正式写路径一个 UoW,旧自动事务入口退出 canonical 路径 |
|
||||
| S1-L5 站点/规则引用原子清理 | `PLANNED` | S1-L4 | SystemConfig+Subscribe 同事务更新,commit 后快照原子发布,并发/故障注入无部分状态 |
|
||||
| S1-L6 Outbox 完成语义 | `PLANNED` | S0 | claim 竞争双发清零;业务提交与 effect pending 可区分;stager/store 分离;handler 幂等与崩溃测试完整 |
|
||||
| S1-L6 Outbox 完成语义 | `VERIFIED` | S0 | 事务内 `OutboxStager` 与独立短事务 `OutboxDispatchStore` 已分离;即时投递与 dispatcher 均先 claim,complete/retry 受 attempt fencing;`PostCommitResult` 区分已提交业务、已完成与 pending effect。事件载荷和宿主 correlation context 携带稳定 event key;旧通知插件保持原签名并承认 at-least-once 重复边界 |
|
||||
|
||||
### S2:进程生命周期、循环与 Adapter 边界
|
||||
|
||||
@@ -122,7 +122,7 @@ canonical 主程序;兼容只经统一 Compat/SDK 门面提供。
|
||||
| S2-L1 日志/消息资源显式生命周期 | `PLANNED` | S0 | import 和非消息 Chain 构造零新增线程;bootstrap 显式创建,失败和正常关闭均收口 |
|
||||
| S2-L2 ChainBase 与 SCC 清零 | `PLANNED` | S0-L2.2 | canonical `app.chain.base` 落地,包根无 eager/重复导出,宿主包根导入清零,Chain SCC 消失 |
|
||||
| S2-L3 GlobalVar/provider 注册收口 | `PLANNED` | S2-L1 | `global_vars` canonical 消费清零,provider 注册进入显式装配阶段并可 reset;Legacy 入口精确保留 |
|
||||
| S2-L4 Passkey 缓存边界 | `PLANNED` | S0-L2.4 | Application 不识别 Redis;原子 consume 由 runtime cache contract + backend 实现 |
|
||||
| S2-L4 Passkey 缓存边界 | `VERIFIED` | S0-L2.4 | `PasskeyChallengeCache` 由 startup 注入,Application 不识别 Redis;严格 `AtomicCacheBackend.store/consume` 由 Memory/Redis backend 分别实现,challenge 仅能被原子领取一次 |
|
||||
| S2-L5 Backup artifact Port | `PLANNED` | S0-L2.4 | Application 不构造 `BackupFiles`,文件 I/O 由注入 Adapter 拥有 |
|
||||
| S2-L6 Application Adapter/DNS 债务清零 | `PLANNED` | S2-L4,S2-L5 | Application 到具体 Adapter 的未批准边归零,SSRF DNS I/O 进入注入 Port,批准通用机制有精确规则和门禁 |
|
||||
| S2-L7 Chain Adapter/宿主 HTTP 债务清零 | `PLANNED` | S2-L6 | Chain 具体 Adapter 与 11 条普通 direct HTTP/Session bridge 归零;SDK/stream/vendor 例外保持精确 containment |
|
||||
@@ -153,7 +153,7 @@ canonical 主程序;兼容只经统一 Compat/SDK 门面提供。
|
||||
| S4-L2 Event strict contract | `PLANNED` | S0-L2.6,S1-L6 | 宿主事件输入/输出按风险 strict,诊断例外只属于第三方插件兼容 |
|
||||
| S4-L3 Complexity v2 | `PLANNED` | S3 | 私有方法、class/file、圈复杂度进入门禁;所有超限通过职责拆分归零 |
|
||||
| S4-L4 全量 mypy 清零 | `PLANNED` | S3,S4-L1,S4-L2 | `mypy-baseline.json` 归零并删除债务接受路径,全宿主 strict 类型通过 |
|
||||
| S4-L5 Ruff 治理债务清零 | `PLANNED` | S3 | 当前受控 868 条诊断归零,规则集扩展经过独立审查且新增诊断为零 |
|
||||
| S4-L5 Ruff 治理债务清零 | `PLANNED` | S3 | 当前受控 840 条诊断归零,规则集扩展经过独立审查且新增诊断为零 |
|
||||
| S4-L6 Coverage/并发/质量证据 | `PLANNED` | S3,S4-L1,S4-L2 | 高风险包纳入 coverage;raw concurrency 分类清零;Module Quality 有真实 evidence test |
|
||||
|
||||
### S5:Plugin、Agent、Domain、Startup 与最终收口
|
||||
|
||||
@@ -64,16 +64,17 @@ to make the directory tree look symmetrical.
|
||||
| `app/application/subscription/` | Subscription use cases: `write.py` owns media-to-row translation and the write port; `contract.py` owns shared metadata/media-key projection; query, mutation, deletion, identity and search stay in their single-word modules |
|
||||
| `app/application/search/` | Search state and later search-plan use cases |
|
||||
| `app/application/download/` | Download task querying/control and selection use cases; `failures.py` owns the frozen failure-cooldown write/query DTOs and persistence Port |
|
||||
| `app/application/history.py` | History use cases and persistence contracts; DownloadHistory owns frozen query/write DTOs and typed ports, while TransferHistory remains a separate unfinished migration surface |
|
||||
| `app/application/music/` | Multi-source music catalog orchestration |
|
||||
| `app/application/chain/` | Injectable Chain runtime capabilities: `context.py` owns the runtime dependency aggregate, `data.py` owns named persistence ports, and `events.py` owns durable event write contracts plus replayable payload conversion |
|
||||
| `app/application/agentdata.py` | Named Agent data ports; canonical Agent consumers use `get_agent_*_port()` and do not alias legacy proxies to Oper classes |
|
||||
| `app/application/outbox.py` | Durable intent and Outbox repository/dispatcher contracts for post-commit side effects |
|
||||
| `app/application/outbox.py` | Durable intent, transaction-only stager, short-transaction dispatch store, claim fencing and structured post-commit result contracts |
|
||||
| `app/application/transfer/` | Durable transfer use cases: `workflow.py` owns admission/planning/queue behavior; `execution.py` owns stable operation identity, step/checkpoint state, retry/manual-review commands and terminal-settlement DTOs |
|
||||
| `app/application/plugin/` | Plugin market catalog, installation command, installed-plugin identity contract and startup migration, runtime port, folder operations and dynamic-route use cases; filenames remain single words (`catalog.py`, `identity.py`, `migration.py`, `install.py`, `runtime.py`, `folders.py`, `routes.py`) |
|
||||
| `app/application/server/` | MoviePilot Server reporting and sharing use cases; local data readers and transport callbacks are injected by startup |
|
||||
| `app/application/site/` | Configured site catalog, authentication level and index-resource capability; the generated extension and its data bundle stay together here |
|
||||
| `app/application/messaging/` | Message rendering/routing, interactions and the Agent-to-message bridge: `ingress.py` owns the single channel-to-host loopback boundary; `interaction.py` shared interaction contracts and view helpers; `router.py` unified interaction priority and callback dispatch; `site.py`/`subscribe.py`/`skill.py` per-command sessions, input parsing and views; `media.py` media interaction state while the business workflow stays in `MediaInteractionChain`; `plugin.py` plugin input capture and plugin button callbacks; `agent.py` agent choice state, callback protocol and WebAgent bridge; `message.py` notification rendering, templates and queue. Not a public SDK recommended for direct plugin use |
|
||||
| `app/application/security/` | Authentication, authorization, cookies, passkeys, OTP/two-factor, path/URL safety, SSRF and signing policy |
|
||||
| `app/application/security/` | Authentication, authorization, frozen user/auth projections, atomic user aggregate commands, per-user configuration publication, cookies, passkeys, OTP/two-factor, path/URL safety, SSRF and signing policy |
|
||||
|
||||
Application services may use domain rules and runtime contracts. They own the
|
||||
persistence Protocol needed by a use case, but must not import `app.db`,
|
||||
@@ -457,10 +458,14 @@ SQLAlchemy models stay under `app/db/models/`; the data access classes live in
|
||||
carries the role. Two verified aggregation exceptions exist: the site family
|
||||
(`Passkey`, `SiteIcon`, `SiteStatistic`, `SiteUserData`) is consolidated in
|
||||
`oper/site.py`, and `AgentTaskRun` lives in `oper/agenttask.py`. DB adapters use
|
||||
Oper classes instead of issuing SQLAlchemy queries directly. Application and
|
||||
Chain code reaches persistence through named Ports/Protocols; concrete DB adapters
|
||||
are the layer that adapts those Ports to Oper classes. Every schema change
|
||||
requires an Alembic migration under `database/versions/`.
|
||||
Oper classes for ordinary entity access. Adapter-owned cross-row locks and
|
||||
compare-and-set transitions may issue focused SQLAlchemy statements when the
|
||||
atomic persistence invariant cannot be expressed by an entity Oper; those
|
||||
statements stay private to the adapter and require concurrency tests.
|
||||
Application and Chain code reaches persistence through named Ports/Protocols;
|
||||
concrete DB adapters are the only layer that adapts those Ports to Oper/Session
|
||||
mechanics. Every schema change requires an Alembic migration under
|
||||
`database/versions/`.
|
||||
|
||||
Oper classes take and return persistence values, not domain objects. Translating
|
||||
`MediaInfo` / `MetaBase` into a row is business logic and belongs in
|
||||
@@ -477,17 +482,40 @@ path cannot forget them. Identity representation rules themselves
|
||||
alongside the two identity mixins; `app/domain/media.py` keeps only source
|
||||
policy. `app/db` therefore has no dependency on `app/domain`.
|
||||
|
||||
User identity is a single aggregate boundary. `app/application/security/user.py`
|
||||
owns frozen user/auth snapshots and the atomic create/update/delete command;
|
||||
`app/db/adapters/user.py` binds each mutation to one request UoW and locks the
|
||||
active-superuser set before a destructive change. The database enforces unique
|
||||
user names and cascades rename/delete to `UserConfig` and delete to `PassKey`.
|
||||
The configured user-configuration repository publishes its in-memory snapshot
|
||||
only after the user transaction commits, and reloads from the database if
|
||||
publication fails.
|
||||
|
||||
Download history is the verified half of the History migration:
|
||||
`app/application/history.py` owns frozen `DownloadHistorySnapshot` /
|
||||
`DownloadFileSnapshot` values and typed query/write ports;
|
||||
`app/db/adapters/history/download.py` performs projection and mutations in short
|
||||
Session scopes. TransferHistory still uses its existing ports, so this must not
|
||||
be described as completion of the entire History boundary.
|
||||
|
||||
Durable post-commit side effects have a separate boundary:
|
||||
|
||||
- `app/application/outbox.py` owns the Outbox intent, repository and dispatcher
|
||||
contracts. An Application command stages the business mutation and its durable
|
||||
intent in the same transaction.
|
||||
- `app/db/adapters/outbox.py` implements the persistence port with SQLAlchemy;
|
||||
- `app/application/outbox.py` separates `OutboxStager`, which only stages in the
|
||||
caller's business transaction, from `OutboxDispatchStore`, whose claim,
|
||||
complete and retry operations own independent short transactions.
|
||||
- `app/db/adapters/outbox.py` implements both roles with SQLAlchemy;
|
||||
`app/startup/composition/subscription.py` and the other composition modules
|
||||
provide the concrete repository, UoW and handlers.
|
||||
- The dispatcher claims an intent with a lease, executes the topic handler, and
|
||||
records retry/dead-letter state. Handlers must be idempotent and must not rely
|
||||
on a live request object.
|
||||
inject the stager, dispatch-store factory, UoW and handlers.
|
||||
- Immediate request-thread delivery and the dispatcher both claim before
|
||||
calling a handler. Lease acquisition is atomic, while complete/retry is fenced
|
||||
by the claimed attempt so an expired owner cannot settle a newer claim.
|
||||
- `PostCommitResult` distinguishes the committed business value from completed
|
||||
and pending effects. This does not provide exactly-once delivery: a process may
|
||||
stop after an external sink succeeds but before complete is persisted. Outbox
|
||||
delivery is therefore at-least-once. Event payloads and the host correlation
|
||||
context carry the stable event key, and consumers that support deduplication
|
||||
should use it. Legacy notification plugins retain their existing method
|
||||
signature, so the host must not claim provider-level exactly-once delivery.
|
||||
- Terminal history is part of the shared data-maintenance policy and is cleaned
|
||||
in bounded daily batches only when that policy is enabled. Completed intents
|
||||
default to 30-day retention and dead letters to 90 days; both values are
|
||||
@@ -546,6 +574,10 @@ expired claimed task remains exclusively owned by fenced recovery APIs.
|
||||
- Startup registers concrete cache factories before decorated business modules
|
||||
are imported. Cache contracts remain in `app/runtime/cache.py`; Redis/file
|
||||
implementations remain in `app/adapters/cache/backends.py`.
|
||||
- Security-sensitive one-shot state uses the strict `AtomicCacheBackend`
|
||||
`store/consume` contract. Memory and Redis implement atomic consume; startup
|
||||
injects that capability through `PasskeyChallengeCache`, so Passkey
|
||||
Application code never identifies or imports the Redis implementation.
|
||||
- `app/runtime/log.py` is a dependency leaf with no `app.*` imports. Foundation
|
||||
emits no runtime logs; upper-layer owners decide whether failures are
|
||||
operationally relevant.
|
||||
@@ -573,6 +605,9 @@ expired claimed task remains exclusively owned by fenced recovery APIs.
|
||||
may use `app.core`, `app.helper`, `app.utils` or `app.log`.
|
||||
- New plugins use `app.sdk`. In DEBUG mode, a legacy plugin import remains
|
||||
functional and emits one actionable warning per plugin and legacy module.
|
||||
- Plugin compatibility changes belong only in curated SDK/Legacy exports or the
|
||||
exact Compat manifest. `app/plugins/**` contains runtime plugin copies and is
|
||||
excluded from host refactors, dependency baselines and ownership migrations.
|
||||
- Delayed imports are not accepted as a way to hide dependency cycles.
|
||||
|
||||
### Dependency facts and semantic policy
|
||||
@@ -677,8 +712,13 @@ driven workflow registration.
|
||||
| `app/application/download/failures.py` | Frozen download-failure cooldown write/query DTOs and Chain persistence Port |
|
||||
| `app/db/adapters/download.py` | Short-session download-failure snapshot and mutation adapter |
|
||||
| `app/db/adapters/mediaserver.py` | Per-operation media-server cache query/upsert/cleanup transaction adapter |
|
||||
| `app/application/outbox.py` | Durable intent, topic handler and Outbox repository contracts |
|
||||
| `app/db/adapters/outbox.py` | SQLAlchemy Outbox persistence, claim/lease and retry state adapter |
|
||||
| `app/application/history.py` | History use cases; frozen DownloadHistory DTOs and typed query/write ports, with TransferHistory migration still pending |
|
||||
| `app/db/adapters/history/download.py` | DownloadHistory short-session snapshot, query and mutation adapter |
|
||||
| `app/application/security/user.py` | Frozen user/auth projections and atomic user aggregate service contracts |
|
||||
| `app/db/adapters/user.py` | User projection plus request-UoW mutation adapter |
|
||||
| `app/db/adapters/configuration.py` | Commit-after UserConfig snapshot publication and fact-source reload adapter |
|
||||
| `app/application/outbox.py` | Durable intent, stager/store, claim fencing, topic handler and structured post-commit contracts |
|
||||
| `app/db/adapters/outbox.py` | SQLAlchemy Outbox transaction-only stagers and short-transaction claim/settlement stores |
|
||||
| `app/application/chain/events.py` | Chain durable-event write port, settlement projection and replayable payload conversion |
|
||||
| `app/application/transfer/workflow.py` | Transfer task, durable admission, versioned planning input/checkpoint contracts and queue use case |
|
||||
| `app/db/adapters/transfer/admission.py` | SQLAlchemy admission/checkpoint persistence, CAS state transition and detached snapshot adapter |
|
||||
@@ -691,6 +731,8 @@ driven workflow registration.
|
||||
| `app/startup/initializers/` | Domain-scoped initialization and shutdown hooks |
|
||||
| `app/chain/agent.py` | `AgentChain(ChainBase)`: the chain-layer entry for Agent sessions; Agent runtime stays in `app/agent/` |
|
||||
| `app/runtime/config.py` | `ConfigModel`, `Settings` and deployment configuration |
|
||||
| `app/runtime/cache.py` | Cache contracts and memory policy, including strict atomic store/consume for one-shot security state |
|
||||
| `app/application/security/passkey.py` | Injected Passkey challenge cache port and one-shot challenge issue/consume policy |
|
||||
| `app/runtime/tasks.py` | TaskRegistry owner, cancellation and bounded shutdown waiting |
|
||||
| `app/runtime/execution.py` | Shared execution/thread-boundary helpers and context propagation |
|
||||
| `app/runtime/correlation.py` | Correlation ID context and propagation boundary |
|
||||
@@ -751,4 +793,4 @@ imports, entrypoint (`api`/`agent`/`monitor`/`workflow`/`doctor`) imports of
|
||||
modules only through `run_module` dispatch), and downloader SDK
|
||||
(`qbittorrentapi`, `transmission_rpc`) imports inside `app/chain`.
|
||||
|
||||
*Last Updated: 2026-08-27*
|
||||
*Last Updated: 2026-08-28*
|
||||
|
||||
@@ -130,18 +130,45 @@ adapters; it does not retain reusable repository implementations.
|
||||
`AgentChatService.delete()`, and `DeletePluginDataCommand`: bind the repository
|
||||
and UoW to one request/operation Session. Legacy plugin-facing Oper methods may
|
||||
remain temporarily, but a new endpoint or startup workflow must call `stage_*`.
|
||||
- User create/update/delete is an aggregate command owned by
|
||||
`app/application/security/user.py`. It uses one request AsyncSession/UoW, locks
|
||||
active superusers before destructive changes, and rejects removal of the last
|
||||
enabled superuser. Query ports return frozen user/auth snapshots rather than
|
||||
ORM rows.
|
||||
- The database is the final user-identity guard: `user.name` is unique;
|
||||
`UserConfig.username` cascades on user rename/delete; `PassKey.user_id`
|
||||
cascades on user delete; `(UserConfig.username, UserConfig.key)` is unique and
|
||||
non-null. A schema change to any of these constraints requires a replay-safe
|
||||
migration that repairs legacy duplicates/orphans before creating constraints.
|
||||
- DownloadHistory is projected into frozen DTOs within the adapter Session.
|
||||
Its typed query/write port and delete mutation use short Session/UoW scopes.
|
||||
TransferHistory has not completed the same migration and must be tracked
|
||||
separately rather than treating the whole History area as typed.
|
||||
|
||||
### Durable post-commit side effects
|
||||
|
||||
Business mutations that must survive process interruption stage their durable
|
||||
intent through `app/application/outbox.py` in the same Session/UoW as the
|
||||
business row. `app/db/adapters/outbox.py` is the SQLAlchemy implementation;
|
||||
startup composition supplies the repository, transaction scope and topic
|
||||
handlers.
|
||||
intent through `OutboxStager` in the same Session/UoW as the business row.
|
||||
`OutboxDispatchStore` owns separate short transactions for claim, complete and
|
||||
retry; a business Session must never call those self-committing operations.
|
||||
`app/db/adapters/outbox.py` implements both roles, and startup composition
|
||||
supplies the stager, store factory, transaction scope and topic handlers.
|
||||
|
||||
The dispatcher claims an intent with a lease, executes an idempotent handler,
|
||||
and records bounded retries or dead-letter state. The shared data-maintenance
|
||||
policy controls bounded terminal-history cleanup, with user-configurable 30-day
|
||||
Immediate delivery and the dispatcher both claim before execution. Claim is
|
||||
atomic and complete/retry is fenced by the claimed attempt, so an expired owner
|
||||
cannot settle a newer lease. `PostCommitResult` separately reports the committed
|
||||
business value plus completed and pending effects; a post-commit failure cannot
|
||||
be represented as a rollback of already committed business data.
|
||||
|
||||
This boundary is at-least-once, not exactly-once. If an external sink succeeds
|
||||
and the process stops before complete is persisted, the intent can be replayed.
|
||||
Event payloads and the host correlation context therefore carry the stable
|
||||
event key, and consumers that support deduplication should use it. Legacy
|
||||
notification plugins retain their existing method signature and remain an
|
||||
at-least-once boundary where duplicate provider delivery is possible. The
|
||||
dispatcher records bounded retries or dead-letter state.
|
||||
The shared data-maintenance policy controls bounded terminal-history cleanup,
|
||||
with user-configurable 30-day
|
||||
completed and 90-day dead-letter defaults; `0` disables either cleanup. It must
|
||||
not delete pending or leased processing rows. The `app/runtime/tasks.py`
|
||||
TaskRegistry is only the owner for in-process work and bounded shutdown waiting;
|
||||
@@ -237,6 +264,15 @@ configuration.set(username="alice", key="notification_enabled", value=True)
|
||||
The no-Session `UserConfigOper()` form is legacy plugin ABI only and must not be
|
||||
copied into host code.
|
||||
|
||||
`TransactionalUserConfigurationRepository` stages a set in a short transaction
|
||||
and publishes the process snapshot only after commit. User rename/delete is
|
||||
first completed by the user aggregate transaction through database cascades;
|
||||
post-commit publication then acquires the write lock and reloads the database
|
||||
fact source so concurrent set/rename/delete operations converge on committed
|
||||
state. If publication fails, the repository reloads that fact source instead of
|
||||
rolling back or hiding the already committed user mutation. Reads and published
|
||||
JSON values are copied so callers cannot mutate shared cache state.
|
||||
|
||||
---
|
||||
|
||||
## Settings / Environment Configuration
|
||||
@@ -280,12 +316,19 @@ def get_movie_detail(tmdb_id: int) -> dict:
|
||||
|
||||
When `REDIS_HOST` is configured, `app/modules/redis/` provides a distributed cache backend. Prefer `FileCache` for single-node deployments.
|
||||
|
||||
Security-sensitive one-shot state uses `AtomicCacheBackend`, not a concrete
|
||||
Redis helper. Its strict `store()` surfaces backend write failures and
|
||||
`consume()` atomically returns-and-removes a value. Both Memory and Redis
|
||||
backends implement this contract; Passkey receives the capability from startup
|
||||
through `PasskeyChallengeCache`, so an authentication or registration challenge
|
||||
can be accepted only once without Application knowing the configured backend.
|
||||
|
||||
---
|
||||
|
||||
## Data Lifecycle Rules
|
||||
|
||||
- **TransferHistory:** Records are inserted after every successful file transfer. Do not delete records without user confirmation.
|
||||
- **DownloadHistory:** Records are inserted when a download task is added. Linked `DownloadFiles` records track individual files within a torrent.
|
||||
- **DownloadHistory:** Records are inserted when a download task is added. Linked `DownloadFiles` records track individual files within a torrent. Host query/write callers use frozen DTOs and the typed DownloadHistory port; ORM rows remain inside the adapter Session.
|
||||
- **SystemConfig:** Values may be read and written freely at runtime. Changes to watched config keys trigger `on_config_changed()` on registered classes via `ConfigReloadMixin`.
|
||||
- **MediaServerItem:** This is a cache of the remote media server library. It is refreshed on media server sync events and can be safely cleared and rebuilt.
|
||||
|
||||
@@ -297,4 +340,4 @@ When `REDIS_HOST` is configured, `app/modules/redis/` provides a distributed cac
|
||||
- `settings.API_TOKEN` and other secret fields must not be included in log output or API responses.
|
||||
- The `config list --show-secrets` flag exists specifically to gate secret visibility in the CLI.
|
||||
|
||||
*Last Updated: 2026-08-27*
|
||||
*Last Updated: 2026-08-28*
|
||||
|
||||
Reference in New Issue
Block a user