Merge remote-tracking branch 'upstream/v3' into codex/chore/floating-build-toolchain

# Conflicts:
#	docs/architecture-optimization-checklist.md
#	docs/architecture-refactor-roadmap.md
This commit is contained in:
InfinityPacer
2026-08-28 10:51:01 +08:00
105 changed files with 6507 additions and 1691 deletions
+43 -36
View File
@@ -69,7 +69,7 @@ MoviePilot V3 已经形成较清晰的模块化单体:`foundation`、`domain`
| 指标 | 当前值 | 解释 |
|---|---:|---|
| 宿主 Python 模块 / 内部依赖边 | 853 / 6,979 | `dependency-baseline.json` 当前快照 |
| 宿主 Python 模块 / 内部依赖边 | 856 / 7,006 | `dependency-baseline.json` 当前快照 |
| 非平凡 SCC | 2 | 新增 Chain 包根环;另一个是隔离的 29 模块 TMDB 移植包环 |
| 跨层 DB 边界债务 | 0 | Application、Chain、API、Agent、Runtime、Workflow 到 DB 的受控债务均为零 |
| Model/Oper 事务债务 | 0 | 自建 Session、自动事务装饰器、直接 commit/rollback 等基线均为零 |
@@ -78,8 +78,8 @@ MoviePilot V3 已经形成较清晰的模块化单体:`foundation`、`domain`
| Python 源码量 | 约 271,400 行 | 60 个文件超过 1,000 行,14 个超过 2,000 行 |
| 长方法 | 281 个超过 80 行 | 67 个超过 150 行,23 个超过 250 行;大量是私有方法 |
| 全量 mypy 历史债务 | 11,808 / 596 文件 | strict frontier 当前覆盖 41 个文件,本批迁移路径的类型债务已清零 |
| Ruff 历史诊断 | 868 | 低水位门禁通过,但规则集只覆盖 `E4/E7/E9/F/I` |
| 覆盖率低水位 | Application 79.02%Domain 79.29% | Chain、Runtime、Agent、Adapter、Startup 未进入包级覆盖率门禁 |
| Ruff 历史诊断 | 840 | 低水位门禁通过,但规则集只覆盖 `E4/E7/E9/F/I` |
| 覆盖率低水位 | Application 79.39%Domain 79.29% | Chain、Runtime、Agent、Adapter、Startup 未进入包级覆盖率门禁 |
### 3.3 热点文件
@@ -101,7 +101,7 @@ MoviePilot V3 已经形成较清晰的模块化单体:`foundation`、`domain`
## 4. 优化清单总表
状态含义:`阻塞` 表示当前门禁已失败;`待执行` 表示尚未开始;`渐进` 表示应按触碰路径逐步收敛。
状态含义:`阻塞` 表示当前门禁已失败;`待执行` 表示尚未开始;`执行中` 表示只完成部分纵切面;`已验证` 表示实现和本地合同已落地但未在本文中声明未知的提交或 CI;`渐进` 表示应按触碰路径逐步收敛。
| ID | 优先级 | 状态 | 事项 | 目标结果 |
|---|---|---|---|---|
@@ -110,10 +110,10 @@ MoviePilot V3 已经形成较清晰的模块化单体:`foundation`、`domain`
| ARCH-102 | P1 | 已交付 | 将 Transfer pending 升级为真实 E3 状态机 | `e9de149db``a2e249f20` 已推送;Unit Tests `33092427327`、Pylint `33092427348` 全绿,崩溃结果未知时进入人工确认 |
| ARCH-103 | P1 | 执行中 | 类型化 Chain/Agent 数据 Port 与 DTO | 宿主主路径不再注入无 Session Oper,不向入口泄漏 ORM |
| ARCH-104 | P1 | 待执行 | 收口跨多次写入的业务事务 | 站点/规则引用清理可整体回滚或幂等恢复 |
| ARCH-105 | P1 | 待执行 | 明确 post-commit 与 Outbox 完成语义 | 业务提交、后置效果 pending”可被调用方正确识别 |
| ARCH-105 | P1 | 已验证 | 明确 post-commit 与 Outbox 完成语义 | 业务提交、effect 完成/pending 可区分;stager/store 分离且 claim/settlement 受 fencing,外部 sink 仍承担 at-least-once 幂等边界 |
| ARCH-106 | P1 | 待执行 | 让线程/队列/日志 writer 由 bootstrap/lifecycle 显式构造 | 导入或普通 Chain 构造不再启动进程资源 |
| ARCH-107 | P1 | 待执行 | 消除 Chain SCC,强化循环门禁 | SCC 只剩精确豁免的 TMDB 移植包环 |
| ARCH-108 | P1 | 执行 | 决策并收口 Application/Chain 到 Adapter 与 HTTP 边界 | 依赖倒置有明确例外、低水位迁移顺序 |
| ARCH-108 | P1 | 执行 | 决策并收口 Application/Chain 到 Adapter 与 HTTP 边界 | Passkey 缓存纵切面已验证,其余 Adapter/HTTP/DNS 债务继续按低水位迁移 |
| ARCH-109 | P1 | 待执行 | 按用例拆分超大 Chain、Scheduler 和厚 API | 稳定 Facade 保留,决策/I/O/状态/生命周期各有 owner |
| ARCH-110 | P1 | 待执行 | Module/Event Contract 分可信级执行 | 宿主 provider 严格,第三方插件仍兼容诊断 |
| ARCH-111 | P1 | 待执行 | 升级复杂度、类型、覆盖率和并发原语门禁 | 高风险私有路径也进入只降不增的治理面 |
@@ -268,14 +268,13 @@ MoviePilot V3 已经形成较清晰的模块化单体:`foundation`、`domain`
**问题与证据**
- `app/application/chain/data.py:14-29,134-176` 的 Oper factory 和 getter 基本都`Any`
`app/application/agentdata.py:91-120` 通过 `__dict__.update()` 动态组装端口。
- `app/startup/initializers/modules.py:848-863,896-910` 仍向生产 Chain/Agent 注入多个无 Session Oper。
- `app/application/chain/data.py` 的 Site/Subscribe/TransferHistory factory 仍`Any`
`app/application/agentdata.py` 通过 `__dict__.update()` 动态组装未迁移端口。
- `app/startup/initializers/modules.py` 仍向生产 Chain/Agent 注入部分无 Session Oper。
- 无 Session Oper 会为单次调用独立创建事务;一个业务操作的“查询后更新”可能被拆成多个事务。
- Workflow query 已在 S1-L2 迁入冻结 DTO adapter-owned Session 投影;其余 Chain/Agent raw data port
返回 `Any`/ORMSubscription mutation 内部也消费 ORM因而仍存在 Session 生命周期外
detached/lazy-load 的潜在风险。公开 Subscription、Site、History QueryService 已经投影 DTO
属于完成项,不应重做。
- Workflow、User 和 DownloadHistory 已迁入冻结 DTO adapter-owned Session 投影;
剩余 Chain/Agent raw data port 仍可返回 `Any`/ORMSubscription mutation 内部也消费 ORM
因而尚有 Session 生命周期外 detached/lazy-load 风险。
- S1-L2 由 `b4f873654``a01a35bcb` 交付;精确 head SHA 的 Unit Tests `33098869736`
Pylint `33098869837` 全绿,Application 覆盖率低水位提升并固化至 `78.78%`。该证据只完成
Workflow query 纵切面,不能替代 S1-L3 对其余 Chain/Agent raw data port 的清零。
@@ -293,15 +292,20 @@ MoviePilot V3 已经形成较清晰的模块化单体:`foundation`、`domain`
伪注入;Workflow 执行服务只在 Application owner 配置一次,不再重复注册到 `ChainDataPorts`
- [x] DownloadFailure/MediaServer 两个 registry 字段改用冻结 DTO 与 typed Repository factory
ORM 不越过短 Session,媒体库远端枚举期间不持有事务,旧 Oper/Compat 与公开 Chain ABI 保持不变。
- [x] User Chain/Agent/认证查询改用冻结 `UserSnapshot`/`UserAuthSnapshot`;创建、更名和
删除由请求级 UoW 原子提交,用户名唯一约束、最后一个启用超级管理员保护与
UserConfig/PassKey 级联约束共同守住身份聚合。
- [x] DownloadHistory 查询和写入改用冻结 DTO、typed Port 与短 Session adapter,删除在单一
UoW 中处理;TransferHistory 仍是 raw portHistory 整体仍在执行中。
- [ ] `ChainDataPorts`/`AgentDataPorts` 可暂时保留为兼容聚合器,但字段必须显式、可类型检查。
- [ ] 以一个业务纵切面迁移并验证后,再迁移下一组,禁止一次替换所有 Oper。
- [ ] 增加 AST 门禁,禁止向 `ChainDataPorts``AgentDataPorts` 和新的 canonical use-case service
注入裸 OperSystemConfig singleton、legacy transaction runner 等兼容边界使用精确 allowlist。
**首批建议**
**后续顺序**
1. Workflow query DTO
2. Chain/Agent 的 Subscribe/History/User/Site raw port。
1. 完成 TransferHistory,收口 History 剩余半边
2. Chain/Agent 的 Subscribe/Site raw port。
3. Subscription mutation 与站点、规则组引用更新。
4. Agent 数据能力。
@@ -350,28 +354,29 @@ MoviePilot V3 已经形成较清晰的模块化单体:`foundation`、`domain`
### ARCH-105 明确 post-commit 与 Outbox 完成语义
**问题与证据**
**已实现事实与语义边界**
- `app/application/outbox.py:167-192` 在业务提交后直接执行 `after_commit()`、即时 publish 和完成标记;
某一步抛错时,调用方可能收到失败,但业务行和 intent 已经提交
- 通用 `DurableEventCommand` commit 后没有先调用已经定义的 `claim_by_event_key()`dispatcher 可在
commit 与请求线程即时 publish 之间先 claim 并投递,随后请求线程再次 publish,形成双发窗口
`app/application/subscription/complete.py:111-145` 已提供先 claim 的正确参考。
- 下载历史及事件 intent 已原子提交;通知在 commit 后同步执行,模块后处理和字幕再投进线程池
进程在提交与这些动作完成之间退出时,未持久化的动作不会自动恢复。
- `SqlAlchemyOutboxRepository` 同时提供不提交的 `stage()` 和内部自提交的 dispatcher 方法,
事务所有权没有由类型清楚表达。
- `OutboxStager` 在业务 Session 中 stage/flush`OutboxDispatchStore` 的 claim、complete 和 retry
每次使用独立短事务,业务与 dispatcher 的事务所有权已由类型分开
- 请求线程即时投递与 dispatcher 均先按稳定 event key 原子 claim;同一 lease 期间只有
一个 owner,过期 owner 不能用旧 attempt 覆盖新 owner 的 complete/retry
- `PostCommitResult`/`PostCommitEffectError` 保留“业务已提交”事实,并逐项列出已完成和
pending effect,后置效果失败不伪装成业务回滚
- lease/attempt fencing 只保护宿主的认领与结算。外部调用成功但 complete 落库前崩溃时,
intent 仍会重放;因此交付承诺是 at-least-once。事件载荷和宿主 correlation context
携带稳定 event key,支持幂等的消费者应使用它;旧通知插件保持原方法签名,不能宣称外部
provider 已获得 exactly-once 或统一幂等能力。
**目标与步骤**
- [ ] Command 返回结构化结果:业务是否提交、哪些后置效果完成、哪些处于 pending。
- [ ] commit 后先取得 delivery lease;未取得时跳过请求线程直投,确保与 dispatcher 排他。
- [ ] 每个 post-commit effect 使用独立 intent 隔离和结算,避免效果 A 失败导致效果 B 被一起重放。
- [x] Command 返回结构化结果:业务是否提交、哪些后置效果完成、哪些处于 pending。
- [x] commit 后先取得 delivery lease;未取得时跳过请求线程直投,确保与 dispatcher 排他。
- [x] 每个 post-commit effect 使用独立 intent 隔离和结算,避免效果 A 失败导致效果 B 被一起重放。
单个外部效果仍是 at-least-once,必须有稳定幂等键和幂等 handler/消费者。
- [ ] 按完成承诺、可重建性、外部不可逆性和业务重要性划分 E0-E3;用户可见性只是因素之一。
- [ ] 拆分 `OutboxStager``OutboxDispatchStore`,避免业务 Session 调用自提交方法。
- [ ] 在 commit、claim、publish、complete、通知和任务提交各断点注入异常/崩溃,验证声明等级与实际恢复一致。
- [ ] 增加请求线程即时投递与 dispatcher 并发竞争测试,以及“外部调用成功、complete 前崩溃”的重放测试。
- [x] 拆分 `OutboxStager``OutboxDispatchStore`,避免业务 Session 调用自提交方法。
- [x] 在 commit、claim、publish、complete、通知和任务提交各断点注入异常/崩溃,验证声明等级与实际恢复一致。
- [x] 增加请求线程即时投递与 dispatcher 并发竞争测试,以及“外部调用成功、complete 前崩溃”的重放测试。
```bash
.venv/bin/python -m pytest \
@@ -469,7 +474,8 @@ MoviePilot V3 已经形成较清晰的模块化单体:`foundation`、`domain`
`security/passkey.py``backup.py``image.py``rss.py``security/cookie.py`
- `app/chain` 有 8 个文件、13 条直接 Adapter 导入,使用 `RequestUtils`、Browser、Cloudflare、
CookieCloud、ServerHelper 等具体能力。
- Passkey Application 服务直接判断 Redis 后端并调用 `RedisHelper.pop()`,安全策略识别了具体实现。
- Passkey Application 已改为消费启动注入的 `PasskeyChallengeCache`,不再判断 Redis 或导入
具体 cache adapterMemory/Redis 均实现严格 `AtomicCacheBackend.store/consume`
- 审计时 LLM streaming、第三方 SDK、移植库和本地控制面没有精确例外表;S0-L2.4b 已建立
66 条完整 egress identity 与 zero-growth policy,其中 11 条普通 HTTP/Session bridge 和 1 条
Application DNS I/O 是清零债务;每条初始边另有独立指纹上界,不能靠同时刷新 baseline/policy
@@ -479,7 +485,7 @@ MoviePilot V3 已经形成较清晰的模块化单体:`foundation`、`domain`
- [x] 建立 Application/Chain 原始 Adapter 直连事实与精确临时 policy,冻结新增、替换和陈旧条目。
- [x] 建立全宿主 direct egress 事实;SDK/stream/vendor/local-control 例外精确到 bindings/uses 指纹。
- [ ] 将 Passkey 原子领取提升为 runtime cache contract,由 Memory/Redis backend 分别实现。
- [x] 将 Passkey 原子领取提升为 runtime cache contract,由 Memory/Redis backend 分别实现。
- [ ] 为 Backup 定义 Application-owned artifact store Port,由 startup 注入文件系统实现。
- [ ] 将 policy 中 11 条普通 HTTP/Session bridge 债务迁移到统一网络能力并把目标收缩为空。
- [ ] 为 Application SSRF 校验注入 DNS 解析 Port,清除 `socket.getaddrinfo` 直接 I/O。
@@ -611,7 +617,8 @@ MoviePilot V3 已经形成较清晰的模块化单体:`foundation`、`domain`
1. ARCH-102 设计并迁移 Transfer E3 状态机。
2. 以站点/规则引用清理为 ARCH-103/104 的第一个 typed Port + UoW 纵切面。
3. 完成 ARCH-105 post-commit 结构化结果Outbox 角色拆分
3. ARCH-105 已完成 post-commit 结构化结果Outbox 角色拆分和 claim fencing
后续只继续清理独立 intent 与更完整的崩溃矩阵。
**退出条件**:故障注入覆盖清单列出的崩溃窗口;schema 有 migration;主路径不再 fail-open;调用方能区分
业务提交与后置效果 pending。
@@ -620,7 +627,7 @@ MoviePilot V3 已经形成较清晰的模块化单体:`foundation`、`domain`
1. ARCH-106 将日志、消息队列和主循环 gateway 纳入 lifecycle。
2. ARCH-107 拆出 `chain/base.py` 并消除新增 SCC。
3. ARCH-108 先迁移 PasskeyBackup按风险迁移外部调用。
3. ARCH-108 Passkey 缓存边界已验证;下一纵切面是 Backup之后按风险迁移外部调用。
**退出条件**:冷导入不启动线程;SCC 只剩 TMDB 精确豁免;Application/Chain 到 Adapter 的债务只降不增。
+2 -2
View File
@@ -704,8 +704,8 @@ flowchart LR
| 指标 | 当前值 |
|---|---:|
| Python 模块 | 853 |
| 内部导入边 | 6,979 |
| Python 模块 | 856 |
| 内部导入边 | 7,006 |
| 非平凡 SCC | 2`ARCH-107` 临时 Chain 包根环;精确 containment 的 TMDB 移植包环) |
| Direct egress | 6612 条待迁移债务,54 条精确 containment |
| Module Contract V2 spec | 217(其中 215 个进入 `run_module` 观察面) |
+5 -5
View File
@@ -103,14 +103,14 @@ canonical 主程序;兼容只经统一 Compat/SDK 门面提供。
| S1-L3.2 Chain registry/DI | `ACTIVE` | S1-L3.1 | 显式类型化 factory,删除 PortProxy 与失效的双重注入,构造器注入真实控制调用 |
| S1-L3.2.1 Registry hygiene | `DELIVERED` | S1-L3.1 | `ac7a20132`:删除零消费者 PortProxy/动态转发和 `ChainRuntimeContext.data_ports` 伪注入;Workflow 退出 Chain registry,只保留 Application owner 单一配置入口;Unit Tests `33120205586`、Pylint `33120205581` 全绿 |
| S1-L3.3 DownloadFailure/MediaServer | `DELIVERED` | S1-L3.2 | `5fb62108a`:两组 raw factory 已替换为冻结 DTO/typed Port;失败冷却在 Session 内投影,媒体库查询只返回标量且每个 upsert/cleanup 独立短事务,远端枚举不持有 Session;旧 Oper 与插件可见 Chain ABI 保持不变;Unit Tests `33127544925`、Pylint `33127544927` 全绿,Application 覆盖率低水位提升至 `78.95%` |
| S1-L3.4 User | `PLANNED` | S1-L3.3 | 认证、偏好与渠道绑定投影冻结快照,User Chain/Agent 不接收 ORM |
| S1-L3.5 History | `PLANNED` | S1-L3.4 | Download/Transfer history 统一 typed query/mutation,删除下载历史双事务 fail-open |
| S1-L3.4 User | `VERIFIED` | S1-L3.3 | User Chain/Agent/认证改用冻结 typed snapshot;创建、更名、删除与最后一个启用超级管理员保护归并到单 UoW;用户名唯一索引及 UserConfig/PassKey 级联迁移落地,UserConfig 在 commit 后持写锁重载数据库事实源并发布内存快照 |
| S1-L3.5 History | `IN_PROGRESS` | S1-L3.4 | DownloadHistory 已迁入冻结 DTO、typed query/write Port 与短 Session adapter,下载历史删除不再拆成 fail-open 双事务;TransferHistory 半边尚未完成,本项不宣称整体交付 |
| S1-L3.6 Site | `PLANNED` | S1-L3.5 | 复用 Site query/health,补齐同步 typed commandSession 内完成 DTO 投影 |
| S1-L3.7 Subscription | `PLANNED` | S1-L3.6 | Chain/Workflow/interaction 全部消费 typed query/command;完成后进入 S1-L4 原子事务收口 |
| S1-L3.8 Agent/Transfer locator gate | `PLANNED` | S1-L3.7 | 删除 AgentDataPorts 与 Chain locator 跨层泄漏,AST 门禁确认 canonical 无 raw getter/Oper/Any |
| S1-L4 Subscription mutation UoW | `PLANNED` | S1-L3 | Subscription mutation 不跨 Session 传 ORM,正式写路径一个 UoW,旧自动事务入口退出 canonical 路径 |
| S1-L5 站点/规则引用原子清理 | `PLANNED` | S1-L4 | SystemConfig+Subscribe 同事务更新,commit 后快照原子发布,并发/故障注入无部分状态 |
| S1-L6 Outbox 完成语义 | `PLANNED` | S0 | claim 竞争双发清零;业务提交与 effect pending 可区分;stager/store 分离;handler 幂等与崩溃测试完整 |
| S1-L6 Outbox 完成语义 | `VERIFIED` | S0 | 事务内 `OutboxStager` 与独立短事务 `OutboxDispatchStore` 已分离;即时投递与 dispatcher 均先 claimcomplete/retry 受 attempt fencing`PostCommitResult` 区分已提交业务、已完成与 pending effect。事件载荷和宿主 correlation context 携带稳定 event key;旧通知插件保持原签名并承认 at-least-once 重复边界 |
### S2:进程生命周期、循环与 Adapter 边界
@@ -122,7 +122,7 @@ canonical 主程序;兼容只经统一 Compat/SDK 门面提供。
| S2-L1 日志/消息资源显式生命周期 | `PLANNED` | S0 | import 和非消息 Chain 构造零新增线程;bootstrap 显式创建,失败和正常关闭均收口 |
| S2-L2 ChainBase 与 SCC 清零 | `PLANNED` | S0-L2.2 | canonical `app.chain.base` 落地,包根无 eager/重复导出,宿主包根导入清零,Chain SCC 消失 |
| S2-L3 GlobalVar/provider 注册收口 | `PLANNED` | S2-L1 | `global_vars` canonical 消费清零,provider 注册进入显式装配阶段并可 reset;Legacy 入口精确保留 |
| S2-L4 Passkey 缓存边界 | `PLANNED` | S0-L2.4 | Application 不识别 Redis;原子 consume 由 runtime cache contract + backend 实现 |
| S2-L4 Passkey 缓存边界 | `VERIFIED` | S0-L2.4 | `PasskeyChallengeCache` 由 startup 注入,Application 不识别 Redis;严格 `AtomicCacheBackend.store/consume` 由 Memory/Redis backend 分别实现,challenge 仅能被原子领取一次 |
| S2-L5 Backup artifact Port | `PLANNED` | S0-L2.4 | Application 不构造 `BackupFiles`,文件 I/O 由注入 Adapter 拥有 |
| S2-L6 Application Adapter/DNS 债务清零 | `PLANNED` | S2-L4,S2-L5 | Application 到具体 Adapter 的未批准边归零,SSRF DNS I/O 进入注入 Port,批准通用机制有精确规则和门禁 |
| S2-L7 Chain Adapter/宿主 HTTP 债务清零 | `PLANNED` | S2-L6 | Chain 具体 Adapter 与 11 条普通 direct HTTP/Session bridge 归零;SDK/stream/vendor 例外保持精确 containment |
@@ -153,7 +153,7 @@ canonical 主程序;兼容只经统一 Compat/SDK 门面提供。
| S4-L2 Event strict contract | `PLANNED` | S0-L2.6,S1-L6 | 宿主事件输入/输出按风险 strict,诊断例外只属于第三方插件兼容 |
| S4-L3 Complexity v2 | `PLANNED` | S3 | 私有方法、class/file、圈复杂度进入门禁;所有超限通过职责拆分归零 |
| S4-L4 全量 mypy 清零 | `PLANNED` | S3,S4-L1,S4-L2 | `mypy-baseline.json` 归零并删除债务接受路径,全宿主 strict 类型通过 |
| S4-L5 Ruff 治理债务清零 | `PLANNED` | S3 | 当前受控 868 条诊断归零,规则集扩展经过独立审查且新增诊断为零 |
| S4-L5 Ruff 治理债务清零 | `PLANNED` | S3 | 当前受控 840 条诊断归零,规则集扩展经过独立审查且新增诊断为零 |
| S4-L6 Coverage/并发/质量证据 | `PLANNED` | S3,S4-L1,S4-L2 | 高风险包纳入 coverageraw concurrency 分类清零;Module Quality 有真实 evidence test |
### S5Plugin、Agent、Domain、Startup 与最终收口
+59 -17
View File
@@ -64,16 +64,17 @@ to make the directory tree look symmetrical.
| `app/application/subscription/` | Subscription use cases: `write.py` owns media-to-row translation and the write port; `contract.py` owns shared metadata/media-key projection; query, mutation, deletion, identity and search stay in their single-word modules |
| `app/application/search/` | Search state and later search-plan use cases |
| `app/application/download/` | Download task querying/control and selection use cases; `failures.py` owns the frozen failure-cooldown write/query DTOs and persistence Port |
| `app/application/history.py` | History use cases and persistence contracts; DownloadHistory owns frozen query/write DTOs and typed ports, while TransferHistory remains a separate unfinished migration surface |
| `app/application/music/` | Multi-source music catalog orchestration |
| `app/application/chain/` | Injectable Chain runtime capabilities: `context.py` owns the runtime dependency aggregate, `data.py` owns named persistence ports, and `events.py` owns durable event write contracts plus replayable payload conversion |
| `app/application/agentdata.py` | Named Agent data ports; canonical Agent consumers use `get_agent_*_port()` and do not alias legacy proxies to Oper classes |
| `app/application/outbox.py` | Durable intent and Outbox repository/dispatcher contracts for post-commit side effects |
| `app/application/outbox.py` | Durable intent, transaction-only stager, short-transaction dispatch store, claim fencing and structured post-commit result contracts |
| `app/application/transfer/` | Durable transfer use cases: `workflow.py` owns admission/planning/queue behavior; `execution.py` owns stable operation identity, step/checkpoint state, retry/manual-review commands and terminal-settlement DTOs |
| `app/application/plugin/` | Plugin market catalog, installation command, installed-plugin identity contract and startup migration, runtime port, folder operations and dynamic-route use cases; filenames remain single words (`catalog.py`, `identity.py`, `migration.py`, `install.py`, `runtime.py`, `folders.py`, `routes.py`) |
| `app/application/server/` | MoviePilot Server reporting and sharing use cases; local data readers and transport callbacks are injected by startup |
| `app/application/site/` | Configured site catalog, authentication level and index-resource capability; the generated extension and its data bundle stay together here |
| `app/application/messaging/` | Message rendering/routing, interactions and the Agent-to-message bridge: `ingress.py` owns the single channel-to-host loopback boundary; `interaction.py` shared interaction contracts and view helpers; `router.py` unified interaction priority and callback dispatch; `site.py`/`subscribe.py`/`skill.py` per-command sessions, input parsing and views; `media.py` media interaction state while the business workflow stays in `MediaInteractionChain`; `plugin.py` plugin input capture and plugin button callbacks; `agent.py` agent choice state, callback protocol and WebAgent bridge; `message.py` notification rendering, templates and queue. Not a public SDK recommended for direct plugin use |
| `app/application/security/` | Authentication, authorization, cookies, passkeys, OTP/two-factor, path/URL safety, SSRF and signing policy |
| `app/application/security/` | Authentication, authorization, frozen user/auth projections, atomic user aggregate commands, per-user configuration publication, cookies, passkeys, OTP/two-factor, path/URL safety, SSRF and signing policy |
Application services may use domain rules and runtime contracts. They own the
persistence Protocol needed by a use case, but must not import `app.db`,
@@ -457,10 +458,14 @@ SQLAlchemy models stay under `app/db/models/`; the data access classes live in
carries the role. Two verified aggregation exceptions exist: the site family
(`Passkey`, `SiteIcon`, `SiteStatistic`, `SiteUserData`) is consolidated in
`oper/site.py`, and `AgentTaskRun` lives in `oper/agenttask.py`. DB adapters use
Oper classes instead of issuing SQLAlchemy queries directly. Application and
Chain code reaches persistence through named Ports/Protocols; concrete DB adapters
are the layer that adapts those Ports to Oper classes. Every schema change
requires an Alembic migration under `database/versions/`.
Oper classes for ordinary entity access. Adapter-owned cross-row locks and
compare-and-set transitions may issue focused SQLAlchemy statements when the
atomic persistence invariant cannot be expressed by an entity Oper; those
statements stay private to the adapter and require concurrency tests.
Application and Chain code reaches persistence through named Ports/Protocols;
concrete DB adapters are the only layer that adapts those Ports to Oper/Session
mechanics. Every schema change requires an Alembic migration under
`database/versions/`.
Oper classes take and return persistence values, not domain objects. Translating
`MediaInfo` / `MetaBase` into a row is business logic and belongs in
@@ -477,17 +482,40 @@ path cannot forget them. Identity representation rules themselves
alongside the two identity mixins; `app/domain/media.py` keeps only source
policy. `app/db` therefore has no dependency on `app/domain`.
User identity is a single aggregate boundary. `app/application/security/user.py`
owns frozen user/auth snapshots and the atomic create/update/delete command;
`app/db/adapters/user.py` binds each mutation to one request UoW and locks the
active-superuser set before a destructive change. The database enforces unique
user names and cascades rename/delete to `UserConfig` and delete to `PassKey`.
The configured user-configuration repository publishes its in-memory snapshot
only after the user transaction commits, and reloads from the database if
publication fails.
Download history is the verified half of the History migration:
`app/application/history.py` owns frozen `DownloadHistorySnapshot` /
`DownloadFileSnapshot` values and typed query/write ports;
`app/db/adapters/history/download.py` performs projection and mutations in short
Session scopes. TransferHistory still uses its existing ports, so this must not
be described as completion of the entire History boundary.
Durable post-commit side effects have a separate boundary:
- `app/application/outbox.py` owns the Outbox intent, repository and dispatcher
contracts. An Application command stages the business mutation and its durable
intent in the same transaction.
- `app/db/adapters/outbox.py` implements the persistence port with SQLAlchemy;
- `app/application/outbox.py` separates `OutboxStager`, which only stages in the
caller's business transaction, from `OutboxDispatchStore`, whose claim,
complete and retry operations own independent short transactions.
- `app/db/adapters/outbox.py` implements both roles with SQLAlchemy;
`app/startup/composition/subscription.py` and the other composition modules
provide the concrete repository, UoW and handlers.
- The dispatcher claims an intent with a lease, executes the topic handler, and
records retry/dead-letter state. Handlers must be idempotent and must not rely
on a live request object.
inject the stager, dispatch-store factory, UoW and handlers.
- Immediate request-thread delivery and the dispatcher both claim before
calling a handler. Lease acquisition is atomic, while complete/retry is fenced
by the claimed attempt so an expired owner cannot settle a newer claim.
- `PostCommitResult` distinguishes the committed business value from completed
and pending effects. This does not provide exactly-once delivery: a process may
stop after an external sink succeeds but before complete is persisted. Outbox
delivery is therefore at-least-once. Event payloads and the host correlation
context carry the stable event key, and consumers that support deduplication
should use it. Legacy notification plugins retain their existing method
signature, so the host must not claim provider-level exactly-once delivery.
- Terminal history is part of the shared data-maintenance policy and is cleaned
in bounded daily batches only when that policy is enabled. Completed intents
default to 30-day retention and dead letters to 90 days; both values are
@@ -546,6 +574,10 @@ expired claimed task remains exclusively owned by fenced recovery APIs.
- Startup registers concrete cache factories before decorated business modules
are imported. Cache contracts remain in `app/runtime/cache.py`; Redis/file
implementations remain in `app/adapters/cache/backends.py`.
- Security-sensitive one-shot state uses the strict `AtomicCacheBackend`
`store/consume` contract. Memory and Redis implement atomic consume; startup
injects that capability through `PasskeyChallengeCache`, so Passkey
Application code never identifies or imports the Redis implementation.
- `app/runtime/log.py` is a dependency leaf with no `app.*` imports. Foundation
emits no runtime logs; upper-layer owners decide whether failures are
operationally relevant.
@@ -573,6 +605,9 @@ expired claimed task remains exclusively owned by fenced recovery APIs.
may use `app.core`, `app.helper`, `app.utils` or `app.log`.
- New plugins use `app.sdk`. In DEBUG mode, a legacy plugin import remains
functional and emits one actionable warning per plugin and legacy module.
- Plugin compatibility changes belong only in curated SDK/Legacy exports or the
exact Compat manifest. `app/plugins/**` contains runtime plugin copies and is
excluded from host refactors, dependency baselines and ownership migrations.
- Delayed imports are not accepted as a way to hide dependency cycles.
### Dependency facts and semantic policy
@@ -677,8 +712,13 @@ driven workflow registration.
| `app/application/download/failures.py` | Frozen download-failure cooldown write/query DTOs and Chain persistence Port |
| `app/db/adapters/download.py` | Short-session download-failure snapshot and mutation adapter |
| `app/db/adapters/mediaserver.py` | Per-operation media-server cache query/upsert/cleanup transaction adapter |
| `app/application/outbox.py` | Durable intent, topic handler and Outbox repository contracts |
| `app/db/adapters/outbox.py` | SQLAlchemy Outbox persistence, claim/lease and retry state adapter |
| `app/application/history.py` | History use cases; frozen DownloadHistory DTOs and typed query/write ports, with TransferHistory migration still pending |
| `app/db/adapters/history/download.py` | DownloadHistory short-session snapshot, query and mutation adapter |
| `app/application/security/user.py` | Frozen user/auth projections and atomic user aggregate service contracts |
| `app/db/adapters/user.py` | User projection plus request-UoW mutation adapter |
| `app/db/adapters/configuration.py` | Commit-after UserConfig snapshot publication and fact-source reload adapter |
| `app/application/outbox.py` | Durable intent, stager/store, claim fencing, topic handler and structured post-commit contracts |
| `app/db/adapters/outbox.py` | SQLAlchemy Outbox transaction-only stagers and short-transaction claim/settlement stores |
| `app/application/chain/events.py` | Chain durable-event write port, settlement projection and replayable payload conversion |
| `app/application/transfer/workflow.py` | Transfer task, durable admission, versioned planning input/checkpoint contracts and queue use case |
| `app/db/adapters/transfer/admission.py` | SQLAlchemy admission/checkpoint persistence, CAS state transition and detached snapshot adapter |
@@ -691,6 +731,8 @@ driven workflow registration.
| `app/startup/initializers/` | Domain-scoped initialization and shutdown hooks |
| `app/chain/agent.py` | `AgentChain(ChainBase)`: the chain-layer entry for Agent sessions; Agent runtime stays in `app/agent/` |
| `app/runtime/config.py` | `ConfigModel`, `Settings` and deployment configuration |
| `app/runtime/cache.py` | Cache contracts and memory policy, including strict atomic store/consume for one-shot security state |
| `app/application/security/passkey.py` | Injected Passkey challenge cache port and one-shot challenge issue/consume policy |
| `app/runtime/tasks.py` | TaskRegistry owner, cancellation and bounded shutdown waiting |
| `app/runtime/execution.py` | Shared execution/thread-boundary helpers and context propagation |
| `app/runtime/correlation.py` | Correlation ID context and propagation boundary |
@@ -751,4 +793,4 @@ imports, entrypoint (`api`/`agent`/`monitor`/`workflow`/`doctor`) imports of
modules only through `run_module` dispatch), and downloader SDK
(`qbittorrentapi`, `transmission_rpc`) imports inside `app/chain`.
*Last Updated: 2026-08-27*
*Last Updated: 2026-08-28*
+52 -9
View File
@@ -130,18 +130,45 @@ adapters; it does not retain reusable repository implementations.
`AgentChatService.delete()`, and `DeletePluginDataCommand`: bind the repository
and UoW to one request/operation Session. Legacy plugin-facing Oper methods may
remain temporarily, but a new endpoint or startup workflow must call `stage_*`.
- User create/update/delete is an aggregate command owned by
`app/application/security/user.py`. It uses one request AsyncSession/UoW, locks
active superusers before destructive changes, and rejects removal of the last
enabled superuser. Query ports return frozen user/auth snapshots rather than
ORM rows.
- The database is the final user-identity guard: `user.name` is unique;
`UserConfig.username` cascades on user rename/delete; `PassKey.user_id`
cascades on user delete; `(UserConfig.username, UserConfig.key)` is unique and
non-null. A schema change to any of these constraints requires a replay-safe
migration that repairs legacy duplicates/orphans before creating constraints.
- DownloadHistory is projected into frozen DTOs within the adapter Session.
Its typed query/write port and delete mutation use short Session/UoW scopes.
TransferHistory has not completed the same migration and must be tracked
separately rather than treating the whole History area as typed.
### Durable post-commit side effects
Business mutations that must survive process interruption stage their durable
intent through `app/application/outbox.py` in the same Session/UoW as the
business row. `app/db/adapters/outbox.py` is the SQLAlchemy implementation;
startup composition supplies the repository, transaction scope and topic
handlers.
intent through `OutboxStager` in the same Session/UoW as the business row.
`OutboxDispatchStore` owns separate short transactions for claim, complete and
retry; a business Session must never call those self-committing operations.
`app/db/adapters/outbox.py` implements both roles, and startup composition
supplies the stager, store factory, transaction scope and topic handlers.
The dispatcher claims an intent with a lease, executes an idempotent handler,
and records bounded retries or dead-letter state. The shared data-maintenance
policy controls bounded terminal-history cleanup, with user-configurable 30-day
Immediate delivery and the dispatcher both claim before execution. Claim is
atomic and complete/retry is fenced by the claimed attempt, so an expired owner
cannot settle a newer lease. `PostCommitResult` separately reports the committed
business value plus completed and pending effects; a post-commit failure cannot
be represented as a rollback of already committed business data.
This boundary is at-least-once, not exactly-once. If an external sink succeeds
and the process stops before complete is persisted, the intent can be replayed.
Event payloads and the host correlation context therefore carry the stable
event key, and consumers that support deduplication should use it. Legacy
notification plugins retain their existing method signature and remain an
at-least-once boundary where duplicate provider delivery is possible. The
dispatcher records bounded retries or dead-letter state.
The shared data-maintenance policy controls bounded terminal-history cleanup,
with user-configurable 30-day
completed and 90-day dead-letter defaults; `0` disables either cleanup. It must
not delete pending or leased processing rows. The `app/runtime/tasks.py`
TaskRegistry is only the owner for in-process work and bounded shutdown waiting;
@@ -237,6 +264,15 @@ configuration.set(username="alice", key="notification_enabled", value=True)
The no-Session `UserConfigOper()` form is legacy plugin ABI only and must not be
copied into host code.
`TransactionalUserConfigurationRepository` stages a set in a short transaction
and publishes the process snapshot only after commit. User rename/delete is
first completed by the user aggregate transaction through database cascades;
post-commit publication then acquires the write lock and reloads the database
fact source so concurrent set/rename/delete operations converge on committed
state. If publication fails, the repository reloads that fact source instead of
rolling back or hiding the already committed user mutation. Reads and published
JSON values are copied so callers cannot mutate shared cache state.
---
## Settings / Environment Configuration
@@ -280,12 +316,19 @@ def get_movie_detail(tmdb_id: int) -> dict:
When `REDIS_HOST` is configured, `app/modules/redis/` provides a distributed cache backend. Prefer `FileCache` for single-node deployments.
Security-sensitive one-shot state uses `AtomicCacheBackend`, not a concrete
Redis helper. Its strict `store()` surfaces backend write failures and
`consume()` atomically returns-and-removes a value. Both Memory and Redis
backends implement this contract; Passkey receives the capability from startup
through `PasskeyChallengeCache`, so an authentication or registration challenge
can be accepted only once without Application knowing the configured backend.
---
## Data Lifecycle Rules
- **TransferHistory:** Records are inserted after every successful file transfer. Do not delete records without user confirmation.
- **DownloadHistory:** Records are inserted when a download task is added. Linked `DownloadFiles` records track individual files within a torrent.
- **DownloadHistory:** Records are inserted when a download task is added. Linked `DownloadFiles` records track individual files within a torrent. Host query/write callers use frozen DTOs and the typed DownloadHistory port; ORM rows remain inside the adapter Session.
- **SystemConfig:** Values may be read and written freely at runtime. Changes to watched config keys trigger `on_config_changed()` on registered classes via `ConfigReloadMixin`.
- **MediaServerItem:** This is a cache of the remote media server library. It is refreshed on media server sync events and can be safely cleared and rebuilt.
@@ -297,4 +340,4 @@ When `REDIS_HOST` is configured, `app/modules/redis/` provides a distributed cac
- `settings.API_TOKEN` and other secret fields must not be included in log output or API responses.
- The `config list --show-secrets` flag exists specifically to gate secret visibility in the CLI.
*Last Updated: 2026-08-27*
*Last Updated: 2026-08-28*