mirror of
https://github.com/jxxghp/MoviePilot.git
synced 2026-09-04 23:17:20 +08:00
docs: sync architecture docs with current structure
This commit is contained in:
@@ -7,7 +7,7 @@
|
||||
> [`docs/rules/04-design-patterns.md`](rules/04-design-patterns.md) 为准,本文与其保持一致;
|
||||
> 如出现差异,以规则文档为准。
|
||||
>
|
||||
> *Last Updated: 2026-08-21*
|
||||
> *Last Updated: 2026-08-24*
|
||||
|
||||
---
|
||||
|
||||
@@ -110,10 +110,9 @@ flowchart TB
|
||||
PluginPkg --> Sdk
|
||||
|
||||
Chain -->|run_module 分发| Modules
|
||||
Chain --> App
|
||||
Chain -->|经应用端口 / Oper 适配| Db
|
||||
Chain -->|Application service / 命名数据端口| App
|
||||
App --> Modules
|
||||
App -->|应用端口 / Oper 适配| Db
|
||||
App -->|Protocol / 持久化端口| Db
|
||||
|
||||
Modules --> Domain
|
||||
App --> Domain
|
||||
@@ -124,7 +123,7 @@ flowchart TB
|
||||
Runtime --> Foundation
|
||||
Adapters --> Domain
|
||||
Adapters --> Foundation
|
||||
App -->|允许的技术适配依赖;优先由 startup 装配| Adapters
|
||||
Startup -->|构造并注入| Adapters
|
||||
|
||||
Startup -.注入/装配.-> Runtime
|
||||
Startup -.注入/装配.-> App
|
||||
@@ -137,20 +136,38 @@ flowchart TB
|
||||
Compat -.精确别名.-> Adapters
|
||||
```
|
||||
|
||||
图中的 `Chain → Db`、`Application → Db` 表示通过应用端口、Oper 或组合根注入的实现完成持久化,
|
||||
不是允许在用例代码中直接创建数据库引擎或拼接 SQL。`compat` 也不是只面向 SDK 的转发层,
|
||||
图中的 `Chain → Application` 与 `Application → Db` 表示通过应用端口和组合根注入的实现完成持久化,
|
||||
具体 DB Adapter 再使用 Oper;这不是允许在用例代码中直接创建数据库引擎或拼接 SQL。`compat` 也不是只面向 SDK 的转发层,
|
||||
它按 `app/runtime/compat/manifest.py` 的白名单把已经删除的旧模块/符号精确映射到各自的 canonical
|
||||
归属。`app/application/subscribe.py` 与 `app/application/plugins.py` 都是 V3 重构过程中新增、
|
||||
未形成插件 ABI 的宿主内部聚合文件,主题实现收口后直接删除,不在 manifest 中制造新的兼容债务。
|
||||
|
||||
当前实际的持久化调用路径可概括为:
|
||||
|
||||
```text
|
||||
入口(API / Agent / CLI / Scheduler / Workflow)
|
||||
-> Chain 或 Application 用例
|
||||
-> 命名 Port / Protocol
|
||||
-> db/adapters 创建短生命周期 Session/UoW
|
||||
-> db/oper 与 db/models
|
||||
```
|
||||
|
||||
`app/application/chain/data.py`、`app/application/agentdata.py` 和
|
||||
`app/application/history.py` 的 `get_*_port()` 是宿主生产代码读取组合根能力的规范入口;
|
||||
兼容期保留的 `*PortProxy` 不能重新被别名为 Oper。需要跨进程恢复或 commit 后可靠执行的
|
||||
业务副作用进入 `app/application/outbox.py` 定义的 Outbox 端口,由
|
||||
`app/db/adapters/outbox.py` 实现;`app/runtime/tasks.py` 的 TaskRegistry 只负责进程内任务所有权、
|
||||
取消和有限等待,不承担 durable queue 语义。
|
||||
|
||||
**依赖方向的核心约束**(由 `tests/test_architecture_dependencies.py` 强制检查):
|
||||
|
||||
| 方向 | 状态 |
|
||||
|---|---|
|
||||
| 入口层 → Chain / Application / Oper | 允许(按工作流复杂度选择) |
|
||||
| 入口层 → Chain / Application / 注入 Port | 允许(按工作流复杂度选择;不得直接构造 Oper) |
|
||||
| Chain → Module | 仅允许通过 `run_module` 方法名分发,禁止直接 import 模块内部 |
|
||||
| Chain → Agent 实现 | 禁止;只能经 `app/application/agent.py` 门面 |
|
||||
| Application → Domain / Runtime / Adapter / Oper | 允许 |
|
||||
| Application → Domain / Runtime / 注入的 Port | 允许;不得直接依赖具体 DB/Oper/Adapter |
|
||||
| DB Adapter → Application 持久化 Protocol / Oper / UoW | 允许;这是依赖倒置的实现方向 |
|
||||
| Module → Module / Chain | 禁止(跨模块编排一律进 Chain) |
|
||||
| Adapter → Application / runtime.extensions / sdk / compat | 禁止 |
|
||||
| Domain → Runtime / Adapter / Application / DB | 禁止 |
|
||||
@@ -166,26 +183,29 @@ flowchart TB
|
||||
|---|---|---|
|
||||
| `app/foundation/` | 无状态、无配置、无 I/O 的底层原语:反射/动态导入、加密、DOM、单例、文本、URL、版本比较 | `reflection.py`、`crypto.py`、`singleton.py` |
|
||||
| `app/domain/` | 纯 MoviePilot 业务语义:媒体上下文、识别解析、站点状态解释、磁力语义、NFO 刮削 | `context.py`、`metainfo.py`、`meta/`、`scraper.py` |
|
||||
| `app/runtime/` | 进程级运行机制:配置、进程拓扑、事件、完整日志、缓存契约与内存后端、并发、调度、限流、本地化、GC、重启状态 | `config.py`、`topology.py`、`events.py`、`log.py`、`cache.py` |
|
||||
| `app/runtime/` | 进程级运行机制:配置、进程拓扑、事件、完整日志、缓存契约与内存后端、任务所有权、执行/关联上下文、并发、调度、限流、本地化、GC、重启状态 | `config.py`、`events.py`、`event/`、`tasks.py`、`execution.py`、`correlation.py`、`log.py`、`cache.py` |
|
||||
| `app/runtime/extensions/` | 模块 / 插件 / 配置化服务 / 托管资源的发现、注册与生命周期适配;旧管理器文件保留稳定 ABI 门面,具体实现拆在主题子包 | `module_manager.py`、`plugin_manager.py`、`plugin/` |
|
||||
| `app/runtime/compat/` | 仅标准库的精确旧模块、包与符号导入路由;不是业务实现,也不是通用 re-export 层 | `manifest.py`、`imports.py` |
|
||||
| `app/adapters/network/` | 通用 HTTP、浏览器、DNS、Cloudflare、IP 传输机制 | `http.py`、`browser.py` |
|
||||
| `app/adapters/cache/` | Redis 与文件缓存的具体实现 | `backends.py`、`redis.py` |
|
||||
| `app/adapters/system/` | OS/文件/进程/stdio/显示/包安装/Rust 加速适配 | `host.py`、`resource.py`、`fsproxy.py` |
|
||||
| `app/adapters/external/` | 命名外部生态:插件市场、CookieCloud、OCR、IP 归属、MP Server、微信加密 | `market.py`、`server.py`、`wechat_crypt.py` |
|
||||
| `app/application/` | 读取配置/持久化状态的聚焦应用服务:识别、过滤、通知、RSS、站点、下载器、媒体服务器、存储、整理规则等;同一主题拆成子包 | `recognition.py`、`rules.py`、`rss.py`、`site/`、`subscription/`、`plugin/` |
|
||||
| `app/adapters/web/` | Web 技术适配:动态插件路由注册、认证依赖和 OpenAPI 重建;不承载插件路由用例 | `plugin/routes.py` |
|
||||
| `app/adapters/observability/` | 可选观测技术适配;核心层只依赖 `runtime/observability` 定义的窄端口 | `otel.py` |
|
||||
| `app/application/` | 读取配置/持久化状态的聚焦应用服务:识别、过滤、通知、RSS、站点、下载器、媒体服务器、存储、整理规则、可靠副作用等;同一主题拆成子包 | `recognition.py`、`rules.py`、`rss.py`、`outbox.py`、`site/`、`subscription/`、`plugin/` |
|
||||
| `app/application/chain/` | Chain 运行时上下文、跨领域数据端口和 durable event 命令;将组合根注入的能力以命名 getter 暴露给 Chain | `context.py`、`data.py`、`durable_events.py` |
|
||||
| `app/application/subscription/` | 订阅新增、查询、变更、删除、媒体身份与搜索契约 | `write.py`、`contract.py`、`mutation.py`、`delete.py`、`identity.py`、`search.py` |
|
||||
| `app/application/plugin/` | 插件市场、安装、运行时端口、文件夹操作和动态路由用例;具体 FastAPI 路由适配器在 adapters 层 | `catalog.py`、`install.py`、`runtime.py`、`folders.py`、`routes.py` |
|
||||
| `app/application/messaging/` | 渠道回环入口、消息渲染/路由、命令交互会话、插件按钮回调、Agent 消息桥接 | `ingress.py`、`message.py`、`router.py`、`agent.py` |
|
||||
| `app/application/security/` | 认证、授权、Cookie、Passkey、OTP/二次认证、SSRF 与 URL/路径安全 | `auth.py`、`url.py`、`twofactor.py` |
|
||||
| `app/chain/` | 跨入口复用的用例编排:订阅、搜索、下载、整理、媒体、消息等 Chain | `subscribe.py`、`search.py`、`transfer.py` |
|
||||
| `app/modules/` | 可插拔后端:下载器、媒体服务器、元数据源、消息渠道、索引器、存储 | `qbittorrent/`、`emby/`、`telegram/`、`themoviedb/` |
|
||||
| `app/db/` | SQLAlchemy 模型(`models/`)与一一对应的数据访问类(`oper/`) | `models/subscribe.py` ↔ `oper/subscribe.py` |
|
||||
| `app/db/` | SQLAlchemy 模型、表级 Oper、会话/UoW 与 Application 持久化适配器;Model 只接受显式 Session,不拥有事务提交 | `models/`、`oper/`、`adapters/`、`uow.py` |
|
||||
| `app/schemas/` | Pydantic 传输模型、枚举(`ModuleType`、`EventType`、`SystemConfigKey` 等) | `types.py`、`context.py` |
|
||||
| `app/api/` | FastAPI 主端点、鉴权依赖、统一 `Response` 响应封装;动态插件端点不走此统一包装 | `apiv1.py`、`endpoints/`、`response.py` |
|
||||
| `app/adapters/web/plugin/` | FastAPI 动态插件路由的技术适配:注册/移除、认证依赖、OpenAPI 重建;保留插件原生响应结构 | `routes.py` |
|
||||
| `app/agent/` | AI Agent:编排器、运行时、工具、中间件、LLM、记忆、技能、策略 | `orchestrator.py`、`runtime_loader.py`、`tools/` |
|
||||
| `app/startup/` | 组合根:装配注入、初始化/关停排序、重启策略 | `lifecycle.py`、`modules_initializer.py` |
|
||||
| `app/startup/` | 唯一组合根:跨层装配、领域初始化、声明式生命周期排序与重启策略 | `composition/`、`initializers/`、`lifecycle/` |
|
||||
| `app/sdk/` | 面向新插件的稳定导入面(网络、缓存、日志、浏览器等);`_legacy/` 只承载旧插件行为适配薄门面 | `network.py`、`browser.py`、`cache.py`、`_legacy/` |
|
||||
| `app/monitor/` | 源目录监控 → 触发整理 | `watcher.py`、`dispatcher.py` |
|
||||
| `app/workflow/` | 工作流引擎 | — |
|
||||
@@ -219,7 +239,7 @@ sequenceDiagram
|
||||
Life->>Init: get_engine() / get_global_async_engine() 预热 + fail-fast
|
||||
Life->>Init: check_connection_budget() 连接预算核算
|
||||
Life->>Init: init_routers(app) 注册 API 路由
|
||||
Life->>Init: init_modules() 发现并初始化模块,返回 HostRuntime
|
||||
Life->>Init: init_modules()(app/startup/initializers/modules.py)发现并初始化模块,返回 HostRuntime
|
||||
Life->>FastAPI: app.state.host_runtime = HostRuntime
|
||||
Life->>Init: init_plugins() / init_scheduler() / init_monitor()
|
||||
Life->>Init: init_command() / init_workflow()
|
||||
@@ -256,7 +276,7 @@ sequenceDiagram
|
||||
- **健康语义**:`/health/live` 只确认进程和事件循环可响应;`/health/ready` 仅在数据库
|
||||
到达当前 head 且生命周期完成后返回 200,启动失败或关停阶段返回 503。两者不公开路径、
|
||||
revision、插件和异常详情,深入诊断继续使用 Doctor。
|
||||
- **关停隔离**:每个关停步骤由 `run_shutdown_step` 独立捕获异常,保证后续资源仍有机会释放。
|
||||
- **关停隔离**:每个关停步骤由 `run_shutdown_step` 独立捕获异常,保证后续资源仍有机会释放;TaskRegistry、事件投递屏障、插件和模块资源按生命周期清单中的 owner 顺序收口。
|
||||
|
||||
---
|
||||
|
||||
@@ -384,6 +404,9 @@ flowchart LR
|
||||
创建、提交、回滚或关闭事务。无会话入口只存在于 Oper,由 `_execute_*` 经组合根事务执行器
|
||||
承接;内置插件必须调用 Oper,不得直接导入宿主 Model。AST 门禁同时约束装饰器、可选 Session
|
||||
和插件到 Model 的依赖,保证提交权不会被底层抢走。
|
||||
- **Outbox 可靠副作用**:业务行与 durable intent 在同一 Session/UoW 中提交;提交后由
|
||||
Outbox dispatcher 依据 topic、claim/lease、有限重试和 dead-letter 执行。完成通知、事件和统计
|
||||
的 post-commit 逻辑必须保持幂等,不能用普通线程或 TaskRegistry 代替持久 intent。
|
||||
- 站点、历史、工作流、Agent 会话删除和插件数据重置已经形成同构事务切片;对应 Application
|
||||
Command/Service 持有 UoW,Oper 的 `stage_*` 方法只修改当前会话。插件数据重置从
|
||||
`startup/initializers/plugins.py` 注入事务能力,插件直接使用 `PluginDataOper` 的旧 ABI 仅作兼容。
|
||||
@@ -623,7 +646,23 @@ flowchart LR
|
||||
SDK 导出(若公开)、`docs/rules/05-architecture.md` 与上述架构测试。
|
||||
- 延迟导入不被接受为隐藏循环依赖的手段。
|
||||
|
||||
### 10.1 2026-08-18 收口状态与后续边界
|
||||
### 10.1 2026-08-24 当前收口状态与后续边界
|
||||
|
||||
当前宿主架构基线(排除 `app/plugins/**`)如下;数字来自
|
||||
`tests/fixtures/architecture/`,更新基线前必须先审查语义变化:
|
||||
|
||||
| 指标 | 当前值 |
|
||||
|---|---:|
|
||||
| Python 模块 | 810 |
|
||||
| 内部导入边 | 6,560 |
|
||||
| 非平凡 SCC | 1(仅隔离的 TMDB 移植包) |
|
||||
| Module Contract V2 spec | 212(其中 211 个进入 `run_module` 观察面) |
|
||||
| Event Contract | 53 |
|
||||
| Model/Oper 自动事务与自建 Session | 0 |
|
||||
| 组合根外 `SystemConfigOper()` | 0 |
|
||||
|
||||
架构专项验证:`tests/test_architecture_dependencies.py` 与
|
||||
`tests/test_architecture_contract_baseline.py` 共 68 passed;当前工作树未修改架构 fixture。
|
||||
|
||||
本总览与本轮架构治理的关系如下:
|
||||
|
||||
@@ -638,6 +677,10 @@ flowchart LR
|
||||
文件夹操作归入 `app/application/plugin/routes.py`、`folders.py`。原
|
||||
`app/application/subscribe.py`、`app/application/plugins.py` 未形成插件 ABI,已经直接删除,
|
||||
宿主调用统一改为 canonical 路径。
|
||||
- 已完成的运行时可靠性收口:TaskRegistry 统一进程内后台任务 owner;durable-required 事件和
|
||||
订阅关键副作用经 `app/application/outbox.py` 与 `app/db/adapters/outbox.py` 进入同事务 Outbox;
|
||||
搜索逐页任务、Agent/消息事件和插件市场子任务均遵守请求或生命周期 owner,不再由入口模块维护
|
||||
无法追踪的裸任务集合。
|
||||
- 判断是否需要新增 manifest 映射的标准:只有当旧物理模块被删除、改名或公开符号迁移时才登记;
|
||||
物理文件仍是稳定入口的,不应为了目录规整新增“自己映射自己”的别名,也不应在 canonical 包中
|
||||
保留多余导出。
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
> 文档性质:现状审计、目标约束、迁移路线和 AI 实施手册
|
||||
> 适用仓库:`MoviePilot`,分支 `v3`
|
||||
> 审计基线:2026-08-18 当前工作树
|
||||
> 审计基线:`7c97d1742`(2026-08-24)
|
||||
> 相关规范:`AGENTS.md`、`docs/rules/05-architecture.md`、`docs/architecture-overview.md`、`docs/refactor/backend-module-refactor-compatibility.md`
|
||||
|
||||
## 1. 文档目的
|
||||
@@ -14,7 +14,7 @@
|
||||
3. 为其他 AI 提供可以直接执行的任务边界、兼容约束、验证命令和完成标准。
|
||||
4. 在不破坏 V3 插件生态的前提下,逐步收敛宿主内部结构,而不是用一次性改名制造新的兼容层。
|
||||
|
||||
本文同时记录治理方案和当前工作树的实施状态。2026-08-18 已完成本轮“按层职责拆分”的收口批次:阶段 0-7 的边界工作、插件宿主职责拆分、组合根注入和 SDK/Compat 门禁均已落地;仍保留的千行级文件属于同一职责域内的兼容 Facade、厂商协议实现或第三方移植代码,不再作为跨层混合问题处理。每个阶段是否完成必须以本文件的机器基线、聚焦测试、插件兼容扫描和完整测试门禁为准,不能只凭目录已经创建判断。
|
||||
本文同时记录治理方案和当前工作树的实施状态。2026-08-24 的当前代码已经完成阶段 0-7 的跨层边界收口,并继续完成模块契约、生命周期 owner、命名数据端口、Outbox 和插件运行时治理切片;仍保留的千行级文件属于同一职责域内的兼容 Facade、厂商协议实现或第三方移植代码,不再作为跨层混合问题处理。每个阶段是否完成必须以本文件的机器基线、聚焦测试、插件兼容扫描和完整测试门禁为准,不能只凭目录已经创建判断。
|
||||
|
||||
### 2026-08-18 收口结论
|
||||
|
||||
@@ -25,10 +25,13 @@
|
||||
3. `PluginManager` 的加载、生命周期、注册表、投影、存储、目录、路径、同步、依赖、克隆和文件监控分别由 `app/runtime/extensions/plugin/` 下的单职责组件承担;旧管理器只保留 V3 ABI 门面和兼容调用顺序。
|
||||
4. 动态插件 API 使用专用 raw 路由;主程序统一响应信封不进入插件 `get_api()`。前端 `pluginApi` 对非 `Response` envelope 的 payload 原样交付调用方。
|
||||
5. 旧插件导入仅由 `app/runtime/compat/manifest.py` 精确映射;canonical 模块不复制旧 Manager/Helper/Oper 导出。`app/plugins/` 仍是运行时副本,继续排除在宿主架构扫描之外。
|
||||
6. 2026-08-24 当前机器基线为 805 个宿主 Python 模块、6,502 条内部导入边;数据库边界、Adapter→DB、Runtime→DB、Application→DB 及新增 API/Agent/Chain 目标边均为 0。架构门禁、插件兼容快照和基线脚本均已重新生成。
|
||||
6. 2026-08-24 当前机器基线为 810 个宿主 Python 模块、6,560 条内部导入边;数据库边界、Adapter→DB、Runtime→DB、Application→DB 及新增 API/Agent/Chain 目标边均为 0。架构门禁、插件兼容快照和基线脚本均已重新生成。
|
||||
7. 订阅写入统一归入 `app/application/subscription/write.py`;插件动态路由和文件夹操作统一归入 `app/application/plugin/routes.py`、`folders.py`。重构期间新增且未形成插件 ABI 的 `app/application/subscribe.py`、`app/application/plugins.py` 已直接删除,不进入 compat manifest。
|
||||
8. 2026-08-24 完成 Module Contract V2 宿主观察面收口:212 个 spec 均使用可执行的显式 aggregation,
|
||||
`legacy` 只保留为未知第三方自定义方法的开放 fallback;插件方法名、kwargs、优先级和异常隔离 ABI 不变。
|
||||
9. 订阅及其它关键业务副作用已通过 `app/application/outbox.py`、`app/db/adapters/outbox.py`
|
||||
和启动组合根形成同事务 durable intent、claim/lease、有限重试与 dead-letter 边界;TaskRegistry
|
||||
仍只负责进程内任务 owner 和关停,不被当作持久队列。
|
||||
|
||||
## 2. 范围与明确排除项
|
||||
|
||||
@@ -93,15 +96,17 @@ MoviePilot V3 已经完成一轮重要基础工作:原 `app/core`、`app/helpe
|
||||
### 4.2 已验证结果
|
||||
|
||||
```text
|
||||
./.venv/bin/python -m pytest tests/test_architecture_dependencies.py -q
|
||||
28 passed
|
||||
./.venv/bin/python -m pytest \
|
||||
tests/test_architecture_dependencies.py \
|
||||
tests/test_architecture_contract_baseline.py -q
|
||||
68 passed
|
||||
```
|
||||
|
||||
这只能证明当前代码符合现有门禁,不能证明符合本文件提出的更完整目标。
|
||||
|
||||
### 4.3 模块规模
|
||||
|
||||
排除 `app/plugins/` 后,2026-08-24 当前静态扫描得到 805 个 Python 模块、6,502 条内部导入边。下表保留 2026-08-18 收口时的一级目录规模快照(代码行数包含注释和空行,用于趋势比较而非质量评分):
|
||||
排除 `app/plugins/` 后,2026-08-24 当前静态扫描得到 810 个 Python 模块、6,560 条内部导入边。下表保留 2026-08-18 收口时的一级目录规模快照(代码行数包含注释和空行,用于趋势比较而非质量评分):
|
||||
|
||||
| 一级目录 | 约代码行数 | Python 文件数 | 判断 |
|
||||
| --- | ---: | ---: | --- |
|
||||
@@ -149,8 +154,8 @@ MoviePilot V3 已经完成一轮重要基础工作:原 `app/core`、`app/helpe
|
||||
|
||||
| 指标 | 初始审计 | 当前基线 | 说明 |
|
||||
| --- | ---: | ---: | --- |
|
||||
| Python 模块数 | 约 654 | 805 | 增量来自单一职责的 Application、Runtime、Adapter、插件组件和维护用例模块 |
|
||||
| 内部导入边 | 约 5,623 | 6,502 | 显式端口增加模块数但移除了反向边;边数不作为单独质量目标 |
|
||||
| Python 模块数 | 约 654 | 810 | 增量来自单一职责的 Application、Runtime、Adapter、插件组件和维护用例模块 |
|
||||
| 内部导入边 | 约 5,623 | 6,560 | 显式端口增加模块数但移除了反向边;边数不作为单独质量目标 |
|
||||
| SCC 数 | 14 | 1 | 自有代码 SCC 已归零,仅保留 TMDB 移植包内部隔离例外 |
|
||||
| `adapters -> db` | 存在 | 0 | `PluginHelper`、`MoviePilotServerHelper` 的本地数据读取已移到组合根/Application |
|
||||
| `runtime -> db` | 存在 | 0 | 插件存储、服务配置均改为启动注入 |
|
||||
@@ -399,6 +404,9 @@ app/chain/transfer.py # 保持 TransferChain 兼容门面
|
||||
- Application 用例拥有事务语义;API 只调用用例。
|
||||
- 复杂跨 Oper 事务可引入小型 `UnitOfWork` Protocol,但不要为单表查询套通用框架。
|
||||
- Event、Scheduler、Server 上报只在提交成功后触发;必要时用显式 after-commit 动作清单。
|
||||
- 需要跨进程恢复、重试或幂等交付的 after-commit 动作进入
|
||||
`app/application/outbox.py`,由 `app/db/adapters/outbox.py` 持久化 intent 并负责 claim/lease;
|
||||
`TaskRegistry` 只覆盖进程内任务的取消与关停等待。
|
||||
|
||||
#### 迁移顺序
|
||||
|
||||
|
||||
@@ -2,11 +2,12 @@
|
||||
|
||||
> 文档性质:当前架构复核、优秀 Python 后端实践对标、AI 可执行任务手册
|
||||
> 适用仓库:`MoviePilot`,分支 `v3`
|
||||
> 审计基线:`6404a3aa583de03bf0770c37b106413461cec1f8`(2026-08-21)
|
||||
> 审计基线:`7c97d1742`(2026-08-24)
|
||||
> 审计范围:宿主后端;排除 `app/plugins/**` 运行时插件副本
|
||||
> 规范优先级:`AGENTS.md` 与 `docs/rules/` 高于本文
|
||||
> 相关文档:`docs/architecture-overview.md`、`docs/refactor/backend-architecture-governance.md`、`docs/refactor/backend-module-refactor-compatibility.md`
|
||||
> 实施进度:阶段 0~6 的宿主架构能力已完成收口;API/Application 公共复杂度基线已清零,启动组合根的 SystemConfigOper 构造点已由 14 降至 1;API 进程内后台任务已完成首批统一登记,插件仓适配和 Outbox 外围扩展仍按风险切片推进。Model/Base 查询与写装饰器、legacy 隐式会话外壳均已清零,插件 SDK 也不再导出宿主 Model。2026-08-23 的长期整改阶段 0 已恢复宿主、启动性能、官方插件和 SDK 契约门禁的可信基线;阶段 1a 已补齐 TaskRegistry owner 零债务门禁和诚实的关停超时语义;阶段 1b1 已收口整理 worker、pending 回放、失败通知、进程内 AI 重试、插件监控与事件投递的生命周期所有权;2026-08-24 的阶段 2 已将 212 个已观察宿主模块方法的 legacy aggregation 清零,并补齐可执行 fanout 与下载器文件 DTO 边界;阶段 3 已将消息交互和远程命令的订阅删除统一到 Application/UoW/outbox,宿主不再调用裸线程统计入口;阶段 4 已统一七种消息渠道的宿主回环与后台执行边界;阶段 5 已补齐事件窗口聚合任务的生命周期所有权;阶段 6 已统一插件文件操作的取消完成语义;阶段 7 已统一插件协程补偿的终态等待;阶段 8 已统一宿主同步函数的异步线程池入口;阶段 9 已统一工作流运行时的宿主获取路径;阶段 10 已统一模块、插件与调度运行时的显式 getter 调用;阶段 11 已清除系统配置 getter 的 Oper 形别名;阶段 12 已完成工作流域的显式 Chain 数据端口迁移;阶段 13 已收口用户、交互与消息链的数据端口;阶段 14 已收口音乐订阅数据端口;阶段 15 已收口站点数据端口;阶段 16 已收口媒体服务器数据端口;阶段 17 已收口下载数据端口;阶段 18 已收口主订阅数据端口;阶段 19 已收口整理数据端口;阶段 20 已收口 Agent 数据端口;阶段 21 已收口监控历史端口;阶段 22 已统一服务配置应用边界;阶段 23 已补齐媒体服务器 API 遗留的类形配置读取路径;阶段 24 已清除 Scheduler 内部无 owner 的协程提交双轨;阶段 25 已补齐 TaskRegistry 跨线程 owner 并迁移整理 AI 接管;阶段 26 已统一 Agent 会话清理提交;阶段 27 已统一历史 AI 进度 owner;阶段 28 已托管旧插件订阅统计线程;阶段 29 已统一 Emby 系条目转换并清零重复代码白名单;阶段 30 已收口插件市场请求级子任务;阶段 31 已托管搜索 AI 推荐任务;阶段 32 已清除事件调度器绕过生命周期 owner 的投递回退;阶段 33 已统一宿主 Agent 运行时的获取路径;阶段 34 已统一 durable-required 事件与 Outbox topic 事实源;阶段 35 已统一 LLM provider 管理 API 的运行时解析路径;阶段 36 已统一 WebAgent 音频能力访问边界;阶段 37 已统一插件输入事件发布路径;阶段 38 已统一 WebAgent 通知事件监听与队列边界;阶段 39 已补齐搜索 SSE 断线时的上游任务清理;阶段 40 已补齐异步防抖取消的终态所有权;阶段 41 已统一优雅重启兜底线程的唯一所有权;阶段 42 已补齐 Telegram typing 的多实例隔离和终态 owner;阶段 43 已统一 Discord typing 的异步 owner 和 shutdown 收尾;阶段 44 已清除 WebAgent 测试临时事件循环提前关闭产生的 CI 红注解;阶段 45 已统一影视与字幕搜索的请求级逐页任务编排;阶段 46 已收口启动性能门禁的托管 runner 假失败与诊断输出;阶段 47 已补齐 Agent 渠道流式刷新任务的重入 owner;阶段 48 已统一工件上传 action 的 Node 24 主版本;阶段 49 已统一插件安装的同步/异步代际解析事实源;阶段 50 已统一插件市场 GitHub 请求降级策略;阶段 51 已统一插件索引请求与响应三态策略;阶段 52 已统一插件 Release 分页策略;阶段 53 已统一远端插件安装模式决策;阶段 54 已补齐同步安装成功后的临时回滚备份清理;阶段 55~56 已收口官方插件观察基线与报告保留策略;阶段 57 已统一进程级运行时 Facade 门禁并补齐 ModuleManager 边界;阶段 58 已消除 AgentTask 关闭回归的跨线程零时长等待竞态。
|
||||
> 当前 canonical 状态:API/Application 公共复杂度基线已清零,组合根外 `SystemConfigOper()` 构造和 Model/Oper 隐式事务均为 0;命名 Chain/Agent 数据端口、TaskRegistry owner、Module Contract V2、typed Event、Outbox durable intent、请求关联和插件运行时 getter 已形成当前路径。插件仓适配、未知第三方 fallback 和其它 E1/E3 副作用仍按风险持续治理。
|
||||
|
||||
## 当前复核结论(2026-08-24)
|
||||
|
||||
@@ -15,7 +16,7 @@
|
||||
|
||||
### 长期整改阶段 0:治理门禁恢复(2026-08-23)
|
||||
|
||||
- 宿主依赖基线已审查 TaskRegistry、有界后台 owner 与插件变更准入接入后的语义差异:当前为 `806` 个模块、`6544` 条内部导入边,12 组重点禁止边继续全部为 `0`,唯一非平凡 SCC 仍是隔离的 TMDB 移植包。
|
||||
- 宿主依赖基线已审查 TaskRegistry、有界后台 owner 与插件变更准入接入后的语义差异:当前为 `810` 个模块、`6560` 条内部导入边,12 组重点禁止边继续全部为 `0`,唯一非平凡 SCC 仍是隔离的 TMDB 移植包。
|
||||
- 启动性能探针会在隔离生命周期中真实创建并释放 TaskRegistry;normal/safe 组件数分别为 `23`/`11`,CI 只读检查使用稳定的宿主模块集合和生命周期组件顺序,不再把 Python/平台模块数量当作硬合同。
|
||||
- 官方插件快照覆盖 `plugins.v3`、`plugins.v2` 以及 V3 实际会从 `package.json` 回退加载的 31 个默认实现;`app/plugins/**` 仍只是宿主运行副本,不进入扫描。
|
||||
- SDK 快照以各模块显式 `__all__` 为公开合同,能够记录赋值别名;`typing`、`__future__` 等实现期导入不再被误冻结,既有数据库备份门面已补精确导出清单。
|
||||
@@ -615,7 +616,7 @@
|
||||
|
||||
- 继续采用单进程控制面是正确选择,不建议现在拆成微服务;插件、调度器、工作流、事件和数据库共享进程内状态,拆分会放大部署、事务和兼容成本。
|
||||
- `foundation/domain/runtime/adapters/application/chain/api/startup` 的职责方向基本成立;宿主架构基线、复杂度 ratchet、异步阻塞 ratchet 当前均通过。
|
||||
- 依赖图当前为 `806` 个 Python 模块、`6546` 条内部导入边;唯一非平凡 SCC 位于隔离的 TMDB 第三方移植包内部,不应为了指标归零重写。
|
||||
- 依赖图当前为 `810` 个 Python 模块、`6560` 条内部导入边;唯一非平凡 SCC 位于隔离的 TMDB 第三方移植包内部,不应为了指标归零重写。
|
||||
- 当前主要风险已经从“目录和依赖失控”转移到运行时协议、后台副作用的可靠性和遗留兼容面。换言之,下一阶段重点应是**语义收口和可验证性**,而不是继续搬文件或机械拆大文件。
|
||||
|
||||
综合评价:架构方向可持续,生产可用性较高;可演进性仍处于中等水平。现阶段没有静态审计发现必须立即推倒重来的 P0 架构问题,但存在需要按 P1/P2 计划治理的真实债务。
|
||||
@@ -762,34 +763,25 @@ MoviePilot V3 当前不是“目录混乱、必须推倒重来”的状态。第
|
||||
|
||||
| 指标 | 当前值 | 判断 |
|
||||
| --- | ---: | --- |
|
||||
| 宿主 Python 模块数 | 753 | 排除 `app/plugins/**` |
|
||||
| 宿主内部导入边 | 6,076 | 边数本身不是质量目标 |
|
||||
| 宿主 Python 模块数 | 810 | 排除 `app/plugins/**` |
|
||||
| 宿主内部导入边 | 6,560 | 边数本身不是质量目标 |
|
||||
| 非平凡 SCC | 1 | 仅 TMDB 移植包内部 |
|
||||
| 重点禁止边 | 0 | Adapter/Runtime/Application/API/Chain 等到 DB 的既有门禁均通过 |
|
||||
| 架构专项测试 | 39 passed | `test_architecture_dependencies` + `test_architecture_contract_baseline` |
|
||||
| 架构专项测试 | 68 passed | `test_architecture_dependencies` + `test_architecture_contract_baseline` |
|
||||
| 宿主 Python 代码行 | 约 241,227 | 含注释和空行,仅用于趋势 |
|
||||
| 已登记模块调用方法 | 211 | 260 个静态调用点,0 个动态方法名调用点 |
|
||||
| 已登记模块调用方法 | 211 | 212 个宿主 spec,其中 211 个进入 `run_module` 观察面 |
|
||||
| legacy 默认模块契约 | 0 个宿主观察方法;未知动态方法保留 fallback | 所有静态宿主方法已有显式 V2 spec;真实 fallback 命中由 `module.contract.legacy_hit` 观测 |
|
||||
| 事件枚举 | 53 | 66 个静态 producer、15 个静态 consumer |
|
||||
| 事件枚举 | 53 | 78 个 producer、15 个 consumer(含动态观察) |
|
||||
| 专用 EventData model | 53 | Event Contract Registry 已为全部事件登记 typed payload/fallback 原因 |
|
||||
| 直接读取 `settings` 的文件 | 105 | 仍按模块族迁移,动态协议和安全端口暂保留 |
|
||||
| 直接读取 `settings` 的文件 | 0 | 当前宿主基线已清零;部署配置通过组合根快照/窄端口提供 |
|
||||
| `SystemConfigOper()` | 1 个 | 仅组合根创建 `SystemConfigService` 时保留 |
|
||||
| Model/Base 上的 DB 装饰器 | 0 | 正式与 legacy 查询/写装饰器全部为 0;`db` 参数必须显式传入 |
|
||||
| 路由端点 | 335 | 11 个已装饰端点超过 80 行,最大 400 行 |
|
||||
| Chain 方法超过 150 行 | 18 | 最大 `TransferChain.do_transfer()` 885 行 |
|
||||
| Application 方法超过 150 行 | 8 | 最大 296 行 |
|
||||
| Agent 方法超过 150 行 | 13 | 最大 713 行 |
|
||||
| API/Application/Chain 公共复杂度超限 | 0 | 当前 `scripts/architecture/complexity.py` ratchet 无新增或增长债务 |
|
||||
| 公共函数缺少返回注解 | 约 1,592 / 7,442 | AST 近似值,适合做 ratchet,不适合直接作为失败阈值 |
|
||||
| 公共参数缺少注解 | 约 858 / 12,763 | 主要集中在 `app/modules` |
|
||||
|
||||
代表性大方法:
|
||||
|
||||
- `app/chain/transfer.py::TransferChain.do_transfer()`:约 885 行;
|
||||
- `app/chain/download.py::DownloadChain.batch_download()`:约 572 行;
|
||||
- `app/chain/subscribe.py::SubscribeChain.match()`:约 417 行;
|
||||
- `app/api/endpoints/agent.py::web_agent_stream()`:约 400 行;
|
||||
- `app/api/endpoints/transfer.py::manual_transfer()`:约 295 行;
|
||||
- `app/scheduler.py::Scheduler.init()`:约 383 行。
|
||||
当前复杂度门禁只对新增/增长负责;同一职责域中的大型兼容 Facade、厂商协议实现和第三方移植代码仍以
|
||||
行为快照、依赖边界和增量 ratchet 为主要治理尺度,不以机械拆文件代替所有权迁移。
|
||||
|
||||
### 2.3 本次检查暴露的基线问题
|
||||
|
||||
|
||||
@@ -66,6 +66,8 @@ to make the directory tree look symmetrical.
|
||||
| `app/application/download/` | Download task querying/control and later submission use cases |
|
||||
| `app/application/music/` | Multi-source music catalog orchestration |
|
||||
| `app/application/chain/` | Injectable Chain runtime context and compatibility provider |
|
||||
| `app/application/agentdata.py` | Named Agent data ports; canonical Agent consumers use `get_agent_*_port()` and do not alias legacy proxies to Oper classes |
|
||||
| `app/application/outbox.py` | Durable intent and Outbox repository/dispatcher contracts for post-commit side effects |
|
||||
| `app/application/plugin/` | Plugin market catalog, installation command, runtime port, folder operations and dynamic-route use cases; filenames remain single words (`catalog.py`, `install.py`, `runtime.py`, `folders.py`, `routes.py`) |
|
||||
| `app/application/server/` | MoviePilot Server reporting and sharing use cases; local data readers and transport callbacks are injected by startup |
|
||||
| `app/application/site/` | Configured site catalog, authentication level and index-resource capability; the generated extension and its data bundle stay together here |
|
||||
@@ -87,11 +89,14 @@ directory categories.
|
||||
| `app/runtime/config.py` | Deployment configuration and resolved runtime settings |
|
||||
| `app/runtime/topology.py` | Process topology policy shared by startup and offline diagnostics |
|
||||
| `app/runtime/events.py` | Event contracts, dispatch and resolver registration |
|
||||
| `app/runtime/event/` | Event registry, explicit handler binding, dispatch barrier/concurrency and isolated error handling |
|
||||
| `app/runtime/observability/` | Low-cardinality metric contracts and no-op-capable observation facade |
|
||||
| `app/runtime/log.py` | Complete console/plugin/file logging runtime and shutdown |
|
||||
| `app/runtime/cache.py` | Cache protocols, memory implementations, decorators and proxies |
|
||||
| `app/runtime/managed_resources.py` | Provider-neutral acquisition, observation and shutdown facade for process-owned optional resources |
|
||||
| `app/runtime/tasks.py` | Lifespan-scoped ownership, cancellation and bounded shutdown waiting for in-process background tasks |
|
||||
| `app/runtime/execution.py` | Shared sync/async execution and cross-thread submission boundary with correlation propagation |
|
||||
| `app/runtime/correlation.py` | Request/cross-thread correlation context and safe propagation into logs and child work |
|
||||
| `app/runtime/state.py` | Process restart and update state |
|
||||
| `app/runtime/extensions/` | Module, plugin, configured-service and managed-resource discovery/registration/lifecycle adapters |
|
||||
| `app/runtime/compat/` | Standard-library-only exact legacy import routing, resource preflight scanning and DEBUG diagnostics |
|
||||
@@ -157,6 +162,8 @@ not a second application-facing service directory.
|
||||
| `app/adapters/network/` | Generic HTTP, browser, DNS, Cloudflare and IP transport mechanisms |
|
||||
| `app/adapters/system/` | OS/filesystem/process facilities, stdio, display, packages, resources and optional Rust acceleration |
|
||||
| `app/adapters/external/` | CookieCloud, plugin market, OCR, IP-location providers and MoviePilot Server |
|
||||
| `app/adapters/web/` | FastAPI-specific technical adapters, including raw dynamic plugin routes |
|
||||
| `app/adapters/observability/` | Optional telemetry exporters; core code depends only on runtime observation ports |
|
||||
| `app/adapters/external/plugin/client.py` | Read-only plugin-market and local-repository client over the established `PluginHelper` implementation |
|
||||
| `app/adapters/system/plugin/` | Plugin package and dependency I/O (`package.py`, `dependency.py`) |
|
||||
| `app/db/adapters/` | SQLAlchemy implementations of Application-owned persistence Protocols |
|
||||
@@ -290,8 +297,8 @@ architecture snapshot, not through incidental module globals.
|
||||
### Chain layer
|
||||
|
||||
`app/chain/` implements use cases shared by API, CLI, Agent, scheduler and other
|
||||
entrypoints. Chains may coordinate modules, application services, Oper classes,
|
||||
events and caches. New chain-to-chain dependencies are allowed only while the
|
||||
entrypoints. Chains may coordinate modules, application services, injected
|
||||
persistence Ports, events and caches. New chain-to-chain dependencies are allowed only while the
|
||||
static graph remains acyclic. Backend protocol details and HTTP request objects
|
||||
do not belong here. Chains interact with modules exclusively through
|
||||
`run_module` dispatch on method-name contracts; direct imports of module
|
||||
@@ -389,9 +396,10 @@ SQLAlchemy models stay under `app/db/models/`; the data access classes live in
|
||||
`oper/subscribe.py`), so a filename carries only the entity and the package name
|
||||
carries the role. Two verified aggregation exceptions exist: the site family
|
||||
(`Passkey`, `SiteIcon`, `SiteStatistic`, `SiteUserData`) is consolidated in
|
||||
`oper/site.py`, and `AgentTaskRun` lives in `oper/agenttask.py`. Chains, modules,
|
||||
application services and endpoints use Oper
|
||||
classes instead of issuing SQLAlchemy queries directly. Every schema change
|
||||
`oper/site.py`, and `AgentTaskRun` lives in `oper/agenttask.py`. DB adapters use
|
||||
Oper classes instead of issuing SQLAlchemy queries directly. Application and
|
||||
Chain code reaches persistence through named Ports/Protocols; concrete DB adapters
|
||||
are the layer that adapts those Ports to Oper classes. Every schema change
|
||||
requires an Alembic migration under `database/versions/`.
|
||||
|
||||
Oper classes take and return persistence values, not domain objects. Translating
|
||||
@@ -409,6 +417,21 @@ path cannot forget them. Identity representation rules themselves
|
||||
alongside the two identity mixins; `app/domain/media.py` keeps only source
|
||||
policy. `app/db` therefore has no dependency on `app/domain`.
|
||||
|
||||
Durable post-commit side effects have a separate boundary:
|
||||
|
||||
- `app/application/outbox.py` owns the Outbox intent, repository and dispatcher
|
||||
contracts. An Application command stages the business mutation and its durable
|
||||
intent in the same transaction.
|
||||
- `app/db/adapters/outbox.py` implements the persistence port with SQLAlchemy;
|
||||
`app/startup/composition/subscription.py` and the other composition modules
|
||||
provide the concrete repository, UoW and handlers.
|
||||
- The dispatcher claims an intent with a lease, executes the topic handler, and
|
||||
records retry/dead-letter state. Handlers must be idempotent and must not rely
|
||||
on a live request object.
|
||||
- `app/runtime/tasks.py` is only the in-process TaskRegistry boundary. It owns
|
||||
cancellation and bounded shutdown waiting, but it is not a durable queue and
|
||||
must not replace an Outbox or persistent task table.
|
||||
|
||||
## Composition and Compatibility Boundaries
|
||||
|
||||
- Startup registers concrete cache factories before decorated business modules
|
||||
@@ -469,6 +492,8 @@ policy. `app/db` therefore has no dependency on `app/domain`.
|
||||
| `app/application/agent.py` | Agent orchestration facade (`get_agent_manager` / `get_prompt_manager` / capability queries / prompt builders); lightweight providers register through `app/startup/initializers/agent.py`, with no static `application -> agent` edge |
|
||||
| `app/agent/runtime_loader.py` | Agent-specific capability discovery and canonical entrypoint/service materialization; reuses the generic Capability Runtime while keeping Agent ownership under `app/agent/` |
|
||||
| `app/application/subscription/write.py` | Subscription media translation and sync/async write-port orchestration |
|
||||
| `app/application/outbox.py` | Durable intent, topic handler and Outbox repository contracts |
|
||||
| `app/db/adapters/outbox.py` | SQLAlchemy Outbox persistence, claim/lease and retry state adapter |
|
||||
| `app/application/scheduling.py` | Runtime scheduler facade for Agent tools and endpoints; `Scheduler` class registered by `app/startup/initializers/scheduler.py` |
|
||||
| `app/application/commands.py` | Command registry facade for Agent tools and endpoints; `Command` class registered by `app/startup/initializers/command.py` |
|
||||
| `app/application/workflow.py` | Workflow use cases plus the runtime port consumed by API and Chain; `WorkFlowManager` is registered by `app/startup/initializers/workflow.py` |
|
||||
@@ -477,6 +502,9 @@ policy. `app/db` therefore has no dependency on `app/domain`.
|
||||
| `app/startup/initializers/` | Domain-scoped initialization and shutdown hooks |
|
||||
| `app/chain/agent.py` | `AgentChain(ChainBase)`: the chain-layer entry for Agent sessions; Agent runtime stays in `app/agent/` |
|
||||
| `app/runtime/config.py` | `ConfigModel`, `Settings` and deployment configuration |
|
||||
| `app/runtime/tasks.py` | TaskRegistry owner, cancellation and bounded shutdown waiting |
|
||||
| `app/runtime/execution.py` | Shared execution/thread-boundary helpers and context propagation |
|
||||
| `app/runtime/correlation.py` | Correlation ID context and propagation boundary |
|
||||
| `app/runtime/topology.py` | Single-worker full-runtime policy and safe-mode topology validation |
|
||||
| `app/runtime/events.py` | `EventManager`/`Event` compatibility facade and global `eventmanager` identity |
|
||||
| `app/runtime/event/registry.py` | Event subscriptions, enable/disable state and dispatch snapshots |
|
||||
@@ -533,4 +561,4 @@ imports, entrypoint (`api`/`agent`/`monitor`/`workflow`/`doctor`) imports of
|
||||
modules only through `run_module` dispatch), and downloader SDK
|
||||
(`qbittorrentapi`, `transmission_rpc`) imports inside `app/chain`.
|
||||
|
||||
*Last Updated: 2026-08-18*
|
||||
*Last Updated: 2026-08-24*
|
||||
|
||||
@@ -129,6 +129,19 @@ adapters; it does not retain reusable repository implementations.
|
||||
and UoW to one request/operation Session. Legacy plugin-facing Oper methods may
|
||||
remain temporarily, but a new endpoint or startup workflow must call `stage_*`.
|
||||
|
||||
### Durable post-commit side effects
|
||||
|
||||
Business mutations that must survive process interruption stage their durable
|
||||
intent through `app/application/outbox.py` in the same Session/UoW as the
|
||||
business row. `app/db/adapters/outbox.py` is the SQLAlchemy implementation;
|
||||
startup composition supplies the repository, transaction scope and topic
|
||||
handlers.
|
||||
|
||||
The dispatcher claims an intent with a lease, executes an idempotent handler,
|
||||
and records bounded retries or dead-letter state. The `app/runtime/tasks.py`
|
||||
TaskRegistry is only the owner for in-process work and bounded shutdown waiting;
|
||||
it is not a durable queue or a replacement for an Outbox/persistent task table.
|
||||
|
||||
Run `./.venv/bin/python scripts/architecture/baseline.py --check-host` after
|
||||
persistence changes. A deliberate debt reduction may refresh the low-water mark
|
||||
with `--write-host`; never refresh it to accept newly introduced debt.
|
||||
@@ -254,4 +267,4 @@ When `REDIS_HOST` is configured, `app/modules/redis/` provides a distributed cac
|
||||
- `settings.API_TOKEN` and other secret fields must not be included in log output or API responses.
|
||||
- The `config list --show-secrets` flag exists specifically to gate secret visibility in the CLI.
|
||||
|
||||
*Last Updated: 2026-08-21*
|
||||
*Last Updated: 2026-08-24*
|
||||
|
||||
@@ -104,4 +104,4 @@ Those managing the application lifecycle post-development:
|
||||
|
||||
---
|
||||
|
||||
*Last Updated: 2026-05-25*
|
||||
*Last Updated: 2026-08-24*
|
||||
|
||||
Reference in New Issue
Block a user