mirror of
https://github.com/jxxghp/MoviePilot.git
synced 2026-09-04 15:09:46 +08:00
refactor: reorganize startup persistence boundaries
This commit is contained in:
jxxghp
@@ -245,12 +245,12 @@ sequenceDiagram
|
||||
和 TestClient 因而共享同一 fail-fast 语义。
|
||||
- **引擎预热 fail-fast**：同步/异步数据库引擎在单线程期完成首次创建，
|
||||
避免调度器放出大量线程后再创建引擎导致连接锁竞争。
|
||||
- **类型化请求装配**：`startup/context.py` 的 frozen slots `HostRuntime` 是 lifespan 内唯一宿主
|
||||
- **类型化请求装配**：`startup/composition/context.py` 的 frozen slots `HostRuntime` 是 lifespan 内唯一宿主
|
||||
上下文，`api/context.py` 从 `app.state` 收窄到具体领域能力。认证、消息、历史、媒体服务器、站点、
|
||||
订阅、工作流和请求事务均使用命名 runtime 字段，不再通过字符串仓储键定位；API、Scheduler、Chain
|
||||
从 `HostRuntime.configuration` 获取 frozen 配置快照。系统设置管理 API 通过
|
||||
`HostRuntime.settings` 的窄服务读写可变部署设置，业务域不接触 Settings 实例；生产与测试组合根统一
|
||||
复用 `startup/configuration.py` 的映射。`ApiDataPorts` 仅保留旧导入 ABI，不参与正式请求链路。
|
||||
复用 `startup/composition/configuration.py` 的映射。`ApiDataPorts` 仅保留旧导入 ABI，不参与正式请求链路。
|
||||
- **安全模式**：`MOVIEPILOT_SAFE_MODE` 会跳过插件、定时器、监控器、命令与工作流，用于故障自救。
|
||||
- **进程拓扑**：全功能 V3 强制 `API_WORKERS=1`，避免每个 worker 重复启动插件和后台控制面；安全模式可临时使用多 worker 诊断，但不是正式扩容方案。
|
||||
- **健康语义**：`/health/live` 只确认进程和事件循环可响应；`/health/ready` 仅在数据库
|
||||
@@ -375,7 +375,8 @@ flowchart LR
|
||||
不再保留主题包之外的第二个写入入口。
|
||||
- 规范写入口中的 Oper 只 stage mutation，不创建独立 Session、不提交；Application Command
|
||||
通过请求或任务入口注入的 UnitOfWork 统一 `commit/rollback`，事件、刷新和上报只在 commit
|
||||
成功后执行。订阅新增样板由 `startup/subscription.py` 创建独占 Session，
|
||||
成功后执行。订阅新增 Port 位于 `application/subscription/write.py`，由
|
||||
`db/adapters/subscription.py` 创建独占 Session，`startup/composition/subscription.py` 只装配回调，
|
||||
`application/subscription/write.py` 决定事务与 post-commit 边界，`SubscribeOper.stage_add()`
|
||||
只查重、`add` 和 `flush`。旧 SDK 显式构造的无会话 Oper 暂留兼容自动短会话，不得被新代码复用。
|
||||
`transaction-debt-baseline.json` 当前要求正式只读查询装饰器保持为 0；原有同步/异步写装饰器
|
||||
@@ -384,7 +385,7 @@ flowchart LR
|
||||
`create/update/delete/truncate` 兼容包装器；AST 门禁保证显式 Session 的提交权不会被底层抢走。
|
||||
- 站点、历史、工作流、Agent 会话删除和插件数据重置已经形成同构事务切片；对应 Application
|
||||
Command/Service 持有 UoW，Oper 的 `stage_*` 方法只修改当前会话。插件数据重置从
|
||||
`startup/plugins_initializer.py` 创建独占会话，插件直接使用 `PluginDataOper` 的旧 ABI 仅作兼容。
|
||||
`startup/initializers/plugins.py` 注入事务能力，插件直接使用 `PluginDataOper` 的旧 ABI 仅作兼容。
|
||||
- 每次表结构变更必须新增 `database/versions/` 下的 Alembic 迁移。
|
||||
- 运行期业务配置使用 `SystemConfigKey` 枚举 + `SystemConfigOper`，禁止裸字符串键；
|
||||
用户级配置使用 `UserConfigOper`。
|
||||
@@ -489,7 +490,7 @@ Agent 采用**门面 + 惰性物化**设计，避免 `application → agent` 形
|
||||
```mermaid
|
||||
flowchart TB
|
||||
Entry["消息渠道 / API / MCP"] --> Facade["app/application/agent.py<br/>编排门面（get_agent_manager 等）"]
|
||||
Reg["app/startup/agent_initializer.py<br/>导入期注册轻量 Provider"]
|
||||
Reg["app/startup/initializers/agent.py<br/>导入期注册轻量 Provider"]
|
||||
Facade -.能力启用或首次使用时物化.-> RT["app/agent/runtime_loader.py<br/>能力发现与服务物化"]
|
||||
RT --> ORC["app/agent/orchestrator.py<br/>会话编排"]
|
||||
ORC --> Tools["app/agent/tools<br/>系统工具（经 application 门面）"]
|
||||
|
||||
@@ -119,7 +119,7 @@ MoviePilot V3 已经完成一轮重要基础工作：原 `app/core`、`app/helpe
|
||||
| 模块 | 静态出度 | 主要原因 |
|
||||
| --- | ---: | --- |
|
||||
| `app.agent.tools.factory` | 99 | 一次性导入全部内置工具并维护集中注册表 |
|
||||
| `app.startup.modules_initializer` | 55 | 组合根职责，这是合理高出度，但仍需声明式管理 |
|
||||
| `app.startup.initializers.modules` | 55 | 组合根职责，这是合理高出度，但仍需声明式管理 |
|
||||
| `app.api.endpoints.system` | 54 | 系统设置、规则测试、日志、网络测试、运行控制混合 |
|
||||
| `app.api.deps` | 49 | 认证、插件配置和跨端点依赖装配集中 |
|
||||
| `app.agent.orchestrator` | 48 | Agent 构建、执行、工具、记忆、审计、用量混合 |
|
||||
@@ -425,7 +425,7 @@ app/chain/transfer.py # 保持 TransferChain 兼容门面
|
||||
|
||||
#### 已有进展
|
||||
|
||||
`app/startup/modules_initializer.py:211-245` 已经承担托管资源、壁纸 Provider、认证载荷、DoH、站点、事件错误通知、模块、Agent 和前端的组合工作。`app/startup/lifecycle.py` 也显式规定数据库预热、路由、模块、插件、调度器、监控器、命令和工作流的顺序。这是正确方向。
|
||||
`app/startup/initializers/modules.py` 承担托管资源、壁纸 Provider、认证载荷、DoH、站点、事件错误通知、模块、Agent 和前端的组合工作。`app/startup/lifecycle/` 显式规定数据库预热、路由、模块、插件、调度器、监控器、命令和工作流的顺序。这是正确方向。
|
||||
|
||||
#### 历史泄漏与当前收口
|
||||
|
||||
@@ -701,7 +701,7 @@ app/application/server/share.py # 订阅/工作流等分享用例
|
||||
|
||||
#### 典型证据
|
||||
|
||||
- `app/application/messaging/skill.py` 通过 `SkillCatalogPort` 消费技能目录，`app.startup.agent_initializer` 才导入并注入 `SkillHelper`。
|
||||
- `app/application/messaging/skill.py` 通过 `SkillCatalogPort` 消费技能目录，`app.startup.initializers.agent` 才导入并注入 `SkillHelper`。
|
||||
- `app/application/plugin/routes.py` 持有 `DynamicRouteRegistry` Protocol 和注册/移除用例；FastAPI app、`app.routes`、`openapi_schema` 和 `setup()` 均封装在 `app/adapters/web/plugin/routes.py`。
|
||||
- 多个 `modules` 直接导入 `app.application.messaging.agent`、`mediaserver`、`storage` 等；其中一部分是合理 SPI 消费，一部分表明应用能力接口和具体实现未区分。
|
||||
- `SystemConfigOper()` 在大量文件中被直接构造，形成持久化配置服务定位器。
|
||||
|
||||
@@ -15,8 +15,8 @@
|
||||
|
||||
### 长期整改阶段 0：治理门禁恢复（2026-08-23）
|
||||
|
||||
- 宿主依赖基线已审查 TaskRegistry 与有界后台 owner 接入后的语义差异：当前为 `800` 个模块、`6482` 条内部导入边，12 组重点禁止边继续全部为 `0`，唯一非平凡 SCC 仍是隔离的 TMDB 移植包。
|
||||
- 启动性能探针会在隔离生命周期中真实创建并释放 TaskRegistry；normal/safe 组件数分别为 `20`/`10`，CI 只读检查使用稳定的宿主模块集合和生命周期组件顺序，不再把 Python/平台模块数量当作硬合同。
|
||||
- 宿主依赖基线已审查本阶段归位后的语义差异：当前为 `805` 个模块、`6514` 条内部导入边，12 组重点禁止边继续全部为 `0`，唯一非平凡 SCC 仍是隔离的 TMDB 移植包。
|
||||
- 启动性能探针会在隔离生命周期中真实创建并释放 TaskRegistry；normal/safe 组件数分别为 `23`/`11`，CI 只读检查使用稳定的宿主模块集合和生命周期组件顺序，不再把 Python/平台模块数量当作硬合同。
|
||||
- 官方插件快照覆盖 `plugins.v3`、`plugins.v2` 以及 V3 实际会从 `package.json` 回退加载的 31 个默认实现；`app/plugins/**` 仍只是宿主运行副本，不进入扫描。
|
||||
- SDK 快照以各模块显式 `__all__` 为公开合同，能够记录赋值别名；`typing`、`__future__` 等实现期导入不再被误冻结，既有数据库备份门面已补精确导出清单。
|
||||
- async 阻塞实际债务已由 fixture 中的 10 项下降到 1 项并固化低水位；剩余项是 Scheduler Agent task 查询，后续阶段迁入异步查询边界后归零。
|
||||
@@ -72,7 +72,7 @@
|
||||
|
||||
- 继续采用单进程控制面是正确选择，不建议现在拆成微服务；插件、调度器、工作流、事件和数据库共享进程内状态，拆分会放大部署、事务和兼容成本。
|
||||
- `foundation/domain/runtime/adapters/application/chain/api/startup` 的职责方向基本成立；宿主架构基线、复杂度 ratchet、异步阻塞 ratchet 当前均通过。
|
||||
- 依赖图当前为 `800` 个 Python 模块、`6479` 条内部导入边；唯一非平凡 SCC 位于隔离的 TMDB 第三方移植包内部，不应为了指标归零重写。
|
||||
- 依赖图当前为 `805` 个 Python 模块、`6514` 条内部导入边；唯一非平凡 SCC 位于隔离的 TMDB 第三方移植包内部，不应为了指标归零重写。
|
||||
- 当前主要风险已经从“目录和依赖失控”转移到运行时协议、后台副作用的可靠性和遗留兼容面。换言之，下一阶段重点应是**语义收口和可验证性**，而不是继续搬文件或机械拆大文件。
|
||||
|
||||
综合评价：架构方向可持续，生产可用性较高；可演进性仍处于中等水平。现阶段没有静态审计发现必须立即推倒重来的 P0 架构问题，但存在需要按 P1/P2 计划治理的真实债务。
|
||||
@@ -82,7 +82,7 @@
|
||||
1. **后台任务的统一所有权已覆盖 API 入口，但仍有更深层任务机制待分级。** `app/runtime/tasks.py` 已建立 lifespan 级 TaskRegistry，启动收尾、插件 Release 刷新、Webhook E0 广播、CookieCloud E1 手工调度、消息入口、Seerr 订阅、整理历史 AI 重做、OpenAI/Anthropic 协议流和 WebAgent 断线后执行/快照保存均不再维护端点模块级任务集合或 Starlette 回调，shutdown 会停止接收、取消并有限等待，且生命周期清单明确登记其顺序。主仓 `app/` 已无裸 FastAPI `BackgroundTasks`；当前仍有约 `50` 个更底层 `create_task`/等价任务创建点，与线程池和 APScheduler 并存，后续需逐项确认 owner、取消、等待、重试、幂等和是否 durable，关键业务副作用优先接入已有 Outbox/恢复表。
|
||||
2. **动态模块契约仍以 legacy 聚合语义为主。** 当前登记 `212` 个模块方法，其中 `194` 个仍使用 `legacy` aggregation，只有 `14` 个 `first_non_empty`、`4` 个 `ordered_list_merge`。`app/runtime/extensions/module/contracts.py:422-455` 已能登记 family、输入/结果标签和基础签名诊断，但 `193` 个方法没有 required parameters，调度器 `app/runtime/extensions/module/dispatcher.py:109-260` 仍主要依赖运行时反射、返回值形状和短路规则。未知第三方方法保留 legacy fallback 是兼容要求，不应删除；宿主高频能力则应逐族补齐可执行的输入校验、结果校验、超时和错误语义。
|
||||
3. **查询侧数据库兼容 ABI 已完成正式装饰器清零。** 写事务装饰器和正式 `db_query/async_db_query` 均为 `0`。站点、消息、用户、订阅、下载/整理历史、工作流、MediaServer、SiteUserData、AgentChat、AgentTaskRun、TransferPending、SystemConfig、PassKey 和 SubscribeHistory 的宿主查询已迁到显式 Session 路径；对应旧插件 Model 调用由独立 `legacy_*` 外壳保留，可同时接受显式 Session 与无 Session 的位置/关键字参数。后续重点转为减少 ORM 对象跨层流转，并保持正式装饰器零回退。
|
||||
4. **组合根和全局状态仍形成复杂的隐式运行时图。** Singleton 实例、模块级 provider、`configure_*` 注册函数和兼容 Facade 同时存在；它们解决了旧 ABI 和启动顺序问题，但增加测试污染、重复装配、实例身份和初始化顺序风险。`app/startup/lifecycle/__init__.py:161-376` 已有声明式生命周期，`app/startup/modules_initializer.py:505-530` 也有分阶段关闭，但尚未做到所有进程级资源都只通过 typed HostRuntime 访问。后续应以“新代码禁止新增 Service Locator/Singleton 依赖、旧入口有命中观测”为 ratchet。
|
||||
4. **组合根和全局状态仍形成复杂的隐式运行时图。** Singleton 实例、模块级 provider、`configure_*` 注册函数和兼容 Facade 同时存在；它们解决了旧 ABI 和启动顺序问题，但增加测试污染、重复装配、实例身份和初始化顺序风险。`app/startup/lifecycle/__init__.py` 已有声明式生命周期，`app/startup/initializers/modules.py` 也有分阶段关闭，但尚未做到所有进程级资源都只通过 typed HostRuntime 访问。后续应以“新代码禁止新增 Service Locator/Singleton 依赖、旧入口有命中观测”为 ratchet。
|
||||
|
||||
### P2：中长期可演进性债务
|
||||
|
||||
@@ -529,7 +529,7 @@ flowchart TB
|
||||
|
||||
**实施记录（2026-08-21）**：
|
||||
|
||||
- `app/startup/subscription.py` 为每次规范新增创建独占同步/异步 Session；
|
||||
- `app/application/subscription/write.py` 定义用例 Port，`app/db/adapters/subscription.py` 为每次规范新增创建独占同步/异步 Session，`app/startup/composition/subscription.py` 只负责注入；
|
||||
`CreateSubscriptionCommand` / `AsyncCreateSubscriptionCommand` 持有 UoW，Oper 只执行
|
||||
查重、`add` 与 `flush`。
|
||||
- `SubscribeOper.stage_add()` 的查重 SQL 已收口到 Oper，不再调用 Model 自动会话装饰器；
|
||||
@@ -584,7 +584,7 @@ flowchart TB
|
||||
**建议结构**：
|
||||
|
||||
```text
|
||||
app/startup/context.py # HostRuntime 及构建结果
|
||||
app/startup/composition/context.py # HostRuntime 及构建结果
|
||||
app/api/context.py # API 可见的最小 AppState / 读取依赖
|
||||
app/api/dependencies/ # 按领域拆分依赖工厂
|
||||
auth.py
|
||||
@@ -612,7 +612,7 @@ app/api/dependencies/ # 按领域拆分依赖工厂
|
||||
|
||||
**实施记录（2026-08-21）**：
|
||||
|
||||
- `app/startup/context.py` 定义 frozen slots `HostRuntime` 与首个窄能力
|
||||
- `app/startup/composition/context.py` 定义 frozen slots `HostRuntime` 与首个窄能力
|
||||
`AgentChatRuntime`，仓储、Session、UoW 字段均为具体 Protocol 工厂，不是字符串字典。
|
||||
- `init_modules()` 保留零参数兼容签名并返回本次 lifespan 唯一 Runtime；生命周期组件把结果挂到
|
||||
`app.state.host_runtime`。`app/api/context.py` 只向 Depends 暴露 Agent chat 的最小能力。
|
||||
@@ -707,7 +707,7 @@ app/api/dependencies/ # 按领域拆分依赖工厂
|
||||
Startup 路径始终使用 HostRuntime 注入。插件 SDK 的 `app.sdk.config.settings`、动态 API 返回和事件字段未改。
|
||||
- 收尾批次把 API 与 Chain 余下直接配置读取全部迁入类型化 snapshot；Scheduler 继续保持为零。
|
||||
`HostRuntime` 新增可变部署设置服务，只供系统设置管理 API 使用，业务 API/Chain 只接收 frozen 字段。
|
||||
snapshot 构造集中到 `app/startup/configuration.py`，生产启动与测试组合根复用同一映射，避免测试默认值
|
||||
snapshot 构造集中到 `app/startup/composition/configuration.py`，生产启动与测试组合根复用同一映射，避免测试默认值
|
||||
漂移。canonical `settings` 直接导入低水位从 154 降到 137，`SystemConfigOper()` 保持 14 个。
|
||||
- `ApiRuntimeConfig` 已覆盖搜索来源、媒体/字幕/音频后缀、重命名格式、WebPush、CookieCloud、根目录和
|
||||
版本标识；`ChainRuntimeConfig` 覆盖搜索、下载、整理、刮削、AI、代理、缓存、链接、路径和 TMDB 图片域。
|
||||
@@ -1080,7 +1080,7 @@ OTel 初始化只能位于 Startup/Adapter；Domain/Application 只依赖 no-op-
|
||||
- 新增 Application command/port
|
||||
- `app/runtime/event/`
|
||||
- `app/runtime/extensions/module/contracts.py`
|
||||
- `app/startup/context.py` / `app/api/context.py`
|
||||
- `app/startup/composition/context.py` / `app/api/context.py`
|
||||
3. 对第三方移植包、旧插件 Facade 和动态 SDK 设置精确豁免，不允许 `app.* = ignore_errors`。
|
||||
4. CI 先检查严格目录；每次迁移扩大 include 范围。
|
||||
5. 类型错误不能用无界 `Any`、`cast(Any, ...)` 或全文件 ignore 消音。
|
||||
@@ -1099,7 +1099,7 @@ OTel 初始化只能位于 Startup/Adapter；Domain/Application 只依赖 no-op-
|
||||
**扩展实施记录（2026-08-22）**：mypy 目标运行时更新到 Python 3.14，严格清单扩大到 20 个源文件；
|
||||
新增纳管配置快照和下载失败事务适配器，仍保持零错误、无全局 ignore。
|
||||
|
||||
Workflow 执行状态 UoW 切片将 `app/application/workflow.py` 与 `app/startup/workflow.py` 纳入 strict 清单，
|
||||
Workflow 执行状态 UoW 切片将 `app/application/workflow.py` 与 `app/db/adapters/workflow.py` 纳入 strict 清单，
|
||||
治理范围扩大到 22 个源文件；事务命令、仓储 Protocol 和短会话适配器保持零错误。
|
||||
|
||||
异步安全与契约收口继续纳管 scheduling facade、Event error policy、Module dispatcher 和 async blocking
|
||||
|
||||
@@ -519,7 +519,7 @@ SYMBOL_ALIASES = {
|
||||
4. Event、模块、插件和安全边界改为由 startup composition root 注入 resolver、回调和错误处理器，迁移模块不再处于强连通分量。
|
||||
5. 插件稳定入口收敛到 `app.sdk`；存量插件无需同步修改，官方插件可以按正常发布节奏迁移。
|
||||
6. 站点二进制和数据资源迁到 `app/application/site`，Build 直接生成 `app.application.site.sites`，Build CI、Resources V3 manifest、Docker 和本地 CLI 使用同一目标路径。
|
||||
7. 媒体识别领域不再直接读取 DB/settings，也不导入 Rust 适配器；`startup/domain_initializer.py` 统一注入实时规则、后缀策略、TMDB 图片地址、默认媒体来源和加速器。
|
||||
7. 媒体识别领域不再直接读取 DB/settings，也不导入 Rust 适配器；`startup/initializers/domain.py` 统一注入实时规则、后缀策略、TMDB 图片地址、默认媒体来源和加速器。
|
||||
8. 缓存按职责拆为 `runtime/cache.py`（契约、内存实现、装饰器、代理）和 `adapters/cache/backends.py`（Redis、文件 I/O）；旧 `app.core.cache` 指向完整 `app.sdk.cache` 门面。
|
||||
9. `application/mediaserver.py` 集中负责媒体服务器的配置化服务发现、Provider ID 规范化和音乐库匹配；通用媒体身份规则继续复用 `domain/media.py`。
|
||||
10. GC 归入 `runtime/gc.py`，外部 IP 归属查询归入 `adapters/external/location.py`，安全能力统一在 `app/application/security/`，URL 安全策略为 `url.py`，二次认证文件为 `twofactor.py`。
|
||||
|
||||
@@ -72,10 +72,13 @@ to make the directory tree look symmetrical.
|
||||
| `app/application/messaging/` | Message rendering/routing, interactions and the Agent-to-message bridge: `interaction.py` shared interaction contracts and view helpers; `router.py` unified interaction priority and callback dispatch; `site.py`/`subscribe.py`/`skill.py` per-command sessions, input parsing and views; `media.py` media interaction state while the business workflow stays in `MediaInteractionChain`; `plugin.py` plugin input capture and plugin button callbacks; `agent.py` agent choice state, callback protocol and WebAgent bridge; `message.py` notification rendering, templates and queue. Not a public SDK recommended for direct plugin use |
|
||||
| `app/application/security/` | Authentication, authorization, cookies, passkeys, OTP/two-factor, path/URL safety, SSRF and signing policy |
|
||||
|
||||
Application services may use domain rules, runtime contracts, Oper classes and
|
||||
adapters. Multi-domain workflows still belong in the existing `app/chain/`
|
||||
package. `Chain`, `Service` and `Manager` remain class patterns; they do not
|
||||
create additional top-level directory categories.
|
||||
Application services may use domain rules and runtime contracts. They own the
|
||||
persistence Protocol needed by a use case, but must not import `app.db`,
|
||||
SQLAlchemy, Session, Oper classes or concrete adapters. `app/db/adapters/`
|
||||
implements those Protocols and startup injects the implementation. Multi-domain
|
||||
workflows still belong in the existing `app/chain/` package. `Chain`, `Service`
|
||||
and `Manager` remain class patterns; they do not create additional top-level
|
||||
directory categories.
|
||||
|
||||
### Runtime boundaries
|
||||
|
||||
@@ -94,8 +97,11 @@ create additional top-level directory categories.
|
||||
| `app/runtime/compat/` | Standard-library-only exact legacy import routing, resource preflight scanning and DEBUG diagnostics |
|
||||
|
||||
`app/startup/` remains the established composition root and is not nested under
|
||||
runtime. It injects providers and callbacks, orders initialization/shutdown and
|
||||
decides restart policy. Lower-level runtime modules must not import startup.
|
||||
runtime. Its root contains only `composition/`, `initializers/` and `lifecycle/`:
|
||||
composition constructs and injects cross-layer dependencies, initializers expose
|
||||
domain-scoped startup/shutdown hooks, and lifecycle orders those hooks and decides
|
||||
restart policy. Reusable persistence implementations belong in `app/db/adapters/`,
|
||||
not startup. Lower-level runtime modules must not import startup.
|
||||
Startup publishes its frozen, slotted `HostRuntime` through FastAPI `app.state`.
|
||||
API dependencies must narrow that object to a domain runtime (for example,
|
||||
`AgentChatRuntime`) instead of adding a string key to a global service map.
|
||||
@@ -105,14 +111,20 @@ API, Scheduler and Chain deployment values are exposed as frozen snapshots from
|
||||
`HostRuntime.configuration`; canonical callers must not add a fresh direct
|
||||
`settings` import when the required field belongs to an existing snapshot.
|
||||
|
||||
`app.schemas` and `app.db` are compatibility facades, not implementation
|
||||
dependency hubs. Host code imports concrete schema submodules; the schema root
|
||||
`app.schemas` and the `app.db` package root are compatibility facades, not
|
||||
implementation dependency hubs. Host code imports concrete schema submodules; the schema root
|
||||
resolves its generated export manifest lazily for plugins and legacy callers.
|
||||
DB internals import `base`, `decorators`, `engine`, `session`, concrete models
|
||||
and Oper modules directly. `app.db.models.load_all_models()` is the explicit
|
||||
composition entry used before metadata creation or migration; importing one
|
||||
model must not import every table.
|
||||
|
||||
`app/db/oper/` owns table-oriented SQLAlchemy access and receives a caller-owned
|
||||
Session. `app/db/adapters/` is the concrete persistence-adapter layer: it may
|
||||
depend on Application-owned Protocols, UoW/Session and Oper implementations.
|
||||
This deliberate dependency inversion is the only `DB implementation ->
|
||||
Application contract` direction; Application must remain free of DB imports.
|
||||
|
||||
### Adapter boundaries
|
||||
|
||||
| Path | Ownership |
|
||||
@@ -123,6 +135,7 @@ model must not import every table.
|
||||
| `app/adapters/external/` | CookieCloud, plugin market, OCR, IP-location providers and MoviePilot Server |
|
||||
| `app/adapters/external/plugin/client.py` | Read-only plugin-market and local-repository client over the established `PluginHelper` implementation |
|
||||
| `app/adapters/system/plugin/` | Plugin package and dependency I/O (`package.py`, `dependency.py`) |
|
||||
| `app/db/adapters/` | SQLAlchemy implementations of Application-owned persistence Protocols |
|
||||
|
||||
Generic protocol transport belongs in `adapters/network`; a named product or
|
||||
ecosystem workflow belongs in `adapters/external`. An adapter may depend on
|
||||
@@ -381,7 +394,7 @@ policy. `app/db` therefore has no dependency on `app/domain`.
|
||||
emits no runtime logs; upper-layer owners decide whether failures are
|
||||
operationally relevant.
|
||||
- `app/adapters/system/resource.py` only reports whether installation occurred;
|
||||
`app/startup/modules_initializer.py` supplies the loaded site-resource
|
||||
`app/startup/initializers/modules.py` supplies the loaded site-resource
|
||||
versions and decides whether to restart. The adapter never imports the site
|
||||
application service.
|
||||
- Configured notification discovery lives in
|
||||
@@ -407,13 +420,15 @@ policy. `app/db` therefore has no dependency on `app/domain`.
|
||||
|
||||
| Direction | Status |
|
||||
|---|---|
|
||||
| `entrypoint -> chain / application / Oper` | Allowed according to workflow complexity |
|
||||
| `chain -> module (only via run_module dispatch) / application / Oper / canonical capability` | Allowed; direct `chain -> module` imports forbidden |
|
||||
| `chain -> agent implementation` | Forbidden; chains reach Agent runtime only through `app/application/agent.py`; `app/startup/agent_initializer.py` registers lightweight providers at import time, and implementations are materialized only when the capability is enabled or first used |
|
||||
| `entrypoint -> chain / application / injected persistence Port` | Allowed according to workflow complexity |
|
||||
| `chain -> module (only via run_module dispatch) / application / injected Port / canonical capability` | Allowed; direct `chain -> module` and `chain -> Oper` imports forbidden |
|
||||
| `chain -> agent implementation` | Forbidden; chains reach Agent runtime only through `app/application/agent.py`; `app/startup/initializers/agent.py` registers lightweight providers at import time, and implementations are materialized only when the capability is enabled or first used |
|
||||
| `agent.tools -> api / scheduler / command` | Forbidden; tools use `app/application/plugin/routes.py`, `plugin/folders.py`, `scheduling.py` and `commands.py` application services |
|
||||
| `api -> factory` | Forbidden; the FastAPI route adapter is injected into `app/application/plugin/routes.py` by the composition root after creation |
|
||||
| `application -> domain / runtime / adapter / Oper` | Allowed |
|
||||
| `module -> canonical capability / Oper` | Allowed |
|
||||
| `application -> domain / runtime contract` | Allowed |
|
||||
| `application -> DB / Oper / concrete adapter` | Forbidden; define a Protocol in Application and inject an implementation |
|
||||
| `db.adapters -> application persistence Protocol / db.oper / UoW` | Allowed; this is dependency inversion, not an upper-layer use-case call |
|
||||
| `module -> canonical capability / Application persistence Port` | Allowed; direct Oper imports are forbidden for new code |
|
||||
| `module -> module / chain` | Forbidden for new code |
|
||||
| `adapter -> application / runtime.extensions / sdk / compat` | Forbidden |
|
||||
| `domain -> runtime / adapter / application / DB` | Forbidden |
|
||||
@@ -426,11 +441,14 @@ policy. `app/db` therefore has no dependency on `app/domain`.
|
||||
|
||||
| Path | Purpose |
|
||||
|---|---|
|
||||
| `app/application/agent.py` | Agent orchestration facade (`get_agent_manager` / `get_prompt_manager` / capability queries / prompt builders); lightweight providers register through `app/startup/agent_initializer.py`, with no static `application -> agent` edge |
|
||||
| `app/application/agent.py` | Agent orchestration facade (`get_agent_manager` / `get_prompt_manager` / capability queries / prompt builders); lightweight providers register through `app/startup/initializers/agent.py`, with no static `application -> agent` edge |
|
||||
| `app/agent/runtime_loader.py` | Agent-specific capability discovery and canonical entrypoint/service materialization; reuses the generic Capability Runtime while keeping Agent ownership under `app/agent/` |
|
||||
| `app/application/subscription/write.py` | Subscription media translation and sync/async write-port orchestration |
|
||||
| `app/application/scheduling.py` | Runtime scheduler facade for Agent tools and endpoints; `Scheduler` class registered by `app/startup/scheduler_initializer.py` |
|
||||
| `app/application/commands.py` | Command registry facade for Agent tools and endpoints; `Command` class registered by `app/startup/command_initializer.py` |
|
||||
| `app/application/scheduling.py` | Runtime scheduler facade for Agent tools and endpoints; `Scheduler` class registered by `app/startup/initializers/scheduler.py` |
|
||||
| `app/application/commands.py` | Command registry facade for Agent tools and endpoints; `Command` class registered by `app/startup/initializers/command.py` |
|
||||
| `app/db/adapters/` | SQLAlchemy repository/UoW implementations for Application-owned persistence Protocols |
|
||||
| `app/startup/composition/` | HostRuntime, configuration snapshots and cross-layer adapter wiring |
|
||||
| `app/startup/initializers/` | Domain-scoped initialization and shutdown hooks |
|
||||
| `app/chain/agent.py` | `AgentChain(ChainBase)`: the chain-layer entry for Agent sessions; Agent runtime stays in `app/agent/` |
|
||||
| `app/runtime/config.py` | `ConfigModel`, `Settings` and deployment configuration |
|
||||
| `app/runtime/topology.py` | Single-worker full-runtime policy and safe-mode topology validation |
|
||||
|
||||
@@ -116,7 +116,9 @@ except:
|
||||
|
||||
- Do not introduce new third-party libraries without placing them in the correct `pyproject.toml` dependency group and updating `uv.lock`: runtime packages belong in `[project].dependencies`, test/lint/build tooling in `[dependency-groups].dev`.
|
||||
- Do not use `requests` or `httpx` directly for external HTTP calls - host code uses `RequestUtils` from `app/adapters/network/http.py`; plugins use `app.sdk.network`.
|
||||
- Do not issue raw SQLAlchemy queries from chains, modules, or endpoints — use the Oper classes in `app/db/oper/`.
|
||||
- Do not issue raw SQLAlchemy queries or import Oper classes from chains, modules,
|
||||
or endpoints. Define/consume an Application persistence Port; its concrete
|
||||
implementation under `app/db/adapters/` may use Oper classes from `app/db/oper/`.
|
||||
- Do not add TODO or FIXME without context. Only keep one if it is genuinely deferred and cannot be addressed in the current task.
|
||||
- Do not add noisy markers like `# change starts here`, `# important`, or `# this is a fix`.
|
||||
- Do not write comments that restate what the code already clearly says.
|
||||
|
||||
@@ -81,33 +81,49 @@ the stub.
|
||||
Oper classes accept and return persistence values. Turning a `MediaInfo` or
|
||||
`MetaBase` into a row is business logic and lives in `app/application/`.
|
||||
|
||||
Application owns use-case commands and persistence Protocols, but does not import
|
||||
`app.db`, SQLAlchemy, Session or Oper. Concrete persistence is used in
|
||||
`app/db/adapters/`: adapters implement those Protocols with explicit Session,
|
||||
UnitOfWork and Oper objects. `app/startup/composition/` creates and injects the
|
||||
adapters; it does not retain reusable repository implementations.
|
||||
|
||||
### Transaction ownership ratchet
|
||||
|
||||
- `tests/fixtures/architecture/transaction-debt-baseline.json` records the
|
||||
existing Model transaction decorators. All formal query and write decorators
|
||||
are now zero and must remain zero; compatibility-only `legacy_*` shells must
|
||||
not be counted as new transaction ownership.
|
||||
- `legacy_db_query` / `legacy_async_db_query` are compatibility-only shells for
|
||||
existing plugin-facing Model methods. Host Oper code must pass an explicit
|
||||
Session through `_execute_sync_query` / `_execute_async_query`; new Model
|
||||
methods must not add either legacy decorator.
|
||||
- `tests/fixtures/architecture/transaction-debt-baseline.json` records formal
|
||||
decorators in concrete files under `app/db/models/`. Their count is zero and
|
||||
must remain zero. Compatibility-only `legacy_*` shells are tracked separately
|
||||
and must never be treated as the target design.
|
||||
- `legacy_db_query` / `legacy_async_db_query` preserve an existing plugin-facing
|
||||
Model method whose no-Session call shape cannot be removed yet. If a Model
|
||||
method has no external ABI obligation, move the query into its Oper and remove
|
||||
the Model method instead of adding `legacy_*`.
|
||||
- `Base.create/get/update/delete/list/truncate` and their async forms are inherited
|
||||
plugin ABI, so `app/db/base.py` deliberately uses legacy query/write wrappers.
|
||||
New host code must not call these convenience methods; Oper staging methods and
|
||||
explicit UoW are the canonical path. Removal requires plugin-usage evidence and
|
||||
a separately announced compatibility break, not a mechanical rename.
|
||||
- Host Oper code must pass an explicit Session through `_execute_sync_query` /
|
||||
`_execute_async_query`; new Model methods must not add any legacy decorator.
|
||||
- New Model methods must not use `db_query`, `db_update`, `async_db_query`, or
|
||||
`async_db_update`, create a Session, or call `commit()` / `rollback()`.
|
||||
- Oper receives a caller-owned Session and may query, add, update, delete, or
|
||||
flush. A composable Oper method must not create its own Session and must not
|
||||
commit or roll back.
|
||||
- The API, Scheduler, Agent, or another logical operation entry creates the
|
||||
Session and adapts it through `app/db/uow.py`. Application command code owns
|
||||
`commit()` / `rollback()`; events, scheduling refresh, reports, and other
|
||||
external effects run only after a successful commit.
|
||||
- API, Scheduler, Agent and Chain consume an injected Application Port; they do
|
||||
not import or create a Session. The concrete `app/db/adapters/` implementation
|
||||
creates the Session and adapts it through `app/db/uow.py`. Application command
|
||||
code decides when the injected UoW commits or rolls back; events, scheduling
|
||||
refresh, reports and other external effects run only after a successful commit.
|
||||
- A synchronous Session is private to one worker thread. An AsyncSession is
|
||||
private to one asyncio task/operation; neither may be stored in a process
|
||||
singleton or reused by concurrent work.
|
||||
- Subscription creation is the reference slice: `app/startup/subscription.py`
|
||||
creates an exclusive Session, `app/application/subscription/write.py` owns the
|
||||
UoW and post-commit callback, and `SubscribeOper.stage_add()` only queries,
|
||||
adds, and flushes. Preserve `SubscribeOper.add()` only for legacy SDK callers;
|
||||
new host code must not use that auto-commit compatibility path.
|
||||
- Subscription creation is the reference slice:
|
||||
`app/application/subscription/write.py` owns the command and persistence Port,
|
||||
`app/db/adapters/subscription.py` creates an exclusive Session and adapts Oper/UoW,
|
||||
and `app/startup/composition/subscription.py` only wires scopes and post-commit
|
||||
callbacks. `SubscribeOper.stage_add()` only queries, adds and flushes. Preserve
|
||||
`SubscribeOper.add()` only for legacy SDK callers; new host code must not use
|
||||
that auto-commit compatibility path.
|
||||
- The same rule applies to `SiteMutationCommand`, history/workflow commands,
|
||||
`AgentChatService.delete()`, and `DeletePluginDataCommand`: bind the repository
|
||||
and UoW to one request/operation Session. Legacy plugin-facing Oper methods may
|
||||
@@ -117,7 +133,18 @@ Run `./.venv/bin/python scripts/architecture/baseline.py --check-host` after
|
||||
persistence changes. A deliberate debt reduction may refresh the low-water mark
|
||||
with `--write-host`; never refresh it to accept newly introduced debt.
|
||||
|
||||
**Standard Oper method conventions:**
|
||||
**Canonical explicit-session Oper conventions:**
|
||||
|
||||
```python
|
||||
with SessionFactory() as session:
|
||||
oper = SubscribeOper(session)
|
||||
subscribe = oper.get(sid=1) # Query in caller-owned Session
|
||||
subscribes = oper.list() # List in caller-owned Session
|
||||
oper.stage_add(Subscribe(...)) # Stage only; caller-owned UoW commits
|
||||
```
|
||||
|
||||
The following no-Session form is legacy plugin ABI only and must not be copied
|
||||
into host code:
|
||||
|
||||
```python
|
||||
oper = SubscribeOper()
|
||||
|
||||
Reference in New Issue
Block a user