diff --git a/docs/architecture-optimization-checklist.md b/docs/architecture-optimization-checklist.md index 51d5b9ce3..a93d61338 100644 --- a/docs/architecture-optimization-checklist.md +++ b/docs/architecture-optimization-checklist.md @@ -156,7 +156,7 @@ MoviePilot V3 已经形成较清晰的模块化单体：`foundation`、`domain` 与生产路径显式 UoW 规则冲突；该项已由 S0-L2.1 修复并增加文档门禁。 - `docs/rules/05-architecture.md:507` 禁止 `application -> concrete adapter`，但同一文档 `177-183` 又要求 RSS 消费 network adapter；当前测试没有形式化这类例外。 -- 规则 `docs/rules/05-architecture.md:516` 禁止任意模块级环，实际 SCC 测试只覆盖特定根。 +- 审计时完整 SCC 只进入生成快照，语义测试只覆盖特定根；S0-L2.2 已增加完整宿主 SCC policy 门禁。 - 架构总览此前仍记录 811 模块、6,572 条边和 1 个 SCC，已经落后于当前基线。 - Event consumer 扫描把任意名为 `.register()` 的调用都当成事件注册，存在明确误报。 @@ -165,7 +165,7 @@ MoviePilot V3 已经形成较清晰的模块化单体：`foundation`、`domain` - [ ] 指定一个机器可读事实源，文档指标由 fixture 生成或只保留不易漂移的语义描述。 - [x] 修正 Oper 示例，分别展示宿主显式 UoW 与插件兼容 Facade，并以文档测试禁止回退。 - [ ] 对 Adapter 规则作出明确决策：允许哪些通用技术机制，哪些命名外部/持久化能力必须注入。 -- [ ] 让 SCC 规则、精确 allowlist 和文档声明一致。 +- [x] 让 SCC 规则、精确 policy 和文档声明一致；Chain 临时债务与 TMDB vendor containment 分开治理。 - [ ] Event 扫描只识别 EventManager 实例/别名和事件装饰器。 - [ ] CI 分开报告“快照一致”与“语义规则通过”，禁止把前者表述为架构完全正确。 diff --git a/docs/architecture-overview.md b/docs/architecture-overview.md index dbe8937e1..152db6299 100644 --- a/docs/architecture-overview.md +++ b/docs/architecture-overview.md @@ -668,6 +668,9 @@ flowchart LR 物理遗留源码、禁止的上向依赖、SDK/compat 反向引用、包含迁移模块的强连通分量、 模块间/模块到 Chain 的 import、入口层对 `app.modules` 内部的 import、 Chain 直接 import 模块内部（必须走 `run_module` 分发）、`app/chain` 内的下载器 SDK 依赖。 +- `tests/fixtures/architecture/dependency-baseline.json` 记录生成事实；人工审查的 SCC 分类单独存入 + `dependency-policy.json`。完整宿主 SCC 必须精确匹配 policy，新增、扩大、变形和陈旧 policy 都失败； + `--write-host` 不会替代人工决策。 - 任何所有权迁移必须同步更新：canonical 导入、`app/runtime/compat/manifest.py`、 SDK 导出（若公开）、`docs/rules/05-architecture.md` 与上述架构测试。 - 延迟导入不被接受为隐藏循环依赖的手段。 @@ -681,7 +684,7 @@ flowchart LR |---|---:| | Python 模块 | 835 | | 内部导入边 | 6,810 | -| 非平凡 SCC | 2（Chain 包根环与隔离的 TMDB 移植包环） | +| 非平凡 SCC | 2（`ARCH-107` 临时 Chain 包根环；精确 containment 的 TMDB 移植包环） | | Module Contract V2 spec | 215（其中 214 个进入 `run_module` 观察面） | | Event Contract | 53 | | Model/Oper 自动事务与自建 Session | 0 | diff --git a/docs/architecture-refactor-roadmap.md b/docs/architecture-refactor-roadmap.md index 867f68a32..5a529b547 100644 --- a/docs/architecture-refactor-roadmap.md +++ b/docs/architecture-refactor-roadmap.md @@ -73,8 +73,8 @@ G-ARCH 只有在以下条件全部满足后才可完成： | Leaf | 状态 | 依赖 | 完成定义 | |---|---|---|---| | S0-L1 可信基线恢复 | `DELIVERED` | 无 | `5df388719`：交付架构审计/路线图，修复 `ARCH-001` 两个 mypy 增量错误；远端 `0/0` | -| S0-L2.1 Host Oper/UoW 规范 | `VERIFIED` | S0-L1 | 宿主规则中未标注兼容范围的无 Session Oper 示例归零，并由文档测试锁定 | -| S0-L2.2 完整宿主 SCC policy | `PLANNED` | S0-L2.1 | 完整宿主 SCC 全部精确分类；Chain 临时债务和 TMDB containment 分离 | +| S0-L2.1 Host Oper/UoW 规范 | `DELIVERED` | S0-L1 | `3bf94ffed`：宿主无 Session Oper 规范债务归零，远端 `0/0` | +| S0-L2.2 完整宿主 SCC policy | `VERIFIED` | S0-L2.1 | 完整宿主 SCC 全部精确分类；Chain 临时债务和 TMDB containment 分离 | | S0-L2.3 Adapter 直连事实 | `PLANNED` | S0-L2.1 | 收集 Application/Chain 的原始 Adapter import，不因父包展开重复计数 | | S0-L2.4 Adapter zero-growth | `PLANNED` | S0-L2.3 | 当前直连均登记迁移 owner，新增/替换失败，删除后要求清理陈旧 policy | | S0-L2.5 Event consumer 识别 | `PLANNED` | S0-L2.1 | consumer 只识别可静态证明的 EventManager 注册，动态误报归零 | @@ -153,38 +153,42 @@ G-ARCH 只有在以下条件全部满足后才可完成： ## 4. 当前活动叶子 -### S0-L2.1 Host Oper/UoW 规范 +### S0-L2.2 完整宿主 SCC policy **Status:** `VERIFIED`（本地验收完成，等待提交、推送和远端一致性确认） **Outcome** -让宿主数据访问规范与已经建立的 Application Port、DB Adapter、显式 Session/UoW -边界一致；无 Session Oper 只作为明确标注的插件 Legacy/Compat ABI 出现。 +让完整宿主静态依赖图中的每个 SCC 都有精确、人工审查的分类；生成快照只记录事实， +不能通过刷新 baseline 自动批准新环。 **Ownership** -- `docs/rules/04-design-patterns.md` 的 Oper、SystemConfig、UserConfig 示例。 -- `docs/rules/10-data-and-persistent.md` 的对应交叉规则。 -- `tests/test_architecture_documentation.py` 的无 Session Oper 文档门禁。 +- `tests/test_architecture_dependencies.py` 的统一图算法和完整 SCC 语义门禁。 +- `tests/fixtures/architecture/dependency-policy.json` 的精确人工 policy。 +- `docs/rules/05-architecture.md` 与 `docs/architecture-overview.md` 的事实/policy 边界。 - 本路线图的叶子状态和交付记录。 **Excluded** -- 不迁移当前 startup 注入的无 Session Oper；该债务由 ARCH-103 逐领域切换。 -- 不修改运行时代码、fixture、`app/plugins/**` 或独立插件仓。 -- 不以删除 Legacy/Compat ABI 伪装宿主规范收口。 +- 不在本叶消除 Chain SCC；由 `ARCH-107`/S2-L2 删除包根环和临时 policy。 +- 不拆分 TMDB 移植包，不把普通单向包外依赖误判为 SCC 成员扩张。 +- 不修改 `app/plugins/**`、运行时代码或独立插件仓。 **Acceptance** ```bash -.venv/bin/python -m pytest tests/test_architecture_documentation.py -q -.venv/bin/python scripts/architecture/baseline.py --check-host -rg -n 'SubscribeOper\(\)|SystemConfigOper\(\)|UserConfigOper\(\)' docs/rules +.venv/bin/python -m pytest \ + tests/test_architecture_dependencies.py \ + tests/test_architecture_baseline_cli.py -q +.venv/bin/python scripts/architecture/baseline.py --check-host --diagnostics +.venv/bin/python scripts/architecture/ruff_ratchet.py +.venv/bin/python scripts/architecture/mypy_ratchet.py +.venv/bin/pylint tests/test_architecture_dependencies.py git diff --check ``` **Delivery** -- 单一提交主题：统一宿主持久化规范并建立文档门禁。 +- 单一提交主题：建立完整宿主 SCC policy 与 zero-growth 语义门禁。 - 推送 `origin/v3` 后确认提交祖先关系、远端 SHA 和 ahead/behind `0/0`。 diff --git a/docs/rules/05-architecture.md b/docs/rules/05-architecture.md index c8d2bf2b4..944a48fff 100644 --- a/docs/rules/05-architecture.md +++ b/docs/rules/05-architecture.md @@ -493,6 +493,29 @@ Durable post-commit side effects have a separate boundary: functional and emits one actionable warning per plugin and legacy module. - Delayed imports are not accepted as a way to hide dependency cycles. +### Dependency facts and semantic policy + +`tests/fixtures/architecture/dependency-baseline.json` is generated evidence: it +records the complete host module graph, edges and SCCs while excluding +`app/plugins/**`. It does not approve those facts. Human-reviewed classifications +live separately in `tests/fixtures/architecture/dependency-policy.json`, and +`scripts/architecture/baseline.py --write-host` must never create or update that +policy. + +The semantic architecture test compares every SCC in the complete host graph with +the exact member sets in policy. A new SCC, member expansion, changed member set, +or stale policy entry fails. The current policy has only two classifications: + +- `temporary_debt`: the three-module `app.chain` package-root cycle, owned by + `ARCH-107` and removed when `ChainBase` moves to `app.chain.base`. +- `contained_vendor`: the exact 29-module TMDB vendored SCC. It may have ordinary + one-way dependencies outside the package, but no outside module may join the SCC + and its member set may not grow. + +The target remains zero canonical host cycles except the precisely contained +vendor component. A temporary policy entry is an executable migration obligation, +not precedent for approving another cycle. + ## Permitted Call Directions | Direction | Status | @@ -513,7 +536,7 @@ Durable post-commit side effects have a separate boundary: | `foundation -> other app packages` | Forbidden | | `canonical implementation -> sdk / compat` | Forbidden | | `compat -> canonical implementation at module import time` | Forbidden | -| Any import that creates a module-level cycle | Forbidden | +| Any import that creates a module-level cycle | Forbidden; the complete host graph must match the exact reviewed SCC policy, and temporary debt must have a removal owner | ## Key File Locations @@ -592,4 +615,4 @@ imports, entrypoint (`api`/`agent`/`monitor`/`workflow`/`doctor`) imports of modules only through `run_module` dispatch), and downloader SDK (`qbittorrentapi`, `transmission_rpc`) imports inside `app/chain`. -*Last Updated: 2026-08-24* +*Last Updated: 2026-08-27* diff --git a/tests/fixtures/architecture/dependency-policy.json b/tests/fixtures/architecture/dependency-policy.json new file mode 100644 index 000000000..333acfa9f --- /dev/null +++ b/tests/fixtures/architecture/dependency-policy.json @@ -0,0 +1,61 @@ +{ + "schema_version": 1, + "scope": { + "dependency_kind": "static_runtime_imports", + "excluded_roots": [ + "app/plugins" + ], + "include_parent_package_initialization": true, + "root": "app" + }, + "allowed_sccs": [ + { + "classification": "temporary_debt", + "id": "chain-package-root", + "modules": [ + "app.chain", + "app.chain._messaging", + "app.chain._recognition" + ], + "reason": "ChainBase 仍由包根导出，包根又加载消息和识别实现；S2-L2 负责拆出 app.chain.base 并清零该环。", + "tracking": "ARCH-107" + }, + { + "classification": "contained_vendor", + "id": "themoviedb-vendored-package", + "modules": [ + "app.modules.themoviedb", + "app.modules.themoviedb.scraper", + "app.modules.themoviedb.tmdbapi", + "app.modules.themoviedb.tmdbv3api", + "app.modules.themoviedb.tmdbv3api.objs.account", + "app.modules.themoviedb.tmdbv3api.objs.auth", + "app.modules.themoviedb.tmdbv3api.objs.certification", + "app.modules.themoviedb.tmdbv3api.objs.change", + "app.modules.themoviedb.tmdbv3api.objs.collection", + "app.modules.themoviedb.tmdbv3api.objs.company", + "app.modules.themoviedb.tmdbv3api.objs.configuration", + "app.modules.themoviedb.tmdbv3api.objs.credit", + "app.modules.themoviedb.tmdbv3api.objs.discover", + "app.modules.themoviedb.tmdbv3api.objs.episode", + "app.modules.themoviedb.tmdbv3api.objs.find", + "app.modules.themoviedb.tmdbv3api.objs.genre", + "app.modules.themoviedb.tmdbv3api.objs.group", + "app.modules.themoviedb.tmdbv3api.objs.keyword", + "app.modules.themoviedb.tmdbv3api.objs.list", + "app.modules.themoviedb.tmdbv3api.objs.movie", + "app.modules.themoviedb.tmdbv3api.objs.network", + "app.modules.themoviedb.tmdbv3api.objs.person", + "app.modules.themoviedb.tmdbv3api.objs.provider", + "app.modules.themoviedb.tmdbv3api.objs.review", + "app.modules.themoviedb.tmdbv3api.objs.search", + "app.modules.themoviedb.tmdbv3api.objs.season", + "app.modules.themoviedb.tmdbv3api.objs.trending", + "app.modules.themoviedb.tmdbv3api.objs.tv", + "app.modules.themoviedb.tmdbv3api.tmdb" + ], + "reason": "隔离的移植代码保持精确成员 containment；普通单向包外依赖允许，任何成员扩张或包外反向入环均失败。", + "tracking": "replace-or-upgrade-vendored-package" + } + ] +} diff --git a/tests/test_architecture_dependencies.py b/tests/test_architecture_dependencies.py index 11d49bb8b..65089a07e 100644 --- a/tests/test_architecture_dependencies.py +++ b/tests/test_architecture_dependencies.py @@ -1,11 +1,23 @@ import ast +import json from functools import lru_cache from pathlib import Path -from scripts.architecture.baseline import iter_runtime_import_nodes +from scripts.architecture.baseline import ( + discover_modules as _discover_modules, +) +from scripts.architecture.baseline import ( + resolve_imports as _resolve_imports, +) +from scripts.architecture.baseline import ( + strongly_connected_components as _strongly_connected_components, +) PROJECT_ROOT = Path(__file__).parents[1] APP_ROOT = PROJECT_ROOT / "app" +DEPENDENCY_POLICY_PATH = ( + PROJECT_ROOT / "tests" / "fixtures" / "architecture" / "dependency-policy.json" +) LEGACY_ROOTS = ("app.core", "app.helper", "app.utils") LEGACY_MODULES = {"app.log"} IMPLEMENTATION_ROOTS = ( @@ -155,111 +167,6 @@ FORBIDDEN_IMPORT_PREFIXES = { } -def _discover_modules() -> dict[str, Path]: - """建立实际 Python 模块名到源码路径的映射。 - - `app/plugins/` 由插件仓自治（包含独立第三方实现与未完成文件）， - 不参与宿主架构图分析。 - """ - modules: dict[str, Path] = {} - for path in APP_ROOT.rglob("*.py"): - relative = path.relative_to(PROJECT_ROOT).with_suffix("") - parts = list(relative.parts) - if parts[0] == "app" and parts[1] == "plugins": - continue - if parts[-1] == "__init__": - parts.pop() - modules[".".join(parts)] = path - return modules - - -def _resolve_imports( - module_name: str, - path: Path, - known_modules: set[str], -) -> set[str]: - """解析一个模块的静态导入，并计入 Python 必然初始化的父包。""" - tree = ast.parse(path.read_text(encoding="utf-8-sig"), filename=str(path)) - package = module_name if path.name == "__init__.py" else module_name.rpartition(".")[0] - dependencies: set[str] = set() - for node in iter_runtime_import_nodes(tree): - candidates: list[str] = [] - if isinstance(node, ast.Import): - candidates.extend(alias.name for alias in node.names) - elif isinstance(node, ast.ImportFrom): - if node.level: - package_parts = package.split(".") - base = ".".join(package_parts[: len(package_parts) - node.level + 1]) - imported_module = ".".join( - part for part in (base, node.module or "") if part - ) - else: - imported_module = node.module or "" - if imported_module: - candidates.append(imported_module) - candidates.extend( - f"{imported_module}.{alias.name}" - for alias in node.names - if alias.name != "*" - ) - - for candidate in candidates: - parts = candidate.split(".") - dependencies.update( - parent - for index in range(2, len(parts)) - if (parent := ".".join(parts[:index])) in known_modules - ) - if candidate in known_modules: - dependencies.add(candidate) - dependencies.discard(module_name) - return dependencies - - -def _strongly_connected_components( - graph: dict[str, set[str]], -) -> list[set[str]]: - """使用 Tarjan 算法返回依赖图中的非平凡强连通分量。""" - indices: dict[str, int] = {} - low_links: dict[str, int] = {} - stack: list[str] = [] - on_stack: set[str] = set() - components: list[set[str]] = [] - - def visit(module_name: str) -> None: - """深度遍历一个模块并在根节点处收集强连通分量。""" - indices[module_name] = len(indices) - low_links[module_name] = indices[module_name] - stack.append(module_name) - on_stack.add(module_name) - for dependency in graph[module_name]: - if dependency not in indices: - visit(dependency) - low_links[module_name] = min( - low_links[module_name], low_links[dependency] - ) - elif dependency in on_stack: - low_links[module_name] = min( - low_links[module_name], indices[dependency] - ) - if low_links[module_name] != indices[module_name]: - return - component: set[str] = set() - while stack: - dependency = stack.pop() - on_stack.remove(dependency) - component.add(dependency) - if dependency == module_name: - break - if len(component) > 1: - components.append(component) - - for module_name in sorted(graph): - if module_name not in indices: - visit(module_name) - return components - - def _legacy_imports(path: Path) -> set[str]: """提取源码中的静态和常量动态旧路径导入。""" tree = ast.parse(path.read_text(encoding="utf-8-sig"), filename=str(path)) @@ -1154,6 +1061,126 @@ def _build_module_graph() -> dict[str, set[str]]: } +def _load_dependency_policy() -> dict: + """读取人工审查的依赖语义 policy；生成基线不得改写该文件。""" + return json.loads(DEPENDENCY_POLICY_PATH.read_text(encoding="utf-8")) + + +def _scc_policy_violations( + graph: dict[str, set[str]], + entries: list[dict], +) -> tuple[list[list[str]], list[list[str]]]: + """返回未审查 SCC 与已经失效但未清理的 policy SCC。""" + actual = { + tuple(component) + for component in _strongly_connected_components(graph) + } + reviewed = {tuple(entry["modules"]) for entry in entries} + return ( + [list(component) for component in sorted(actual - reviewed)], + [list(component) for component in sorted(reviewed - actual)], + ) + + +def test_complete_host_sccs_match_reviewed_policy() -> None: + """完整宿主图的每个 SCC 都必须精确匹配人工 policy，且不得保留陈旧项。""" + policy = _load_dependency_policy() + entries = policy["allowed_sccs"] + + assert policy["schema_version"] == 1 + assert policy["scope"] == { + "dependency_kind": "static_runtime_imports", + "excluded_roots": ["app/plugins"], + "include_parent_package_initialization": True, + "root": "app", + } + assert {entry["classification"] for entry in entries} == { + "contained_vendor", + "temporary_debt", + } + assert len({entry["id"] for entry in entries}) == len(entries) + assert all(entry["modules"] == sorted(set(entry["modules"])) for entry in entries) + assert all(entry["reason"] and entry["tracking"] for entry in entries) + all_members = [member for entry in entries for member in entry["modules"]] + assert len(all_members) == len(set(all_members)) + + temporary = next( + entry for entry in entries if entry["classification"] == "temporary_debt" + ) + assert temporary["id"] == "chain-package-root" + assert temporary["tracking"] == "ARCH-107" + assert temporary["modules"] == [ + "app.chain", + "app.chain._messaging", + "app.chain._recognition", + ] + + vendor = next( + entry for entry in entries if entry["classification"] == "contained_vendor" + ) + assert vendor["id"] == "themoviedb-vendored-package" + assert vendor["tracking"] == "replace-or-upgrade-vendored-package" + assert len(vendor["modules"]) == 29 + assert all( + member.startswith("app.modules.themoviedb") + for member in vendor["modules"] + ) + + unreviewed, stale = _scc_policy_violations(_build_module_graph(), entries) + assert unreviewed == [] + assert stale == [] + + +def test_scc_policy_rejects_unreviewed_cycles_in_every_host_root() -> None: + """API、Chain、Module 与 Startup 内的新环都必须落入未审查集合。""" + for root in ("app.api", "app.chain", "app.modules", "app.startup"): + first = f"{root}.first" + second = f"{root}.second" + unreviewed, stale = _scc_policy_violations( + {first: {second}, second: {first}}, + [], + ) + assert unreviewed == [[first, second]] + assert stale == [] + + +def test_scc_policy_rejects_changed_or_stale_membership() -> None: + """已知 SCC 缩小、扩大或消失后必须同步审查 policy，不能静默放行。""" + entries = [{"modules": ["app.chain.first", "app.chain.second", "app.chain.third"]}] + unreviewed, stale = _scc_policy_violations( + { + "app.chain.first": {"app.chain.second"}, + "app.chain.second": {"app.chain.first"}, + "app.chain.third": set(), + }, + entries, + ) + assert unreviewed == [["app.chain.first", "app.chain.second"]] + assert stale == [["app.chain.first", "app.chain.second", "app.chain.third"]] + + +def test_contained_vendor_scc_may_have_one_way_outbound_dependency() -> None: + """vendor SCC 的普通单向向下依赖不应把包外模块误纳入 containment。""" + first = "app.modules.themoviedb.first" + second = "app.modules.themoviedb.second" + domain = "app.domain.media" + entries = [{"modules": [first, second]}] + unreviewed, stale = _scc_policy_violations( + {first: {second, domain}, second: {first}, domain: set()}, + entries, + ) + assert unreviewed == [] + assert stale == [] + + +def test_host_dependency_graph_excludes_plugin_copies() -> None: + """`app/plugins/**` 是运行时副本，不能进入宿主模块或 SCC 图。""" + assert all( + not module.startswith("app.plugins") + for module in _discover_modules() + ) + + def test_chain_does_not_import_agent_implementation(): """编排层不得反向依赖 Agent 实现，跨域编排经 application 门面。""" violations: dict[str, set[str]] = {}