GithubBackup/MoviePilot
1
0
Fork 0
mirror of https://github.com/jxxghp/MoviePilot.git synced 2026-08-29 03:56:43 +08:00
Code Issues Packages Projects Releases Wiki Activity

test(architecture): govern complete host cycles

Browse Source
This commit is contained in:
jxxghp
2026-08-27 05:50:39 +08:00
parent 3bf94ffeda
commit a884ab5c2b
6 changed files with 244 additions and 126 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture-optimization-checklist.md
+2 -2
View File
@@ -156,7 +156,7 @@ MoviePilot V3 已经形成较清晰的模块化单体：`foundation`、`domain`
与生产路径显式 UoW 规则冲突；该项已由 S0-L2.1 修复并增加文档门禁。
- `docs/rules/05-architecture.md:507` 禁止 `application -> concrete adapter`，但同一文档
`177-183` 又要求 RSS 消费 network adapter；当前测试没有形式化这类例外。
- 规则 `docs/rules/05-architecture.md:516` 禁止任意模块级环，实际 SCC 测试只覆盖特定根。
- 审计时完整 SCC 只进入生成快照，语义测试只覆盖特定根；S0-L2.2 已增加完整宿主 SCC policy 门禁。
- 架构总览此前仍记录 811 模块、6,572 条边和 1 个 SCC，已经落后于当前基线。
- Event consumer 扫描把任意名为 `.register()` 的调用都当成事件注册，存在明确误报。
@@ -165,7 +165,7 @@ MoviePilot V3 已经形成较清晰的模块化单体：`foundation`、`domain`
- [ ] 指定一个机器可读事实源，文档指标由 fixture 生成或只保留不易漂移的语义描述。
- [x] 修正 Oper 示例，分别展示宿主显式 UoW 与插件兼容 Facade，并以文档测试禁止回退。
- [ ] 对 Adapter 规则作出明确决策：允许哪些通用技术机制，哪些命名外部/持久化能力必须注入。
- [ ] 让 SCC 规则、精确 allowlist 和文档声明一致。
- [x] 让 SCC 规则、精确 policy 和文档声明一致；Chain 临时债务与 TMDB vendor containment 分开治理。
- [ ] Event 扫描只识别 EventManager 实例/别名和事件装饰器。
- [ ] CI 分开报告“快照一致”与“语义规则通过”，禁止把前者表述为架构完全正确。
docs/architecture-overview.md
+4 -1
View File
@@ -668,6 +668,9 @@ flowchart LR
物理遗留源码、禁止的上向依赖、SDK/compat 反向引用、包含迁移模块的强连通分量、
模块间/模块到 Chain 的 import、入口层对 `app.modules` 内部的 import、
Chain 直接 import 模块内部（必须走 `run_module` 分发）、`app/chain` 内的下载器 SDK 依赖。
- `tests/fixtures/architecture/dependency-baseline.json` 记录生成事实；人工审查的 SCC 分类单独存入
`dependency-policy.json`。完整宿主 SCC 必须精确匹配 policy，新增、扩大、变形和陈旧 policy 都失败；
`--write-host` 不会替代人工决策。
- 任何所有权迁移必须同步更新：canonical 导入、`app/runtime/compat/manifest.py`、
SDK 导出（若公开）、`docs/rules/05-architecture.md` 与上述架构测试。
- 延迟导入不被接受为隐藏循环依赖的手段。
@@ -681,7 +684,7 @@ flowchart LR
|---|---:|
| Python 模块 | 835 |
| 内部导入边 | 6,810 |
| 非平凡 SCC | 2（Chain 包根环与隔离的 TMDB 移植包环） |
| 非平凡 SCC | 2（`ARCH-107` 临时 Chain 包根环；精确 containment 的 TMDB 移植包环） |
| Module Contract V2 spec | 215（其中 214 个进入 `run_module` 观察面） |
| Event Contract | 53 |
| Model/Oper 自动事务与自建 Session | 0 |
docs/architecture-refactor-roadmap.md
+19 -15
View File
@@ -73,8 +73,8 @@ G-ARCH 只有在以下条件全部满足后才可完成：
| Leaf | 状态 | 依赖 | 完成定义 |
|---|---|---|---|
| S0-L1 可信基线恢复 | `DELIVERED` | 无 | `5df388719`：交付架构审计/路线图，修复 `ARCH-001` 两个 mypy 增量错误；远端 `0/0` |
| S0-L2.1 Host Oper/UoW 规范 | `VERIFIED` | S0-L1 | 宿主规则中未标注兼容范围的无 Session Oper 示例归零，并由文档测试锁定 |
| S0-L2.2 完整宿主 SCC policy | `PLANNED` | S0-L2.1 | 完整宿主 SCC 全部精确分类；Chain 临时债务和 TMDB containment 分离 |
| S0-L2.1 Host Oper/UoW 规范 | `DELIVERED` | S0-L1 | `3bf94ffed`：宿主无 Session Oper 规范债务归零，远端 `0/0` |
| S0-L2.2 完整宿主 SCC policy | `VERIFIED` | S0-L2.1 | 完整宿主 SCC 全部精确分类；Chain 临时债务和 TMDB containment 分离 |
| S0-L2.3 Adapter 直连事实 | `PLANNED` | S0-L2.1 | 收集 Application/Chain 的原始 Adapter import，不因父包展开重复计数 |
| S0-L2.4 Adapter zero-growth | `PLANNED` | S0-L2.3 | 当前直连均登记迁移 owner，新增/替换失败，删除后要求清理陈旧 policy |
| S0-L2.5 Event consumer 识别 | `PLANNED` | S0-L2.1 | consumer 只识别可静态证明的 EventManager 注册，动态误报归零 |
@@ -153,38 +153,42 @@ G-ARCH 只有在以下条件全部满足后才可完成：
## 4. 当前活动叶子
### S0-L2.1 Host Oper/UoW 规范
### S0-L2.2 完整宿主 SCC policy
**Status:** `VERIFIED`（本地验收完成，等待提交、推送和远端一致性确认）
**Outcome**
让宿主数据访问规范与已经建立的 Application Port、DB Adapter、显式 Session/UoW
边界一致；无 Session Oper 只作为明确标注的插件 Legacy/Compat ABI 出现。
让完整宿主静态依赖图中的每个 SCC 都有精确、人工审查的分类；生成快照只记录事实，
不能通过刷新 baseline 自动批准新环。
**Ownership**
- `docs/rules/04-design-patterns.md` 的 Oper、SystemConfig、UserConfig 示例。
- `docs/rules/10-data-and-persistent.md` 的对应交叉规则。
- `tests/test_architecture_documentation.py` 的无 Session Oper 文档门禁。
- `tests/test_architecture_dependencies.py` 的统一图算法和完整 SCC 语义门禁。
- `tests/fixtures/architecture/dependency-policy.json` 的精确人工 policy。
- `docs/rules/05-architecture.md` 与 `docs/architecture-overview.md` 的事实/policy 边界。
- 本路线图的叶子状态和交付记录。
**Excluded**
- 不迁移当前 startup 注入的无 Session Oper；该债务由 ARCH-103 逐领域切换。
- 不修改运行时代码、fixture、`app/plugins/**` 或独立插件仓。
- 不以删除 Legacy/Compat ABI 伪装宿主规范收口。
- 不在本叶消除 Chain SCC；由 `ARCH-107`/S2-L2 删除包根环和临时 policy。
- 不拆分 TMDB 移植包，不把普通单向包外依赖误判为 SCC 成员扩张。
- 不修改 `app/plugins/**`、运行时代码或独立插件仓。
**Acceptance**
```bash
.venv/bin/python -m pytest tests/test_architecture_documentation.py -q
.venv/bin/python scripts/architecture/baseline.py --check-host
rg -n 'SubscribeOper\(\)|SystemConfigOper\(\)|UserConfigOper\(\)' docs/rules
.venv/bin/python -m pytest \
tests/test_architecture_dependencies.py \
tests/test_architecture_baseline_cli.py -q
.venv/bin/python scripts/architecture/baseline.py --check-host --diagnostics
.venv/bin/python scripts/architecture/ruff_ratchet.py
.venv/bin/python scripts/architecture/mypy_ratchet.py
.venv/bin/pylint tests/test_architecture_dependencies.py
git diff --check
```
**Delivery**
- 单一提交主题：统一宿主持久化规范并建立文档门禁。
- 单一提交主题：建立完整宿主 SCC policy 与 zero-growth 语义门禁。
- 推送 `origin/v3` 后确认提交祖先关系、远端 SHA 和 ahead/behind `0/0`。
docs/rules/05-architecture.md
+25 -2
View File
@@ -493,6 +493,29 @@ Durable post-commit side effects have a separate boundary:
functional and emits one actionable warning per plugin and legacy module.
- Delayed imports are not accepted as a way to hide dependency cycles.
### Dependency facts and semantic policy
`tests/fixtures/architecture/dependency-baseline.json` is generated evidence: it
records the complete host module graph, edges and SCCs while excluding
`app/plugins/**`. It does not approve those facts. Human-reviewed classifications
live separately in `tests/fixtures/architecture/dependency-policy.json`, and
`scripts/architecture/baseline.py --write-host` must never create or update that
policy.
The semantic architecture test compares every SCC in the complete host graph with
the exact member sets in policy. A new SCC, member expansion, changed member set,
or stale policy entry fails. The current policy has only two classifications:
- `temporary_debt`: the three-module `app.chain` package-root cycle, owned by
`ARCH-107` and removed when `ChainBase` moves to `app.chain.base`.
- `contained_vendor`: the exact 29-module TMDB vendored SCC. It may have ordinary
one-way dependencies outside the package, but no outside module may join the SCC
and its member set may not grow.
The target remains zero canonical host cycles except the precisely contained
vendor component. A temporary policy entry is an executable migration obligation,
not precedent for approving another cycle.
## Permitted Call Directions
| Direction | Status |
@@ -513,7 +536,7 @@ Durable post-commit side effects have a separate boundary:
| `foundation -> other app packages` | Forbidden |
| `canonical implementation -> sdk / compat` | Forbidden |
| `compat -> canonical implementation at module import time` | Forbidden |
| Any import that creates a module-level cycle | Forbidden |
| Any import that creates a module-level cycle | Forbidden; the complete host graph must match the exact reviewed SCC policy, and temporary debt must have a removal owner |
## Key File Locations
@@ -592,4 +615,4 @@ imports, entrypoint (`api`/`agent`/`monitor`/`workflow`/`doctor`) imports of
modules only through `run_module` dispatch), and downloader SDK
(`qbittorrentapi`, `transmission_rpc`) imports inside `app/chain`.
*Last Updated: 2026-08-24*
*Last Updated: 2026-08-27*
tests/fixtures/architecture/dependency-policy.json
Vendored
+61
View File
@@ -0,0 +1,61 @@
{
"schema_version": 1,
"scope": {
"dependency_kind": "static_runtime_imports",
"excluded_roots": [
"app/plugins"
],
"include_parent_package_initialization": true,
"root": "app"
},
"allowed_sccs": [
{
"classification": "temporary_debt",
"id": "chain-package-root",
"modules": [
"app.chain",
"app.chain._messaging",
"app.chain._recognition"
],
"reason": "ChainBase 仍由包根导出，包根又加载消息和识别实现；S2-L2 负责拆出 app.chain.base 并清零该环。",
"tracking": "ARCH-107"
},
{
"classification": "contained_vendor",
"id": "themoviedb-vendored-package",
"modules": [
"app.modules.themoviedb",
"app.modules.themoviedb.scraper",
"app.modules.themoviedb.tmdbapi",
"app.modules.themoviedb.tmdbv3api",
"app.modules.themoviedb.tmdbv3api.objs.account",
"app.modules.themoviedb.tmdbv3api.objs.auth",
"app.modules.themoviedb.tmdbv3api.objs.certification",
"app.modules.themoviedb.tmdbv3api.objs.change",
"app.modules.themoviedb.tmdbv3api.objs.collection",
"app.modules.themoviedb.tmdbv3api.objs.company",
"app.modules.themoviedb.tmdbv3api.objs.configuration",
"app.modules.themoviedb.tmdbv3api.objs.credit",
"app.modules.themoviedb.tmdbv3api.objs.discover",
"app.modules.themoviedb.tmdbv3api.objs.episode",
"app.modules.themoviedb.tmdbv3api.objs.find",
"app.modules.themoviedb.tmdbv3api.objs.genre",
"app.modules.themoviedb.tmdbv3api.objs.group",
"app.modules.themoviedb.tmdbv3api.objs.keyword",
"app.modules.themoviedb.tmdbv3api.objs.list",
"app.modules.themoviedb.tmdbv3api.objs.movie",
"app.modules.themoviedb.tmdbv3api.objs.network",
"app.modules.themoviedb.tmdbv3api.objs.person",
"app.modules.themoviedb.tmdbv3api.objs.provider",
"app.modules.themoviedb.tmdbv3api.objs.review",
"app.modules.themoviedb.tmdbv3api.objs.search",
"app.modules.themoviedb.tmdbv3api.objs.season",
"app.modules.themoviedb.tmdbv3api.objs.trending",
"app.modules.themoviedb.tmdbv3api.objs.tv",
"app.modules.themoviedb.tmdbv3api.tmdb"
],
"reason": "隔离的移植代码保持精确成员 containment；普通单向包外依赖允许，任何成员扩张或包外反向入环均失败。",
"tracking": "replace-or-upgrade-vendored-package"
}
]
}
tests/test_architecture_dependencies.py
+133 -106
View File
@@ -1,11 +1,23 @@
import ast
import json
from functools import lru_cache
from pathlib import Path
from scripts.architecture.baseline import iter_runtime_import_nodes
from scripts.architecture.baseline import (
discover_modules as _discover_modules,
)
from scripts.architecture.baseline import (
resolve_imports as _resolve_imports,
)
from scripts.architecture.baseline import (
strongly_connected_components as _strongly_connected_components,
)
PROJECT_ROOT = Path(__file__).parents[1]
APP_ROOT = PROJECT_ROOT / "app"
DEPENDENCY_POLICY_PATH = (
PROJECT_ROOT / "tests" / "fixtures" / "architecture" / "dependency-policy.json"
)
LEGACY_ROOTS = ("app.core", "app.helper", "app.utils")
LEGACY_MODULES = {"app.log"}
IMPLEMENTATION_ROOTS = (
@@ -155,111 +167,6 @@ FORBIDDEN_IMPORT_PREFIXES = {
}
def _discover_modules() -> dict[str, Path]:
"""建立实际 Python 模块名到源码路径的映射。
`app/plugins/` 由插件仓自治（包含独立第三方实现与未完成文件），
不参与宿主架构图分析。
"""
modules: dict[str, Path] = {}
for path in APP_ROOT.rglob("*.py"):
relative = path.relative_to(PROJECT_ROOT).with_suffix("")
parts = list(relative.parts)
if parts[0] == "app" and parts[1] == "plugins":
continue
if parts[-1] == "__init__":
parts.pop()
modules[".".join(parts)] = path
return modules
def _resolve_imports(
module_name: str,
path: Path,
known_modules: set[str],
) -> set[str]:
"""解析一个模块的静态导入，并计入 Python 必然初始化的父包。"""
tree = ast.parse(path.read_text(encoding="utf-8-sig"), filename=str(path))
package = module_name if path.name == "__init__.py" else module_name.rpartition(".")[0]
dependencies: set[str] = set()
for node in iter_runtime_import_nodes(tree):
candidates: list[str] = []
if isinstance(node, ast.Import):
candidates.extend(alias.name for alias in node.names)
elif isinstance(node, ast.ImportFrom):
if node.level:
package_parts = package.split(".")
base = ".".join(package_parts[: len(package_parts) - node.level + 1])
imported_module = ".".join(
part for part in (base, node.module or "") if part
)
else:
imported_module = node.module or ""
if imported_module:
candidates.append(imported_module)
candidates.extend(
f"{imported_module}.{alias.name}"
for alias in node.names
if alias.name != "*"
)
for candidate in candidates:
parts = candidate.split(".")
dependencies.update(
parent
for index in range(2, len(parts))
if (parent := ".".join(parts[:index])) in known_modules
)
if candidate in known_modules:
dependencies.add(candidate)
dependencies.discard(module_name)
return dependencies
def _strongly_connected_components(
graph: dict[str, set[str]],
) -> list[set[str]]:
"""使用 Tarjan 算法返回依赖图中的非平凡强连通分量。"""
indices: dict[str, int] = {}
low_links: dict[str, int] = {}
stack: list[str] = []
on_stack: set[str] = set()
components: list[set[str]] = []
def visit(module_name: str) -> None:
"""深度遍历一个模块并在根节点处收集强连通分量。"""
indices[module_name] = len(indices)
low_links[module_name] = indices[module_name]
stack.append(module_name)
on_stack.add(module_name)
for dependency in graph[module_name]:
if dependency not in indices:
visit(dependency)
low_links[module_name] = min(
low_links[module_name], low_links[dependency]
)
elif dependency in on_stack:
low_links[module_name] = min(
low_links[module_name], indices[dependency]
)
if low_links[module_name] != indices[module_name]:
return
component: set[str] = set()
while stack:
dependency = stack.pop()
on_stack.remove(dependency)
component.add(dependency)
if dependency == module_name:
break
if len(component) > 1:
components.append(component)
for module_name in sorted(graph):
if module_name not in indices:
visit(module_name)
return components
def _legacy_imports(path: Path) -> set[str]:
"""提取源码中的静态和常量动态旧路径导入。"""
tree = ast.parse(path.read_text(encoding="utf-8-sig"), filename=str(path))
@@ -1154,6 +1061,126 @@ def _build_module_graph() -> dict[str, set[str]]:
}
def _load_dependency_policy() -> dict:
"""读取人工审查的依赖语义 policy；生成基线不得改写该文件。"""
return json.loads(DEPENDENCY_POLICY_PATH.read_text(encoding="utf-8"))
def _scc_policy_violations(
graph: dict[str, set[str]],
entries: list[dict],
) -> tuple[list[list[str]], list[list[str]]]:
"""返回未审查 SCC 与已经失效但未清理的 policy SCC。"""
actual = {
tuple(component)
for component in _strongly_connected_components(graph)
}
reviewed = {tuple(entry["modules"]) for entry in entries}
return (
[list(component) for component in sorted(actual - reviewed)],
[list(component) for component in sorted(reviewed - actual)],
)
def test_complete_host_sccs_match_reviewed_policy() -> None:
"""完整宿主图的每个 SCC 都必须精确匹配人工 policy，且不得保留陈旧项。"""
policy = _load_dependency_policy()
entries = policy["allowed_sccs"]
assert policy["schema_version"] == 1
assert policy["scope"] == {
"dependency_kind": "static_runtime_imports",
"excluded_roots": ["app/plugins"],
"include_parent_package_initialization": True,
"root": "app",
}
assert {entry["classification"] for entry in entries} == {
"contained_vendor",
"temporary_debt",
}
assert len({entry["id"] for entry in entries}) == len(entries)
assert all(entry["modules"] == sorted(set(entry["modules"])) for entry in entries)
assert all(entry["reason"] and entry["tracking"] for entry in entries)
all_members = [member for entry in entries for member in entry["modules"]]
assert len(all_members) == len(set(all_members))
temporary = next(
entry for entry in entries if entry["classification"] == "temporary_debt"
)
assert temporary["id"] == "chain-package-root"
assert temporary["tracking"] == "ARCH-107"
assert temporary["modules"] == [
"app.chain",
"app.chain._messaging",
"app.chain._recognition",
]
vendor = next(
entry for entry in entries if entry["classification"] == "contained_vendor"
)
assert vendor["id"] == "themoviedb-vendored-package"
assert vendor["tracking"] == "replace-or-upgrade-vendored-package"
assert len(vendor["modules"]) == 29
assert all(
member.startswith("app.modules.themoviedb")
for member in vendor["modules"]
)
unreviewed, stale = _scc_policy_violations(_build_module_graph(), entries)
assert unreviewed == []
assert stale == []
def test_scc_policy_rejects_unreviewed_cycles_in_every_host_root() -> None:
"""API、Chain、Module 与 Startup 内的新环都必须落入未审查集合。"""
for root in ("app.api", "app.chain", "app.modules", "app.startup"):
first = f"{root}.first"
second = f"{root}.second"
unreviewed, stale = _scc_policy_violations(
{first: {second}, second: {first}},
[],
)
assert unreviewed == [[first, second]]
assert stale == []
def test_scc_policy_rejects_changed_or_stale_membership() -> None:
"""已知 SCC 缩小、扩大或消失后必须同步审查 policy，不能静默放行。"""
entries = [{"modules": ["app.chain.first", "app.chain.second", "app.chain.third"]}]
unreviewed, stale = _scc_policy_violations(
{
"app.chain.first": {"app.chain.second"},
"app.chain.second": {"app.chain.first"},
"app.chain.third": set(),
},
entries,
)
assert unreviewed == [["app.chain.first", "app.chain.second"]]
assert stale == [["app.chain.first", "app.chain.second", "app.chain.third"]]
def test_contained_vendor_scc_may_have_one_way_outbound_dependency() -> None:
"""vendor SCC 的普通单向向下依赖不应把包外模块误纳入 containment。"""
first = "app.modules.themoviedb.first"
second = "app.modules.themoviedb.second"
domain = "app.domain.media"
entries = [{"modules": [first, second]}]
unreviewed, stale = _scc_policy_violations(
{first: {second, domain}, second: {first}, domain: set()},
entries,
)
assert unreviewed == []
assert stale == []
def test_host_dependency_graph_excludes_plugin_copies() -> None:
"""`app/plugins/**` 是运行时副本，不能进入宿主模块或 SCC 图。"""
assert all(
not module.startswith("app.plugins")
for module in _discover_modules()
)
def test_chain_does_not_import_agent_implementation():
"""编排层不得反向依赖 Agent 实现，跨域编排经 application 门面。"""
violations: dict[str, set[str]] = {}