mirror of
https://github.com/jxxghp/MoviePilot.git
synced 2026-09-01 05:27:02 +08:00
ci: enforce backend architecture contracts
This commit is contained in:
@@ -0,0 +1,69 @@
|
||||
name: Official Plugin Architecture Observation
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
schedule:
|
||||
- cron: '17 3 * * 1'
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: official-plugin-architecture-observation
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
observe:
|
||||
runs-on: ubuntu-latest
|
||||
name: Compare latest official plugin contracts
|
||||
timeout-minutes: 15
|
||||
|
||||
steps:
|
||||
- name: Checkout MoviePilot
|
||||
uses: actions/checkout@v5
|
||||
with:
|
||||
path: MoviePilot
|
||||
|
||||
- name: Checkout MoviePilot-Plugins
|
||||
uses: actions/checkout@v5
|
||||
with:
|
||||
repository: jxxghp/MoviePilot-Plugins
|
||||
ref: main
|
||||
path: MoviePilot-Plugins
|
||||
|
||||
- name: Set up uv
|
||||
uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
|
||||
with:
|
||||
version: '0.12.5'
|
||||
python-version: '3.12'
|
||||
enable-cache: true
|
||||
cache-dependency-glob: |
|
||||
MoviePilot/pyproject.toml
|
||||
MoviePilot/uv.lock
|
||||
|
||||
- name: Install dependencies
|
||||
working-directory: MoviePilot
|
||||
run: uv sync --locked
|
||||
|
||||
- name: Compare plugin contracts without updating fixtures
|
||||
id: compare
|
||||
continue-on-error: true
|
||||
working-directory: MoviePilot
|
||||
run: |
|
||||
uv run --locked --no-sync python scripts/architecture/baseline.py \
|
||||
--check-plugins \
|
||||
--plugin-repo ../MoviePilot-Plugins \
|
||||
--report official-plugin-architecture-report.json
|
||||
|
||||
- name: Upload semantic comparison report
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v6
|
||||
with:
|
||||
name: official-plugin-architecture-report
|
||||
path: MoviePilot/official-plugin-architecture-report.json
|
||||
if-no-files-found: warn
|
||||
retention-days: 14
|
||||
|
||||
- name: Fail when plugin contracts changed
|
||||
if: steps.compare.outcome == 'failure'
|
||||
run: exit 1
|
||||
@@ -1,12 +1,21 @@
|
||||
name: Pylint Code Quality Check
|
||||
|
||||
on:
|
||||
# 允许手动触发
|
||||
pull_request:
|
||||
branches:
|
||||
- v3
|
||||
push:
|
||||
branches:
|
||||
- v3
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: pylint-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
pylint:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
@@ -19,6 +19,37 @@ concurrency:
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
architecture:
|
||||
runs-on: ubuntu-latest
|
||||
name: Architecture Contract Gate
|
||||
timeout-minutes: 10
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v5
|
||||
|
||||
- name: Set up uv
|
||||
uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
|
||||
with:
|
||||
version: '0.12.5'
|
||||
python-version: '3.12'
|
||||
enable-cache: true
|
||||
cache-dependency-glob: |
|
||||
pyproject.toml
|
||||
uv.lock
|
||||
|
||||
- name: Install dependencies
|
||||
run: uv sync --locked
|
||||
|
||||
- name: Check host architecture contracts
|
||||
run: |
|
||||
uv run --locked --no-sync pytest \
|
||||
tests/test_architecture_dependencies.py \
|
||||
tests/test_architecture_contract_baseline.py \
|
||||
tests/test_architecture_baseline_cli.py -q
|
||||
uv run --locked --no-sync python \
|
||||
scripts/architecture/baseline.py --check-host
|
||||
|
||||
pytest:
|
||||
runs-on: ubuntu-latest
|
||||
name: Unit Tests (${{ matrix.shard }})
|
||||
|
||||
@@ -181,6 +181,20 @@ Safety 直接识别项目清单和锁文件,不需要生成或维护 requireme
|
||||
参数启动对应分片。需要单进程调试时使用 `python tests/run.py --serial`。覆盖率报告
|
||||
按需通过 `Unit Tests` workflow 的手动触发串行生成,不阻塞常规 PR / push 门禁。
|
||||
|
||||
4. **运行架构与静态门禁**:主仓架构检查不依赖独立插件仓;官方插件兼容观察单独运行,
|
||||
任何检查命令都不会写入 fixture。
|
||||
|
||||
```bash
|
||||
uv run --locked --no-sync python scripts/architecture/baseline.py --check-host
|
||||
uv run --locked --no-sync python scripts/architecture/baseline.py \
|
||||
--check-plugins --plugin-repo ../MoviePilot-Plugins \
|
||||
--report official-plugin-architecture-report.json
|
||||
uv run --locked --no-sync pylint app/
|
||||
```
|
||||
|
||||
GitHub Actions 会在 `v3` 的 PR/push 中独立执行宿主架构与 Pylint 门禁;最新官方插件仓
|
||||
通过每周或手工观察工作流检查,只上传语义差异报告,不会自动更新已提交基线。
|
||||
|
||||
### 7. 参考资源
|
||||
|
||||
- [uv 官方文档](https://docs.astral.sh/uv/)
|
||||
|
||||
@@ -6,6 +6,7 @@
|
||||
> 审计范围:宿主后端;排除 `app/plugins/**` 运行时插件副本
|
||||
> 规范优先级:`AGENTS.md` 与 `docs/rules/` 高于本文
|
||||
> 相关文档:`docs/architecture-overview.md`、`docs/refactor/backend-architecture-governance.md`、`docs/refactor/backend-module-refactor-compatibility.md`
|
||||
> 实施进度:阶段 0(ARCH-201~203)已完成,后续阶段按任务 ID 独立提交和回滚
|
||||
|
||||
## 1. 结论先行
|
||||
|
||||
@@ -106,6 +107,10 @@ MoviePilot V3 当前不是“目录混乱、必须推倒重来”的状态。第
|
||||
|
||||
**本次审计没有更新任何基线文件。**上述意外写入已恢复,最终工作树只包含本文和文档索引改动。
|
||||
|
||||
阶段 0 实施后,宿主与插件基线已使用独立 check/write 入口;运行契约行号只进入按需诊断,
|
||||
插件 commit、源码摘要和文件数只作为 provenance。当前宿主和官方插件语义检查均通过,CI 会在
|
||||
主仓 PR/push 执行宿主硬门禁,并在定时/手工工作流中上传最新插件仓的语义差异报告。
|
||||
|
||||
## 3. 优秀 Python 后端实践对标
|
||||
|
||||
本节只采用与 MoviePilot 当前形态相近、能转化为具体约束的实践。参考不是为了照抄目录,而是为了验证职责、生命周期和失败语义。
|
||||
|
||||
+3
-1
@@ -145,7 +145,9 @@ def test_recognize_prefers_explicit_identity(sample_meta, monkeypatch):
|
||||
|
||||
## CI 与 PR
|
||||
|
||||
- **门禁**:`.github/workflows/test.yml` 在指向 `v3` 的 `pull_request` / `push` 及手动触发时,从 `uv.lock` 同步环境,通过 `tests/run.py --shard N/TOTAL` 把全量测试文件稳定分到 4 个独立 pytest job。每个分片都有独立进程和临时 `CONFIG_DIR`,不共用 SQLite 或进程级状态。
|
||||
- **门禁**:`.github/workflows/test.yml` 在指向 `v3` 的 `pull_request` / `push` 及手动触发时,从 `uv.lock` 同步环境。独立 `architecture` job 先运行宿主依赖、运行契约和基线 CLI 快速门禁;全量测试再通过 `tests/run.py --shard N/TOTAL` 稳定分到 4 个 pytest job。每个分片都有独立进程和临时 `CONFIG_DIR`,不共用 SQLite 或进程级状态。
|
||||
- **跨仓观察**:`.github/workflows/architecture-observe.yml` 每周或手工检出官方插件仓最新 `main`,使用 `--check-plugins` 比较公开导入、Hook 和动态 API 契约。它只上传 `official-plugin-architecture-report.json`,不会自动刷新 fixture;语义变化必须人工审查后显式执行 `--write-plugins`。
|
||||
- **静态检查**:`.github/workflows/pylint.yml` 对指向 `v3` 的 PR、推送和手工触发运行现有严重错误门禁,不再仅依赖维护者手工执行。
|
||||
- **PR**:产品代码、测试基础设施、依赖或运行行为发生变化时,运行 `uv run --locked --no-sync python tests/run.py`,默认以 4 个独立 pytest 进程完成全量;需要断点、输出顺序或测试污染诊断时使用 `--serial`。确认本次改动涉及的路径通过且 socket 探针零真实出站。若存在无关失败,必须在当前 `upstream/v3` 基线上独立复现并在 PR 中如实说明;不得静默扩大当前 PR 去修复基线问题。纯文档变更按实际内容执行文本、结构和 diff 检查,CI 仍会运行全量门禁。
|
||||
- 覆盖率不参与常规 PR / push 的合并门禁;需要覆盖率制品时手动触发 `Unit Tests` workflow,独立的 `Coverage Report` job 会通过 `tests/run.py --serial` 跑串行全量并上传 JSON / XML 报告。
|
||||
- 复现 CI 使用 `uv sync --locked`;主程序运行依赖位于 `[project].dependencies`,pytest 与覆盖率工具位于默认 `dev` 依赖组。
|
||||
|
||||
@@ -862,6 +862,72 @@ def semantic_baseline(path: Path, value: dict[str, Any]) -> dict[str, Any]:
|
||||
return value
|
||||
|
||||
|
||||
def _compare_semantic_values(
|
||||
expected: Any,
|
||||
actual: Any,
|
||||
path: str,
|
||||
report: dict[str, list[dict[str, Any]]],
|
||||
) -> None:
|
||||
"""递归比较语义 JSON,把增删改记录为可审查条目。"""
|
||||
if isinstance(expected, dict) and isinstance(actual, dict):
|
||||
expected_keys = set(expected)
|
||||
actual_keys = set(actual)
|
||||
for key in sorted(expected_keys - actual_keys):
|
||||
report["removed"].append(
|
||||
{"path": f"{path}.{key}", "value": expected[key]}
|
||||
)
|
||||
for key in sorted(actual_keys - expected_keys):
|
||||
report["added"].append(
|
||||
{"path": f"{path}.{key}", "value": actual[key]}
|
||||
)
|
||||
for key in sorted(expected_keys & actual_keys):
|
||||
_compare_semantic_values(
|
||||
expected[key],
|
||||
actual[key],
|
||||
f"{path}.{key}",
|
||||
report,
|
||||
)
|
||||
return
|
||||
if isinstance(expected, list) and isinstance(actual, list):
|
||||
expected_items = {
|
||||
json.dumps(item, ensure_ascii=False, sort_keys=True): item
|
||||
for item in expected
|
||||
}
|
||||
actual_items = {
|
||||
json.dumps(item, ensure_ascii=False, sort_keys=True): item
|
||||
for item in actual
|
||||
}
|
||||
for key in sorted(expected_items.keys() - actual_items.keys()):
|
||||
report["removed"].append({"path": path, "value": expected_items[key]})
|
||||
for key in sorted(actual_items.keys() - expected_items.keys()):
|
||||
report["added"].append({"path": path, "value": actual_items[key]})
|
||||
return
|
||||
if expected != actual:
|
||||
report["changed"].append(
|
||||
{"path": path, "expected": expected, "actual": actual}
|
||||
)
|
||||
|
||||
|
||||
def build_comparison_report(path: Path, actual: dict[str, Any]) -> dict[str, Any]:
|
||||
"""生成包含语义增删改和 provenance 的机器可读审查报告。"""
|
||||
expected = json.loads(path.read_text(encoding="utf-8"))
|
||||
expected_semantic = semantic_baseline(path, expected)
|
||||
actual_semantic = semantic_baseline(path, actual)
|
||||
differences: dict[str, list[dict[str, Any]]] = {
|
||||
"added": [],
|
||||
"removed": [],
|
||||
"changed": [],
|
||||
}
|
||||
_compare_semantic_values(expected_semantic, actual_semantic, "$", differences)
|
||||
return {
|
||||
"baseline": str(_display_path(path)),
|
||||
"semantic_match": expected_semantic == actual_semantic,
|
||||
"expected_provenance": expected.get("provenance"),
|
||||
"actual_provenance": actual.get("provenance"),
|
||||
**differences,
|
||||
}
|
||||
|
||||
|
||||
def check_json(
|
||||
path: Path,
|
||||
actual: dict[str, Any],
|
||||
@@ -912,6 +978,11 @@ def parse_args(argv: Optional[list[str]] = None) -> argparse.Namespace:
|
||||
action="store_true",
|
||||
help="随宿主检查打印当前运行契约源码位置",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--report",
|
||||
type=Path,
|
||||
help="将检查结果写为独立 JSON 报告,不修改任何 fixture",
|
||||
)
|
||||
args = parser.parse_args(argv)
|
||||
if args.check or args.write:
|
||||
if not args.scope:
|
||||
@@ -932,6 +1003,8 @@ def parse_args(argv: Optional[list[str]] = None) -> argparse.Namespace:
|
||||
parser.error("--plugin-repo 只能用于插件基线操作")
|
||||
if args.diagnostics and not args.check_host:
|
||||
parser.error("--diagnostics 只能与 --check-host 一起使用")
|
||||
if args.report and not (args.check_host or args.check_plugins):
|
||||
parser.error("--report 只能与检查操作一起使用")
|
||||
return args
|
||||
|
||||
|
||||
@@ -966,6 +1039,18 @@ def main(argv: Optional[list[str]] = None) -> int:
|
||||
check_json(path, baseline, write_hint=write_hint)
|
||||
for path, baseline in baselines
|
||||
]
|
||||
if args.report:
|
||||
report_path = args.report.resolve()
|
||||
report_value = {
|
||||
"schema_version": 1,
|
||||
"checks": [
|
||||
build_comparison_report(path, baseline)
|
||||
for path, baseline in baselines
|
||||
],
|
||||
}
|
||||
print(f"即将写入报告:{_display_path(report_path)}")
|
||||
write_json(report_path, report_value)
|
||||
print(f"已写入报告:{_display_path(report_path)}")
|
||||
if args.diagnostics:
|
||||
print(json.dumps(collect_runtime_diagnostics(), ensure_ascii=False, indent=2))
|
||||
return 0 if all(checks) else 1
|
||||
|
||||
@@ -80,6 +80,17 @@ def test_architecture_diagnostics_only_support_host_check(capsys):
|
||||
assert "只能与 --check-host" in capsys.readouterr().err
|
||||
|
||||
|
||||
def test_architecture_report_only_supports_check_operations(capsys, tmp_path: Path):
|
||||
"""审查报告不能与 fixture 写操作混用。"""
|
||||
with pytest.raises(SystemExit) as error:
|
||||
architecture_baseline.parse_args(
|
||||
["--write-host", "--report", str(tmp_path / "report.json")]
|
||||
)
|
||||
|
||||
assert error.value.code == 2
|
||||
assert "只能与检查操作" in capsys.readouterr().err
|
||||
|
||||
|
||||
def test_runtime_semantics_ignore_line_changes_but_keep_call_count(tmp_path: Path):
|
||||
"""旧 fixture 的行号变化不影响门禁,重复调用次数仍属于语义。"""
|
||||
baseline_path = tmp_path / "runtime-contract-baseline.json"
|
||||
@@ -272,6 +283,66 @@ def test_architecture_write_plugins_only_updates_plugin_file(
|
||||
assert not runtime_path.exists()
|
||||
|
||||
|
||||
def test_architecture_plugin_check_writes_review_report_only_when_requested(
|
||||
tmp_path: Path,
|
||||
monkeypatch,
|
||||
):
|
||||
"""跨仓检查失败时应产出语义差异报告,并保持 fixture 不变。"""
|
||||
plugin_repo = tmp_path / "MoviePilot-Plugins"
|
||||
plugin_repo.mkdir()
|
||||
baseline_path = tmp_path / "official-plugin-baseline.json"
|
||||
report_path = tmp_path / "report.json"
|
||||
expected = {
|
||||
"schema_version": 3,
|
||||
"scope": {"repository": "MoviePilot-Plugins", "roots": ["plugins.v3"]},
|
||||
"provenance": {
|
||||
"head": "a" * 40,
|
||||
"python_file_count": 1,
|
||||
"source_sha256": "a" * 64,
|
||||
},
|
||||
"imports": {},
|
||||
"hooks": {},
|
||||
"api_routes": {},
|
||||
}
|
||||
actual = json.loads(json.dumps(expected))
|
||||
actual["provenance"]["head"] = "b" * 40
|
||||
actual["imports"] = {
|
||||
"app.sdk.logging": {"file_count": 1, "files": ["plugins.v3/demo.py"]}
|
||||
}
|
||||
baseline_path.write_text(json.dumps(expected), encoding="utf-8")
|
||||
content_before = baseline_path.read_bytes()
|
||||
monkeypatch.setattr(
|
||||
architecture_baseline,
|
||||
"PLUGIN_BASELINE_PATH",
|
||||
baseline_path,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
architecture_baseline,
|
||||
"collect_official_plugin_baseline",
|
||||
lambda _repository: actual,
|
||||
)
|
||||
|
||||
assert architecture_baseline.main(
|
||||
[
|
||||
"--check-plugins",
|
||||
"--plugin-repo",
|
||||
str(plugin_repo),
|
||||
"--report",
|
||||
str(report_path),
|
||||
]
|
||||
) == 1
|
||||
|
||||
report = json.loads(report_path.read_text(encoding="utf-8"))
|
||||
assert baseline_path.read_bytes() == content_before
|
||||
assert report["checks"][0]["semantic_match"] is False
|
||||
assert report["checks"][0]["added"] == [
|
||||
{
|
||||
"path": "$.imports.app.sdk.logging",
|
||||
"value": {"file_count": 1, "files": ["plugins.v3/demo.py"]},
|
||||
}
|
||||
]
|
||||
|
||||
|
||||
def test_performance_default_print_does_not_write_fixture(
|
||||
tmp_path: Path,
|
||||
monkeypatch,
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
"""后端架构治理工作流的静态契约测试。"""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from ruamel.yaml import YAML
|
||||
|
||||
|
||||
PROJECT_ROOT = Path(__file__).parents[1]
|
||||
WORKFLOW_ROOT = PROJECT_ROOT / ".github" / "workflows"
|
||||
|
||||
|
||||
def _load_workflow(name: str) -> dict:
|
||||
"""以 YAML 1.2 解析 GitHub Actions 工作流。"""
|
||||
yaml = YAML(typ="safe")
|
||||
return yaml.load((WORKFLOW_ROOT / name).read_text(encoding="utf-8"))
|
||||
|
||||
|
||||
def _step_commands(workflow: dict, job_name: str) -> str:
|
||||
"""拼接指定 job 的命令,便于断言关键门禁没有被移除。"""
|
||||
return "\n".join(
|
||||
step.get("run", "")
|
||||
for step in workflow["jobs"][job_name]["steps"]
|
||||
)
|
||||
|
||||
|
||||
def test_unit_test_workflow_has_independent_host_architecture_gate():
|
||||
"""主仓 PR 与推送必须在全量分片外快速执行宿主架构门禁。"""
|
||||
workflow = _load_workflow("test.yml")
|
||||
commands = _step_commands(workflow, "architecture")
|
||||
|
||||
assert "pull_request" in workflow["on"]
|
||||
assert "push" in workflow["on"]
|
||||
assert "tests/test_architecture_dependencies.py" in commands
|
||||
assert "tests/test_architecture_contract_baseline.py" in commands
|
||||
assert "scripts/architecture/baseline.py --check-host" in commands
|
||||
|
||||
|
||||
def test_official_plugin_observation_is_scheduled_and_never_writes_fixture():
|
||||
"""跨仓观察应定时或手工运行,只上传语义报告而不刷新 fixture。"""
|
||||
workflow = _load_workflow("architecture-observe.yml")
|
||||
commands = _step_commands(workflow, "observe")
|
||||
steps = workflow["jobs"]["observe"]["steps"]
|
||||
|
||||
assert "schedule" in workflow["on"]
|
||||
assert "workflow_dispatch" in workflow["on"]
|
||||
assert "--check-plugins" in commands
|
||||
assert "--report official-plugin-architecture-report.json" in commands
|
||||
assert "--write" not in commands
|
||||
assert any(
|
||||
step.get("uses", "").startswith("actions/upload-artifact@")
|
||||
for step in steps
|
||||
)
|
||||
|
||||
|
||||
def test_pylint_workflow_runs_for_v3_pull_requests_and_pushes():
|
||||
"""严重 Python 静态错误检查不能继续只依赖手工触发。"""
|
||||
workflow = _load_workflow("pylint.yml")
|
||||
|
||||
assert workflow["on"]["pull_request"]["branches"] == ["v3"]
|
||||
assert workflow["on"]["push"]["branches"] == ["v3"]
|
||||
Reference in New Issue
Block a user