mirror of
https://github.com/jxxghp/MoviePilot.git
synced 2026-09-04 15:09:46 +08:00
test(architecture): govern direct egress
This commit is contained in:
@@ -415,15 +415,19 @@ MoviePilot V3 已经形成较清晰的模块化单体:`foundation`、`domain`
|
||||
- `app/chain` 有 8 个文件、13 条直接 Adapter 导入,使用 `RequestUtils`、Browser、Cloudflare、
|
||||
CookieCloud、ServerHelper 等具体能力。
|
||||
- Passkey Application 服务直接判断 Redis 后端并调用 `RedisHelper.pop()`,安全策略识别了具体实现。
|
||||
- “所有 HTTP 必须经 RequestUtils”与 LLM streaming、第三方 SDK、移植库中的直接 HTTP 客户端现状
|
||||
没有一份精确例外表,也没有 zero-growth gate。
|
||||
- 审计时 LLM streaming、第三方 SDK、移植库和本地控制面没有精确例外表;S0-L2.4b 已建立
|
||||
66 条完整 egress identity 与 zero-growth policy,其中 11 条普通 HTTP/Session bridge 和 1 条
|
||||
Application DNS I/O 是清零债务;每条初始边另有独立指纹上界,不能靠同时刷新 baseline/policy
|
||||
掩盖同一边的调用面增长,债务删除后也不得恢复。
|
||||
|
||||
**目标与步骤**
|
||||
|
||||
- [x] 建立 Application/Chain 原始 Adapter 直连事实与精确临时 policy,冻结新增、替换和陈旧条目。
|
||||
- [x] 建立全宿主 direct egress 事实;SDK/stream/vendor/local-control 例外精确到 bindings/uses 指纹。
|
||||
- [ ] 将 Passkey 原子领取提升为 runtime cache contract,由 Memory/Redis backend 分别实现。
|
||||
- [ ] 为 Backup 定义 Application-owned artifact store Port,由 startup 注入文件系统实现。
|
||||
- [ ] 普通外部请求迁移到统一网络能力;SDK transport、streaming 和 vendor code 例外必须精确到路径与原因。
|
||||
- [ ] 将 policy 中 11 条普通 HTTP/Session bridge 债务迁移到统一网络能力并把目标收缩为空。
|
||||
- [ ] 为 Application SSRF 校验注入 DNS 解析 Port,清除 `socket.getaddrinfo` 直接 I/O。
|
||||
- [ ] 命名外部产品、安全敏感能力及通用技术 Adapter 均改为注入 Port;不在 Application/Chain 保留直连例外。
|
||||
- [ ] 最终把基线收缩到零或少量书面化例外,而不是一次性禁止后再大量豁免。
|
||||
|
||||
|
||||
@@ -673,6 +673,11 @@ flowchart LR
|
||||
`--write-host` 不会替代人工决策。
|
||||
- 同一 baseline 的 `direct_adapter_imports` 记录现存原始直连;policy 将其全部标为有 owner 的
|
||||
`temporary_debt`,并以初始 28 条冻结上界、目标为空集合。新增、替换、删除后未清理 policy 都会失败。
|
||||
- `direct_egress` 记录全宿主 66 条 raw transport、network SDK 和协议操作 identity;11 条普通 HTTP/
|
||||
Session bridge 与 1 条 Application DNS I/O 是清零债务,54 条 canonical transport、SDK、
|
||||
stream/vendor/diagnostic/control-plane 事实是精确 containment。每条初始边的指纹由测试独立冻结,
|
||||
bindings/uses 变化、分类互换、通配导入和初始边增长都会失败;债务删除时同步删除冻结项以禁止恢复,
|
||||
`--write-host` 不会改写人工 policy 或冻结上界。
|
||||
- 任何所有权迁移必须同步更新:canonical 导入、`app/runtime/compat/manifest.py`、
|
||||
SDK 导出(若公开)、`docs/rules/05-architecture.md` 与上述架构测试。
|
||||
- 延迟导入不被接受为隐藏循环依赖的手段。
|
||||
@@ -687,6 +692,7 @@ flowchart LR
|
||||
| Python 模块 | 835 |
|
||||
| 内部导入边 | 6,810 |
|
||||
| 非平凡 SCC | 2(`ARCH-107` 临时 Chain 包根环;精确 containment 的 TMDB 移植包环) |
|
||||
| Direct egress | 66(12 条待迁移债务,54 条精确 containment) |
|
||||
| Module Contract V2 spec | 215(其中 214 个进入 `run_module` 观察面) |
|
||||
| Event Contract | 53 |
|
||||
| Model/Oper 自动事务与自建 Session | 0 |
|
||||
|
||||
@@ -76,8 +76,8 @@ G-ARCH 只有在以下条件全部满足后才可完成:
|
||||
| S0-L2.1 Host Oper/UoW 规范 | `DELIVERED` | S0-L1 | `3bf94ffed`:宿主无 Session Oper 规范债务归零,远端 `0/0` |
|
||||
| S0-L2.2 完整宿主 SCC policy | `DELIVERED` | S0-L2.1 | `a884ab5c2`:完整宿主 SCC 精确 policy 生效,远端 `0/0` |
|
||||
| S0-L2.3 Adapter 直连事实 | `DELIVERED` | S0-L2.1 | `e1483e85d`:锁定 28 条原始 Adapter import 事实,远端 `0/0` |
|
||||
| S0-L2.4 Adapter zero-growth | `VERIFIED` | S0-L2.3 | 当前直连均登记迁移 owner,新增/替换失败,删除后要求清理陈旧 policy |
|
||||
| S0-L2.4b HTTP/Egress 事实与政策 | `PLANNED` | S0-L2.4 | direct HTTP/SDK/stream/vendor egress 分类完整,债务与精确例外分别受不可增长政策约束 |
|
||||
| S0-L2.4 Adapter zero-growth | `DELIVERED` | S0-L2.3 | `2553226f3`:冻结 28 条直连及 owner,收缩/新增/stale policy 门禁生效,远端 `0/0` |
|
||||
| S0-L2.4b HTTP/Egress 事实与政策 | `VERIFIED` | S0-L2.4 | 66 条事实、12 条债务和 54 条精确例外已冻结;collector、policy、全量测试与本地质量门禁通过,待推送后确认远端 CI |
|
||||
| S0-L2.5 Event consumer 识别 | `PLANNED` | S0-L2.1 | consumer 只识别可静态证明的 EventManager 注册,动态误报归零 |
|
||||
| S0-L2.6 事实源与 CI 投影 | `PLANNED` | S0-L2.2,S0-L2.4b,S0-L2.5 | fixture/policy/overview 职责固定,CI 分开报告语义 policy 与快照一致性 |
|
||||
|
||||
@@ -107,8 +107,8 @@ G-ARCH 只有在以下条件全部满足后才可完成:
|
||||
| S2-L3 GlobalVar/provider 注册收口 | `PLANNED` | S2-L1 | `global_vars` canonical 消费清零,provider 注册进入显式装配阶段并可 reset;Legacy 入口精确保留 |
|
||||
| S2-L4 Passkey 缓存边界 | `PLANNED` | S0-L4 | Application 不识别 Redis;原子 consume 由 runtime cache contract + backend 实现 |
|
||||
| S2-L5 Backup artifact Port | `PLANNED` | S0-L4 | Application 不构造 `BackupFiles`,文件 I/O 由注入 Adapter 拥有 |
|
||||
| S2-L6 Application Adapter 债务清零 | `PLANNED` | S2-L4,S2-L5 | Application 到具体 Adapter 的未批准边归零,批准通用机制有精确规则和门禁 |
|
||||
| S2-L7 Chain Adapter/HTTP 债务清零 | `PLANNED` | S2-L6 | Chain 具体 Adapter 与普通 direct HTTP 依赖归零;SDK/stream/vendor 例外精确 containment |
|
||||
| S2-L6 Application Adapter/DNS 债务清零 | `PLANNED` | S2-L4,S2-L5 | Application 到具体 Adapter 的未批准边归零,SSRF DNS I/O 进入注入 Port,批准通用机制有精确规则和门禁 |
|
||||
| S2-L7 Chain Adapter/宿主 HTTP 债务清零 | `PLANNED` | S2-L6 | Chain 具体 Adapter 与 11 条普通 direct HTTP/Session bridge 归零;SDK/stream/vendor 例外保持精确 containment |
|
||||
|
||||
### S3:大型编排器职责清零
|
||||
|
||||
@@ -154,50 +154,53 @@ G-ARCH 只有在以下条件全部满足后才可完成:
|
||||
|
||||
## 4. 当前活动叶子
|
||||
|
||||
### S0-L2.4 Adapter zero-growth
|
||||
### S0-L2.4b HTTP/Egress 事实与政策
|
||||
|
||||
**Status:** `VERIFIED`(本地验收完成,等待提交、推送和远端一致性确认)
|
||||
**Status:** `VERIFIED`(本地验收完成,等待提交、推送和远端 CI 确认)
|
||||
|
||||
**Outcome**
|
||||
|
||||
把 S0-L2.3 收集的 Application/Chain 到具体 Adapter 直连全部登记为有迁移 owner 的临时债务。
|
||||
初始 28 条形成只减不增的冻结上界,政策目标固定为空集合;新增、替换和删除后未同步清理的
|
||||
陈旧登记都必须失败。
|
||||
扫描宿主 raw transport、network SDK 和库名扫描会漏掉的协议操作,事实保留 import provenance、
|
||||
稳定 callable/operation,不保存行号。普通 HTTP/Session bridge 是待迁移债务;canonical transport、
|
||||
SDK、streaming、contained vendor 和 local control-plane 只允许精确 containment。
|
||||
|
||||
**Ownership**
|
||||
|
||||
- `tests/fixtures/architecture/dependency-policy.json` 的人工 Adapter policy。
|
||||
- `tests/test_architecture_adapter_imports.py` 的精确集合、owner 和 stale-policy 门禁。
|
||||
- `AGENTS.md`、`docs/rules/05-architecture.md` 与架构总览中的规范边界。
|
||||
- `.github/workflows/test.yml` 的快速架构 CI 投影。
|
||||
- `scripts/architecture/` 的 direct egress AST collector 与 registry。
|
||||
- `tests/fixtures/architecture/dependency-baseline.json` 的生成事实和
|
||||
`dependency-policy.json` 的人工分类。
|
||||
- `tests/test_architecture_egress.py` 的 collector、当前事实、policy 和 zero-growth 门禁。
|
||||
- HTTP 规范、架构总览、优化清单与快速架构 CI 投影。
|
||||
- 本路线图的叶子状态和交付记录。
|
||||
|
||||
**Excluded**
|
||||
|
||||
- 不在本叶迁移 28 条运行时依赖;迁移由 S2-L4 至 S2-L7 按 owner 债务清零。
|
||||
- 不修改生成事实、完整依赖图、SCC、digest、`app/plugins/**` 或运行时代码。
|
||||
- 不把 direct HTTP/SDK/stream/vendor egress 混入 Adapter policy;这些由 S0-L2.4b 独立治理。
|
||||
- 不在本叶迁移生产 HTTP 调用;本叶完成完整事实、分类和不可增长门禁。
|
||||
- 不修改 Adapter 直连 policy、完整依赖图/SCC/digest、运行时代码或 `app/plugins/**`。
|
||||
- 未登记 registry 的任意第三方包不能被猜测为网络 SDK;新增 SDK 必须显式扩展 registry 与 policy。
|
||||
|
||||
**Acceptance**
|
||||
|
||||
```bash
|
||||
.venv/bin/python -m pytest \
|
||||
tests/test_architecture_adapter_imports.py \
|
||||
tests/test_architecture_egress.py \
|
||||
tests/test_architecture_contract_baseline.py \
|
||||
tests/test_architecture_dependencies.py \
|
||||
tests/test_architecture_baseline_cli.py \
|
||||
tests/test_architecture_ci.py \
|
||||
tests/test_architecture_documentation.py -q
|
||||
tests/test_plugin_identity_transitions.py -q
|
||||
.venv/bin/python scripts/architecture/baseline.py --check-host
|
||||
.venv/bin/python scripts/architecture/ruff_ratchet.py
|
||||
.venv/bin/python scripts/architecture/mypy_ratchet.py
|
||||
.venv/bin/pylint tests/test_architecture_adapter_imports.py \
|
||||
.venv/bin/pylint scripts/architecture/baseline.py \
|
||||
scripts/architecture/egress.py \
|
||||
tests/test_architecture_egress.py \
|
||||
tests/test_architecture_contract_baseline.py \
|
||||
tests/test_architecture_dependencies.py \
|
||||
tests/test_architecture_ci.py
|
||||
tests/test_architecture_ci.py \
|
||||
tests/test_plugin_identity_transitions.py
|
||||
git diff --check
|
||||
```
|
||||
|
||||
**Delivery**
|
||||
|
||||
- 单一提交主题:冻结现有 Adapter 直连债务并阻止增长。
|
||||
- 单一提交主题:建立并冻结 direct egress 事实与精确政策。
|
||||
- 推送 `origin/v3` 后确认提交祖先关系、远端 SHA 和 ahead/behind `0/0`。
|
||||
|
||||
@@ -528,6 +528,25 @@ the Port required by its use case, startup injects the concrete Adapter, and Cha
|
||||
consumes the Application capability or an injected Port. A `canonical capability`
|
||||
never means permission to import a concrete `app.adapters.*` implementation.
|
||||
|
||||
Direct egress is a separate boundary from Adapter imports. The generated
|
||||
`direct_egress` facts scan the complete host `app` tree except `app.plugins` and
|
||||
record raw transports, registered network SDKs and exact protocol operations.
|
||||
Each identity contains import provenance plus stable callable/operation uses and
|
||||
has no line number. The manual policy classifies every full fingerprint as either
|
||||
`temporary_debt` with a removal leaf and empty target state, or an
|
||||
`approved_exception` with an exact owner and reason. Canonical transports, SDKs,
|
||||
streaming protocols, contained vendor code, diagnostics and control planes are
|
||||
contained exceptions, not category-wide permissions. In policy, `owner: "$source"`
|
||||
means the fact's exact `source` module is the owner; it does not authorize sibling
|
||||
or child modules. Runtime wildcard imports from a registered egress root are
|
||||
forbidden. Updating the generated baseline never updates this policy; additions,
|
||||
fact changes, classification swaps and stale entries fail independently. Current
|
||||
debt may shrink without changing a fixed count, but no initial edge may grow or be
|
||||
reclassified. Tests independently freeze every initial edge fingerprint, so
|
||||
refreshing both generated facts and manual policy cannot hide growth on the same
|
||||
`source/target`; when debt is removed, its frozen edge and fingerprint must be
|
||||
removed in the same reviewed change so that it cannot return.
|
||||
|
||||
## Permitted Call Directions
|
||||
|
||||
| Direction | Status |
|
||||
|
||||
@@ -2,11 +2,19 @@
|
||||
|
||||
## HTTP Client Conventions
|
||||
|
||||
**Rule:** Host outbound HTTP transport implementations must go through `RequestUtils` from
|
||||
**Rule:** Host-authored ordinary outbound HTTP must go through `RequestUtils` from
|
||||
`app/adapters/network/http.py`. This transport rule does not authorize Application or Chain to
|
||||
import the concrete Adapter; they own/use a Port and receive its implementation from startup.
|
||||
Plugins import the curated facade from `app.sdk.network`. Do not use `requests`, `httpx`, or
|
||||
`aiohttp` directly outside a separately reviewed SDK, streaming, or vendored transport boundary.
|
||||
Plugins import the curated facade from `app.sdk.network`.
|
||||
|
||||
Direct `requests`/`httpx` clients, SDK transports, product streaming protocols, contained vendor
|
||||
code, diagnostics and control-plane access are governed by the exact `direct_egress` facts and
|
||||
manual policy. An approved fingerprint is containment for that exact source/binding/use identity
|
||||
only; `owner: "$source"` names that source module and does not authorize another caller or
|
||||
operation. New SDK roots must be added to the registry and receive explicit policy review.
|
||||
Ordinary direct HTTP, RequestUtils Session bridges and Application-owned DNS I/O remain
|
||||
`temporary_debt` with an empty target state. Runtime wildcard imports from registered egress roots
|
||||
are forbidden.
|
||||
|
||||
`RequestUtils` handles:
|
||||
- Proxy configuration (from `settings.PROXY_*`)
|
||||
|
||||
Reference in New Issue
Block a user