test(architecture): govern direct egress

This commit is contained in:
jxxghp
2026-08-27 07:28:42 +08:00
parent 3b7f8c357d
commit 47f0de745c
19 changed files with 3598 additions and 38 deletions
+7 -3
View File
@@ -415,15 +415,19 @@ MoviePilot V3 已经形成较清晰的模块化单体:`foundation`、`domain`
- `app/chain` 有 8 个文件、13 条直接 Adapter 导入,使用 `RequestUtils`、Browser、Cloudflare、
CookieCloud、ServerHelper 等具体能力。
- Passkey Application 服务直接判断 Redis 后端并调用 `RedisHelper.pop()`,安全策略识别了具体实现。
- “所有 HTTP 必须经 RequestUtils”与 LLM streaming、第三方 SDK、移植库中的直接 HTTP 客户端现状
没有一份精确例外表,也没有 zero-growth gate。
- 审计时 LLM streaming、第三方 SDK、移植库和本地控制面没有精确例外表;S0-L2.4b 已建立
66 条完整 egress identity 与 zero-growth policy,其中 11 条普通 HTTP/Session bridge 和 1 条
Application DNS I/O 是清零债务;每条初始边另有独立指纹上界,不能靠同时刷新 baseline/policy
掩盖同一边的调用面增长,债务删除后也不得恢复。
**目标与步骤**
- [x] 建立 Application/Chain 原始 Adapter 直连事实与精确临时 policy,冻结新增、替换和陈旧条目。
- [x] 建立全宿主 direct egress 事实;SDK/stream/vendor/local-control 例外精确到 bindings/uses 指纹。
- [ ] 将 Passkey 原子领取提升为 runtime cache contract,由 Memory/Redis backend 分别实现。
- [ ] 为 Backup 定义 Application-owned artifact store Port,由 startup 注入文件系统实现。
- [ ] 普通外部请求迁移到统一网络能力;SDK transport、streaming 和 vendor code 例外必须精确到路径与原因
- [ ] 将 policy 中 11 条普通 HTTP/Session bridge 债务迁移到统一网络能力并把目标收缩为空
- [ ] 为 Application SSRF 校验注入 DNS 解析 Port,清除 `socket.getaddrinfo` 直接 I/O。
- [ ] 命名外部产品、安全敏感能力及通用技术 Adapter 均改为注入 Port;不在 Application/Chain 保留直连例外。
- [ ] 最终把基线收缩到零或少量书面化例外,而不是一次性禁止后再大量豁免。
+6
View File
@@ -673,6 +673,11 @@ flowchart LR
`--write-host` 不会替代人工决策。
- 同一 baseline 的 `direct_adapter_imports` 记录现存原始直连;policy 将其全部标为有 owner 的
`temporary_debt`,并以初始 28 条冻结上界、目标为空集合。新增、替换、删除后未清理 policy 都会失败。
- `direct_egress` 记录全宿主 66 条 raw transport、network SDK 和协议操作 identity11 条普通 HTTP/
Session bridge 与 1 条 Application DNS I/O 是清零债务,54 条 canonical transport、SDK、
stream/vendor/diagnostic/control-plane 事实是精确 containment。每条初始边的指纹由测试独立冻结,
bindings/uses 变化、分类互换、通配导入和初始边增长都会失败;债务删除时同步删除冻结项以禁止恢复,
`--write-host` 不会改写人工 policy 或冻结上界。
- 任何所有权迁移必须同步更新:canonical 导入、`app/runtime/compat/manifest.py`
SDK 导出(若公开)、`docs/rules/05-architecture.md` 与上述架构测试。
- 延迟导入不被接受为隐藏循环依赖的手段。
@@ -687,6 +692,7 @@ flowchart LR
| Python 模块 | 835 |
| 内部导入边 | 6,810 |
| 非平凡 SCC | 2`ARCH-107` 临时 Chain 包根环;精确 containment 的 TMDB 移植包环) |
| Direct egress | 6612 条待迁移债务,54 条精确 containment |
| Module Contract V2 spec | 215(其中 214 个进入 `run_module` 观察面) |
| Event Contract | 53 |
| Model/Oper 自动事务与自建 Session | 0 |
+26 -23
View File
@@ -76,8 +76,8 @@ G-ARCH 只有在以下条件全部满足后才可完成:
| S0-L2.1 Host Oper/UoW 规范 | `DELIVERED` | S0-L1 | `3bf94ffed`:宿主无 Session Oper 规范债务归零,远端 `0/0` |
| S0-L2.2 完整宿主 SCC policy | `DELIVERED` | S0-L2.1 | `a884ab5c2`:完整宿主 SCC 精确 policy 生效,远端 `0/0` |
| S0-L2.3 Adapter 直连事实 | `DELIVERED` | S0-L2.1 | `e1483e85d`:锁定 28 条原始 Adapter import 事实,远端 `0/0` |
| S0-L2.4 Adapter zero-growth | `VERIFIED` | S0-L2.3 | 当前直连均登记迁移 owner,新增/替换失败,删除后要求清理陈旧 policy |
| S0-L2.4b HTTP/Egress 事实与政策 | `PLANNED` | S0-L2.4 | direct HTTP/SDK/stream/vendor egress 分类完整,债务与精确例外分别受不可增长政策约束 |
| S0-L2.4 Adapter zero-growth | `DELIVERED` | S0-L2.3 | `2553226f3`:冻结 28 条直连及 owner,收缩/新增/stale policy 门禁生效,远端 `0/0` |
| S0-L2.4b HTTP/Egress 事实与政策 | `VERIFIED` | S0-L2.4 | 66 条事实、12 条债务和 54 条精确例外已冻结;collector、policy、全量测试与本地质量门禁通过,待推送后确认远端 CI |
| S0-L2.5 Event consumer 识别 | `PLANNED` | S0-L2.1 | consumer 只识别可静态证明的 EventManager 注册,动态误报归零 |
| S0-L2.6 事实源与 CI 投影 | `PLANNED` | S0-L2.2,S0-L2.4b,S0-L2.5 | fixture/policy/overview 职责固定,CI 分开报告语义 policy 与快照一致性 |
@@ -107,8 +107,8 @@ G-ARCH 只有在以下条件全部满足后才可完成:
| S2-L3 GlobalVar/provider 注册收口 | `PLANNED` | S2-L1 | `global_vars` canonical 消费清零,provider 注册进入显式装配阶段并可 reset;Legacy 入口精确保留 |
| S2-L4 Passkey 缓存边界 | `PLANNED` | S0-L4 | Application 不识别 Redis;原子 consume 由 runtime cache contract + backend 实现 |
| S2-L5 Backup artifact Port | `PLANNED` | S0-L4 | Application 不构造 `BackupFiles`,文件 I/O 由注入 Adapter 拥有 |
| S2-L6 Application Adapter 债务清零 | `PLANNED` | S2-L4,S2-L5 | Application 到具体 Adapter 的未批准边归零,批准通用机制有精确规则和门禁 |
| S2-L7 Chain Adapter/HTTP 债务清零 | `PLANNED` | S2-L6 | Chain 具体 Adapter 与普通 direct HTTP 依赖归零;SDK/stream/vendor 例外精确 containment |
| S2-L6 Application Adapter/DNS 债务清零 | `PLANNED` | S2-L4,S2-L5 | Application 到具体 Adapter 的未批准边归零,SSRF DNS I/O 进入注入 Port批准通用机制有精确规则和门禁 |
| S2-L7 Chain Adapter/宿主 HTTP 债务清零 | `PLANNED` | S2-L6 | Chain 具体 Adapter 与 11 条普通 direct HTTP/Session bridge 归零;SDK/stream/vendor 例外保持精确 containment |
### S3:大型编排器职责清零
@@ -154,50 +154,53 @@ G-ARCH 只有在以下条件全部满足后才可完成:
## 4. 当前活动叶子
### S0-L2.4 Adapter zero-growth
### S0-L2.4b HTTP/Egress 事实与政策
**Status:** `VERIFIED`(本地验收完成,等待提交、推送和远端一致性确认)
**Status:** `VERIFIED`(本地验收完成,等待提交、推送和远端 CI 确认)
**Outcome**
把 S0-L2.3 收集的 Application/Chain 到具体 Adapter 直连全部登记为有迁移 owner 的临时债务。
初始 28 条形成只减不增的冻结上界,政策目标固定为空集合;新增、替换和删除后未同步清理的
陈旧登记都必须失败
扫描宿主 raw transport、network SDK 和库名扫描会漏掉的协议操作,事实保留 import provenance、
稳定 callable/operation,不保存行号。普通 HTTP/Session bridge 是待迁移债务;canonical transport、
SDK、streaming、contained vendor 和 local control-plane 只允许精确 containment
**Ownership**
- `tests/fixtures/architecture/dependency-policy.json` 的人工 Adapter policy。
- `tests/test_architecture_adapter_imports.py` 的精确集合、owner 和 stale-policy 门禁。
- `AGENTS.md``docs/rules/05-architecture.md` 与架构总览中的规范边界
- `.github/workflows/test.yml` 的快速架构 CI 投影
- `scripts/architecture/` 的 direct egress AST collector 与 registry。
- `tests/fixtures/architecture/dependency-baseline.json` 的生成事实和
`dependency-policy.json` 的人工分类
- `tests/test_architecture_egress.py` 的 collector、当前事实、policy 和 zero-growth 门禁
- HTTP 规范、架构总览、优化清单与快速架构 CI 投影。
- 本路线图的叶子状态和交付记录。
**Excluded**
- 不在本叶迁移 28 条运行时依赖;迁移由 S2-L4 至 S2-L7 按 owner 债务清零
- 不修改生成事实、完整依赖图SCCdigest、`app/plugins/**` 或运行时代码
- 不把 direct HTTP/SDK/stream/vendor egress 混入 Adapter policy;这些由 S0-L2.4b 独立治理
- 不在本叶迁移生产 HTTP 调用;本叶完成完整事实、分类和不可增长门禁
- 不修改 Adapter 直连 policy、完整依赖图/SCC/digest、运行时代码或 `app/plugins/**`
- 未登记 registry 的任意第三方包不能被猜测为网络 SDK;新增 SDK 必须显式扩展 registry 与 policy
**Acceptance**
```bash
.venv/bin/python -m pytest \
tests/test_architecture_adapter_imports.py \
tests/test_architecture_egress.py \
tests/test_architecture_contract_baseline.py \
tests/test_architecture_dependencies.py \
tests/test_architecture_baseline_cli.py \
tests/test_architecture_ci.py \
tests/test_architecture_documentation.py -q
tests/test_plugin_identity_transitions.py -q
.venv/bin/python scripts/architecture/baseline.py --check-host
.venv/bin/python scripts/architecture/ruff_ratchet.py
.venv/bin/python scripts/architecture/mypy_ratchet.py
.venv/bin/pylint tests/test_architecture_adapter_imports.py \
.venv/bin/pylint scripts/architecture/baseline.py \
scripts/architecture/egress.py \
tests/test_architecture_egress.py \
tests/test_architecture_contract_baseline.py \
tests/test_architecture_dependencies.py \
tests/test_architecture_ci.py
tests/test_architecture_ci.py \
tests/test_plugin_identity_transitions.py
git diff --check
```
**Delivery**
- 单一提交主题:冻结现有 Adapter 直连债务并阻止增长
- 单一提交主题:建立并冻结 direct egress 事实与精确政策
- 推送 `origin/v3` 后确认提交祖先关系、远端 SHA 和 ahead/behind `0/0`
+19
View File
@@ -528,6 +528,25 @@ the Port required by its use case, startup injects the concrete Adapter, and Cha
consumes the Application capability or an injected Port. A `canonical capability`
never means permission to import a concrete `app.adapters.*` implementation.
Direct egress is a separate boundary from Adapter imports. The generated
`direct_egress` facts scan the complete host `app` tree except `app.plugins` and
record raw transports, registered network SDKs and exact protocol operations.
Each identity contains import provenance plus stable callable/operation uses and
has no line number. The manual policy classifies every full fingerprint as either
`temporary_debt` with a removal leaf and empty target state, or an
`approved_exception` with an exact owner and reason. Canonical transports, SDKs,
streaming protocols, contained vendor code, diagnostics and control planes are
contained exceptions, not category-wide permissions. In policy, `owner: "$source"`
means the fact's exact `source` module is the owner; it does not authorize sibling
or child modules. Runtime wildcard imports from a registered egress root are
forbidden. Updating the generated baseline never updates this policy; additions,
fact changes, classification swaps and stale entries fail independently. Current
debt may shrink without changing a fixed count, but no initial edge may grow or be
reclassified. Tests independently freeze every initial edge fingerprint, so
refreshing both generated facts and manual policy cannot hide growth on the same
`source/target`; when debt is removed, its frozen edge and fingerprint must be
removed in the same reviewed change so that it cannot return.
## Permitted Call Directions
| Direction | Status |
+11 -3
View File
@@ -2,11 +2,19 @@
## HTTP Client Conventions
**Rule:** Host outbound HTTP transport implementations must go through `RequestUtils` from
**Rule:** Host-authored ordinary outbound HTTP must go through `RequestUtils` from
`app/adapters/network/http.py`. This transport rule does not authorize Application or Chain to
import the concrete Adapter; they own/use a Port and receive its implementation from startup.
Plugins import the curated facade from `app.sdk.network`. Do not use `requests`, `httpx`, or
`aiohttp` directly outside a separately reviewed SDK, streaming, or vendored transport boundary.
Plugins import the curated facade from `app.sdk.network`.
Direct `requests`/`httpx` clients, SDK transports, product streaming protocols, contained vendor
code, diagnostics and control-plane access are governed by the exact `direct_egress` facts and
manual policy. An approved fingerprint is containment for that exact source/binding/use identity
only; `owner: "$source"` names that source module and does not authorize another caller or
operation. New SDK roots must be added to the registry and receive explicit policy review.
Ordinary direct HTTP, RequestUtils Session bridges and Application-owned DNS I/O remain
`temporary_debt` with an empty target state. Runtime wildcard imports from registered egress roots
are forbidden.
`RequestUtils` handles:
- Proxy configuration (from `settings.PROXY_*`)