GithubBackup/MoviePilot
1
0
Fork 0
mirror of https://github.com/jxxghp/MoviePilot.git synced 2026-08-29 12:06:51 +08:00
Code Issues Packages Projects Releases Wiki Activity

refactor: add transaction ownership ratchet

Browse Source
This commit is contained in:
jxxghp
2026-08-21 20:16:30 +08:00
parent bce440e97c
commit de2957b9de
8 changed files with 1330 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture-overview.md
+10 -2
View File
@@ -342,13 +342,18 @@ SQLAlchemy 查询。`models/` 与 `oper/` 按文件一一镜像（站点族聚
```mermaid
flowchart LR
Caller["Chain / Application / 端点 / Module"]
Entry["API / Scheduler / Agent<br/>逻辑操作入口"]
Command["Application Command<br/>事务所有者"]
UoW["app/db/uow.py<br/>commit / rollback"]
Oper["app/db/oper/*.py<br/>SubscribeOper / TransferHistoryOper ..."]
Models["app/db/models/*.py<br/>SQLAlchemy 模型"]
Engine["app/db/engine.py<br/>同步 + 异步引擎"]
DB[("PostgreSQL / SQLite")]
Caller --> Oper --> Models --> Engine --> DB
Entry --> Command --> Oper --> Models --> Engine --> DB
Entry --> UoW --> Engine
Command -.提交或回滚.-> UoW
Command -.commit 后副作用.-> Effects["Event / Scheduler / Report"]
Models -.before_insert/before_update.-> Norm["_identity.py<br/>media_source/media_id 归一化"]
```
@@ -356,6 +361,9 @@ flowchart LR
归 `app/application/`（见 `application/subscription/write.py`、`application/history.py`）。
订阅新增、查询、变更、删除、身份和搜索契约已经统一收口在 `application/subscription/`，
不再保留主题包之外的第二个写入入口。
- Oper 只 stage mutation，不创建独立 Session、不提交；Application Command 通过请求或任务
入口注入的 UnitOfWork 统一 `commit/rollback`，事件、刷新和上报只在 commit 成功后执行。
`transaction-debt-baseline.json` 将存量 178 个 Model 事务装饰器冻结为只降不增低水位。
- 每次表结构变更必须新增 `database/versions/` 下的 Alembic 迁移。
- 运行期业务配置使用 `SystemConfigKey` 枚举 + `SystemConfigOper`，禁止裸字符串键；
用户级配置使用 `UserConfigOper`。
docs/refactor/backend-architecture-next-stage.md
+1 -1
View File
@@ -6,7 +6,7 @@
> 审计范围：宿主后端；排除 `app/plugins/**` 运行时插件副本
> 规范优先级：`AGENTS.md` 与 `docs/rules/` 高于本文
> 相关文档：`docs/architecture-overview.md`、`docs/refactor/backend-architecture-governance.md`、`docs/refactor/backend-module-refactor-compatibility.md`
> 实施进度：阶段 0（ARCH-201～203）与阶段 1（ARCH-210～212）已完成，后续任务按 ID 独立提交和回滚
> 实施进度：阶段 0（ARCH-201～203）、阶段 1（ARCH-210～212）与 ARCH-220 已完成，后续任务按 ID 独立提交和回滚
## 1. 结论先行
docs/rules/10-data-and-persistent.md
+23
View File
@@ -81,6 +81,29 @@ the stub.
Oper classes accept and return persistence values. Turning a `MediaInfo` or
`MetaBase` into a row is business logic and lives in `app/application/`.
### Transaction ownership ratchet
- `tests/fixtures/architecture/transaction-debt-baseline.json` records the
existing Model transaction decorators. The current 178 legacy decorators are
migration debt: they may decrease but must never increase or move to a new
Model method.
- New Model methods must not use `db_query`, `db_update`, `async_db_query`, or
`async_db_update`, create a Session, or call `commit()` / `rollback()`.
- Oper receives a caller-owned Session and may query, add, update, delete, or
flush. A composable Oper method must not create its own Session and must not
commit or roll back.
- The API, Scheduler, Agent, or another logical operation entry creates the
Session and adapts it through `app/db/uow.py`. Application command code owns
`commit()` / `rollback()`; events, scheduling refresh, reports, and other
external effects run only after a successful commit.
- A synchronous Session is private to one worker thread. An AsyncSession is
private to one asyncio task/operation; neither may be stored in a process
singleton or reused by concurrent work.
Run `./.venv/bin/python scripts/architecture/baseline.py --check-host` after
persistence changes. A deliberate debt reduction may refresh the low-water mark
with `--write-host`; never refresh it to accept newly introduced debt.
**Standard Oper method conventions:**
```python
scripts/architecture/baseline.py
+210 -1
View File
@@ -19,6 +19,7 @@ APP_ROOT = PROJECT_ROOT / "app"
BASELINE_ROOT = PROJECT_ROOT / "tests" / "fixtures" / "architecture"
DEPENDENCY_BASELINE_PATH = BASELINE_ROOT / "dependency-baseline.json"
RUNTIME_BASELINE_PATH = BASELINE_ROOT / "runtime-contract-baseline.json"
TRANSACTION_BASELINE_PATH = BASELINE_ROOT / "transaction-debt-baseline.json"
PLUGIN_BASELINE_PATH = BASELINE_ROOT / "official-plugin-baseline.json"
PLUGIN_HOOKS = (
"get_actions",
@@ -37,6 +38,25 @@ PLUGIN_HOOKS = (
"init_plugin",
"stop_service",
)
MODEL_TRANSACTION_DECORATORS = {
"async_db_query",
"async_db_update",
"db_query",
"db_update",
}
SESSION_FACTORY_NAMES = {
"AsyncSession",
"AsyncSessionFactory",
"ScopedSession",
"Session",
"SessionFactory",
"async_session_scope",
"get_async_db",
"get_async_session_factory",
"get_db",
"get_scoped_session",
"get_session_factory",
}
def discover_modules() -> dict[str, Path]:
@@ -250,6 +270,137 @@ def collect_dependency_baseline() -> dict[str, Any]:
}
def _expression_name(node: ast.AST) -> str:
"""返回调用或装饰器表达式的点分名称，无法静态解析时返回空串。"""
if isinstance(node, ast.Name):
return node.id
if isinstance(node, ast.Attribute):
prefix = _expression_name(node.value)
return ".".join(part for part in (prefix, node.attr) if part)
if isinstance(node, ast.Call):
return _expression_name(node.func)
return ""
def _iter_owned_functions(tree: ast.Module) -> list[tuple[str, ast.AST]]:
"""收集模块函数与类方法的稳定限定名，不记录易漂移源码行号。"""
methods: list[tuple[str, ast.AST]] = []
def visit_class(node: ast.ClassDef, parents: tuple[str, ...]) -> None:
"""递归访问嵌套类，并收集直接定义的方法。"""
class_path = (*parents, node.name)
for child in node.body:
if isinstance(child, (ast.FunctionDef, ast.AsyncFunctionDef)):
methods.append((".".join((*class_path, child.name)), child))
elif isinstance(child, ast.ClassDef):
visit_class(child, class_path)
for statement in tree.body:
if isinstance(statement, ast.ClassDef):
visit_class(statement, ())
elif isinstance(statement, (ast.FunctionDef, ast.AsyncFunctionDef)):
methods.append((statement.name, statement))
return methods
def _collect_method_calls(
root: Path,
*,
operations: set[str],
) -> list[dict[str, str]]:
"""按文件、方法和操作收集指定调用，作为只降不增的事务债务清单。"""
calls: list[dict[str, str]] = []
for path in sorted(root.rglob("*.py")):
relative = path.relative_to(PROJECT_ROOT).as_posix()
for method, node in _iter_owned_functions(parse_source(path)):
for call in ast.walk(node):
if not isinstance(call, ast.Call):
continue
expression = _expression_name(call.func)
operation = expression.rsplit(".", 1)[-1]
if operation in operations:
calls.append(
{
"file": relative,
"method": method,
"operation": operation,
}
)
return sorted(
calls,
key=lambda item: (item["file"], item["method"], item["operation"]),
)
def collect_transaction_debt_baseline() -> dict[str, Any]:
"""记录 Model 自动事务和 Oper 会话所有权债务，供 CI 执行单向 ratchet。"""
model_root = APP_ROOT / "db" / "models"
oper_root = APP_ROOT / "db" / "oper"
decorated_methods: list[dict[str, str]] = []
for path in sorted(model_root.rglob("*.py")):
relative = path.relative_to(PROJECT_ROOT).as_posix()
for method, node in _iter_owned_functions(parse_source(path)):
for decorator in node.decorator_list:
decorator_name = _expression_name(decorator).rsplit(".", 1)[-1]
if decorator_name in MODEL_TRANSACTION_DECORATORS:
decorated_methods.append(
{
"decorator": decorator_name,
"file": relative,
"method": method,
}
)
decorated_methods.sort(
key=lambda item: (item["file"], item["method"], item["decorator"])
)
model_transaction_calls = _collect_method_calls(
model_root,
operations={"commit", "rollback"},
)
model_session_factories = _collect_method_calls(
model_root,
operations=SESSION_FACTORY_NAMES,
)
oper_transaction_calls = _collect_method_calls(
oper_root,
operations={"commit", "rollback"},
)
oper_session_factories = _collect_method_calls(
oper_root,
operations=SESSION_FACTORY_NAMES,
)
decorator_counts = {
decorator: sum(
item["decorator"] == decorator
for item in decorated_methods
)
for decorator in sorted(MODEL_TRANSACTION_DECORATORS)
}
return {
"schema_version": 1,
"scope": "app/db/models and app/db/oper transaction ownership debt",
"model_decorators": {
"count": len(decorated_methods),
"by_kind": decorator_counts,
"methods": decorated_methods,
},
"model_transaction_calls": {
"count": len(model_transaction_calls),
"calls": model_transaction_calls,
},
"model_session_factories": {
"count": len(model_session_factories),
"calls": model_session_factories,
},
"oper_transaction_calls": {
"count": len(oper_transaction_calls),
"calls": oper_transaction_calls,
},
"oper_session_factories": {
"count": len(oper_session_factories),
"calls": oper_session_factories,
},
}
def _collect_run_module_locations() -> tuple[
dict[str, list[dict[str, Any]]],
list[dict[str, Any]],
@@ -862,6 +1013,45 @@ def semantic_baseline(path: Path, value: dict[str, Any]) -> dict[str, Any]:
return value
def transaction_ratchet_matches(
expected: dict[str, Any],
actual: dict[str, Any],
) -> bool:
"""事务债务只允许删除既有条目，不允许新增或提高任一分类计数。"""
if expected.get("schema_version") != actual.get("schema_version"):
return False
if expected.get("scope") != actual.get("scope"):
return False
sections = (
("model_decorators", "methods"),
("model_transaction_calls", "calls"),
("model_session_factories", "calls"),
("oper_transaction_calls", "calls"),
("oper_session_factories", "calls"),
)
for section, entries_key in sections:
expected_section = expected.get(section, {})
actual_section = actual.get(section, {})
if actual_section.get("count", 0) > expected_section.get("count", 0):
return False
expected_entries = {
json.dumps(item, ensure_ascii=False, sort_keys=True)
for item in expected_section.get(entries_key, [])
}
actual_entries = {
json.dumps(item, ensure_ascii=False, sort_keys=True)
for item in actual_section.get(entries_key, [])
}
if not actual_entries.issubset(expected_entries):
return False
expected_kinds = expected.get("model_decorators", {}).get("by_kind", {})
actual_kinds = actual.get("model_decorators", {}).get("by_kind", {})
return all(
count <= expected_kinds.get(decorator, 0)
for decorator, count in actual_kinds.items()
)
def _compare_semantic_values(
expected: Any,
actual: Any,
@@ -919,9 +1109,12 @@ def build_comparison_report(path: Path, actual: dict[str, Any]) -> dict[str, Any
"changed": [],
}
_compare_semantic_values(expected_semantic, actual_semantic, "$", differences)
semantic_match = expected_semantic == actual_semantic
if path.name == TRANSACTION_BASELINE_PATH.name:
semantic_match = transaction_ratchet_matches(expected, actual)
return {
"baseline": str(_display_path(path)),
"semantic_match": expected_semantic == actual_semantic,
"semantic_match": semantic_match,
"expected_provenance": expected.get("provenance"),
"actual_provenance": actual.get("provenance"),
**differences,
@@ -936,6 +1129,21 @@ def check_json(
) -> bool:
"""比较当前扫描结果和已提交基线并输出限定范围的更新提示。"""
expected = json.loads(path.read_text(encoding="utf-8"))
if path.name == TRANSACTION_BASELINE_PATH.name:
if transaction_ratchet_matches(expected, actual):
if expected != actual:
print(
"事务债务已下降；门禁继续通过，可在本任务提交中显式运行 "
"scripts/architecture/baseline.py --write-host 固化新低水位",
file=sys.stderr,
)
return True
print(
f"事务债务出现新增：{_display_path(path)}；"
"Model 自动事务、直接 commit/rollback 或 Oper 自建 Session 不得增长",
file=sys.stderr,
)
return False
expected_semantic = semantic_baseline(path, expected)
actual_semantic = semantic_baseline(path, actual)
if expected_semantic == actual_semantic:
@@ -1016,6 +1224,7 @@ def main(argv: Optional[list[str]] = None) -> int:
baselines = [
(DEPENDENCY_BASELINE_PATH, collect_dependency_baseline()),
(RUNTIME_BASELINE_PATH, collect_runtime_baseline()),
(TRANSACTION_BASELINE_PATH, collect_transaction_debt_baseline()),
]
write_hint = "--write-host"
else:
tests/fixtures/architecture/transaction-debt-baseline.json
Vendored
+921
View File
@@ -0,0 +1,921 @@
{
"model_decorators": {
"by_kind": {
"async_db_query": 49,
"async_db_update": 13,
"db_query": 75,
"db_update": 41
},
"count": 178,
"methods": [
{
"decorator": "async_db_query",
"file": "app/db/models/agentchat.py",
"method": "AgentChat.async_get_by_session"
},
{
"decorator": "async_db_query",
"file": "app/db/models/agentchat.py",
"method": "AgentChat.async_list_by_page"
},
{
"decorator": "db_query",
"file": "app/db/models/agentchat.py",
"method": "AgentChat.get_by_session"
},
{
"decorator": "db_query",
"file": "app/db/models/agentchat.py",
"method": "AgentChat.list_by_page"
},
{
"decorator": "db_update",
"file": "app/db/models/agenttask.py",
"method": "AgentTask.add_task"
},
{
"decorator": "db_query",
"file": "app/db/models/agenttask.py",
"method": "AgentTask.get_for_user"
},
{
"decorator": "db_query",
"file": "app/db/models/agenttask.py",
"method": "AgentTask.list_for_user"
},
{
"decorator": "db_update",
"file": "app/db/models/agenttask.py",
"method": "AgentTask.update_task"
},
{
"decorator": "db_update",
"file": "app/db/models/agenttaskrun.py",
"method": "AgentTaskRun.begin_run"
},
{
"decorator": "db_update",
"file": "app/db/models/agenttaskrun.py",
"method": "AgentTaskRun.delete_task_and_runs"
},
{
"decorator": "db_update",
"file": "app/db/models/agenttaskrun.py",
"method": "AgentTaskRun.finish_run"
},
{
"decorator": "db_query",
"file": "app/db/models/agenttaskrun.py",
"method": "AgentTaskRun.get_by_run_id"
},
{
"decorator": "db_update",
"file": "app/db/models/agenttaskrun.py",
"method": "AgentTaskRun.interrupt_task"
},
{
"decorator": "db_query",
"file": "app/db/models/agenttaskrun.py",
"method": "AgentTaskRun.list_for_task"
},
{
"decorator": "db_update",
"file": "app/db/models/downloadfailure.py",
"method": "DownloadFailure.delete_expired"
},
{
"decorator": "db_query",
"file": "app/db/models/downloadfailure.py",
"method": "DownloadFailure.get_active_by_fingerprints"
},
{
"decorator": "db_update",
"file": "app/db/models/downloadfailure.py",
"method": "DownloadFailure.record_failure"
},
{
"decorator": "db_update",
"file": "app/db/models/downloadhistory.py",
"method": "DownloadFiles.delete_by_fullpath"
},
{
"decorator": "db_update",
"file": "app/db/models/downloadhistory.py",
"method": "DownloadFiles.delete_orphans"
},
{
"decorator": "db_query",
"file": "app/db/models/downloadhistory.py",
"method": "DownloadFiles.get_by_fullpath"
},
{
"decorator": "db_query",
"file": "app/db/models/downloadhistory.py",
"method": "DownloadFiles.get_by_hash"
},
{
"decorator": "db_query",
"file": "app/db/models/downloadhistory.py",
"method": "DownloadFiles.get_by_savepath"
},
{
"decorator": "async_db_query",
"file": "app/db/models/downloadhistory.py",
"method": "DownloadHistory.async_count"
},
{
"decorator": "async_db_query",
"file": "app/db/models/downloadhistory.py",
"method": "DownloadHistory.async_count_by_title"
},
{
"decorator": "async_db_query",
"file": "app/db/models/downloadhistory.py",
"method": "DownloadHistory.async_list_by_page"
},
{
"decorator": "async_db_query",
"file": "app/db/models/downloadhistory.py",
"method": "DownloadHistory.async_list_by_title"
},
{
"decorator": "db_update",
"file": "app/db/models/downloadhistory.py",
"method": "DownloadHistory.delete_before"
},
{
"decorator": "db_query",
"file": "app/db/models/downloadhistory.py",
"method": "DownloadHistory.get_by_hash"
},
{
"decorator": "db_query",
"file": "app/db/models/downloadhistory.py",
"method": "DownloadHistory.get_by_hashes"
},
{
"decorator": "db_query",
"file": "app/db/models/downloadhistory.py",
"method": "DownloadHistory.get_by_media_identity"
},
{
"decorator": "db_query",
"file": "app/db/models/downloadhistory.py",
"method": "DownloadHistory.get_by_path"
},
{
"decorator": "db_query",
"file": "app/db/models/downloadhistory.py",
"method": "DownloadHistory.get_last_by"
},
{
"decorator": "db_query",
"file": "app/db/models/downloadhistory.py",
"method": "DownloadHistory.list_by_date"
},
{
"decorator": "db_query",
"file": "app/db/models/downloadhistory.py",
"method": "DownloadHistory.list_by_page"
},
{
"decorator": "db_query",
"file": "app/db/models/downloadhistory.py",
"method": "DownloadHistory.list_by_type"
},
{
"decorator": "db_query",
"file": "app/db/models/downloadhistory.py",
"method": "DownloadHistory.list_by_user_date"
},
{
"decorator": "async_db_query",
"file": "app/db/models/mediaserver.py",
"method": "MediaServerItem.async_exist_by_media_identity"
},
{
"decorator": "async_db_query",
"file": "app/db/models/mediaserver.py",
"method": "MediaServerItem.async_exists_by_title"
},
{
"decorator": "async_db_query",
"file": "app/db/models/mediaserver.py",
"method": "MediaServerItem.async_get_by_itemid"
},
{
"decorator": "db_update",
"file": "app/db/models/mediaserver.py",
"method": "MediaServerItem.delete_excluded_servers"
},
{
"decorator": "db_update",
"file": "app/db/models/mediaserver.py",
"method": "MediaServerItem.delete_stale"
},
{
"decorator": "db_update",
"file": "app/db/models/mediaserver.py",
"method": "MediaServerItem.empty"
},
{
"decorator": "db_query",
"file": "app/db/models/mediaserver.py",
"method": "MediaServerItem.exist_by_media_identity"
},
{
"decorator": "db_query",
"file": "app/db/models/mediaserver.py",
"method": "MediaServerItem.exists_by_title"
},
{
"decorator": "db_query",
"file": "app/db/models/mediaserver.py",
"method": "MediaServerItem.get_by_itemid"
},
{
"decorator": "db_query",
"file": "app/db/models/mediaserver.py",
"method": "MediaServerItem.get_by_server_itemid"
},
{
"decorator": "async_db_query",
"file": "app/db/models/message.py",
"method": "Message.async_list_by_page"
},
{
"decorator": "async_db_query",
"file": "app/db/models/message.py",
"method": "Message.async_list_sent_by_page"
},
{
"decorator": "db_update",
"file": "app/db/models/message.py",
"method": "Message.create_and_to_dict"
},
{
"decorator": "db_update",
"file": "app/db/models/message.py",
"method": "Message.delete_before"
},
{
"decorator": "db_query",
"file": "app/db/models/message.py",
"method": "Message.exists_by_source"
},
{
"decorator": "db_query",
"file": "app/db/models/message.py",
"method": "Message.list_by_page"
},
{
"decorator": "async_db_update",
"file": "app/db/models/passkey.py",
"method": "PassKey.async_delete_by_id"
},
{
"decorator": "async_db_query",
"file": "app/db/models/passkey.py",
"method": "PassKey.async_get_by_credential_id"
},
{
"decorator": "async_db_query",
"file": "app/db/models/passkey.py",
"method": "PassKey.async_get_by_id"
},
{
"decorator": "async_db_query",
"file": "app/db/models/passkey.py",
"method": "PassKey.async_get_by_user_id"
},
{
"decorator": "async_db_update",
"file": "app/db/models/passkey.py",
"method": "PassKey.async_update_last_used"
},
{
"decorator": "db_update",
"file": "app/db/models/passkey.py",
"method": "PassKey.delete_by_id"
},
{
"decorator": "db_query",
"file": "app/db/models/passkey.py",
"method": "PassKey.get_by_credential_id"
},
{
"decorator": "db_query",
"file": "app/db/models/passkey.py",
"method": "PassKey.get_by_id"
},
{
"decorator": "db_query",
"file": "app/db/models/passkey.py",
"method": "PassKey.get_by_user_id"
},
{
"decorator": "db_update",
"file": "app/db/models/passkey.py",
"method": "PassKey.update_last_used"
},
{
"decorator": "async_db_query",
"file": "app/db/models/plugindata.py",
"method": "PluginData.async_get_plugin_data"
},
{
"decorator": "async_db_query",
"file": "app/db/models/plugindata.py",
"method": "PluginData.async_get_plugin_data_by_key"
},
{
"decorator": "async_db_query",
"file": "app/db/models/plugindata.py",
"method": "PluginData.async_get_plugin_data_by_plugin_id"
},
{
"decorator": "db_update",
"file": "app/db/models/plugindata.py",
"method": "PluginData.del_plugin_data"
},
{
"decorator": "db_update",
"file": "app/db/models/plugindata.py",
"method": "PluginData.del_plugin_data_by_key"
},
{
"decorator": "db_query",
"file": "app/db/models/plugindata.py",
"method": "PluginData.get_plugin_data"
},
{
"decorator": "db_query",
"file": "app/db/models/plugindata.py",
"method": "PluginData.get_plugin_data_by_key"
},
{
"decorator": "db_query",
"file": "app/db/models/plugindata.py",
"method": "PluginData.get_plugin_data_by_plugin_id"
},
{
"decorator": "async_db_query",
"file": "app/db/models/site.py",
"method": "Site.async_get_actives"
},
{
"decorator": "async_db_query",
"file": "app/db/models/site.py",
"method": "Site.async_get_by_domain"
},
{
"decorator": "async_db_query",
"file": "app/db/models/site.py",
"method": "Site.async_get_by_name"
},
{
"decorator": "async_db_query",
"file": "app/db/models/site.py",
"method": "Site.async_list_order_by_pri"
},
{
"decorator": "async_db_update",
"file": "app/db/models/site.py",
"method": "Site.async_reset"
},
{
"decorator": "db_query",
"file": "app/db/models/site.py",
"method": "Site.get_actives"
},
{
"decorator": "db_query",
"file": "app/db/models/site.py",
"method": "Site.get_by_domain"
},
{
"decorator": "db_query",
"file": "app/db/models/site.py",
"method": "Site.get_domains_by_ids"
},
{
"decorator": "db_query",
"file": "app/db/models/site.py",
"method": "Site.list_order_by_pri"
},
{
"decorator": "db_update",
"file": "app/db/models/site.py",
"method": "Site.reset"
},
{
"decorator": "async_db_query",
"file": "app/db/models/siteicon.py",
"method": "SiteIcon.async_get_by_domain"
},
{
"decorator": "db_query",
"file": "app/db/models/siteicon.py",
"method": "SiteIcon.get_by_domain"
},
{
"decorator": "async_db_query",
"file": "app/db/models/sitestatistic.py",
"method": "SiteStatistic.async_get_by_domain"
},
{
"decorator": "db_query",
"file": "app/db/models/sitestatistic.py",
"method": "SiteStatistic.get_by_domain"
},
{
"decorator": "db_update",
"file": "app/db/models/sitestatistic.py",
"method": "SiteStatistic.reset"
},
{
"decorator": "async_db_query",
"file": "app/db/models/siteuserdata.py",
"method": "SiteUserData.async_get_by_domain"
},
{
"decorator": "async_db_query",
"file": "app/db/models/siteuserdata.py",
"method": "SiteUserData.async_get_latest"
},
{
"decorator": "db_update",
"file": "app/db/models/siteuserdata.py",
"method": "SiteUserData.delete_before"
},
{
"decorator": "db_query",
"file": "app/db/models/siteuserdata.py",
"method": "SiteUserData.get_by_date"
},
{
"decorator": "db_query",
"file": "app/db/models/siteuserdata.py",
"method": "SiteUserData.get_by_domain"
},
{
"decorator": "db_query",
"file": "app/db/models/siteuserdata.py",
"method": "SiteUserData.get_latest"
},
{
"decorator": "async_db_update",
"file": "app/db/models/subscribe.py",
"method": "Subscribe.async_delete_by_media_identity"
},
{
"decorator": "async_db_query",
"file": "app/db/models/subscribe.py",
"method": "Subscribe.async_exists"
},
{
"decorator": "async_db_query",
"file": "app/db/models/subscribe.py",
"method": "Subscribe.async_exists_by_username"
},
{
"decorator": "async_db_query",
"file": "app/db/models/subscribe.py",
"method": "Subscribe.async_get_by"
},
{
"decorator": "async_db_query",
"file": "app/db/models/subscribe.py",
"method": "Subscribe.async_get_by_state"
},
{
"decorator": "async_db_query",
"file": "app/db/models/subscribe.py",
"method": "Subscribe.async_get_by_title"
},
{
"decorator": "async_db_query",
"file": "app/db/models/subscribe.py",
"method": "Subscribe.async_list_by_media_identity"
},
{
"decorator": "async_db_query",
"file": "app/db/models/subscribe.py",
"method": "Subscribe.async_list_by_title"
},
{
"decorator": "async_db_query",
"file": "app/db/models/subscribe.py",
"method": "Subscribe.async_list_by_type"
},
{
"decorator": "async_db_query",
"file": "app/db/models/subscribe.py",
"method": "Subscribe.async_list_by_username"
},
{
"decorator": "db_update",
"file": "app/db/models/subscribe.py",
"method": "Subscribe.delete_by_media_identity"
},
{
"decorator": "db_query",
"file": "app/db/models/subscribe.py",
"method": "Subscribe.exists"
},
{
"decorator": "db_query",
"file": "app/db/models/subscribe.py",
"method": "Subscribe.exists_by_username"
},
{
"decorator": "db_query",
"file": "app/db/models/subscribe.py",
"method": "Subscribe.get_by"
},
{
"decorator": "db_query",
"file": "app/db/models/subscribe.py",
"method": "Subscribe.get_by_state"
},
{
"decorator": "db_query",
"file": "app/db/models/subscribe.py",
"method": "Subscribe.get_by_title"
},
{
"decorator": "db_query",
"file": "app/db/models/subscribe.py",
"method": "Subscribe.list_by_media_identity"
},
{
"decorator": "db_query",
"file": "app/db/models/subscribe.py",
"method": "Subscribe.list_by_type"
},
{
"decorator": "db_query",
"file": "app/db/models/subscribe.py",
"method": "Subscribe.list_by_username"
},
{
"decorator": "async_db_query",
"file": "app/db/models/subscribehistory.py",
"method": "SubscribeHistory.async_exists"
},
{
"decorator": "async_db_query",
"file": "app/db/models/subscribehistory.py",
"method": "SubscribeHistory.async_list_by_type"
},
{
"decorator": "async_db_query",
"file": "app/db/models/subscribehistory.py",
"method": "SubscribeHistory.async_list_by_type_and_username"
},
{
"decorator": "db_query",
"file": "app/db/models/subscribehistory.py",
"method": "SubscribeHistory.exists"
},
{
"decorator": "db_query",
"file": "app/db/models/subscribehistory.py",
"method": "SubscribeHistory.list_by_type"
},
{
"decorator": "async_db_query",
"file": "app/db/models/systemconfig.py",
"method": "SystemConfig.async_get_by_key"
},
{
"decorator": "db_update",
"file": "app/db/models/systemconfig.py",
"method": "SystemConfig.delete_by_key"
},
{
"decorator": "db_query",
"file": "app/db/models/systemconfig.py",
"method": "SystemConfig.get_by_key"
},
{
"decorator": "async_db_query",
"file": "app/db/models/transferhistory.py",
"method": "TransferHistory.async_count"
},
{
"decorator": "async_db_query",
"file": "app/db/models/transferhistory.py",
"method": "TransferHistory.async_count_by_title"
},
{
"decorator": "async_db_query",
"file": "app/db/models/transferhistory.py",
"method": "TransferHistory.async_list_by_page"
},
{
"decorator": "async_db_query",
"file": "app/db/models/transferhistory.py",
"method": "TransferHistory.async_list_by_title"
},
{
"decorator": "async_db_query",
"file": "app/db/models/transferhistory.py",
"method": "TransferHistory.async_statistic"
},
{
"decorator": "db_query",
"file": "app/db/models/transferhistory.py",
"method": "TransferHistory.count"
},
{
"decorator": "db_query",
"file": "app/db/models/transferhistory.py",
"method": "TransferHistory.count_by_title"
},
{
"decorator": "db_update",
"file": "app/db/models/transferhistory.py",
"method": "TransferHistory.delete_before"
},
{
"decorator": "db_query",
"file": "app/db/models/transferhistory.py",
"method": "TransferHistory.get_by_dest"
},
{
"decorator": "db_query",
"file": "app/db/models/transferhistory.py",
"method": "TransferHistory.get_by_hash"
},
{
"decorator": "db_query",
"file": "app/db/models/transferhistory.py",
"method": "TransferHistory.get_by_media_identity"
},
{
"decorator": "db_query",
"file": "app/db/models/transferhistory.py",
"method": "TransferHistory.get_by_src"
},
{
"decorator": "db_query",
"file": "app/db/models/transferhistory.py",
"method": "TransferHistory.get_success_by_src"
},
{
"decorator": "db_query",
"file": "app/db/models/transferhistory.py",
"method": "TransferHistory.list_by"
},
{
"decorator": "db_query",
"file": "app/db/models/transferhistory.py",
"method": "TransferHistory.list_by_date"
},
{
"decorator": "db_query",
"file": "app/db/models/transferhistory.py",
"method": "TransferHistory.list_by_hash"
},
{
"decorator": "db_query",
"file": "app/db/models/transferhistory.py",
"method": "TransferHistory.list_by_page"
},
{
"decorator": "db_query",
"file": "app/db/models/transferhistory.py",
"method": "TransferHistory.list_by_title"
},
{
"decorator": "db_query",
"file": "app/db/models/transferhistory.py",
"method": "TransferHistory.list_success_by_src"
},
{
"decorator": "db_query",
"file": "app/db/models/transferhistory.py",
"method": "TransferHistory.list_success_move_by_dest"
},
{
"decorator": "db_query",
"file": "app/db/models/transferhistory.py",
"method": "TransferHistory.monthly_media_statistics"
},
{
"decorator": "db_update",
"file": "app/db/models/transferhistory.py",
"method": "TransferHistory.replace_by_src"
},
{
"decorator": "db_query",
"file": "app/db/models/transferhistory.py",
"method": "TransferHistory.statistic"
},
{
"decorator": "db_update",
"file": "app/db/models/transferhistory.py",
"method": "TransferHistory.update_download_hash"
},
{
"decorator": "db_update",
"file": "app/db/models/transferpending.py",
"method": "TransferPending.clear"
},
{
"decorator": "db_update",
"file": "app/db/models/transferpending.py",
"method": "TransferPending.discard"
},
{
"decorator": "db_query",
"file": "app/db/models/transferpending.py",
"method": "TransferPending.list_all"
},
{
"decorator": "db_update",
"file": "app/db/models/transferpending.py",
"method": "TransferPending.register"
},
{
"decorator": "async_db_update",
"file": "app/db/models/user.py",
"method": "User.async_delete_by_id"
},
{
"decorator": "async_db_update",
"file": "app/db/models/user.py",
"method": "User.async_delete_by_name"
},
{
"decorator": "async_db_query",
"file": "app/db/models/user.py",
"method": "User.async_get_by_id"
},
{
"decorator": "async_db_query",
"file": "app/db/models/user.py",
"method": "User.async_get_by_name"
},
{
"decorator": "async_db_update",
"file": "app/db/models/user.py",
"method": "User.async_update_otp_by_name"
},
{
"decorator": "db_update",
"file": "app/db/models/user.py",
"method": "User.delete_by_id"
},
{
"decorator": "db_update",
"file": "app/db/models/user.py",
"method": "User.delete_by_name"
},
{
"decorator": "db_query",
"file": "app/db/models/user.py",
"method": "User.get_by_id"
},
{
"decorator": "db_query",
"file": "app/db/models/user.py",
"method": "User.get_by_name"
},
{
"decorator": "db_update",
"file": "app/db/models/user.py",
"method": "User.update_otp_by_name"
},
{
"decorator": "db_update",
"file": "app/db/models/userconfig.py",
"method": "UserConfig.delete_by_key"
},
{
"decorator": "db_query",
"file": "app/db/models/userconfig.py",
"method": "UserConfig.get_by_key"
},
{
"decorator": "async_db_update",
"file": "app/db/models/workflow.py",
"method": "Workflow.async_fail"
},
{
"decorator": "async_db_query",
"file": "app/db/models/workflow.py",
"method": "Workflow.async_get_by_name"
},
{
"decorator": "async_db_query",
"file": "app/db/models/workflow.py",
"method": "Workflow.async_get_enabled_workflows"
},
{
"decorator": "async_db_query",
"file": "app/db/models/workflow.py",
"method": "Workflow.async_get_event_triggered_workflows"
},
{
"decorator": "async_db_query",
"file": "app/db/models/workflow.py",
"method": "Workflow.async_get_timer_triggered_workflows"
},
{
"decorator": "async_db_update",
"file": "app/db/models/workflow.py",
"method": "Workflow.async_reset"
},
{
"decorator": "async_db_update",
"file": "app/db/models/workflow.py",
"method": "Workflow.async_start"
},
{
"decorator": "async_db_update",
"file": "app/db/models/workflow.py",
"method": "Workflow.async_success"
},
{
"decorator": "async_db_update",
"file": "app/db/models/workflow.py",
"method": "Workflow.async_update_current_action"
},
{
"decorator": "async_db_update",
"file": "app/db/models/workflow.py",
"method": "Workflow.async_update_state"
},
{
"decorator": "db_update",
"file": "app/db/models/workflow.py",
"method": "Workflow.fail"
},
{
"decorator": "db_query",
"file": "app/db/models/workflow.py",
"method": "Workflow.get_by_name"
},
{
"decorator": "db_query",
"file": "app/db/models/workflow.py",
"method": "Workflow.get_enabled_workflows"
},
{
"decorator": "db_query",
"file": "app/db/models/workflow.py",
"method": "Workflow.get_event_triggered_workflows"
},
{
"decorator": "db_query",
"file": "app/db/models/workflow.py",
"method": "Workflow.get_timer_triggered_workflows"
},
{
"decorator": "db_update",
"file": "app/db/models/workflow.py",
"method": "Workflow.reset"
},
{
"decorator": "db_update",
"file": "app/db/models/workflow.py",
"method": "Workflow.start"
},
{
"decorator": "db_update",
"file": "app/db/models/workflow.py",
"method": "Workflow.success"
},
{
"decorator": "db_update",
"file": "app/db/models/workflow.py",
"method": "Workflow.update_current_action"
},
{
"decorator": "db_update",
"file": "app/db/models/workflow.py",
"method": "Workflow.update_state"
}
]
},
"model_session_factories": {
"calls": [],
"count": 0
},
"model_transaction_calls": {
"calls": [],
"count": 0
},
"oper_session_factories": {
"calls": [],
"count": 0
},
"oper_transaction_calls": {
"calls": [],
"count": 0
},
"schema_version": 1,
"scope": "app/db/models and app/db/oper transaction ownership debt"
}
tests/test_architecture_baseline_cli.py
+72
View File
@@ -45,6 +45,28 @@ def _performance_sample(*, loaded_module_count: int = 10) -> dict:
}
def _transaction_sample(methods: list[dict[str, str]]) -> dict:
"""构造最小事务债务 fixture，供单向 ratchet 行为测试。"""
return {
"schema_version": 1,
"scope": "app/db/models and app/db/oper transaction ownership debt",
"model_decorators": {
"count": len(methods),
"by_kind": {
"async_db_query": 0,
"async_db_update": 0,
"db_query": 0,
"db_update": len(methods),
},
"methods": methods,
},
"model_transaction_calls": {"count": 0, "calls": []},
"model_session_factories": {"count": 0, "calls": []},
"oper_transaction_calls": {"count": 0, "calls": []},
"oper_session_factories": {"count": 0, "calls": []},
}
def test_architecture_legacy_action_requires_scope(capsys):
"""旧操作未明确宿主或插件范围时必须拒绝执行。"""
with pytest.raises(SystemExit) as error:
@@ -209,6 +231,34 @@ def test_plugin_v2_fixture_migrates_before_semantic_comparison(tmp_path: Path):
) == architecture_baseline.semantic_baseline(baseline_path, new_value)
def test_transaction_ratchet_allows_removal_but_rejects_new_method() -> None:
"""事务债务低水位允许下降，替换或新增 Model 自动事务仍必须失败。"""
first = {
"decorator": "db_update",
"file": "app/db/models/demo.py",
"method": "Demo.save",
}
second = {
"decorator": "db_update",
"file": "app/db/models/demo.py",
"method": "Demo.delete",
}
expected = _transaction_sample([first])
assert architecture_baseline.transaction_ratchet_matches(
expected,
_transaction_sample([]),
)
assert not architecture_baseline.transaction_ratchet_matches(
expected,
_transaction_sample([first, second]),
)
assert not architecture_baseline.transaction_ratchet_matches(
expected,
_transaction_sample([second]),
)
def test_architecture_write_host_only_updates_host_files(
tmp_path: Path,
monkeypatch,
@@ -217,6 +267,7 @@ def test_architecture_write_host_only_updates_host_files(
"""宿主写操作不得连带覆盖官方插件 fixture。"""
dependency_path = tmp_path / "dependency.json"
runtime_path = tmp_path / "runtime.json"
transaction_path = tmp_path / "transaction.json"
plugin_path = tmp_path / "plugin.json"
monkeypatch.setattr(
architecture_baseline,
@@ -228,6 +279,11 @@ def test_architecture_write_host_only_updates_host_files(
"RUNTIME_BASELINE_PATH",
runtime_path,
)
monkeypatch.setattr(
architecture_baseline,
"TRANSACTION_BASELINE_PATH",
transaction_path,
)
monkeypatch.setattr(architecture_baseline, "PLUGIN_BASELINE_PATH", plugin_path)
monkeypatch.setattr(
architecture_baseline,
@@ -239,16 +295,25 @@ def test_architecture_write_host_only_updates_host_files(
"collect_runtime_baseline",
lambda: {"scope": "host-runtime"},
)
monkeypatch.setattr(
architecture_baseline,
"collect_transaction_debt_baseline",
lambda: {"scope": "host-transaction"},
)
assert architecture_baseline.main(["--write-host"]) == 0
assert json.loads(dependency_path.read_text()) == {"scope": "host-dependency"}
assert json.loads(runtime_path.read_text()) == {"scope": "host-runtime"}
assert json.loads(transaction_path.read_text()) == {
"scope": "host-transaction"
}
assert not plugin_path.exists()
output = capsys.readouterr().out
assert "即将写入" in output
assert "dependency.json" in output
assert "runtime.json" in output
assert "transaction.json" in output
def test_architecture_write_plugins_only_updates_plugin_file(
@@ -261,6 +326,7 @@ def test_architecture_write_plugins_only_updates_plugin_file(
(plugin_repo / "plugins.v3").mkdir()
dependency_path = tmp_path / "dependency.json"
runtime_path = tmp_path / "runtime.json"
transaction_path = tmp_path / "transaction.json"
plugin_path = tmp_path / "plugin.json"
monkeypatch.setattr(
architecture_baseline,
@@ -272,6 +338,11 @@ def test_architecture_write_plugins_only_updates_plugin_file(
"RUNTIME_BASELINE_PATH",
runtime_path,
)
monkeypatch.setattr(
architecture_baseline,
"TRANSACTION_BASELINE_PATH",
transaction_path,
)
monkeypatch.setattr(architecture_baseline, "PLUGIN_BASELINE_PATH", plugin_path)
assert architecture_baseline.main(
@@ -281,6 +352,7 @@ def test_architecture_write_plugins_only_updates_plugin_file(
assert plugin_path.is_file()
assert not dependency_path.exists()
assert not runtime_path.exists()
assert not transaction_path.exists()
def test_architecture_plugin_check_writes_review_report_only_when_requested(
tests/test_architecture_contract_baseline.py
+27
View File
@@ -16,6 +16,7 @@ def test_architecture_contract_baselines_match_current_source():
baseline_paths = (
BASELINE_ROOT / "dependency-baseline.json",
BASELINE_ROOT / "runtime-contract-baseline.json",
BASELINE_ROOT / "transaction-debt-baseline.json",
)
contents_before = {
path: path.read_bytes()
@@ -56,6 +57,18 @@ def test_official_plugin_baseline_records_external_source():
)
def test_dependency_baseline_records_nonempty_host_graph() -> None:
"""宿主依赖 fixture 不得因收集器提前返回而被静默写成空值。"""
baseline_path = BASELINE_ROOT / "dependency-baseline.json"
baseline = json.loads(baseline_path.read_text(encoding="utf-8"))
assert baseline["schema_version"] == 1
assert baseline["module_count"] == len(baseline["modules"])
assert baseline["edge_count"] == len(baseline["edges"])
assert baseline["module_count"] > 0
assert baseline["edge_count"] > 0
def test_official_discovery_plugins_explicitly_keep_host_page_envelope():
"""宿主探索页消费的官方插件 API 不得依赖动态路由隐式包装。"""
baseline_path = BASELINE_ROOT / "official-plugin-baseline.json"
@@ -105,6 +118,20 @@ def test_runtime_contract_baseline_excludes_diagnostic_line_numbers():
assert '"line"' not in json.dumps(baseline)
def test_transaction_debt_baseline_is_a_model_and_oper_ratchet() -> None:
"""事务 fixture 必须冻结存量 Model 自动提交，并保持 Oper 自提交为零。"""
baseline_path = BASELINE_ROOT / "transaction-debt-baseline.json"
baseline = json.loads(baseline_path.read_text(encoding="utf-8"))
assert baseline["schema_version"] == 1
assert baseline["model_decorators"]["count"] == 178
assert sum(baseline["model_decorators"]["by_kind"].values()) == 178
assert baseline["model_transaction_calls"] == {"count": 0, "calls": []}
assert baseline["model_session_factories"] == {"count": 0, "calls": []}
assert baseline["oper_transaction_calls"] == {"count": 0, "calls": []}
assert baseline["oper_session_factories"] == {"count": 0, "calls": []}
def test_startup_performance_baseline_records_normal_and_safe_lifecycle_resources():
"""非功能基线必须同时记录正常/安全模式和隔离资源增量。"""
baseline_path = BASELINE_ROOT / "startup-performance-baseline.json"
tests/test_db_session_lifecycle.py
+66
View File
@@ -6,9 +6,12 @@
缓存的池化引擎，漏掉任何一类都是连接泄漏。
"""
import asyncio
import threading
from concurrent.futures import ThreadPoolExecutor
from unittest.mock import AsyncMock, MagicMock
import pytest
from sqlalchemy.orm import scoped_session, sessionmaker
from app.runtime.config import global_vars, settings
from app.db import engine as engine_module
@@ -90,6 +93,69 @@ def test_get_async_db_yields_session_from_scope(monkeypatch):
assert used == ["enter", "exit"], "会话作用域未正确进入/退出"
def test_scoped_sessions_are_not_shared_across_worker_threads(monkeypatch):
"""同步入口必须为并行工作线程提供不同 Session 实例。"""
registry = scoped_session(sessionmaker())
barrier = threading.Barrier(2)
monkeypatch.setattr(session_module, "_scoped_session", registry)
def open_in_thread() -> int:
"""在线程内持有会话直到另一个线程也完成解析。"""
session = session_module.ScopedSession()
try:
barrier.wait(timeout=5)
return id(session)
finally:
registry.remove()
with ThreadPoolExecutor(max_workers=2) as executor:
session_ids = list(executor.map(lambda _: open_in_thread(), range(2)))
assert len(set(session_ids)) == 2
def test_async_session_scopes_are_not_shared_across_tasks(monkeypatch):
"""并发异步任务必须各自创建和关闭 AsyncSession 作用域。"""
created: list[object] = []
class FakeAsyncSession:
"""记录每次作用域构造的独立异步会话替身。"""
def __init__(self, **_kwargs) -> None:
"""创建可由异步上下文管理器返回的唯一实例。"""
created.append(self)
async def __aenter__(self):
"""返回当前会话实例。"""
return self
async def __aexit__(self, *_exc) -> bool:
"""模拟正常释放且不吞掉异常。"""
return False
monkeypatch.setattr(
session_module,
"_resolve_async_engine",
lambda: (object(), True),
)
monkeypatch.setattr(session_module, "AsyncSession", FakeAsyncSession)
async def open_in_task() -> int:
"""进入一个任务私有的异步会话作用域。"""
async with session_module.async_session_scope() as session:
await asyncio.sleep(0)
return id(session)
async def run() -> list[int]:
"""并发执行两个会话作用域。"""
return await asyncio.gather(open_in_task(), open_in_task())
session_ids = asyncio.run(run())
assert len(created) == 2
assert len(set(session_ids)) == 2
def test_close_database_disposes_pooled_engines(monkeypatch):
"""
close_database 必须释放按事件循环缓存的池化引擎。