mirror of
https://github.com/jxxghp/MoviePilot.git
synced 2026-06-22 07:54:06 +08:00
Compare commits
12 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
05943287c0 | ||
|
|
94633173b1 | ||
|
|
7ab1a668cb | ||
|
|
d57deb1df1 | ||
|
|
d940373f6b | ||
|
|
ca01b8ec3f | ||
|
|
384d6a3fe1 | ||
|
|
922e8473c5 | ||
|
|
01c3451679 | ||
|
|
98e3ea4e6f | ||
|
|
0e8bcb4df6 | ||
|
|
784672af5c |
192
AGENTS.md
192
AGENTS.md
@@ -1,153 +1,103 @@
|
||||
# MoviePilot AI Agent Guide
|
||||
# AGENTS.md
|
||||
|
||||
This file defines the default behavior for AI agents working in the MoviePilot repository. Unless a deeper directory provides another `AGENTS.md`, these rules apply to the entire repo.
|
||||
This file is the primary instruction set for all AI agents and LLMs working in this repository. Local documentation takes precedence over general training data. You must follow this file and the rule documents it references.
|
||||
|
||||
## 1. Project Scope
|
||||
---
|
||||
|
||||
- This repository contains the MoviePilot backend, CLI, MCP/API, Docker assets, and AI skills.
|
||||
- The backend is based on FastAPI, with most code under `app/`.
|
||||
- Frontend source code is not in this repository. The frontend source repository is `MoviePilot-Frontend`.
|
||||
- This repository also includes the local CLI, database migrations, developer docs, tests, Docker scripts, and AI skills.
|
||||
## Task-to-Documentation Mapping
|
||||
|
||||
## 2. Working Principles
|
||||
Before executing any task, identify the domain and load the corresponding document.
|
||||
|
||||
- Read the relevant implementation, tests, and docs before changing code. Do not infer behavior from directory names alone.
|
||||
- Prefer the smallest correct change. Reuse existing functions, patterns, and naming whenever possible.
|
||||
- Do not perform unrelated large refactors, mass renames, or formatting-only cleanup.
|
||||
- Before adding a new abstraction, check whether it is actually reusable. If the logic fits well inside an existing function, class, or flow, keep it there.
|
||||
- The worktree may contain user changes. Do not revert, overwrite, or reorganize changes you do not fully understand.
|
||||
- Default to writing conclusions, validation results, and risk notes in Chinese unless the user asks otherwise.
|
||||
### Architectural Decisions
|
||||
* **Primary Reference:** `docs/rules/05-architecture.md`
|
||||
* **Required Constraints:** Respect layer boundaries and dependency flow. Do not introduce circular dependencies. Verify the correct layer for any new capability before implementing.
|
||||
|
||||
## 3. Key Directories
|
||||
### Business Logic and Design Patterns
|
||||
* **Primary Reference:** `docs/rules/04-design-patterns.md`
|
||||
* **Required Constraints:** Use the project's established Module, Chain, Event, and Oper structural patterns. Do not introduce abstractions the project has not adopted.
|
||||
|
||||
- `app/api/endpoints/`: HTTP entrypoints. Handles auth, parameters, responses, and simple CRUD.
|
||||
- `app/chain/`: Business orchestration layer for search, recognition, subscriptions, downloads, messaging flows, and similar use cases.
|
||||
- `app/modules/`: Dynamically loaded system modules. Encapsulates pluggable downloaders, media servers, message channels, and other backend capabilities.
|
||||
- `app/plugins/`: Directory where plugins are installed and managed.
|
||||
- `app/helper/`: Reusable low-level helper logic. Not a place for full business orchestration.
|
||||
- `app/core/config.py`: Environment variables, deployment parameters, and startup-level settings.
|
||||
- `app/schemas/types.py`: Shared enums and types such as `SystemConfigKey` and module categories.
|
||||
- `app/db/`: Database models, sessions, and `*_oper.py` data access wrappers.
|
||||
- `moviepilot`: Local CLI entrypoint and help text.
|
||||
- `database/versions/`: Alembic migration scripts.
|
||||
- `docs/`: CLI, MCP/API, and development workflow documentation.
|
||||
- `skills/`: AI agent skills and related scripts.
|
||||
- `tests/`: Pytest tests and a few manual test scripts.
|
||||
- `config/`, `.moviepilot.env`, and `*.db`: Local config or runtime data. Do not modify or commit them unless the user explicitly asks for it.
|
||||
### Coding Standards and Style
|
||||
* **Primary Reference:** `docs/rules/06-code-styles.md`
|
||||
* **Required Constraints:** Match the style of the surrounding file. Type annotations, Pydantic models, and async/await usage must all conform to the documented standards.
|
||||
|
||||
## 4. Layering And Access Boundaries
|
||||
### Identifiers and Naming
|
||||
* **Primary Reference:** `docs/rules/07-naming-conventions.md`
|
||||
* **Required Constraints:** All filenames, class names, function names, and constants must follow the project's taxonomy. No arbitrary abbreviations or mixed casing styles.
|
||||
|
||||
### API / Endpoint Layer
|
||||
### Comments and Documentation
|
||||
* **Primary Reference:** `docs/rules/08-comment-styles.md`
|
||||
* **Required Constraints:** All public classes and methods require Chinese docstrings. Comments must explain the *why*, not restate the code.
|
||||
* **⚠️ MANDATORY GATE:** Code that is missing proper Chinese docstrings on public interfaces is **REJECTED** at review. No exceptions.
|
||||
|
||||
- Endpoints should only handle HTTP concerns: auth, parameter parsing, response models, streaming adaptation, and simple input validation.
|
||||
- Simple list, detail, toggle, settings read/write, and pure CRUD endpoints may directly call `app/db/` or an existing `helper`.
|
||||
- If the logic coordinates multiple modules, triggers events, touches caches, or combines search, recognition, subscription, or download workflows, move it into `chain`.
|
||||
- Prefer adding new endpoints to an existing domain file. Create a new endpoint file only when introducing a new top-level resource domain.
|
||||
- After adding a new endpoint, register it in `app/api/apiv1.py`.
|
||||
### External Communication and Interfaces
|
||||
* **Primary Reference:** `docs/rules/09-external-response.md`
|
||||
* **Required Constraints:** All third-party HTTP requests must go through `RequestUtils`. Response formats must use the project's standard schemas. Error handling must follow the per-layer conventions.
|
||||
|
||||
### Chain Layer
|
||||
### Data and Persistence
|
||||
* **Primary Reference:** `docs/rules/10-data-and-persistent.md`
|
||||
* **Required Constraints:** Any database model change requires a matching Alembic migration. Runtime configuration must be managed via `SystemConfigKey` + `SystemConfigOper`. Raw string keys are forbidden.
|
||||
|
||||
- `chain` is the business orchestration layer shared by API, CLI, message interaction, agents, schedulers, and similar entrypoints.
|
||||
- `chain` is responsible for composing `module`, `helper`, `db`, events, caches, and other stable `chain` capabilities.
|
||||
- Inside `chain`, prefer calling module capabilities through `run_module()` or `async_run_module()`. Only use `ModuleManager` or similar helpers directly when you truly need to enumerate modules, inspect instances, or run health checks.
|
||||
- `chain` should focus on use cases and workflows. It should not hold low-level protocol details, HTTP request objects, or page-specific parameter assembly.
|
||||
- Before adding a new `chain`, ask whether this is a reusable business use case shared by multiple entrypoints, or a flow that coordinates multiple modules or resources. If it is just short logic for one endpoint, do not create a new `chain`.
|
||||
- `chain` may call other `chain` classes when reusing stable domain logic, but avoid introducing new circular dependencies.
|
||||
### Quality and Security
|
||||
* **Primary Reference:** `docs/rules/11-quality-and-security.md`
|
||||
* **Required Constraints:** All code changes must pass the relevant pytest tests and pylint checks. Dependency changes require a passing safety scan.
|
||||
|
||||
### Module Layer
|
||||
### Commands and Development Workflow
|
||||
* **Primary Reference:** `docs/rules/03-commands.md`
|
||||
* **Required Constraints:** Only suggest or execute commands documented in that file. Do not assume tool defaults or global flags.
|
||||
|
||||
- `module` is the pluggable capability layer discovered and loaded by `ModuleManager`.
|
||||
- Put logic in `module` when it represents a new downloader, media server, message channel, recognition backend, filtering backend, file-management backend, or any other capability that needs lifecycle management, priority, configuration switches, or independent testing.
|
||||
- New modules should follow the existing base-class contract and implement or align with `init_module()`, `init_setting()`, `get_name()`, `get_type()`, `get_subtype()`, `get_priority()`, `test()`, and `stop()`.
|
||||
- A `module` should focus on one backend or one capability implementation. It should return domain results, not HTTP responses, and should not depend on endpoint auth or FastAPI request objects.
|
||||
- `chain -> module` is the intended main direction. The repository contains a small number of historical `module -> chain` usages. Do not expand that pattern in new code. If a module needs shared business logic, prefer moving that logic up into `chain` or down into `helper`.
|
||||
- Do not add direct `module -> module` coupling for new code. Cross-module orchestration should be handled by `chain`.
|
||||
---
|
||||
|
||||
### Helper Layer
|
||||
## Agent Execution Rules
|
||||
|
||||
- `helper` is for reusable low-level support logic such as path handling, config aggregation, site index loading, protocol wrappers, rate limiting, cache helpers, and page parsing.
|
||||
- Add a new `helper` only when the logic is reused in multiple places, or when it is clearly a standalone low-level concern.
|
||||
- If logic is used only by a single `chain` or a single `module`, prefer keeping it in the original file instead of turning `helper` into a dumping ground.
|
||||
- If the code needs configuration switches, runtime loading, priorities, independent test entrypoints, or multi-implementation dispatch, it is probably a `module`, not a `helper`.
|
||||
- `helper` must not become another orchestration layer. Full business workflows still belong in `chain`.
|
||||
### Pre-Flight Check
|
||||
|
||||
### Preferred Call Directions
|
||||
Before generating any code or proposing changes, you must:
|
||||
|
||||
- Preferred direction: `endpoint/CLI/agent/command -> chain -> module/helper/db`
|
||||
- Allowed direction: `chain -> chain`, as long as the reused logic is stable and does not introduce cycles.
|
||||
- Cautious direction: `endpoint -> db/model/oper/helper`, only for simple queries, simple CRUD, or input normalization.
|
||||
- Avoid for new code: `module -> chain`, `module -> module`, `helper -> chain`, `helper -> endpoint`.
|
||||
1. Identify the task domain (architecture / business logic / coding style / naming / comments / external interfaces / data / quality).
|
||||
2. Load the corresponding document from `docs/rules/`.
|
||||
3. Explicitly verify that your proposed solution does not violate the following three mandatory constraints:
|
||||
- **Naming Conventions (07):** Are all files, classes, functions, and constants named correctly?
|
||||
- **Architecture Boundaries (05):** Is the code placed in the correct layer? Are all call directions valid?
|
||||
- **Comment Standards (08):** Do all new public classes and methods include Chinese docstrings?
|
||||
|
||||
## 5. Where New Capabilities Should Go
|
||||
### Implementation Guidelines
|
||||
|
||||
- Scenario: adding a new business workflow such as search, recognition, subscription, download orchestration, or message interaction.
|
||||
Action: prefer `app/chain/` so API, CLI, agents, and schedulers can share the same orchestration logic.
|
||||
- Scenario: adding a new downloader, media server, message channel, or other pluggable backend integration.
|
||||
Action: put it in `app/modules/`. If this introduces a new module category or subtype, also check `app/schemas/types.py` and related schemas.
|
||||
- Scenario: adding a new public HTTP API.
|
||||
Action: put it in `app/api/endpoints/`, register it in `app/api/apiv1.py`, and add auth, schemas, docs, and tests. Move complex logic into `chain`.
|
||||
- Scenario: adding a new low-level utility, parser, config reader, or protocol wrapper.
|
||||
Action: put it in `app/helper/`, but only if it is not a one-off implementation and not a full business use case.
|
||||
- Scenario: adding a deployment-level, environment-level, or startup-time config such as ports, paths, proxies, switches, keys, or third-party service addresses.
|
||||
Action: put it in `ConfigModel` or `Settings` inside `app/core/config.py`.
|
||||
- Scenario: adding a runtime business config, user-editable rule, or persistent system option.
|
||||
Action: prefer `SystemConfigKey` plus `SystemConfigOper`. Do not scatter raw string keys.
|
||||
- Scenario: a config change should automatically reload a long-lived object.
|
||||
Action: add `CONFIG_WATCH`, `on_config_changed()`, and `get_reload_name()` where appropriate on the related `chain`, `module`, `helper`, or manager class.
|
||||
- Scenario: adding a few dozen lines of private logic inside one `chain` or `module`.
|
||||
Action: prefer a private function or private method in the same file. Do not create a new `helper` by default.
|
||||
* **Pattern Adherence:** Avoid generic boilerplate. If `04-design-patterns.md` defines a project-level pattern for a scenario, you are required to use it.
|
||||
* **Documentation Standards:** Docstring style for any new function or module must match `08-comment-styles.md`.
|
||||
* **⚠️ MANDATORY GATE:** Public classes, methods, and functions without proper Chinese docstrings are **REJECTED**. No exceptions.
|
||||
* **Command Reliance:** Only suggest commands listed in `03-commands.md`. Do not rely on inferred tool defaults.
|
||||
* **Minimal Change Principle:** Prefer the smallest correct change. Do not perform unrelated refactors, mass renames, or formatting-only cleanup.
|
||||
* **Output Language:** Summaries, validation results, and risk notes default to Chinese unless the user requests otherwise.
|
||||
|
||||
## 6. Code And Comment Requirements
|
||||
### Conflict Resolution
|
||||
|
||||
- Preserve the existing code style. Do not introduce a new abstraction layer without a clear payoff.
|
||||
- The repository already uses short docstrings for many public classes and methods. For new public classes and methods, follow the local style of the surrounding file.
|
||||
- Comments and docstrings should default to Chinese. If the surrounding file is already consistently in English, match the local style.
|
||||
- Comments should explain why the code is written that way and what non-obvious constraints exist, such as edge cases, compatibility reasons, call ordering, cache or reload semantics, and external system limitations.
|
||||
- Do not write line-by-line translation comments. Do not comment obvious assignments, branches, or straightforward calls.
|
||||
- For complex notes, place the comment above the code block instead of using long end-of-line comments.
|
||||
- When changing code, update or remove stale comments so the documentation stays aligned with the implementation.
|
||||
- Do not add TODO or FIXME without context. Only keep one if it is genuinely useful and cannot be addressed as part of the current task.
|
||||
- Do not add noisy comments like "change starts here", "change ends here", or "this is important".
|
||||
If existing code appears to contradict the documentation:
|
||||
|
||||
## 7. Dependency And Environment Conventions
|
||||
1. Stop implementation immediately.
|
||||
2. Identify the specific file and line of the contradiction.
|
||||
3. Prompt the user: "The documentation in `[File]` requires Pattern A, but the current implementation uses Pattern B. Which is the current standard?"
|
||||
|
||||
- Target Python version is `3.11+`. Current CI uses Python `3.12`.
|
||||
- The dependency source file is `requirements.in`.
|
||||
- `requirements.txt` is the lock file generated by `pip-compile requirements.in`. Do not maintain it manually.
|
||||
- Install dependencies with `pip install -r requirements.txt`.
|
||||
- When adding or upgrading dependencies:
|
||||
1. Update `requirements.in`
|
||||
2. Run `pip-compile requirements.in`
|
||||
3. Run the relevant tests and security checks
|
||||
---
|
||||
|
||||
## 8. Coupled Updates
|
||||
## Coupled Update Rules
|
||||
|
||||
- When fixing a bug, prefer adding a test that reproduces it. When adding a feature, prefer the smallest useful test coverage.
|
||||
- When changing CLI behavior, also check and update `moviepilot`, `docs/cli.md`, and related tests.
|
||||
- When changing MCP or REST API behavior, exposed tools, or AI interaction behavior, also check and update `docs/mcp-api.md`, related `skills/*/SKILL.md` files or scripts, and related tests.
|
||||
- When changing development workflow, dependency management, or security-check procedures, also update `docs/development-setup.md`.
|
||||
- When changing database structure, add an Alembic migration under `database/versions/`. Do not update models without a migration.
|
||||
- When changing user-visible config, defaults, or initialization flow, also check related docs, help text, setup or init flows, and tests.
|
||||
- When adding a new skill, follow the existing `skills/<name>/SKILL.md` structure, keep the YAML front matter, and prefer script paths relative to the `SKILL.md` file.
|
||||
When modifying the following, you must also update the listed artifacts:
|
||||
|
||||
## 9. Validation Requirements
|
||||
| Changed Content | Must Also Update |
|
||||
|---|---|
|
||||
| CLI behavior | `moviepilot` entrypoint, `docs/cli.md`, related tests |
|
||||
| MCP / REST API, exposed tools | `docs/mcp-api.md`, `skills/*/SKILL.md`, related tests |
|
||||
| Dev workflow, dependency management, security checks | `docs/development-setup.md` |
|
||||
| Database model schema | New Alembic migration under `database/versions/` |
|
||||
| User-visible config or init flow | Related docs, help text, setup/init flows, tests |
|
||||
| New skill | Follow `skills/<name>/SKILL.md` structure, keep YAML front matter |
|
||||
|
||||
- Run at least the tests directly related to the change, for example `pytest tests/test_xxx.py`.
|
||||
- If the change affects common modules, startup flow, CLI, or agent runtime behavior, expand the validation scope.
|
||||
- After Python code changes, at minimum ensure the change does not introduce new error-level issues in `pylint app/`.
|
||||
- When changing CLI behavior, validate the relevant help output such as `moviepilot help` or the specific subcommand help.
|
||||
- When changing dependencies, also run `pip-compile requirements.in` and `safety check -r requirements.txt --policy-file=safety.policy.yml`.
|
||||
- If the task only changes documentation, explicitly say that tests were not run. Do not claim checks that were not executed.
|
||||
---
|
||||
|
||||
## 10. Commit And Release Conventions
|
||||
## Primary Entry Point
|
||||
|
||||
- Only create a commit when the user explicitly asks for one.
|
||||
- Prefer Conventional Commits such as `feat: ...`, `fix: ...`, and `docs: ...`.
|
||||
- This is not just stylistic. The release workflow uses Conventional Commits to categorize changelog entries.
|
||||
- Do not casually change version numbers, release settings, or Docker release flow unless the task explicitly involves them.
|
||||
For the full documentation map and cross-references, refer to:
|
||||
|
||||
## 11. Output Requirements
|
||||
**[Documentation Hub Index](./docs/rules/README.md)**
|
||||
|
||||
- Result summaries should focus on three things: what changed, how it was validated, and what risks remain.
|
||||
- Do not write vague summaries. Do not describe unexecuted checks as completed.
|
||||
- If there is compatibility impact, config migration risk, or user-data risk, call it out explicitly.
|
||||
*Last Updated: 2026-05-25*
|
||||
|
||||
@@ -32,29 +32,87 @@ class LLMTestTimeout(TimeoutError):
|
||||
def _patch_gemini_thought_signature():
|
||||
"""
|
||||
修复 langchain-google-genai 中 Gemini 2.5 思考模型的 thought_signature 兼容问题。
|
||||
langchain-google-genai 的 _is_gemini_3_or_later() 仅检查 "gemini-3",
|
||||
导致 Gemini 2.5 思考模型(如 gemini-2.5-flash、gemini-2.5-pro)在工具调用时
|
||||
缺少 thought_signature 而报错 400。
|
||||
此补丁将检查范围扩展到 Gemini 2.5 模型。
|
||||
|
||||
问题 1:_is_gemini_3_or_later() 仅检查 "gemini-3",不包含 Gemini 2.5 模型,
|
||||
导致 _parse_chat_history 的 thought_signature 强制注入逻辑被跳过。
|
||||
|
||||
问题 2:强制注入逻辑使用 first_fc_seen 标志,只给每个 model 消息中
|
||||
第一个缺少 thought_signature 的 function_call 补 dummy,后续并行
|
||||
function_call 仍缺失签名,导致 Gemini API 返回 400。
|
||||
|
||||
此补丁同时修复以上两个问题。
|
||||
"""
|
||||
try:
|
||||
import langchain_google_genai.chat_models as _cm
|
||||
|
||||
# 检查版本:需要 >= 4.0 才支持 _is_gemini_3_or_later
|
||||
try:
|
||||
from importlib.metadata import version
|
||||
_version = version("langchain-google-genai") or ""
|
||||
except Exception:
|
||||
_version = ""
|
||||
try:
|
||||
_major = int(_version.split(".")[0]) if _version else 0
|
||||
except (ValueError, TypeError):
|
||||
_major = 0
|
||||
if _major < 4:
|
||||
logger.error(
|
||||
f"langchain-google-genai 版本 {_version or '未知'} 过旧,"
|
||||
f"不支持 Gemini 2.5+ 模型的 thought_signature 处理,"
|
||||
f"请升级到 4.2.3+:pip install langchain-google-genai~=4.2.3"
|
||||
)
|
||||
return
|
||||
|
||||
# 仅在未修补时执行
|
||||
if getattr(_cm, "_thought_signature_patched", False):
|
||||
return
|
||||
|
||||
if not hasattr(_cm, "_is_gemini_3_or_later"):
|
||||
logger.error(
|
||||
"langchain-google-genai 缺少 _is_gemini_3_or_later,"
|
||||
"无法修补 thought_signature 兼容性,请检查包版本"
|
||||
)
|
||||
return
|
||||
|
||||
# 补丁 1:扩展 _is_gemini_3_or_later,使 Gemini 2.5 模型也能触发
|
||||
# _parse_chat_history 中的 thought_signature 强制注入逻辑
|
||||
def _patched_is_gemini_3_or_later(model_name: str) -> bool:
|
||||
if not model_name:
|
||||
return False
|
||||
name = model_name.lower().replace("models/", "")
|
||||
# Gemini 2.5 思考模型也需要 thought_signature 支持
|
||||
return "gemini-3" in name or "gemini-2.5" in name
|
||||
|
||||
_cm._is_gemini_3_or_later = _patched_is_gemini_3_or_later
|
||||
|
||||
# 补丁 2:修复 _parse_chat_history 中 first_fc_seen 只修复第一个
|
||||
# function_call 的问题。用 wrapper 在原函数返回后,确保所有 model
|
||||
# 消息中所有 function_call 都带有 thought_signature。
|
||||
_original_parse_chat_history = _cm._parse_chat_history
|
||||
|
||||
def _patched_parse_chat_history(*args, **kwargs):
|
||||
result = _original_parse_chat_history(*args, **kwargs)
|
||||
system_instruction, formatted_messages = result
|
||||
|
||||
# 从参数中提取 model 名称
|
||||
model = kwargs.get("model")
|
||||
if model is None and len(args) >= 4:
|
||||
model = args[3]
|
||||
|
||||
if model and _patched_is_gemini_3_or_later(model):
|
||||
dummy = _cm.DUMMY_THOUGHT_SIGNATURE
|
||||
for content_msg in formatted_messages:
|
||||
if content_msg.role == "model":
|
||||
for part in content_msg.parts or []:
|
||||
if part.function_call and not part.thought_signature:
|
||||
part.thought_signature = dummy
|
||||
|
||||
return result
|
||||
|
||||
_cm._parse_chat_history = _patched_parse_chat_history
|
||||
_cm._thought_signature_patched = True
|
||||
logger.debug(
|
||||
"已修补 langchain-google-genai thought_signature 兼容性(覆盖 Gemini 2.5 模型)"
|
||||
"已修补 langchain-google-genai thought_signature 兼容性"
|
||||
"(覆盖 Gemini 2.5 模型 + 修复并行 function_call 签名缺失)"
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f"修补 langchain-google-genai thought_signature 失败: {e}")
|
||||
|
||||
@@ -19,10 +19,8 @@ from app.schemas.types import SystemConfigKey
|
||||
class AddCustomFilterRuleInput(BaseModel):
|
||||
"""新增自定义过滤规则工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
rule_id: str = Field(
|
||||
...,
|
||||
description="Unique custom rule ID. Only letters and numbers are allowed.",
|
||||
|
||||
@@ -22,7 +22,7 @@ from app.utils.crypto import HashUtils
|
||||
|
||||
class AddDownloadInput(BaseModel):
|
||||
"""添加下载工具的输入参数模型"""
|
||||
explanation: str = Field(..., description="Clear explanation of why this tool is being used in the current context")
|
||||
explanation: Optional[str] = Field(None, description="Clear explanation of why this tool is being used in the current context")
|
||||
torrent_url: List[str] = Field(
|
||||
...,
|
||||
description="One or more torrent_url values. Supports refs from get_search_results (`hash:id`) and magnet links."
|
||||
|
||||
@@ -23,10 +23,8 @@ from app.schemas.types import SystemConfigKey
|
||||
class AddRuleGroupInput(BaseModel):
|
||||
"""新增过滤规则组工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
name: str = Field(..., description="New rule group name.")
|
||||
rule_string: str = Field(
|
||||
...,
|
||||
|
||||
@@ -14,10 +14,8 @@ from app.schemas.types import MediaType, MessageChannel
|
||||
class AddSubscribeInput(BaseModel):
|
||||
"""添加订阅工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
title: str = Field(
|
||||
...,
|
||||
description="The title of the media to subscribe to (e.g., 'The Matrix', 'Breaking Bad')",
|
||||
|
||||
@@ -38,10 +38,8 @@ class UserChoiceOptionInput(BaseModel):
|
||||
class AskUserChoiceInput(BaseModel):
|
||||
"""按钮选择工具输入。"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why the agent needs the user to choose from buttons",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why the agent needs the user to choose from buttons",)
|
||||
message: str = Field(
|
||||
...,
|
||||
description="Question or prompt shown to the user together with the buttons",
|
||||
|
||||
@@ -38,10 +38,8 @@ class BrowserAction(str, Enum):
|
||||
class BrowseWebpageInput(BaseModel):
|
||||
"""浏览器操作工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this browser action is being performed",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this browser action is being performed",)
|
||||
action: str = Field(
|
||||
...,
|
||||
description=(
|
||||
|
||||
@@ -19,10 +19,8 @@ from app.schemas.types import SystemConfigKey
|
||||
class DeleteCustomFilterRuleInput(BaseModel):
|
||||
"""删除自定义过滤规则工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
rule_id: str = Field(..., description="Custom rule ID to delete.")
|
||||
|
||||
|
||||
|
||||
@@ -12,10 +12,8 @@ from app.log import logger
|
||||
class DeleteDownloadInput(BaseModel):
|
||||
"""删除下载任务工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
hash: str = Field(
|
||||
..., description="Task hash (can be obtained from query_download_tasks tool)"
|
||||
)
|
||||
|
||||
@@ -13,10 +13,8 @@ from app.log import logger
|
||||
class DeleteDownloadHistoryInput(BaseModel):
|
||||
"""删除下载历史记录工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
history_id: int = Field(
|
||||
..., description="The ID of the download history record to delete"
|
||||
)
|
||||
|
||||
@@ -18,10 +18,8 @@ from app.schemas.types import SystemConfigKey
|
||||
class DeleteRuleGroupInput(BaseModel):
|
||||
"""删除过滤规则组工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
name: str = Field(..., description="Rule group name to delete.")
|
||||
|
||||
|
||||
|
||||
@@ -15,10 +15,8 @@ from app.schemas.types import EventType
|
||||
class DeleteSubscribeInput(BaseModel):
|
||||
"""删除订阅工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
subscribe_id: int = Field(
|
||||
...,
|
||||
description="The ID of the subscription to delete (can be obtained from query_subscribes tool)",
|
||||
|
||||
@@ -12,10 +12,8 @@ from app.log import logger
|
||||
class DeleteTransferHistoryInput(BaseModel):
|
||||
"""删除整理历史记录工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
history_id: int = Field(
|
||||
..., description="The ID of the transfer history record to delete"
|
||||
)
|
||||
|
||||
@@ -141,9 +141,7 @@ class _CommandOutput:
|
||||
class ExecuteCommandInput(BaseModel):
|
||||
"""执行 Shell 命令工具的输入参数模型。"""
|
||||
|
||||
explanation: str = Field(
|
||||
..., description="Clear explanation of why this command action is needed"
|
||||
)
|
||||
explanation: Optional[str] = Field(None, description="Clear explanation of why this command action is needed")
|
||||
action: Optional[Literal["start", "read", "wait", "write", "kill", "run"]] = Field(
|
||||
"start",
|
||||
description=(
|
||||
|
||||
@@ -14,10 +14,8 @@ from app.schemas.types import MediaType, media_type_to_agent
|
||||
class GetRecommendationsInput(BaseModel):
|
||||
"""获取推荐工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
source: Optional[str] = Field(
|
||||
"tmdb_trending",
|
||||
description="Recommendation source: "
|
||||
|
||||
@@ -20,10 +20,8 @@ from ._torrent_search_utils import (
|
||||
class GetSearchResultsInput(BaseModel):
|
||||
"""获取搜索结果工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
site: Optional[List[str]] = Field(None, description="Site name filters")
|
||||
season: Optional[List[str]] = Field(None, description="Season or episode filters")
|
||||
free_state: Optional[List[str]] = Field(None, description="Promotion state filters")
|
||||
|
||||
@@ -18,10 +18,8 @@ from app.log import logger
|
||||
class InstallPluginInput(BaseModel):
|
||||
"""安装插件工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
plugin_id: str = Field(
|
||||
...,
|
||||
description="Exact plugin ID to install. Use query_market_plugins first to find the correct plugin_id.",
|
||||
|
||||
@@ -16,7 +16,7 @@ from app.utils.string import StringUtils
|
||||
|
||||
class ListDirectoryInput(BaseModel):
|
||||
"""查询文件系统目录内容工具的输入参数模型"""
|
||||
explanation: str = Field(..., description="Clear explanation of why this tool is being used in the current context")
|
||||
explanation: Optional[str] = Field(None, description="Clear explanation of why this tool is being used in the current context")
|
||||
path: str = Field(..., description="Directory path to list contents (e.g., '/home/user/downloads' or 'C:/Downloads')")
|
||||
storage: Optional[str] = Field("local", description="Storage type (default: 'local' for local file system, can be 'smb', 'alist', etc.)")
|
||||
sort_by: Optional[str] = Field("name", description="Sort order: 'name' for alphabetical sorting, 'time' for modification time sorting (default: 'name')")
|
||||
|
||||
@@ -12,10 +12,8 @@ from app.log import logger
|
||||
class ListSlashCommandsInput(BaseModel):
|
||||
"""查询所有可用斜杠命令工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
|
||||
|
||||
class ListSlashCommandsTool(MoviePilotTool):
|
||||
|
||||
@@ -12,10 +12,8 @@ from app.log import logger
|
||||
class ModifyDownloadInput(BaseModel):
|
||||
"""修改下载任务工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
hash: str = Field(
|
||||
..., description="Task hash (can be obtained from query_download_tasks tool)"
|
||||
)
|
||||
|
||||
@@ -17,10 +17,8 @@ from app.log import logger
|
||||
class QueryBuiltinFilterRulesInput(BaseModel):
|
||||
"""查询内置过滤规则工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
rule_ids: Optional[List[str]] = Field(
|
||||
None,
|
||||
description="Optional list of built-in rule IDs to query. If omitted, return all built-in rules.",
|
||||
|
||||
@@ -18,10 +18,8 @@ from app.log import logger
|
||||
class QueryCustomFilterRulesInput(BaseModel):
|
||||
"""查询自定义过滤规则工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
rule_ids: Optional[List[str]] = Field(
|
||||
None,
|
||||
description="Optional list of custom rule IDs to query. If omitted, return all custom rules.",
|
||||
|
||||
@@ -14,10 +14,8 @@ from app.schemas.types import SystemConfigKey
|
||||
class QueryCustomIdentifiersInput(BaseModel):
|
||||
"""查询自定义识别词工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
|
||||
|
||||
class QueryCustomIdentifiersTool(MoviePilotTool):
|
||||
|
||||
@@ -12,7 +12,7 @@ from app.log import logger
|
||||
|
||||
class QueryDirectorySettingsInput(BaseModel):
|
||||
"""查询系统目录设置工具的输入参数模型"""
|
||||
explanation: str = Field(..., description="Clear explanation of why this tool is being used in the current context")
|
||||
explanation: Optional[str] = Field(None, description="Clear explanation of why this tool is being used in the current context")
|
||||
directory_type: Optional[str] = Field("all",
|
||||
description="Filter directories by type: 'download' for download directories, 'library' for media library directories, 'all' for all directories")
|
||||
storage_type: Optional[str] = Field("all",
|
||||
|
||||
@@ -15,7 +15,7 @@ from app.schemas.types import TorrentStatus, media_type_to_agent
|
||||
|
||||
class QueryDownloadTasksInput(BaseModel):
|
||||
"""查询下载工具的输入参数模型"""
|
||||
explanation: str = Field(..., description="Clear explanation of why this tool is being used in the current context")
|
||||
explanation: Optional[str] = Field(None, description="Clear explanation of why this tool is being used in the current context")
|
||||
downloader: Optional[str] = Field(None,
|
||||
description="Name of specific downloader to query (optional, if not provided queries all configured downloaders)")
|
||||
status: Optional[str] = Field("all",
|
||||
|
||||
@@ -13,7 +13,7 @@ from app.schemas.types import SystemConfigKey
|
||||
|
||||
class QueryDownloadersInput(BaseModel):
|
||||
"""查询下载器工具的输入参数模型"""
|
||||
explanation: str = Field(..., description="Clear explanation of why this tool is being used in the current context")
|
||||
explanation: Optional[str] = Field(None, description="Clear explanation of why this tool is being used in the current context")
|
||||
|
||||
|
||||
class QueryDownloadersTool(MoviePilotTool):
|
||||
|
||||
@@ -12,7 +12,7 @@ from app.log import logger
|
||||
|
||||
class QueryEpisodeScheduleInput(BaseModel):
|
||||
"""查询剧集上映时间工具的输入参数模型"""
|
||||
explanation: str = Field(..., description="Clear explanation of why this tool is being used in the current context")
|
||||
explanation: Optional[str] = Field(None, description="Clear explanation of why this tool is being used in the current context")
|
||||
tmdb_id: int = Field(..., description="TMDB ID of the TV series (can be obtained from search_media tool)")
|
||||
season: int = Field(..., description="Season number to query")
|
||||
episode_group: Optional[str] = Field(None, description="Episode group ID (optional)")
|
||||
|
||||
@@ -20,10 +20,8 @@ from app.log import logger
|
||||
class QueryInstalledPluginsInput(BaseModel):
|
||||
"""查询已安装插件工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
query: Optional[str] = Field(
|
||||
None,
|
||||
description="Optional keyword to filter installed plugins by plugin ID, name, description, or author.",
|
||||
|
||||
@@ -76,7 +76,7 @@ def _build_tv_server_result(existing_seasons: OrderedDict, total_seasons: Ordere
|
||||
|
||||
class QueryLibraryExistsInput(BaseModel):
|
||||
"""查询媒体库工具的输入参数模型"""
|
||||
explanation: str = Field(..., description="Clear explanation of why this tool is being used in the current context")
|
||||
explanation: Optional[str] = Field(None, description="Clear explanation of why this tool is being used in the current context")
|
||||
tmdb_id: Optional[int] = Field(None, description="TMDB ID (can be obtained from search_media tool). Either tmdb_id or douban_id must be provided.")
|
||||
douban_id: Optional[str] = Field(None, description="Douban ID (can be obtained from search_media tool). Either tmdb_id or douban_id must be provided.")
|
||||
media_type: Optional[str] = Field(None, description="Allowed values: movie, tv")
|
||||
|
||||
@@ -17,10 +17,8 @@ PAGE_SIZE = 20
|
||||
class QueryLibraryLatestInput(BaseModel):
|
||||
"""查询媒体服务器最近入库影片工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
server: Optional[str] = Field(
|
||||
None,
|
||||
description="Media server name (optional, if not specified queries all enabled media servers)",
|
||||
|
||||
@@ -20,10 +20,8 @@ from app.log import logger
|
||||
class QueryMarketPluginsInput(BaseModel):
|
||||
"""查询插件市场工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
query: Optional[str] = Field(
|
||||
None,
|
||||
description="Optional keyword to filter plugin market results by plugin ID, name, description, or author.",
|
||||
|
||||
@@ -17,7 +17,7 @@ SEASON_PREVIEW_LIMIT = 100
|
||||
|
||||
class QueryMediaDetailInput(BaseModel):
|
||||
"""查询媒体详情工具的输入参数模型"""
|
||||
explanation: str = Field(..., description="Clear explanation of why this tool is being used in the current context")
|
||||
explanation: Optional[str] = Field(None, description="Clear explanation of why this tool is being used in the current context")
|
||||
tmdb_id: Optional[int] = Field(None, description="TMDB ID of the media (movie or TV series, can be obtained from search_media tool)")
|
||||
douban_id: Optional[str] = Field(None, description="Douban ID of the media (alternative to tmdb_id)")
|
||||
media_type: str = Field(..., description="Allowed values: movie, tv")
|
||||
|
||||
@@ -13,10 +13,8 @@ from app.log import logger
|
||||
class QueryPersonasInput(BaseModel):
|
||||
"""查询人格工具的输入参数模型。"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
query: Optional[str] = Field(
|
||||
None,
|
||||
description=(
|
||||
|
||||
@@ -13,10 +13,8 @@ from app.log import logger
|
||||
class QueryPluginCapabilitiesInput(BaseModel):
|
||||
"""查询插件能力工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
plugin_id: Optional[str] = Field(
|
||||
None,
|
||||
description="Optional plugin ID to query capabilities for a specific plugin. "
|
||||
|
||||
@@ -14,10 +14,8 @@ from app.log import logger
|
||||
class QueryPluginConfigInput(BaseModel):
|
||||
"""查询插件配置工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
plugin_id: str = Field(
|
||||
...,
|
||||
description="The plugin ID to query. Use query_installed_plugins first to discover valid plugin IDs.",
|
||||
|
||||
@@ -18,10 +18,8 @@ from app.log import logger
|
||||
class QueryPluginDataInput(BaseModel):
|
||||
"""查询插件数据工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
plugin_id: str = Field(
|
||||
...,
|
||||
description="The plugin ID to query. Use query_installed_plugins first to discover valid plugin IDs.",
|
||||
|
||||
@@ -17,7 +17,7 @@ MAX_PAGE_SIZE = 50
|
||||
|
||||
class QueryPopularSubscribesInput(BaseModel):
|
||||
"""查询热门订阅工具的输入参数模型"""
|
||||
explanation: str = Field(..., description="Clear explanation of why this tool is being used in the current context")
|
||||
explanation: Optional[str] = Field(None, description="Clear explanation of why this tool is being used in the current context")
|
||||
media_type: str = Field(..., description="Allowed values: movie, tv")
|
||||
page: Optional[int] = Field(1, description="Page number for pagination (default: 1)")
|
||||
count: Optional[int] = Field(30, description="Number of items per page (default: 30, max: 50)")
|
||||
|
||||
@@ -18,10 +18,8 @@ from app.log import logger
|
||||
class QueryRuleGroupsInput(BaseModel):
|
||||
"""查询规则组工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
group_names: Optional[List[str]] = Field(
|
||||
None,
|
||||
description="Optional list of rule group names to query. If omitted, return all rule groups.",
|
||||
|
||||
@@ -11,7 +11,7 @@ from app.log import logger
|
||||
|
||||
class QuerySchedulersInput(BaseModel):
|
||||
"""查询定时服务工具的输入参数模型"""
|
||||
explanation: str = Field(..., description="Clear explanation of why this tool is being used in the current context")
|
||||
explanation: Optional[str] = Field(None, description="Clear explanation of why this tool is being used in the current context")
|
||||
|
||||
|
||||
class QuerySchedulersTool(MoviePilotTool):
|
||||
|
||||
@@ -23,10 +23,8 @@ def _preview_list(value, limit: int = SITE_USERDATA_DETAIL_PREVIEW_LIMIT) -> tup
|
||||
class QuerySiteUserdataInput(BaseModel):
|
||||
"""查询站点用户数据工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
site_id: int = Field(
|
||||
...,
|
||||
description="The ID of the site to query user data for (can be obtained from query_sites tool)",
|
||||
|
||||
@@ -13,10 +13,8 @@ from app.log import logger
|
||||
class QuerySitesInput(BaseModel):
|
||||
"""查询站点工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
status: Optional[str] = Field(
|
||||
"all",
|
||||
description="Filter sites by status: 'active' for enabled sites, 'inactive' for disabled sites, 'all' for all sites",
|
||||
|
||||
@@ -17,10 +17,8 @@ PAGE_SIZE = 20
|
||||
class QuerySubscribeHistoryInput(BaseModel):
|
||||
"""查询订阅历史工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
media_type: Optional[str] = Field(
|
||||
"all", description="Allowed values: movie, tv, all"
|
||||
)
|
||||
|
||||
@@ -14,7 +14,7 @@ MAX_PAGE_SIZE = 50
|
||||
|
||||
class QuerySubscribeSharesInput(BaseModel):
|
||||
"""查询订阅分享工具的输入参数模型"""
|
||||
explanation: str = Field(..., description="Clear explanation of why this tool is being used in the current context")
|
||||
explanation: Optional[str] = Field(None, description="Clear explanation of why this tool is being used in the current context")
|
||||
name: Optional[str] = Field(None, description="Filter shares by media name (partial match, optional)")
|
||||
page: Optional[int] = Field(1, description="Page number for pagination (default: 1)")
|
||||
count: Optional[int] = Field(30, description="Number of items per page (default: 30, max: 50)")
|
||||
|
||||
@@ -47,10 +47,8 @@ QUERY_SUBSCRIBE_OUTPUT_FIELDS = [
|
||||
class QuerySubscribesInput(BaseModel):
|
||||
"""查询订阅工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
status: Optional[str] = Field(
|
||||
"all",
|
||||
description="Filter subscriptions by status: 'R' for enabled subscriptions, 'S' for paused ones, 'all' for all subscriptions",
|
||||
|
||||
@@ -19,10 +19,8 @@ from app.log import logger
|
||||
class QuerySystemSettingsInput(BaseModel):
|
||||
"""查询系统设置工具的输入参数模型。"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
setting_key: Optional[str] = Field(
|
||||
None,
|
||||
description=(
|
||||
|
||||
@@ -15,7 +15,7 @@ from app.utils.jieba import cut as jieba_cut
|
||||
|
||||
class QueryTransferHistoryInput(BaseModel):
|
||||
"""查询整理历史记录工具的输入参数模型"""
|
||||
explanation: str = Field(..., description="Clear explanation of why this tool is being used in the current context")
|
||||
explanation: Optional[str] = Field(None, description="Clear explanation of why this tool is being used in the current context")
|
||||
title: Optional[str] = Field(None, description="Search by title (optional, supports partial match)")
|
||||
status: Optional[str] = Field("all",
|
||||
description="Filter by status: 'success' for successful transfers, 'failed' for failed transfers, 'all' for all records (default: 'all')")
|
||||
|
||||
@@ -13,7 +13,7 @@ from app.log import logger
|
||||
|
||||
class QueryWorkflowsInput(BaseModel):
|
||||
"""查询工作流工具的输入参数模型"""
|
||||
explanation: str = Field(..., description="Clear explanation of why this tool is being used in the current context")
|
||||
explanation: Optional[str] = Field(None, description="Clear explanation of why this tool is being used in the current context")
|
||||
state: Optional[str] = Field("all", description="Filter workflows by state: 'W' for waiting, 'R' for running, 'P' for paused, 'S' for success, 'F' for failed, 'all' for all workflows (default: 'all')")
|
||||
name: Optional[str] = Field(None, description="Filter workflows by name (partial match, optional)")
|
||||
trigger_type: Optional[str] = Field("all", description="Filter workflows by trigger type: 'timer' for scheduled, 'event' for event-triggered, 'manual' for manual, 'all' for all types (default: 'all')")
|
||||
|
||||
@@ -15,7 +15,7 @@ from app.schemas.types import media_type_to_agent
|
||||
|
||||
class RecognizeMediaInput(BaseModel):
|
||||
"""识别媒体信息工具的输入参数模型"""
|
||||
explanation: str = Field(..., description="Clear explanation of why this tool is being used in the current context")
|
||||
explanation: Optional[str] = Field(None, description="Clear explanation of why this tool is being used in the current context")
|
||||
title: Optional[str] = Field(None, description="The title of the torrent/media to recognize (required for torrent recognition)")
|
||||
subtitle: Optional[str] = Field(None, description="The subtitle or description of the torrent (optional, helps improve recognition accuracy)")
|
||||
path: Optional[str] = Field(None, description="The file path to recognize (required for file recognition, mutually exclusive with title)")
|
||||
|
||||
@@ -16,10 +16,8 @@ from app.log import logger
|
||||
class ReloadPluginInput(BaseModel):
|
||||
"""重载插件工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
plugin_id: str = Field(
|
||||
...,
|
||||
description="The plugin ID to reload so the latest saved config takes effect.",
|
||||
|
||||
@@ -11,10 +11,8 @@ from app.log import logger
|
||||
class RunSchedulerInput(BaseModel):
|
||||
"""运行定时服务工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
job_id: str = Field(
|
||||
...,
|
||||
description="The ID of the scheduled job to run (can be obtained from query_schedulers tool)",
|
||||
|
||||
@@ -14,10 +14,8 @@ from app.schemas.types import EventType, MessageChannel
|
||||
class RunSlashCommandInput(BaseModel):
|
||||
"""运行斜杠命令工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
command: str = Field(
|
||||
...,
|
||||
description="The slash command to execute, e.g. '/cookiecloud'. "
|
||||
|
||||
@@ -14,10 +14,8 @@ from app.log import logger
|
||||
class RunWorkflowInput(BaseModel):
|
||||
"""执行工作流工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
workflow_id: int = Field(
|
||||
..., description="Workflow ID (can be obtained from query_workflows tool)"
|
||||
)
|
||||
|
||||
@@ -15,10 +15,8 @@ from app.schemas import FileItem
|
||||
class ScrapeMetadataInput(BaseModel):
|
||||
"""刮削媒体元数据工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
path: str = Field(
|
||||
...,
|
||||
description="Path to the file or directory to scrape metadata for (e.g., '/path/to/file.mkv' or '/path/to/directory')",
|
||||
|
||||
@@ -13,7 +13,7 @@ from app.schemas.types import MediaType, media_type_to_agent
|
||||
|
||||
class SearchMediaInput(BaseModel):
|
||||
"""搜索媒体工具的输入参数模型"""
|
||||
explanation: str = Field(..., description="Clear explanation of why this tool is being used in the current context")
|
||||
explanation: Optional[str] = Field(None, description="Clear explanation of why this tool is being used in the current context")
|
||||
title: str = Field(..., description="The title of the media to search for (e.g., 'The Matrix', 'Breaking Bad')")
|
||||
year: Optional[str] = Field(None, description="Release year of the media (optional, helps narrow down results)")
|
||||
media_type: Optional[str] = Field(None,
|
||||
|
||||
@@ -12,7 +12,7 @@ from app.log import logger
|
||||
|
||||
class SearchPersonInput(BaseModel):
|
||||
"""搜索人物工具的输入参数模型"""
|
||||
explanation: str = Field(..., description="Clear explanation of why this tool is being used in the current context")
|
||||
explanation: Optional[str] = Field(None, description="Clear explanation of why this tool is being used in the current context")
|
||||
name: str = Field(..., description="The name of the person to search for (e.g., 'Tom Hanks', '周杰伦')")
|
||||
|
||||
|
||||
|
||||
@@ -14,7 +14,7 @@ from app.log import logger
|
||||
|
||||
class SearchPersonCreditsInput(BaseModel):
|
||||
"""搜索演员参演作品工具的输入参数模型"""
|
||||
explanation: str = Field(..., description="Clear explanation of why this tool is being used in the current context")
|
||||
explanation: Optional[str] = Field(None, description="Clear explanation of why this tool is being used in the current context")
|
||||
person_id: int = Field(..., description="The ID of the person/actor to search for credits (e.g., 31 for Tom Hanks in TMDB)")
|
||||
source: str = Field(..., description="The data source: 'tmdb' for TheMovieDB, 'douban' for Douban, 'bangumi' for Bangumi")
|
||||
page: Optional[int] = Field(1, description="Page number for pagination (default: 1)")
|
||||
|
||||
@@ -14,7 +14,7 @@ from app.schemas.types import media_type_to_agent
|
||||
|
||||
class SearchSubscribeInput(BaseModel):
|
||||
"""搜索订阅缺失剧集工具的输入参数模型"""
|
||||
explanation: str = Field(..., description="Clear explanation of why this tool is being used in the current context")
|
||||
explanation: Optional[str] = Field(None, description="Clear explanation of why this tool is being used in the current context")
|
||||
subscribe_id: int = Field(..., description="The ID of the subscription to search for missing episodes (can be obtained from query_subscribes tool)")
|
||||
manual: Optional[bool] = Field(False, description="Whether this is a manual search (default: False)")
|
||||
filter_groups: Optional[List[str]] = Field(None,
|
||||
|
||||
@@ -19,7 +19,7 @@ from ._torrent_search_utils import (
|
||||
|
||||
class SearchTorrentsInput(BaseModel):
|
||||
"""搜索种子工具的输入参数模型"""
|
||||
explanation: str = Field(..., description="Clear explanation of why this tool is being used in the current context")
|
||||
explanation: Optional[str] = Field(None, description="Clear explanation of why this tool is being used in the current context")
|
||||
tmdb_id: Optional[int] = Field(None, description="TMDB ID (can be obtained from search_media tool). Either tmdb_id or douban_id must be provided.")
|
||||
douban_id: Optional[str] = Field(None, description="Douban ID (can be obtained from search_media tool). Either tmdb_id or douban_id must be provided.")
|
||||
media_type: Optional[str] = Field(None, description="Allowed values: movie, tv")
|
||||
|
||||
@@ -19,10 +19,8 @@ SEARCH_TIMEOUT = 20
|
||||
class SearchWebInput(BaseModel):
|
||||
"""搜索网络内容工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
query: str = Field(
|
||||
..., description="The search query string to search for on the web"
|
||||
)
|
||||
|
||||
@@ -15,10 +15,8 @@ from app.schemas.types import MessageChannel
|
||||
class SendLocalFileInput(BaseModel):
|
||||
"""发送本地附件工具输入。"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why sending this local file helps the user",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why sending this local file helps the user",)
|
||||
file_path: str = Field(
|
||||
...,
|
||||
description="Absolute path to the local image or file to send to the user",
|
||||
|
||||
@@ -11,10 +11,8 @@ from app.log import logger
|
||||
class SendMessageInput(BaseModel):
|
||||
"""发送消息工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
message: Optional[str] = Field(
|
||||
None,
|
||||
description="The message content to send to the user (should be clear and informative)",
|
||||
|
||||
@@ -15,10 +15,8 @@ from app.schemas import Notification, NotificationType
|
||||
class SendVoiceMessageInput(BaseModel):
|
||||
"""发送语音消息工具输入。"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why a voice reply is the best fit in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why a voice reply is the best fit in the current context",)
|
||||
message: str = Field(
|
||||
...,
|
||||
description="The spoken content to send back to the user",
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
"""切换当前激活人格工具。"""
|
||||
|
||||
import json
|
||||
from typing import Type
|
||||
from typing import Type, Optional
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
@@ -13,10 +13,8 @@ from app.log import logger
|
||||
class SwitchPersonaInput(BaseModel):
|
||||
"""切换人格工具的输入参数模型。"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
persona_id: str = Field(
|
||||
...,
|
||||
description=(
|
||||
|
||||
@@ -12,7 +12,7 @@ from app.log import logger
|
||||
|
||||
class TestSiteInput(BaseModel):
|
||||
"""测试站点连通性工具的输入参数模型"""
|
||||
explanation: str = Field(..., description="Clear explanation of why this tool is being used in the current context")
|
||||
explanation: Optional[str] = Field(None, description="Clear explanation of why this tool is being used in the current context")
|
||||
site_identifier: int = Field(..., description="Site ID to test (can be obtained from query_sites tool)")
|
||||
|
||||
|
||||
|
||||
@@ -13,10 +13,8 @@ from app.schemas import FileItem, MediaType
|
||||
class TransferFileInput(BaseModel):
|
||||
"""整理文件或目录工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
file_path: str = Field(
|
||||
...,
|
||||
description="Path to the file or directory to transfer (e.g., '/path/to/file.mkv' or '/path/to/directory')",
|
||||
|
||||
@@ -17,10 +17,8 @@ from app.log import logger
|
||||
class UninstallPluginInput(BaseModel):
|
||||
"""卸载插件工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
plugin_id: str = Field(
|
||||
...,
|
||||
description="Exact plugin ID to uninstall. Use query_installed_plugins first to find the correct plugin_id.",
|
||||
|
||||
@@ -22,10 +22,8 @@ from app.schemas.types import SystemConfigKey
|
||||
class UpdateCustomFilterRuleInput(BaseModel):
|
||||
"""更新自定义过滤规则工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
current_rule_id: str = Field(
|
||||
..., description="Existing custom rule ID to update."
|
||||
)
|
||||
|
||||
@@ -14,10 +14,8 @@ from app.schemas.types import SystemConfigKey
|
||||
class UpdateCustomIdentifiersInput(BaseModel):
|
||||
"""更新自定义识别词工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
identifiers: List[str] = Field(
|
||||
...,
|
||||
description=(
|
||||
|
||||
@@ -13,10 +13,8 @@ from app.log import logger
|
||||
class UpdatePersonaDefinitionInput(BaseModel):
|
||||
"""更新人格定义工具的输入参数模型。"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
persona_id: str = Field(
|
||||
...,
|
||||
description=(
|
||||
|
||||
@@ -14,10 +14,8 @@ from app.log import logger
|
||||
class UpdatePluginConfigInput(BaseModel):
|
||||
"""修改插件配置工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
plugin_id: str = Field(
|
||||
...,
|
||||
description="The plugin ID to update. Use query_plugin_config first to inspect the current config.",
|
||||
|
||||
@@ -24,10 +24,8 @@ from app.schemas.types import SystemConfigKey
|
||||
class UpdateRuleGroupInput(BaseModel):
|
||||
"""更新过滤规则组工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
current_name: str = Field(..., description="Existing rule group name to update.")
|
||||
new_name: Optional[str] = Field(
|
||||
None,
|
||||
|
||||
@@ -17,10 +17,8 @@ from app.utils.string import StringUtils
|
||||
class UpdateSiteInput(BaseModel):
|
||||
"""更新站点工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
site_id: int = Field(
|
||||
...,
|
||||
description="The ID of the site to update (can be obtained from query_sites tool)",
|
||||
|
||||
@@ -13,10 +13,8 @@ from app.log import logger
|
||||
class UpdateSiteCookieInput(BaseModel):
|
||||
"""更新站点Cookie和UA工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
site_identifier: int = Field(
|
||||
...,
|
||||
description="Site ID to update Cookie and User-Agent for (can be obtained from query_sites tool)",
|
||||
|
||||
@@ -16,10 +16,8 @@ from app.schemas.types import EventType
|
||||
class UpdateSubscribeInput(BaseModel):
|
||||
"""更新订阅工具的输入参数模型"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
subscribe_id: int = Field(
|
||||
...,
|
||||
description="The ID of the subscription to update (can be obtained from query_subscribes tool)",
|
||||
|
||||
@@ -25,10 +25,8 @@ SettingValue = Optional[Union[list, dict, bool, int, float, str]]
|
||||
class UpdateSystemSettingsInput(BaseModel):
|
||||
"""更新系统设置工具的输入参数模型。"""
|
||||
|
||||
explanation: str = Field(
|
||||
...,
|
||||
description="Clear explanation of why this tool is being used in the current context",
|
||||
)
|
||||
explanation: Optional[str] = Field(None,
|
||||
description="Clear explanation of why this tool is being used in the current context",)
|
||||
setting_key: str = Field(
|
||||
...,
|
||||
description=(
|
||||
|
||||
@@ -35,6 +35,7 @@ from app.helper.progress import ProgressHelper
|
||||
from app.helper.rule import RuleHelper
|
||||
from app.helper.subscribe import SubscribeHelper
|
||||
from app.helper.system import SystemHelper
|
||||
from app.helper.usage import UsageHelper
|
||||
from app.log import logger
|
||||
from app.scheduler import Scheduler
|
||||
from app.schemas import ConfigChangeEventData
|
||||
@@ -360,8 +361,11 @@ async def fetch_image(
|
||||
|
||||
fetch_url = SecurityUtils.strip_url_signature(url)
|
||||
# 验证URL安全性
|
||||
if not SecurityUtils.is_safe_url(
|
||||
url, allowed_domains, block_private=True
|
||||
if not await SecurityUtils.is_safe_url_async(
|
||||
url,
|
||||
allowed_domains,
|
||||
block_private=True,
|
||||
allowed_private_ranges=settings.IMAGE_PROXY_ALLOWED_PRIVATE_RANGES,
|
||||
) and not (fetch_url := SecurityUtils.verify_signed_url(url)):
|
||||
logger.warn(f"Blocked unsafe image URL: {url}")
|
||||
return None
|
||||
@@ -517,6 +521,14 @@ async def get_env_setting(_: User = Depends(get_current_active_user_async)):
|
||||
return schemas.Response(success=True, data=info)
|
||||
|
||||
|
||||
@router.get("/usage/statistic", summary="查询安装版本统计报表", response_model=schemas.Response)
|
||||
async def usage_statistic(_: User = Depends(get_current_active_user_async)):
|
||||
"""
|
||||
查询安装版本统计报表
|
||||
"""
|
||||
return schemas.Response(success=True, data=await UsageHelper().async_get_statistic())
|
||||
|
||||
|
||||
@router.post("/env", summary="更新系统配置", response_model=schemas.Response)
|
||||
async def set_env_setting(
|
||||
env: dict, _: User = Depends(get_current_active_superuser_async)
|
||||
|
||||
@@ -437,6 +437,8 @@ class ConfigModel(BaseModel):
|
||||
)
|
||||
# 插件安装数据共享
|
||||
PLUGIN_STATISTIC_SHARE: bool = True
|
||||
# 安装版本统计上报
|
||||
USAGE_STATISTIC_SHARE: bool = True
|
||||
# 是否开启插件热加载
|
||||
PLUGIN_AUTO_RELOAD: bool = False
|
||||
# 本地插件仓库目录,多个地址使用,分隔
|
||||
@@ -510,6 +512,8 @@ class ConfigModel(BaseModel):
|
||||
"qpic.cn",
|
||||
]
|
||||
)
|
||||
# 图片代理允许访问的非公网 IP/CIDR,默认不放行任何非公网解析结果
|
||||
IMAGE_PROXY_ALLOWED_PRIVATE_RANGES: list = Field(default=[])
|
||||
# 允许的图片文件后缀格式
|
||||
SECURITY_IMAGE_SUFFIXES: list = Field(
|
||||
default=[".jpg", ".jpeg", ".png", ".webp", ".gif", ".svg", ".avif"]
|
||||
|
||||
@@ -49,7 +49,7 @@ class MetaVideo(MetaBase):
|
||||
_part_re = r"(^PART[0-9ABI]{0,2}$|^CD[0-9]{0,2}$|^DVD[0-9]{0,2}$|^DISK[0-9]{0,2}$|^DISC[0-9]{0,2}$)"
|
||||
_roman_numerals = r"^(?=[MDCLXVI])M*(C[MD]|D?C{0,3})(X[CL]|L?X{0,3})(I[XV]|V?I{0,3})$"
|
||||
_source_re = r"^BLURAY$|^HDTV$|^UHDTV$|^HDDVD$|^WEBRIP$|^DVDRIP$|^BDRIP$|^BLU$|^WEB$|^BD$|^HDRip$|^REMUX$|^UHD$"
|
||||
_effect_re = r"^SDR$|^HDR\d*$|^DOLBY$|^DOVI$|^DV$|^3D$|^REPACK$|^HLG$|^HDR10(\+|Plus)$|^EDR$|^HQ$"
|
||||
_effect_re = r"^SDR$|^HDR\d*$|^DOLBY$|^DOVI$|^DV$|^3D$|^REPACK$|^HLG$|^HDR10(\+|Plus)$|^HDR10P$|^VIVID$|^EDR$|^HQ$"
|
||||
_resources_type_re = r"%s|%s" % (_source_re, _effect_re)
|
||||
_name_no_begin_re = r"^[\[【].+?[\]】]"
|
||||
_name_no_chinese_re = r".*版|.*字幕"
|
||||
|
||||
119
app/helper/usage.py
Normal file
119
app/helper/usage.py
Normal file
@@ -0,0 +1,119 @@
|
||||
import platform
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict
|
||||
|
||||
from app.core.config import settings
|
||||
from app.log import logger
|
||||
from app.utils.http import AsyncRequestUtils, RequestUtils
|
||||
from app.utils.singleton import WeakSingleton
|
||||
from app.utils.system import SystemUtils
|
||||
from version import APP_VERSION, FRONTEND_VERSION
|
||||
|
||||
|
||||
class UsageHelper(metaclass=WeakSingleton):
|
||||
"""
|
||||
安装版本统计上报
|
||||
"""
|
||||
|
||||
_usage_report = f"{settings.MP_SERVER_HOST}/usage/report"
|
||||
_usage_statistic = f"{settings.MP_SERVER_HOST}/usage/statistic"
|
||||
|
||||
@staticmethod
|
||||
def get_frontend_version() -> str:
|
||||
"""
|
||||
获取当前前端版本。
|
||||
"""
|
||||
if SystemUtils.is_frozen() and SystemUtils.is_windows():
|
||||
version_file = settings.CONFIG_PATH.parent / "nginx" / "html" / "version.txt"
|
||||
else:
|
||||
version_file = Path(settings.FRONTEND_PATH) / "version.txt"
|
||||
if version_file.exists():
|
||||
try:
|
||||
with open(version_file, "r") as file:
|
||||
version = str(file.read()).strip()
|
||||
return version or FRONTEND_VERSION
|
||||
except Exception as err:
|
||||
logger.debug(f"加载版本文件 {version_file} 出错:{str(err)}")
|
||||
return FRONTEND_VERSION
|
||||
|
||||
@staticmethod
|
||||
def build_payload() -> Dict[str, Any]:
|
||||
"""
|
||||
构建安装版本统计上报载荷。
|
||||
"""
|
||||
return {
|
||||
"user_uid": SystemUtils.generate_user_unique_id(),
|
||||
"backend_version": APP_VERSION,
|
||||
"frontend_version": UsageHelper.get_frontend_version(),
|
||||
"version_flag": settings.VERSION_FLAG,
|
||||
"platform": f"{platform.system()} {platform.release()}".strip(),
|
||||
"arch": SystemUtils.cpu_arch(),
|
||||
}
|
||||
|
||||
def report(self) -> bool:
|
||||
"""
|
||||
上报当前安装实例的版本统计。
|
||||
"""
|
||||
if not settings.USAGE_STATISTIC_SHARE:
|
||||
return False
|
||||
payload = self.build_payload()
|
||||
if not payload.get("user_uid"):
|
||||
return False
|
||||
try:
|
||||
res = RequestUtils(
|
||||
proxies=settings.PROXY,
|
||||
content_type="application/json",
|
||||
timeout=5,
|
||||
).post(self._usage_report, json=payload)
|
||||
return bool(res is not None and res.status_code == 200)
|
||||
except Exception as err:
|
||||
logger.debug(f"上报安装版本统计失败:{str(err)}")
|
||||
return False
|
||||
|
||||
async def async_report(self) -> bool:
|
||||
"""
|
||||
异步上报当前安装实例的版本统计。
|
||||
"""
|
||||
if not settings.USAGE_STATISTIC_SHARE:
|
||||
return False
|
||||
payload = self.build_payload()
|
||||
if not payload.get("user_uid"):
|
||||
return False
|
||||
try:
|
||||
res = await AsyncRequestUtils(
|
||||
proxies=settings.PROXY,
|
||||
content_type="application/json",
|
||||
timeout=5,
|
||||
).post(self._usage_report, json=payload)
|
||||
return bool(res is not None and res.status_code == 200)
|
||||
except Exception as err:
|
||||
logger.debug(f"异步上报安装版本统计失败:{str(err)}")
|
||||
return False
|
||||
|
||||
def get_statistic(self) -> Dict[str, Any]:
|
||||
"""
|
||||
获取安装版本统计报表。
|
||||
"""
|
||||
if not settings.USAGE_STATISTIC_SHARE:
|
||||
return {}
|
||||
try:
|
||||
res = RequestUtils(proxies=settings.PROXY, timeout=10).get_res(self._usage_statistic)
|
||||
if res is not None and res.status_code == 200:
|
||||
return res.json()
|
||||
except Exception as err:
|
||||
logger.debug(f"获取安装版本统计报表失败:{str(err)}")
|
||||
return {}
|
||||
|
||||
async def async_get_statistic(self) -> Dict[str, Any]:
|
||||
"""
|
||||
异步获取安装版本统计报表。
|
||||
"""
|
||||
if not settings.USAGE_STATISTIC_SHARE:
|
||||
return {}
|
||||
try:
|
||||
res = await AsyncRequestUtils(proxies=settings.PROXY, timeout=10).get_res(self._usage_statistic)
|
||||
if res is not None and res.status_code == 200:
|
||||
return res.json()
|
||||
except Exception as err:
|
||||
logger.debug(f"异步获取安装版本统计报表失败:{str(err)}")
|
||||
return {}
|
||||
@@ -36,6 +36,7 @@ from app.db.systemconfig_oper import SystemConfigOper
|
||||
from app.helper.image import WallpaperHelper
|
||||
from app.helper.message import MessageHelper
|
||||
from app.helper.sites import SitesHelper # noqa
|
||||
from app.helper.usage import UsageHelper
|
||||
from app.log import logger
|
||||
from app.schemas import Notification, NotificationType, Workflow
|
||||
from app.schemas.types import EventType, SystemConfigKey
|
||||
@@ -264,6 +265,7 @@ class Scheduler(ConfigReloadMixin, metaclass=SingletonClass):
|
||||
"DATA_CLEANUP_DOWNLOAD_HISTORY_DAYS",
|
||||
"DATA_CLEANUP_SITE_USERDATA_DAYS",
|
||||
"DATA_CLEANUP_TRANSFER_HISTORY_DAYS",
|
||||
"USAGE_STATISTIC_SHARE",
|
||||
}
|
||||
|
||||
def __init__(self):
|
||||
@@ -401,6 +403,11 @@ class Scheduler(ConfigReloadMixin, metaclass=SingletonClass):
|
||||
"func": self.agent_heartbeat,
|
||||
"running": False,
|
||||
},
|
||||
"usage_report": {
|
||||
"name": "安装版本统计上报",
|
||||
"func": UsageHelper().report,
|
||||
"running": False,
|
||||
},
|
||||
}
|
||||
|
||||
# 创建定时服务
|
||||
@@ -644,6 +651,17 @@ class Scheduler(ConfigReloadMixin, metaclass=SingletonClass):
|
||||
kwargs={"job_id": "agent_heartbeat"},
|
||||
)
|
||||
|
||||
# 安装版本统计上报
|
||||
if settings.USAGE_STATISTIC_SHARE:
|
||||
self._scheduler.add_job(
|
||||
self.start,
|
||||
"interval",
|
||||
id="usage_report",
|
||||
name="安装版本统计上报",
|
||||
hours=24,
|
||||
kwargs={"job_id": "usage_report"},
|
||||
)
|
||||
|
||||
# 初始化工作流服务
|
||||
self.init_workflow_jobs()
|
||||
|
||||
|
||||
@@ -6,6 +6,7 @@ from fastapi import FastAPI
|
||||
from app.chain.system import SystemChain
|
||||
from app.core.config import global_vars
|
||||
from app.helper.system import SystemHelper
|
||||
from app.helper.usage import UsageHelper
|
||||
from app.startup.command_initializer import init_command, stop_command, restart_command
|
||||
from app.startup.modules_initializer import init_modules, stop_modules
|
||||
from app.startup.monitor_initializer import stop_monitor, init_monitor
|
||||
@@ -29,6 +30,8 @@ async def init_extra():
|
||||
SystemHelper().set_system_modified()
|
||||
# 重启完成
|
||||
SystemChain().restart_finish()
|
||||
# 上报当前安装版本
|
||||
await UsageHelper().async_report()
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
|
||||
@@ -1,21 +1,61 @@
|
||||
import asyncio
|
||||
import hmac
|
||||
import ipaddress
|
||||
import socket
|
||||
import time
|
||||
import threading
|
||||
from hashlib import sha256
|
||||
from pathlib import Path
|
||||
from typing import List, Optional, Set, Union
|
||||
from typing import Dict, Iterable, List, Optional, Set, Union
|
||||
from urllib.parse import parse_qsl, quote, urlencode, urlparse, urlunparse
|
||||
|
||||
from anyio import Path as AsyncPath
|
||||
from cachetools import TTLCache
|
||||
|
||||
from app.core.config import settings
|
||||
from app.log import logger
|
||||
|
||||
|
||||
# DNS 解析结果缓存。
|
||||
# 正向缓存 TTL 选择 120s,短于常见 CDN / fake-ip 的 DNS TTL,避免长期持有失效 IP;
|
||||
# 负向缓存 TTL 选择 15s,避免临时解析失败把目标长时间拉黑。
|
||||
_DNS_CACHE_MAXSIZE = 1024
|
||||
_DNS_CACHE_TTL_POSITIVE = 120
|
||||
_DNS_CACHE_TTL_NEGATIVE = 15
|
||||
_dns_positive_cache: "TTLCache[str, List[ipaddress._BaseAddress]]" = TTLCache(
|
||||
maxsize=_DNS_CACHE_MAXSIZE, ttl=_DNS_CACHE_TTL_POSITIVE
|
||||
)
|
||||
_dns_negative_cache: "TTLCache[str, bool]" = TTLCache(
|
||||
maxsize=_DNS_CACHE_MAXSIZE, ttl=_DNS_CACHE_TTL_NEGATIVE
|
||||
)
|
||||
# 同步路径下保护 TTLCache 读写:`cachetools.TTLCache` 本身非线程安全。
|
||||
# 锁只覆盖缓存读写,不包 `getaddrinfo`,避免把 DNS 查询本身串行化。
|
||||
_dns_cache_lock = threading.Lock()
|
||||
# 同 hostname 的并发异步解析去重:同一 hostname 首次未命中时建立锁,
|
||||
# 后续并发请求 await 同一把锁,避免对同一目标重复发起 `getaddrinfo`。
|
||||
_dns_inflight_locks: Dict[str, asyncio.Lock] = {}
|
||||
_dns_inflight_meta_lock = threading.Lock()
|
||||
|
||||
|
||||
def _resolve_addrinfo_to_ips(
|
||||
address_infos: Iterable,
|
||||
) -> Optional[List[ipaddress._BaseAddress]]:
|
||||
"""
|
||||
将 `socket.getaddrinfo` 返回的结果归一化为 IP 列表。
|
||||
|
||||
任一条目无法解析为 IP 即视为异常情况,整体返回 None 让上层按"不安全目标"
|
||||
处理,避免出现"部分 IP 漏校验"的情况。
|
||||
"""
|
||||
addresses: List[ipaddress._BaseAddress] = []
|
||||
for address_info in address_infos:
|
||||
try:
|
||||
addresses.append(ipaddress.ip_address(address_info[4][0]))
|
||||
except ValueError:
|
||||
return None
|
||||
return addresses or None
|
||||
|
||||
|
||||
class SecurityUtils:
|
||||
_SIGNED_URL_PURPOSE = "image-proxy"
|
||||
_SIGNED_URL_EXPIRE_SECONDS = 86400
|
||||
|
||||
@staticmethod
|
||||
def is_safe_path(base_path: Path, user_path: Path,
|
||||
@@ -79,57 +119,282 @@ class SecurityUtils:
|
||||
logger.debug(f"Error occurred while validating paths: {e}")
|
||||
return False
|
||||
|
||||
@staticmethod
|
||||
def _literal_ip(hostname: str) -> Optional[ipaddress._BaseAddress]:
|
||||
"""
|
||||
若 hostname 是字面量 IP(含 IPv6 的 `[::1]` 形式)则返回 IP 对象,否则 None。
|
||||
"""
|
||||
if not hostname:
|
||||
return None
|
||||
candidate = hostname
|
||||
if candidate.startswith("[") and candidate.endswith("]"):
|
||||
candidate = candidate[1:-1]
|
||||
try:
|
||||
return ipaddress.ip_address(candidate)
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
@staticmethod
|
||||
def _cache_lookup(hostname: str) -> tuple[bool, Optional[List[ipaddress._BaseAddress]]]:
|
||||
"""
|
||||
在 TTL 缓存中查找 hostname,返回 (是否命中, 命中值)。
|
||||
|
||||
命中值为 `None` 表示命中负向缓存(先前解析失败)。
|
||||
"""
|
||||
with _dns_cache_lock:
|
||||
cached = _dns_positive_cache.get(hostname)
|
||||
if cached is not None:
|
||||
return True, cached
|
||||
if hostname in _dns_negative_cache:
|
||||
return True, None
|
||||
return False, None
|
||||
|
||||
@staticmethod
|
||||
def _cache_store(
|
||||
hostname: str, addresses: Optional[List[ipaddress._BaseAddress]]
|
||||
) -> None:
|
||||
"""
|
||||
将解析结果写入对应的正向/负向缓存。
|
||||
"""
|
||||
with _dns_cache_lock:
|
||||
if addresses is None:
|
||||
_dns_negative_cache[hostname] = True
|
||||
else:
|
||||
_dns_positive_cache[hostname] = addresses
|
||||
|
||||
@staticmethod
|
||||
def _hostname_addresses(hostname: str) -> Optional[List[ipaddress._BaseAddress]]:
|
||||
"""
|
||||
同步解析主机名并返回全部 IP 地址,结果走 TTL 缓存。
|
||||
|
||||
字面量 IP 直接返回自身;DNS 解析失败或结果异常时返回 None,由上层按
|
||||
不安全目标处理。async 调用方应使用 `_hostname_addresses_async`。
|
||||
"""
|
||||
if not hostname:
|
||||
return None
|
||||
literal = SecurityUtils._literal_ip(hostname)
|
||||
if literal is not None:
|
||||
return [literal]
|
||||
|
||||
hit, value = SecurityUtils._cache_lookup(hostname)
|
||||
if hit:
|
||||
return value
|
||||
|
||||
try:
|
||||
address_infos = socket.getaddrinfo(hostname, None, type=socket.SOCK_STREAM)
|
||||
except socket.gaierror:
|
||||
SecurityUtils._cache_store(hostname, None)
|
||||
return None
|
||||
addresses = _resolve_addrinfo_to_ips(address_infos)
|
||||
SecurityUtils._cache_store(hostname, addresses)
|
||||
return addresses
|
||||
|
||||
@staticmethod
|
||||
def _get_inflight_lock(hostname: str) -> asyncio.Lock:
|
||||
"""
|
||||
取得 hostname 对应的 in-flight 锁,不存在则按需创建。
|
||||
|
||||
用 `threading.Lock` 保护字典写入,避免多个事件循环线程并发创建出多把锁
|
||||
破坏去重语义;锁本身是 `asyncio.Lock`,归属当前事件循环。
|
||||
"""
|
||||
with _dns_inflight_meta_lock:
|
||||
lock = _dns_inflight_locks.get(hostname)
|
||||
if lock is None:
|
||||
lock = asyncio.Lock()
|
||||
_dns_inflight_locks[hostname] = lock
|
||||
return lock
|
||||
|
||||
@staticmethod
|
||||
def _release_inflight_lock(hostname: str, lock: asyncio.Lock) -> None:
|
||||
"""
|
||||
请求结束后清理 in-flight 锁,避免长期持有大量已闲置的 `asyncio.Lock`。
|
||||
|
||||
仅当字典中登记的仍是当前 lock,且 `lock.locked()` 为 False 时才删除。
|
||||
`asyncio.Lock` 公平 FIFO:持有者释放后若仍有等待者,锁会立刻被下一个
|
||||
等待者接走、`locked()` 重新变为 True,因此该守卫可同时排除"仍有持有者"
|
||||
与"刚被等待者接走"两种情况,避免误删后续协程仍在使用的字典条目。
|
||||
"""
|
||||
with _dns_inflight_meta_lock:
|
||||
current = _dns_inflight_locks.get(hostname)
|
||||
if current is lock and not lock.locked():
|
||||
_dns_inflight_locks.pop(hostname, None)
|
||||
|
||||
@staticmethod
|
||||
async def _hostname_addresses_async(
|
||||
hostname: str,
|
||||
) -> Optional[List[ipaddress._BaseAddress]]:
|
||||
"""
|
||||
异步解析主机名并返回全部 IP 地址,与同步版本共用同一份 TTL 缓存。
|
||||
|
||||
通过事件循环的默认线程池执行 `getaddrinfo`,不阻塞 asyncio 事件循环;
|
||||
同 hostname 的并发未命中请求通过 in-flight 锁去重,只发起一次 DNS 查询。
|
||||
"""
|
||||
if not hostname:
|
||||
return None
|
||||
literal = SecurityUtils._literal_ip(hostname)
|
||||
if literal is not None:
|
||||
return [literal]
|
||||
|
||||
hit, value = SecurityUtils._cache_lookup(hostname)
|
||||
if hit:
|
||||
return value
|
||||
|
||||
lock = SecurityUtils._get_inflight_lock(hostname)
|
||||
try:
|
||||
async with lock:
|
||||
# 等到锁后再查一次缓存,前一个持锁者可能已经回填结果
|
||||
hit, value = SecurityUtils._cache_lookup(hostname)
|
||||
if hit:
|
||||
return value
|
||||
|
||||
loop = asyncio.get_running_loop()
|
||||
try:
|
||||
address_infos = await loop.getaddrinfo(
|
||||
hostname, None, type=socket.SOCK_STREAM
|
||||
)
|
||||
except socket.gaierror:
|
||||
SecurityUtils._cache_store(hostname, None)
|
||||
return None
|
||||
addresses = _resolve_addrinfo_to_ips(address_infos)
|
||||
SecurityUtils._cache_store(hostname, addresses)
|
||||
return addresses
|
||||
finally:
|
||||
# 必须在 `async with` 释放锁之后再清理字典:`_release_inflight_lock`
|
||||
# 以 `not lock.locked()` 为清理守卫,持锁状态下调用会跳过 pop。
|
||||
SecurityUtils._release_inflight_lock(hostname, lock)
|
||||
|
||||
@staticmethod
|
||||
def _addresses_all_global(
|
||||
addresses: Optional[List[ipaddress._BaseAddress]],
|
||||
) -> bool:
|
||||
"""
|
||||
判断解析结果是否全部为公网地址(空列表/None 视为非公网)。
|
||||
"""
|
||||
if not addresses:
|
||||
return False
|
||||
return all(address.is_global for address in addresses)
|
||||
|
||||
@staticmethod
|
||||
def _is_global_hostname(hostname: str) -> bool:
|
||||
"""
|
||||
判断主机名解析结果是否全部为公网地址。
|
||||
判断主机名解析结果是否全部为公网地址(同步版本)。
|
||||
|
||||
图片代理会访问用户可控的 URL,这里必须在 allowlist 命中前后都排除
|
||||
私有、回环、链路本地、保留地址等非公网目标,避免通过 DNS 或字面量 IP
|
||||
绕过域名白名单访问内网服务。
|
||||
"""
|
||||
if not hostname:
|
||||
return False
|
||||
try:
|
||||
return ipaddress.ip_address(hostname).is_global
|
||||
except ValueError:
|
||||
pass
|
||||
|
||||
try:
|
||||
address_infos = socket.getaddrinfo(hostname, None, type=socket.SOCK_STREAM)
|
||||
except socket.gaierror:
|
||||
return False
|
||||
|
||||
if not address_infos:
|
||||
return False
|
||||
|
||||
for address_info in address_infos:
|
||||
try:
|
||||
address = ipaddress.ip_address(address_info[4][0])
|
||||
except ValueError:
|
||||
return False
|
||||
if not address.is_global:
|
||||
return False
|
||||
return True
|
||||
return SecurityUtils._addresses_all_global(
|
||||
SecurityUtils._hostname_addresses(hostname)
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _url_signature_payload(url: str, expires_at: int, purpose: str) -> bytes:
|
||||
async def _is_global_hostname_async(hostname: str) -> bool:
|
||||
"""
|
||||
判断主机名解析结果是否全部为公网地址(异步版本)。语义与 `_is_global_hostname` 一致。
|
||||
"""
|
||||
return SecurityUtils._addresses_all_global(
|
||||
await SecurityUtils._hostname_addresses_async(hostname)
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _parse_ip_networks(ranges: Optional[Iterable[str]]) -> List[ipaddress._BaseNetwork]:
|
||||
"""
|
||||
解析用户配置的 IP/CIDR 网段。
|
||||
|
||||
配置错误的条目会被忽略并写入 debug 日志,避免单个无效值导致所有图片代理
|
||||
校验失败。调用方仍然需要先完成域名白名单匹配,不能单独依赖该网段放行。
|
||||
"""
|
||||
networks = []
|
||||
for value in ranges or []:
|
||||
if not value:
|
||||
continue
|
||||
try:
|
||||
networks.append(ipaddress.ip_network(str(value).strip(), strict=False))
|
||||
except ValueError:
|
||||
logger.debug(f"忽略无效的图片代理允许网段配置: {value}")
|
||||
return networks
|
||||
|
||||
@staticmethod
|
||||
def _match_private_addresses(
|
||||
addresses: Optional[List[ipaddress._BaseAddress]],
|
||||
networks: List[ipaddress._BaseNetwork],
|
||||
) -> Optional[tuple[List[ipaddress._BaseAddress], List[ipaddress._BaseNetwork]]]:
|
||||
"""
|
||||
在已解析出的地址列表中匹配显式允许的非公网网段。
|
||||
|
||||
所有解析地址都必须命中至少一个允许网段才放行;只要有一个 IP 落在允许
|
||||
网段外(或解析结果是全公网),就视为不匹配私网放行规则。
|
||||
"""
|
||||
if not addresses or not networks:
|
||||
return None
|
||||
if all(address.is_global for address in addresses):
|
||||
return None
|
||||
|
||||
matched_networks: List[ipaddress._BaseNetwork] = []
|
||||
for address in addresses:
|
||||
matched_for_address = [
|
||||
network for network in networks if address in network
|
||||
]
|
||||
if not matched_for_address:
|
||||
return None
|
||||
matched_networks.extend(matched_for_address)
|
||||
return addresses, list(dict.fromkeys(matched_networks))
|
||||
|
||||
@staticmethod
|
||||
def _is_allowed_private_hostname(
|
||||
hostname: str,
|
||||
allowed_private_ranges: Optional[Iterable[str]],
|
||||
) -> Optional[tuple[List[ipaddress._BaseAddress], List[ipaddress._BaseNetwork]]]:
|
||||
"""
|
||||
返回主机名命中的显式允许非公网地址和网段(同步版本)。
|
||||
|
||||
该能力只用于图片代理的受控例外,例如 TUN fake-ip 或内网 CDN。必须由
|
||||
`is_safe_url` 先完成域名 allowlist 校验后再调用,避免把任意用户 URL
|
||||
变成 SSRF 绕过入口。
|
||||
"""
|
||||
networks = SecurityUtils._parse_ip_networks(allowed_private_ranges)
|
||||
if not networks:
|
||||
return None
|
||||
return SecurityUtils._match_private_addresses(
|
||||
SecurityUtils._hostname_addresses(hostname), networks
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
async def _is_allowed_private_hostname_async(
|
||||
hostname: str,
|
||||
allowed_private_ranges: Optional[Iterable[str]],
|
||||
) -> Optional[tuple[List[ipaddress._BaseAddress], List[ipaddress._BaseNetwork]]]:
|
||||
"""
|
||||
`_is_allowed_private_hostname` 的异步版本,语义保持一致。
|
||||
"""
|
||||
networks = SecurityUtils._parse_ip_networks(allowed_private_ranges)
|
||||
if not networks:
|
||||
return None
|
||||
return SecurityUtils._match_private_addresses(
|
||||
await SecurityUtils._hostname_addresses_async(hostname), networks
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _url_signature_payload(url: str, purpose: str) -> bytes:
|
||||
"""
|
||||
构造 URL 签名载荷。
|
||||
|
||||
签名覆盖用途、过期时间和完整 URL,确保同一个签名不能挪用到其它
|
||||
内网地址或其它代理用途。
|
||||
签名覆盖用途与完整 URL,确保同一个签名不能挪用到其它代理用途或其它 URL。
|
||||
"""
|
||||
return f"{purpose}\n{expires_at}\n{url}".encode("utf-8")
|
||||
return f"{purpose}\n{url}".encode("utf-8")
|
||||
|
||||
@staticmethod
|
||||
def _sign_url_payload(url: str, expires_at: int, purpose: str) -> str:
|
||||
def _sign_url_payload(url: str, purpose: str) -> str:
|
||||
"""
|
||||
使用 RESOURCE_SECRET_KEY 对 URL 签名载荷生成 HMAC。
|
||||
|
||||
相同 `(url, purpose, RESOURCE_SECRET_KEY)` 组合在进程生命周期内输出
|
||||
完全一致;签名的失效边界绑定在 `RESOURCE_SECRET_KEY` 上,进程重启
|
||||
或显式轮换密钥时所有旧签名一起作废。
|
||||
"""
|
||||
return hmac.new(
|
||||
settings.RESOURCE_SECRET_KEY.encode("utf-8"),
|
||||
SecurityUtils._url_signature_payload(url, expires_at, purpose),
|
||||
SecurityUtils._url_signature_payload(url, purpose),
|
||||
sha256,
|
||||
).hexdigest()
|
||||
|
||||
@@ -149,14 +414,19 @@ class SecurityUtils:
|
||||
@staticmethod
|
||||
def sign_url(
|
||||
url: str,
|
||||
expires_in: int = _SIGNED_URL_EXPIRE_SECONDS,
|
||||
purpose: str = _SIGNED_URL_PURPOSE,
|
||||
) -> str:
|
||||
"""
|
||||
给服务端返回的资源 URL 添加临时签名。
|
||||
给服务端返回的资源 URL 添加稳定签名。
|
||||
|
||||
该签名用于允许 `/system/img` 代理访问服务端已经确认过的私网图片 URL,
|
||||
避免代理端点重新依赖媒体服务器的具体路径规则。
|
||||
签名作为 `/system/img` 代理放行私网图片 URL 的能力凭证:图片代理默认
|
||||
拒绝解析到非公网地址的 URL(防 SSRF),合法媒体服务器 URL 必须由后端
|
||||
预先签名后才能跳过该限制。
|
||||
|
||||
签名为 `(url, purpose, RESOURCE_SECRET_KEY)` 的确定性 HMAC,**不带
|
||||
过期时间**:相同 URL 多次调用结果完全一致,让浏览器与 Service Worker
|
||||
的缓存能稳定命中;失效边界由 `RESOURCE_SECRET_KEY` 控制——进程重启
|
||||
自动重生成、或者运维显式轮换后所有历史签名一起作废。
|
||||
"""
|
||||
if not url:
|
||||
return url
|
||||
@@ -164,11 +434,9 @@ class SecurityUtils:
|
||||
if parsed_url.scheme not in {"http", "https"} or not parsed_url.netloc:
|
||||
return url
|
||||
clean_url = SecurityUtils.strip_url_signature(url)
|
||||
expires_at = int(time.time() + expires_in)
|
||||
signature = SecurityUtils._sign_url_payload(clean_url, expires_at, purpose)
|
||||
signature = SecurityUtils._sign_url_payload(clean_url, purpose)
|
||||
fragment = urlencode(
|
||||
{
|
||||
"mp_exp": str(expires_at),
|
||||
"mp_sig": signature,
|
||||
"mp_purpose": purpose,
|
||||
}
|
||||
@@ -182,6 +450,9 @@ class SecurityUtils:
|
||||
) -> Optional[str]:
|
||||
"""
|
||||
验证 URL fragment 中的代理签名,成功时返回去签名后的真实 URL。
|
||||
|
||||
签名只校验 `(url, purpose, RESOURCE_SECRET_KEY)`,密钥轮换/进程重启
|
||||
后旧签名自动失效。
|
||||
"""
|
||||
if not url:
|
||||
return None
|
||||
@@ -189,78 +460,153 @@ class SecurityUtils:
|
||||
if parsed_url.scheme not in {"http", "https"} or not parsed_url.netloc:
|
||||
return None
|
||||
fragment_params = dict(parse_qsl(parsed_url.fragment, keep_blank_values=True))
|
||||
expires_at = fragment_params.get("mp_exp")
|
||||
signature = fragment_params.get("mp_sig")
|
||||
signed_purpose = fragment_params.get("mp_purpose")
|
||||
if not expires_at or not signature or signed_purpose != purpose:
|
||||
return None
|
||||
try:
|
||||
expires_at_int = int(expires_at)
|
||||
except ValueError:
|
||||
return None
|
||||
if expires_at_int < int(time.time()):
|
||||
if not signature or signed_purpose != purpose:
|
||||
return None
|
||||
|
||||
clean_url = SecurityUtils.strip_url_signature(url)
|
||||
expected_signature = SecurityUtils._sign_url_payload(
|
||||
clean_url, expires_at_int, purpose
|
||||
)
|
||||
expected_signature = SecurityUtils._sign_url_payload(clean_url, purpose)
|
||||
if not hmac.compare_digest(signature, expected_signature):
|
||||
return None
|
||||
return clean_url
|
||||
|
||||
@staticmethod
|
||||
def _check_url_allowlist(
|
||||
url: str,
|
||||
allowed_domains: Union[Set[str], List[str]],
|
||||
strict: bool,
|
||||
) -> Optional[str]:
|
||||
"""
|
||||
执行"协议 + netloc + 域名白名单"前置校验,命中返回 hostname,未命中返回 None。
|
||||
|
||||
DNS 校验(SSRF 防御)由调用方自行接续,本方法不发起 DNS 查询。
|
||||
"""
|
||||
try:
|
||||
parsed_url = urlparse(url)
|
||||
except Exception as e: # noqa: BLE001 - 任何解析异常都视为不安全 URL
|
||||
logger.debug(f"Error occurred while validating URL: {e}")
|
||||
return None
|
||||
|
||||
# 如果 URL 没有包含有效的 scheme,或者无法从中提取到有效的 netloc,则认为该 URL 是无效的
|
||||
if not parsed_url.scheme or not parsed_url.netloc:
|
||||
return None
|
||||
# 仅允许 http 或 https 协议
|
||||
if parsed_url.scheme not in {"http", "https"}:
|
||||
return None
|
||||
|
||||
# 获取完整的 netloc(包括 IP 和端口)并转换为小写
|
||||
netloc = parsed_url.netloc.lower()
|
||||
if not netloc:
|
||||
return None
|
||||
|
||||
# 检查每个允许的域名
|
||||
normalized_allowed = {d.lower() for d in allowed_domains}
|
||||
domain_allowed = False
|
||||
for domain in normalized_allowed:
|
||||
parsed_allowed_url = urlparse(domain)
|
||||
allowed_netloc = parsed_allowed_url.netloc or parsed_allowed_url.path
|
||||
|
||||
if strict:
|
||||
# 严格模式下,要求完全匹配域名和端口
|
||||
if netloc == allowed_netloc:
|
||||
domain_allowed = True
|
||||
break
|
||||
else:
|
||||
# 非严格模式下,允许子域名匹配
|
||||
if netloc == allowed_netloc or netloc.endswith("." + allowed_netloc):
|
||||
domain_allowed = True
|
||||
break
|
||||
|
||||
if not domain_allowed:
|
||||
return None
|
||||
return parsed_url.hostname or ""
|
||||
|
||||
@staticmethod
|
||||
def _log_private_range_allowed(
|
||||
url: str,
|
||||
match: tuple[List[ipaddress._BaseAddress], List[ipaddress._BaseNetwork]],
|
||||
) -> None:
|
||||
"""
|
||||
记录"图片代理允许访问配置的非公网网段"放行日志,便于运维排查。
|
||||
"""
|
||||
addresses, matched_networks = match
|
||||
logger.debug(
|
||||
"图片代理允许访问配置的非公网网段: "
|
||||
f"url={url}, ips={','.join(map(str, addresses))}, "
|
||||
f"ranges={','.join(map(str, matched_networks))}"
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def is_safe_url(
|
||||
url: str,
|
||||
allowed_domains: Union[Set[str], List[str]],
|
||||
strict: bool = False,
|
||||
block_private: bool = False,
|
||||
allowed_private_ranges: Optional[Iterable[str]] = None,
|
||||
) -> bool:
|
||||
"""
|
||||
验证URL是否在允许的域名列表中,包括带有端口的域名
|
||||
验证URL是否在允许的域名列表中,包括带有端口的域名(同步版本)
|
||||
|
||||
:param url: 需要验证的 URL
|
||||
:param allowed_domains: 允许的域名集合,域名可以包含端口
|
||||
:param strict: 是否严格匹配一级域名(默认为 False,允许多级域名)
|
||||
:param block_private: 是否拦截解析到非公网地址的 URL,防止 SSRF
|
||||
:param allowed_private_ranges: 域名命中后额外允许的非公网 IP/CIDR 网段
|
||||
:return: 如果URL合法且在允许的域名列表中,返回 True;否则返回 False
|
||||
|
||||
注意:`block_private=True` 时会同步调用 `getaddrinfo`;async 上下文请改用
|
||||
`is_safe_url_async`。
|
||||
"""
|
||||
try:
|
||||
# 解析URL
|
||||
parsed_url = urlparse(url)
|
||||
|
||||
# 如果 URL 没有包含有效的 scheme,或者无法从中提取到有效的 netloc,则认为该 URL 是无效的
|
||||
if not parsed_url.scheme or not parsed_url.netloc:
|
||||
hostname = SecurityUtils._check_url_allowlist(url, allowed_domains, strict)
|
||||
if hostname is None:
|
||||
return False
|
||||
|
||||
# 仅允许 http 或 https 协议
|
||||
if parsed_url.scheme not in {"http", "https"}:
|
||||
if block_private and not SecurityUtils._is_global_hostname(hostname):
|
||||
private_match = SecurityUtils._is_allowed_private_hostname(
|
||||
hostname, allowed_private_ranges
|
||||
)
|
||||
if private_match:
|
||||
SecurityUtils._log_private_range_allowed(url, private_match)
|
||||
return True
|
||||
return False
|
||||
|
||||
# 获取完整的 netloc(包括 IP 和端口)并转换为小写
|
||||
netloc = parsed_url.netloc.lower()
|
||||
if not netloc:
|
||||
return False
|
||||
|
||||
if block_private and not SecurityUtils._is_global_hostname(parsed_url.hostname or ""):
|
||||
return False
|
||||
|
||||
# 检查每个允许的域名
|
||||
allowed_domains = {d.lower() for d in allowed_domains}
|
||||
for domain in allowed_domains:
|
||||
parsed_allowed_url = urlparse(domain)
|
||||
allowed_netloc = parsed_allowed_url.netloc or parsed_allowed_url.path
|
||||
|
||||
if strict:
|
||||
# 严格模式下,要求完全匹配域名和端口
|
||||
if netloc == allowed_netloc:
|
||||
return True
|
||||
else:
|
||||
# 非严格模式下,允许子域名匹配
|
||||
if netloc == allowed_netloc or netloc.endswith('.' + allowed_netloc):
|
||||
return True
|
||||
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.debug(f"Error occurred while validating URL: {e}")
|
||||
return False
|
||||
|
||||
@staticmethod
|
||||
async def is_safe_url_async(
|
||||
url: str,
|
||||
allowed_domains: Union[Set[str], List[str]],
|
||||
strict: bool = False,
|
||||
block_private: bool = False,
|
||||
allowed_private_ranges: Optional[Iterable[str]] = None,
|
||||
) -> bool:
|
||||
"""
|
||||
`is_safe_url` 的异步版本,参数与返回值含义不变。
|
||||
|
||||
DNS 解析通过事件循环线程池执行,并复用 TTL 缓存。
|
||||
"""
|
||||
try:
|
||||
hostname = SecurityUtils._check_url_allowlist(url, allowed_domains, strict)
|
||||
if hostname is None:
|
||||
return False
|
||||
|
||||
if block_private and not await SecurityUtils._is_global_hostname_async(
|
||||
hostname
|
||||
):
|
||||
private_match = await SecurityUtils._is_allowed_private_hostname_async(
|
||||
hostname, allowed_private_ranges
|
||||
)
|
||||
if private_match:
|
||||
SecurityUtils._log_private_range_allowed(url, private_match)
|
||||
return True
|
||||
return False
|
||||
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.debug(f"Error occurred while validating URL: {e}")
|
||||
return False
|
||||
|
||||
84
docs/rules/01-project-overview.md
Normal file
84
docs/rules/01-project-overview.md
Normal file
@@ -0,0 +1,84 @@
|
||||
# 01 — Project Overview
|
||||
|
||||
## System Purpose
|
||||
|
||||
MoviePilot is a self-hosted media automation platform targeting Chinese-language users. It automates the full lifecycle of media acquisition and organization:
|
||||
|
||||
1. **Discovery** — monitors RSS feeds, subscription lists, and recommendation sources for new media releases.
|
||||
2. **Search** — queries configured torrent indexers to locate suitable torrents for subscribed media.
|
||||
3. **Download** — sends torrent tasks to a configured download client (qBittorrent, Transmission, rTorrent).
|
||||
4. **Transfer** — moves or hard-links completed downloads into a structured media library.
|
||||
5. **Scraping** — fetches metadata (posters, descriptions, episode info) from TMDB, TheTVDB, Douban, and Bangumi.
|
||||
6. **Media Server Integration** — notifies and refreshes Emby, Jellyfin, or Plex after files are organized.
|
||||
7. **Messaging** — sends status notifications through Telegram, WeChat, Feishu, Slack, Discord, and other channels.
|
||||
8. **AI Agent** — provides a conversational agent interface (via MCP and LLM chain) for natural-language management tasks.
|
||||
|
||||
---
|
||||
|
||||
## Repository Boundaries
|
||||
|
||||
### What Is in This Repository
|
||||
|
||||
| Path | Content |
|
||||
|---|---|
|
||||
| `app/` | FastAPI backend application |
|
||||
| `moviepilot` | Local CLI entrypoint (install, init, start, stop, update, agent) |
|
||||
| `app/api/endpoints/` | HTTP endpoint handlers |
|
||||
| `app/chain/` | Business orchestration layer |
|
||||
| `app/modules/` | Pluggable backend integrations (downloaders, media servers, etc.) |
|
||||
| `app/helper/` | Reusable low-level utilities |
|
||||
| `app/db/` | SQLAlchemy models and data access wrappers |
|
||||
| `app/core/` | Config, event system, module manager, plugin manager, security |
|
||||
| `app/schemas/` | Pydantic request/response models and shared enums |
|
||||
| `app/agent/` | LLM agent runtime |
|
||||
| `app/workflow/` | Workflow engine |
|
||||
| `database/versions/` | Alembic migration scripts |
|
||||
| `docs/` | CLI, MCP/API, and development workflow documentation |
|
||||
| `skills/` | AI agent skills and associated scripts |
|
||||
| `tests/` | Pytest test suite |
|
||||
|
||||
### What Is NOT in This Repository
|
||||
|
||||
* **Frontend source code** — lives in the separate `MoviePilot-Frontend` repository (Vue/TypeScript). Only the built `dist/` artifact is consumed here.
|
||||
* **Plugin source code** — plugins are installed into `app/plugins/` at runtime from external sources; they are not part of this repository.
|
||||
* **User config and runtime data** — `config/`, `.moviepilot.env`, `*.db` files are local runtime state. Do not modify or commit them unless explicitly requested.
|
||||
|
||||
---
|
||||
|
||||
## Deployment Models
|
||||
|
||||
### Docker (Primary)
|
||||
|
||||
The standard deployment method. A Docker image bundles the backend, frontend static files, and resource data. Users configure via environment variables and mount a config directory.
|
||||
|
||||
### Local CLI
|
||||
|
||||
An alternative for users running from source. The `moviepilot` CLI handles installation, initialization, service management, and updates. See `docs/cli.md` for the full command reference.
|
||||
|
||||
---
|
||||
|
||||
## Key External Dependencies (Domain Context)
|
||||
|
||||
| Service Type | Supported Backends |
|
||||
|---|---|
|
||||
| Torrent indexers | Site-specific spiders, Jackett/Prowlarr compatible |
|
||||
| Download clients | qBittorrent, Transmission, rTorrent |
|
||||
| Media servers | Emby, Jellyfin, Plex, TrimMedia, Zspace, Ugreen |
|
||||
| Metadata sources | TMDB, TheTVDB, Douban, Bangumi, Fanart |
|
||||
| Message channels | Telegram, WeChat, WeChatClawBot, Feishu, Slack, Discord, VoceChat, Synology Chat, WebPush, QQBot |
|
||||
| LLM providers | OpenAI-compatible, Anthropic, and other configurable providers |
|
||||
|
||||
---
|
||||
|
||||
## Business Domain Vocabulary
|
||||
|
||||
| Term | Meaning |
|
||||
|---|---|
|
||||
| Subscribe | A tracked media item (movie or TV series) that MoviePilot will automatically search and download |
|
||||
| Transfer | The process of moving or hard-linking downloaded files into the organized media library |
|
||||
| Chain | A business orchestration class that coordinates multiple modules for a use case |
|
||||
| Module | A pluggable backend integration loaded by the module manager |
|
||||
| Skill | A packaged AI agent capability that can be invoked via the MCP interface |
|
||||
| SystemConfig | Runtime key-value configuration stored in the database and managed via `SystemConfigKey` |
|
||||
|
||||
*Last Updated: 2026-05-25*
|
||||
144
docs/rules/02-tech-stack.md
Normal file
144
docs/rules/02-tech-stack.md
Normal file
@@ -0,0 +1,144 @@
|
||||
# 02 — Tech Stack
|
||||
|
||||
## Runtime and Language
|
||||
|
||||
| Item | Detail |
|
||||
|---|---|
|
||||
| Language | Python 3.11+ |
|
||||
| CI Python version | Python 3.12 |
|
||||
| Async runtime | asyncio (native), integrated with FastAPI/Uvicorn |
|
||||
|
||||
---
|
||||
|
||||
## Backend Framework
|
||||
|
||||
| Item | Detail |
|
||||
|---|---|
|
||||
| Web framework | FastAPI |
|
||||
| ASGI server | Uvicorn |
|
||||
| Data validation | Pydantic v2 (`BaseModel`, `BaseSettings`, `model_validator`) |
|
||||
| Settings management | `pydantic-settings` (`BaseSettings` class in `app/core/config.py`) |
|
||||
|
||||
---
|
||||
|
||||
## Database
|
||||
|
||||
| Item | Detail |
|
||||
|---|---|
|
||||
| Default database | SQLite |
|
||||
| Optional database | PostgreSQL (configured via `DB_TYPE` and related env vars) |
|
||||
| ORM | SQLAlchemy |
|
||||
| Migration tool | Alembic (`database/versions/`) |
|
||||
| PostgreSQL extras | `app/modules/postgresql/` module; setup guide at `docs/postgresql-setup.md` |
|
||||
|
||||
---
|
||||
|
||||
## Caching
|
||||
|
||||
| Item | Detail |
|
||||
|---|---|
|
||||
| File-based cache | `FileCache` / `AsyncFileCache` in `app/core/cache.py` |
|
||||
| Redis | Optional; `app/modules/redis/` module; used for distributed caching when configured |
|
||||
| In-process cache | Decorator helpers `fresh` / `async_fresh` on `FileCache` |
|
||||
|
||||
---
|
||||
|
||||
## LLM and AI Agent
|
||||
|
||||
| Item | Detail |
|
||||
|---|---|
|
||||
| Agent runtime | `app/agent/` — custom LLM agent orchestration |
|
||||
| LLM abstraction | LangChain-based with multi-provider support |
|
||||
| Supported providers | OpenAI-compatible APIs, Anthropic, and other configurable providers |
|
||||
| Configuration | `LLM_PROVIDER`, `LLM_MODEL`, `LLM_API_KEY`, `LLM_BASE_URL` in settings |
|
||||
| Enable flag | `AI_AGENT_ENABLE` |
|
||||
| MCP protocol | JSON-RPC 2.0 at `/api/v1/mcp`; see `docs/mcp-api.md` |
|
||||
|
||||
---
|
||||
|
||||
## Module Integrations
|
||||
|
||||
### Download Clients
|
||||
| Module | Directory |
|
||||
|---|---|
|
||||
| qBittorrent | `app/modules/qbittorrent/` |
|
||||
| Transmission | `app/modules/transmission/` |
|
||||
| rTorrent | `app/modules/rtorrent/` |
|
||||
|
||||
### Media Servers
|
||||
| Module | Directory |
|
||||
|---|---|
|
||||
| Emby | `app/modules/emby/` |
|
||||
| Jellyfin | `app/modules/jellyfin/` |
|
||||
| Plex | `app/modules/plex/` |
|
||||
| TrimMedia | `app/modules/trimemedia/` |
|
||||
| Zspace | `app/modules/zspace/` |
|
||||
| Ugreen | `app/modules/ugreen/` |
|
||||
|
||||
### Message Channels
|
||||
| Module | Directory |
|
||||
|---|---|
|
||||
| Telegram | `app/modules/telegram/` |
|
||||
| WeChat | `app/modules/wechat/` |
|
||||
| WeChatClawBot | `app/modules/wechatclawbot/` |
|
||||
| Feishu | `app/modules/feishu/` |
|
||||
| Slack | `app/modules/slack/` |
|
||||
| Discord | `app/modules/discord/` |
|
||||
| VoceChat | `app/modules/vocechat/` |
|
||||
| Synology Chat | `app/modules/synologychat/` |
|
||||
| WebPush | `app/modules/webpush/` |
|
||||
| QQBot | `app/modules/qqbot/` |
|
||||
|
||||
### Metadata Sources
|
||||
| Module | Directory |
|
||||
|---|---|
|
||||
| TMDB | `app/modules/themoviedb/` |
|
||||
| TheTVDB | `app/modules/thetvdb/` |
|
||||
| Douban | `app/modules/douban/` |
|
||||
| Bangumi | `app/modules/bangumi/` |
|
||||
| Fanart | `app/modules/fanart/` |
|
||||
|
||||
---
|
||||
|
||||
## Dependency Management
|
||||
|
||||
| Item | Detail |
|
||||
|---|---|
|
||||
| Source file | `requirements.in` — edit this to add or upgrade dependencies |
|
||||
| Lock file | `requirements.txt` — generated by `pip-compile`; never edit manually |
|
||||
| Tool | `pip-tools` (`pip-compile`, `pip-sync`) |
|
||||
| Install | `pip install -r requirements.txt` |
|
||||
|
||||
---
|
||||
|
||||
## Performance Extension
|
||||
|
||||
| Item | Detail |
|
||||
|---|---|
|
||||
| Rust extension | `moviepilot_rust` — optional compiled accelerator for core processing paths |
|
||||
| Build | Requires Rust `cargo`; built automatically by `moviepilot install deps` |
|
||||
| Skip flag | `MOVIEPILOT_SKIP_RUST_ACCEL=1` disables build (falls back to Python implementation) |
|
||||
| Toggle | Can be disabled/re-enabled at runtime via frontend Advanced Settings → Lab |
|
||||
|
||||
---
|
||||
|
||||
## Quality Tooling
|
||||
|
||||
| Tool | Purpose | Command |
|
||||
|---|---|---|
|
||||
| pytest | Test runner | `pytest tests/test_xxx.py` |
|
||||
| pylint | Static analysis | `pylint app/` |
|
||||
| safety | Dependency vulnerability scan | `safety check -r requirements.txt --policy-file=safety.policy.yml` |
|
||||
|
||||
---
|
||||
|
||||
## Deployment
|
||||
|
||||
| Method | Detail |
|
||||
|---|---|
|
||||
| Docker | Primary deployment; image bundles backend + frontend static files + resources |
|
||||
| Local CLI | `moviepilot` CLI for source-based install; see `docs/cli.md` |
|
||||
| Frontend | Vue/TypeScript SPA served from `public/`; source in `MoviePilot-Frontend` repo |
|
||||
| Frontend proxy | Local Node `service.js` proxies `/api` and `/cookiecloud` to the backend |
|
||||
|
||||
*Last Updated: 2026-05-25*
|
||||
259
docs/rules/03-commands.md
Normal file
259
docs/rules/03-commands.md
Normal file
@@ -0,0 +1,259 @@
|
||||
# 03 — Commands
|
||||
|
||||
Only suggest or execute commands that appear in this document. Do not assume standard tool defaults, global flags, or operating-system-specific behavior unless explicitly listed here.
|
||||
|
||||
---
|
||||
|
||||
## Development Environment Setup
|
||||
|
||||
```bash
|
||||
# Create and activate virtual environment
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate # macOS / Linux
|
||||
.\venv\Scripts\activate # Windows
|
||||
|
||||
# Install pip-tools
|
||||
pip install pip-tools
|
||||
|
||||
# Install project dependencies
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Dependency Management
|
||||
|
||||
```bash
|
||||
# Compile requirements.txt from requirements.in (full recompile)
|
||||
pip-compile requirements.in
|
||||
|
||||
# Upgrade a single package without touching others
|
||||
pip-compile --upgrade-package <package-name> requirements.in
|
||||
|
||||
# Install from the generated lock file
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
**Rules:**
|
||||
- Always edit `requirements.in` to add or change dependencies.
|
||||
- Never edit `requirements.txt` manually — it is a generated lock file.
|
||||
- After any change to `requirements.in`, re-run `pip-compile requirements.in` and commit both files together.
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
# Run a specific test file
|
||||
pytest tests/test_xxx.py
|
||||
|
||||
# Run all tests
|
||||
pytest
|
||||
|
||||
# Run tests with verbose output
|
||||
pytest -v tests/test_xxx.py
|
||||
|
||||
# Run a specific test function
|
||||
pytest tests/test_xxx.py::test_function_name
|
||||
```
|
||||
|
||||
**Rules:**
|
||||
- Run at minimum the tests directly related to the change.
|
||||
- If the change affects common modules, startup flow, CLI, or agent runtime behavior, expand the scope to the full test suite.
|
||||
- If the task only changes documentation, state explicitly that tests were not run. Do not claim checks that were not executed.
|
||||
|
||||
---
|
||||
|
||||
## Static Analysis
|
||||
|
||||
```bash
|
||||
# Run pylint on the application package
|
||||
pylint app/
|
||||
|
||||
# Run pylint on a specific module
|
||||
pylint app/chain/download.py
|
||||
```
|
||||
|
||||
**Rules:**
|
||||
- After Python code changes, ensure no new error-level issues are introduced.
|
||||
- Warning-level issues in new code should be minimized but are not an absolute gate.
|
||||
|
||||
---
|
||||
|
||||
## Security Scan
|
||||
|
||||
```bash
|
||||
# Run safety check against the lock file
|
||||
safety check -r requirements.txt --policy-file=safety.policy.yml
|
||||
|
||||
# Save report to file
|
||||
safety check -r requirements.txt --policy-file=safety.policy.yml > safety_report.txt
|
||||
```
|
||||
|
||||
**Rules:**
|
||||
- Run after every change to `requirements.txt`.
|
||||
- No new high-severity vulnerabilities may be introduced.
|
||||
|
||||
---
|
||||
|
||||
## Local CLI — Service Management
|
||||
|
||||
```bash
|
||||
moviepilot start
|
||||
moviepilot start --timeout 60
|
||||
moviepilot stop
|
||||
moviepilot stop --timeout 30 --force
|
||||
moviepilot restart
|
||||
moviepilot restart --start-timeout 60 --stop-timeout 30
|
||||
moviepilot status
|
||||
moviepilot version
|
||||
```
|
||||
|
||||
```bash
|
||||
moviepilot logs
|
||||
moviepilot logs --lines 100
|
||||
moviepilot logs --stdio
|
||||
moviepilot logs --frontend
|
||||
moviepilot logs --follow
|
||||
moviepilot logs --frontend --follow
|
||||
moviepilot logs --stdio --follow
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Local CLI — Installation and Setup
|
||||
|
||||
```bash
|
||||
# One-line bootstrap installer
|
||||
curl -fsSL https://raw.githubusercontent.com/jxxghp/MoviePilot/v2/scripts/bootstrap-local.sh | bash
|
||||
|
||||
# Install backend dependencies
|
||||
moviepilot install deps
|
||||
moviepilot install deps --python python3.11
|
||||
moviepilot install deps --venv /path/to/venv
|
||||
moviepilot install deps --recreate
|
||||
|
||||
# Install frontend release
|
||||
moviepilot install frontend
|
||||
moviepilot install frontend --version latest
|
||||
moviepilot install frontend --version v2.9.31
|
||||
|
||||
# Install resource files
|
||||
moviepilot install resources
|
||||
|
||||
# Initialize local config
|
||||
moviepilot init
|
||||
moviepilot init --wizard
|
||||
moviepilot init --force-token
|
||||
moviepilot init --superuser admin --superuser-password 'ChangeMe123!'
|
||||
|
||||
# All-in-one setup
|
||||
moviepilot setup
|
||||
moviepilot setup --wizard
|
||||
moviepilot setup --recreate
|
||||
moviepilot setup --superuser admin --superuser-password 'ChangeMe123!'
|
||||
|
||||
# Uninstall
|
||||
moviepilot uninstall
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Local CLI — Update
|
||||
|
||||
```bash
|
||||
moviepilot update backend
|
||||
moviepilot update backend --ref latest
|
||||
moviepilot update backend --ref v2.9.31
|
||||
|
||||
moviepilot update frontend
|
||||
moviepilot update frontend --frontend-version latest
|
||||
|
||||
moviepilot update all
|
||||
moviepilot update all --ref latest --frontend-version latest
|
||||
moviepilot update all --skip-resources
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Local CLI — Startup on Boot
|
||||
|
||||
```bash
|
||||
moviepilot startup status
|
||||
moviepilot startup enable
|
||||
moviepilot startup disable
|
||||
moviepilot startup enable --venv /path/to/venv
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Local CLI — Configuration
|
||||
|
||||
```bash
|
||||
moviepilot config path
|
||||
moviepilot config list
|
||||
moviepilot config list --show-secrets
|
||||
moviepilot config get PORT
|
||||
moviepilot config set PORT 3001
|
||||
moviepilot config keys
|
||||
moviepilot config keys DB_
|
||||
moviepilot config keys --show-current
|
||||
moviepilot config describe PORT
|
||||
moviepilot config describe API_TOKEN --show-secrets
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Local CLI — Tools and Scheduler
|
||||
|
||||
```bash
|
||||
# List all MCP tools
|
||||
moviepilot tool list
|
||||
|
||||
# Show tool parameters
|
||||
moviepilot tool show query_schedulers
|
||||
moviepilot tool show search_torrents
|
||||
|
||||
# Run a tool directly
|
||||
moviepilot tool run query_schedulers
|
||||
moviepilot tool run search_torrents media_type=movie tmdb_id=12345
|
||||
|
||||
# List scheduled tasks
|
||||
moviepilot scheduler list
|
||||
|
||||
# Immediately run a scheduled task
|
||||
moviepilot scheduler run subscribe_refresh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Local CLI — Agent
|
||||
|
||||
```bash
|
||||
moviepilot agent "Help me analyze the last search failure"
|
||||
moviepilot agent --user-id admin "Check the current downloader configuration"
|
||||
moviepilot agent --session cli-debug-1 "Why was the last transfer not triggered?"
|
||||
moviepilot agent --new-session "Summarize any obvious problems with the current system config"
|
||||
```
|
||||
|
||||
**Prerequisites:** `AI_AGENT_ENABLE` must be set to true, and LLM provider settings (`LLM_PROVIDER`, `LLM_MODEL`, `LLM_API_KEY`) must be configured.
|
||||
|
||||
---
|
||||
|
||||
## Local CLI — Help Discovery
|
||||
|
||||
```bash
|
||||
moviepilot --help
|
||||
moviepilot help
|
||||
moviepilot commands
|
||||
moviepilot help install
|
||||
moviepilot help init
|
||||
moviepilot help setup
|
||||
moviepilot help update
|
||||
moviepilot help agent
|
||||
moviepilot help config
|
||||
moviepilot help tool
|
||||
moviepilot help scheduler
|
||||
```
|
||||
|
||||
*Last Updated: 2026-05-25*
|
||||
219
docs/rules/04-design-patterns.md
Normal file
219
docs/rules/04-design-patterns.md
Normal file
@@ -0,0 +1,219 @@
|
||||
# 04 — Design Patterns
|
||||
|
||||
This document defines the structural patterns used across this codebase. When implementing complex features, you are required to use these patterns rather than inventing new abstractions.
|
||||
|
||||
---
|
||||
|
||||
## 1. Module Pattern (Pluggable Backends)
|
||||
|
||||
**When to use:** Adding a new downloader, media server, message channel, storage backend, or any other capability that requires lifecycle management, configuration switches, priority ordering, or independent testing.
|
||||
|
||||
**Base class:** `_ModuleBase` in `app/modules/__init__.py`
|
||||
|
||||
**Specialized base classes:**
|
||||
- `_DownloaderBase` — for download clients
|
||||
- `_MediaServerBase` — for media servers (implied by existing patterns)
|
||||
|
||||
**Required methods every module must implement:**
|
||||
|
||||
```python
|
||||
class ExampleModule(_ModuleBase, _DownloaderBase):
|
||||
|
||||
def init_module(self) -> None:
|
||||
"""模块初始化"""
|
||||
super().init_service(service_name=..., service_type=...)
|
||||
|
||||
def init_setting(self) -> Tuple[str, Union[str, bool]]:
|
||||
"""返回控制此模块开关的配置项名称和匹配值"""
|
||||
return "DOWNLOADER", "example"
|
||||
|
||||
@staticmethod
|
||||
def get_name() -> str:
|
||||
return "Example"
|
||||
|
||||
@staticmethod
|
||||
def get_type() -> ModuleType:
|
||||
return ModuleType.Downloader
|
||||
|
||||
@staticmethod
|
||||
def get_subtype() -> DownloaderType:
|
||||
return DownloaderType.Example
|
||||
|
||||
@staticmethod
|
||||
def get_priority() -> int:
|
||||
return 1
|
||||
|
||||
def test(self) -> Optional[Tuple[bool, str]]:
|
||||
"""测试模块连通性"""
|
||||
...
|
||||
|
||||
def stop(self):
|
||||
pass
|
||||
```
|
||||
|
||||
**Module directory convention:** `app/modules/<backend_name>/` containing at minimum `__init__.py` (the module class) and the implementation class.
|
||||
|
||||
**Module types** are defined in `app/schemas/types.py` as `ModuleType`, `DownloaderType`, `MediaServerType`, `MessageChannel`, `StorageSchema`, `OtherModulesType`. When adding a new category, update these enums.
|
||||
|
||||
---
|
||||
|
||||
## 2. Chain Orchestration Pattern
|
||||
|
||||
**When to use:** Adding a new business workflow that is shared across multiple entrypoints (API endpoint, CLI, agent, scheduler, webhook). Chains coordinate modules, helpers, databases, events, and caches.
|
||||
|
||||
**Base class:** `ChainBase` in `app/chain/__init__.py`
|
||||
|
||||
**Calling modules from a chain:**
|
||||
|
||||
```python
|
||||
# Preferred: call via run_module / async_run_module
|
||||
result = self.run_module("method_name", kwarg1=val1, kwarg2=val2)
|
||||
result = await self.async_run_module("method_name", kwarg1=val1)
|
||||
|
||||
# Only use ModuleManager directly when you need to enumerate modules,
|
||||
# inspect instances, or run health checks.
|
||||
```
|
||||
|
||||
**Chain-to-chain calls:** A chain may call another chain to reuse stable domain logic. Avoid introducing new circular dependencies between chains.
|
||||
|
||||
**File convention:** `app/chain/<domain>.py`, class name `<Domain>Chain` (e.g., `DownloadChain`, `SearchChain`, `SubscribeChain`).
|
||||
|
||||
---
|
||||
|
||||
## 3. Event / Observer Pattern
|
||||
|
||||
**When to use:** Triggering cross-cutting reactions (e.g., notifying the media server after a transfer completes, reloading a module after config changes, dispatching user messages to message channels).
|
||||
|
||||
**Core classes:** `EventManager` (singleton instance `eventmanager`) and `Event` in `app/core/event.py`.
|
||||
|
||||
**Registering a handler:**
|
||||
|
||||
```python
|
||||
from app.core.event import eventmanager, Event
|
||||
from app.schemas.types import EventType
|
||||
|
||||
@eventmanager.register(EventType.TransferComplete)
|
||||
def on_transfer_complete(self, event: Event):
|
||||
event_data = event.event_data
|
||||
...
|
||||
```
|
||||
|
||||
**Sending an event:**
|
||||
|
||||
```python
|
||||
eventmanager.send_event(EventType.TransferComplete, data_dict)
|
||||
```
|
||||
|
||||
**Event types** are defined as `EventType` and `ChainEventType` enums in `app/schemas/types.py`. Add new event types there when extending the event system.
|
||||
|
||||
---
|
||||
|
||||
## 4. Repository (Oper) Pattern
|
||||
|
||||
**When to use:** All database reads and writes. Never issue SQLAlchemy queries directly from chain, module, or endpoint code.
|
||||
|
||||
**Convention:** Each SQLAlchemy model in `app/db/models/` has a corresponding `<Model>Oper` class in `app/db/<model>_oper.py`.
|
||||
|
||||
```
|
||||
app/db/models/subscribe.py → app/db/subscribe_oper.py (SubscribeOper)
|
||||
app/db/models/systemconfig.py → app/db/systemconfig_oper.py (SystemConfigOper)
|
||||
app/db/models/transferhistory.py → app/db/transferhistory_oper.py (TransferHistoryOper)
|
||||
```
|
||||
|
||||
**Usage:**
|
||||
|
||||
```python
|
||||
from app.db.subscribe_oper import SubscribeOper
|
||||
|
||||
oper = SubscribeOper()
|
||||
subscribe = oper.get(sid=1)
|
||||
oper.add(Subscribe(name="Example", type="电影"))
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Config Reload Pattern
|
||||
|
||||
**When to use:** A chain, module, or helper holds a long-lived object that must be rebuilt when specific configuration keys change (e.g., a downloader client reconnects when its host/port changes).
|
||||
|
||||
**Mixin:** `ConfigReloadMixin` in `app/utils/mixins.py`
|
||||
|
||||
**How it works:**
|
||||
1. Inherit `ConfigReloadMixin`.
|
||||
2. Define a `CONFIG_WATCH` class attribute as a set of config key names.
|
||||
3. Implement `on_config_changed()` — called automatically when any watched key changes.
|
||||
4. Optionally implement `get_reload_name()` to provide a descriptive name for log messages.
|
||||
|
||||
```python
|
||||
class MyChain(ChainBase, ConfigReloadMixin):
|
||||
|
||||
CONFIG_WATCH = {"DOWNLOADER", "QB_HOST", "QB_PORT"}
|
||||
|
||||
def on_config_changed(self):
|
||||
self.init_module()
|
||||
```
|
||||
|
||||
`_ModuleBase` already inherits `ConfigReloadMixin` and calls `init_module()` from `on_config_changed()` by default. Modules typically only need to declare `CONFIG_WATCH`.
|
||||
|
||||
---
|
||||
|
||||
## 6. Singleton Pattern
|
||||
|
||||
**When to use:** Classes that must have exactly one instance shared application-wide (e.g., `EventManager`, `ModuleManager`, `PluginManager`).
|
||||
|
||||
**Implementation:** Inherit from `Singleton` in `app/utils/singleton.py`.
|
||||
|
||||
```python
|
||||
from app.utils.singleton import Singleton
|
||||
|
||||
class MyManager(metaclass=Singleton):
|
||||
...
|
||||
```
|
||||
|
||||
Do not introduce new singletons unless the class genuinely manages global shared state. Prefer dependency injection or parameter passing for everything else.
|
||||
|
||||
---
|
||||
|
||||
## 7. SystemConfig Pattern
|
||||
|
||||
**When to use:** Storing runtime business configuration that is user-editable, persistent across restarts, and not tied to a specific deployment environment.
|
||||
|
||||
**Enum:** `SystemConfigKey` in `app/schemas/types.py`
|
||||
|
||||
**Oper class:** `SystemConfigOper` in `app/db/systemconfig_oper.py`
|
||||
|
||||
```python
|
||||
from app.schemas.types import SystemConfigKey
|
||||
from app.db.systemconfig_oper import SystemConfigOper
|
||||
|
||||
oper = SystemConfigOper()
|
||||
value = oper.get(SystemConfigKey.RssUrls)
|
||||
oper.set(SystemConfigKey.RssUrls, ["https://..."])
|
||||
```
|
||||
|
||||
**Rule:** Never use raw string literals as SystemConfig keys. Always add a new entry to the `SystemConfigKey` enum first.
|
||||
|
||||
---
|
||||
|
||||
## 8. UserConfig Pattern
|
||||
|
||||
**When to use:** Per-user settings that must survive across sessions but differ by user.
|
||||
|
||||
**Oper class:** `UserConfigOper` in `app/db/userconfig_oper.py`
|
||||
|
||||
Usage mirrors `SystemConfigOper` but scoped to a `user_id`.
|
||||
|
||||
---
|
||||
|
||||
## Anti-Patterns to Avoid
|
||||
|
||||
| Anti-Pattern | Correct Alternative |
|
||||
|---|---|
|
||||
| `module -> chain` coupling | Move shared logic into `chain` or down into `helper` |
|
||||
| `module -> module` direct calls | Use `chain` to orchestrate cross-module workflows |
|
||||
| `helper -> chain` dependency | `helper` must remain a low-level utility; move orchestration to `chain` |
|
||||
| Raw SQLAlchemy queries in endpoints or chains | Use the corresponding `*_oper.py` class |
|
||||
| Raw string keys for SystemConfig | Define and use a `SystemConfigKey` enum entry |
|
||||
| HTTP requests via `requests` or `httpx` directly | Use `RequestUtils` from `app/utils/http.py` |
|
||||
|
||||
*Last Updated: 2026-05-25*
|
||||
169
docs/rules/05-architecture.md
Normal file
169
docs/rules/05-architecture.md
Normal file
@@ -0,0 +1,169 @@
|
||||
# 05 — Architecture and Modules
|
||||
|
||||
## Layer Overview
|
||||
|
||||
The application is structured as four distinct layers. Each layer has a defined responsibility, and dependency may only flow in permitted directions.
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────┐
|
||||
│ Entrypoints │
|
||||
│ (API Endpoints / CLI / Agent / Scheduler / │
|
||||
│ Webhook / Message Interaction) │
|
||||
└────────────────────┬─────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────────────────────────────────────┐
|
||||
│ Chain Layer (app/chain/) │
|
||||
│ Business orchestration: search, download, │
|
||||
│ subscribe, transfer, message, recommend, etc. │
|
||||
└──────┬──────────────┬───────────────┬────────────┘
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌────────────┐ ┌──────────┐ ┌────────────────┐
|
||||
│ Module │ │ Helper │ │ DB / Oper │
|
||||
│ Layer │ │ Layer │ │ Layer │
|
||||
│ (app/ │ │ (app/ │ │ (app/db/) │
|
||||
│ modules/) │ │ helper/)│ │ │
|
||||
└────────────┘ └──────────┘ └────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Layer Responsibilities and Boundaries
|
||||
|
||||
### Entrypoint Layer
|
||||
|
||||
**Directories:** `app/api/endpoints/`, `moviepilot` (CLI), `app/agent/`, scheduler callbacks, webhook handlers, message interactions.
|
||||
|
||||
**Responsibilities:**
|
||||
- HTTP concerns: authentication, parameter parsing, response model serialization, streaming adaptation, simple input validation.
|
||||
- Simple list, detail, toggle, settings read/write, and pure CRUD endpoints may call `app/db/` or a helper directly.
|
||||
- Any logic that coordinates multiple modules, triggers events, touches caches, or combines workflows must be moved into `chain`.
|
||||
|
||||
**Rules:**
|
||||
- Prefer adding new endpoints to an existing domain file. Create a new endpoint file only when introducing a new top-level resource domain.
|
||||
- After adding a new endpoint, register it in `app/api/apiv1.py`.
|
||||
- Endpoints must not contain business logic that belongs in `chain`.
|
||||
|
||||
---
|
||||
|
||||
### Chain Layer
|
||||
|
||||
**Directory:** `app/chain/`
|
||||
|
||||
**Responsibilities:**
|
||||
- Business orchestration shared by API, CLI, agent, scheduler, and other entrypoints.
|
||||
- Composes module capabilities, helpers, database access, events, and caches.
|
||||
- Focuses on use cases and workflows.
|
||||
|
||||
**Rules:**
|
||||
- Call module capabilities via `run_module()` or `async_run_module()`. Use `ModuleManager` directly only when enumerating, inspecting, or running health checks.
|
||||
- Do not hold low-level protocol details, HTTP request objects, or page-specific parameter assembly.
|
||||
- Before creating a new chain file, verify the workflow is genuinely reused across multiple entrypoints, or coordinates multiple modules. If it is short logic for a single endpoint, keep it in the endpoint.
|
||||
- Chain-to-chain calls are allowed when reusing stable domain logic. Avoid introducing new circular dependencies.
|
||||
|
||||
---
|
||||
|
||||
### Module Layer
|
||||
|
||||
**Directory:** `app/modules/`
|
||||
|
||||
**Responsibilities:**
|
||||
- Pluggable capability implementations: downloaders, media servers, message channels, metadata sources, storage backends, subtitle backends, filter backends, etc.
|
||||
- Manages lifecycle (init, stop), configuration switches, priority ordering, and independent testability.
|
||||
|
||||
**Module categories (defined in `app/schemas/types.py`):**
|
||||
|
||||
| Enum | Examples |
|
||||
|---|---|
|
||||
| `ModuleType.Downloader` | qBittorrent, Transmission, rTorrent |
|
||||
| `ModuleType.MediaServer` | Emby, Jellyfin, Plex, TrimMedia, Zspace, Ugreen |
|
||||
| `ModuleType.MessageChannel` | Telegram, WeChat, Feishu, Slack, Discord |
|
||||
| `ModuleType.MetaData` | TMDB, TheTVDB, Douban, Bangumi, Fanart |
|
||||
| `ModuleType.Indexer` | Site-specific torrent indexers |
|
||||
| `ModuleType.Storage` | Alist, rclone, u115, local storage |
|
||||
|
||||
**Rules:**
|
||||
- A module must focus on one backend or one capability. It returns domain result objects, not HTTP responses, and must not depend on FastAPI request objects or endpoint auth.
|
||||
- Do not add direct `module → module` coupling for new code. Cross-module orchestration must go through `chain`.
|
||||
- Do not expand the historical `module → chain` usage pattern. If a module needs shared business logic, move that logic into `chain` or down into `helper`.
|
||||
|
||||
---
|
||||
|
||||
### Helper Layer
|
||||
|
||||
**Directory:** `app/helper/`
|
||||
|
||||
**Responsibilities:**
|
||||
- Reusable low-level support: path handling, config aggregation, site index loading, protocol wrappers, rate limiting, cache utilities, page parsing, notification helpers.
|
||||
|
||||
**Rules:**
|
||||
- Add a new helper only when the logic is reused in multiple places, or it is clearly a standalone low-level concern.
|
||||
- If logic is used only by a single chain or module, keep it in the original file. Do not turn `helper` into a dumping ground.
|
||||
- If the code needs configuration switches, runtime loading, priorities, or multi-implementation dispatch, it is a `module`, not a `helper`.
|
||||
- `helper` must not contain full business workflows.
|
||||
|
||||
---
|
||||
|
||||
### DB / Oper Layer
|
||||
|
||||
**Directory:** `app/db/`
|
||||
|
||||
**Responsibilities:**
|
||||
- SQLAlchemy models under `app/db/models/`.
|
||||
- Data access wrappers (`*_oper.py`) that encapsulate all database queries.
|
||||
|
||||
**Rules:**
|
||||
- Never issue SQLAlchemy queries directly from chain, module, or endpoint code. Always use the corresponding `*_oper.py` class.
|
||||
- Any schema change requires a new Alembic migration under `database/versions/`.
|
||||
|
||||
---
|
||||
|
||||
## Permitted Call Directions
|
||||
|
||||
| Direction | Status |
|
||||
|---|---|
|
||||
| `endpoint / CLI / agent / scheduler → chain` | ✅ Preferred |
|
||||
| `endpoint / CLI / agent / scheduler → db / helper` | ✅ Allowed for simple CRUD and input normalization only |
|
||||
| `chain → chain` | ✅ Allowed when reusing stable, non-circular domain logic |
|
||||
| `chain → module` | ✅ Via `run_module()` / `async_run_module()` |
|
||||
| `chain → helper` | ✅ Allowed |
|
||||
| `chain → db` | ✅ Via `*_oper.py` classes |
|
||||
| `module → chain` | ⚠️ Exists in legacy code; do not expand in new code |
|
||||
| `module → module` | ❌ Forbidden in new code |
|
||||
| `helper → chain` | ❌ Forbidden |
|
||||
| `helper → endpoint` | ❌ Forbidden |
|
||||
|
||||
---
|
||||
|
||||
## Key File Locations
|
||||
|
||||
| Path | Purpose |
|
||||
|---|---|
|
||||
| `app/api/apiv1.py` | API router registration — register new endpoints here |
|
||||
| `app/core/config.py` | `ConfigModel` and `Settings` — all deployment/env-level config |
|
||||
| `app/schemas/types.py` | `SystemConfigKey`, `EventType`, `ModuleType`, and all shared enums |
|
||||
| `app/core/module.py` | `ModuleManager` — discovers and manages module instances |
|
||||
| `app/core/plugin.py` | `PluginManager` — discovers and manages plugin instances |
|
||||
| `app/core/event.py` | `EventManager` + `Event` — the application event bus |
|
||||
| `app/core/context.py` | `Context`, `MediaInfo`, `TorrentInfo` — shared domain context objects |
|
||||
| `app/main.py` | Application startup and FastAPI instance |
|
||||
| `database/versions/` | Alembic migration scripts |
|
||||
|
||||
---
|
||||
|
||||
## Where New Capabilities Go
|
||||
|
||||
| Scenario | Action |
|
||||
|---|---|
|
||||
| New business workflow shared by multiple entrypoints | `app/chain/` |
|
||||
| New downloader, media server, message channel, or storage backend | `app/modules/<backend>/` |
|
||||
| New public HTTP API endpoint | `app/api/endpoints/`, register in `app/api/apiv1.py` |
|
||||
| New low-level utility reused in multiple places | `app/helper/` |
|
||||
| New deployment/env/startup config (ports, paths, API keys) | `ConfigModel` in `app/core/config.py` |
|
||||
| New runtime business config, user-editable rule, or persistent system option | `SystemConfigKey` + `SystemConfigOper` |
|
||||
| Config change should reload a long-lived object | Add `CONFIG_WATCH` + `on_config_changed()` to the relevant class |
|
||||
| Few dozen lines of private logic in one chain or module | Private function in the same file; do not create a new helper |
|
||||
| New module category or subtype | Also update `app/schemas/types.py` |
|
||||
|
||||
*Last Updated: 2026-05-25*
|
||||
121
docs/rules/06-code-styles.md
Normal file
121
docs/rules/06-code-styles.md
Normal file
@@ -0,0 +1,121 @@
|
||||
# 06 — Code Standards and Style
|
||||
|
||||
## General Principles
|
||||
|
||||
- Preserve the style of the surrounding file. When in doubt, read neighboring code first.
|
||||
- Prefer the smallest correct change. Do not introduce a new abstraction layer without a clear payoff.
|
||||
- Do not add features, refactors, or abstractions beyond what the task requires.
|
||||
- Do not add error handling or validation for scenarios that cannot happen. Trust internal code and framework guarantees; only validate at system boundaries (user input, external API responses).
|
||||
|
||||
---
|
||||
|
||||
## Python Version and Typing
|
||||
|
||||
- Target: **Python 3.11+**. CI runs Python 3.12.
|
||||
- **Type annotations are required** on all public methods and function signatures.
|
||||
- Use `Optional[X]` for nullable types (do not use `X | None` — keep consistency with the existing codebase style).
|
||||
- Use `Union[X, Y]` for multi-type parameters.
|
||||
- Prefer `list[X]`, `dict[K, V]`, `tuple[X, Y]` built-in generics in new code (Python 3.9+); match the style of the surrounding file.
|
||||
- Use `pathlib.Path` for all file path operations. Never use raw string concatenation for paths.
|
||||
|
||||
---
|
||||
|
||||
## Pydantic Models
|
||||
|
||||
- All request body and response models must be defined as Pydantic `BaseModel` subclasses in `app/schemas/`.
|
||||
- Use `Field(...)` for required fields; use `Field(default=...)` or `Field(None)` for optional fields.
|
||||
- Do not define ad-hoc `dict` return types for API responses — define a schema class.
|
||||
- Settings and deployment configuration live in `ConfigModel` / `Settings` in `app/core/config.py` using `pydantic-settings`.
|
||||
- Use `model_validator` for cross-field validation logic.
|
||||
|
||||
---
|
||||
|
||||
## Async and Concurrency
|
||||
|
||||
- Prefer `async def` for I/O-bound operations (network requests, database queries, file operations).
|
||||
- Use `await` consistently; do not mix sync and async code paths in the same function without using `run_in_threadpool` from FastAPI or `asyncio.to_thread`.
|
||||
- For CPU-bound work that must not block the event loop, submit to `ThreadHelper` (see `app/helper/thread.py`).
|
||||
- Do not use bare `threading.Thread` in new code; use `ThreadHelper.submit()`.
|
||||
|
||||
---
|
||||
|
||||
## Imports
|
||||
|
||||
Order imports as follows, separated by blank lines:
|
||||
|
||||
1. Standard library (`import os`, `import json`, etc.)
|
||||
2. Third-party packages (`from fastapi import ...`, `from pydantic import ...`)
|
||||
3. Local application packages (`from app.chain import ...`, `from app.schemas import ...`)
|
||||
|
||||
Within each group, sort alphabetically. Do not use wildcard imports (`from module import *`) in application code.
|
||||
|
||||
---
|
||||
|
||||
## String Formatting
|
||||
|
||||
- Use **f-strings** for all string interpolation. Do not use `%` formatting or `.format()`.
|
||||
- For log messages, use `logger.info(f"...")` — do not use lazy `%s` format in logger calls (the project does not rely on lazy evaluation here).
|
||||
|
||||
---
|
||||
|
||||
## Error Handling
|
||||
|
||||
- In **chain and module layers**: do not raise HTTP exceptions. Catch exceptions, log them, and return `None` or a domain-level error object so the caller can decide how to proceed.
|
||||
- In **endpoint layer**: use FastAPI's `HTTPException` or the project's standard response schemas for errors.
|
||||
- Never swallow exceptions silently. At minimum log the error with `logger.error(f"...: {str(err)}")`.
|
||||
- Do not use bare `except:` — always catch a specific exception type or at minimum `Exception`.
|
||||
|
||||
```python
|
||||
# Correct
|
||||
try:
|
||||
result = self.do_work()
|
||||
except Exception as err:
|
||||
logger.error(f"Failed to do work: {str(err)}")
|
||||
return None
|
||||
|
||||
# Wrong — swallowing silently
|
||||
try:
|
||||
result = self.do_work()
|
||||
except:
|
||||
pass
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Logging
|
||||
|
||||
- Use `logger` from `app/log.py`. Do not import the standard library `logging` directly in application code.
|
||||
- Log levels:
|
||||
- `logger.debug(...)` — detailed diagnostic information, disabled by default.
|
||||
- `logger.info(...)` — normal operational events.
|
||||
- `logger.warning(...)` — unexpected but recoverable situations.
|
||||
- `logger.error(...)` — failures that affect functionality.
|
||||
- Keep log messages in Chinese unless the surrounding file consistently uses English.
|
||||
|
||||
---
|
||||
|
||||
## Constants and Magic Values
|
||||
|
||||
- Do not scatter raw string keys for `SystemConfig`. Add a `SystemConfigKey` enum entry and reference it.
|
||||
- Do not use magic numbers or magic strings inline. Define a named constant or enum value.
|
||||
|
||||
---
|
||||
|
||||
## File Organization
|
||||
|
||||
- One primary class per file is the norm for chains, modules, and helpers.
|
||||
- Private helper functions in the same file are preferable to extracting a new helper for single-use logic.
|
||||
- Keep files focused on one domain concern.
|
||||
|
||||
---
|
||||
|
||||
## What Not To Do
|
||||
|
||||
- Do not introduce new third-party libraries without updating `requirements.in` and running `pip-compile`.
|
||||
- Do not use `requests` or `httpx` directly for external HTTP calls — use `RequestUtils` from `app/utils/http.py`.
|
||||
- Do not issue raw SQLAlchemy queries from chains, modules, or endpoints — use the `*_oper.py` classes.
|
||||
- Do not add TODO or FIXME without context. Only keep one if it is genuinely deferred and cannot be addressed in the current task.
|
||||
- Do not add noisy markers like `# change starts here`, `# important`, or `# this is a fix`.
|
||||
- Do not write comments that restate what the code already clearly says.
|
||||
|
||||
*Last Updated: 2026-05-25*
|
||||
102
docs/rules/07-naming-conventions.md
Normal file
102
docs/rules/07-naming-conventions.md
Normal file
@@ -0,0 +1,102 @@
|
||||
# 07 — Naming Conventions
|
||||
|
||||
All new code must follow these conventions. Consistent naming is how the codebase communicates intent without comments.
|
||||
|
||||
---
|
||||
|
||||
## Files
|
||||
|
||||
| Context | Convention | Examples |
|
||||
|---|---|---|
|
||||
| Python source files | `snake_case.py` | `download_chain.py`, `qbittorrent.py` |
|
||||
| Module package directories | `snake_case/` | `qbittorrent/`, `synologychat/` |
|
||||
| Test files | `test_<domain>.py` | `test_download_chain.py`, `test_subscribe_endpoint.py` |
|
||||
| Alembic migrations | Auto-generated by Alembic; do not rename | `20240101_add_column.py` |
|
||||
| Skill directories | `<kebab-case>/` | `transfer-failed-retry/`, `moviepilot-cli/` |
|
||||
|
||||
---
|
||||
|
||||
## Classes
|
||||
|
||||
| Context | Convention | Examples |
|
||||
|---|---|---|
|
||||
| Chain classes | `<Domain>Chain` | `DownloadChain`, `SearchChain`, `SubscribeChain` |
|
||||
| Module classes | `<Backend>Module` | `QbittorrentModule`, `EmbyModule`, `TelegramModule` |
|
||||
| Oper (data access) classes | `<Model>Oper` | `SubscribeOper`, `SystemConfigOper`, `TransferHistoryOper` |
|
||||
| Helper classes | `<Domain>Helper` | `TorrentHelper`, `DirectoryHelper`, `MessageHelper` |
|
||||
| Pydantic schema models | `PascalCase`, noun-focused | `MediaInfo`, `TorrentInfo`, `DownloadingTorrent` |
|
||||
| SQLAlchemy model classes | `PascalCase`, singular noun | `Subscribe`, `TransferHistory`, `SystemConfig` |
|
||||
| Enum classes | `PascalCase` | `MediaType`, `EventType`, `ModuleType` |
|
||||
| Manager classes | `<Domain>Manager` | `ModuleManager`, `PluginManager`, `EventManager` |
|
||||
| General classes | `PascalCase` | `MetaInfo`, `Context`, `ChainBase` |
|
||||
|
||||
---
|
||||
|
||||
## Functions and Methods
|
||||
|
||||
| Context | Convention | Examples |
|
||||
|---|---|---|
|
||||
| All functions and methods | `snake_case` | `get_subscribe`, `run_module`, `on_config_changed` |
|
||||
| Private methods | `_snake_case` (leading underscore) | `_submit_download_added_task`, `_parse_result` |
|
||||
| Event handler methods | `on_<event_name>` or descriptive | `on_transfer_complete`, `handle_config_changed` |
|
||||
| Module interface methods | Match `_ModuleBase` contract | `init_module`, `init_setting`, `get_name`, `get_type`, `test`, `stop` |
|
||||
| Oper methods | Verb + noun | `get`, `add`, `update`, `delete`, `list` |
|
||||
|
||||
---
|
||||
|
||||
## Variables and Parameters
|
||||
|
||||
| Context | Convention | Examples |
|
||||
|---|---|---|
|
||||
| Local variables | `snake_case` | `torrent_info`, `media_type`, `download_dir` |
|
||||
| Instance attributes | `snake_case` | `self.download_history`, `self.config` |
|
||||
| Constants (module-level) | `UPPER_SNAKE_CASE` | `DEFAULT_EVENT_PRIORITY`, `MIN_EVENT_CONSUMER_THREADS` |
|
||||
| Private variables | `_snake_case` (leading underscore) | `_instance`, `_lock` |
|
||||
| Type variables | `PascalCase` with `TypeVar` | `T = TypeVar("T")` |
|
||||
|
||||
---
|
||||
|
||||
## Enums
|
||||
|
||||
| Context | Convention | Examples |
|
||||
|---|---|---|
|
||||
| Enum class name | `PascalCase` | `MediaType`, `TorrentStatus`, `EventType` |
|
||||
| Enum members | `PascalCase` (for complex enums) | `MediaType.MOVIE`, `EventType.TransferComplete` |
|
||||
| String enum values | Match the domain language | `MediaType.MOVIE = '电影'`, `TorrentStatus.TRANSFER = '可转移'` |
|
||||
| `SystemConfigKey` values | Match the config key as a string | `SystemConfigKey.RssUrls = "RssUrls"` |
|
||||
|
||||
---
|
||||
|
||||
## Configuration and Settings
|
||||
|
||||
| Context | Convention | Examples |
|
||||
|---|---|---|
|
||||
| `Settings` / `ConfigModel` fields | `UPPER_SNAKE_CASE` | `API_TOKEN`, `LLM_MODEL`, `QB_HOST` |
|
||||
| `SystemConfigKey` enum members | `PascalCase` | `SystemConfigKey.RssUrls`, `SystemConfigKey.SubscribeFilter` |
|
||||
| Environment variable names | `UPPER_SNAKE_CASE` | `AI_AGENT_ENABLE`, `DB_TYPE` |
|
||||
|
||||
---
|
||||
|
||||
## API Endpoints and Routers
|
||||
|
||||
| Context | Convention | Examples |
|
||||
|---|---|---|
|
||||
| Endpoint function names | `snake_case`, verb-first | `get_subscribe_list`, `add_download`, `delete_history` |
|
||||
| URL path segments | `kebab-case` or `snake_case` matching existing patterns | `/api/v1/subscribe`, `/api/v1/transfer/history` |
|
||||
| Router tags | Match the resource domain name | `"subscribe"`, `"download"`, `"media"` |
|
||||
|
||||
---
|
||||
|
||||
## Anti-Patterns
|
||||
|
||||
| Wrong | Correct |
|
||||
|---|---|
|
||||
| `class downloadchain:` | `class DownloadChain:` |
|
||||
| `class QBModule:` | `class QbittorrentModule:` |
|
||||
| `def GetSubscribe():` | `def get_subscribe():` |
|
||||
| `TORRENT_info = ...` | `torrent_info = ...` |
|
||||
| `def handleConfigChanged():` | `def on_config_changed():` or `def handle_config_changed():` |
|
||||
| `SystemConfigOper().get("RssUrls")` | `SystemConfigOper().get(SystemConfigKey.RssUrls)` |
|
||||
| `class subscribe_oper:` | `class SubscribeOper:` |
|
||||
|
||||
*Last Updated: 2026-05-25*
|
||||
140
docs/rules/08-comment-styles.md
Normal file
140
docs/rules/08-comment-styles.md
Normal file
@@ -0,0 +1,140 @@
|
||||
# 08 — Comments and Documentation Style
|
||||
|
||||
## ⚠️ Mandatory Gate
|
||||
|
||||
All **public classes**, **public methods**, and **public functions** in this project must have Chinese docstrings. Code submitted without compliant docstrings on public interfaces will be **rejected at review**. No exceptions.
|
||||
|
||||
"Public" means anything not prefixed with `_`. This includes all methods on `ChainBase` subclasses, `_ModuleBase` subclasses, Pydantic schema classes, and endpoint functions.
|
||||
|
||||
---
|
||||
|
||||
## Docstring Format
|
||||
|
||||
### Single-line (for simple, obvious descriptions)
|
||||
|
||||
```python
|
||||
def get_name() -> str:
|
||||
"""获取模块名称"""
|
||||
return "Qbittorrent"
|
||||
```
|
||||
|
||||
### Multi-line (for methods with parameters, return values, or non-obvious behavior)
|
||||
|
||||
```python
|
||||
def download(
|
||||
self,
|
||||
context: Context,
|
||||
torrent: TorrentInfo,
|
||||
download_dir: Path,
|
||||
) -> Optional[str]:
|
||||
"""
|
||||
添加下载任务到下载器。
|
||||
|
||||
:param context: 当前媒体上下文,包含识别结果和种子选择信息
|
||||
:param torrent: 要下载的种子信息
|
||||
:param download_dir: 目标保存目录
|
||||
:return: 成功时返回下载任务 ID,失败时返回 None
|
||||
"""
|
||||
...
|
||||
```
|
||||
|
||||
### Class docstrings
|
||||
|
||||
```python
|
||||
class DownloadChain(ChainBase):
|
||||
"""
|
||||
下载处理链,负责协调搜索结果的种子选择、下载器调度和下载后处理。
|
||||
"""
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Docstring Language Rule
|
||||
|
||||
- **Default:** Chinese.
|
||||
- **Exception:** If the surrounding file is entirely and consistently in English, match the local style.
|
||||
- Do not mix languages within a single docstring. Pick one and stay consistent for the whole file.
|
||||
|
||||
---
|
||||
|
||||
## Inline Comments
|
||||
|
||||
**Only add an inline or block comment when the WHY is non-obvious.** Good reasons to add a comment:
|
||||
|
||||
- A hidden external constraint (e.g., "this API returns stale data for up to 60 seconds after update")
|
||||
- A subtle invariant the code must maintain
|
||||
- A workaround for a specific third-party bug
|
||||
- Call ordering or initialization requirements that are not apparent from the code
|
||||
- Compatibility reasons with a specific client version or protocol
|
||||
|
||||
**Do not add a comment when:**
|
||||
|
||||
- The code already explains itself through well-named identifiers
|
||||
- The comment would just restate what the code does in words
|
||||
- The logic is straightforward branching or assignment
|
||||
|
||||
---
|
||||
|
||||
## Correct Examples
|
||||
|
||||
```python
|
||||
# qBittorrent API 在添加种子后立即查询时可能返回空,需要短暂等待
|
||||
time.sleep(0.5)
|
||||
result = self.client.get_torrent(hash_id)
|
||||
```
|
||||
|
||||
```python
|
||||
# 此处必须先检查 module 是否已初始化,否则多线程并发调用时 get_instances() 可能返回空列表
|
||||
if not self._initialized:
|
||||
self.init_module()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Incorrect Examples
|
||||
|
||||
```python
|
||||
# 获取订阅列表 ← 这只是在重述代码,不需要
|
||||
subscribes = SubscribeOper().list()
|
||||
|
||||
# 如果 result 为 None 则返回 ← 无意义
|
||||
if result is None:
|
||||
return None
|
||||
|
||||
# change starts here ← 噪音,禁止
|
||||
# fix: handle edge case ← 噪音,改成提交信息里写
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Comment Placement
|
||||
|
||||
- Place block comments **above** the code they describe, not on the same line.
|
||||
- Use same-line end-of-line comments only for very short clarifications (e.g., unit of a constant).
|
||||
- For long explanations, prefer a block comment above the code rather than a multiline end-of-line comment.
|
||||
|
||||
```python
|
||||
# 优先使用已有的下载目录映射,避免重复计算路径
|
||||
effective_dir = self._resolve_download_dir(torrent) or download_dir
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Stale Comment Rule
|
||||
|
||||
When modifying code, update or remove any comment that no longer accurately describes the implementation. A stale comment is worse than no comment — it actively misleads future readers.
|
||||
|
||||
---
|
||||
|
||||
## Prohibited Patterns
|
||||
|
||||
| Pattern | Why |
|
||||
|---|---|
|
||||
| `# change starts here` / `# change ends here` | Editorial noise; belongs in git history, not source |
|
||||
| `# TODO` without context or assignee | Accepted only when the deferral is genuinely unavoidable and the reason is documented |
|
||||
| `# FIXME` left in submitted code | Fix it now or document exactly why it cannot be fixed |
|
||||
| `# this is important` | Every line of code is important; this adds nothing |
|
||||
| Commented-out dead code | Delete it; git history preserves it |
|
||||
| Docstrings in English on new public interfaces | Violation of the mandatory Chinese docstring gate |
|
||||
|
||||
*Last Updated: 2026-05-25*
|
||||
174
docs/rules/09-external-response.md
Normal file
174
docs/rules/09-external-response.md
Normal file
@@ -0,0 +1,174 @@
|
||||
# 09 — External APIs, Protocols, and Responses
|
||||
|
||||
## HTTP Client Conventions
|
||||
|
||||
**Rule:** All outbound HTTP requests must go through `RequestUtils` from `app/utils/http.py`. Do not use `requests`, `httpx`, or `aiohttp` directly.
|
||||
|
||||
`RequestUtils` handles:
|
||||
- Proxy configuration (from `settings.PROXY_*`)
|
||||
- Timeouts
|
||||
- SSL verification settings
|
||||
- User-Agent headers
|
||||
- Retry logic
|
||||
|
||||
```python
|
||||
from app.utils.http import RequestUtils
|
||||
|
||||
res = RequestUtils(
|
||||
ua=settings.USER_AGENT,
|
||||
proxies=settings.PROXY,
|
||||
timeout=30,
|
||||
).get_res(url="https://api.example.com/data")
|
||||
|
||||
if res and res.status_code == 200:
|
||||
data = res.json()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Response Format — REST API
|
||||
|
||||
All REST API responses use Pydantic schema models from `app/schemas/`. Do not return raw `dict` objects from endpoints.
|
||||
|
||||
### Standard Response Patterns
|
||||
|
||||
```python
|
||||
# Success with data
|
||||
from app.schemas.response import Response
|
||||
|
||||
return Response(success=True, message="", data=result)
|
||||
|
||||
# Success without data
|
||||
return Response(success=True, message="操作成功")
|
||||
|
||||
# Error
|
||||
return Response(success=False, message="错误原因描述")
|
||||
```
|
||||
|
||||
### List Responses
|
||||
|
||||
For paginated lists, follow the pattern of existing endpoint files. Check `app/api/endpoints/` for examples matching the resource domain.
|
||||
|
||||
### Error Responses (Endpoint Layer Only)
|
||||
|
||||
In endpoints, raise `HTTPException` for request-level errors:
|
||||
|
||||
```python
|
||||
from fastapi import HTTPException
|
||||
|
||||
raise HTTPException(status_code=404, detail="Resource not found")
|
||||
raise HTTPException(status_code=403, detail="Permission denied")
|
||||
```
|
||||
|
||||
Do not raise `HTTPException` in chain or module code. Chains and modules return `None` or domain-level error objects on failure; the endpoint translates that into an HTTP response.
|
||||
|
||||
---
|
||||
|
||||
## Error Handling by Layer
|
||||
|
||||
| Layer | On external API failure |
|
||||
|---|---|
|
||||
| Module | Log the error, return `None` or `(False, "error message")` tuple |
|
||||
| Chain | Log the error, return `None` or an appropriate domain object with failure indication |
|
||||
| Endpoint | Translate `None` or failure result into a `Response(success=False, ...)` or `HTTPException` |
|
||||
|
||||
```python
|
||||
# Module layer
|
||||
def test(self) -> Optional[Tuple[bool, str]]:
|
||||
"""测试模块连通性"""
|
||||
try:
|
||||
ok = self.client.ping()
|
||||
return (True, "连接成功") if ok else (False, "连接失败")
|
||||
except Exception as err:
|
||||
logger.error(f"测试连通性失败:{str(err)}")
|
||||
return (False, str(err))
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## MCP Protocol
|
||||
|
||||
MoviePilot exposes an MCP (Model Context Protocol) interface for AI agent integration.
|
||||
|
||||
- **Transport:** HTTP, JSON-RPC 2.0
|
||||
- **Base path:** `/api/v1/mcp`
|
||||
- **Protocol versions supported:** `2025-11-25`, `2025-06-18`, `2024-11-05`
|
||||
|
||||
### Authentication
|
||||
|
||||
```
|
||||
Header: X-API-KEY: <api_key>
|
||||
Query: ?apikey=<api_key>
|
||||
```
|
||||
|
||||
### Supported Methods
|
||||
|
||||
| Method | Description |
|
||||
|---|---|
|
||||
| `initialize` | Initialize session, negotiate protocol version and capabilities |
|
||||
| `notifications/initialized` | Client confirmation of initialization |
|
||||
| `tools/list` | List all available tools |
|
||||
| `tools/call` | Invoke a specific tool |
|
||||
| `ping` | Connection liveness check |
|
||||
|
||||
### Error Codes
|
||||
|
||||
| Code | Message | Meaning |
|
||||
|---|---|---|
|
||||
| -32700 | Parse error | Malformed JSON |
|
||||
| -32600 | Invalid Request | Invalid JSON-RPC request structure |
|
||||
| -32601 | Method not found | Unknown method |
|
||||
| -32602 | Invalid params | Parameter validation failure |
|
||||
| -32002 | Session not found | Session does not exist or has expired |
|
||||
| -32003 | Not initialized | Session has not completed initialization |
|
||||
| -32603 | Internal error | Server-side error |
|
||||
|
||||
### Tool Response Format
|
||||
|
||||
MCP tools return structured content. Errors must use the JSON-RPC error object format, not HTTP status codes.
|
||||
|
||||
---
|
||||
|
||||
## Notification and Messaging
|
||||
|
||||
Internal notifications use the `Notification` schema and the event system:
|
||||
|
||||
```python
|
||||
from app.schemas import Notification
|
||||
from app.schemas.types import NotificationType, MessageChannel
|
||||
from app.core.event import eventmanager
|
||||
from app.schemas.types import EventType
|
||||
|
||||
eventmanager.send_event(
|
||||
EventType.NoticeMessage,
|
||||
{
|
||||
"channel": MessageChannel.Telegram,
|
||||
"type": NotificationType.Download,
|
||||
"title": "下载成功",
|
||||
"text": f"{media_name} 已添加到下载队列",
|
||||
"image": poster_url,
|
||||
}
|
||||
)
|
||||
```
|
||||
|
||||
Do not call message channel modules directly from chain code. Use the event bus to decouple senders from channels.
|
||||
|
||||
---
|
||||
|
||||
## Media Metadata API Conventions
|
||||
|
||||
When calling TMDB, TheTVDB, Douban, or Bangumi via the module layer:
|
||||
|
||||
- Always check the module return for `None` before using the result — modules return `None` when the backend is not configured or the request fails.
|
||||
- Cache responses using `FileCache` / `AsyncFileCache` where the result is stable and repeated requests would be expensive.
|
||||
- Return domain objects (`MediaInfo`, `TmdbEpisode`, `MediaPerson`, etc.) from modules, never raw API response dicts.
|
||||
|
||||
---
|
||||
|
||||
## Webhook Handling
|
||||
|
||||
Webhook payloads arrive at `app/api/endpoints/webhook.py` and are dispatched via `eventmanager.send_event(EventType.WebhookMessage, ...)`. Processing logic lives in the chain layer (`app/chain/webhook.py`).
|
||||
|
||||
Do not add webhook-specific business logic directly in the endpoint. The endpoint parses the payload and fires the event; the chain handles the response.
|
||||
|
||||
*Last Updated: 2026-05-25*
|
||||
177
docs/rules/10-data-and-persistent.md
Normal file
177
docs/rules/10-data-and-persistent.md
Normal file
@@ -0,0 +1,177 @@
|
||||
# 10 — Data and Persistent Management
|
||||
|
||||
## Database Models
|
||||
|
||||
**Location:** `app/db/models/`
|
||||
|
||||
Models are SQLAlchemy declarative classes. Each model maps to one database table.
|
||||
|
||||
| Model | Table Domain |
|
||||
|---|---|
|
||||
| `Subscribe` | Media subscriptions |
|
||||
| `SubscribeHistory` | Completed subscription records |
|
||||
| `TransferHistory` | File transfer history |
|
||||
| `DownloadHistory` / `DownloadFiles` | Download task history and file list |
|
||||
| `MediaServerItem` | Media server library item cache |
|
||||
| `SystemConfig` | Runtime key-value configuration store |
|
||||
| `UserConfig` | Per-user configuration store |
|
||||
| `User` | User accounts |
|
||||
| `Site` / `SiteIcon` / `SiteStatistic` / `SiteUserData` | Torrent site records and statistics |
|
||||
| `Message` | Message log |
|
||||
| `PluginData` | Plugin-persisted data |
|
||||
| `PassKey` | Passkey authentication records |
|
||||
| `Workflow` | Workflow definitions |
|
||||
|
||||
---
|
||||
|
||||
## Alembic Migrations
|
||||
|
||||
**Location:** `database/versions/`
|
||||
|
||||
**Rule:** Any change to a SQLAlchemy model schema (adding a column, renaming a column, changing a column type, adding a table, removing a table) **requires a new Alembic migration script**. Never update models without a corresponding migration.
|
||||
|
||||
**Generating a migration:**
|
||||
|
||||
```bash
|
||||
# Auto-generate from model diff
|
||||
alembic revision --autogenerate -m "describe the change"
|
||||
|
||||
# Create a blank migration for manual SQL
|
||||
alembic revision -m "describe the change"
|
||||
```
|
||||
|
||||
**Review the auto-generated migration before committing** — auto-generation can miss nullable changes, index modifications, or SQLite-incompatible operations.
|
||||
|
||||
---
|
||||
|
||||
## Data Access Layer (Oper Pattern)
|
||||
|
||||
**Location:** `app/db/`
|
||||
|
||||
Each model has a corresponding `*_oper.py` file containing the data access class. Do not write SQLAlchemy queries directly in chain, module, or endpoint code.
|
||||
|
||||
| Oper Class | File |
|
||||
|---|---|
|
||||
| `SubscribeOper` | `subscribe_oper.py` |
|
||||
| `SystemConfigOper` | `systemconfig_oper.py` |
|
||||
| `TransferHistoryOper` | `transferhistory_oper.py` |
|
||||
| `DownloadHistoryOper` | `downloadhistory_oper.py` |
|
||||
| `MediaServerOper` | `mediaserver_oper.py` |
|
||||
| `UserOper` | `user_oper.py` |
|
||||
| `UserConfigOper` | `userconfig_oper.py` |
|
||||
| `MessageOper` | `message_oper.py` |
|
||||
| `SiteOper` | `site_oper.py` |
|
||||
| `PluginDataOper` | `plugindata_oper.py` |
|
||||
| `WorkflowOper` | `workflow_oper.py` |
|
||||
|
||||
**Standard Oper method conventions:**
|
||||
|
||||
```python
|
||||
oper = SubscribeOper()
|
||||
subscribe = oper.get(sid=1) # Get by primary key or filter
|
||||
subscribes = oper.list() # List all
|
||||
oper.add(Subscribe(...)) # Insert
|
||||
oper.update(sid=1, name="New Name") # Update by key
|
||||
oper.delete(sid=1) # Delete by key
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## SystemConfig — Runtime Configuration
|
||||
|
||||
**Purpose:** Runtime business configuration that is user-editable, persisted in the database, and survives application restarts.
|
||||
|
||||
**Enum:** `SystemConfigKey` in `app/schemas/types.py`
|
||||
|
||||
**Oper:** `SystemConfigOper` in `app/db/systemconfig_oper.py`
|
||||
|
||||
```python
|
||||
from app.schemas.types import SystemConfigKey
|
||||
from app.db.systemconfig_oper import SystemConfigOper
|
||||
|
||||
oper = SystemConfigOper()
|
||||
|
||||
# Read
|
||||
rss_urls = oper.get(SystemConfigKey.RssUrls)
|
||||
|
||||
# Write
|
||||
oper.set(SystemConfigKey.RssUrls, ["https://example.com/rss"])
|
||||
```
|
||||
|
||||
**Rule:** Never use raw string literals as `SystemConfig` keys. Always define a new `SystemConfigKey` enum entry first. Raw string key lookups are not searchable and cannot be refactored safely.
|
||||
|
||||
---
|
||||
|
||||
## UserConfig — Per-User Configuration
|
||||
|
||||
**Purpose:** Settings that differ per user account. Uses `UserConfigOper`.
|
||||
|
||||
```python
|
||||
from app.db.userconfig_oper import UserConfigOper
|
||||
|
||||
oper = UserConfigOper()
|
||||
value = oper.get(user_id=1, key="notification_enabled")
|
||||
oper.set(user_id=1, key="notification_enabled", value=True)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Settings / Environment Configuration
|
||||
|
||||
**Purpose:** Deployment-level, environment-level, and startup-time configuration such as ports, paths, proxies, switches, API keys, and third-party service addresses.
|
||||
|
||||
**Location:** `ConfigModel` and `Settings` in `app/core/config.py`
|
||||
|
||||
These values are read from environment variables (or `.moviepilot.env`) at startup and are immutable at runtime. They are not stored in the database.
|
||||
|
||||
**Access:**
|
||||
|
||||
```python
|
||||
from app.core.config import settings
|
||||
|
||||
host = settings.QB_HOST
|
||||
port = settings.QB_PORT
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Caching
|
||||
|
||||
### FileCache / AsyncFileCache
|
||||
|
||||
**Location:** `app/core/cache.py`
|
||||
|
||||
Used to cache expensive external API responses to disk. Cache entries have a configurable TTL.
|
||||
|
||||
```python
|
||||
from app.core.cache import FileCache, fresh
|
||||
|
||||
cache = FileCache(cache_name="tmdb", ttl=3600)
|
||||
|
||||
@fresh(cache=cache, key_func=lambda tmdb_id: f"movie_{tmdb_id}")
|
||||
def get_movie_detail(tmdb_id: int) -> dict:
|
||||
return self._tmdb_client.get_movie(tmdb_id)
|
||||
```
|
||||
|
||||
### Redis (Optional)
|
||||
|
||||
When `REDIS_HOST` is configured, `app/modules/redis/` provides a distributed cache backend. Prefer `FileCache` for single-node deployments.
|
||||
|
||||
---
|
||||
|
||||
## Data Lifecycle Rules
|
||||
|
||||
- **TransferHistory:** Records are inserted after every successful file transfer. Do not delete records without user confirmation.
|
||||
- **DownloadHistory:** Records are inserted when a download task is added. Linked `DownloadFiles` records track individual files within a torrent.
|
||||
- **SystemConfig:** Values may be read and written freely at runtime. Changes to watched config keys trigger `on_config_changed()` on registered classes via `ConfigReloadMixin`.
|
||||
- **MediaServerItem:** This is a cache of the remote media server library. It is refreshed on media server sync events and can be safely cleared and rebuilt.
|
||||
|
||||
---
|
||||
|
||||
## Sensitive Data Handling
|
||||
|
||||
- Never log database record contents that include personal data (user credentials, passkeys, API tokens).
|
||||
- `settings.API_TOKEN` and other secret fields must not be included in log output or API responses.
|
||||
- The `config list --show-secrets` flag exists specifically to gate secret visibility in the CLI.
|
||||
|
||||
*Last Updated: 2026-05-25*
|
||||
139
docs/rules/11-quality-and-security.md
Normal file
139
docs/rules/11-quality-and-security.md
Normal file
@@ -0,0 +1,139 @@
|
||||
# 11 — Code Quality and Security
|
||||
|
||||
## Testing Requirements
|
||||
|
||||
### What to Run
|
||||
|
||||
```bash
|
||||
# Minimum: run tests directly related to the change
|
||||
pytest tests/test_<domain>.py
|
||||
|
||||
# If the change affects common modules, startup flow, CLI, or agent runtime
|
||||
pytest
|
||||
```
|
||||
|
||||
### When to Expand Scope
|
||||
|
||||
Run the full test suite when changing:
|
||||
- `app/core/` — config, event system, module manager, plugin manager
|
||||
- `app/chain/__init__.py` — chain base class
|
||||
- `app/modules/__init__.py` — module base class
|
||||
- `app/main.py` — application startup
|
||||
- The CLI entrypoint (`moviepilot`)
|
||||
- Agent runtime (`app/agent/`)
|
||||
- Any shared schema in `app/schemas/types.py`
|
||||
|
||||
### Honest Reporting
|
||||
|
||||
- If a task only changes documentation, state explicitly that tests were not run.
|
||||
- Do not claim "all tests pass" unless you ran them.
|
||||
- Do not describe unexecuted checks as completed.
|
||||
|
||||
### Writing New Tests
|
||||
|
||||
- When fixing a bug, prefer adding a test that reproduces it first.
|
||||
- When adding a feature, add at minimum the smallest useful test coverage.
|
||||
- Test files go in `tests/`, named `test_<domain>.py`.
|
||||
- Use the patterns established in adjacent test files (fixtures, mock patterns, assertion style).
|
||||
- Agent-related tests are under `tests/test_agent_*.py`. Integration-style tests may be in `tests/cases/` or `tests/manual/`.
|
||||
|
||||
---
|
||||
|
||||
## Static Analysis
|
||||
|
||||
```bash
|
||||
pylint app/
|
||||
```
|
||||
|
||||
- After any Python code change, ensure no new **error-level** pylint issues are introduced.
|
||||
- Warning-level issues in new code should be minimized but are not an absolute gate for submission.
|
||||
- Do not suppress pylint warnings with `# pylint: disable` without a documented reason.
|
||||
|
||||
---
|
||||
|
||||
## Dependency Security Scan
|
||||
|
||||
```bash
|
||||
safety check -r requirements.txt --policy-file=safety.policy.yml
|
||||
```
|
||||
|
||||
- Run after every change to `requirements.txt`.
|
||||
- No new high-severity vulnerabilities may be introduced.
|
||||
- If a vulnerability cannot be patched immediately, document it explicitly in the PR description.
|
||||
|
||||
---
|
||||
|
||||
## Authentication and Authorization
|
||||
|
||||
### API Authentication
|
||||
|
||||
All REST and MCP API endpoints require authentication. The project supports two mechanisms:
|
||||
|
||||
| Method | Format |
|
||||
|---|---|
|
||||
| Request header | `X-API-KEY: <api_key>` |
|
||||
| Query parameter | `?apikey=<api_key>` |
|
||||
|
||||
The `API_TOKEN` value in `settings` is the source of truth. It is set at initialization and never exposed in logs or API responses.
|
||||
|
||||
### Endpoint Authorization
|
||||
|
||||
- Use the existing FastAPI dependency functions (e.g., `get_current_user`, `get_current_active_superuser`) — check `app/api/endpoints/` for usage patterns.
|
||||
- Do not add manual token parsing inside endpoint functions. Always use the project's dependency injection.
|
||||
- Superuser-only operations must explicitly require the superuser dependency.
|
||||
|
||||
---
|
||||
|
||||
## Input Validation
|
||||
|
||||
- Validate user input at the **endpoint layer only**, using Pydantic models.
|
||||
- Do not duplicate validation logic in chain or module code. Trust that the endpoint has already validated what it passes down.
|
||||
- For external API responses, validate using Pydantic models or explicit `None` checks before accessing fields.
|
||||
|
||||
---
|
||||
|
||||
## Secrets Management
|
||||
|
||||
- Never hardcode secrets (API keys, passwords, tokens) in source code.
|
||||
- All secrets are configured via environment variables or `.moviepilot.env` and accessed through `settings`.
|
||||
- Never log or serialize `settings.API_TOKEN`, `settings.DB_PASSWORD`, or any field with `Secret` in its name.
|
||||
- Do not commit `.moviepilot.env`, `*.db`, or any file under `config/` — these are local runtime state.
|
||||
|
||||
---
|
||||
|
||||
## SQL Injection Prevention
|
||||
|
||||
- All database access goes through SQLAlchemy ORM via the `*_oper.py` classes. No raw SQL string construction.
|
||||
- If a raw SQL query is ever genuinely necessary, use SQLAlchemy's `text()` with parameterized binds — never string interpolation.
|
||||
|
||||
---
|
||||
|
||||
## XSS and Injection in Notifications
|
||||
|
||||
- When constructing notification messages that include user-provided data (media titles, filenames, usernames), treat those values as untrusted strings.
|
||||
- Do not render user data in HTML contexts without escaping. Notification channels that render HTML (e.g., Telegram with `parse_mode=HTML`) must escape user-controlled strings.
|
||||
|
||||
---
|
||||
|
||||
## File Path Security
|
||||
|
||||
- Use `pathlib.Path` for all file path operations.
|
||||
- Never construct file paths by concatenating user-provided strings.
|
||||
- When transferring files to a user-configured path, verify the destination is within an allowed base directory before writing.
|
||||
|
||||
---
|
||||
|
||||
## Pre-Submission Checklist
|
||||
|
||||
Before marking any task as complete:
|
||||
|
||||
- [ ] Related pytest tests pass
|
||||
- [ ] No new pylint error-level issues in `pylint app/`
|
||||
- [ ] If dependencies changed: `pip-compile requirements.in` was run and `safety check` passes
|
||||
- [ ] If CLI behavior changed: `docs/cli.md` and related tests are updated
|
||||
- [ ] If MCP/API behavior changed: `docs/mcp-api.md` and related skill files are updated
|
||||
- [ ] If database schema changed: a new Alembic migration exists under `database/versions/`
|
||||
- [ ] No secrets are included in code, logs, or committed files
|
||||
- [ ] Public classes and methods have Chinese docstrings
|
||||
|
||||
*Last Updated: 2026-05-25*
|
||||
123
docs/rules/12-collaboration-and-distribution.md
Normal file
123
docs/rules/12-collaboration-and-distribution.md
Normal file
@@ -0,0 +1,123 @@
|
||||
# 12 — Collaboration, Versioning, Build, and Release
|
||||
|
||||
## Commit Conventions
|
||||
|
||||
This project uses **Conventional Commits**. The release workflow parses commit messages to categorize changelog entries. This is not stylistic — it is functional.
|
||||
|
||||
### Format
|
||||
|
||||
```
|
||||
<type>(<optional scope>): <description>
|
||||
|
||||
[optional body]
|
||||
|
||||
[optional footer]
|
||||
```
|
||||
|
||||
### Commit Types
|
||||
|
||||
| Type | When to use |
|
||||
|---|---|
|
||||
| `feat` | A new feature visible to users |
|
||||
| `fix` | A bug fix |
|
||||
| `docs` | Documentation only changes |
|
||||
| `chore` | Maintenance, dependency updates, tooling changes |
|
||||
| `refactor` | Code restructuring without behavior change |
|
||||
| `test` | Adding or modifying tests |
|
||||
| `ci` | CI/CD pipeline changes |
|
||||
| `perf` | Performance improvements |
|
||||
|
||||
### Examples
|
||||
|
||||
```
|
||||
feat: support MiniMax audio provider
|
||||
fix: sign media server image proxy URLs
|
||||
docs: add MCP client configuration examples
|
||||
chore: upgrade pydantic to 2.9.0
|
||||
refactor: extract transfer path resolution into helper
|
||||
test: add subscribe endpoint validation tests
|
||||
ci: improve docker build cache
|
||||
```
|
||||
|
||||
### Rules
|
||||
|
||||
- **Only create a commit when the user explicitly asks for one.**
|
||||
- Keep the subject line under 72 characters.
|
||||
- Use the imperative mood in the subject line ("add", "fix", "remove", not "added", "fixed", "removed").
|
||||
- If a commit introduces a breaking change, append `!` after the type and include `BREAKING CHANGE:` in the footer.
|
||||
|
||||
---
|
||||
|
||||
## Branch Policy
|
||||
|
||||
- Do not casually create, rename, or delete branches without user instruction.
|
||||
- The main development branch is the project default — check `git branch` rather than assuming it is `main` or `master`.
|
||||
- Feature work lives on dedicated branches and is merged via pull request.
|
||||
- Do not force-push to shared branches.
|
||||
|
||||
---
|
||||
|
||||
## Version Numbers
|
||||
|
||||
- Do not casually change version numbers in `version.py` or related files.
|
||||
- Version changes are part of the release workflow and are only made when the task explicitly involves a release.
|
||||
- The `FRONTEND_VERSION` field in `version.py` controls which frontend release the CLI and Docker build will download. Only update it as part of a coordinated frontend release.
|
||||
|
||||
---
|
||||
|
||||
## Docker Build and Release
|
||||
|
||||
- The primary Docker image bundles the backend (Python app), frontend static files (from `public/`), and resource data.
|
||||
- Docker build and release are managed by CI. Do not manually trigger or alter the Docker release flow unless the task explicitly requires it.
|
||||
- If a Dockerfile change is needed, update `Dockerfile` and verify the build locally before submitting.
|
||||
|
||||
---
|
||||
|
||||
## CI/CD
|
||||
|
||||
- CI runs on every push and pull request. The pipeline typically includes:
|
||||
- Dependency installation
|
||||
- pytest test suite
|
||||
- pylint static analysis
|
||||
- Docker image build (on main branch or tags)
|
||||
- Do not merge code that fails CI unless there is an explicit, documented reason and user approval.
|
||||
|
||||
---
|
||||
|
||||
## Pull Request Guidelines
|
||||
|
||||
- Keep PRs focused on a single concern. Separate refactors, features, and bug fixes into distinct PRs when practical.
|
||||
- Include in the PR description:
|
||||
- What changed and why
|
||||
- How the change was validated
|
||||
- Any known risks or compatibility impact
|
||||
- Migration steps if config or database schema changed
|
||||
- Tag the PR with the appropriate label (`bug`, `feature`, `docs`, `chore`).
|
||||
|
||||
---
|
||||
|
||||
## Dependency Release Process
|
||||
|
||||
When updating a dependency:
|
||||
|
||||
1. Update `requirements.in` with the new version constraint.
|
||||
2. Run `pip-compile requirements.in` to regenerate `requirements.txt`.
|
||||
3. Run `safety check -r requirements.txt --policy-file=safety.policy.yml`.
|
||||
4. Run the full test suite: `pytest`.
|
||||
5. Commit both `requirements.in` and `requirements.txt` together.
|
||||
|
||||
---
|
||||
|
||||
## Local CLI Release
|
||||
|
||||
The `moviepilot` CLI is the local-mode entrypoint. Its update path is:
|
||||
|
||||
```bash
|
||||
moviepilot update all # updates backend + frontend + resources
|
||||
moviepilot update backend # git pull + reinstall deps
|
||||
moviepilot update frontend
|
||||
```
|
||||
|
||||
Bootstrap installer changes live in `scripts/bootstrap-local.sh`. Only modify this script if the task explicitly involves the bootstrap flow.
|
||||
|
||||
*Last Updated: 2026-05-25*
|
||||
107
docs/rules/README.md
Normal file
107
docs/rules/README.md
Normal file
@@ -0,0 +1,107 @@
|
||||
# Documentation Hub
|
||||
|
||||
This repository maintains a structured documentation library covering the full development lifecycle. All rule documents live in the `docs/rules/` directory. This index maps each file to its technical domain and intended reader.
|
||||
|
||||
---
|
||||
|
||||
## Technical Document Index
|
||||
|
||||
### Section I: Foundation and Environment
|
||||
|
||||
* **01 Project Overview**
|
||||
* File: `01-project-overview.md`
|
||||
* Scope: System goals, business domain, deployment models, and what is and is not in this repository.
|
||||
|
||||
* **02 Tech Stack**
|
||||
* File: `02-tech-stack.md`
|
||||
* Scope: Frameworks, languages, libraries, runtime environments, and third-party integrations.
|
||||
|
||||
* **03 Commands**
|
||||
* File: `03-commands.md`
|
||||
* Scope: CLI reference, development triggers, testing commands, linting, and dependency management.
|
||||
|
||||
### Section II: Architecture and Logic
|
||||
|
||||
* **04 Design Patterns**
|
||||
* File: `04-design-patterns.md`
|
||||
* Scope: Project-specific structural, creational, and behavioral patterns: Module, Chain, Event, Oper, Config Reload, Singleton.
|
||||
|
||||
* **05 Architecture and Modules**
|
||||
* File: `05-architecture.md`
|
||||
* Scope: Layer boundaries, dependency directions, module categories, and the canonical call graph.
|
||||
|
||||
* **09 External APIs, Protocols, and Responses**
|
||||
* File: `09-external-response.md`
|
||||
* Scope: HTTP client conventions, MCP protocol, standardized response formats, and error handling by layer.
|
||||
|
||||
* **10 Data and Persistent Management**
|
||||
* File: `10-data-and-persistent.md`
|
||||
* Scope: SQLAlchemy models, Alembic migrations, Oper access layer, SystemConfig, caching patterns.
|
||||
|
||||
### Section III: Implementation Standards
|
||||
|
||||
* **06 Code Standards and Style**
|
||||
* File: `06-code-styles.md`
|
||||
* Scope: Type annotations, Pydantic usage, async patterns, imports, formatting, and error handling rules.
|
||||
|
||||
* **07 Naming Conventions**
|
||||
* File: `07-naming-conventions.md`
|
||||
* Scope: Strict taxonomy for files, classes, functions, constants, and schema models.
|
||||
|
||||
* **08 Comments and Documentation Style**
|
||||
* File: `08-comment-styles.md`
|
||||
* Scope: Chinese docstring requirements, inline comment rules, and prohibited comment anti-patterns.
|
||||
|
||||
### Section IV: Quality and Governance
|
||||
|
||||
* **11 Code Quality and Security**
|
||||
* File: `11-quality-and-security.md`
|
||||
* Scope: Testing requirements, pylint gates, safety scans, authentication patterns, and input validation rules.
|
||||
|
||||
* **12 Collaboration, Versioning, Build, and Release**
|
||||
* File: `12-collaboration-and-distribution.md`
|
||||
* Scope: Conventional Commits, branch policy, release workflow, Docker build, and version management.
|
||||
|
||||
---
|
||||
|
||||
## Reader Persona Guidance
|
||||
|
||||
### Core Developers and Implementers
|
||||
|
||||
Developers actively writing or modifying features should follow this reading path:
|
||||
|
||||
1. **07 Naming Conventions** — establishes the lexicon for the feature.
|
||||
2. **06 Code Standards** — ensures linting and logic compliance.
|
||||
3. **04 Design Patterns** — identifies the correct structural approach.
|
||||
4. **03 Commands** — required for local execution and validation.
|
||||
|
||||
### System Architects and Reviewers
|
||||
|
||||
Personnel focused on system integrity and long-term maintenance:
|
||||
|
||||
1. **05 Architecture and Modules** — for verifying structural boundaries.
|
||||
2. **10 Data and Persistent Management** — for auditing data integrity and storage efficiency.
|
||||
3. **09 External APIs** — for reviewing integration security and protocol compliance.
|
||||
4. **11 Code Quality and Security** — for establishing the PR approval baseline.
|
||||
|
||||
### Operations and Release Engineers
|
||||
|
||||
Those managing the application lifecycle post-development:
|
||||
|
||||
1. **12 Collaboration and Versioning** — for release tags and branch management.
|
||||
2. **02 Tech Stack** — for environment provisioning and dependency management.
|
||||
3. **11 Code Quality and Security** — for verifying deployment-ready security posture.
|
||||
|
||||
---
|
||||
|
||||
## Document Interconnectivity
|
||||
|
||||
* **Architecture (05)** references **Code Standards (06)** for layer isolation and module boundary rules.
|
||||
* **Naming Conventions (07)** works in tandem with **Comment Styles (08)** to define overall code readability.
|
||||
* **External APIs (09)** relies on **Tech Stack (02)** for transport layer specifications and HTTP client selection.
|
||||
* **Data Management (10)** is governed by **Quality and Security (11)** for sensitive data handling requirements.
|
||||
* **Design Patterns (04)** is the implementation reference for decisions documented in **Architecture (05)**.
|
||||
|
||||
---
|
||||
|
||||
*Last Updated: 2026-05-25*
|
||||
@@ -126,7 +126,7 @@ static SOURCE_PATTERN: Lazy<Regex> = Lazy::new(|| {
|
||||
});
|
||||
static EFFECT_PATTERN: Lazy<Regex> = Lazy::new(|| {
|
||||
RegexBuilder::new(
|
||||
r"(^SDR$|^HDR\d*$|^DOLBY$|^DOVI$|^DV$|^3D$|^REPACK$|^HLG$|^HDR10(\+|Plus)$|^EDR$|^HQ$)",
|
||||
r"(^SDR$|^HDR\d*$|^DOLBY$|^DOVI$|^DV$|^3D$|^REPACK$|^HLG$|^HDR10(\+|Plus)$|^HDR10P$|^VIVID$|^EDR$|^HQ$)",
|
||||
)
|
||||
.case_insensitive(true)
|
||||
.build()
|
||||
|
||||
@@ -2,10 +2,23 @@ import socket
|
||||
from unittest import TestCase
|
||||
from unittest.mock import patch
|
||||
|
||||
from app.utils.security import SecurityUtils
|
||||
from app.utils.security import (
|
||||
SecurityUtils,
|
||||
_dns_inflight_locks,
|
||||
_dns_negative_cache,
|
||||
_dns_positive_cache,
|
||||
)
|
||||
|
||||
|
||||
class SecurityUtilsTest(TestCase):
|
||||
def setUp(self) -> None:
|
||||
"""
|
||||
每个用例前清空 DNS TTL 缓存与 in-flight 锁,避免跨用例状态污染。
|
||||
"""
|
||||
_dns_positive_cache.clear()
|
||||
_dns_negative_cache.clear()
|
||||
_dns_inflight_locks.clear()
|
||||
|
||||
def test_signed_url_roundtrip_returns_clean_url(self):
|
||||
"""
|
||||
URL 签名验证成功后返回不含签名片段的真实请求地址。
|
||||
@@ -14,7 +27,8 @@ class SecurityUtilsTest(TestCase):
|
||||
|
||||
signed_url = SecurityUtils.sign_url(url)
|
||||
|
||||
self.assertIn("#mp_exp=", signed_url)
|
||||
self.assertIn("#mp_sig=", signed_url)
|
||||
self.assertIn("mp_purpose=image-proxy", signed_url)
|
||||
self.assertEqual(SecurityUtils.verify_signed_url(signed_url), url)
|
||||
self.assertEqual(SecurityUtils.strip_url_signature(signed_url), url)
|
||||
|
||||
@@ -32,19 +46,50 @@ class SecurityUtilsTest(TestCase):
|
||||
|
||||
self.assertIsNone(SecurityUtils.verify_signed_url(tampered_url))
|
||||
|
||||
def test_signed_url_rejects_expired_signature(self):
|
||||
def test_signed_url_is_deterministic_for_same_inputs(self):
|
||||
"""
|
||||
已过期签名不能继续放行私网图片代理请求。
|
||||
相同 URL 与 RESOURCE_SECRET_KEY 多次签名结果必须完全一致,
|
||||
保证浏览器 / Service Worker 缓存能稳定命中。
|
||||
"""
|
||||
with patch("app.utils.security.time.time", return_value=1000):
|
||||
signed_url = SecurityUtils.sign_url(
|
||||
"http://192.168.1.50:8096/Items/abc/Images/Primary",
|
||||
expires_in=10,
|
||||
)
|
||||
url = "http://192.168.1.50:8096/Items/abc/Images/Primary"
|
||||
|
||||
with patch("app.utils.security.time.time", return_value=1011):
|
||||
first = SecurityUtils.sign_url(url)
|
||||
second = SecurityUtils.sign_url(url)
|
||||
|
||||
self.assertEqual(first, second)
|
||||
self.assertEqual(SecurityUtils.verify_signed_url(first), url)
|
||||
|
||||
def test_signed_url_invalidated_after_secret_rotation(self):
|
||||
"""
|
||||
`RESOURCE_SECRET_KEY` 变更(进程重启或运维显式轮换)后旧签名必须作废,
|
||||
作为签名长期有效模型的失效兜底。
|
||||
"""
|
||||
url = "http://192.168.1.50:8096/Items/abc/Images/Primary"
|
||||
|
||||
with patch(
|
||||
"app.utils.security.settings.RESOURCE_SECRET_KEY",
|
||||
"old-secret-value-aaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
):
|
||||
signed_url = SecurityUtils.sign_url(url)
|
||||
self.assertEqual(SecurityUtils.verify_signed_url(signed_url), url)
|
||||
|
||||
with patch(
|
||||
"app.utils.security.settings.RESOURCE_SECRET_KEY",
|
||||
"new-secret-value-bbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
):
|
||||
self.assertIsNone(SecurityUtils.verify_signed_url(signed_url))
|
||||
|
||||
def test_signed_url_rejects_other_purpose(self):
|
||||
"""
|
||||
签名绑定 `purpose`,挪用到其它签名用途必须被拒绝。
|
||||
"""
|
||||
url = "http://192.168.1.50:8096/Items/abc/Images/Primary"
|
||||
signed_url = SecurityUtils.sign_url(url)
|
||||
|
||||
self.assertIsNone(
|
||||
SecurityUtils.verify_signed_url(signed_url, purpose="other-purpose")
|
||||
)
|
||||
|
||||
def test_is_safe_url_keeps_default_allowlist_behavior(self):
|
||||
"""
|
||||
默认 URL 校验保持历史 allowlist 行为,避免影响非代理调用方。
|
||||
@@ -162,3 +207,477 @@ class SecurityUtilsTest(TestCase):
|
||||
block_private=True,
|
||||
)
|
||||
)
|
||||
|
||||
def test_is_safe_url_allows_configured_private_range_after_domain_match(self):
|
||||
"""
|
||||
图片域名命中 allowlist 后,可通过配置允许 TUN fake-ip 等特定非公网网段。
|
||||
"""
|
||||
with patch(
|
||||
"app.utils.security.socket.getaddrinfo",
|
||||
return_value=[
|
||||
(
|
||||
socket.AF_INET,
|
||||
socket.SOCK_STREAM,
|
||||
0,
|
||||
"",
|
||||
("198.18.16.96", 0),
|
||||
)
|
||||
],
|
||||
), patch("app.utils.security.logger.debug") as debug_log:
|
||||
self.assertTrue(
|
||||
SecurityUtils.is_safe_url(
|
||||
"https://img1.doubanio.com/poster.webp",
|
||||
{"doubanio.com"},
|
||||
block_private=True,
|
||||
allowed_private_ranges=["198.18.0.0/15"],
|
||||
)
|
||||
)
|
||||
debug_message = debug_log.call_args.args[0]
|
||||
self.assertIn("ips=198.18.16.96", debug_message)
|
||||
self.assertIn("ranges=198.18.0.0/15", debug_message)
|
||||
|
||||
def test_is_safe_url_blocks_configured_private_range_without_domain_match(self):
|
||||
"""
|
||||
非公网网段例外必须依附域名白名单,不能单独放行任意用户 URL。
|
||||
"""
|
||||
with patch(
|
||||
"app.utils.security.socket.getaddrinfo",
|
||||
return_value=[
|
||||
(
|
||||
socket.AF_INET,
|
||||
socket.SOCK_STREAM,
|
||||
0,
|
||||
"",
|
||||
("198.18.16.96", 0),
|
||||
)
|
||||
],
|
||||
):
|
||||
self.assertFalse(
|
||||
SecurityUtils.is_safe_url(
|
||||
"https://attacker.example.com/poster.webp",
|
||||
{"doubanio.com"},
|
||||
block_private=True,
|
||||
allowed_private_ranges=["198.18.0.0/15"],
|
||||
)
|
||||
)
|
||||
|
||||
def test_is_safe_url_blocks_private_result_outside_configured_range(self):
|
||||
"""
|
||||
仅允许显式配置的非公网网段,其它内网解析结果仍按 SSRF 风险拦截。
|
||||
"""
|
||||
with patch(
|
||||
"app.utils.security.socket.getaddrinfo",
|
||||
return_value=[
|
||||
(
|
||||
socket.AF_INET,
|
||||
socket.SOCK_STREAM,
|
||||
0,
|
||||
"",
|
||||
("10.0.0.8", 0),
|
||||
)
|
||||
],
|
||||
):
|
||||
self.assertFalse(
|
||||
SecurityUtils.is_safe_url(
|
||||
"https://assets.example.com/poster.jpg",
|
||||
{"example.com"},
|
||||
block_private=True,
|
||||
allowed_private_ranges=["198.18.0.0/15"],
|
||||
)
|
||||
)
|
||||
|
||||
def test_is_safe_url_blocks_mixed_allowed_and_disallowed_private_results(self):
|
||||
"""
|
||||
同一域名的解析结果必须全部落在允许网段内,避免部分安全结果掩盖风险地址。
|
||||
"""
|
||||
with patch(
|
||||
"app.utils.security.socket.getaddrinfo",
|
||||
return_value=[
|
||||
(
|
||||
socket.AF_INET,
|
||||
socket.SOCK_STREAM,
|
||||
0,
|
||||
"",
|
||||
("198.18.16.96", 0),
|
||||
),
|
||||
(
|
||||
socket.AF_INET,
|
||||
socket.SOCK_STREAM,
|
||||
0,
|
||||
"",
|
||||
("10.0.0.8", 0),
|
||||
),
|
||||
],
|
||||
):
|
||||
self.assertFalse(
|
||||
SecurityUtils.is_safe_url(
|
||||
"https://assets.example.com/poster.jpg",
|
||||
{"example.com"},
|
||||
block_private=True,
|
||||
allowed_private_ranges=["198.18.0.0/15"],
|
||||
)
|
||||
)
|
||||
|
||||
def test_is_safe_url_async_uses_event_loop_resolver(self):
|
||||
"""
|
||||
异步版本通过事件循环的非阻塞 getaddrinfo 完成 SSRF 校验,
|
||||
且语义与同步版本保持一致:解析到非公网地址时仍然拒绝。
|
||||
"""
|
||||
import asyncio
|
||||
|
||||
async def fake_getaddrinfo(host, *_args, **_kwargs):
|
||||
self.assertEqual(host, "internal.example.com")
|
||||
return [
|
||||
(
|
||||
socket.AF_INET,
|
||||
socket.SOCK_STREAM,
|
||||
0,
|
||||
"",
|
||||
("127.0.0.1", 0),
|
||||
)
|
||||
]
|
||||
|
||||
async def run() -> bool:
|
||||
loop = asyncio.get_running_loop()
|
||||
with patch.object(loop, "getaddrinfo", side_effect=fake_getaddrinfo):
|
||||
return await SecurityUtils.is_safe_url_async(
|
||||
"http://internal.example.com/secret.png",
|
||||
{"example.com"},
|
||||
block_private=True,
|
||||
)
|
||||
|
||||
self.assertFalse(asyncio.run(run()))
|
||||
|
||||
def test_is_safe_url_async_hits_dns_cache(self):
|
||||
"""
|
||||
异步与同步版本共享 DNS TTL 缓存:同步预热后,异步版本不应再发起 DNS 查询。
|
||||
"""
|
||||
import asyncio
|
||||
|
||||
# 先用同步路径预热缓存
|
||||
with patch(
|
||||
"app.utils.security.socket.getaddrinfo",
|
||||
return_value=[
|
||||
(
|
||||
socket.AF_INET,
|
||||
socket.SOCK_STREAM,
|
||||
0,
|
||||
"",
|
||||
("93.184.216.34", 0),
|
||||
)
|
||||
],
|
||||
):
|
||||
self.assertTrue(
|
||||
SecurityUtils.is_safe_url(
|
||||
"https://assets.example.com/poster.jpg",
|
||||
{"example.com"},
|
||||
block_private=True,
|
||||
)
|
||||
)
|
||||
|
||||
async def run() -> bool:
|
||||
loop = asyncio.get_running_loop()
|
||||
with patch.object(
|
||||
loop,
|
||||
"getaddrinfo",
|
||||
side_effect=AssertionError("缓存命中后不应再次发起 DNS 查询"),
|
||||
):
|
||||
return await SecurityUtils.is_safe_url_async(
|
||||
"https://assets.example.com/poster.jpg",
|
||||
{"example.com"},
|
||||
block_private=True,
|
||||
)
|
||||
|
||||
self.assertTrue(asyncio.run(run()))
|
||||
|
||||
def test_is_safe_url_async_allows_public_dns_result(self):
|
||||
"""
|
||||
异步版本对全公网解析结果且命中 allowlist 时放行。
|
||||
"""
|
||||
import asyncio
|
||||
|
||||
async def fake_getaddrinfo(host, *_args, **_kwargs):
|
||||
self.assertEqual(host, "assets.example.com")
|
||||
return [
|
||||
(
|
||||
socket.AF_INET,
|
||||
socket.SOCK_STREAM,
|
||||
0,
|
||||
"",
|
||||
("93.184.216.34", 0),
|
||||
)
|
||||
]
|
||||
|
||||
async def run() -> bool:
|
||||
loop = asyncio.get_running_loop()
|
||||
with patch.object(loop, "getaddrinfo", side_effect=fake_getaddrinfo):
|
||||
return await SecurityUtils.is_safe_url_async(
|
||||
"https://assets.example.com/poster.jpg",
|
||||
{"example.com"},
|
||||
block_private=True,
|
||||
)
|
||||
|
||||
self.assertTrue(asyncio.run(run()))
|
||||
|
||||
def test_dns_resolution_failure_populates_negative_cache(self):
|
||||
"""
|
||||
DNS 解析失败应回填负向缓存,避免短期内对同一目标反复触发 `getaddrinfo`。
|
||||
"""
|
||||
from app.utils.security import _dns_negative_cache as neg_cache
|
||||
|
||||
with patch(
|
||||
"app.utils.security.socket.getaddrinfo",
|
||||
side_effect=socket.gaierror,
|
||||
) as mock_resolve:
|
||||
self.assertFalse(
|
||||
SecurityUtils.is_safe_url(
|
||||
"https://assets.example.com/poster.jpg",
|
||||
{"example.com"},
|
||||
block_private=True,
|
||||
)
|
||||
)
|
||||
self.assertEqual(mock_resolve.call_count, 1)
|
||||
self.assertIn("assets.example.com", neg_cache)
|
||||
|
||||
self.assertFalse(
|
||||
SecurityUtils.is_safe_url(
|
||||
"https://assets.example.com/another.jpg",
|
||||
{"example.com"},
|
||||
block_private=True,
|
||||
)
|
||||
)
|
||||
self.assertEqual(
|
||||
mock_resolve.call_count,
|
||||
1,
|
||||
"命中负向缓存后不应再次调用 getaddrinfo",
|
||||
)
|
||||
|
||||
def test_literal_ip_skips_dns_cache(self):
|
||||
"""
|
||||
URL 中的字面量 IP 走快路径,不应进入 DNS 缓存或触发 `getaddrinfo`。
|
||||
"""
|
||||
from app.utils.security import (
|
||||
_dns_negative_cache as neg_cache,
|
||||
_dns_positive_cache as pos_cache,
|
||||
)
|
||||
|
||||
with patch(
|
||||
"app.utils.security.socket.getaddrinfo",
|
||||
side_effect=AssertionError("字面量 IP 不应触发 getaddrinfo"),
|
||||
):
|
||||
self.assertFalse(
|
||||
SecurityUtils.is_safe_url(
|
||||
"http://10.0.0.5:8080/secret.png",
|
||||
{"http://10.0.0.5:8080"},
|
||||
block_private=True,
|
||||
)
|
||||
)
|
||||
self.assertNotIn("10.0.0.5", pos_cache)
|
||||
self.assertNotIn("10.0.0.5", neg_cache)
|
||||
|
||||
def test_literal_ipv6_in_brackets_is_recognized(self):
|
||||
"""
|
||||
`urlparse` 已为 IPv6 字面量脱壳,`_literal_ip` 兼容直接传入带方括号的形式。
|
||||
"""
|
||||
self.assertEqual(
|
||||
str(SecurityUtils._literal_ip("[::1]")),
|
||||
"::1",
|
||||
)
|
||||
self.assertEqual(
|
||||
str(SecurityUtils._literal_ip("::1")),
|
||||
"::1",
|
||||
)
|
||||
self.assertIsNone(SecurityUtils._literal_ip("not-an-ip"))
|
||||
|
||||
def test_is_safe_url_async_dedupes_concurrent_inflight_queries(self):
|
||||
"""
|
||||
同 hostname 的并发未命中请求应通过 in-flight 锁去重,只触发一次 DNS 查询。
|
||||
"""
|
||||
import asyncio
|
||||
|
||||
call_count = 0
|
||||
|
||||
async def run() -> None:
|
||||
nonlocal call_count
|
||||
loop = asyncio.get_running_loop()
|
||||
release = asyncio.Event()
|
||||
|
||||
async def slow_getaddrinfo(host, *_args, **_kwargs):
|
||||
nonlocal call_count
|
||||
call_count += 1
|
||||
await release.wait()
|
||||
return [
|
||||
(
|
||||
socket.AF_INET,
|
||||
socket.SOCK_STREAM,
|
||||
0,
|
||||
"",
|
||||
("93.184.216.34", 0),
|
||||
)
|
||||
]
|
||||
|
||||
with patch.object(loop, "getaddrinfo", side_effect=slow_getaddrinfo):
|
||||
tasks = [
|
||||
asyncio.create_task(
|
||||
SecurityUtils.is_safe_url_async(
|
||||
"https://assets.example.com/poster.jpg",
|
||||
{"example.com"},
|
||||
block_private=True,
|
||||
)
|
||||
)
|
||||
for _ in range(5)
|
||||
]
|
||||
# 让所有任务都进入 in-flight 等待状态
|
||||
await asyncio.sleep(0)
|
||||
await asyncio.sleep(0)
|
||||
release.set()
|
||||
results = await asyncio.gather(*tasks)
|
||||
|
||||
self.assertTrue(all(results))
|
||||
self.assertEqual(call_count, 1, "并发未命中应去重为单次 DNS 查询")
|
||||
|
||||
asyncio.run(run())
|
||||
|
||||
def test_sync_cache_access_is_thread_safe(self):
|
||||
"""
|
||||
同步路径下并发线程访问 DNS 缓存不应触发异常或拿到不一致结果。
|
||||
TTLCache 自身非线程安全,依赖模块级 `_dns_cache_lock` 串行化读写。
|
||||
"""
|
||||
import threading
|
||||
|
||||
with patch(
|
||||
"app.utils.security.socket.getaddrinfo",
|
||||
return_value=[
|
||||
(
|
||||
socket.AF_INET,
|
||||
socket.SOCK_STREAM,
|
||||
0,
|
||||
"",
|
||||
("93.184.216.34", 0),
|
||||
)
|
||||
],
|
||||
):
|
||||
results: list = []
|
||||
errors: list = []
|
||||
|
||||
def worker() -> None:
|
||||
try:
|
||||
for _ in range(50):
|
||||
results.append(
|
||||
SecurityUtils.is_safe_url(
|
||||
"https://assets.example.com/poster.jpg",
|
||||
{"example.com"},
|
||||
block_private=True,
|
||||
)
|
||||
)
|
||||
except Exception as exc: # noqa: BLE001 - 用例需捕获任意异常
|
||||
errors.append(exc)
|
||||
|
||||
threads = [threading.Thread(target=worker) for _ in range(8)]
|
||||
for t in threads:
|
||||
t.start()
|
||||
for t in threads:
|
||||
t.join()
|
||||
|
||||
self.assertEqual(errors, [])
|
||||
self.assertTrue(all(results))
|
||||
self.assertEqual(len(results), 8 * 50)
|
||||
|
||||
def test_async_dns_resolution_failure_releases_inflight_lock(self):
|
||||
"""
|
||||
DNS 解析失败后 in-flight 锁字典中必须被清理,避免每个解析失败的 hostname
|
||||
都在 `_dns_inflight_locks` 里残留一把 `asyncio.Lock`。
|
||||
"""
|
||||
import asyncio
|
||||
|
||||
async def fail_getaddrinfo(*_args, **_kwargs):
|
||||
raise socket.gaierror()
|
||||
|
||||
async def run() -> None:
|
||||
loop = asyncio.get_running_loop()
|
||||
with patch.object(loop, "getaddrinfo", side_effect=fail_getaddrinfo):
|
||||
result = await SecurityUtils._hostname_addresses_async(
|
||||
"bad-host.example"
|
||||
)
|
||||
self.assertIsNone(result)
|
||||
|
||||
asyncio.run(run())
|
||||
self.assertNotIn(
|
||||
"bad-host.example",
|
||||
_dns_inflight_locks,
|
||||
"解析失败路径必须释放 in-flight 锁字典条目",
|
||||
)
|
||||
|
||||
def test_async_dns_resolution_success_releases_inflight_lock(self):
|
||||
"""
|
||||
正常解析完成后 in-flight 锁字典也必须被清理,避免 hostname 累积。
|
||||
"""
|
||||
import asyncio
|
||||
|
||||
async def fake_getaddrinfo(*_args, **_kwargs):
|
||||
return [
|
||||
(
|
||||
socket.AF_INET,
|
||||
socket.SOCK_STREAM,
|
||||
0,
|
||||
"",
|
||||
("93.184.216.34", 0),
|
||||
)
|
||||
]
|
||||
|
||||
async def run() -> None:
|
||||
loop = asyncio.get_running_loop()
|
||||
with patch.object(loop, "getaddrinfo", side_effect=fake_getaddrinfo):
|
||||
result = await SecurityUtils._hostname_addresses_async(
|
||||
"ok-host.example"
|
||||
)
|
||||
self.assertIsNotNone(result)
|
||||
|
||||
asyncio.run(run())
|
||||
self.assertNotIn(
|
||||
"ok-host.example",
|
||||
_dns_inflight_locks,
|
||||
"正常解析路径必须释放 in-flight 锁字典条目",
|
||||
)
|
||||
|
||||
def test_async_dns_concurrent_waiters_release_inflight_lock(self):
|
||||
"""
|
||||
并发未命中场景下,所有等待者完成后 in-flight 锁字典也必须被清理,
|
||||
覆盖"等到锁但缓存已被前一个协程回填"的二次返回路径。
|
||||
"""
|
||||
import asyncio
|
||||
|
||||
async def run() -> None:
|
||||
loop = asyncio.get_running_loop()
|
||||
release = asyncio.Event()
|
||||
|
||||
async def slow_getaddrinfo(*_args, **_kwargs):
|
||||
await release.wait()
|
||||
return [
|
||||
(
|
||||
socket.AF_INET,
|
||||
socket.SOCK_STREAM,
|
||||
0,
|
||||
"",
|
||||
("93.184.216.34", 0),
|
||||
)
|
||||
]
|
||||
|
||||
with patch.object(loop, "getaddrinfo", side_effect=slow_getaddrinfo):
|
||||
tasks = [
|
||||
asyncio.create_task(
|
||||
SecurityUtils._hostname_addresses_async("multi-host.example")
|
||||
)
|
||||
for _ in range(5)
|
||||
]
|
||||
await asyncio.sleep(0)
|
||||
await asyncio.sleep(0)
|
||||
release.set()
|
||||
await asyncio.gather(*tasks)
|
||||
|
||||
asyncio.run(run())
|
||||
self.assertNotIn(
|
||||
"multi-host.example",
|
||||
_dns_inflight_locks,
|
||||
"并发等待者全部退出后必须释放 in-flight 锁字典条目",
|
||||
)
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import asyncio
|
||||
import ipaddress
|
||||
import sys
|
||||
import unittest
|
||||
from types import ModuleType, SimpleNamespace
|
||||
@@ -134,6 +135,47 @@ class NettestSecurityTest(unittest.TestCase):
|
||||
|
||||
self.assertIsNone(resp)
|
||||
|
||||
def test_fetch_image_allows_configured_private_range_after_domain_match(self):
|
||||
"""
|
||||
图片代理在域名白名单命中后,可按配置放行指定非公网解析网段。
|
||||
"""
|
||||
image_helper = Mock()
|
||||
image_helper.async_fetch_image = AsyncMock(return_value=b"image-bytes")
|
||||
|
||||
with patch.object(system_endpoint, "ImageHelper", return_value=image_helper), patch.object(
|
||||
system_endpoint.HashUtils, "md5", return_value="etag", create=True
|
||||
), patch.object(
|
||||
system_endpoint.RequestUtils, "generate_cache_headers", return_value={}, create=True
|
||||
), patch.object(
|
||||
system_endpoint.SecurityUtils,
|
||||
"_is_global_hostname",
|
||||
return_value=False,
|
||||
), patch.object(
|
||||
system_endpoint.SecurityUtils,
|
||||
"_hostname_addresses",
|
||||
return_value=[ipaddress.ip_address("198.18.16.96")],
|
||||
), patch.object(
|
||||
system_endpoint.settings,
|
||||
"IMAGE_PROXY_ALLOWED_PRIVATE_RANGES",
|
||||
["198.18.0.0/15"],
|
||||
), patch(
|
||||
"app.utils.security.logger.debug",
|
||||
):
|
||||
resp = asyncio.run(
|
||||
system_endpoint.fetch_image(
|
||||
url="https://img1.doubanio.com/poster.webp",
|
||||
allowed_domains={"doubanio.com"},
|
||||
)
|
||||
)
|
||||
|
||||
self.assertEqual(resp.status_code, 200)
|
||||
image_helper.async_fetch_image.assert_awaited_once_with(
|
||||
url="https://img1.doubanio.com/poster.webp",
|
||||
proxy=None,
|
||||
use_cache=False,
|
||||
cookies=None,
|
||||
)
|
||||
|
||||
def test_fetch_image_blocks_tampered_signed_private_url(self):
|
||||
"""
|
||||
私网签名绑定完整 URL,改动路径后不能继续代理。
|
||||
|
||||
@@ -1,2 +1,2 @@
|
||||
APP_VERSION = 'v2.13.0'
|
||||
FRONTEND_VERSION = 'v2.13.0'
|
||||
APP_VERSION = 'v2.13.1'
|
||||
FRONTEND_VERSION = 'v2.13.1'
|
||||
|
||||
Reference in New Issue
Block a user