chore: split runtime and development dependencies (#5985)

This commit is contained in:
InfinityPacer
2026-06-22 17:44:06 +08:00
committed by GitHub
parent 7a7b27858e
commit e44a6f41b5
11 changed files with 68 additions and 96 deletions

View File

@@ -1,6 +1,6 @@
## 开发环境设置指南
本文档旨在帮助开发者快速设置开发环境,并介绍如何使用 `pip-tools` 管理依赖项和使用 `safety` 进行安全检查
本文档旨在帮助开发者快速设置开发环境,并说明主程序、开发测试、构建工具和插件依赖的管理边界
### 环境准备
@@ -33,57 +33,38 @@ Rust 加速扩展通过 `moviepilot-rust` PyPI 包安装,主项目本地开发
虚拟环境确保项目的依赖项与系统全局环境隔离,防止冲突。
### 2. 使用 pip-tools 管理依赖项
### 2. 依赖分层与安装
我们使用 `pip-tools` 来管理项目的 Python 依赖项,这有助于保持 `requirements.txt` 文件的一致性和更新性。
主程序依赖按使用场景分层,避免运行时镜像携带只在开发、测试或构建时需要的工具:
#### 安装 pip-tools
| 文件 | 用途 | 典型安装场景 |
| --- | --- | --- |
| `requirements.in` | 主程序运行时依赖。只放启动、后台任务、插件运行框架和内置功能在生产环境需要导入的包。 | Docker 镜像、CLI 本地运行、运行时依赖自愈。 |
| `requirements-dev.in` | 开发、测试、静态检查和源码构建辅助依赖。 | CI 单测、本地跑测、Pylint、显式源码构建。 |
| `requirements.txt` | 兼容入口,默认只委托到 `requirements.in`。它不是跨平台完整锁文件,不应在本地开发机上直接维护一份平台相关锁定结果。 | 旧脚本、Docker 运行时恢复、CLI 安装入口。 |
首先,您需要安装 `pip-tools` 以便管理依赖
运行主程序只需要安装运行时依赖:
```bash
pip install pip-tools
pip install -r requirements.txt
```
#### 管理依赖项
开发、测试、静态检查或执行源码编译时安装开发依赖入口:
1. **修改 `requirements.in` 文件**
```bash
pip install -r requirements-dev.in
```
`requirements.in` 文件是项目依赖项的源文件。要添加或更新依赖项,请直接编辑该文件。
### 3. 修改主程序依赖
2. **更新特定的依赖项**
新增或升级依赖时,先确认依赖属于哪个层级
如果你只想更新 `requirements.in` 中的某个特定依赖包,而不影响其他依赖项,可以使用 `--upgrade-package` 选项,指定要升级的包:
1. **运行时依赖**:被 `app/` 生产代码直接导入,或是生产功能、后台任务、插件框架启动必需,写入 `requirements.in`。
2. **开发 / 测试 / 静态检查 / 构建依赖**只用于单测、覆盖率、lint 辅助、源码构建等,不应进入生产运行时,写入 `requirements-dev.in`。
3. **工具依赖**`pip-tools`、`uv`、`safety` 这类安装或审计工具不属于主程序运行依赖,按脚本或 CI 场景显式安装。
4. **插件依赖**:由插件声明并在插件安装阶段处理,不直接并入主程序 `requirements.in`。
```bash
pip-compile --upgrade-package <package-name> requirements.in
```
例如,要只升级 `requests` 这个包,你可以运行以下命令:
```bash
pip-compile --upgrade-package requests requirements.in
```
3. **全量更新依赖项**
如果你想更新 `requirements.in` 中的所有依赖包,运行以下命令生成或更新 `requirements.txt` 文件:
```bash
pip-compile requirements.in
```
这将根据 `requirements.in` 中指定的依赖项生成一个锁定的 `requirements.txt` 文件。
4. **安装依赖项**
使用以下命令安装 `requirements.txt` 文件中列出的依赖项:
```bash
pip install -r requirements.txt
```
### 3. 准备资源与插件目录
### 4. 准备资源与插件目录
本地源码开发时,主程序需要读取资源文件和插件源码。相关文件需要放到主程序实际加载的目录下:
@@ -92,9 +73,9 @@ pip install pip-tools
如果资源文件没有放到 `app/helper/`,站点索引、规则和内置资源相关能力可能无法按本地开发预期工作;如果插件没有放到 `app/plugins/`,主程序也不会在本地运行时发现该插件。
### 4. 运行安全检查
### 5. 运行安全检查
我们使用 `safety` 工具检查依赖项中是否存在已知安全漏洞。请确保在每次更新依赖项后都运行安全检查,以确保项目的安全性
我们使用 `safety` 工具检查依赖项中是否存在已知安全漏洞。更新运行时依赖后,应至少检查运行时入口;更新开发测试依赖时,也应覆盖开发入口
#### 安装 safety
@@ -106,7 +87,7 @@ pip install safety
#### 执行安全检查
运行以下命令检查 `requirements.txt` 文件中列出的依赖项是否存在安全漏洞
运行以下命令检查运行时入口
```bash
safety check -r requirements.txt --policy-file=safety.policy.yml > safety_report.txt
@@ -114,11 +95,11 @@ safety check -r requirements.txt --policy-file=safety.policy.yml > safety_report
这将生成一个名为 `safety_report.txt` 的报告文件,您可以查看其中的漏洞报告并进行相应处理。
### 5. 提交代码前的检查
### 6. 提交代码前的检查
在提交代码之前,请确保完成以下步骤:
1. **确依赖项已更新**:如果您对 `requirements.in` 进行了更改,请重新生成 `requirements.txt` 并安装依赖
1. **确依赖分层正确**:运行时包进入 `requirements.in`;测试、覆盖率、静态检查和构建辅助进入 `requirements-dev.in`;插件依赖不并入主程序运行时依赖。
2. **运行安全检查**:确保 `safety` 检查通过,没有新的安全漏洞。
@@ -128,9 +109,10 @@ safety check -r requirements.txt --policy-file=safety.policy.yml > safety_report
pytest
```
### 6. 参考资源
### 7. 参考资源
- [pip-tools 官方文档](https://github.com/jazzband/pip-tools)
- [uv 官方文档](https://docs.astral.sh/uv/)
- [safety 官方文档](https://pyup.io/safety/)
- [MoviePilot-Resources](https://github.com/jxxghp/MoviePilot-Resources)
- [MoviePilot-Plugins](https://github.com/jxxghp/MoviePilot-Plugins)

View File

@@ -104,10 +104,11 @@
| Item | Detail |
|---|---|
| Source file | `requirements.in`edit this to add or upgrade dependencies |
| Lock file | `requirements.txt` — generated by `pip-compile`; never edit manually |
| Tool | `pip-tools` (`pip-compile`, `pip-sync`) |
| Install | `pip install -r requirements.txt` |
| Runtime source | `requirements.in`production/runtime dependencies only |
| Dev/test/lint/build source | `requirements-dev.in` — includes runtime plus pytest, coverage tooling, pylint, and build support |
| Compatibility entry | `requirements.txt` — delegates to `requirements.in`; not a committed cross-platform lock |
| Runtime install | `pip install -r requirements.txt` |
| Dev/test/lint/build install | `pip install -r requirements-dev.in` |
---

View File

@@ -12,11 +12,11 @@ python3 -m venv venv
source venv/bin/activate # macOS / Linux
.\venv\Scripts\activate # Windows
# Install pip-tools
pip install pip-tools
# Install project dependencies
# Install runtime dependencies
pip install -r requirements.txt
# Install development/test/lint/build dependencies
pip install -r requirements-dev.in
```
---
@@ -24,20 +24,17 @@ pip install -r requirements.txt
## Dependency Management
```bash
# Compile requirements.txt from requirements.in (full recompile)
pip-compile requirements.in
# Upgrade a single package without touching others
pip-compile --upgrade-package <package-name> requirements.in
# Install from the generated lock file
# Install runtime dependencies
pip install -r requirements.txt
# Install test/lint/build dependencies
pip install -r requirements-dev.in
```
**Rules:**
- Always edit `requirements.in` to add or change dependencies.
- Never edit `requirements.txt` manually — it is a generated lock file.
- After any change to `requirements.in`, re-run `pip-compile requirements.in` and commit both files together.
- Runtime dependencies belong in `requirements.in`.
- Test, coverage, lint, and explicit build tooling belong in `requirements-dev.in`.
- `requirements.txt` is a compatibility entry that delegates to `requirements.in`; do not replace it with a local cross-platform lock file.
---
@@ -83,7 +80,7 @@ pylint app/chain/download.py
## Security Scan
```bash
# Run safety check against the lock file
# Run safety check against the runtime compatibility entry
safety check -r requirements.txt --policy-file=safety.policy.yml
# Save report to file
@@ -91,7 +88,7 @@ safety check -r requirements.txt --policy-file=safety.policy.yml > safety_report
```
**Rules:**
- Run after every change to `requirements.txt`.
- Run after runtime dependency changes; include `requirements-dev.in` when development/test/lint/build dependencies change.
- No new high-severity vulnerabilities may be introduced.
---

View File

@@ -111,7 +111,7 @@ except:
## What Not To Do
- Do not introduce new third-party libraries without updating `requirements.in` and running `pip-compile`.
- Do not introduce new third-party libraries without placing them in the correct dependency entry: runtime packages in `requirements.in`, test/lint/build tooling in `requirements-dev.in`.
- Do not use `requests` or `httpx` directly for external HTTP calls — use `RequestUtils` from `app/utils/http.py`.
- Do not issue raw SQLAlchemy queries from chains, modules, or endpoints — use the `*_oper.py` classes.
- Do not add TODO or FIXME without context. Only keep one if it is genuinely deferred and cannot be addressed in the current task.

View File

@@ -57,7 +57,7 @@ pylint app/
safety check -r requirements.txt --policy-file=safety.policy.yml
```
- Run after every change to `requirements.txt`.
- Run after runtime dependency changes; scan the development dependency entry as well when `requirements-dev.in` changes.
- No new high-severity vulnerabilities may be introduced.
- If a vulnerability cannot be patched immediately, document it explicitly in the PR description.
@@ -129,7 +129,7 @@ Before marking any task as complete:
- [ ] Related pytest tests pass
- [ ] No new pylint error-level issues in `pylint app/`
- [ ] If dependencies changed: `pip-compile requirements.in` was run and `safety check` passes
- [ ] If dependencies changed: the package is in the correct runtime or dev dependency entry, and `safety check` passes for the affected entry
- [ ] If CLI behavior changed: `docs/cli.md` and related tests are updated
- [ ] If MCP/API behavior changed: `docs/mcp-api.md` and related skill files are updated
- [ ] If database schema changed: a new Alembic migration exists under `database/versions/`

View File

@@ -100,11 +100,10 @@ ci: improve docker build cache
When updating a dependency:
1. Update `requirements.in` with the new version constraint.
2. Run `pip-compile requirements.in` to regenerate `requirements.txt`.
3. Run `safety check -r requirements.txt --policy-file=safety.policy.yml`.
1. Decide the dependency layer: runtime packages go to `requirements.in`; test, coverage, lint, and explicit build tooling go to `requirements-dev.in`.
2. Keep `requirements.txt` as the compatibility entry that delegates to `requirements.in`; do not commit a locally generated cross-platform lock file.
3. Run `safety check -r requirements.txt --policy-file=safety.policy.yml`; include the dev dependency entry when `requirements-dev.in` changed.
4. Run the full test suite: `pytest`.
5. Commit both `requirements.in` and `requirements.txt` together.
---

View File

@@ -15,7 +15,7 @@ python tests/run.py # 等价于 pytest 全量(参数透
- 不再使用 `python -m unittest discover`:它不导入 `tests` 包、收不到纯函数用例,且绕过 `conftest.py` 的隔离。
- 不再依赖 `python tests/test_xxx.py` 直跑:所有 `if __name__ == "__main__": unittest.main()` 尾巴已移除。
- **复现 CI 用干净环境**:建议用一个仅 `pip install -r requirements.in pytest` 的虚拟环境运行,避免本地额外包或编译产物掩盖问题。
- **复现 CI 用干净环境**:建议用一个仅 `pip install -r requirements-dev.in` 的虚拟环境运行,避免本地额外包或编译产物掩盖问题。
## 隔离模型(`tests/conftest.py`
@@ -133,4 +133,4 @@ def test_recognize_prefers_explicit_id(sample_meta, monkeypatch):
- **门禁**`.github/workflows/test.yml` 在指向 `v2` 的 `pull_request` / `push` 及手动触发时,用 `python tests/run.py` 跑全量单测。
- **PR**`python tests/run.py` 确认全绿、且 socket 探针零真实出站,避免把红的改动推上去空耗门禁。
- 复现 CI 用仅安装 `requirements.in` 的干净环境(含 pytest 与可选扩展),保证可选扩展、动态模块的存在性与 CI 一致
- 复现 CI 用仅安装 `requirements-dev.in` 的干净环境`requirements.in` 只承载运行时依赖pytest 与覆盖率插件由开发依赖入口提供