diff --git a/.github/workflows/pylint.yml b/.github/workflows/pylint.yml index e7d3263a..93e86cb1 100644 --- a/.github/workflows/pylint.yml +++ b/.github/workflows/pylint.yml @@ -23,24 +23,15 @@ jobs: uses: actions/cache@v4 with: path: ~/.cache/pip - key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt', '**/requirements.in') }} + key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.in', '**/requirements-dev.in', '**/requirements.txt') }} restore-keys: | ${{ runner.os }}-pip- - name: Install dependencies run: | python -m pip install --upgrade pip setuptools wheel - pip install pylint - # 安装项目依赖 - if [ -f requirements.txt ]; then - echo "📦 安装 requirements.txt 中的依赖..." - pip install -r requirements.txt - elif [ -f requirements.in ]; then - echo "📦 安装 requirements.in 中的依赖..." - pip install -r requirements.in - else - echo "⚠️ 未找到依赖文件,仅安装 pylint" - fi + # Pylint 属于开发/静态检查依赖,统一通过 dev 入口安装。 + pip install -r requirements-dev.in - name: Verify pylint config run: | @@ -88,4 +79,4 @@ jobs: run: | echo "🎉 Pylint 检查完成!" echo "✅ 没有发现语法错误或严重问题" - echo "📊 详细报告已保存为构建工件" \ No newline at end of file + echo "📊 详细报告已保存为构建工件" diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 64e5dc3d..900b2381 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -38,16 +38,15 @@ jobs: uses: actions/cache@v5 with: path: ~/.cache/pip - key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.in', '**/requirements.txt') }} + key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.in', '**/requirements-dev.in', '**/requirements.txt') }} restore-keys: | ${{ runner.os }}-pip- - name: Install dependencies run: | python -m pip install --upgrade pip setuptools wheel - # 用 requirements.in 还原 CI / 全新环境(含 pytest 与 moviepilot-rust 等可选扩展), - # 与本地"干净 venv 复现"一致;测试运行器 pytest 已在 requirements.in 中声明。 - pip install -r requirements.in + # 单测需要开发/测试依赖;运行时入口 requirements.in 不携带测试与构建辅助工具。 + pip install -r requirements-dev.in - name: Run tests timeout-minutes: 10 diff --git a/docs/development-setup.md b/docs/development-setup.md index 9e13fc41..e0e9705f 100644 --- a/docs/development-setup.md +++ b/docs/development-setup.md @@ -1,6 +1,6 @@ ## 开发环境设置指南 -本文档旨在帮助开发者快速设置开发环境,并介绍如何使用 `pip-tools` 管理依赖项和使用 `safety` 进行安全检查。 +本文档旨在帮助开发者快速设置开发环境,并说明主程序、开发测试、构建工具和插件依赖的管理边界。 ### 环境准备 @@ -33,57 +33,38 @@ Rust 加速扩展通过 `moviepilot-rust` PyPI 包安装,主项目本地开发 虚拟环境确保项目的依赖项与系统全局环境隔离,防止冲突。 -### 2. 使用 pip-tools 管理依赖项 +### 2. 依赖分层与安装 -我们使用 `pip-tools` 来管理项目的 Python 依赖项,这有助于保持 `requirements.txt` 文件的一致性和更新性。 +主程序依赖按使用场景分层,避免运行时镜像携带只在开发、测试或构建时需要的工具: -#### 安装 pip-tools +| 文件 | 用途 | 典型安装场景 | +| --- | --- | --- | +| `requirements.in` | 主程序运行时依赖。只放启动、后台任务、插件运行框架和内置功能在生产环境需要导入的包。 | Docker 镜像、CLI 本地运行、运行时依赖自愈。 | +| `requirements-dev.in` | 开发、测试、静态检查和源码构建辅助依赖。 | CI 单测、本地跑测、Pylint、显式源码构建。 | +| `requirements.txt` | 兼容入口,默认只委托到 `requirements.in`。它不是跨平台完整锁文件,不应在本地开发机上直接维护一份平台相关锁定结果。 | 旧脚本、Docker 运行时恢复、CLI 安装入口。 | -首先,您需要安装 `pip-tools` 以便管理依赖项: +运行主程序只需要安装运行时依赖: ```bash -pip install pip-tools +pip install -r requirements.txt ``` -#### 管理依赖项 +开发、测试、静态检查或执行源码编译时安装开发依赖入口: -1. **修改 `requirements.in` 文件**: +```bash +pip install -r requirements-dev.in +``` - `requirements.in` 文件是项目依赖项的源文件。要添加或更新依赖项,请直接编辑该文件。 +### 3. 修改主程序依赖 -2. **更新特定的依赖项**: +新增或升级依赖时,先确认依赖属于哪个层级: - 如果你只想更新 `requirements.in` 中的某个特定依赖包,而不影响其他依赖项,可以使用 `--upgrade-package` 选项,指定要升级的包: +1. **运行时依赖**:被 `app/` 生产代码直接导入,或是生产功能、后台任务、插件框架启动必需,写入 `requirements.in`。 +2. **开发 / 测试 / 静态检查 / 构建依赖**:只用于单测、覆盖率、lint 辅助、源码构建等,不应进入生产运行时,写入 `requirements-dev.in`。 +3. **工具依赖**:`pip-tools`、`uv`、`safety` 这类安装或审计工具不属于主程序运行依赖,按脚本或 CI 场景显式安装。 +4. **插件依赖**:由插件声明并在插件安装阶段处理,不直接并入主程序 `requirements.in`。 - ```bash - pip-compile --upgrade-package requirements.in - ``` - - 例如,要只升级 `requests` 这个包,你可以运行以下命令: - - ```bash - pip-compile --upgrade-package requests requirements.in - ``` - -3. **全量更新依赖项**: - - 如果你想更新 `requirements.in` 中的所有依赖包,运行以下命令生成或更新 `requirements.txt` 文件: - - ```bash - pip-compile requirements.in - ``` - - 这将根据 `requirements.in` 中指定的依赖项生成一个锁定的 `requirements.txt` 文件。 - -4. **安装依赖项**: - - 使用以下命令安装 `requirements.txt` 文件中列出的依赖项: - - ```bash - pip install -r requirements.txt - ``` - -### 3. 准备资源与插件目录 +### 4. 准备资源与插件目录 本地源码开发时,主程序需要读取资源文件和插件源码。相关文件需要放到主程序实际加载的目录下: @@ -92,9 +73,9 @@ pip install pip-tools 如果资源文件没有放到 `app/helper/`,站点索引、规则和内置资源相关能力可能无法按本地开发预期工作;如果插件没有放到 `app/plugins/`,主程序也不会在本地运行时发现该插件。 -### 4. 运行安全检查 +### 5. 运行安全检查 -我们使用 `safety` 工具来检查依赖项中是否存在已知的安全漏洞。请确保在每次更新依赖项后都运行安全检查,以确保项目的安全性。 +我们使用 `safety` 工具检查依赖项中是否存在已知安全漏洞。更新运行时依赖后,应至少检查运行时入口;更新开发测试依赖时,也应覆盖开发入口。 #### 安装 safety @@ -106,7 +87,7 @@ pip install safety #### 执行安全检查 -运行以下命令以检查 `requirements.txt` 文件中列出的依赖项是否存在安全漏洞: +运行以下命令检查运行时入口: ```bash safety check -r requirements.txt --policy-file=safety.policy.yml > safety_report.txt @@ -114,11 +95,11 @@ safety check -r requirements.txt --policy-file=safety.policy.yml > safety_report 这将生成一个名为 `safety_report.txt` 的报告文件,您可以查看其中的漏洞报告并进行相应处理。 -### 5. 提交代码前的检查 +### 6. 提交代码前的检查 在提交代码之前,请确保完成以下步骤: -1. **确保依赖项已更新**:如果您对 `requirements.in` 进行了更改,请重新生成 `requirements.txt` 并安装依赖项。 +1. **确认依赖分层正确**:运行时包进入 `requirements.in`;测试、覆盖率、静态检查和构建辅助进入 `requirements-dev.in`;插件依赖不并入主程序运行时依赖。 2. **运行安全检查**:确保 `safety` 检查通过,没有新的安全漏洞。 @@ -128,9 +109,10 @@ safety check -r requirements.txt --policy-file=safety.policy.yml > safety_report pytest ``` -### 6. 参考资源 +### 7. 参考资源 - [pip-tools 官方文档](https://github.com/jazzband/pip-tools) +- [uv 官方文档](https://docs.astral.sh/uv/) - [safety 官方文档](https://pyup.io/safety/) - [MoviePilot-Resources](https://github.com/jxxghp/MoviePilot-Resources) - [MoviePilot-Plugins](https://github.com/jxxghp/MoviePilot-Plugins) diff --git a/docs/rules/02-tech-stack.md b/docs/rules/02-tech-stack.md index 37ba57f7..4e733d53 100644 --- a/docs/rules/02-tech-stack.md +++ b/docs/rules/02-tech-stack.md @@ -104,10 +104,11 @@ | Item | Detail | |---|---| -| Source file | `requirements.in` — edit this to add or upgrade dependencies | -| Lock file | `requirements.txt` — generated by `pip-compile`; never edit manually | -| Tool | `pip-tools` (`pip-compile`, `pip-sync`) | -| Install | `pip install -r requirements.txt` | +| Runtime source | `requirements.in` — production/runtime dependencies only | +| Dev/test/lint/build source | `requirements-dev.in` — includes runtime plus pytest, coverage tooling, pylint, and build support | +| Compatibility entry | `requirements.txt` — delegates to `requirements.in`; not a committed cross-platform lock | +| Runtime install | `pip install -r requirements.txt` | +| Dev/test/lint/build install | `pip install -r requirements-dev.in` | --- diff --git a/docs/rules/03-commands.md b/docs/rules/03-commands.md index fd0380f4..be87bb63 100644 --- a/docs/rules/03-commands.md +++ b/docs/rules/03-commands.md @@ -12,11 +12,11 @@ python3 -m venv venv source venv/bin/activate # macOS / Linux .\venv\Scripts\activate # Windows -# Install pip-tools -pip install pip-tools - -# Install project dependencies +# Install runtime dependencies pip install -r requirements.txt + +# Install development/test/lint/build dependencies +pip install -r requirements-dev.in ``` --- @@ -24,20 +24,17 @@ pip install -r requirements.txt ## Dependency Management ```bash -# Compile requirements.txt from requirements.in (full recompile) -pip-compile requirements.in - -# Upgrade a single package without touching others -pip-compile --upgrade-package requirements.in - -# Install from the generated lock file +# Install runtime dependencies pip install -r requirements.txt + +# Install test/lint/build dependencies +pip install -r requirements-dev.in ``` **Rules:** -- Always edit `requirements.in` to add or change dependencies. -- Never edit `requirements.txt` manually — it is a generated lock file. -- After any change to `requirements.in`, re-run `pip-compile requirements.in` and commit both files together. +- Runtime dependencies belong in `requirements.in`. +- Test, coverage, lint, and explicit build tooling belong in `requirements-dev.in`. +- `requirements.txt` is a compatibility entry that delegates to `requirements.in`; do not replace it with a local cross-platform lock file. --- @@ -83,7 +80,7 @@ pylint app/chain/download.py ## Security Scan ```bash -# Run safety check against the lock file +# Run safety check against the runtime compatibility entry safety check -r requirements.txt --policy-file=safety.policy.yml # Save report to file @@ -91,7 +88,7 @@ safety check -r requirements.txt --policy-file=safety.policy.yml > safety_report ``` **Rules:** -- Run after every change to `requirements.txt`. +- Run after runtime dependency changes; include `requirements-dev.in` when development/test/lint/build dependencies change. - No new high-severity vulnerabilities may be introduced. --- diff --git a/docs/rules/06-code-styles.md b/docs/rules/06-code-styles.md index 8c6553e8..5ac2fe80 100644 --- a/docs/rules/06-code-styles.md +++ b/docs/rules/06-code-styles.md @@ -111,7 +111,7 @@ except: ## What Not To Do -- Do not introduce new third-party libraries without updating `requirements.in` and running `pip-compile`. +- Do not introduce new third-party libraries without placing them in the correct dependency entry: runtime packages in `requirements.in`, test/lint/build tooling in `requirements-dev.in`. - Do not use `requests` or `httpx` directly for external HTTP calls — use `RequestUtils` from `app/utils/http.py`. - Do not issue raw SQLAlchemy queries from chains, modules, or endpoints — use the `*_oper.py` classes. - Do not add TODO or FIXME without context. Only keep one if it is genuinely deferred and cannot be addressed in the current task. diff --git a/docs/rules/11-quality-and-security.md b/docs/rules/11-quality-and-security.md index e103f324..0197030b 100644 --- a/docs/rules/11-quality-and-security.md +++ b/docs/rules/11-quality-and-security.md @@ -57,7 +57,7 @@ pylint app/ safety check -r requirements.txt --policy-file=safety.policy.yml ``` -- Run after every change to `requirements.txt`. +- Run after runtime dependency changes; scan the development dependency entry as well when `requirements-dev.in` changes. - No new high-severity vulnerabilities may be introduced. - If a vulnerability cannot be patched immediately, document it explicitly in the PR description. @@ -129,7 +129,7 @@ Before marking any task as complete: - [ ] Related pytest tests pass - [ ] No new pylint error-level issues in `pylint app/` -- [ ] If dependencies changed: `pip-compile requirements.in` was run and `safety check` passes +- [ ] If dependencies changed: the package is in the correct runtime or dev dependency entry, and `safety check` passes for the affected entry - [ ] If CLI behavior changed: `docs/cli.md` and related tests are updated - [ ] If MCP/API behavior changed: `docs/mcp-api.md` and related skill files are updated - [ ] If database schema changed: a new Alembic migration exists under `database/versions/` diff --git a/docs/rules/12-collaboration-and-distribution.md b/docs/rules/12-collaboration-and-distribution.md index 16cf0620..9edee76f 100644 --- a/docs/rules/12-collaboration-and-distribution.md +++ b/docs/rules/12-collaboration-and-distribution.md @@ -100,11 +100,10 @@ ci: improve docker build cache When updating a dependency: -1. Update `requirements.in` with the new version constraint. -2. Run `pip-compile requirements.in` to regenerate `requirements.txt`. -3. Run `safety check -r requirements.txt --policy-file=safety.policy.yml`. +1. Decide the dependency layer: runtime packages go to `requirements.in`; test, coverage, lint, and explicit build tooling go to `requirements-dev.in`. +2. Keep `requirements.txt` as the compatibility entry that delegates to `requirements.in`; do not commit a locally generated cross-platform lock file. +3. Run `safety check -r requirements.txt --policy-file=safety.policy.yml`; include the dev dependency entry when `requirements-dev.in` changed. 4. Run the full test suite: `pytest`. -5. Commit both `requirements.in` and `requirements.txt` together. --- diff --git a/docs/testing.md b/docs/testing.md index 6682fac3..4339f98e 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -15,7 +15,7 @@ python tests/run.py # 等价于 pytest 全量(参数透 - 不再使用 `python -m unittest discover`:它不导入 `tests` 包、收不到纯函数用例,且绕过 `conftest.py` 的隔离。 - 不再依赖 `python tests/test_xxx.py` 直跑:所有 `if __name__ == "__main__": unittest.main()` 尾巴已移除。 -- **复现 CI 用干净环境**:建议用一个仅 `pip install -r requirements.in pytest` 的虚拟环境运行,避免本地额外包或编译产物掩盖问题。 +- **复现 CI 用干净环境**:建议用一个仅 `pip install -r requirements-dev.in` 的虚拟环境运行,避免本地额外包或编译产物掩盖问题。 ## 隔离模型(`tests/conftest.py`) @@ -133,4 +133,4 @@ def test_recognize_prefers_explicit_id(sample_meta, monkeypatch): - **门禁**:`.github/workflows/test.yml` 在指向 `v2` 的 `pull_request` / `push` 及手动触发时,用 `python tests/run.py` 跑全量单测。 - **PR**:`python tests/run.py` 确认全绿、且 socket 探针零真实出站,避免把红的改动推上去空耗门禁。 -- 复现 CI 用仅安装 `requirements.in` 的干净环境(含 pytest 与可选扩展),保证可选扩展、动态模块的存在性与 CI 一致。 +- 复现 CI 用仅安装 `requirements-dev.in` 的干净环境;`requirements.in` 只承载运行时依赖,pytest 与覆盖率插件由开发依赖入口提供。 diff --git a/requirements-dev.in b/requirements-dev.in new file mode 100644 index 00000000..79e90643 --- /dev/null +++ b/requirements-dev.in @@ -0,0 +1,7 @@ +-r requirements.in + +Cython~=3.2.5 +pylint~=4.0.6 +pytest~=9.0.3 +pytest-cov~=7.1.0 +pytest-timeout~=2.4.0 diff --git a/requirements.in b/requirements.in index 80828459..9a302c87 100644 --- a/requirements.in +++ b/requirements.in @@ -1,4 +1,3 @@ -Cython~=3.2.5 moviepilot-rust~=0.1.10 pydantic>=2.13.4,<3.0.0 pydantic-settings>=2.14.1,<3.0.0 @@ -91,6 +90,3 @@ google-genai~=2.8.0 ddgs~=9.14.4 websocket-client~=1.9.0 lark-oapi~=1.6.8 -pytest~=9.0.3 -pytest-cov~=7.1.0 -pytest-timeout~=2.4.0