mirror of
https://github.com/JefferyHcool/BiliNote.git
synced 2026-06-17 21:50:15 +08:00
Compare commits
16 Commits
copilot/fi
...
master
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
7baaefd76e | ||
|
|
3002e311ac | ||
|
|
ad57bc5489 | ||
|
|
a7d8995f3a | ||
|
|
67486c4d66 | ||
|
|
54714faa73 | ||
|
|
f6ba347718 | ||
|
|
67253a1bf2 | ||
|
|
f79dc612fb | ||
|
|
ab9ca6a026 | ||
|
|
2ba409880e | ||
|
|
16a0dd4aec | ||
|
|
39d051cc36 | ||
|
|
71c931be55 | ||
|
|
24b4d4c57c | ||
|
|
7d4573f84e |
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"$schema": "../node_modules/@tauri-apps/cli/config.schema.json",
|
||||
"productName": "BiliNote",
|
||||
"version": "2.4.0",
|
||||
"version": "2.4.2",
|
||||
"identifier": "com.jefferyhuang.bilinote",
|
||||
"build": {
|
||||
"frontendDist": "../dist",
|
||||
|
||||
17
CHANGELOG.md
17
CHANGELOG.md
@@ -2,6 +2,23 @@
|
||||
|
||||
本项目所有重要变更记录于此。格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/),遵循 [语义化版本](https://semver.org/lang/zh-CN/)。
|
||||
|
||||
## [2.4.2] - 2026-06-17
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Docker 部署打开显示 nginx 欢迎页**:`nginx/default.conf` 被 docker-compose(多容器)与 `Dockerfile.complete`(单镜像)共用,但两种模式对 `location /` 的需求相反(多容器需反代独立的 frontend 容器,单镜像需直接服务本地静态文件),导致其中一种部署方式总会回退到 nginx 默认欢迎页。现拆分为两份配置:`nginx/default.conf`(compose,反代 frontend 容器)与新增的 `nginx/standalone.conf`(单镜像,静态前端 + 本地 backend 代理);`Dockerfile.complete` 改用后者并删除 Debian 默认站点,两种部署方式均恢复正常。
|
||||
|
||||
## [2.4.1] - 2026-06-17
|
||||
|
||||
### Added
|
||||
|
||||
- **YouTube Shorts 链接支持**:后端 URL 校验与 video id 提取支持 `youtube.com/shorts/<id>` 形态,Shorts 链接可正常提交生成笔记(#381)。
|
||||
|
||||
### Fixed
|
||||
|
||||
- **B 站 412(wbi/playurl 风控)**:B 站 `x/player/wbi/playurl` 网关新增 `dm_img_list`/`dm_img_str`/`dm_cover_img_str`/`dm_img_inter` + `web_location` 风控校验,缺失即返回 HTTP 412。多数视频网页内嵌 playinfo、yt-dlp 不调此 API;而网页不内嵌 playinfo、必须走 API 的视频(如 BV1X9L16oEgB)会撞上风控,刷新 cookie 无效、yt-dlp(含最新版)上游尚未适配。现于 wbi 签名前注入哑值 dm_img 风控参数(形态对齐 yt-dlp 自身 arc/search 用法)恢复 200(#410)。
|
||||
- **B 站分 P 视频字幕取错集**:分 P 视频提交 `?p=N` 时,字幕优先链路未透传 p 参数,始终取第 1 集 cid,导致笔记内容与实际下载的 p=N 音频不一致。现从 `data.pages[N-1]` 取对应分 P 的 cid(#409)。
|
||||
|
||||
## [2.4.0] - 2026-06-07
|
||||
|
||||
### Added
|
||||
|
||||
@@ -90,9 +90,11 @@ WORKDIR /app/backend
|
||||
# 复制前端静态文件到 nginx
|
||||
COPY --from=frontend-builder /tmp/frontend/dist /usr/share/nginx/html
|
||||
|
||||
# 配置 nginx
|
||||
# 配置 nginx(单镜像版:前端静态文件 + 本地 backend 代理,见 nginx/standalone.conf)
|
||||
RUN rm -rf /etc/nginx/conf.d/default.conf
|
||||
COPY ./nginx/default.conf /etc/nginx/conf.d/default.conf
|
||||
# 删除默认 nginx site,防止 default_server 劫持 80 端口
|
||||
RUN rm -f /etc/nginx/sites-enabled/default
|
||||
COPY ./nginx/standalone.conf /etc/nginx/conf.d/default.conf
|
||||
|
||||
# 创建 supervisor 配置
|
||||
# 关键点:supervisord 默认 *不* 把自己的环境变量传给子进程。
|
||||
@@ -127,9 +129,7 @@ priority=20
|
||||
environment=BACKEND_PORT="%(ENV_BACKEND_PORT)s",BACKEND_HOST="%(ENV_BACKEND_HOST)s",TRANSCRIBER_TYPE="%(ENV_TRANSCRIBER_TYPE)s",WHISPER_MODEL_SIZE="%(ENV_WHISPER_MODEL_SIZE)s",FFMPEG_BIN_PATH="%(ENV_FFMPEG_BIN_PATH)s",HF_ENDPOINT="%(ENV_HF_ENDPOINT)s",STATIC="%(ENV_STATIC)s",OUT_DIR="%(ENV_OUT_DIR)s",DATA_DIR="%(ENV_DATA_DIR)s",NOTE_OUTPUT_DIR="%(ENV_NOTE_OUTPUT_DIR)s",DATABASE_URL="%(ENV_DATABASE_URL)s",IMAGE_BASE_URL="%(ENV_IMAGE_BASE_URL)s",ENV="%(ENV_ENV)s",GROQ_TRANSCRIBER_MODEL="%(ENV_GROQ_TRANSCRIBER_MODEL)s"
|
||||
EOF
|
||||
|
||||
# 修改 nginx 配置以使用本地 backend
|
||||
RUN sed -i 's/proxy_pass http:\/\/backend:8483/proxy_pass http:\/\/127.0.0.1:8483/g' /etc/nginx/conf.d/default.conf && \
|
||||
sed -i 's/proxy_pass http:\/\/frontend:80/proxy_pass http:\/\/127.0.0.1:8080/g' /etc/nginx/conf.d/default.conf
|
||||
# nginx/standalone.conf 已直接写好本地 backend(127.0.0.1:8483)与前端静态服务,无需再 sed 改写。
|
||||
|
||||
# 启动 supervisor
|
||||
# 推荐启动方式(覆盖默认 env):
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
<p align="center">
|
||||
<img src="./doc/icon.svg" alt="BiliNote Banner" width="50" height="50" />
|
||||
</p>
|
||||
<h1 align="center" > BiliNote v2.4.0</h1>
|
||||
<h1 align="center" > BiliNote v2.4.2</h1>
|
||||
</div>
|
||||
|
||||
<p align="center"><i>AI 视频笔记生成工具 让 AI 为你的视频做笔记</i></p>
|
||||
|
||||
71
backend/app/downloaders/bilibili_dm_patch.py
Normal file
71
backend/app/downloaders/bilibili_dm_patch.py
Normal file
@@ -0,0 +1,71 @@
|
||||
"""
|
||||
Patch yt-dlp's Bilibili extractor to inject the dm_img_* / web_location
|
||||
risk-control parameters required by Bilibili's wbi/playurl gateway.
|
||||
|
||||
Background
|
||||
----------
|
||||
Around 2026-06 Bilibili's ``x/player/wbi/playurl`` gateway began rejecting
|
||||
requests that omit the browser fingerprint params
|
||||
``dm_img_list`` / ``dm_img_str`` / ``dm_cover_img_str`` / ``dm_img_inter`` +
|
||||
``web_location`` with **HTTP 412**. Current yt-dlp (incl. the latest release)
|
||||
does not send these for the playurl endpoint, so any video whose web page does
|
||||
*not* inline ``playinfo`` — forcing yt-dlp onto the API path — fails with 412.
|
||||
Refreshing cookies does not help; the params themselves are missing.
|
||||
|
||||
We inject dummy-but-well-formed values *before* wbi signing. The value shapes
|
||||
deliberately mirror yt-dlp's own usage of the same fields for the
|
||||
``x/space/wbi/arc/search`` endpoint (``BiliBiliSpaceIE``), which is the only
|
||||
place upstream currently sends them.
|
||||
"""
|
||||
import base64
|
||||
import logging
|
||||
import random
|
||||
import string
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def build_dm_img_params() -> dict:
|
||||
"""Return dummy ``dm_img_*`` / ``web_location`` params the gateway expects."""
|
||||
return {
|
||||
'web_location': 1550101,
|
||||
'dm_img_list': '[]',
|
||||
'dm_img_str': base64.b64encode(
|
||||
''.join(random.choices(string.printable, k=random.randint(16, 64))).encode()
|
||||
)[:-2].decode(),
|
||||
'dm_cover_img_str': base64.b64encode(
|
||||
''.join(random.choices(string.printable, k=random.randint(32, 128))).encode()
|
||||
)[:-2].decode(),
|
||||
'dm_img_inter': '{"ds":[],"wh":[6093,6631,31],"of":[430,760,380]}',
|
||||
}
|
||||
|
||||
|
||||
def apply_bilibili_dm_img_patch() -> bool:
|
||||
"""
|
||||
Monkey-patch ``BilibiliBaseIE._download_playinfo`` to inject dm_img params.
|
||||
|
||||
Idempotent and defensive: returns ``True`` if the patch is in place (whether
|
||||
applied now or previously), ``False`` if yt-dlp's internals could not be
|
||||
patched (logged, never raised — the caller stays functional).
|
||||
"""
|
||||
try:
|
||||
from yt_dlp.extractor.bilibili import BilibiliBaseIE
|
||||
except Exception as e: # yt-dlp missing or module layout changed upstream
|
||||
logger.warning("Bilibili dm_img patch skipped, cannot import extractor: %s", e)
|
||||
return False
|
||||
|
||||
original = BilibiliBaseIE._download_playinfo
|
||||
if getattr(original, '_bili_dm_patched', False):
|
||||
return True
|
||||
|
||||
def _patched_download_playinfo(self, bvid, cid, headers=None, query=None):
|
||||
# dm_* are merged into the query that the original method signs via
|
||||
# _sign_wbi; caller-supplied query params (e.g. try_look/qn) take
|
||||
# precedence over the injected dummies.
|
||||
merged_query = {**build_dm_img_params(), **(query or {})}
|
||||
return original(self, bvid, cid, headers=headers, query=merged_query)
|
||||
|
||||
_patched_download_playinfo._bili_dm_patched = True
|
||||
BilibiliBaseIE._download_playinfo = _patched_download_playinfo
|
||||
logger.info("Applied Bilibili wbi/playurl dm_img patch to yt-dlp BilibiliBaseIE")
|
||||
return True
|
||||
@@ -1,9 +1,6 @@
|
||||
import base64
|
||||
import os
|
||||
import json
|
||||
import logging
|
||||
import random
|
||||
import string
|
||||
import tempfile
|
||||
from abc import ABC
|
||||
from typing import Union, Optional, List
|
||||
@@ -11,6 +8,7 @@ from typing import Union, Optional, List
|
||||
import yt_dlp
|
||||
|
||||
from app.downloaders.base import Downloader, DownloadQuality, QUALITY_MAP
|
||||
from app.downloaders.bilibili_dm_patch import apply_bilibili_dm_img_patch
|
||||
from app.downloaders.bilibili_subtitle import BilibiliSubtitleFetcher
|
||||
from app.models.notes_model import AudioDownloadResult
|
||||
from app.models.transcriber_model import TranscriptResult, TranscriptSegment
|
||||
@@ -20,53 +18,10 @@ from app.services.cookie_manager import CookieConfigManager
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _patch_bilibili_extractor():
|
||||
"""
|
||||
Monkey-patch yt-dlp's BilibiliBaseIE._download_playinfo to inject the
|
||||
dm_img_* / web_location risk-control parameters that Bilibili's
|
||||
wbi/playurl gateway started requiring (returns HTTP 412 without them).
|
||||
|
||||
The parameter format (string.printable source + [:-2] base64 truncation)
|
||||
mirrors yt-dlp's own implementation for the same fields in the channel
|
||||
search endpoint (yt_dlp/extractor/bilibili.py, BiliBiliSpaceIE).
|
||||
"""
|
||||
try:
|
||||
from yt_dlp.extractor.bilibili import BilibiliBaseIE
|
||||
|
||||
# Guard: skip if already patched (e.g. module reloaded in tests)
|
||||
if getattr(BilibiliBaseIE._download_playinfo, '_bili_dm_patched', False):
|
||||
return
|
||||
|
||||
_original_download_playinfo = BilibiliBaseIE._download_playinfo
|
||||
|
||||
def _patched_download_playinfo(self, bvid, cid, headers=None, query=None):
|
||||
# Inject dummy risk-control fingerprints expected by Bilibili's gateway.
|
||||
# The [:-2] truncation and string.printable source intentionally match
|
||||
# yt-dlp's own pattern used for the x/space/wbi/arc/search endpoint.
|
||||
extra = {
|
||||
'web_location': 1550101,
|
||||
'dm_img_list': '[]',
|
||||
'dm_img_str': base64.b64encode(
|
||||
''.join(random.choices(string.printable, k=random.randint(16, 64))).encode()
|
||||
)[:-2].decode(),
|
||||
'dm_cover_img_str': base64.b64encode(
|
||||
''.join(random.choices(string.printable, k=random.randint(32, 128))).encode()
|
||||
)[:-2].decode(),
|
||||
'dm_img_inter': '{"ds":[],"wh":[6093,6631,31],"of":[430,760,380]}',
|
||||
}
|
||||
# Caller-supplied query params take precedence over the dummy values
|
||||
merged_query = {**extra, **(query or {})}
|
||||
return _original_download_playinfo(self, bvid, cid, headers=headers, query=merged_query)
|
||||
|
||||
_patched_download_playinfo._bili_dm_patched = True
|
||||
BilibiliBaseIE._download_playinfo = _patched_download_playinfo
|
||||
logger.info("Applied Bilibili wbi/playurl dm_img patch to yt-dlp BilibiliBaseIE")
|
||||
except Exception as e:
|
||||
logger.warning("Failed to apply Bilibili dm_img patch: %s", e)
|
||||
|
||||
|
||||
_patch_bilibili_extractor()
|
||||
# Inject the dm_img_* / web_location risk-control params Bilibili's wbi/playurl
|
||||
# gateway now requires; without them the API path returns HTTP 412. See
|
||||
# app/downloaders/bilibili_dm_patch.py for details.
|
||||
apply_bilibili_dm_img_patch()
|
||||
|
||||
|
||||
class BilibiliDownloader(Downloader, ABC):
|
||||
|
||||
@@ -3,12 +3,13 @@
|
||||
|
||||
流程:
|
||||
1. 从 URL 提 BV id(已有 utils.url_parser.extract_video_id)
|
||||
2. GET /x/web-interface/view?bvid=BVxxx → 拿 cid
|
||||
3. GET /x/player/wbi/v2?bvid=...&cid=... → 返回 data.subtitle.subtitles[]
|
||||
2. 从 URL 提 p 参数(分 P 序号,已有 utils.url_parser.extract_bilibili_p_number)
|
||||
3. GET /x/web-interface/view?bvid=BVxxx&p=N → 拿第 N 集的 cid
|
||||
4. GET /x/player/wbi/v2?bvid=...&cid=... → 返回 data.subtitle.subtitles[]
|
||||
每条带 subtitle_url(B 站后端已经签好 auth_key 的完整地址)
|
||||
4. 按优先级(人工 zh-CN > AI zh-CN > 任意 zh > 任意非空)选一条
|
||||
5. fetch subtitle_url → JSON {body:[{from,to,content,...}]}
|
||||
6. 解析为 TranscriptResult
|
||||
5. 按优先级(人工 zh-CN > AI zh-CN > 任意 zh > 任意非空)选一条
|
||||
6. fetch subtitle_url → JSON {body:[{from,to,content,...}]}
|
||||
7. 解析为 TranscriptResult
|
||||
|
||||
AI 字幕需要登录态 cookie(SESSDATA);通过 CookieConfigManager 注入。
|
||||
"""
|
||||
@@ -20,7 +21,7 @@ import requests
|
||||
from app.models.transcriber_model import TranscriptResult, TranscriptSegment
|
||||
from app.services.cookie_manager import CookieConfigManager
|
||||
from app.utils.logger import get_logger
|
||||
from app.utils.url_parser import extract_video_id
|
||||
from app.utils.url_parser import extract_video_id, extract_bilibili_p_number, resolve_bilibili_short_url
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
@@ -45,10 +46,13 @@ class BilibiliSubtitleFetcher:
|
||||
h["Cookie"] = self._cookie
|
||||
return h
|
||||
|
||||
def _get_cid(self, bvid: str) -> Optional[int]:
|
||||
def _get_cid(self, bvid: str, p: Optional[int] = None) -> Optional[int]:
|
||||
url = "https://api.bilibili.com/x/web-interface/view"
|
||||
params = {"bvid": bvid}
|
||||
if p is not None and p >= 1:
|
||||
params["p"] = p
|
||||
try:
|
||||
resp = requests.get(url, params={"bvid": bvid}, headers=self._headers(), timeout=10)
|
||||
resp = requests.get(url, params=params, headers=self._headers(), timeout=10)
|
||||
data = resp.json()
|
||||
except Exception as e:
|
||||
logger.warning(f"获取 cid 失败: {e}")
|
||||
@@ -56,6 +60,19 @@ class BilibiliSubtitleFetcher:
|
||||
if data.get("code") != 0:
|
||||
logger.warning(f"view API 返回错误: code={data.get('code')}, msg={data.get('message')}")
|
||||
return None
|
||||
# 分 P 视频:data.pages[N-1] 对应第 N 集
|
||||
pages = data.get("data", {}).get("pages", [])
|
||||
if pages:
|
||||
if p is not None and 1 <= p <= len(pages):
|
||||
cid = pages[p - 1].get("cid")
|
||||
logger.info(f"分 P 视频: bvid={bvid} p={p} 共 {len(pages)} 集, 取第 {p} 集 cid={cid}")
|
||||
return int(cid) if cid else None
|
||||
else:
|
||||
# 没有 p 参数或 p 超出范围,取第 1 集
|
||||
cid = pages[0].get("cid")
|
||||
logger.info(f"非分 P 或 p 无效: bvid={bvid} 取第 1 集 cid={cid}")
|
||||
return int(cid) if cid else None
|
||||
# 单集视频
|
||||
cid = data.get("data", {}).get("cid")
|
||||
return int(cid) if cid else None
|
||||
|
||||
@@ -109,14 +126,21 @@ class BilibiliSubtitleFetcher:
|
||||
return None
|
||||
|
||||
def fetch_subtitles(self, video_url: str) -> Optional[TranscriptResult]:
|
||||
# 统一 resolve 短链,避免 extract_video_id 和 extract_bilibili_p_number 各 resolve 一次
|
||||
if "b23.tv" in video_url:
|
||||
video_url = resolve_bilibili_short_url(video_url) or video_url
|
||||
|
||||
bvid = extract_video_id(video_url, "bilibili")
|
||||
if not bvid:
|
||||
logger.info("无法从 URL 提取 BV id")
|
||||
return None
|
||||
|
||||
cid = self._get_cid(bvid)
|
||||
# 提取分 P 序号
|
||||
p = extract_bilibili_p_number(video_url)
|
||||
|
||||
cid = self._get_cid(bvid, p)
|
||||
if not cid:
|
||||
logger.info(f"{bvid} 没有取到 cid")
|
||||
logger.info(f"{bvid} (p={p}) 没有取到 cid")
|
||||
return None
|
||||
|
||||
subtitles = self._list_subtitles(bvid, cid)
|
||||
@@ -149,7 +173,7 @@ class BilibiliSubtitleFetcher:
|
||||
return None
|
||||
|
||||
full_text = " ".join(s.text for s in segments)
|
||||
logger.info(f"B站直拉字幕成功: {bvid} lan={lan} 共 {len(segments)} 段")
|
||||
logger.info(f"B站直拉字幕成功: {bvid} p={p} lan={lan} 共 {len(segments)} 段")
|
||||
return TranscriptResult(
|
||||
language=lan,
|
||||
full_text=full_text,
|
||||
@@ -158,6 +182,7 @@ class BilibiliSubtitleFetcher:
|
||||
"source": "bilibili_player_api",
|
||||
"bvid": bvid,
|
||||
"cid": cid,
|
||||
"p": p,
|
||||
"lan": lan,
|
||||
"ai_type": track.get("ai_type"),
|
||||
},
|
||||
|
||||
@@ -23,8 +23,8 @@ def extract_video_id(url: str, platform: str) -> Optional[str]:
|
||||
return f"BV{match.group(1)}" if match else None
|
||||
|
||||
elif platform == "youtube":
|
||||
# 匹配 v=xxxxx 或 youtu.be/xxxxx,ID 长度通常为 11
|
||||
match = re.search(r"(?:v=|youtu\.be/)([0-9A-Za-z_-]{11})", url)
|
||||
# 匹配 v=xxxxx、youtu.be/xxxxx 或 shorts/xxxxx,ID 长度通常为 11
|
||||
match = re.search(r"(?:v=|youtu\.be/|shorts/)([0-9A-Za-z_-]{11})", url)
|
||||
return match.group(1) if match else None
|
||||
|
||||
elif platform == "douyin":
|
||||
@@ -48,3 +48,36 @@ def resolve_bilibili_short_url(short_url: str) -> Optional[str]:
|
||||
except requests.RequestException as e:
|
||||
print(f"Error resolving short URL: {e}")
|
||||
return None
|
||||
|
||||
|
||||
def extract_bilibili_p_number(url: str) -> Optional[int]:
|
||||
"""
|
||||
从 B 站分 P 视频 URL 中提取 p 参数(分 P 序号)。
|
||||
|
||||
支持格式:
|
||||
- https://www.bilibili.com/video/BVxxx/?p=36
|
||||
- https://www.bilibili.com/video/BVxxx?p=5
|
||||
- https://b23.tv/xxxxx?p=10
|
||||
- https://www.bilibili.com/video/BVxxx/pN (尾缀形式)
|
||||
|
||||
:param url: B 站视频链接
|
||||
:return: 分 P 序号(从 1 开始),非分 P 视频返回 None
|
||||
"""
|
||||
if "b23.tv" in url:
|
||||
url = resolve_bilibili_short_url(url) or url
|
||||
|
||||
# 匹配 ?p=NNN 或 &p=NNN
|
||||
match = re.search(r'[?&]p=(\d+)', url)
|
||||
if match:
|
||||
p = int(match.group(1))
|
||||
if p >= 1:
|
||||
return p
|
||||
|
||||
# 匹配 /pN 尾缀形式(较少见)
|
||||
match = re.search(r'/p(\d+)(?:/?$|\?|&)', url)
|
||||
if match:
|
||||
p_val = int(match.group(1))
|
||||
if p_val >= 1:
|
||||
return p_val
|
||||
|
||||
return None
|
||||
|
||||
@@ -4,7 +4,7 @@ from urllib.parse import urlparse
|
||||
|
||||
SUPPORTED_PLATFORMS = {
|
||||
"bilibili": r"(https?://)?(www\.)?bilibili\.com/video/[a-zA-Z0-9]+",
|
||||
"youtube": r"(https?://)?(www\.)?(youtube\.com/watch\?v=|youtu\.be/)[\w\-]+",
|
||||
"youtube": r"(https?://)?(www\.)?(youtube\.com/(watch\?v=|shorts/)|youtu\.be/)[\w\-]+",
|
||||
"douyin": "douyin",
|
||||
"kuaishou": "kuaishou"
|
||||
}
|
||||
|
||||
94
backend/tests/test_bilibili_dm_patch.py
Normal file
94
backend/tests/test_bilibili_dm_patch.py
Normal file
@@ -0,0 +1,94 @@
|
||||
"""
|
||||
TDD coverage for the Bilibili wbi/playurl dm_img risk-control patch.
|
||||
|
||||
Background: around 2026-06, Bilibili's `x/player/wbi/playurl` gateway began
|
||||
rejecting requests that omit the browser fingerprint params
|
||||
(dm_img_list / dm_img_str / dm_cover_img_str / dm_img_inter + web_location)
|
||||
with HTTP 412. yt-dlp (incl. latest) does not yet send these for playurl, so
|
||||
videos whose web page does not inline playinfo (forcing the API call) fail.
|
||||
|
||||
These tests verify our yt-dlp monkey-patch injects those params *before* wbi
|
||||
signing, and that caller-supplied query params still win.
|
||||
"""
|
||||
import importlib.util
|
||||
import pathlib
|
||||
import unittest
|
||||
|
||||
ROOT = pathlib.Path(__file__).resolve().parents[1]
|
||||
MODULE_PATH = ROOT / "app" / "downloaders" / "bilibili_dm_patch.py"
|
||||
spec = importlib.util.spec_from_file_location("bilibili_dm_patch", MODULE_PATH)
|
||||
if spec is None or spec.loader is None:
|
||||
raise ImportError("bilibili_dm_patch module spec not found")
|
||||
bilibili_dm_patch = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(bilibili_dm_patch)
|
||||
|
||||
REQUIRED_KEYS = {
|
||||
"web_location",
|
||||
"dm_img_list",
|
||||
"dm_img_str",
|
||||
"dm_cover_img_str",
|
||||
"dm_img_inter",
|
||||
}
|
||||
|
||||
|
||||
class BuildDmImgParamsTest(unittest.TestCase):
|
||||
def test_contains_all_required_risk_control_keys(self):
|
||||
params = bilibili_dm_patch.build_dm_img_params()
|
||||
self.assertTrue(REQUIRED_KEYS.issubset(params.keys()))
|
||||
|
||||
def test_web_location_is_expected_sentinel(self):
|
||||
self.assertEqual(bilibili_dm_patch.build_dm_img_params()["web_location"], 1550101)
|
||||
|
||||
|
||||
class ApplyPatchTest(unittest.TestCase):
|
||||
def setUp(self):
|
||||
try:
|
||||
import yt_dlp.extractor.bilibili # noqa: F401
|
||||
except Exception as exc: # pragma: no cover - env without yt-dlp
|
||||
self.skipTest(f"yt-dlp not importable: {exc}")
|
||||
|
||||
def test_patch_is_idempotent(self):
|
||||
from yt_dlp.extractor.bilibili import BilibiliBaseIE
|
||||
|
||||
self.assertTrue(bilibili_dm_patch.apply_bilibili_dm_img_patch())
|
||||
first = BilibiliBaseIE._download_playinfo
|
||||
self.assertTrue(bilibili_dm_patch.apply_bilibili_dm_img_patch())
|
||||
self.assertIs(BilibiliBaseIE._download_playinfo, first)
|
||||
|
||||
def test_dm_params_reach_wbi_signing_with_caller_query_preserved(self):
|
||||
from yt_dlp import YoutubeDL
|
||||
from yt_dlp.extractor.bilibili import BilibiliBaseIE
|
||||
|
||||
bilibili_dm_patch.apply_bilibili_dm_img_patch()
|
||||
|
||||
captured = {}
|
||||
|
||||
def fake_sign_wbi(params, video_id):
|
||||
# Capture the exact params handed to wbi signing (just before the
|
||||
# HTTP request). dm_* must already be present here, pre-signature.
|
||||
captured.update(params)
|
||||
return params
|
||||
|
||||
def fake_download_json(url, video_id, **kwargs):
|
||||
# Avoid any network; the real playurl call would 412 without dm_*.
|
||||
return {"data": {"ok": True}}
|
||||
|
||||
ie = BilibiliBaseIE(YoutubeDL({"quiet": True}))
|
||||
ie._sign_wbi = fake_sign_wbi
|
||||
ie._download_json = fake_download_json
|
||||
|
||||
ie._download_playinfo("BV1X9L16oEgB", 4242, headers={}, query={"qn": 64})
|
||||
|
||||
self.assertTrue(
|
||||
REQUIRED_KEYS.issubset(captured.keys()),
|
||||
f"missing dm_* keys, got: {sorted(captured)}",
|
||||
)
|
||||
self.assertEqual(captured["web_location"], 1550101)
|
||||
# caller-supplied query must survive the merge
|
||||
self.assertEqual(captured["qn"], 64)
|
||||
# the original method still builds its base params
|
||||
self.assertEqual(captured["bvid"], "BV1X9L16oEgB")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
50
backend/tests/test_video_url_support.py
Normal file
50
backend/tests/test_video_url_support.py
Normal file
@@ -0,0 +1,50 @@
|
||||
import importlib.util
|
||||
import pathlib
|
||||
import unittest
|
||||
|
||||
|
||||
ROOT = pathlib.Path(__file__).resolve().parents[1]
|
||||
|
||||
|
||||
def _load_module(name, relative_path):
|
||||
module_path = ROOT / relative_path
|
||||
spec = importlib.util.spec_from_file_location(name, module_path)
|
||||
if spec is None or spec.loader is None:
|
||||
raise ImportError(f"{name} module spec not found")
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
url_parser = _load_module("url_parser", pathlib.Path("app") / "utils" / "url_parser.py")
|
||||
video_url_validator = _load_module(
|
||||
"video_url_validator",
|
||||
pathlib.Path("app") / "validators" / "video_url_validator.py",
|
||||
)
|
||||
|
||||
|
||||
class TestVideoUrlSupport(unittest.TestCase):
|
||||
def test_extract_youtube_video_id_from_supported_url_shapes(self):
|
||||
expected_id = "dQw4w9WgXcQ"
|
||||
|
||||
cases = [
|
||||
f"https://www.youtube.com/watch?v={expected_id}",
|
||||
f"https://youtu.be/{expected_id}",
|
||||
f"https://www.youtube.com/shorts/{expected_id}",
|
||||
]
|
||||
|
||||
for url in cases:
|
||||
with self.subTest(url=url):
|
||||
self.assertEqual(
|
||||
url_parser.extract_video_id(url, "youtube"),
|
||||
expected_id,
|
||||
)
|
||||
|
||||
def test_accepts_youtube_shorts_url(self):
|
||||
url = "https://www.youtube.com/shorts/dQw4w9WgXcQ"
|
||||
|
||||
self.assertTrue(video_url_validator.is_supported_video_url(url))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -2,19 +2,18 @@ server {
|
||||
listen 80;
|
||||
client_max_body_size 10G;
|
||||
|
||||
# gzip 压缩
|
||||
gzip on;
|
||||
gzip_vary on;
|
||||
gzip_min_length 1024;
|
||||
gzip_proxied any;
|
||||
gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript image/svg+xml;
|
||||
|
||||
# 所有非 /api 请求全部代理给 frontend 容器
|
||||
# 多容器(docker-compose)部署:前端在独立的 frontend 容器,代理过去。
|
||||
# 单镜像(Dockerfile.complete)部署请勿用本文件,改用 nginx/standalone.conf。
|
||||
location / {
|
||||
proxy_pass http://frontend:80;
|
||||
}
|
||||
|
||||
# 所有 /api 请求代理给 backend 容器
|
||||
location /api/ {
|
||||
proxy_pass http://backend:8483;
|
||||
proxy_set_header Host $host;
|
||||
|
||||
41
nginx/standalone.conf
Normal file
41
nginx/standalone.conf
Normal file
@@ -0,0 +1,41 @@
|
||||
# 单镜像(Dockerfile.complete / 一体化部署)专用 nginx 配置。
|
||||
#
|
||||
# 与 nginx/default.conf(docker-compose 多容器版)的关键区别:
|
||||
# - 前端不再由独立的 frontend 容器提供,构建产物已直接 COPY 到本镜像的
|
||||
# /usr/share/nginx/html,所以 location / 走【静态文件】而非反代 frontend;
|
||||
# - backend 与 nginx 同处一个容器,所以 /api、/static 代理到 127.0.0.1:8483。
|
||||
#
|
||||
# 注意:请勿把本文件的 location / 改成代理 frontend,否则单镜像里没有 frontend
|
||||
# 服务,会回退到 nginx 默认欢迎页。多容器(compose)请改 nginx/default.conf。
|
||||
server {
|
||||
listen 80;
|
||||
client_max_body_size 10G;
|
||||
|
||||
gzip on;
|
||||
gzip_vary on;
|
||||
gzip_min_length 1024;
|
||||
gzip_proxied any;
|
||||
gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript image/svg+xml;
|
||||
|
||||
# 前端静态文件由本容器直接服务(构建产物已 COPY 到此目录)
|
||||
location / {
|
||||
root /usr/share/nginx/html;
|
||||
index index.html;
|
||||
try_files $uri $uri/ /index.html;
|
||||
}
|
||||
|
||||
# backend 与 nginx 同容器,代理到本地
|
||||
location /api/ {
|
||||
proxy_pass http://127.0.0.1:8483;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
}
|
||||
|
||||
location /static/ {
|
||||
proxy_pass http://127.0.0.1:8483/static/;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
expires 7d;
|
||||
add_header Cache-Control "public, immutable";
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user