mirror of
https://github.com/JefferyHcool/BiliNote.git
synced 2026-08-28 19:46:57 +08:00
pumpkinperson996
3e579b1434
## 问题
输入 YouTube 链接生成笔记时任务必定失败：
ERROR: [youtube] <id>: Requested format is not available
即使字幕已经抓取成功（日志里能看到"成功获取 YouTube 字幕，共 N 段"），
任务仍然在下载阶段崩掉。
## 原因
两个独立的问题叠加：
1. `requirements.txt` 把 yt-dlp 钉在 `2025.3.31`。YouTube 之后轮换过
player，旧版 yt-dlp 解不出 nsig 签名，所有音视频格式都被丢弃，只剩下
storyboard 图片（日志：`Only images are available for download`），
格式选择随即抛错。
2. 有字幕时 `NoteGenerator` 走的是"只取元信息"的路径
（`download(skip_download=True)`，只要 title/duration/cover），但
`YoutubeDownloader.download` 无条件设置了
`format='bestaudio[ext=m4a]/bestaudio/best'`。于是一个根本不需要媒体流的
调用，也会因为选不出格式而失败——把一个字幕已经到手的任务整个带崩。
## 改动
- `youtube_downloader.py`：`skip_download` 时设置
`ignore_no_formats_error=True`。yt-dlp 落后于 YouTube 时，只取元信息的路径
降级为"没有音频"，而不是让整个笔记失败。
- `youtube_downloader.py`：`ext = info.get("ext") or "m4a"`。跳过下载时
yt-dlp 返回 `ext=None`，`dict.get` 的默认值对显式 None 不生效，
会拼出 `xxx.None` 这样的路径。
- `requirements.txt`：`yt-dlp==2025.3.31` → `>=2026.7.4`。yt-dlp 是对抗
YouTube 变化的滚动依赖，精确钉版本本身就是这个 bug 的成因；用 `>=` 与同文件
的 `youtube-transcript-api>=1.0.0` 保持一致。
- `bilibili_dm_patch.py`：wrapper 改为透传 `**kwargs`。升级 yt-dlp 后
`_real_extract` 会以 `fatal=False` 调用 `_download_playinfo`，而 wrapper
钉死了签名，导致 **所有 B 站下载** 抛
`TypeError: ... got an unexpected keyword argument 'fatal'`。
- 测试：新增 `test_youtube_metadata_only.py` 覆盖上面两条 YouTube 保证；
`test_bilibili_dm_patch.py` 新增未知 kwargs 透传用例，并让 fake 响应带上
`code` 字段（yt-dlp 2026.x 会先校验信封再返回 data）。
## 验证
- 真实跑通：YouTube（有字幕，走元信息路径）与 B 站（无字幕，走完整下载 +
转写）均能生成笔记。
- `pytest tests/` → 46 passed。唯一失败的
`test_task_serial_executor` 在升级前后表现一致，与本次改动无关，未作改动。
75 lines
3.2 KiB
Python
75 lines
3.2 KiB
Python
"""
|
|
Patch yt-dlp's Bilibili extractor to inject the dm_img_* / web_location
|
|
risk-control parameters required by Bilibili's wbi/playurl gateway.
|
|
|
|
Background
|
|
----------
|
|
Around 2026-06 Bilibili's ``x/player/wbi/playurl`` gateway began rejecting
|
|
requests that omit the browser fingerprint params
|
|
``dm_img_list`` / ``dm_img_str`` / ``dm_cover_img_str`` / ``dm_img_inter`` +
|
|
``web_location`` with **HTTP 412**. Current yt-dlp (incl. the latest release)
|
|
does not send these for the playurl endpoint, so any video whose web page does
|
|
*not* inline ``playinfo`` — forcing yt-dlp onto the API path — fails with 412.
|
|
Refreshing cookies does not help; the params themselves are missing.
|
|
|
|
We inject dummy-but-well-formed values *before* wbi signing. The value shapes
|
|
deliberately mirror yt-dlp's own usage of the same fields for the
|
|
``x/space/wbi/arc/search`` endpoint (``BiliBiliSpaceIE``), which is the only
|
|
place upstream currently sends them.
|
|
"""
|
|
import base64
|
|
import logging
|
|
import random
|
|
import string
|
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
|
|
def build_dm_img_params() -> dict:
|
|
"""Return dummy ``dm_img_*`` / ``web_location`` params the gateway expects."""
|
|
return {
|
|
'web_location': 1550101,
|
|
'dm_img_list': '[]',
|
|
'dm_img_str': base64.b64encode(
|
|
''.join(random.choices(string.printable, k=random.randint(16, 64))).encode()
|
|
)[:-2].decode(),
|
|
'dm_cover_img_str': base64.b64encode(
|
|
''.join(random.choices(string.printable, k=random.randint(32, 128))).encode()
|
|
)[:-2].decode(),
|
|
'dm_img_inter': '{"ds":[],"wh":[6093,6631,31],"of":[430,760,380]}',
|
|
}
|
|
|
|
|
|
def apply_bilibili_dm_img_patch() -> bool:
|
|
"""
|
|
Monkey-patch ``BilibiliBaseIE._download_playinfo`` to inject dm_img params.
|
|
|
|
Idempotent and defensive: returns ``True`` if the patch is in place (whether
|
|
applied now or previously), ``False`` if yt-dlp's internals could not be
|
|
patched (logged, never raised — the caller stays functional).
|
|
"""
|
|
try:
|
|
from yt_dlp.extractor.bilibili import BilibiliBaseIE
|
|
except Exception as e: # yt-dlp missing or module layout changed upstream
|
|
logger.warning("Bilibili dm_img patch skipped, cannot import extractor: %s", e)
|
|
return False
|
|
|
|
original = BilibiliBaseIE._download_playinfo
|
|
if getattr(original, '_bili_dm_patched', False):
|
|
return True
|
|
|
|
def _patched_download_playinfo(self, bvid, cid, headers=None, query=None, **kwargs):
|
|
# dm_* are merged into the query that the original method signs via
|
|
# _sign_wbi; caller-supplied query params (e.g. try_look/qn) take
|
|
# precedence over the injected dummies.
|
|
# **kwargs stays open on purpose: yt-dlp keeps adding parameters to
|
|
# _download_playinfo (2026.x added `fatal`), and a wrapper that pins the
|
|
# signature turns every such addition into a TypeError at download time.
|
|
merged_query = {**build_dm_img_params(), **(query or {})}
|
|
return original(self, bvid, cid, headers=headers, query=merged_query, **kwargs)
|
|
|
|
_patched_download_playinfo._bili_dm_patched = True
|
|
BilibiliBaseIE._download_playinfo = _patched_download_playinfo
|
|
logger.info("Applied Bilibili wbi/playurl dm_img patch to yt-dlp BilibiliBaseIE")
|
|
return True
|