Compare commits

...

89 Commits

Author SHA1 Message Date
jxxghp
634e5a4c55 Merge pull request #4496 from wikrin/v2 2025-06-29 07:51:24 +08:00
Attente
332b154f15 fix(api): 适配 FastAPI 请求参数兼容性问题
修复系统配置和用户配置接口无法正常工作的问题。
2025-06-29 05:31:25 +08:00
jxxghp
b446d4db28 更新 GitHub 工作流配置,排除带有 RFC 标签的 issue 2025-06-28 22:24:51 +08:00
jxxghp
ce0397a140 fix update.sh 2025-06-28 22:03:18 +08:00
jxxghp
f278cccef3 for test 2025-06-28 21:42:28 +08:00
jxxghp
cbf1dbcd2e fix 恢复插件后安装依赖 2025-06-28 21:42:03 +08:00
jxxghp
037c6b02fa Merge pull request #4493 from Miralia/v2 2025-06-28 20:07:12 +08:00
Miralia
5f44e4322d Fix and add more 2025-06-28 19:47:33 +08:00
Miralia
6cebe97d6d add FPT Play 2025-06-28 19:12:00 +08:00
jxxghp
82ec146446 更新 plugin.py 2025-06-28 16:49:09 +08:00
jxxghp
3928c352c6 fix update 2025-06-28 15:01:25 +08:00
jxxghp
0ba36d21a9 Revert "fix security"
This reverts commit c7800df801.
2025-06-28 14:37:22 +08:00
jxxghp
6152727e9b fix Dockerfile 2025-06-28 14:33:33 +08:00
jxxghp
53c02fa706 resource v2 2025-06-28 14:26:14 +08:00
jxxghp
c7800df801 fix security 2025-06-28 14:12:24 +08:00
jxxghp
562c1de0c9 aList => OpenList 2025-06-28 08:43:09 +08:00
jxxghp
e2c90639f3 更新 message.py 2025-06-27 19:54:13 +08:00
jxxghp
92e175a8d1 Merge pull request #4488 from Miralia/v2 2025-06-27 17:29:10 +08:00
jxxghp
cf7bca75f6 fix res.text 2025-06-27 17:23:32 +08:00
Miralia
24a173f075 Update streamingplatform.py 2025-06-27 17:21:27 +08:00
jxxghp
8d695dda55 fix log 2025-06-27 17:16:08 +08:00
jxxghp
93eec6c4b8 fix cache 2025-06-27 15:24:57 +08:00
jxxghp
a2cc1a2926 upgrade packages 2025-06-27 14:34:35 +08:00
jxxghp
11729d0eca fix 2025-06-27 13:34:27 +08:00
jxxghp
978819be38 fix db pool size 2025-06-27 12:41:03 +08:00
jxxghp
23c9862eb3 fix site parser 2025-06-27 12:26:17 +08:00
jxxghp
a9f18ea3ef fix #4475 2025-06-27 10:05:19 +08:00
jxxghp
574257edf8 add SystemConfModel 2025-06-27 09:54:15 +08:00
jxxghp
bb4438ac42 feat:非大内存模式下主动gc 2025-06-27 09:44:47 +08:00
jxxghp
0baf6e5fe7 fix SiteParser close session 2025-06-27 08:38:02 +08:00
jxxghp
d8a53da8ee auto close RequestUtils 2025-06-27 08:30:57 +08:00
jxxghp
9555ac6305 fix RequestUtils 2025-06-27 08:09:38 +08:00
jxxghp
4dd5ea8e2f add del 2025-06-27 07:53:10 +08:00
jxxghp
8068523d88 fix downloader 2025-06-26 20:52:17 +08:00
jxxghp
27dd681d9f fix RequestUtils 2025-06-26 17:36:22 +08:00
jxxghp
152f814fb6 fix base chain 2025-06-26 13:28:11 +08:00
jxxghp
2700e639f1 fix chain 2025-06-26 13:16:10 +08:00
jxxghp
c440ce3045 fix oper 2025-06-26 08:33:43 +08:00
jxxghp
2829a3cb4e fix 2025-06-26 08:18:37 +08:00
jxxghp
a487091be8 Revert "fix resource helper"
This reverts commit e7524774da.
2025-06-25 13:32:28 +08:00
jxxghp
e7524774da fix resource helper 2025-06-25 12:50:00 +08:00
jxxghp
3918c876c5 Merge pull request #4478 from Miralia/v2 2025-06-24 21:07:55 +08:00
Miralia
f07f87735c fix 2025-06-24 19:52:14 +08:00
Miralia
b7566e8fe8 feat(meta): 扩展流媒体平台列表,增加更多平台支持。 2025-06-24 19:46:01 +08:00
jxxghp
73eba90f2f 更新 version.py 2025-06-24 10:34:42 +08:00
jxxghp
62e74f6fd1 fix 2025-06-24 08:19:10 +08:00
jxxghp
4375e48840 Merge pull request #4476 from Miralia/v2 2025-06-23 20:52:15 +08:00
Miralia
a1d6e94e90 feat(meta): 新增 WEB 平台来源识别并支持更多音视频格式。 2025-06-23 20:36:58 +08:00
jxxghp
1f44e13ff0 add reload logging 2025-06-23 10:14:22 +08:00
jxxghp
d2992f9ced fix plugin load 2025-06-23 09:31:56 +08:00
jxxghp
950337bccc fix plugin load 2025-06-23 08:19:22 +08:00
jxxghp
757c3be359 更新 version.py 2025-06-22 10:08:17 +08:00
jxxghp
269ab9adfc fix:删除消息能力 2025-06-22 10:04:21 +08:00
jxxghp
bd241a5164 feat:删除消息能力 2025-06-22 09:37:01 +08:00
jxxghp
3d92b57f24 fix 2025-06-22 09:04:03 +08:00
jxxghp
70d8cb3697 fix #4461 2025-06-22 08:51:29 +08:00
jxxghp
9e4ec5841c fix #4470 2025-06-22 08:47:43 +08:00
jxxghp
682f4fe608 fix message cache 2025-06-20 17:33:08 +08:00
jxxghp
ce8a077e07 优化按钮回调数据,简化为仅使用索引值 2025-06-19 15:54:07 +08:00
jxxghp
d5f63bcdb3 remove Commands DEV flag 2025-06-18 13:33:37 +08:00
jxxghp
5c3756fd1b v2.5.7-1 2025-06-17 20:02:45 +08:00
jxxghp
99939e1a3d fix 2025-06-17 19:42:16 +08:00
jxxghp
56742ace11 fix:带UA下载图片 2025-06-17 19:27:53 +08:00
jxxghp
742cb7a8da 更新 version.py 2025-06-17 18:56:47 +08:00
jxxghp
98327d1750 fix download message 2025-06-17 15:35:38 +08:00
jxxghp
b944306302 v2.5.7 2025-06-16 22:15:54 +08:00
jxxghp
02ab1d4111 fix settings 2025-06-16 21:29:57 +08:00
jxxghp
28552fb0ce 更新 transmission.py 2025-06-16 19:38:19 +08:00
jxxghp
bf52fcb2ec fix message 2025-06-16 11:45:26 +08:00
jxxghp
bab1f73480 修复:slack消息交互 2025-06-16 09:49:01 +08:00
jxxghp
c06001d921 feat:内建重启前主动备份插件 2025-06-16 08:57:21 +08:00
jxxghp
0fa49bb9c6 fix 消息定向发送时不检查消息类型匹配 2025-06-16 08:06:47 +08:00
jxxghp
bf23fe6ce2 更新 subscribe.py 2025-06-15 23:31:13 +08:00
jxxghp
7c6137b742 更新 download.py 2025-06-15 23:30:01 +08:00
jxxghp
3823a7c9b6 fix:消息发送范围 2025-06-15 23:18:07 +08:00
jxxghp
a944975be2 fix:交互消息立即发送 2025-06-15 23:06:25 +08:00
jxxghp
6da65d3b03 add MessageAction 2025-06-15 21:25:14 +08:00
jxxghp
0d938f2dca refactor:减少Alipan及115的Api调用 2025-06-15 20:41:32 +08:00
jxxghp
4fa9bb3c1f feat: 插件消息的事件回调 [PLUGIN]插件ID|内容 2025-06-15 19:47:04 +08:00
jxxghp
2f5b22a81f fix 2025-06-15 19:41:24 +08:00
jxxghp
fcd5ca3fda feat:Slack支持编辑消息 2025-06-15 19:28:05 +08:00
jxxghp
c18247f3b1 增强消息处理功能,支持编辑消息 2025-06-15 19:18:18 +08:00
jxxghp
f8fbfdbba7 优化消息处理逻辑 2025-06-15 18:40:36 +08:00
jxxghp
21addfb947 更新 message.py 2025-06-15 16:56:48 +08:00
jxxghp
8672bd12c4 fix bug 2025-06-15 16:31:09 +08:00
jxxghp
be8054e81e fix bug 2025-06-15 15:57:58 +08:00
jxxghp
82f46c6010 feat:回调消息路由给插件 2025-06-15 15:56:38 +08:00
jxxghp
95a827e8a2 feat:Telegram、Slack 支持按钮 2025-06-15 15:34:06 +08:00
jxxghp
c534e3dcb8 feat:未安装的插件,不加载模块 2025-06-15 09:55:20 +08:00
91 changed files with 3934 additions and 2395 deletions

View File

@@ -27,4 +27,6 @@ jobs:
# 忽略所有的 Pull Request只处理 Issue
days-before-pr-stale: -1
days-before-pr-close: -1
# 排除带有RFC标签的issue
exempt-issue-labels: "RFC"
repo-token: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -1,5 +1,6 @@
from typing import List, Any, Optional
import jieba
from fastapi import APIRouter, Depends
from sqlalchemy.orm import Session
@@ -57,6 +58,8 @@ def transfer_history(title: Optional[str] = None,
status = True
if title:
words = jieba.cut(title, HMM=False)
title = "%".join(words)
total = TransferHistory.count_by_title(db, title=title, status=status)
result = TransferHistory.list_by_title(db, title=title, page=page,
count=count, status=status)

View File

@@ -147,7 +147,7 @@ def all_plugins(_: schemas.TokenPayload = Depends(get_current_active_superuser),
installed_plugins = [plugin for plugin in local_plugins if plugin.installed]
if state == "installed":
return installed_plugins
# 未安装的本地插件
not_installed_plugins = [plugin for plugin in local_plugins if not plugin.installed]
# 在线插件
@@ -178,7 +178,7 @@ def all_plugins(_: schemas.TokenPayload = Depends(get_current_active_superuser),
if state == "market":
# 返回未安装的插件
return market_plugins
# 返回所有插件
return installed_plugins + market_plugins
@@ -523,7 +523,7 @@ def clone_plugin(plugin_id: str,
version=clone_data.get("version", ""),
icon=clone_data.get("icon", "")
)
if success:
# 注册插件服务
reload_plugin(message)
@@ -547,7 +547,7 @@ def _add_clone_to_plugin_folder(original_plugin_id: str, clone_plugin_id: str):
config_oper = SystemConfigOper()
# 获取插件文件夹配置
folders = config_oper.get(SystemConfigKey.PluginFolders) or {}
# 查找原插件所在的文件夹
target_folder = None
for folder_name, folder_data in folders.items():
@@ -561,7 +561,7 @@ def _add_clone_to_plugin_folder(original_plugin_id: str, clone_plugin_id: str):
if original_plugin_id in folder_data:
target_folder = folder_name
break
# 如果找到了原插件所在的文件夹,则将分身插件也添加到该文件夹中
if target_folder:
folder_data = folders[target_folder]
@@ -575,12 +575,12 @@ def _add_clone_to_plugin_folder(original_plugin_id: str, clone_plugin_id: str):
if clone_plugin_id not in folder_data:
folder_data.append(clone_plugin_id)
logger.info(f"已将分身插件 {clone_plugin_id} 添加到文件夹 '{target_folder}'")
# 保存更新后的文件夹配置
config_oper.set(SystemConfigKey.PluginFolders, folders)
else:
logger.info(f"原插件 {original_plugin_id} 不在任何文件夹中,分身插件 {clone_plugin_id} 将保持独立")
except Exception as e:
logger.error(f"处理插件文件夹时出错:{str(e)}")
# 文件夹处理失败不影响插件分身创建的整体流程
@@ -595,10 +595,10 @@ def _remove_plugin_from_folders(plugin_id: str):
config_oper = SystemConfigOper()
# 获取插件文件夹配置
folders = config_oper.get(SystemConfigKey.PluginFolders) or {}
# 标记是否有修改
modified = False
# 遍历所有文件夹,移除指定插件
for folder_name, folder_data in folders.items():
if isinstance(folder_data, dict) and 'plugins' in folder_data:
@@ -613,13 +613,13 @@ def _remove_plugin_from_folders(plugin_id: str):
folder_data.remove(plugin_id)
logger.info(f"已从文件夹 '{folder_name}' 中移除插件 {plugin_id}")
modified = True
# 如果有修改,保存更新后的文件夹配置
if modified:
config_oper.set(SystemConfigKey.PluginFolders, folders)
else:
logger.debug(f"插件 {plugin_id} 不在任何文件夹中,无需移除")
except Exception as e:
logger.error(f"从文件夹中移除插件时出错:{str(e)}")
# 文件夹处理失败不影响插件卸载的整体流程

View File

@@ -11,7 +11,7 @@ from typing import Optional, Union, Annotated
import aiofiles
import pillow_avif # noqa 用于自动注册AVIF支持
from PIL import Image
from fastapi import APIRouter, Depends, HTTPException, Header, Request, Response
from fastapi import APIRouter, Body, Depends, HTTPException, Header, Request, Response
from fastapi.responses import StreamingResponse
from app import schemas
@@ -288,8 +288,11 @@ def get_setting(key: str,
@router.post("/setting/{key}", summary="更新系统设置", response_model=schemas.Response)
def set_setting(key: str, value: Union[list, dict, bool, int, str] = None,
_: User = Depends(get_current_active_superuser)):
def set_setting(
key: str,
value: Annotated[Union[list, dict, bool, int, str] | None, Body()] = None,
_: User = Depends(get_current_active_superuser),
):
"""
更新系统设置(仅管理员)
"""

View File

@@ -1,8 +1,8 @@
import base64
import re
from typing import Any, List, Union
from typing import Annotated, Any, List, Union
from fastapi import APIRouter, Depends, HTTPException, UploadFile, File
from fastapi import APIRouter, Body, Depends, HTTPException, UploadFile, File
from sqlalchemy.orm import Session
from app import schemas
@@ -164,8 +164,11 @@ def get_config(key: str,
@router.post("/config/{key}", summary="更新用户配置", response_model=schemas.Response)
def set_config(key: str, value: Union[list, dict, bool, int, str] = None,
current_user: User = Depends(get_current_active_user)):
def set_config(
key: str,
value: Annotated[Union[list, dict, bool, int, str] | None, Body()] = None,
current_user: User = Depends(get_current_active_user),
):
"""
更新用户配置
"""

View File

@@ -22,7 +22,7 @@ from app.helper.service import ServiceConfigHelper
from app.log import logger
from app.schemas import TransferInfo, TransferTorrent, ExistMediaInfo, DownloadingTorrent, CommingMessage, Notification, \
WebhookEventInfo, TmdbEpisode, MediaPerson, FileItem, TransferDirectoryConf
from app.schemas.types import TorrentStatus, MediaType, MediaImageType, EventType
from app.schemas.types import TorrentStatus, MediaType, MediaImageType, EventType, MessageChannel
from app.utils.object import ObjectUtils
@@ -94,9 +94,8 @@ class ChainBase(metaclass=ABCMeta):
return ret is None
result = None
plugin_modules = self.pluginmanager.get_plugin_modules()
# 插件模块
for plugin, module_dict in plugin_modules.items():
for plugin, module_dict in self.pluginmanager.get_plugin_modules().items():
plugin_id, plugin_name = plugin
if method in module_dict:
func = module_dict[method]
@@ -138,10 +137,7 @@ class ChainBase(metaclass=ABCMeta):
# 系统模块
logger.debug(f"请求系统模块执行:{method} ...")
modules = self.modulemanager.get_running_modules(method)
# 按优先级排序
modules = sorted(modules, key=lambda x: x.get_priority())
for module in modules:
for module in sorted(self.modulemanager.get_running_modules(method), key=lambda x: x.get_priority()):
module_id = module.__class__.__name__
try:
module_name = module.get_name()
@@ -612,7 +608,8 @@ class ChainBase(metaclass=ABCMeta):
# 发送消息事件
self.eventmanager.send_event(etype=EventType.NoticeMessage, data={**message.dict(), "type": message.mtype})
# 按原消息发送
self.messagequeue.send_message("post_message", message=message)
self.messagequeue.send_message("post_message", message=message,
immediately=True if message.userid else False)
def post_medias_message(self, message: Notification, medias: List[MediaInfo]) -> None:
"""
@@ -624,7 +621,8 @@ class ChainBase(metaclass=ABCMeta):
note_list = [media.to_dict() for media in medias]
self.messagehelper.put(message, role="user", note=note_list, title=message.title)
self.messageoper.add(**message.dict(), note=note_list)
return self.messagequeue.send_message("post_medias_message", message=message, medias=medias)
return self.messagequeue.send_message("post_medias_message", message=message, medias=medias,
immediately=True if message.userid else False)
def post_torrents_message(self, message: Notification, torrents: List[Context]) -> None:
"""
@@ -636,7 +634,21 @@ class ChainBase(metaclass=ABCMeta):
note_list = [torrent.torrent_info.to_dict() for torrent in torrents]
self.messagehelper.put(message, role="user", note=note_list, title=message.title)
self.messageoper.add(**message.dict(), note=note_list)
return self.messagequeue.send_message("post_torrents_message", message=message, torrents=torrents)
return self.messagequeue.send_message("post_torrents_message", message=message, torrents=torrents,
immediately=True if message.userid else False)
def delete_message(self, channel: MessageChannel, source: str,
message_id: Union[str, int], chat_id: Optional[Union[str, int]] = None) -> bool:
"""
删除消息
:param channel: 消息渠道
:param source: 消息源(指定特定的消息模块)
:param message_id: 消息ID
:param chat_id: 聊天ID如群组ID
:return: 删除是否成功
"""
return self.run_module("delete_message", channel=channel, source=source,
message_id=message_id, chat_id=chat_id)
def metadata_img(self, mediainfo: MediaInfo,
season: Optional[int] = None, episode: Optional[int] = None) -> Optional[dict]:

View File

@@ -324,10 +324,12 @@ class DownloadChain(ChainBase):
self.post_message(
Notification(
channel=channel,
source=source if channel else None,
mtype=NotificationType.Download,
ctype=ContentType.DownloadAdded,
image=_media.get_message_image(),
link=settings.MP_DOMAIN('/#/downloading'),
userid=userid,
username=username
),
meta=_meta,

View File

@@ -427,7 +427,7 @@ class MediaChain(ChainBase):
"""
try:
logger.info(f"正在下载图片:{_url} ...")
r = RequestUtils(proxies=settings.PROXY).get_res(url=_url)
r = RequestUtils(proxies=settings.PROXY, ua=settings.USER_AGENT).get_res(url=_url)
if r:
return r.content
else:
@@ -506,7 +506,9 @@ class MediaChain(ChainBase):
# 根据图片类型检查开关
if 'poster' in image_name.lower():
should_scrape = scraping_switchs.get('movie_poster', True)
elif 'backdrop' in image_name.lower() or 'fanart' in image_name.lower():
elif ('backdrop' in image_name.lower()
or 'fanart' in image_name.lower()
or 'background' in image_name.lower()):
should_scrape = scraping_switchs.get('movie_backdrop', True)
elif 'logo' in image_name.lower():
should_scrape = scraping_switchs.get('movie_logo', True)
@@ -700,7 +702,9 @@ class MediaChain(ChainBase):
# 根据电视剧图片类型检查开关
if 'poster' in image_name.lower():
should_scrape = scraping_switchs.get('tv_poster', True)
elif 'backdrop' in image_name.lower() or 'fanart' in image_name.lower():
elif ('backdrop' in image_name.lower()
or 'fanart' in image_name.lower()
or 'background' in image_name.lower()):
should_scrape = scraping_switchs.get('tv_backdrop', True)
elif 'banner' in image_name.lower():
should_scrape = scraping_switchs.get('tv_banner', True)

File diff suppressed because it is too large Load Diff

View File

@@ -202,16 +202,15 @@ class SearchChain(ChainBase):
# 过滤完成
progress.update(value=50, text=f'过滤完成,剩余 {len(torrents)} 个资源', key=ProgressKey.Search)
# 开始匹配
_match_torrents = []
# 总数
_total = len(torrents)
# 已处理数
_count = 0
# 开始匹配
_match_torrents = []
torrenthelper = TorrentHelper()
if mediainfo:
try:
# 英文标题应该在别名/原标题中,不需要再匹配
logger.info(f"开始匹配结果 标题:{mediainfo.title},原标题:{mediainfo.original_title},别名:{mediainfo.names}")
progress.update(value=51, text=f'开始匹配,总 {_total} 个资源 ...', key=ProgressKey.Search)
@@ -256,16 +255,18 @@ class SearchChain(ChainBase):
progress.update(value=97,
text=f'匹配完成,共匹配到 {len(_match_torrents)} 个资源',
key=ProgressKey.Search)
else:
_match_torrents = [(t, MetaInfo(title=t.title, subtitle=t.description)) for t in torrents]
# 去掉mediainfo中多余的数据
mediainfo.clear()
# 组装上下文
contexts = [Context(torrent_info=t[0],
media_info=mediainfo,
meta_info=t[1]) for t in _match_torrents]
# 去掉mediainfo中多余的数据
mediainfo.clear()
# 组装上下文
contexts = [Context(torrent_info=t[0],
media_info=mediainfo,
meta_info=t[1]) for t in _match_torrents]
finally:
torrents.clear()
del torrents
_match_torrents.clear()
del _match_torrents
# 排序
progress.update(value=99,
@@ -364,6 +365,7 @@ class SearchChain(ChainBase):
logger.info(f"站点搜索完成,有效资源数:{len(results)},总耗时 {(end_time - start_time).seconds}")
# 结束进度
progress.end(ProgressKey.Search)
# 返回
return results

View File

@@ -1,4 +1,5 @@
import base64
import gc
import re
from datetime import datetime
from typing import Optional, Tuple, Union, Dict
@@ -92,10 +93,9 @@ class SiteChain(ChainBase):
"""
刷新所有站点的用户数据
"""
sites = SitesHelper().get_indexers()
any_site_updated = False
result = {}
for site in sites:
for site in SitesHelper().get_indexers():
if global_vars.is_system_stopped:
return None
if site.get("is_active"):
@@ -107,6 +107,11 @@ class SiteChain(ChainBase):
EventManager().send_event(EventType.SiteRefreshed, {
"site_id": "*"
})
# 如果不是大内存模式,进行垃圾回收
if not settings.BIG_MEMORY_MODE:
gc.collect()
return result
def is_special_site(self, domain: str) -> bool:
@@ -351,9 +356,10 @@ class SiteChain(ChainBase):
ua=settings.USER_AGENT
).get_res(url=domain_url)
if res and res.status_code in [200, 500, 403]:
if not indexer.get("public") and not SiteUtils.is_logged_in(res.text):
content = res.text
if not indexer.get("public") and not SiteUtils.is_logged_in(content):
_fail_count += 1
if under_challenge(res.text):
if under_challenge(content):
logger.warn(f"站点 {indexer.get('name')} 被Cloudflare防护无法登录无法添加站点")
continue
logger.warn(
@@ -571,8 +577,9 @@ class SiteChain(ChainBase):
).get_res(url=site_url)
# 判断登录状态
if res and res.status_code in [200, 500, 403]:
if not public and not SiteUtils.is_logged_in(res.text):
if under_challenge(res.text):
content = res.text
if not public and not SiteUtils.is_logged_in(content):
if under_challenge(content):
msg = "站点被Cloudflare防护请打开站点浏览器仿真"
elif res.status_code == 200:
msg = "Cookie已失效"

View File

@@ -1,4 +1,5 @@
import copy
import gc
import json
import random
import threading
@@ -221,10 +222,13 @@ class SubscribeChain(ChainBase):
# 订阅成功按规则发送消息
self.post_message(
schemas.Notification(
channel=channel,
source=source,
mtype=NotificationType.Subscribe,
ctype=ContentType.SubscribeAdded,
image=mediainfo.get_message_image(),
link=link,
userid=userid,
username=username
),
meta=metainfo,
@@ -283,148 +287,161 @@ class SubscribeChain(ChainBase):
subscribes = [subscribe] if subscribe else []
else:
subscribes = subscribeoper.list(self.get_states_for_search(state))
# 遍历订阅
for subscribe in subscribes:
if global_vars.is_system_stopped:
break
mediakey = subscribe.tmdbid or subscribe.doubanid
custom_word_list = subscribe.custom_words.split("\n") if subscribe.custom_words else None
# 校验当前时间减订阅创建时间是否大于1分钟否则跳过先留出编辑订阅的时间
if subscribe.date:
now = datetime.now()
subscribe_time = datetime.strptime(subscribe.date, '%Y-%m-%d %H:%M:%S')
if (now - subscribe_time).total_seconds() < 60:
logger.debug(f"订阅标题:{subscribe.name} 新增小于1分钟暂不搜索...")
continue
# 随机休眠1-5分钟
if not sid and state in ['R', 'P']:
sleep_time = random.randint(60, 300)
logger.info(f'订阅搜索随机休眠 {sleep_time} 秒 ...')
time.sleep(sleep_time)
try:
logger.info(f'开始搜索订阅,标题:{subscribe.name} ...')
# 生成元数据
meta = MetaInfo(subscribe.name)
meta.year = subscribe.year
meta.begin_season = subscribe.season or None
try:
# 遍历订阅
for subscribe in subscribes:
if global_vars.is_system_stopped:
break
mediakey = subscribe.tmdbid or subscribe.doubanid
custom_word_list = subscribe.custom_words.split("\n") if subscribe.custom_words else None
# 校验当前时间减订阅创建时间是否大于1分钟否则跳过先留出编辑订阅的时间
if subscribe.date:
now = datetime.now()
subscribe_time = datetime.strptime(subscribe.date, '%Y-%m-%d %H:%M:%S')
if (now - subscribe_time).total_seconds() < 60:
logger.debug(f"订阅标题:{subscribe.name} 新增小于1分钟暂不搜索...")
continue
# 随机休眠1-5分钟
if not sid and state in ['R', 'P']:
sleep_time = random.randint(60, 300)
logger.info(f'订阅搜索随机休眠 {sleep_time} 秒 ...')
time.sleep(sleep_time)
try:
meta.type = MediaType(subscribe.type)
except ValueError:
logger.error(f'订阅 {subscribe.name} 类型错误:{subscribe.type}')
continue
# 识别媒体信息
mediainfo: MediaInfo = self.recognize_media(meta=meta, mtype=meta.type,
tmdbid=subscribe.tmdbid,
doubanid=subscribe.doubanid,
episode_group=subscribe.episode_group,
cache=False)
if not mediainfo:
logger.warn(
f'未识别到媒体信息,标题:{subscribe.name}tmdbid{subscribe.tmdbid}doubanid{subscribe.doubanid}')
continue
logger.info(f'开始搜索订阅,标题:{subscribe.name} ...')
# 生成元数据
meta = MetaInfo(subscribe.name)
meta.year = subscribe.year
meta.begin_season = subscribe.season or None
try:
meta.type = MediaType(subscribe.type)
except ValueError:
logger.error(f'订阅 {subscribe.name} 类型错误:{subscribe.type}')
continue
# 识别媒体信息
mediainfo: MediaInfo = self.recognize_media(meta=meta, mtype=meta.type,
tmdbid=subscribe.tmdbid,
doubanid=subscribe.doubanid,
episode_group=subscribe.episode_group,
cache=False)
if not mediainfo:
logger.warn(
f'未识别到媒体信息,标题:{subscribe.name}tmdbid{subscribe.tmdbid}doubanid{subscribe.doubanid}')
continue
# 如果媒体已存在或已下载完毕,跳过当前订阅处理
exist_flag, no_exists = self.check_and_handle_existing_media(subscribe=subscribe,
meta=meta,
mediainfo=mediainfo,
mediakey=mediakey)
if exist_flag:
continue
# 如果媒体已存在或已下载完毕,跳过当前订阅处理
exist_flag, no_exists = self.check_and_handle_existing_media(subscribe=subscribe,
meta=meta,
mediainfo=mediainfo,
mediakey=mediakey)
if exist_flag:
continue
# 站点范围
sites = self.get_sub_sites(subscribe)
# 站点范围
sites = self.get_sub_sites(subscribe)
# 优先级过滤规则
if subscribe.best_version:
rule_groups = subscribe.filter_groups \
or SystemConfigOper().get(SystemConfigKey.BestVersionFilterRuleGroups) or []
else:
rule_groups = subscribe.filter_groups \
or SystemConfigOper().get(SystemConfigKey.SubscribeFilterRuleGroups) or []
# 搜索,同时电视剧会过滤掉不需要的剧集
contexts = SearchChain().process(mediainfo=mediainfo,
keyword=subscribe.keyword,
no_exists=no_exists,
sites=sites,
rule_groups=rule_groups,
area="imdbid" if subscribe.search_imdbid else "title",
custom_words=custom_word_list,
filter_params=self.get_params(subscribe))
if not contexts:
logger.warn(f'订阅 {subscribe.keyword or subscribe.name} 未搜索到资源')
self.finish_subscribe_or_not(subscribe=subscribe, meta=meta,
mediainfo=mediainfo, lefts=no_exists)
continue
# 过滤搜索结果
matched_contexts = []
for context in contexts:
if global_vars.is_system_stopped:
break
torrent_meta = context.meta_info
torrent_info = context.torrent_info
torrent_mediainfo = context.media_info
# 洗版
# 优先级过滤规则
if subscribe.best_version:
# 洗版时,非整季不要
if torrent_mediainfo.type == MediaType.TV:
if torrent_meta.episode_list:
logger.info(f'{subscribe.name} 正在洗版,{torrent_info.title} 不是整季')
continue
# 洗版时,优先级小于等于已下载优先级的不要
if subscribe.current_priority \
and torrent_info.pri_order <= subscribe.current_priority:
logger.info(
f'{subscribe.name} 正在洗版,{torrent_info.title} 优先级低于或等于已下载优先级')
continue
# 更新订阅自定义属性
if subscribe.media_category:
torrent_mediainfo.category = subscribe.media_category
if subscribe.episode_group:
torrent_mediainfo.episode_group = subscribe.episode_group
matched_contexts.append(context)
rule_groups = subscribe.filter_groups \
or SystemConfigOper().get(SystemConfigKey.BestVersionFilterRuleGroups) or []
else:
rule_groups = subscribe.filter_groups \
or SystemConfigOper().get(SystemConfigKey.SubscribeFilterRuleGroups) or []
if not matched_contexts:
logger.warn(f'订阅 {subscribe.name} 没有符合过滤条件的资源')
self.finish_subscribe_or_not(subscribe=subscribe, meta=meta,
mediainfo=mediainfo, lefts=no_exists)
continue
# 搜索,同时电视剧会过滤掉不需要的剧集
contexts = SearchChain().process(mediainfo=mediainfo,
keyword=subscribe.keyword,
no_exists=no_exists,
sites=sites,
rule_groups=rule_groups,
area="imdbid" if subscribe.search_imdbid else "title",
custom_words=custom_word_list,
filter_params=self.get_params(subscribe))
if not contexts:
logger.warn(f'订阅 {subscribe.keyword or subscribe.name} 未搜索到资源')
self.finish_subscribe_or_not(subscribe=subscribe, meta=meta,
mediainfo=mediainfo, lefts=no_exists)
continue
# 自动下载
downloads, lefts = DownloadChain().batch_download(
contexts=matched_contexts,
no_exists=no_exists,
userid=subscribe.username,
username=subscribe.username,
save_path=subscribe.save_path,
downloader=subscribe.downloader,
source=self.get_subscribe_source_keyword(subscribe)
)
# 过滤搜索结果
matched_contexts = []
try:
for context in contexts:
if global_vars.is_system_stopped:
break
torrent_meta = context.meta_info
torrent_info = context.torrent_info
torrent_mediainfo = context.media_info
# 同步外部修改,更新订阅信息
subscribe = subscribeoper.get(subscribe.id)
# 洗版
if subscribe.best_version:
# 洗版时,非整季不要
if torrent_mediainfo.type == MediaType.TV:
if torrent_meta.episode_list:
logger.info(f'{subscribe.name} 正在洗版,{torrent_info.title} 不是整季')
continue
# 洗版时,优先级小于等于已下载优先级的不要
if subscribe.current_priority \
and torrent_info.pri_order <= subscribe.current_priority:
logger.info(
f'{subscribe.name} 正在洗版,{torrent_info.title} 优先级低于或等于已下载优先级')
continue
# 更新订阅自定义属性
if subscribe.media_category:
torrent_mediainfo.category = subscribe.media_category
if subscribe.episode_group:
torrent_mediainfo.episode_group = subscribe.episode_group
matched_contexts.append(context)
finally:
contexts.clear()
del contexts
# 判断是否应完成订阅
if subscribe:
self.finish_subscribe_or_not(subscribe=subscribe, meta=meta, mediainfo=mediainfo,
downloads=downloads, lefts=lefts)
finally:
# 如果状态为N则更新为R
if subscribe and subscribe.state == 'N':
subscribeoper.update(subscribe.id, {'state': 'R'})
if not matched_contexts:
logger.warn(f'订阅 {subscribe.name} 没有符合过滤条件的资源')
self.finish_subscribe_or_not(subscribe=subscribe, meta=meta,
mediainfo=mediainfo, lefts=no_exists)
continue
# 手动触发时发送系统消息
if manual:
if subscribes:
if sid:
self.messagehelper.put(f'{subscribes[0].name} 搜索完成!', title="订阅搜索", role="system")
# 自动下载
downloads, lefts = DownloadChain().batch_download(
contexts=matched_contexts,
no_exists=no_exists,
username=subscribe.username,
save_path=subscribe.save_path,
downloader=subscribe.downloader,
source=self.get_subscribe_source_keyword(subscribe)
)
# 同步外部修改,更新订阅信息
subscribe = subscribeoper.get(subscribe.id)
# 判断是否应完成订阅
if subscribe:
self.finish_subscribe_or_not(subscribe=subscribe, meta=meta, mediainfo=mediainfo,
downloads=downloads, lefts=lefts)
finally:
# 如果状态为N则更新为R
if subscribe and subscribe.state == 'N':
subscribeoper.update(subscribe.id, {'state': 'R'})
# 手动触发时发送系统消息
if manual:
if subscribes:
if sid:
self.messagehelper.put(f'{subscribes[0].name} 搜索完成!', title="订阅搜索", role="system")
else:
self.messagehelper.put('所有订阅搜索完成!', title="订阅搜索", role="system")
else:
self.messagehelper.put('所有订阅搜索完成', title="订阅搜索", role="system")
else:
self.messagehelper.put('没有找到订阅!', title="订阅搜索", role="system")
logger.debug(f"search Lock released at {datetime.now()}")
self.messagehelper.put('没有找到订阅', title="订阅搜索", role="system")
logger.debug(f"search Lock released at {datetime.now()}")
finally:
subscribes.clear()
del subscribes
# 如果不是大内存模式,进行垃圾回收
if not settings.BIG_MEMORY_MODE:
gc.collect()
def update_subscribe_priority(self, subscribe: Subscribe, meta: MetaBase,
mediainfo: MediaInfo, downloads: Optional[List[Context]]):
@@ -497,6 +514,9 @@ class SubscribeChain(ChainBase):
self.match(
TorrentsChain().refresh(sites=sites)
)
# 如果不是大内存模式,进行垃圾回收
if not settings.BIG_MEMORY_MODE:
gc.collect()
@staticmethod
def get_sub_sites(subscribe: Subscribe) -> List[int]:
@@ -525,13 +545,9 @@ class SubscribeChain(ChainBase):
获取订阅中涉及的所有站点清单(节约资源)
:return: 返回[]代表所有站点命中返回None代表没有订阅
"""
# 查询所有订阅
subscribes = SubscribeOper().list(self.get_states_for_search('R'))
if not subscribes:
return None
ret_sites = []
# 刷新订阅选中的Rss站点
for subscribe in subscribes:
for subscribe in SubscribeOper().list(self.get_states_for_search('R')):
# 刷新选中的站点
ret_sites.extend(self.get_sub_sites(subscribe))
# 去重
@@ -550,8 +566,6 @@ class SubscribeChain(ChainBase):
with self._rlock:
logger.debug(f"match lock acquired at {datetime.now()}")
# 所有订阅
subscribes = SubscribeOper().list(self.get_states_for_search('R'))
# 预识别所有未识别的种子
processed_torrents: Dict[str, List[Context]] = {}
@@ -574,232 +588,242 @@ class SubscribeChain(ChainBase):
# 添加已预处理
processed_torrents[domain].append(context)
# 遍历订阅
for subscribe in subscribes:
if global_vars.is_system_stopped:
break
logger.info(f'开始匹配订阅,标题:{subscribe.name} ...')
mediakey = subscribe.tmdbid or subscribe.doubanid
# 生成元数据
meta = MetaInfo(subscribe.name)
meta.year = subscribe.year
meta.begin_season = subscribe.season or None
try:
meta.type = MediaType(subscribe.type)
except ValueError:
logger.error(f'订阅 {subscribe.name} 类型错误:{subscribe.type}')
continue
# 订阅的站点域名列表
domains = []
if subscribe.sites:
domains = SiteOper().get_domains_by_ids(subscribe.sites)
# 识别媒体信息
mediainfo: MediaInfo = self.recognize_media(meta=meta, mtype=meta.type,
tmdbid=subscribe.tmdbid,
doubanid=subscribe.doubanid,
episode_group=subscribe.episode_group,
cache=False)
if not mediainfo:
logger.warn(
f'未识别到媒体信息,标题:{subscribe.name}tmdbid{subscribe.tmdbid}doubanid{subscribe.doubanid}')
continue
# 如果媒体已存在或已下载完毕,跳过当前订阅处理
exist_flag, no_exists = self.check_and_handle_existing_media(subscribe=subscribe, meta=meta,
mediainfo=mediainfo,
mediakey=mediakey)
if exist_flag:
continue
# 清理多余信息
mediainfo.clear()
# 订阅识别词
if subscribe.custom_words:
custom_words_list = subscribe.custom_words.split("\n")
else:
custom_words_list = None
# 遍历预识别后的种子
_match_context = []
torrenthelper = TorrentHelper()
systemconfig = SystemConfigOper()
wordsmatcher = WordsMatcher()
for domain, contexts in processed_torrents.items():
# 所有订阅
subscribes = SubscribeOper().list(self.get_states_for_search('R'))
try:
for subscribe in subscribes:
if global_vars.is_system_stopped:
break
if domains and domain not in domains:
logger.info(f'开始匹配订阅,标题:{subscribe.name} ...')
mediakey = subscribe.tmdbid or subscribe.doubanid
# 生成元数据
meta = MetaInfo(subscribe.name)
meta.year = subscribe.year
meta.begin_season = subscribe.season or None
try:
meta.type = MediaType(subscribe.type)
except ValueError:
logger.error(f'订阅 {subscribe.name} 类型错误:{subscribe.type}')
continue
logger.debug(f'开始匹配站点:{domain},共缓存了 {len(contexts)} 个种子...')
for context in contexts:
# 订阅的站点域名列表
domains = []
if subscribe.sites:
domains = SiteOper().get_domains_by_ids(subscribe.sites)
# 识别媒体信息
mediainfo: MediaInfo = self.recognize_media(meta=meta, mtype=meta.type,
tmdbid=subscribe.tmdbid,
doubanid=subscribe.doubanid,
episode_group=subscribe.episode_group,
cache=False)
if not mediainfo:
logger.warn(
f'未识别到媒体信息,标题:{subscribe.name}tmdbid{subscribe.tmdbid}doubanid{subscribe.doubanid}')
continue
# 如果媒体已存在或已下载完毕,跳过当前订阅处理
exist_flag, no_exists = self.check_and_handle_existing_media(subscribe=subscribe, meta=meta,
mediainfo=mediainfo,
mediakey=mediakey)
if exist_flag:
continue
# 清理多余信息
mediainfo.clear()
# 订阅识别词
if subscribe.custom_words:
custom_words_list = subscribe.custom_words.split("\n")
else:
custom_words_list = None
# 遍历预识别后的种子
_match_context = []
torrenthelper = TorrentHelper()
systemconfig = SystemConfigOper()
wordsmatcher = WordsMatcher()
for domain, contexts in processed_torrents.items():
if global_vars.is_system_stopped:
break
# 提取信息
_context = copy.copy(context)
torrent_meta = _context.meta_info
torrent_mediainfo = _context.media_info
torrent_info = _context.torrent_info
# 不在订阅站点范围的不处理
sub_sites = self.get_sub_sites(subscribe)
if sub_sites and torrent_info.site not in sub_sites:
logger.debug(f"{torrent_info.site_name} - {torrent_info.title} 不符合订阅站点要求")
if domains and domain not in domains:
continue
logger.debug(f'开始匹配站点:{domain},共缓存了 {len(contexts)} 个种子...')
try:
for context in contexts:
if global_vars.is_system_stopped:
break
# 提取信息
_context = copy.copy(context)
torrent_meta = _context.meta_info
torrent_mediainfo = _context.media_info
torrent_info = _context.torrent_info
# 有自定义识别词时,需要判断是否需要重新识别
if custom_words_list:
# 使用org_string应用一次后理论上不能再次应用
_, apply_words = wordsmatcher.prepare(torrent_meta.org_string,
custom_words=custom_words_list)
if apply_words:
logger.info(
f'{torrent_info.site_name} - {torrent_info.title} 因订阅存在自定义识别词,重新识别元数据...')
# 重新识别元数据
torrent_meta = MetaInfo(title=torrent_info.title, subtitle=torrent_info.description,
custom_words=custom_words_list)
# 更新元数据缓存
_context.meta_info = torrent_meta
# 重新识别媒体信息
torrent_mediainfo = self.recognize_media(meta=torrent_meta,
episode_group=subscribe.episode_group)
if torrent_mediainfo:
# 清理多余信息
torrent_mediainfo.clear()
# 更新种子缓存
_context.media_info = torrent_mediainfo
# 如果仍然没有识别到媒体信息,尝试标题匹配
if not torrent_mediainfo or (not torrent_mediainfo.tmdb_id and not torrent_mediainfo.douban_id):
logger.info(
f'{torrent_info.site_name} - {torrent_info.title} 重新识别失败,尝试通过标题匹配...')
if torrenthelper.match_torrent(mediainfo=mediainfo,
torrent_meta=torrent_meta,
torrent=torrent_info):
# 匹配成功
logger.info(
f'{mediainfo.title_year} 通过标题匹配到可选资源:{torrent_info.site_name} - {torrent_info.title}')
torrent_mediainfo = mediainfo
# 更新种子缓存
_context.media_info = mediainfo
else:
continue
# 直接比对媒体信息
if torrent_mediainfo and (torrent_mediainfo.tmdb_id or torrent_mediainfo.douban_id):
if torrent_mediainfo.type != mediainfo.type:
continue
if torrent_mediainfo.tmdb_id \
and torrent_mediainfo.tmdb_id != mediainfo.tmdb_id:
continue
if torrent_mediainfo.douban_id \
and torrent_mediainfo.douban_id != mediainfo.douban_id:
continue
logger.info(
f'{mediainfo.title_year} 通过媒体信ID匹配到可选资源{torrent_info.site_name} - {torrent_info.title}')
else:
continue
# 如果是电视剧
if torrent_mediainfo.type == MediaType.TV:
# 有多季的不要
if len(torrent_meta.season_list) > 1:
logger.debug(f'{torrent_info.title} 有多季,不处理')
continue
# 比对季
if torrent_meta.begin_season:
if meta.begin_season != torrent_meta.begin_season:
logger.debug(f'{torrent_info.title} 季不匹配')
# 不在订阅站点范围的不处理
sub_sites = self.get_sub_sites(subscribe)
if sub_sites and torrent_info.site not in sub_sites:
logger.debug(f"{torrent_info.site_name} - {torrent_info.title} 不符合订阅站点要求")
continue
elif meta.begin_season != 1:
logger.debug(f'{torrent_info.title} 季不匹配')
continue
# 非洗版
if not subscribe.best_version:
# 不是缺失的剧集不要
if no_exists and no_exists.get(mediakey):
# 缺失集
no_exists_info = no_exists.get(mediakey).get(subscribe.season)
if no_exists_info:
# 是否有交集
if no_exists_info.episodes and \
torrent_meta.episode_list and \
not set(no_exists_info.episodes).intersection(
set(torrent_meta.episode_list)
):
logger.debug(
f'{torrent_info.title} 对应剧集 {torrent_meta.episode_list} 未包含缺失的剧集'
)
continue
else:
# 洗版时,非整季不要
if meta.type == MediaType.TV:
if torrent_meta.episode_list:
logger.debug(f'{subscribe.name} 正在洗版,{torrent_info.title} 不是整季')
# 有自定义识别词时,需要判断是否需要重新识别
if custom_words_list:
# 使用org_string应用一次后理论上不能再次应用
_, apply_words = wordsmatcher.prepare(torrent_meta.org_string,
custom_words=custom_words_list)
if apply_words:
logger.info(
f'{torrent_info.site_name} - {torrent_info.title} 因订阅存在自定义识别词,重新识别元数据...')
# 重新识别元数据
torrent_meta = MetaInfo(title=torrent_info.title, subtitle=torrent_info.description,
custom_words=custom_words_list)
# 更新元数据缓存
_context.meta_info = torrent_meta
# 重新识别媒体信息
torrent_mediainfo = self.recognize_media(meta=torrent_meta,
episode_group=subscribe.episode_group)
if torrent_mediainfo:
# 清理多余信息
torrent_mediainfo.clear()
# 更新种子缓存
_context.media_info = torrent_mediainfo
# 如果仍然没有识别到媒体信息,尝试标题匹配
if not torrent_mediainfo or (not torrent_mediainfo.tmdb_id and not torrent_mediainfo.douban_id):
logger.info(
f'{torrent_info.site_name} - {torrent_info.title} 重新识别失败,尝试通过标题匹配...')
if torrenthelper.match_torrent(mediainfo=mediainfo,
torrent_meta=torrent_meta,
torrent=torrent_info):
# 匹配成功
logger.info(
f'{mediainfo.title_year} 通过标题匹配到可选资源:{torrent_info.site_name} - {torrent_info.title}')
torrent_mediainfo = mediainfo
# 更新种子缓存
_context.media_info = mediainfo
else:
continue
# 匹配订阅附加参数
if not torrenthelper.filter_torrent(torrent_info=torrent_info,
filter_params=self.get_params(subscribe)):
continue
# 直接比对媒体信息
if torrent_mediainfo and (torrent_mediainfo.tmdb_id or torrent_mediainfo.douban_id):
if torrent_mediainfo.type != mediainfo.type:
continue
if torrent_mediainfo.tmdb_id \
and torrent_mediainfo.tmdb_id != mediainfo.tmdb_id:
continue
if torrent_mediainfo.douban_id \
and torrent_mediainfo.douban_id != mediainfo.douban_id:
continue
logger.info(
f'{mediainfo.title_year} 通过媒体信ID匹配到可选资源{torrent_info.site_name} - {torrent_info.title}')
else:
continue
# 优先级过滤规则
if subscribe.best_version:
rule_groups = subscribe.filter_groups \
or systemconfig.get(SystemConfigKey.BestVersionFilterRuleGroups)
else:
rule_groups = subscribe.filter_groups \
or systemconfig.get(SystemConfigKey.SubscribeFilterRuleGroups)
result: List[TorrentInfo] = self.filter_torrents(
rule_groups=rule_groups,
torrent_list=[torrent_info],
mediainfo=torrent_mediainfo)
if result is not None and not result:
# 不符合过滤规则
logger.debug(f"{torrent_info.title} 不匹配过滤规则")
continue
# 如果是电视剧
if torrent_mediainfo.type == MediaType.TV:
# 有多季的不要
if len(torrent_meta.season_list) > 1:
logger.debug(f'{torrent_info.title} 有多季,不处理')
continue
# 比对季
if torrent_meta.begin_season:
if meta.begin_season != torrent_meta.begin_season:
logger.debug(f'{torrent_info.title} 季不匹配')
continue
elif meta.begin_season != 1:
logger.debug(f'{torrent_info.title} 季不匹配')
continue
# 非洗版
if not subscribe.best_version:
# 不是缺失的剧集不要
if no_exists and no_exists.get(mediakey):
# 缺失集
no_exists_info = no_exists.get(mediakey).get(subscribe.season)
if no_exists_info:
# 是否有交集
if no_exists_info.episodes and \
torrent_meta.episode_list and \
not set(no_exists_info.episodes).intersection(
set(torrent_meta.episode_list)
):
logger.debug(
f'{torrent_info.title} 对应剧集 {torrent_meta.episode_list} 未包含缺失的剧集'
)
continue
else:
# 洗版时,非整季不要
if meta.type == MediaType.TV:
if torrent_meta.episode_list:
logger.debug(f'{subscribe.name} 正在洗版,{torrent_info.title} 不是整季')
continue
# 洗版时,优先级小于已下载优先级的不要
if subscribe.best_version:
if subscribe.current_priority \
and torrent_info.pri_order <= subscribe.current_priority:
logger.info(
f'{subscribe.name} 正在洗版,{torrent_info.title} 优先级低于或等于已下载优先级')
continue
# 匹配订阅附加参数
if not torrenthelper.filter_torrent(torrent_info=torrent_info,
filter_params=self.get_params(subscribe)):
continue
# 匹配成功
logger.info(f'{mediainfo.title_year} 匹配成功:{torrent_info.title}')
# 自定义属性
if subscribe.media_category:
torrent_mediainfo.category = subscribe.media_category
if subscribe.episode_group:
torrent_mediainfo.episode_group = subscribe.episode_group
_match_context.append(_context)
# 优先级过滤规则
if subscribe.best_version:
rule_groups = subscribe.filter_groups \
or systemconfig.get(SystemConfigKey.BestVersionFilterRuleGroups)
else:
rule_groups = subscribe.filter_groups \
or systemconfig.get(SystemConfigKey.SubscribeFilterRuleGroups)
result: List[TorrentInfo] = self.filter_torrents(
rule_groups=rule_groups,
torrent_list=[torrent_info],
mediainfo=torrent_mediainfo)
if result is not None and not result:
# 不符合过滤规则
logger.debug(f"{torrent_info.title} 不匹配过滤规则")
continue
if not _match_context:
# 未匹配到资源
logger.info(f'{mediainfo.title_year} 未匹配到符合条件的资源')
self.finish_subscribe_or_not(subscribe=subscribe, meta=meta,
mediainfo=mediainfo, lefts=no_exists)
continue
# 洗版时,优先级小于已下载优先级的不要
if subscribe.best_version:
if subscribe.current_priority \
and torrent_info.pri_order <= subscribe.current_priority:
logger.info(
f'{subscribe.name} 正在洗版,{torrent_info.title} 优先级低于或等于已下载优先级')
continue
# 开始批量择优下载
logger.info(f'{mediainfo.title_year} 匹配完成,共匹配到{len(_match_context)}个资源')
downloads, lefts = DownloadChain().batch_download(contexts=_match_context,
no_exists=no_exists,
userid=subscribe.username,
username=subscribe.username,
save_path=subscribe.save_path,
downloader=subscribe.downloader,
source=self.get_subscribe_source_keyword(subscribe)
)
# 匹配成功
logger.info(f'{mediainfo.title_year} 匹配成功:{torrent_info.title}')
# 自定义属性
if subscribe.media_category:
torrent_mediainfo.category = subscribe.media_category
if subscribe.episode_group:
torrent_mediainfo.episode_group = subscribe.episode_group
_match_context.append(_context)
finally:
contexts.clear()
del contexts
# 同步外部修改,更新订阅信息
subscribe = SubscribeOper().get(subscribe.id)
if not _match_context:
# 未匹配到资源
logger.info(f'{mediainfo.title_year} 未匹配到符合条件的资源')
self.finish_subscribe_or_not(subscribe=subscribe, meta=meta,
mediainfo=mediainfo, lefts=no_exists)
continue
# 判断是否要完成订阅
if subscribe:
self.finish_subscribe_or_not(subscribe=subscribe, meta=meta, mediainfo=mediainfo,
downloads=downloads, lefts=lefts)
# 开始批量择优下载
logger.info(f'{mediainfo.title_year} 匹配完成,共匹配到{len(_match_context)}个资源')
downloads, lefts = DownloadChain().batch_download(contexts=_match_context,
no_exists=no_exists,
username=subscribe.username,
save_path=subscribe.save_path,
downloader=subscribe.downloader,
source=self.get_subscribe_source_keyword(subscribe)
)
# 同步外部修改,更新订阅信息
subscribe = SubscribeOper().get(subscribe.id)
# 判断是否要完成订阅
if subscribe:
self.finish_subscribe_or_not(subscribe=subscribe, meta=meta, mediainfo=mediainfo,
downloads=downloads, lefts=lefts)
finally:
processed_torrents.clear()
del processed_torrents
subscribes.clear()
del subscribes
logger.debug(f"match Lock released at {datetime.now()}")
@@ -809,12 +833,8 @@ class SubscribeChain(ChainBase):
"""
# 查询所有订阅
subscribeoper = SubscribeOper()
subscribes = subscribeoper.list()
if not subscribes:
# 没有订阅不运行
return
# 遍历订阅
for subscribe in subscribes:
for subscribe in subscribeoper.list():
if global_vars.is_system_stopped:
break
logger.info(f'开始更新订阅元数据:{subscribe.name} ...')
@@ -870,11 +890,10 @@ class SubscribeChain(ChainBase):
follow_users: List[str] = SystemConfigOper().get(SystemConfigKey.FollowSubscribers)
if not follow_users:
return
share_subs = SubscribeHelper().get_shares()
logger.info(f'开始刷新follow用户分享订阅 ...')
success_count = 0
subscribeoper = SubscribeOper()
for share_sub in share_subs:
for share_sub in SubscribeHelper().get_shares():
if global_vars.is_system_stopped:
break
uid = share_sub.get("share_uid")

View File

@@ -1,5 +1,6 @@
import json
import re
import shutil
from pathlib import Path
from typing import Union, Optional
@@ -10,6 +11,7 @@ from app.schemas import Notification, MessageChannel
from app.utils.http import RequestUtils
from app.utils.system import SystemUtils
from app.helper.system import SystemHelper
from app.helper.plugin import PluginHelper
from version import FRONTEND_VERSION, APP_VERSION
@@ -42,11 +44,124 @@ class SystemChain(ChainBase):
"channel": channel.value,
"userid": userid
}, self._restart_file)
# 主动备份一次插件
self.backup_plugins()
# 设置停止标志,通知所有模块准备停止
global_vars.stop_system()
# 重启
SystemHelper.restart()
@staticmethod
def backup_plugins():
"""
备份插件到用户配置目录仅docker环境
"""
# 非docker环境不处理
if not SystemUtils.is_docker():
return
try:
# 使用绝对路径确保准确性
plugins_dir = settings.ROOT_PATH / "app" / "plugins"
backup_dir = settings.CONFIG_PATH / "plugins_backup"
if not plugins_dir.exists():
logger.info("插件目录不存在,跳过备份")
return
# 确保备份目录存在
backup_dir.mkdir(parents=True, exist_ok=True)
# 需要排除的文件和目录
exclude_items = {"__init__.py", "__pycache__", ".DS_Store"}
# 遍历插件目录,备份除排除项外的所有内容
for item in plugins_dir.iterdir():
if item.name in exclude_items:
continue
target_path = backup_dir / item.name
# 如果是目录
if item.is_dir():
if target_path.exists():
continue
shutil.copytree(item, target_path)
logger.info(f"已备份插件目录: {item.name}")
# 如果是文件
elif item.is_file():
if target_path.exists():
continue
shutil.copy2(item, target_path)
logger.info(f"已备份插件文件: {item.name}")
logger.info(f"插件备份完成,备份位置: {backup_dir}")
except Exception as e:
logger.error(f"插件备份失败: {str(e)}")
@staticmethod
def restore_plugins():
"""
从备份恢复插件到app/plugins目录恢复完成后删除备份仅docker环境
"""
# 非docker环境不处理
if not SystemUtils.is_docker():
return
# 使用绝对路径确保准确性
plugins_dir = settings.ROOT_PATH / "app" / "plugins"
backup_dir = settings.CONFIG_PATH / "plugins_backup"
if not backup_dir.exists():
logger.info("插件备份目录不存在,跳过恢复")
return
# 系统被重置才恢复插件
if SystemHelper().is_system_reset():
# 确保插件目录存在
plugins_dir.mkdir(parents=True, exist_ok=True)
# 遍历备份目录,恢复所有内容
restored_count = 0
for item in backup_dir.iterdir():
target_path = plugins_dir / item.name
try:
# 如果是目录,且目录内有内容
if item.is_dir() and any(item.iterdir()):
if target_path.exists():
shutil.rmtree(target_path)
shutil.copytree(item, target_path)
logger.info(f"已恢复插件目录: {item.name}")
# 安装依赖
requirements_file = target_path / "requirements.txt"
if requirements_file.exists():
logger.info(f"正在安装插件 {item.name} 的依赖...")
success, message = PluginHelper.pip_install_with_fallback(requirements_file)
if not success:
logger.warn(f"插件 {item.name} 依赖安装失败: {message}")
restored_count += 1
# 如果是文件
elif item.is_file():
shutil.copy2(item, target_path)
logger.info(f"已恢复插件文件: {item.name}")
restored_count += 1
except Exception as e:
logger.error(f"恢复插件 {item.name} 时发生错误: {str(e)}")
continue
logger.info(f"插件恢复完成,共恢复 {restored_count} 个项目")
# 删除备份目录
try:
shutil.rmtree(backup_dir)
logger.info(f"已删除插件备份目录: {backup_dir}")
except Exception as e:
logger.warning(f"删除备份目录失败: {str(e)}")
def __get_version_message(self) -> str:
"""
获取版本信息文本

View File

@@ -98,6 +98,7 @@ class TorrentsChain(ChainBase):
if not site.get("rss"):
logger.error(f'站点 {domain} 未配置RSS地址')
return []
# 解析RSS
rss_items = RssHelper().parse(site.get("rss"), True if site.get("proxy") else False,
timeout=int(site.get("timeout") or 30))
if rss_items is None:
@@ -109,25 +110,28 @@ class TorrentsChain(ChainBase):
return []
# 组装种子
ret_torrents: List[TorrentInfo] = []
for item in rss_items:
if not item.get("title"):
continue
torrentinfo = TorrentInfo(
site=site.get("id"),
site_name=site.get("name"),
site_cookie=site.get("cookie"),
site_ua=site.get("ua") or settings.USER_AGENT,
site_proxy=site.get("proxy"),
site_order=site.get("pri"),
site_downloader=site.get("downloader"),
title=item.get("title"),
enclosure=item.get("enclosure"),
page_url=item.get("link"),
size=item.get("size"),
pubdate=item["pubdate"].strftime("%Y-%m-%d %H:%M:%S") if item.get("pubdate") else None,
)
ret_torrents.append(torrentinfo)
try:
for item in rss_items:
if not item.get("title"):
continue
torrentinfo = TorrentInfo(
site=site.get("id"),
site_name=site.get("name"),
site_cookie=site.get("cookie"),
site_ua=site.get("ua") or settings.USER_AGENT,
site_proxy=site.get("proxy"),
site_order=site.get("pri"),
site_downloader=site.get("downloader"),
title=item.get("title"),
enclosure=item.get("enclosure"),
page_url=item.get("link"),
size=item.get("size"),
pubdate=item["pubdate"].strftime("%Y-%m-%d %H:%M:%S") if item.get("pubdate") else None,
)
ret_torrents.append(torrentinfo)
finally:
rss_items.clear()
del rss_items
return ret_torrents
def refresh(self, stype: Optional[str] = None, sites: List[int] = None) -> Dict[str, List[Context]]:
@@ -152,13 +156,10 @@ class TorrentsChain(ChainBase):
torrents_cache[_domain] = [_torrent for _torrent in _torrents
if not TorrentHelper().is_invalid(_torrent.torrent_info.enclosure)]
# 所有站点索引
indexers = SitesHelper().get_indexers()
# 需要刷新的站点domain
domains = []
# 遍历站点缓存资源
for indexer in indexers:
for indexer in SitesHelper().get_indexers():
if global_vars.is_system_stopped:
break
# 未开启的站点不刷新
@@ -175,7 +176,7 @@ class TorrentsChain(ChainBase):
# 按pubdate降序排列
torrents.sort(key=lambda x: x.pubdate or '', reverse=True)
# 取前N条
torrents = torrents[:settings.CONF["refresh"]]
torrents = torrents[:settings.CONF.refresh]
if torrents:
# 过滤出没有处理过的种子 - 优化:使用集合查找,避免重复创建字符串列表
cached_signatures = {f'{t.torrent_info.title}{t.torrent_info.description}'
@@ -187,36 +188,40 @@ class TorrentsChain(ChainBase):
else:
logger.info(f'{indexer.get("name")} 没有新种子')
continue
for torrent in torrents:
if global_vars.is_system_stopped:
break
logger.info(f'处理资源:{torrent.title} ...')
# 识别
meta = MetaInfo(title=torrent.title, subtitle=torrent.description)
if torrent.title != meta.org_string:
logger.info(f'种子名称应用识别词后发生改变:{torrent.title} => {meta.org_string}')
# 使用站点种子分类,校正类型识别
if meta.type != MediaType.TV \
and torrent.category == MediaType.TV.value:
meta.type = MediaType.TV
# 识别媒体信息
mediainfo: MediaInfo = MediaChain().recognize_by_meta(meta)
if not mediainfo:
logger.warn(f'{torrent.title} 未识别到媒体信息')
# 存储空的媒体信息
mediainfo = MediaInfo()
# 清理多余数据,减少内存占用
mediainfo.clear()
# 上下文
context = Context(meta_info=meta, media_info=mediainfo, torrent_info=torrent)
# 添加到缓存
if not torrents_cache.get(domain):
torrents_cache[domain] = [context]
else:
torrents_cache[domain].append(context)
# 如果超过了限制条数则移除掉前面的
if len(torrents_cache[domain]) > settings.CONF["torrents"]:
torrents_cache[domain] = torrents_cache[domain][-settings.CONF["torrents"]:]
try:
for torrent in torrents:
if global_vars.is_system_stopped:
break
logger.info(f'处理资源:{torrent.title} ...')
# 识别
meta = MetaInfo(title=torrent.title, subtitle=torrent.description)
if torrent.title != meta.org_string:
logger.info(f'种子名称应用识别词后发生改变:{torrent.title} => {meta.org_string}')
# 使用站点种子分类,校正类型识别
if meta.type != MediaType.TV \
and torrent.category == MediaType.TV.value:
meta.type = MediaType.TV
# 识别媒体信息
mediainfo: MediaInfo = MediaChain().recognize_by_meta(meta)
if not mediainfo:
logger.warn(f'{torrent.title} 未识别到媒体信息')
# 存储空的媒体信息
mediainfo = MediaInfo()
# 清理多余数据,减少内存占用
mediainfo.clear()
# 上下文
context = Context(meta_info=meta, media_info=mediainfo, torrent_info=torrent)
# 添加到缓存
if not torrents_cache.get(domain):
torrents_cache[domain] = [context]
else:
torrents_cache[domain].append(context)
# 如果超过了限制条数则移除掉前面的
if len(torrents_cache[domain]) > settings.CONF.torrents:
torrents_cache[domain] = torrents_cache[domain][-settings.CONF.torrents:]
finally:
torrents.clear()
del torrents
else:
logger.info(f'{indexer.get("name")} 没有获取到种子')

View File

@@ -1,3 +1,4 @@
import gc
import queue
import re
import threading
@@ -788,6 +789,7 @@ class TransferChain(ChainBase, metaclass=Singleton):
for dir_info in download_dirs):
return True
logger.info("开始整理下载器中已经完成下载的文件 ...")
# 从下载器获取种子列表
torrents: Optional[List[TransferTorrent]] = self.list_torrents(status=TorrentStatus.TRANSFER)
if not torrents:
@@ -796,70 +798,78 @@ class TransferChain(ChainBase, metaclass=Singleton):
logger.info(f"获取到 {len(torrents)} 个已完成的下载任务")
for torrent in torrents:
if global_vars.is_system_stopped:
break
# 文件路径
file_path = torrent.path
if not file_path.exists():
logger.warn(f"文件不存在:{file_path}")
continue
# 检查是否为下载器监控目录中的文件
is_downloader_monitor = False
for dir_info in download_dirs:
if dir_info.monitor_type != "downloader":
continue
if not dir_info.download_path:
continue
if file_path.is_relative_to(Path(dir_info.download_path)):
is_downloader_monitor = True
try:
for torrent in torrents:
if global_vars.is_system_stopped:
break
if not is_downloader_monitor:
logger.debug(f"文件 {file_path} 不在下载器监控目录中,不通过下载器进行整理")
continue
# 查询下载记录识别情况
downloadhis: DownloadHistory = DownloadHistoryOper().get_by_hash(torrent.hash)
if downloadhis:
# 类型
try:
mtype = MediaType(downloadhis.type)
except ValueError:
mtype = MediaType.TV
# 按TMDBID识别
mediainfo = self.recognize_media(mtype=mtype,
tmdbid=downloadhis.tmdbid,
doubanid=downloadhis.doubanid,
episode_group=downloadhis.episode_group)
if mediainfo:
# 补充图片
self.obtain_images(mediainfo)
# 更新自定义媒体类别
if downloadhis.media_category:
mediainfo.category = downloadhis.media_category
else:
# 非MoviePilot下载的任务按文件识别
mediainfo = None
# 文件路径
file_path = torrent.path
if not file_path.exists():
logger.warn(f"文件不存在:{file_path}")
continue
# 检查是否为下载器监控目录中的文件
is_downloader_monitor = False
for dir_info in download_dirs:
if dir_info.monitor_type != "downloader":
continue
if not dir_info.download_path:
continue
if file_path.is_relative_to(Path(dir_info.download_path)):
is_downloader_monitor = True
break
if not is_downloader_monitor:
logger.debug(f"文件 {file_path} 不在下载器监控目录中,不通过下载器进行整理")
continue
# 查询下载记录识别情况
downloadhis: DownloadHistory = DownloadHistoryOper().get_by_hash(torrent.hash)
if downloadhis:
# 类型
try:
mtype = MediaType(downloadhis.type)
except ValueError:
mtype = MediaType.TV
# 按TMDBID识别
mediainfo = self.recognize_media(mtype=mtype,
tmdbid=downloadhis.tmdbid,
doubanid=downloadhis.doubanid,
episode_group=downloadhis.episode_group)
if mediainfo:
# 补充图片
self.obtain_images(mediainfo)
# 更新自定义媒体类别
if downloadhis.media_category:
mediainfo.category = downloadhis.media_category
else:
# 非MoviePilot下载的任务按文件识别
mediainfo = None
# 执行实时整理,匹配源目录
state, errmsg = self.do_transfer(
fileitem=FileItem(
storage="local",
path=str(file_path).replace("\\", "/"),
type="dir" if not file_path.is_file() else "file",
name=file_path.name,
size=file_path.stat().st_size,
extension=file_path.suffix.lstrip('.'),
),
mediainfo=mediainfo,
downloader=torrent.downloader,
download_hash=torrent.hash,
background=False,
)
# 执行实时整理,匹配源目录
state, errmsg = self.do_transfer(
fileitem=FileItem(
storage="local",
path=str(file_path).replace("\\", "/"),
type="dir" if not file_path.is_file() else "file",
name=file_path.name,
size=file_path.stat().st_size,
extension=file_path.suffix.lstrip('.'),
),
mediainfo=mediainfo,
downloader=torrent.downloader,
download_hash=torrent.hash,
background=False,
)
# 设置下载任务状态
if not state:
logger.warn(f"整理下载器任务失败:{torrent.hash} - {errmsg}")
self.transfer_completed(hashs=torrent.hash, downloader=torrent.downloader)
# 设置下载任务状态
if not state:
logger.warn(f"整理下载器任务失败:{torrent.hash} - {errmsg}")
self.transfer_completed(hashs=torrent.hash, downloader=torrent.downloader)
finally:
torrents.clear()
del torrents
# 如果不是大内存模式,进行垃圾回收
if not settings.BIG_MEMORY_MODE:
gc.collect()
# 结束
logger.info("所有下载器中下载完成的文件已整理完成")
@@ -1032,111 +1042,115 @@ class TransferChain(ChainBase, metaclass=Singleton):
# 整理所有文件
transfer_tasks: List[TransferTask] = []
for file_item, bluray_dir in file_items:
if global_vars.is_system_stopped:
break
if continue_callback and not continue_callback():
break
file_path = Path(file_item.path)
# 回收站及隐藏的文件不处理
if file_item.path.find('/@Recycle/') != -1 \
or file_item.path.find('/#recycle/') != -1 \
or file_item.path.find('/.') != -1 \
or file_item.path.find('/@eaDir') != -1:
logger.debug(f"{file_item.path} 是回收站或隐藏的文件")
continue
# 整理屏蔽词不处理
is_blocked = False
if transfer_exclude_words:
for keyword in transfer_exclude_words:
if not keyword:
continue
if keyword and re.search(r"%s" % keyword, file_item.path, re.IGNORECASE):
logger.info(f"{file_item.path} 命中整理屏蔽词 {keyword},不处理")
is_blocked = True
break
if is_blocked:
continue
# 整理成功的不再处理
if not force:
transferd = TransferHistoryOper().get_by_src(file_item.path, storage=file_item.storage)
if transferd:
if not transferd.status:
all_success = False
logger.info(f"{file_item.path} 已整理过,如需重新处理,请删除整理记录。")
err_msgs.append(f"{file_item.name} 已整理过")
try:
for file_item, bluray_dir in file_items:
if global_vars.is_system_stopped:
break
if continue_callback and not continue_callback():
break
file_path = Path(file_item.path)
# 回收站及隐藏的文件不处理
if file_item.path.find('/@Recycle/') != -1 \
or file_item.path.find('/#recycle/') != -1 \
or file_item.path.find('/.') != -1 \
or file_item.path.find('/@eaDir') != -1:
logger.debug(f"{file_item.path} 是回收站或隐藏的文件")
continue
if not meta:
# 文件元数据
file_meta = MetaInfoPath(file_path)
else:
file_meta = meta
# 整理屏蔽词不处理
is_blocked = False
if transfer_exclude_words:
for keyword in transfer_exclude_words:
if not keyword:
continue
if keyword and re.search(r"%s" % keyword, file_item.path, re.IGNORECASE):
logger.info(f"{file_item.path} 命中整理屏蔽词 {keyword},不处理")
is_blocked = True
break
if is_blocked:
continue
# 合并季
if season is not None:
file_meta.begin_season = season
# 整理成功的不再处理
if not force:
transferd = TransferHistoryOper().get_by_src(file_item.path, storage=file_item.storage)
if transferd:
if not transferd.status:
all_success = False
logger.info(f"{file_item.path} 已整理过,如需重新处理,请删除整理记录。")
err_msgs.append(f"{file_item.name} 已整理过")
continue
if not file_meta:
all_success = False
logger.error(f"{file_path.name} 无法识别有效信息")
err_msgs.append(f"{file_path.name} 无法识别有效信息")
continue
if not meta:
# 文件元数据
file_meta = MetaInfoPath(file_path)
else:
file_meta = meta
# 自定义识别
if formaterHandler:
# 开始集、结束集、PART
begin_ep, end_ep, part = formaterHandler.split_episode(file_name=file_path.name, file_meta=file_meta)
if begin_ep is not None:
file_meta.begin_episode = begin_ep
file_meta.part = part
if end_ep is not None:
file_meta.end_episode = end_ep
# 合并季
if season is not None:
file_meta.begin_season = season
# 根据父路径获取下载历史
download_history = None
downloadhis = DownloadHistoryOper()
if bluray_dir:
# 蓝光原盘,按目录名查询
download_history = downloadhis.get_by_path(str(file_path))
else:
# 按文件全路径查询
download_file = downloadhis.get_file_by_fullpath(str(file_path))
if download_file:
download_history = downloadhis.get_by_hash(download_file.download_hash)
if not file_meta:
all_success = False
logger.error(f"{file_path.name} 无法识别有效信息")
err_msgs.append(f"{file_path.name} 无法识别有效信息")
continue
# 获取下载Hash
if download_history and (not downloader or not download_hash):
downloader = download_history.downloader
download_hash = download_history.download_hash
# 自定义识别
if formaterHandler:
# 开始集、结束集、PART
begin_ep, end_ep, part = formaterHandler.split_episode(file_name=file_path.name, file_meta=file_meta)
if begin_ep is not None:
file_meta.begin_episode = begin_ep
file_meta.part = part
if end_ep is not None:
file_meta.end_episode = end_ep
# 后台整理
transfer_task = TransferTask(
fileitem=file_item,
meta=file_meta,
mediainfo=mediainfo,
target_directory=target_directory,
target_storage=target_storage,
target_path=target_path,
transfer_type=transfer_type,
scrape=scrape,
library_type_folder=library_type_folder,
library_category_folder=library_category_folder,
downloader=downloader,
download_hash=download_hash,
download_history=download_history,
manual=manual,
background=background
)
if background:
self.put_to_queue(task=transfer_task)
logger.info(f"{file_path.name} 已添加到整理队列")
else:
# 加入列表
self.__put_to_jobview(transfer_task)
transfer_tasks.append(transfer_task)
# 根据父路径获取下载历史
download_history = None
downloadhis = DownloadHistoryOper()
if bluray_dir:
# 蓝光原盘,按目录名查询
download_history = downloadhis.get_by_path(str(file_path))
else:
# 按文件全路径查询
download_file = downloadhis.get_file_by_fullpath(str(file_path))
if download_file:
download_history = downloadhis.get_by_hash(download_file.download_hash)
# 获取下载Hash
if download_history and (not downloader or not download_hash):
downloader = download_history.downloader
download_hash = download_history.download_hash
# 后台整理
transfer_task = TransferTask(
fileitem=file_item,
meta=file_meta,
mediainfo=mediainfo,
target_directory=target_directory,
target_storage=target_storage,
target_path=target_path,
transfer_type=transfer_type,
scrape=scrape,
library_type_folder=library_type_folder,
library_category_folder=library_category_folder,
downloader=downloader,
download_hash=download_hash,
download_history=download_history,
manual=manual,
background=background
)
if background:
self.put_to_queue(task=transfer_task)
logger.info(f"{file_path.name} 已添加到整理队列")
else:
# 加入列表
self.__put_to_jobview(transfer_task)
transfer_tasks.append(transfer_task)
finally:
file_items.clear()
del file_items
# 实时整理
if transfer_tasks:
@@ -1155,29 +1169,32 @@ class TransferChain(ChainBase, metaclass=Singleton):
progress.update(value=0,
text=__process_msg,
key=ProgressKey.FileTransfer)
for transfer_task in transfer_tasks:
if global_vars.is_system_stopped:
break
if continue_callback and not continue_callback():
break
# 更新进度
__process_msg = f"正在整理 {processed_num + fail_num + 1}/{total_num}{transfer_task.fileitem.name} ..."
logger.info(__process_msg)
progress.update(value=(processed_num + fail_num) / total_num * 100,
text=__process_msg,
key=ProgressKey.FileTransfer)
state, err_msg = self.__handle_transfer(
task=transfer_task,
callback=self.__default_callback
)
if not state:
all_success = False
logger.warn(f"{transfer_task.fileitem.name} {err_msg}")
err_msgs.append(f"{transfer_task.fileitem.name} {err_msg}")
fail_num += 1
else:
processed_num += 1
try:
for transfer_task in transfer_tasks:
if global_vars.is_system_stopped:
break
if continue_callback and not continue_callback():
break
# 更新进度
__process_msg = f"正在整理 {processed_num + fail_num + 1}/{total_num}{transfer_task.fileitem.name} ..."
logger.info(__process_msg)
progress.update(value=(processed_num + fail_num) / total_num * 100,
text=__process_msg,
key=ProgressKey.FileTransfer)
state, err_msg = self.__handle_transfer(
task=transfer_task,
callback=self.__default_callback
)
if not state:
all_success = False
logger.warn(f"{transfer_task.fileitem.name} {err_msg}")
err_msgs.append(f"{transfer_task.fileitem.name} {err_msg}")
fail_num += 1
else:
processed_num += 1
finally:
transfer_tasks.clear()
del transfer_tasks
# 整理结束
__end_msg = f"整理队列处理完成,共整理 {total_num} 个文件,失败 {fail_num}"

View File

@@ -9,7 +9,6 @@ from app.chain.site import SiteChain
from app.chain.subscribe import SubscribeChain
from app.chain.system import SystemChain
from app.chain.transfer import TransferChain
from app.core.config import settings
from app.core.event import Event as ManagerEvent, eventmanager, Event
from app.core.plugin import PluginManager
from app.helper.message import MessageHelper
@@ -162,10 +161,6 @@ class Command(metaclass=Singleton):
"""
初始化菜单命令
"""
if settings.DEV:
logger.debug("Development mode active. Skipping command initialization.")
return
# 使用线程池提交后台任务,避免引起阻塞
ThreadHelper().submit(self.__init_commands_background, pid)

View File

@@ -131,7 +131,7 @@ class CacheToolsBackend(CacheBackend):
- 不支持按 `key` 独立隔离 TTL 和 Maxsize仅支持作用于 region 级别
"""
def __init__(self, maxsize: Optional[int] = 1000, ttl: Optional[int] = 1800):
def __init__(self, maxsize: Optional[int] = 512, ttl: Optional[int] = 1800):
"""
初始化缓存实例
@@ -454,7 +454,7 @@ class RedisBackend(CacheBackend):
self.client.close()
def get_cache_backend(maxsize: Optional[int] = 1000, ttl: Optional[int] = 1800) -> CacheBackend:
def get_cache_backend(maxsize: Optional[int] = 512, ttl: Optional[int] = 1800) -> CacheBackend:
"""
根据配置获取缓存后端实例
@@ -482,13 +482,13 @@ def get_cache_backend(maxsize: Optional[int] = 1000, ttl: Optional[int] = 1800)
return CacheToolsBackend(maxsize=maxsize, ttl=ttl)
def cached(region: Optional[str] = None, maxsize: Optional[int] = 1000, ttl: Optional[int] = 1800,
def cached(region: Optional[str] = None, maxsize: Optional[int] = 512, ttl: Optional[int] = 1800,
skip_none: Optional[bool] = True, skip_empty: Optional[bool] = False):
"""
自定义缓存装饰器,支持为每个 key 动态传递 maxsize 和 ttl
:param region: 缓存的区
:param maxsize: 缓存的最大条目数,默认值为 1000
:param maxsize: 缓存的最大条目数,默认值为 512
:param ttl: 缓存的存活时间,单位秒,默认值为 1800
:param skip_none: 跳过 None 缓存,默认为 True
:param skip_empty: 跳过空值缓存(如 None, [], {}, "", set()),默认为 False

View File

@@ -15,6 +15,34 @@ from app.utils.system import SystemUtils
from app.utils.url import UrlUtils
class SystemConfModel(BaseModel):
"""
系统关键资源大小配置
"""
# 缓存种子数量
torrents: int = 0
# 订阅刷新处理数量
refresh: int = 0
# TMDB请求缓存数量
tmdb: int = 0
# 豆瓣请求缓存数量
douban: int = 0
# Bangumi请求缓存数量
bangumi: int = 0
# Fanart请求缓存数量
fanart: int = 0
# 元数据缓存过期时间(秒)
meta: int = 0
# 调度器数量
scheduler: int = 0
# 线程池大小
threadpool: int = 0
# 数据库连接池大小
dbpool: int = 0
# 数据库连接池溢出数量
dbpooloverflow: int = 0
class ConfigModel(BaseModel):
"""
Pydantic 配置模型,描述所有配置项及其类型和默认值
@@ -57,16 +85,12 @@ class ConfigModel(BaseModel):
DB_ECHO: bool = False
# 数据库连接池类型QueuePool, NullPool
DB_POOL_TYPE: str = "QueuePool"
# 是否在获取连接时进行预先 ping 操作,默认关闭
DB_POOL_PRE_PING: bool = False
# 数据库连接池的大小,默认 100
DB_POOL_SIZE: int = 100
# 数据库连接的回收时间(秒),默认 1800 秒
DB_POOL_RECYCLE: int = 1800
# 数据库连接池获取连接的超时时间(秒),默认 60 秒
DB_POOL_TIMEOUT: int = 60
# 数据库连接池最大溢出连接数,默认 500
DB_MAX_OVERFLOW: int = 500
# 是否在获取连接时进行预先 ping 操作
DB_POOL_PRE_PING: bool = True
# 数据库连接的回收时间(秒)
DB_POOL_RECYCLE: int = 300
# 数据库连接池获取连接的超时时间(秒)
DB_POOL_TIMEOUT: int = 30
# SQLite 的 busy_timeout 参数,默认为 60 秒
DB_TIMEOUT: int = 60
# SQLite 是否启用 WAL 模式,默认开启
@@ -124,6 +148,8 @@ class ConfigModel(BaseModel):
ALIPAN_APP_ID: str = "ac1bf04dc9fd4d9aaabb65b4a668d403"
# 元数据识别缓存过期时间(小时)
META_CACHE_EXPIRE: int = 0
# 电视剧动漫的分类genre_ids
ANIME_GENREIDS: List[int] = Field(default=[16])
# 用户认证站点
AUTH_SITE: str = ""
# 重启自动升级
@@ -251,7 +277,7 @@ class ConfigModel(BaseModel):
# 是否启用内存监控
MEMORY_ANALYSIS: bool = False
# 内存快照间隔(分钟)
MEMORY_SNAPSHOT_INTERVAL: int = 60
MEMORY_SNAPSHOT_INTERVAL: int = 30
# 保留的内存快照文件数量
MEMORY_SNAPSHOT_KEEP_COUNT: int = 20
# 全局图片缓存,将媒体图片缓存到本地
@@ -523,43 +549,37 @@ class Settings(BaseSettings, ConfigModel, LogConfigModel):
return self.CONFIG_PATH / "cookies"
@property
def CONF(self):
def CONF(self) -> SystemConfModel:
"""
{
"torrents": "缓存种子数量",
"refresh": "订阅刷新处理数量",
"tmdb": "TMDB请求缓存数量",
"douban": "豆瓣请求缓存数量",
"fanart": "Fanart请求缓存数量",
"meta": "元数据缓存过期时间(秒)",
"memory": "最大占用内存MB",
"scheduler": "调度器缓存数量"
"threadpool": "线程池数量"
}
根据内存模式返回系统配置
"""
if self.BIG_MEMORY_MODE:
return {
"torrents": 200,
"refresh": 100,
"tmdb": 1024,
"douban": 512,
"bangumi": 512,
"fanart": 512,
"meta": (self.META_CACHE_EXPIRE or 24) * 3600,
"scheduler": 100,
"threadpool": 100
}
return {
"torrents": 100,
"refresh": 50,
"tmdb": 256,
"douban": 256,
"bangumi": 256,
"fanart": 128,
"meta": (self.META_CACHE_EXPIRE or 2) * 3600,
"scheduler": 50,
"threadpool": 50
}
return SystemConfModel(
torrents=200,
refresh=100,
tmdb=1024,
douban=512,
bangumi=512,
fanart=512,
meta=(self.META_CACHE_EXPIRE or 24) * 3600,
scheduler=100,
threadpool=100,
dbpool=100,
dbpooloverflow=50
)
return SystemConfModel(
torrents=100,
refresh=50,
tmdb=256,
douban=256,
bangumi=256,
fanart=128,
meta=(self.META_CACHE_EXPIRE or 2) * 3600,
scheduler=50,
threadpool=50,
dbpool=50,
dbpooloverflow=20
)
@property
def PROXY(self):

View File

@@ -6,7 +6,6 @@ import threading
import time
import traceback
import uuid
from functools import lru_cache
from queue import Empty, PriorityQueue
from typing import Callable, Dict, List, Optional, Union
@@ -263,7 +262,6 @@ class EventManager(metaclass=Singleton):
return handler_info
@classmethod
@lru_cache(maxsize=1000)
def __get_handler_identifier(cls, target: Union[Callable, type]) -> Optional[str]:
"""
获取处理器或处理器类的唯一标识符,包括模块名和类名/方法名
@@ -279,7 +277,6 @@ class EventManager(metaclass=Singleton):
return f"{module_name}.{qualname}"
@classmethod
@lru_cache(maxsize=1000)
def __get_class_from_callable(cls, handler: Callable) -> Optional[str]:
"""
获取可调用对象所属类的唯一标识符

View File

@@ -10,6 +10,7 @@ from app.core.meta.releasegroup import ReleaseGroupsMatcher
from app.schemas.types import MediaType
from app.utils.string import StringUtils
from app.utils.tokens import Tokens
from app.core.meta.streamingplatform import StreamingPlatforms
class MetaVideo(MetaBase):
@@ -31,7 +32,7 @@ class MetaVideo(MetaBase):
_part_re = r"(^PART[0-9ABI]{0,2}$|^CD[0-9]{0,2}$|^DVD[0-9]{0,2}$|^DISK[0-9]{0,2}$|^DISC[0-9]{0,2}$)"
_roman_numerals = r"^(?=[MDCLXVI])M*(C[MD]|D?C{0,3})(X[CL]|L?X{0,3})(I[XV]|V?I{0,3})$"
_source_re = r"^BLURAY$|^HDTV$|^UHDTV$|^HDDVD$|^WEBRIP$|^DVDRIP$|^BDRIP$|^BLU$|^WEB$|^BD$|^HDRip$|^REMUX$|^UHD$"
_effect_re = r"^SDR$|^HDR\d*$|^DOLBY$|^DOVI$|^DV$|^3D$|^REPACK$|^HLG$|^HDR10(\+|Plus)$"
_effect_re = r"^SDR$|^HDR\d*$|^DOLBY$|^DOVI$|^DV$|^3D$|^REPACK$|^HLG$|^HDR10(\+|Plus)$|^EDR$|^HQ$"
_resources_type_re = r"%s|%s" % (_source_re, _effect_re)
_name_no_begin_re = r"^[\[【].+?[\]】]"
_name_no_chinese_re = r".*版|.*字幕"
@@ -51,7 +52,7 @@ class MetaVideo(MetaBase):
_resources_pix_re = r"^[SBUHD]*(\d{3,4}[PI]+)|\d{3,4}X(\d{3,4})"
_resources_pix_re2 = r"(^[248]+K)"
_video_encode_re = r"^(H26[45])$|^(x26[45])$|^AVC$|^HEVC$|^VC\d?$|^MPEG\d?$|^Xvid$|^DivX$|^AV1$|^HDR\d*$|^AVS(\+|[23])$"
_audio_encode_re = r"^DTS\d?$|^DTSHD$|^DTSHDMA$|^Atmos$|^TrueHD\d?$|^AC3$|^\dAudios?$|^DDP\d?$|^DD\+\d?$|^DD\d?$|^LPCM\d?$|^AAC\d?$|^FLAC\d?$|^HD\d?$|^MA\d?$|^HR\d?$|^Opus\d?$|^Vorbis\d?$"
_audio_encode_re = r"^DTS\d?$|^DTSHD$|^DTSHDMA$|^Atmos$|^TrueHD\d?$|^AC3$|^\dAudios?$|^DDP\d?$|^DD\+\d?$|^DD\d?$|^LPCM\d?$|^AAC\d?$|^FLAC\d?$|^HD\d?$|^MA\d?$|^HR\d?$|^Opus\d?$|^Vorbis\d?$|^AV[3S]A$"
def __init__(self, title: str, subtitle: str = None, isfile: bool = False):
"""
@@ -66,6 +67,8 @@ class MetaVideo(MetaBase):
original_title = title
self._source = ""
self._effect = []
self.web_source = None
self._index = 0
# 判断是否纯数字命名
if isfile \
and title.isdigit() \
@@ -93,9 +96,12 @@ class MetaVideo(MetaBase):
# 拆分tokens
tokens = Tokens(title)
self.tokens = tokens
# 实例化StreamingPlatforms对象
streaming_platforms = StreamingPlatforms()
# 解析名称、年份、季、集、资源类型、分辨率等
token = tokens.get_next()
while token:
self._index += 1 # 更新当前处理的token索引
# Part
self.__init_part(token)
# 标题
@@ -116,6 +122,9 @@ class MetaVideo(MetaBase):
# 资源类型
if self._continue_flag:
self.__init_resource_type(token)
# 流媒体平台
if self._continue_flag:
self.__init_web_source(token, streaming_platforms)
# 视频编码
if self._continue_flag:
self.__init_video_encode(token)
@@ -131,6 +140,9 @@ class MetaVideo(MetaBase):
self.resource_effect = " ".join(self._effect)
if self._source:
self.resource_type = self._source.strip()
# 添加流媒体平台
if self.web_source:
self.resource_type = f"{self.web_source} {self.resource_type}"
# 提取原盘DIY
if self.resource_type and "BluRay" in self.resource_type:
if (self.subtitle and re.findall(r'D[Ii]Y', self.subtitle)) \
@@ -574,6 +586,57 @@ class MetaVideo(MetaBase):
self._effect.append(effect)
self._last_token = effect.upper()
def __init_web_source(self, token: str, streaming_platforms: StreamingPlatforms):
"""
识别流媒体平台
"""
if not self.name:
return
platform_name = None
query_range = 1
prev_token = None
prev_idx = self._index - 2
if 0 <= prev_idx < len(self.tokens.tokens):
prev_token = self.tokens.tokens[prev_idx]
next_token = self.tokens.peek()
if streaming_platforms.is_streaming_platform(token):
platform_name = streaming_platforms.get_streaming_platform_name(token)
else:
for adjacent_token, is_next in [(prev_token, False), (next_token, True)]:
if not adjacent_token or platform_name:
continue
for separator in [" ", "-"]:
if is_next:
combined_token = f"{token}{separator}{adjacent_token}"
else:
combined_token = f"{adjacent_token}{separator}{token}"
if streaming_platforms.is_streaming_platform(combined_token):
platform_name = streaming_platforms.get_streaming_platform_name(combined_token)
query_range = 2
if is_next:
self.tokens.get_next()
break
if not platform_name:
return
web_tokens = ["WEB", "DL", "WEBDL", "WEBRIP"]
match_start_idx = self._index - query_range
match_end_idx = self._index - 1
start_index = max(0, match_start_idx - query_range)
end_index = min(len(self.tokens.tokens), match_end_idx + 1 + query_range)
tokens_to_check = self.tokens.tokens[start_index:end_index]
if any(tok and tok.upper() in web_tokens for tok in tokens_to_check):
self.web_source = platform_name
self._continue_flag = False
def __init_video_encode(self, token: str):
"""
识别视频编码

View File

@@ -0,0 +1,315 @@
from typing import Optional, List, Tuple
from app.utils.singleton import Singleton
class StreamingPlatforms(metaclass=Singleton):
"""
流媒体平台简称与全称。
"""
STREAMING_PLATFORMS: List[Tuple[str, str]] = [
("AMZN", "Amazon"),
("NF", "Netflix"),
("ATVP", "Apple TV+"),
("iT", "iTunes"),
("DSNP", "Disney+"),
("HS", "Hotstar"),
("APPS", "Disney+ MENA"),
("PMTP", "Paramount+"),
("HMAX", "Max"),
("", "Max"),
("HULU", "Hulu Networks"),
("MA", "Movies Anywhere"),
("BCORE", "Bravia Core"),
("MS", "Microsoft Store"),
("SHO", "Showtime"),
("STAN", "Stan"),
("PCOK", "Peacock"),
("SKST", "SkyShowtime"),
("NOW", "Now"),
("FXTL", "Foxtel Now"),
("BNGE", "Binge"),
("CRKL", "Crackle"),
("RKTN", "Rakuten TV"),
("ALL4", "Channel 4"),
("AS", "Adult Swim"),
("BRTB", "Brtb TV"),
("CNLP", "Canal+"),
("CRIT", "Criterion Channel"),
("DSCP", "Discovery+"),
("FOOD", "Food Network"),
("MUBI", "Mubi"),
("PLAY", "Google Play"),
("YT", "YouTube"),
("", "friDay"),
("", "KKTV"),
("", "ofiii"),
("", "LiTV"),
("", "MyVideo"),
("Hami", "Hami Video"),
("HamiVideo", "Hami Video"),
("MW", "meWATCH"),
("CATCHPLAY", "CATCHPLAY+"),
("CPP", "CATCHPLAY+"),
("LINETV", "LINE TV"),
("VIU", "Viu"),
("IQ", ""),
("", "WeTV"),
("ABMA", "Abema"),
("ADN", ""),
("AT-X", ""),
("Baha", ""),
("BG", "B-Global"),
("CR", "Crunchyroll"),
("", "DMM"),
("FOD", ""),
("FUNi", "Funimation"),
("HIDI", "HIDIVE"),
("UNXT", "U-NEXT"),
("FAA", "Filmarchiv Austria"),
("CC", "Comedy Central"),
("iP", "BBC iPlayer"),
("9NOW", "9Now"),
("ABC", ""),
("", "AMC"),
("", "ZEE5"),
("", "WAVO"),
("SHAHID", "Shahid"),
("Flixole", "FlixOlé"),
("TOU", "Ici TOU.TV"),
("ROKU", "Roku"),
("KNPY", "Kanopy"),
("SNXT", "Sun NXT"),
("CUR", "Curiosity Stream"),
("MY5", "Channel 5"),
("AHA", "aha"),
("WOWP", "WOW Presents Plus"),
("JC", "JioCinema"),
("", "Dekkoo"),
("FILMZIE", "Filmzie"),
("HoiChoi", "Hoichoi"),
("VIKI", "Rakuten Viki"),
("SF", "SF Anytime"),
("PLEX", "Plex"),
("SHDR", "Shudder"),
("CRAV", "Crave"),
("CPE", "Cineplex Entertainment"),
("JF HC", ""),
("JF", ""),
("JFFP", ""),
("VIAP", "Viaplay"),
("TUBI", "TubiTV"),
("", "PBS"),
("PBSK", "PBS KIDS"),
("LGP", "Lionsgate Play"),
("", "CTV"),
("", "Cineverse"),
("LN", "Love Nature"),
("MP", "Movistar Plus+"),
("RUNTIME", "Runtime"),
("STZ", "STARZ"),
("FUBO", "fuboTV"),
("TENK", "Tënk"),
("KNOW", "Knowledge Network"),
("TVO", "tvo"),
("", "OVID"),
("CBC", "CBC Gem"),
("FANDOR", "fandor"),
("CW", "The CW"),
("KNPY", "Kanopy"),
("FREE", "Freeform"),
("AE", "A&E"),
("LIFE", "Lifetime"),
("WWEN", "WWE Network"),
("CMAX", "Cinemax"),
("HLMK", "Hallmark"),
("BYU", "BYUtv"),
("", "ViX"),
("VICE", "Viceland"),
("", "TVING"),
("USAN", "USA Network"),
("FOX", ""),
("", "TCM"),
("BRAV", "BravoTV"),
("", "TNT"),
("", "ZDF"),
("", "IndieFlix"),
("", "TLC"),
("", "HGTV"),
("ANPL", "Animal Planet"),
("TRVL", "Travel Channel"),
("", "VH1"),
("SAINA", "Saina Play"),
("SP", "Saina Play"),
("OXGN", "Oxygen"),
("PSN", "PlayStation Network"),
("PMNT", "Paramount Network"),
("FAWESOME", "Fawesome"),
("KLASSIKI", "Klassiki"),
("STRP", "Star+"),
("NATG", "National Geographic"),
("REVEEL", "Reveel"),
("FYI", "FYI Network"),
("WatchiT", "WATCH IT"),
("ITVX", "ITV"),
("GAIA", "Gaia"),
("", "FlixLatino"),
("CNNP", "CNN+"),
("TROMA", "Troma"),
("IVI", "Ivi"),
("9NOW", "9Now"),
("A3P", "Atresplayer"),
("7PLUS", "7plus"),
("", "SBS"),
("TEN", "10Play"),
("AUBC", ""),
("DSNY", "Disney Networks"),
("OSN", "OSN+"),
("SVT", "Sveriges Television"),
("LACINETEK", "LaCinetek"),
("", "Maxdome"),
("RTL", "RTL+"),
("ARTE", "Arte"),
("JOYN", "Joyn"),
("TV2", "TV 2"),
("3SAT", "3sat"),
("FILMINGO", "filmingo"),
("", "WOW"),
("OKKO", "Okko"),
("", "Go3"),
("ARGP", "Argo"),
("VOYO", "Voyo"),
("VMAX", "vivamax"),
("FILMIN", "Filmin"),
("", "Mitele"),
("MY5", "Channel 5"),
("", "ARD"),
("BK", "Bentkey"),
("BOOM", "Boomerang"),
("", "CBS"),
("CLBI", "Club illico"),
("CMOR", "C More"),
("CMT", ""),
("", "CNBC"),
("COOK", "Cooking Channel"),
("CWS", "CW Seed"),
("DCU", "DC Universe"),
("DDY", "Digiturk Dilediğin Yerde"),
("DEST", "Destination America"),
("DISC", "Discovery Channel"),
("DW", "DailyWire+"),
("DLWP", "DailyWire+"),
("DPLY", "dplay"),
("DRPO", "Dropout"),
("EPIX", "EPIX MGM+"),
("ESQ", "Esquire"),
("ETV", "E!"),
("FBWatch", "Facebook Watch"),
("FPT", "FPT Play"),
("FTV", "France.tv"),
("GLOB", "GloboSat Play"),
("GLBO", "Globoplay"),
("GO90", "go90"),
("HIST", "History Channel"),
("HPLAY", "Hungama Play"),
("KS", "Kaleidescape"),
("", "MBC"),
("MMAX", "ManoramaMAX"),
("MNBC", "MSNBC"),
("MTOD", "Motor Trend OnDemand"),
("NBC", ""),
("NBLA", "Nebula"),
("NICK", "Nickelodeon"),
("ODK", "OnDemandKorea"),
("POGO", "PokerGO"),
("PUHU", "puhutv"),
("QIBI", "Quibi"),
("RTE", "RTÉ"),
("SESO", "Seeso"),
("SPIK", "Spike"),
("SS", "Simply South"),
("SYFY", "SyFy"),
("TIMV", "TIMvision"),
("TK", "Tentkotta"),
("", "TV4"),
("TVL", "TV Land"),
("", "TVNZ"),
("", "UKTV"),
("VLCT", "Discovery Velocity"),
("VMEO", "Vimeo"),
("VRV", "VRV Defunct"),
("WTCH", "Watcha"),
("", "NowPlayer"),
("HuluJP", "Hulu Networks"),
("Gaga", "GagaOOLala"),
("MyTVS", "MyTVSuper"),
("", "BBC"),
("CC", "Comedy Central"),
("NowE", "Now E"),
("WAVVE", "Wavve"),
("SE", ""),
("", "BritBox"),
("AOD", "Anime on Demand"),
("AF", ""),
("BCH", "Bandai Channel"),
("VMJ", "VideoMarket"),
("LFTL", "Laftel"),
("WAKA", "Wakanim"),
("WAKANIM", "Wakanim"),
("AO", "AnimeOnegai"),
("", "Lemino"),
("VIDIO", "Vidio"),
("TVER", "TVer"),
("", "MBS"),
("LFTLNET", "Laftel"),
("JONU", "Jonu Play"),
("PlutoTV", "Pluto TV"),
("AbemaTV", "Abema"),
("", "dTV"),
("NYMEY", "Nymey"),
("SMNS", "SAMANSA"),
("CTHP", "CATCHPLAY+"),
("HBOGO", "HBO GO"),
("HBO", "HBO"),
("FPTP", "FPT Play"),
("", "LOCIPO"),
("DANT", "DANET"),
("OV", "OceanVeil"),
]
def __init__(self):
"""初始化流媒体平台匹配器"""
self._lookup_cache = {}
self._build_cache()
def _build_cache(self) -> None:
"""
构建查询缓存。
"""
self._lookup_cache.clear()
for short_name, full_name in self.STREAMING_PLATFORMS:
canonical_name = full_name or short_name
if not canonical_name:
continue
aliases = {short_name, full_name}
for alias in aliases:
if alias:
self._lookup_cache[alias.upper()] = canonical_name
def get_streaming_platform_name(self, platform_code: str) -> Optional[str]:
"""
根据流媒体平台简称或全称获取标准名称。
"""
if platform_code is None:
return None
return self._lookup_cache.get(platform_code.upper())
def is_streaming_platform(self, name: str) -> bool:
"""
判断给定的字符串是否为已知的流媒体平台代码或名称。
"""
if name is None:
return False
return name.upper() in self._lookup_cache

View File

@@ -19,7 +19,6 @@ from app.core.config import settings
from app.core.event import eventmanager, Event
from app.db.plugindata_oper import PluginDataOper
from app.db.systemconfig_oper import SystemConfigOper
from app.helper.module import ModuleHelper
from app.helper.plugin import PluginHelper
from app.helper.sites import SitesHelper
from app.log import logger
@@ -122,21 +121,10 @@ class PluginManager(metaclass=Singleton):
return False
return True
# 扫描插件目录
if pid:
# 加载指定插件
plugins = ModuleHelper.load_with_pre_filter(
"app.plugins",
filter_func=lambda name, obj: check_module(obj) and name == pid
)
else:
# 加载所有插件
plugins = ModuleHelper.load(
"app.plugins",
filter_func=lambda _, obj: check_module(obj)
)
# 已安装插件
installed_plugins = SystemConfigOper().get(SystemConfigKey.UserInstalledPlugins) or []
# 扫描插件目录,只加载符合条件的插件
plugins = self._load_selective_plugins(pid, installed_plugins, check_module)
# 排序
plugins.sort(key=lambda x: x.plugin_order if hasattr(x, "plugin_order") else 0)
for plugin in plugins:
@@ -152,11 +140,6 @@ class PluginManager(metaclass=Singleton):
continue
# 存储Class
self._plugins[plugin_id] = plugin
# 未安装的不加载
if plugin_id not in installed_plugins:
# 设置事件状态为不可用
eventmanager.disable_event_handler(plugin)
continue
# 生成实例
plugin_obj = plugin()
# 生效插件配置
@@ -201,7 +184,7 @@ class PluginManager(metaclass=Singleton):
logger.info(f"正在停止插件 {pid}...")
plugin_obj = self._running_plugins.get(pid)
if not plugin_obj:
logger.warning(f"插件 {pid} 不存在或未加载")
logger.debug(f"插件 {pid} 不存在或未加载")
return
plugins = {pid: plugin_obj}
else:
@@ -213,6 +196,7 @@ class PluginManager(metaclass=Singleton):
# 清空对像
if pid:
# 清空指定插件
self._plugins.pop(pid, None)
self._running_plugins.pop(pid, None)
else:
# 清空
@@ -220,6 +204,80 @@ class PluginManager(metaclass=Singleton):
self._running_plugins = {}
logger.info("插件停止完成")
@staticmethod
def _load_selective_plugins(pid: Optional[str], installed_plugins: List[str],
check_module_func: Callable) -> List[Any]:
"""
选择性加载插件只import符合条件的插件
:param pid: 指定插件ID为空则加载所有已安装插件
:param installed_plugins: 已安装插件列表
:param check_module_func: 模块检查函数
:return: 插件类列表
"""
import importlib
plugins = []
plugins_dir = settings.ROOT_PATH / "app" / "plugins"
if not plugins_dir.exists():
logger.warning(f"插件目录不存在:{plugins_dir}")
return plugins
# 确定需要加载的插件目录名称列表
if pid:
# 加载指定插件
target_plugins = [pid.lower()]
else:
# 加载已安装插件
target_plugins = [plugin_id.lower() for plugin_id in installed_plugins]
if not target_plugins:
logger.debug("没有需要加载的插件")
return plugins
# 扫描plugins目录
_loaded_modules = set()
for plugin_dir in plugins_dir.iterdir():
if not plugin_dir.is_dir() or plugin_dir.name.startswith('_'):
continue
# 检查是否是需要加载的插件
if plugin_dir.name not in target_plugins:
logger.debug(f"跳过插件目录:{plugin_dir.name}(不在加载列表中)")
continue
# 检查__init__.py是否存在
init_file = plugin_dir / "__init__.py"
if not init_file.exists():
logger.debug(f"跳过插件目录:{plugin_dir.name}缺少__init__.py")
continue
try:
# 构建模块名
module_name = f"app.plugins.{plugin_dir.name}"
logger.debug(f"正在导入插件模块:{module_name}")
# 导入模块
module = importlib.import_module(module_name)
importlib.reload(module)
# 检查模块中的类
for name, obj in module.__dict__.items():
if name.startswith('_') or not isinstance(obj, type):
continue
if name in _loaded_modules:
continue
if check_module_func(obj):
_loaded_modules.add(name)
plugins.append(obj)
logger.debug(f"找到符合条件的插件类:{name}")
break
except Exception as err:
logger.error(f"加载插件 {plugin_dir.name} 失败:{str(err)} - {traceback.format_exc()}")
return plugins
@property
def running_plugins(self) -> Dict[str, Any]:
"""
@@ -247,6 +305,7 @@ class PluginManager(metaclass=Singleton):
event_data: schemas.ConfigChangeEventData = event.event_data
if event_data.key not in ['DEV', 'PLUGIN_AUTO_RELOAD']:
return
logger.info("配置变更,重新加载插件文件修改监测...")
self.reload_monitor()
def reload_monitor(self):
@@ -354,8 +413,7 @@ class PluginManager(metaclass=Singleton):
# 确定需要安装的插件
plugins_to_install = [
plugin for plugin in online_plugins
if plugin.id in install_plugins
and not self.is_plugin_exists(plugin.id, plugin.plugin_version)
if plugin.id in install_plugins and not self.is_plugin_exists(plugin.id, plugin.plugin_version)
]
if not plugins_to_install:

View File

@@ -24,9 +24,9 @@ db_kwargs = {
# 当使用 QueuePool 时,添加 QueuePool 特有的参数
if pool_class == QueuePool:
db_kwargs.update({
"pool_size": settings.DB_POOL_SIZE,
"pool_size": settings.CONF.dbpool,
"pool_timeout": settings.DB_POOL_TIMEOUT,
"max_overflow": settings.DB_MAX_OVERFLOW
"max_overflow": settings.CONF.dbpooloverflow
})
# 创建数据库引擎
Engine = create_engine(**db_kwargs)
@@ -221,8 +221,7 @@ class Base:
@classmethod
@db_query
def list(cls, db: Session) -> List[Self]:
result = db.query(cls).all()
return list(result)
return db.query(cls).all()
def to_dict(self):
return {c.name: getattr(self, c.name, None) for c in self.__table__.columns} # noqa

View File

@@ -1,7 +1,7 @@
import time
from typing import Optional
from sqlalchemy import Column, Integer, String, Sequence, JSON, or_
from sqlalchemy import Column, Integer, String, Sequence, JSON
from sqlalchemy.orm import Session
from app.db import db_query, db_update, Base
@@ -74,8 +74,7 @@ class DownloadHistory(Base):
@staticmethod
@db_query
def list_by_page(db: Session, page: Optional[int] = 1, count: Optional[int] = 30):
result = db.query(DownloadHistory).offset((page - 1) * count).limit(count).all()
return list(result)
return db.query(DownloadHistory).offset((page - 1) * count).limit(count).all()
@staticmethod
@db_query
@@ -91,50 +90,47 @@ class DownloadHistory(Base):
据tmdbid、season、season_episode查询下载记录
tmdbid + mtype 或 title + year
"""
result = None
# TMDBID + 类型
if tmdbid and mtype:
# 电视剧某季某集
if season and episode:
result = db.query(DownloadHistory).filter(DownloadHistory.tmdbid == tmdbid,
DownloadHistory.type == mtype,
DownloadHistory.seasons == season,
DownloadHistory.episodes == episode).order_by(
return db.query(DownloadHistory).filter(DownloadHistory.tmdbid == tmdbid,
DownloadHistory.type == mtype,
DownloadHistory.seasons == season,
DownloadHistory.episodes == episode).order_by(
DownloadHistory.id.desc()).all()
# 电视剧某季
elif season:
result = db.query(DownloadHistory).filter(DownloadHistory.tmdbid == tmdbid,
DownloadHistory.type == mtype,
DownloadHistory.seasons == season).order_by(
return db.query(DownloadHistory).filter(DownloadHistory.tmdbid == tmdbid,
DownloadHistory.type == mtype,
DownloadHistory.seasons == season).order_by(
DownloadHistory.id.desc()).all()
else:
# 电视剧所有季集/电影
result = db.query(DownloadHistory).filter(DownloadHistory.tmdbid == tmdbid,
DownloadHistory.type == mtype).order_by(
return db.query(DownloadHistory).filter(DownloadHistory.tmdbid == tmdbid,
DownloadHistory.type == mtype).order_by(
DownloadHistory.id.desc()).all()
# 标题 + 年份
elif title and year:
# 电视剧某季某集
if season and episode:
result = db.query(DownloadHistory).filter(DownloadHistory.title == title,
DownloadHistory.year == year,
DownloadHistory.seasons == season,
DownloadHistory.episodes == episode).order_by(
return db.query(DownloadHistory).filter(DownloadHistory.title == title,
DownloadHistory.year == year,
DownloadHistory.seasons == season,
DownloadHistory.episodes == episode).order_by(
DownloadHistory.id.desc()).all()
# 电视剧某季
elif season:
result = db.query(DownloadHistory).filter(DownloadHistory.title == title,
DownloadHistory.year == year,
DownloadHistory.seasons == season).order_by(
return db.query(DownloadHistory).filter(DownloadHistory.title == title,
DownloadHistory.year == year,
DownloadHistory.seasons == season).order_by(
DownloadHistory.id.desc()).all()
else:
# 电视剧所有季集/电影
result = db.query(DownloadHistory).filter(DownloadHistory.title == title,
DownloadHistory.year == year).order_by(
return db.query(DownloadHistory).filter(DownloadHistory.title == title,
DownloadHistory.year == year).order_by(
DownloadHistory.id.desc()).all()
if result:
return list(result)
return []
@staticmethod
@@ -144,13 +140,12 @@ class DownloadHistory(Base):
查询某用户某时间之后的下载历史
"""
if username:
result = db.query(DownloadHistory).filter(DownloadHistory.date < date,
DownloadHistory.username == username).order_by(
return db.query(DownloadHistory).filter(DownloadHistory.date < date,
DownloadHistory.username == username).order_by(
DownloadHistory.id.desc()).all()
else:
result = db.query(DownloadHistory).filter(DownloadHistory.date < date).order_by(
return db.query(DownloadHistory).filter(DownloadHistory.date < date).order_by(
DownloadHistory.id.desc()).all()
return list(result)
@staticmethod
@db_query
@@ -173,12 +168,11 @@ class DownloadHistory(Base):
@staticmethod
@db_query
def list_by_type(db: Session, mtype: str, days: int):
result = db.query(DownloadHistory) \
return db.query(DownloadHistory) \
.filter(DownloadHistory.type == mtype,
DownloadHistory.date >= time.strftime("%Y-%m-%d %H:%M:%S",
time.localtime(time.time() - 86400 * int(days)))
).all()
return list(result)
class DownloadFiles(Base):
@@ -205,12 +199,10 @@ class DownloadFiles(Base):
@db_query
def get_by_hash(db: Session, download_hash: str, state: Optional[int] = None):
if state:
result = db.query(DownloadFiles).filter(DownloadFiles.download_hash == download_hash,
DownloadFiles.state == state).all()
return db.query(DownloadFiles).filter(DownloadFiles.download_hash == download_hash,
DownloadFiles.state == state).all()
else:
result = db.query(DownloadFiles).filter(DownloadFiles.download_hash == download_hash).all()
return list(result)
return db.query(DownloadFiles).filter(DownloadFiles.download_hash == download_hash).all()
@staticmethod
@db_query
@@ -225,8 +217,7 @@ class DownloadFiles(Base):
@staticmethod
@db_query
def get_by_savepath(db: Session, savepath: str):
result = db.query(DownloadFiles).filter(DownloadFiles.savepath == savepath).all()
return list(result)
return db.query(DownloadFiles).filter(DownloadFiles.savepath == savepath).all()
@staticmethod
@db_update

View File

@@ -37,7 +37,4 @@ class Message(Base):
@staticmethod
@db_query
def list_by_page(db: Session, page: Optional[int] = 1, count: Optional[int] = 30):
result = db.query(Message).order_by(Message.reg_time.desc()).offset((page - 1) * count).limit(
count).all()
result.sort(key=lambda x: x.reg_time, reverse=False)
return list(result)
return db.query(Message).order_by(Message.reg_time.desc()).offset((page - 1) * count).limit(count).all()

View File

@@ -16,8 +16,7 @@ class PluginData(Base):
@staticmethod
@db_query
def get_plugin_data(db: Session, plugin_id: str):
result = db.query(PluginData).filter(PluginData.plugin_id == plugin_id).all()
return list(result)
return db.query(PluginData).filter(PluginData.plugin_id == plugin_id).all()
@staticmethod
@db_query
@@ -37,5 +36,4 @@ class PluginData(Base):
@staticmethod
@db_query
def get_plugin_data_by_plugin_id(db: Session, plugin_id: str):
result = db.query(PluginData).filter(PluginData.plugin_id == plugin_id).all()
return list(result)
return db.query(PluginData).filter(PluginData.plugin_id == plugin_id).all()

View File

@@ -62,20 +62,17 @@ class Site(Base):
@staticmethod
@db_query
def get_actives(db: Session):
result = db.query(Site).filter(Site.is_active == 1).all()
return list(result)
return db.query(Site).filter(Site.is_active == 1).all()
@staticmethod
@db_query
def list_order_by_pri(db: Session):
result = db.query(Site).order_by(Site.pri).all()
return list(result)
return db.query(Site).order_by(Site.pri).all()
@staticmethod
@db_query
def get_domains_by_ids(db: Session, ids: list):
result = db.query(Site.domain).filter(Site.id.in_(ids)).all()
return [r[0] for r in result]
return [r[0] for r in db.query(Site.domain).filter(Site.id.in_(ids)).all()]
@staticmethod
@db_update

View File

@@ -104,12 +104,10 @@ class Subscribe(Base):
def get_by_state(db: Session, state: str):
# 如果 state 为空或 None返回所有订阅
if not state:
result = db.query(Subscribe).all()
return db.query(Subscribe).all()
else:
# 如果传入的状态不为空,拆分成多个状态
states = state.split(',')
result = db.query(Subscribe).filter(Subscribe.state.in_(states)).all()
return list(result)
return db.query(Subscribe).filter(Subscribe.state.in_(state.split(','))).all()
@staticmethod
@db_query
@@ -123,11 +121,10 @@ class Subscribe(Base):
@db_query
def get_by_tmdbid(db: Session, tmdbid: int, season: Optional[int] = None):
if season:
result = db.query(Subscribe).filter(Subscribe.tmdbid == tmdbid,
Subscribe.season == season).all()
return db.query(Subscribe).filter(Subscribe.tmdbid == tmdbid,
Subscribe.season == season).all()
else:
result = db.query(Subscribe).filter(Subscribe.tmdbid == tmdbid).all()
return list(result)
return db.query(Subscribe).filter(Subscribe.tmdbid == tmdbid).all()
@staticmethod
@db_query
@@ -170,26 +167,24 @@ class Subscribe(Base):
def list_by_username(db: Session, username: str, state: Optional[str] = None, mtype: Optional[str] = None):
if mtype:
if state:
result = db.query(Subscribe).filter(Subscribe.state == state,
Subscribe.username == username,
Subscribe.type == mtype).all()
return db.query(Subscribe).filter(Subscribe.state == state,
Subscribe.username == username,
Subscribe.type == mtype).all()
else:
result = db.query(Subscribe).filter(Subscribe.username == username,
Subscribe.type == mtype).all()
return db.query(Subscribe).filter(Subscribe.username == username,
Subscribe.type == mtype).all()
else:
if state:
result = db.query(Subscribe).filter(Subscribe.state == state,
Subscribe.username == username).all()
return db.query(Subscribe).filter(Subscribe.state == state,
Subscribe.username == username).all()
else:
result = db.query(Subscribe).filter(Subscribe.username == username).all()
return list(result)
return db.query(Subscribe).filter(Subscribe.username == username).all()
@staticmethod
@db_query
def list_by_type(db: Session, mtype: str, days: int):
result = db.query(Subscribe) \
return db.query(Subscribe) \
.filter(Subscribe.type == mtype,
Subscribe.date >= time.strftime("%Y-%m-%d %H:%M:%S",
time.localtime(time.time() - 86400 * int(days)))
).all()
return list(result)

View File

@@ -75,12 +75,11 @@ class SubscribeHistory(Base):
@staticmethod
@db_query
def list_by_type(db: Session, mtype: str, page: Optional[int] = 1, count: Optional[int] = 30):
result = db.query(SubscribeHistory).filter(
return db.query(SubscribeHistory).filter(
SubscribeHistory.type == mtype
).order_by(
SubscribeHistory.date.desc()
).offset((page - 1) * count).limit(count).all()
return list(result)
@staticmethod
@db_query

View File

@@ -63,35 +63,33 @@ class TransferHistory(Base):
@db_query
def list_by_title(db: Session, title: str, page: Optional[int] = 1, count: Optional[int] = 30, status: bool = None):
if status is not None:
result = db.query(TransferHistory).filter(
return db.query(TransferHistory).filter(
TransferHistory.status == status
).order_by(
TransferHistory.date.desc()
).offset((page - 1) * count).limit(count).all()
else:
result = db.query(TransferHistory).filter(or_(
return db.query(TransferHistory).filter(or_(
TransferHistory.title.like(f'%{title}%'),
TransferHistory.src.like(f'%{title}%'),
TransferHistory.dest.like(f'%{title}%'),
)).order_by(
TransferHistory.date.desc()
).offset((page - 1) * count).limit(count).all()
return list(result)
@staticmethod
@db_query
def list_by_page(db: Session, page: Optional[int] = 1, count: Optional[int] = 30, status: bool = None):
if status is not None:
result = db.query(TransferHistory).filter(
return db.query(TransferHistory).filter(
TransferHistory.status == status
).order_by(
TransferHistory.date.desc()
).offset((page - 1) * count).limit(count).all()
else:
result = db.query(TransferHistory).order_by(
return db.query(TransferHistory).order_by(
TransferHistory.date.desc()
).offset((page - 1) * count).limit(count).all()
return list(result)
@staticmethod
@db_query
@@ -115,8 +113,7 @@ class TransferHistory(Base):
@staticmethod
@db_query
def list_by_hash(db: Session, download_hash: str):
result = db.query(TransferHistory).filter(TransferHistory.download_hash == download_hash).all()
return list(result)
return db.query(TransferHistory).filter(TransferHistory.download_hash == download_hash).all()
@staticmethod
@db_query
@@ -128,8 +125,7 @@ class TransferHistory(Base):
TransferHistory.id.label('id')).filter(
TransferHistory.date >= time.strftime("%Y-%m-%d %H:%M:%S",
time.localtime(time.time() - 86400 * days))).subquery()
result = db.query(sub_query.c.date, func.count(sub_query.c.id)).group_by(sub_query.c.date).all()
return list(result)
return db.query(sub_query.c.date, func.count(sub_query.c.id)).group_by(sub_query.c.date).all()
@staticmethod
@db_query
@@ -153,70 +149,67 @@ class TransferHistory(Base):
@staticmethod
@db_query
def list_by(db: Session, mtype: Optional[str] = None, title: Optional[str] = None, year: Optional[str] = None, season: Optional[str] = None,
def list_by(db: Session, mtype: Optional[str] = None, title: Optional[str] = None, year: Optional[str] = None,
season: Optional[str] = None,
episode: Optional[str] = None, tmdbid: Optional[int] = None, dest: Optional[str] = None):
"""
据tmdbid、season、season_episode查询转移记录
tmdbid + mtype 或 title + year 必输
"""
result = None
# TMDBID + 类型
if tmdbid and mtype:
# 电视剧某季某集
if season and episode:
result = db.query(TransferHistory).filter(TransferHistory.tmdbid == tmdbid,
TransferHistory.type == mtype,
TransferHistory.seasons == season,
TransferHistory.episodes == episode,
TransferHistory.dest == dest).all()
return db.query(TransferHistory).filter(TransferHistory.tmdbid == tmdbid,
TransferHistory.type == mtype,
TransferHistory.seasons == season,
TransferHistory.episodes == episode,
TransferHistory.dest == dest).all()
# 电视剧某季
elif season:
result = db.query(TransferHistory).filter(TransferHistory.tmdbid == tmdbid,
TransferHistory.type == mtype,
TransferHistory.seasons == season).all()
return db.query(TransferHistory).filter(TransferHistory.tmdbid == tmdbid,
TransferHistory.type == mtype,
TransferHistory.seasons == season).all()
else:
if dest:
# 电影
result = db.query(TransferHistory).filter(TransferHistory.tmdbid == tmdbid,
TransferHistory.type == mtype,
TransferHistory.dest == dest).all()
return db.query(TransferHistory).filter(TransferHistory.tmdbid == tmdbid,
TransferHistory.type == mtype,
TransferHistory.dest == dest).all()
else:
# 电视剧所有季集
result = db.query(TransferHistory).filter(TransferHistory.tmdbid == tmdbid,
TransferHistory.type == mtype).all()
return db.query(TransferHistory).filter(TransferHistory.tmdbid == tmdbid,
TransferHistory.type == mtype).all()
# 标题 + 年份
elif title and year:
# 电视剧某季某集
if season and episode:
result = db.query(TransferHistory).filter(TransferHistory.title == title,
TransferHistory.year == year,
TransferHistory.seasons == season,
TransferHistory.episodes == episode,
TransferHistory.dest == dest).all()
return db.query(TransferHistory).filter(TransferHistory.title == title,
TransferHistory.year == year,
TransferHistory.seasons == season,
TransferHistory.episodes == episode,
TransferHistory.dest == dest).all()
# 电视剧某季
elif season:
result = db.query(TransferHistory).filter(TransferHistory.title == title,
TransferHistory.year == year,
TransferHistory.seasons == season).all()
return db.query(TransferHistory).filter(TransferHistory.title == title,
TransferHistory.year == year,
TransferHistory.seasons == season).all()
else:
if dest:
# 电影
result = db.query(TransferHistory).filter(TransferHistory.title == title,
TransferHistory.year == year,
TransferHistory.dest == dest).all()
return db.query(TransferHistory).filter(TransferHistory.title == title,
TransferHistory.year == year,
TransferHistory.dest == dest).all()
else:
# 电视剧所有季集
result = db.query(TransferHistory).filter(TransferHistory.title == title,
TransferHistory.year == year).all()
return db.query(TransferHistory).filter(TransferHistory.title == title,
TransferHistory.year == year).all()
# 类型 + 转移路径emby webhook season无tmdbid场景
elif mtype and season and dest:
# 电视剧某季
result = db.query(TransferHistory).filter(TransferHistory.type == mtype,
TransferHistory.seasons == season,
TransferHistory.dest.like(f"{dest}%")).all()
if result:
return list(result)
return db.query(TransferHistory).filter(TransferHistory.type == mtype,
TransferHistory.seasons == season,
TransferHistory.dest.like(f"{dest}%")).all()
return []
@staticmethod

View File

@@ -78,7 +78,3 @@ class SystemConfigOper(DbOper, metaclass=Singleton):
if conf:
conf.delete(self._db, conf.id)
return True
def __del__(self):
if self._db:
self._db.close()

View File

@@ -50,10 +50,6 @@ class UserConfigOper(DbOper, metaclass=Singleton):
return self.__get_config_caches(username=username)
return self.__get_config_cache(username=username, key=key)
def __del__(self):
if self._db:
self._db.close()
def __set_config_cache(self, username: str, key: str, value: Any):
"""
设置配置缓存

View File

@@ -68,6 +68,7 @@ def enable_doh(enable: bool):
else:
socket.getaddrinfo = _orig_getaddrinfo
class DohHelper(metaclass=Singleton):
def __init__(self):
enable_doh(settings.DOH_ENABLE)

View File

@@ -241,7 +241,7 @@ class TemplateContextBuilder:
"total_size": StringUtils.str_filesize(transferinfo.total_size),
"err_msg": transferinfo.message,
}
self._context.update(ctx)
return self._context.update(ctx)
def _add_file_info(self, file_extension: Optional[str]):
"""
@@ -363,7 +363,7 @@ class TemplateHelper(metaclass=SingletonClass):
self.set_cache_context(rendered, context)
# 返回渲染结果
return rendered
return None
except Exception as e:
logger.error(f"模板处理失败: {str(e)}")
raise ValueError(f"模板处理失败: {str(e)}") from e
@@ -645,7 +645,8 @@ class MessageQueueManager(metaclass=SingletonClass):
"""
发送消息(立即发送或加入队列)
"""
if self._is_in_scheduled_time(datetime.now()):
immediately = kwargs.pop("immediately", False)
if immediately or self._is_in_scheduled_time(datetime.now()):
self._send(*args, **kwargs)
else:
self.queue.put({

View File

@@ -1,15 +1,16 @@
import sys
import importlib
import json
import shutil
import traceback
import site
import importlib
import sys
import traceback
from pathlib import Path
from typing import Any, Dict, List, Optional, Tuple, Set
from typing import Dict, List, Optional, Tuple, Set
from packaging.specifiers import SpecifierSet, InvalidSpecifier
from packaging.version import Version, InvalidVersion
from pkg_resources import Requirement, working_set
from requests import Response
from app.core.cache import cached
from app.core.config import settings
@@ -85,11 +86,13 @@ class PluginHelper(metaclass=Singleton):
if res is None:
return None
if res:
content = res.text
try:
return json.loads(res.text)
return json.loads(content)
except json.JSONDecodeError:
logger.error(f"插件包数据解析失败:{res.text}")
return None
if "404: Not Found" not in content:
logger.warn(f"插件包数据解析失败:{content}")
return None
return {}
def get_plugin_package_version(self, pid: str, repo_url: str, package_version: Optional[str] = None) -> Optional[str]:
@@ -109,14 +112,11 @@ class PluginHelper(metaclass=Singleton):
package_version = settings.VERSION_FLAG
# 优先检查指定版本的插件,即 package.v(x).json 文件中是否存在该插件,如果存在,返回该版本号
plugins = self.get_plugins(repo_url, package_version)
if pid in plugins:
if pid in (self.get_plugins(repo_url, package_version) or []):
return package_version
# 如果指定版本的插件不存在,检查全局 package.json 文件,查看插件是否兼容指定的版本
global_plugins = self.get_plugins(repo_url)
plugin = global_plugins.get(pid, None)
plugin = (self.get_plugins(repo_url) or {}).get(pid, None)
# 检查插件是否明确支持当前指定的版本(如 v2 或 v3如果支持返回空字符串表示使用 package.jsonv1
if plugin and plugin.get(package_version) is True:
return ""
@@ -317,7 +317,7 @@ class PluginHelper(metaclass=Singleton):
else:
return None, "插件在仓库中不存在或返回数据格式不正确"
except Exception as e:
logger.error(f"插件数据解析失败:{res.text}{e}")
logger.error(f"插件数据解析失败:{e}")
return None, "插件数据解析失败"
def __download_files(self, pid: str, file_list: List[dict], user_repo: str,
@@ -399,8 +399,7 @@ class PluginHelper(metaclass=Singleton):
with open(requirements_file_path, "w", encoding="utf-8") as f:
f.write(requirements_txt)
success, message = self.__pip_install_with_fallback(requirements_file_path)
return success, message
return self.pip_install_with_fallback(requirements_file_path)
return True, "" # 如果 requirements.txt 为空,视作成功
@@ -417,7 +416,7 @@ class PluginHelper(metaclass=Singleton):
# 检查是否存在 requirements.txt 文件
if requirements_file.exists():
logger.info(f"{pid} 存在依赖,开始尝试安装依赖")
success, error_message = self.__pip_install_with_fallback(requirements_file)
success, error_message = self.pip_install_with_fallback(requirements_file)
if success:
return True, True, ""
else:
@@ -475,7 +474,7 @@ class PluginHelper(metaclass=Singleton):
shutil.rmtree(plugin_dir, ignore_errors=True)
@staticmethod
def __pip_install_with_fallback(requirements_file: Path) -> Tuple[bool, str]:
def pip_install_with_fallback(requirements_file: Path) -> Tuple[bool, str]:
"""
使用自动降级策略安装依赖,并确保新安装的包可被动态导入
:param requirements_file: 依赖的 requirements.txt 文件路径
@@ -520,7 +519,7 @@ class PluginHelper(metaclass=Singleton):
def __request_with_fallback(url: str,
headers: Optional[dict] = None,
timeout: Optional[int] = 60,
is_api: bool = False) -> Optional[Any]:
is_api: bool = False) -> Optional[Response]:
"""
使用自动降级策略,请求资源,优先级依次为镜像站、代理、直连
:param url: 目标URL
@@ -596,7 +595,6 @@ class PluginHelper(metaclass=Singleton):
def install_dependencies(self, dependencies: List[str]) -> Tuple[bool, str]:
"""
安装指定的依赖项列表
:param dependencies: 需要安装或更新的依赖项列表
:return: (success, message)
"""
@@ -611,12 +609,12 @@ class PluginHelper(metaclass=Singleton):
with open(requirements_temp_file, "w", encoding="utf-8") as f:
for dep in dependencies:
f.write(dep + "\n")
# 使用自动降级策略安装依赖
success, message = self.__pip_install_with_fallback(requirements_temp_file)
# 删除临时文件
requirements_temp_file.unlink()
return success, message
try:
# 使用自动降级策略安装依赖
return self.pip_install_with_fallback(requirements_temp_file)
finally:
# 删除临时文件
requirements_temp_file.unlink()
except Exception as e:
logger.error(f"安装依赖项时发生错误:{e}")
return False, f"安装依赖项时发生错误:{e}"

View File

@@ -15,8 +15,8 @@ class ResourceHelper:
检测和更新资源包
"""
# 资源包的git仓库地址
_repo = f"{settings.GITHUB_PROXY}https://raw.githubusercontent.com/jxxghp/MoviePilot-Resources/main/package.json"
_files_api = f"https://api.github.com/repos/jxxghp/MoviePilot-Resources/contents/resources"
_repo = f"{settings.GITHUB_PROXY}https://raw.githubusercontent.com/jxxghp/MoviePilot-Resources/main/package.v2.json"
_files_api = f"https://api.github.com/repos/jxxghp/MoviePilot-Resources/contents/resources.v2"
_base_dir: Path = settings.ROOT_PATH
def __init__(self):
@@ -58,9 +58,15 @@ class ResourceHelper:
if rtype == "auth":
# 站点认证资源
local_version = SitesHelper().auth_version
# 阻断v2.3.0以下的版本直接更新,避免无限重启
if StringUtils.compare_version(local_version, "<", "2.3.0"):
continue
elif rtype == "sites":
# 站点索引资源
local_version = SitesHelper().indexer_version
# 阻断v2.0.0以下的版本直接更新,避免无限重启
if StringUtils.compare_version(local_version, "<", "2.0.0"):
continue
else:
continue
if StringUtils.compare_version(version, ">", local_version):

View File

@@ -306,6 +306,7 @@ class RssHelper:
return False
finally:
if parser is not None:
parser.close()
del parser
if root is None:
@@ -319,70 +320,72 @@ class RssHelper:
items_count = min(len(items), self.MAX_RSS_ITEMS)
if len(items) > self.MAX_RSS_ITEMS:
logger.warning(f"RSS条目过多: {len(items)},仅处理前{self.MAX_RSS_ITEMS}")
try:
for item in items[:items_count]:
try:
# 使用xpath提取信息更高效
title_nodes = item.xpath('.//title')
title = title_nodes[0].text if title_nodes and title_nodes[0].text else ""
if not title:
continue
for item in items[:items_count]:
try:
# 使用xpath提取信息更高效
title_nodes = item.xpath('.//title')
title = title_nodes[0].text if title_nodes and title_nodes[0].text else ""
if not title:
# 描述
desc_nodes = item.xpath('.//description | .//summary')
description = desc_nodes[0].text if desc_nodes and desc_nodes[0].text else ""
# 种子页面
link_nodes = item.xpath('.//link')
if link_nodes:
link = link_nodes[0].text if hasattr(link_nodes[0], 'text') and link_nodes[0].text else link_nodes[0].get('href', '')
else:
link = ""
# 种子链接
enclosure_nodes = item.xpath('.//enclosure')
enclosure = enclosure_nodes[0].get('url', '') if enclosure_nodes else ""
if not enclosure and not link:
continue
# 部分RSS只有link没有enclosure
if not enclosure and link:
enclosure = link
# 大小
size = 0
if enclosure_nodes:
size_attr = enclosure_nodes[0].get('length', '0')
if size_attr and str(size_attr).isdigit():
size = int(size_attr)
# 发布日期
pubdate_nodes = item.xpath('.//pubDate | .//published | .//updated')
pubdate = ""
if pubdate_nodes and pubdate_nodes[0].text:
pubdate = StringUtils.get_time(pubdate_nodes[0].text)
# 获取豆瓣昵称
nickname_nodes = item.xpath('.//*[local-name()="creator"]')
nickname = nickname_nodes[0].text if nickname_nodes and nickname_nodes[0].text else ""
# 返回对象
tmp_dict = {
'title': title,
'enclosure': enclosure,
'size': size,
'description': description,
'link': link,
'pubdate': pubdate
}
# 如果豆瓣昵称不为空返回数据增加豆瓣昵称供doubansync插件获取
if nickname:
tmp_dict['nickname'] = nickname
ret_array.append(tmp_dict)
except Exception as e1:
logger.debug(f"解析RSS条目失败{str(e1)} - {traceback.format_exc()}")
continue
# 描述
desc_nodes = item.xpath('.//description | .//summary')
description = desc_nodes[0].text if desc_nodes and desc_nodes[0].text else ""
# 种子页面
link_nodes = item.xpath('.//link')
if link_nodes:
link = link_nodes[0].text if hasattr(link_nodes[0], 'text') and link_nodes[0].text else \
link_nodes[0].get('href', '')
else:
link = ""
# 种子链接
enclosure_nodes = item.xpath('.//enclosure')
enclosure = enclosure_nodes[0].get('url', '') if enclosure_nodes else ""
if not enclosure and not link:
continue
# 部分RSS只有link没有enclosure
if not enclosure and link:
enclosure = link
# 大小
size = 0
if enclosure_nodes:
size_attr = enclosure_nodes[0].get('length', '0')
if size_attr and str(size_attr).isdigit():
size = int(size_attr)
# 发布日期
pubdate_nodes = item.xpath('.//pubDate | .//published | .//updated')
pubdate = ""
if pubdate_nodes and pubdate_nodes[0].text:
pubdate = StringUtils.get_time(pubdate_nodes[0].text)
# 获取豆瓣昵称
nickname_nodes = item.xpath('.//*[local-name()="creator"]')
nickname = nickname_nodes[0].text if nickname_nodes and nickname_nodes[0].text else ""
# 返回对象
tmp_dict = {
'title': title,
'enclosure': enclosure,
'size': size,
'description': description,
'link': link,
'pubdate': pubdate
}
# 如果豆瓣昵称不为空返回数据增加豆瓣昵称供doubansync插件获取
if nickname:
tmp_dict['nickname'] = nickname
ret_array.append(tmp_dict)
except Exception as e1:
logger.debug(f"解析RSS条目失败{str(e1)} - {traceback.format_exc()}")
continue
finally:
items.clear()
del items
except Exception as e2:
logger.error(f"解析RSS失败{str(e2)} - {traceback.format_exc()}")

View File

@@ -107,8 +107,7 @@ class ServiceBaseHelper(Generic[TConf]):
迭代所有模块的实例及其对应的配置,返回 ServiceInfo 实例
"""
configs = self.get_configs()
modules = self.modulemanager.get_running_type_modules(self.module_type)
for module in modules:
for module in self.modulemanager.get_running_type_modules(self.module_type):
if not module:
continue
module_instances = module.get_instances()

View File

@@ -58,7 +58,7 @@ class SubscribeHelper(metaclass=Singleton):
self.get_user_uuid()
self.get_github_user()
@cached(maxsize=20, ttl=1800)
@cached(maxsize=5, ttl=1800)
def get_statistic(self, stype: str, page: Optional[int] = 1, count: Optional[int] = 30) -> List[dict]:
"""
获取订阅统计数据

View File

@@ -32,6 +32,7 @@ class SystemHelper:
if event_data.key not in ['DEBUG', 'LOG_LEVEL', 'LOG_MAX_FILE_SIZE', 'LOG_BACKUP_COUNT',
'LOG_FILE_FORMAT', 'LOG_CONSOLE_FORMAT']:
return
logger.info("配置变更,更新日志设置...")
logger.update_loggers()
@staticmethod
@@ -109,7 +110,6 @@ class SystemHelper:
try:
# 检查容器是否配置了自动重启策略
has_restart_policy = SystemHelper._check_restart_policy()
if has_restart_policy:
# 有重启策略,使用优雅退出方式
logger.info("检测到容器配置了自动重启策略,使用优雅重启方式...")
@@ -120,7 +120,6 @@ class SystemHelper:
# 没有重启策略使用Docker API强制重启
logger.info("容器未配置自动重启策略使用Docker API重启...")
return SystemHelper._docker_api_restart()
except Exception as err:
logger.error(f"重启失败: {str(err)}")
# 降级为Docker API重启
@@ -141,7 +140,6 @@ class SystemHelper:
# 重启容器
client.containers.get(container_id).restart()
return True, ""
except Exception as docker_err:
return False, f"重启时发生错误:{str(docker_err)}"

View File

@@ -1,8 +1,7 @@
from concurrent.futures import ThreadPoolExecutor
from typing import Optional
from app.utils.singleton import Singleton
from app.core.config import settings
from app.utils.singleton import Singleton
class ThreadHelper(metaclass=Singleton):
@@ -10,7 +9,7 @@ class ThreadHelper(metaclass=Singleton):
线程池管理
"""
def __init__(self):
self.pool = ThreadPoolExecutor(max_workers=settings.CONF['threadpool'])
self.pool = ThreadPoolExecutor(max_workers=settings.CONF.threadpool)
def submit(self, func, *args, **kwargs):
"""
@@ -28,6 +27,3 @@ class ThreadHelper(metaclass=Singleton):
:return:
"""
self.pool.shutdown()
def __del__(self):
self.shutdown()

View File

@@ -112,7 +112,7 @@ class ServiceBase(Generic[TService, TConf], metaclass=ABCMeta):
# 通过服务类型或工厂函数来创建实例
if isinstance(service_type, type):
# 如果传入的是类类型,调用构造函数实例化
self._instances[conf.name] = service_type(**conf.config)
self._instances[conf.name] = service_type(name=conf.name, **conf.config)
else:
# 如果传入的是工厂函数,直接调用工厂函数
self._instances[conf.name] = service_type(conf)
@@ -210,8 +210,8 @@ class _MessageBase(ServiceBase[TService, NotificationConf]):
# 检查消息来源
if message.source and message.source != source:
return False
# 检查消息类型开关
if message.mtype:
# 不是定向发送时,检查消息类型开关
if not message.userid and message.mtype:
conf = self.get_config(source)
if conf:
switchs = conf.switchs or []

View File

@@ -30,7 +30,7 @@ class BangumiApi(object):
self._session = requests.Session()
self._req = RequestUtils(session=self._session)
@cached(maxsize=settings.CONF["bangumi"], ttl=settings.CONF["meta"])
@cached(maxsize=settings.CONF.bangumi, ttl=settings.CONF.meta)
def __invoke(self, url, key: Optional[str] = None, **kwargs):
req_url = self._base_url + url
params = {}

View File

@@ -171,14 +171,14 @@ class DoubanApi(metaclass=Singleton):
).digest()
).decode()
@cached(maxsize=settings.CONF["douban"], ttl=settings.CONF["meta"])
@cached(maxsize=settings.CONF.douban, ttl=settings.CONF.meta)
def __invoke_recommend(self, url: str, **kwargs) -> dict:
"""
推荐/发现类API
"""
return self.__invoke(url, **kwargs)
@cached(maxsize=settings.CONF["douban"], ttl=settings.CONF["meta"])
@cached(maxsize=settings.CONF.douban, ttl=settings.CONF.meta)
def __invoke_search(self, url: str, **kwargs) -> dict:
"""
搜索类API
@@ -213,7 +213,7 @@ class DoubanApi(metaclass=Singleton):
return resp.json()
return resp.json() if resp else {}
@cached(maxsize=settings.CONF["douban"], ttl=settings.CONF["meta"])
@cached(maxsize=settings.CONF.douban, ttl=settings.CONF.meta)
def __post(self, url: str, **kwargs) -> dict:
"""
POST请求

View File

@@ -16,7 +16,7 @@ from app.schemas.types import MediaType
lock = RLock()
CACHE_EXPIRE_TIMESTAMP_STR = "cache_expire_timestamp"
EXPIRE_TIMESTAMP = settings.CONF["meta"]
EXPIRE_TIMESTAMP = settings.CONF.meta
class DoubanCache(metaclass=Singleton):

View File

@@ -29,6 +29,7 @@ class EmbyModule(_ModuleBase, _MediaServerBase[Emby]):
event_data: schemas.ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.MediaServers.value]:
return
logger.info("配置变更重新初始化Emby模块...")
self.init_module()
@staticmethod

View File

@@ -399,28 +399,30 @@ class FanartModule(_ModuleBase):
if not mediainfo.get_image(season_image):
mediainfo.set_image(season_image, image_obj.get('url'))
else:
# 其他图片优先环境变量指定语言再like最多
def pick_best_image(images):
def __pick_best_image(_images):
lang_env = settings.FANART_LANG
if lang_env:
langs = [lang.strip() for lang in lang_env.split(",") if lang.strip()]
for lang in langs:
lang_images = [img for img in images if img.get('lang') == lang]
lang_images = [img for img in _images if img.get('lang') == lang]
if lang_images:
lang_images.sort(key=lambda x: int(x.get('likes', 0)), reverse=True)
return lang_images[0]
# 没设置或没找到,按原逻辑 zh、en、like最多
zh_images = [img for img in images if img.get('lang') == 'zh']
zh_images = [img for img in _images if img.get('lang') == 'zh']
if zh_images:
zh_images.sort(key=lambda x: int(x.get('likes', 0)), reverse=True)
return zh_images[0]
en_images = [img for img in images if img.get('lang') == 'en']
en_images = [img for img in _images if img.get('lang') == 'en']
if en_images:
en_images.sort(key=lambda x: int(x.get('likes', 0)), reverse=True)
return en_images[0]
images.sort(key=lambda x: int(x.get('likes', 0)), reverse=True)
return images[0]
image_obj = pick_best_image(images)
_images.sort(key=lambda x: int(x.get('likes', 0)), reverse=True)
return _images[0]
image_obj = __pick_best_image(images)
# 设置图片,没有图片才设置
if not mediainfo.get_image(image_name):
mediainfo.set_image(image_name, image_obj.get('url'))
@@ -438,7 +440,7 @@ class FanartModule(_ModuleBase):
return result
@classmethod
@cached(maxsize=settings.CONF["fanart"], ttl=settings.CONF["meta"])
@cached(maxsize=settings.CONF.fanart, ttl=settings.CONF.meta)
def __request_fanart(cls, media_type: MediaType, queryid: Union[str, int]) -> Optional[dict]:
if media_type == MediaType.MOVIE:
image_url = cls._movie_url % queryid

View File

@@ -5,7 +5,7 @@ import secrets
import threading
import time
from pathlib import Path
from typing import List, Dict, Optional, Tuple, Union
from typing import List, Optional, Tuple, Union
import requests
from tqdm import tqdm
@@ -52,9 +52,6 @@ class AliPan(StorageBase, metaclass=Singleton):
# 基础url
base_url = "https://openapi.alipan.com"
# CID和路径缓存
_id_cache: Dict[str, Tuple[str, str]] = {}
def __init__(self):
super().__init__()
self.session = requests.Session()
@@ -279,61 +276,6 @@ class AliPan(StorageBase, metaclass=Singleton):
return ret_data.get(result_key)
return ret_data
def _path_to_id(self, drive_id: str, path: str) -> Tuple[str, str]:
"""
路径转drive_id, file_id带缓存机制
"""
# 根目录
if path == "/":
return drive_id, "root"
if len(path) > 1 and path.endswith("/"):
path = path[:-1]
# 检查缓存
if path in self._id_cache:
return self._id_cache[path]
# 逐级查找缓存
file_id = "root"
file_path = "/"
for p in Path(path).parents:
if str(p) in self._id_cache:
file_path = str(p)
file_id = self._id_cache[file_path]
break
# 计算相对路径
rel_path = Path(path).relative_to(file_path)
for part in Path(rel_path).parts:
find_part = False
next_marker = None
while True:
resp = self._request_api(
"POST",
"/adrive/v1.0/openFile/list",
json={
"drive_id": drive_id,
"limit": 100,
"marker": next_marker,
"parent_file_id": file_id,
}
)
if not resp:
break
for item in resp.get("items", []):
if item["name"] == part:
file_id = item["file_id"]
find_part = True
break
if find_part:
break
if len(resp.get("items")) < 100:
break
if not find_part:
raise FileNotFoundError(f"【阿里云盘】{path} 不存在")
if file_id == "root":
raise FileNotFoundError(f"【阿里云盘】{path} 不存在")
# 缓存路径
self._id_cache[path] = (drive_id, file_id)
return drive_id, file_id
def __get_fileitem(self, fileinfo: dict, parent: str = "/") -> schemas.FileItem:
"""
获取文件信息
@@ -427,9 +369,6 @@ class AliPan(StorageBase, metaclass=Singleton):
break
next_marker = resp.get("next_marker")
for item in resp.get("items", []):
# 更新缓存
path = f"{fileitem.path}{item.get('name')}"
self._id_cache[path] = (drive_id, item.get("file_id"))
items.append(self.__get_fileitem(item, parent=fileitem.path))
if len(resp.get("items")) < 100:
break
@@ -467,7 +406,6 @@ class AliPan(StorageBase, metaclass=Singleton):
return None
# 缓存新目录
new_path = Path(parent_item.path) / name
self._id_cache[str(new_path)] = (resp.get("drive_id"), resp.get("file_id"))
return self._delay_get_item(new_path)
@staticmethod
@@ -837,15 +775,9 @@ class AliPan(StorageBase, metaclass=Singleton):
if resp.get("code"):
logger.warn(f"【阿里云盘】重命名失败: {resp.get('message')}")
return False
if fileitem.path in self._id_cache:
del self._id_cache[fileitem.path]
for key in list(self._id_cache.keys()):
if key.startswith(fileitem.path):
del self._id_cache[key]
self._id_cache[str(Path(fileitem.path).parent / name)] = (resp.get("drive_id"), resp.get("file_id"))
return True
def get_item(self, path: Path) -> Optional[schemas.FileItem]:
def get_item(self, path: Path, drive_id: str = None) -> Optional[schemas.FileItem]:
"""
获取指定路径的文件/目录项
"""
@@ -854,7 +786,7 @@ class AliPan(StorageBase, metaclass=Singleton):
"POST",
"/adrive/v1.0/openFile/get_by_path",
json={
"drive_id": self._default_drive_id,
"drive_id": drive_id or self._default_drive_id,
"file_path": str(path)
}
)
@@ -910,9 +842,15 @@ class AliPan(StorageBase, metaclass=Singleton):
def copy(self, fileitem: schemas.FileItem, path: Path, new_name: str) -> bool:
"""
企业级复制实现(支持目录递归复制)
复制文件到指定路径
:param fileitem: 要复制的文件项
:param path: 目标目录路径
:param new_name: 新文件名
"""
dest_cid = self._path_to_id(fileitem.drive_id, str(path))
dest_fileitem = self.get_item(path, drive_id=fileitem.drive_id)
if not dest_fileitem or dest_fileitem.type != "dir":
logger.warn(f"【阿里云盘】目标路径 {path} 不存在或不是目录!")
return False
resp = self._request_api(
"POST",
"/adrive/v1.0/openFile/copy",
@@ -920,7 +858,7 @@ class AliPan(StorageBase, metaclass=Singleton):
"drive_id": fileitem.drive_id,
"file_id": fileitem.fileid,
"to_drive_id": fileitem.drive_id,
"to_parent_file_id": dest_cid
"to_parent_file_id": dest_fileitem.fileid,
}
)
if not resp:
@@ -932,18 +870,20 @@ class AliPan(StorageBase, metaclass=Singleton):
new_path = Path(path) / fileitem.name
new_file = self._delay_get_item(new_path)
self.rename(new_file, new_name)
# 更新缓存
del self._id_cache[fileitem.path]
rename_new_path = Path(path) / new_name
self._id_cache[str(rename_new_path)] = (resp.get("drive_id"), resp.get("file_id"))
return True
def move(self, fileitem: schemas.FileItem, path: Path, new_name: str) -> bool:
"""
原子性移动操作实现
移动文件到指定路径
:param fileitem: 要移动的文件项
:param path: 目标目录路径
:param new_name: 新文件名
"""
src_fid = fileitem.fileid
target_id = self._path_to_id(fileitem.drive_id, str(path))
target_fileitem = self.get_item(path, drive_id=fileitem.drive_id)
if not target_fileitem or target_fileitem.type != "dir":
logger.warn(f"【阿里云盘】目标路径 {path} 不存在或不是目录!")
return False
resp = self._request_api(
"POST",
@@ -951,7 +891,7 @@ class AliPan(StorageBase, metaclass=Singleton):
json={
"drive_id": fileitem.drive_id,
"file_id": src_fid,
"to_parent_file_id": target_id,
"to_parent_file_id": target_fileitem.fileid,
"new_name": new_name
}
)
@@ -960,10 +900,6 @@ class AliPan(StorageBase, metaclass=Singleton):
if resp.get("code"):
logger.warn(f"【阿里云盘】移动文件失败: {resp.get('message')}")
return False
# 更新缓存
del self._id_cache[fileitem.path]
rename_new_path = Path(path) / new_name
self._id_cache[str(rename_new_path)] = (resp.get("drive_id"), resp.get("file_id"))
return True
def link(self, fileitem: schemas.FileItem, target_file: Path) -> bool:

View File

@@ -4,7 +4,6 @@ from pathlib import Path
from typing import Optional, List, Dict
import requests
from requests import Response
from app import schemas
from app.core.cache import cached
@@ -20,7 +19,7 @@ from app.utils.url import UrlUtils
class Alist(StorageBase, metaclass=Singleton):
"""
Alist相关操作
api文档https://alist.nn.ci/zh/guide/api
api文档https://oplist.org/zh/
"""
# 存储类型
@@ -77,7 +76,7 @@ class Alist(StorageBase, metaclass=Singleton):
token = conf.get("token")
if token:
return str(token)
resp: Response = RequestUtils(headers={
resp = RequestUtils(headers={
'Content-Type': 'application/json'
}).post_res(
self.__get_api_url("/api/auth/login"),
@@ -102,20 +101,20 @@ class Alist(StorageBase, metaclass=Singleton):
"""
if resp is None:
logger.warning("alist】请求登录失败无法连接alist服务")
logger.warning("OpenList】请求登录失败无法连接alist服务")
return ""
if resp.status_code != 200:
logger.warning(f"alist】更新令牌请求发送失败状态码{resp.status_code}")
logger.warning(f"OpenList】更新令牌请求发送失败状态码{resp.status_code}")
return ""
result = resp.json()
if result["code"] != 200:
logger.critical(f'alist】更新令牌错误信息{result["message"]}')
logger.critical(f'OpenList】更新令牌错误信息{result["message"]}')
return ""
logger.debug("alist】AList获取令牌成功")
logger.debug("OpenList】AList获取令牌成功")
return result["data"]["token"]
def __get_header_with_token(self) -> dict:
@@ -151,7 +150,7 @@ class Alist(StorageBase, metaclass=Singleton):
if item:
return [item]
return []
resp: Response = RequestUtils(
resp = RequestUtils(
headers=self.__get_header_with_token()
).post_res(
self.__get_api_url("/api/fs/list"),
@@ -200,11 +199,11 @@ class Alist(StorageBase, metaclass=Singleton):
"""
if resp is None:
logger.warn(f"alist】请求获取目录 {fileitem.path} 的文件列表失败无法连接alist服务")
logger.warn(f"OpenList】请求获取目录 {fileitem.path} 的文件列表失败无法连接alist服务")
return []
if resp.status_code != 200:
logger.warn(
f"alist】请求获取目录 {fileitem.path} 的文件列表失败,状态码:{resp.status_code}"
f"OpenList】请求获取目录 {fileitem.path} 的文件列表失败,状态码:{resp.status_code}"
)
return []
@@ -212,7 +211,7 @@ class Alist(StorageBase, metaclass=Singleton):
if result["code"] != 200:
logger.warn(
f'alist】获取目录 {fileitem.path} 的文件列表失败,错误信息:{result["message"]}'
f'OpenList】获取目录 {fileitem.path} 的文件列表失败,错误信息:{result["message"]}'
)
return []
@@ -240,7 +239,7 @@ class Alist(StorageBase, metaclass=Singleton):
:param name: 目录名
"""
path = Path(fileitem.path) / name
resp: Response = RequestUtils(
resp = RequestUtils(
headers=self.__get_header_with_token()
).post_res(
self.__get_api_url("/api/fs/mkdir"),
@@ -258,15 +257,15 @@ class Alist(StorageBase, metaclass=Singleton):
}
"""
if resp is None:
logger.warn(f"alist】请求创建目录 {path} 失败无法连接alist服务")
logger.warn(f"OpenList】请求创建目录 {path} 失败无法连接alist服务")
return None
if resp.status_code != 200:
logger.warn(f"alist】请求创建目录 {path} 失败,状态码:{resp.status_code}")
logger.warn(f"OpenList】请求创建目录 {path} 失败,状态码:{resp.status_code}")
return None
result = resp.json()
if result["code"] != 200:
logger.warn(f'alist】创建目录 {path} 失败,错误信息:{result["message"]}')
logger.warn(f'OpenList】创建目录 {path} 失败,错误信息:{result["message"]}')
return None
return self.get_item(path)
@@ -304,7 +303,7 @@ class Alist(StorageBase, metaclass=Singleton):
:param per_page: 每页数量
:param refresh: 是否刷新
"""
resp: Response = RequestUtils(
resp = RequestUtils(
headers=self.__get_header_with_token()
).post_res(
self.__get_api_url("/api/fs/get"),
@@ -348,15 +347,15 @@ class Alist(StorageBase, metaclass=Singleton):
}
"""
if resp is None:
logger.warn(f"alist】请求获取文件 {path} 失败无法连接alist服务")
logger.warn(f"OpenList】请求获取文件 {path} 失败无法连接alist服务")
return None
if resp.status_code != 200:
logger.warn(f"alist】请求获取文件 {path} 失败,状态码:{resp.status_code}")
logger.warn(f"OpenList】请求获取文件 {path} 失败,状态码:{resp.status_code}")
return None
result = resp.json()
if result["code"] != 200:
logger.debug(f'alist】获取文件 {path} 失败,错误信息:{result["message"]}')
logger.debug(f'OpenList】获取文件 {path} 失败,错误信息:{result["message"]}')
return None
return schemas.FileItem(
@@ -381,7 +380,7 @@ class Alist(StorageBase, metaclass=Singleton):
"""
删除文件
"""
resp: Response = RequestUtils(
resp = RequestUtils(
headers=self.__get_header_with_token()
).post_res(
self.__get_api_url("/api/fs/remove"),
@@ -405,18 +404,18 @@ class Alist(StorageBase, metaclass=Singleton):
}
"""
if resp is None:
logger.warn(f"alist】请求删除文件 {fileitem.path} 失败无法连接alist服务")
logger.warn(f"OpenList】请求删除文件 {fileitem.path} 失败无法连接alist服务")
return False
if resp.status_code != 200:
logger.warn(
f"alist】请求删除文件 {fileitem.path} 失败,状态码:{resp.status_code}"
f"OpenList】请求删除文件 {fileitem.path} 失败,状态码:{resp.status_code}"
)
return False
result = resp.json()
if result["code"] != 200:
logger.warn(
f'alist】删除文件 {fileitem.path} 失败,错误信息:{result["message"]}'
f'OpenList】删除文件 {fileitem.path} 失败,错误信息:{result["message"]}'
)
return False
return True
@@ -425,7 +424,7 @@ class Alist(StorageBase, metaclass=Singleton):
"""
重命名文件
"""
resp: Response = RequestUtils(
resp = RequestUtils(
headers=self.__get_header_with_token()
).post_res(
self.__get_api_url("/api/fs/rename"),
@@ -447,18 +446,18 @@ class Alist(StorageBase, metaclass=Singleton):
}
"""
if not resp:
logger.warn(f"alist】请求重命名文件 {fileitem.path} 失败无法连接alist服务")
logger.warn(f"OpenList】请求重命名文件 {fileitem.path} 失败无法连接alist服务")
return False
if resp.status_code != 200:
logger.warn(
f"alist】请求重命名文件 {fileitem.path} 失败,状态码:{resp.status_code}"
f"OpenList】请求重命名文件 {fileitem.path} 失败,状态码:{resp.status_code}"
)
return False
result = resp.json()
if result["code"] != 200:
logger.warn(
f'alist】重命名文件 {fileitem.path} 失败,错误信息:{result["message"]}'
f'OpenList】重命名文件 {fileitem.path} 失败,错误信息:{result["message"]}'
)
return False
@@ -476,7 +475,7 @@ class Alist(StorageBase, metaclass=Singleton):
:param path: 文件保存路径
:param password: 文件密码
"""
resp: Response = RequestUtils(
resp = RequestUtils(
headers=self.__get_header_with_token()
).post_res(
self.__get_api_url("/api/fs/get"),
@@ -512,15 +511,15 @@ class Alist(StorageBase, metaclass=Singleton):
}
"""
if not resp:
logger.warn(f"alist】请求获取文件 {path} 失败无法连接alist服务")
logger.warn(f"OpenList】请求获取文件 {path} 失败无法连接alist服务")
return None
if resp.status_code != 200:
logger.warn(f"alist】请求获取文件 {path} 失败,状态码:{resp.status_code}")
logger.warn(f"OpenList】请求获取文件 {path} 失败,状态码:{resp.status_code}")
return None
result = resp.json()
if result["code"] != 200:
logger.warn(f'alist】获取文件 {path} 失败,错误信息:{result["message"]}')
logger.warn(f'OpenList】获取文件 {path} 失败,错误信息:{result["message"]}')
return None
if result["data"]["raw_url"]:
@@ -561,13 +560,13 @@ class Alist(StorageBase, metaclass=Singleton):
headers.setdefault("As-Task", str(task).lower())
headers.setdefault("File-Path", encoded_path)
with open(path, "rb") as f:
resp: Response = RequestUtils(headers=headers).put_res(
resp = RequestUtils(headers=headers).put_res(
self.__get_api_url("/api/fs/put"),
data=f,
)
if resp.status_code != 200:
logger.warn(f"alist】请求上传文件 {path} 失败,状态码:{resp.status_code}")
logger.warn(f"OpenList】请求上传文件 {path} 失败,状态码:{resp.status_code}")
return None
new_item = self.get_item(Path(fileitem.path) / path.name)
@@ -590,7 +589,7 @@ class Alist(StorageBase, metaclass=Singleton):
:param path: 目标目录
:param new_name: 新文件名
"""
resp: Response = RequestUtils(
resp = RequestUtils(
headers=self.__get_header_with_token()
).post_res(
self.__get_api_url("/api/fs/copy"),
@@ -617,19 +616,19 @@ class Alist(StorageBase, metaclass=Singleton):
"""
if resp is None:
logger.warn(
f"alist】请求复制文件 {fileitem.path} 失败无法连接alist服务"
f"OpenList】请求复制文件 {fileitem.path} 失败无法连接alist服务"
)
return False
if resp.status_code != 200:
logger.warn(
f"alist】请求复制文件 {fileitem.path} 失败,状态码:{resp.status_code}"
f"OpenList】请求复制文件 {fileitem.path} 失败,状态码:{resp.status_code}"
)
return False
result = resp.json()
if result["code"] != 200:
logger.warn(
f'alist】复制文件 {fileitem.path} 失败,错误信息:{result["message"]}'
f'OpenList】复制文件 {fileitem.path} 失败,错误信息:{result["message"]}'
)
return False
# 重命名
@@ -649,7 +648,7 @@ class Alist(StorageBase, metaclass=Singleton):
# 先重命名
if fileitem.name != new_name:
self.rename(fileitem, new_name)
resp: Response = RequestUtils(
resp = RequestUtils(
headers=self.__get_header_with_token()
).post_res(
self.__get_api_url("/api/fs/move"),
@@ -676,19 +675,19 @@ class Alist(StorageBase, metaclass=Singleton):
"""
if resp is None:
logger.warn(
f"alist】请求移动文件 {fileitem.path} 失败无法连接alist服务"
f"OpenList】请求移动文件 {fileitem.path} 失败无法连接alist服务"
)
return False
if resp.status_code != 200:
logger.warn(
f"alist】请求移动文件 {fileitem.path} 失败,状态码:{resp.status_code}"
f"OpenList】请求移动文件 {fileitem.path} 失败,状态码:{resp.status_code}"
)
return False
result = resp.json()
if result["code"] != 200:
logger.warn(
f'alist】移动文件 {fileitem.path} 失败,错误信息:{result["message"]}'
f'OpenList】移动文件 {fileitem.path} 失败,错误信息:{result["message"]}'
)
return False
return True

View File

@@ -5,7 +5,7 @@ import secrets
import threading
import time
from pathlib import Path
from typing import List, Dict, Optional, Tuple, Union
from typing import List, Optional, Tuple, Union
import oss2
import requests
@@ -51,9 +51,6 @@ class U115Pan(StorageBase, metaclass=Singleton):
# 基础url
base_url = "https://proapi.115.com"
# CID和路径缓存
_id_cache: Dict[str, str] = {}
def __init__(self):
super().__init__()
self.session = requests.Session()
@@ -238,58 +235,6 @@ class U115Pan(StorageBase, metaclass=Singleton):
return ret_data.get(result_key)
return ret_data
def _path_to_id(self, path: str) -> str:
"""
路径转FID带缓存机制
"""
# 根目录
if path == "/":
return '0'
if len(path) > 1 and path.endswith("/"):
path = path[:-1]
# 检查缓存
if path in self._id_cache:
return self._id_cache[path]
# 逐级查找缓存
current_id = 0
parent_path = "/"
for p in Path(path).parents:
if str(p) in self._id_cache:
parent_path = str(p)
current_id = self._id_cache[parent_path]
break
# 计算相对路径
rel_path = Path(path).relative_to(parent_path)
for part in Path(rel_path).parts:
offset = 0
find_part = False
while True:
resp = self._request_api(
"GET",
"/open/ufile/files",
"data",
params={"cid": current_id, "limit": 1000, "offset": offset, "cur": True, "show_dir": 1}
)
if not resp:
break
for item in resp:
if item["fn"] == part:
current_id = item["fid"]
find_part = True
break
if find_part:
break
if len(resp) < 1000:
break
offset += len(resp)
if not find_part:
raise FileNotFoundError(f"【115】{path} 不存在")
if not current_id:
raise FileNotFoundError(f"【115】{path} 不存在")
# 缓存路径
self._id_cache[path] = str(current_id)
return str(current_id)
@staticmethod
def _calc_sha1(filepath: Path, size: Optional[int] = None) -> str:
"""
@@ -335,7 +280,11 @@ class U115Pan(StorageBase, metaclass=Singleton):
else:
cid = fileitem.fileid
if not cid:
cid = self._path_to_id(fileitem.path)
_fileitem = self.get_item(Path(fileitem.path))
if not _fileitem:
logger.warn(f"【115】获取目录 {fileitem.path} 失败!")
return []
cid = _fileitem.fileid
items = []
offset = 0
@@ -354,8 +303,6 @@ class U115Pan(StorageBase, metaclass=Singleton):
for item in resp:
# 更新缓存
path = f"{fileitem.path}{item['fn']}"
self._id_cache[path] = str(item["fid"])
file_path = path + ("/" if item["fc"] == "0" else "")
items.append(schemas.FileItem(
storage=self.schema.value,
@@ -398,8 +345,6 @@ class U115Pan(StorageBase, metaclass=Singleton):
return self.get_item(new_path)
logger.warn(f"【115】创建目录失败: {resp.get('error')}")
return None
# 缓存新目录
self._id_cache[str(new_path)] = str(resp["data"]["file_id"])
return schemas.FileItem(
storage=self.schema.value,
fileid=str(resp["data"]["file_id"]),
@@ -716,13 +661,6 @@ class U115Pan(StorageBase, metaclass=Singleton):
if not resp:
return False
if resp["state"]:
if fileitem.path in self._id_cache:
del self._id_cache[fileitem.path]
for key in list(self._id_cache.keys()):
if key.startswith(fileitem.path):
del self._id_cache[key]
new_path = Path(fileitem.path).parent / name
self._id_cache[str(new_path)] = fileitem.fileid
return True
return False
@@ -731,15 +669,12 @@ class U115Pan(StorageBase, metaclass=Singleton):
获取指定路径的文件/目录项
"""
try:
file_id = self._path_to_id(str(path))
if not file_id:
return None
resp = self._request_api(
"GET",
"POST",
"/open/folder/get_info",
"data",
params={
"file_id": int(file_id)
data={
"path": str(path)
}
)
if not resp:
@@ -753,7 +688,7 @@ class U115Pan(StorageBase, metaclass=Singleton):
basename=Path(resp["file_name"]).stem,
extension=Path(resp["file_name"]).suffix[1:] if resp["file_category"] == "1" else None,
pickcode=resp["pick_code"],
size=StringUtils.num_filesize(resp['size']) if resp["file_category"] == "1" else None,
size=resp['size_byte'] if resp["file_category"] == "1" else None,
modify_time=resp["utime"]
)
except Exception as e:
@@ -805,14 +740,17 @@ class U115Pan(StorageBase, metaclass=Singleton):
企业级复制实现(支持目录递归复制)
"""
src_fid = fileitem.fileid
dest_cid = self._path_to_id(str(path))
dest_fileitem = self.get_item(path)
if not dest_fileitem or dest_fileitem.type != "dir":
logger.warn(f"【115】目标路径 {path} 不是一个有效的目录!")
return False
resp = self._request_api(
"POST",
"/open/ufile/copy",
data={
"file_id": int(src_fid),
"pid": int(dest_cid)
"pid": int(dest_fileitem.fileid),
}
)
if not resp:
@@ -821,10 +759,6 @@ class U115Pan(StorageBase, metaclass=Singleton):
new_path = Path(path) / fileitem.name
new_item = self._delay_get_item(new_path)
self.rename(new_item, new_name)
# 更新缓存
del self._id_cache[fileitem.path]
rename_new_path = Path(path) / new_name
self._id_cache[str(rename_new_path)] = new_item.fileid
return True
return False
@@ -833,14 +767,16 @@ class U115Pan(StorageBase, metaclass=Singleton):
原子性移动操作实现
"""
src_fid = fileitem.fileid
dest_cid = self._path_to_id(str(path))
dest_fileitem = self.get_item(path)
if not dest_fileitem or dest_fileitem.type != "dir":
logger.warn(f"【115】目标路径 {path} 不是一个有效的目录!")
return False
resp = self._request_api(
"POST",
"/open/ufile/move",
data={
"file_ids": int(src_fid),
"to_cid": int(dest_cid)
"to_cid": int(dest_fileitem.fileid),
}
)
if not resp:
@@ -849,10 +785,6 @@ class U115Pan(StorageBase, metaclass=Singleton):
new_path = Path(path) / fileitem.name
new_file = self._delay_get_item(new_path)
self.rename(new_file, new_name)
# 更新缓存
del self._id_cache[fileitem.path]
rename_new_path = Path(path) / new_name
self._id_cache[str(rename_new_path)] = src_fid
return True
return False

View File

@@ -286,26 +286,29 @@ class IndexerModule(_ModuleBase):
return None
# 获取用户数据
logger.info(f"站点 {site.get('name')} 开始以 {site.get('schema')} 模型解析数据...")
site_obj.parse()
logger.debug(f"站点 {site.get('name')} 数据解析完成")
return SiteUserData(
domain=StringUtils.get_url_domain(site.get("url")),
userid=site_obj.userid,
username=site_obj.username,
user_level=site_obj.user_level,
join_at=site_obj.join_at,
upload=site_obj.upload,
download=site_obj.download,
ratio=site_obj.ratio,
bonus=site_obj.bonus,
seeding=site_obj.seeding,
seeding_size=site_obj.seeding_size,
seeding_info=site_obj.seeding_info or [],
leeching=site_obj.leeching,
leeching_size=site_obj.leeching_size,
message_unread=site_obj.message_unread,
message_unread_contents=site_obj.message_unread_contents or [],
updated_day=datetime.now().strftime('%Y-%m-%d'),
err_msg=site_obj.err_msg
)
try:
logger.info(f"站点 {site.get('name')} 开始以 {site.get('schema')} 模型解析数据...")
site_obj.parse()
logger.debug(f"站点 {site.get('name')} 数据解析完成")
return SiteUserData(
domain=StringUtils.get_url_domain(site.get("url")),
userid=site_obj.userid,
username=site_obj.username,
user_level=site_obj.user_level,
join_at=site_obj.join_at,
upload=site_obj.upload,
download=site_obj.download,
ratio=site_obj.ratio,
bonus=site_obj.bonus,
seeding=site_obj.seeding,
seeding_size=site_obj.seeding_size,
seeding_info=site_obj.seeding_info or [],
leeching=site_obj.leeching,
leeching_size=site_obj.leeching_size,
message_unread=site_obj.message_unread,
message_unread_contents=site_obj.message_unread_contents or [],
updated_day=datetime.now().strftime('%Y-%m-%d'),
err_msg=site_obj.err_msg
)
finally:
site_obj.clear()

View File

@@ -156,53 +156,57 @@ class SiteParserBase(metaclass=ABCMeta):
解析站点信息
:return:
"""
# Cookie模式时获取站点首页html
if self.request_mode == "apikey":
if not self.apikey and not self.token:
logger.warn(f"{self._site_name} 未设置cookie 或 apikey/token跳过后续操作")
return
self._index_html = {}
else:
# 检查是否已经登录
self._index_html = self._get_page_content(url=self._site_url)
if not self._parse_logged_in(self._index_html):
return
# 解析站点页面
self._parse_site_page(self._index_html)
# 解析用户基础信息
if self._user_basic_page:
self._parse_user_base_info(
self._get_page_content(
url=urljoin(self._base_url, self._user_basic_page),
params=self._user_basic_params,
headers=self._user_basic_headers
try:
# Cookie模式时获取站点首页html
if self.request_mode == "apikey":
if not self.apikey and not self.token:
logger.warn(f"{self._site_name} 未设置cookie 或 apikey/token跳过后续操作")
return
self._index_html = {}
else:
# 检查是否已经登录
self._index_html = self._get_page_content(url=self._site_url)
if not self._parse_logged_in(self._index_html):
return
# 解析站点页面
self._parse_site_page(self._index_html)
# 解析用户基础信息
if self._user_basic_page:
self._parse_user_base_info(
self._get_page_content(
url=urljoin(self._base_url, self._user_basic_page),
params=self._user_basic_params,
headers=self._user_basic_headers
)
)
)
else:
self._parse_user_base_info(self._index_html)
# 解析用户详细信息
if self._user_detail_page:
self._parse_user_detail_info(
self._get_page_content(
url=urljoin(self._base_url, self._user_detail_page),
params=self._user_detail_params,
headers=self._user_detail_headers
else:
self._parse_user_base_info(self._index_html)
# 解析用户详细信息
if self._user_detail_page:
self._parse_user_detail_info(
self._get_page_content(
url=urljoin(self._base_url, self._user_detail_page),
params=self._user_detail_params,
headers=self._user_detail_headers
)
)
)
# 解析用户未读消息
if settings.SITE_MESSAGE:
self._pase_unread_msgs()
# 解析用户上传、下载、分享率等信息
if self._user_traffic_page:
self._parse_user_traffic_info(
self._get_page_content(
url=urljoin(self._base_url, self._user_traffic_page),
params=self._user_traffic_params,
headers=self._user_traffic_headers
# 解析用户未读消息
if settings.SITE_MESSAGE:
self._pase_unread_msgs()
# 解析用户上传、下载、分享率等信息
if self._user_traffic_page:
self._parse_user_traffic_info(
self._get_page_content(
url=urljoin(self._base_url, self._user_traffic_page),
params=self._user_traffic_params,
headers=self._user_traffic_headers
)
)
)
# 解析用户做种信息
self._parse_seeding_pages()
# 解析用户做种信息
self._parse_seeding_pages()
finally:
# 关闭连接
self.close()
def _pase_unread_msgs(self):
"""
@@ -430,6 +434,22 @@ class SiteParserBase(metaclass=ABCMeta):
"""
pass
def close(self):
"""
关闭会话
"""
if self._session:
self._session.close()
self._session = None
def clear(self):
"""
清除当前解析器的所有信息
"""
self._index_html = ""
self.seeding_info.clear()
self.message_unread_contents.clear()
def to_dict(self):
"""
转化为字典

View File

@@ -30,6 +30,7 @@ class JellyfinModule(_ModuleBase, _MediaServerBase[Jellyfin]):
event_data: schemas.ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.MediaServers.value]:
return
logger.info("配置变更重新初始化Jellyfin模块...")
self.init_module()
@staticmethod

View File

@@ -30,6 +30,7 @@ class PlexModule(_ModuleBase, _MediaServerBase[Plex]):
event_data: schemas.ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.MediaServers.value]:
return
logger.info("配置变更重新初始化Plex模块...")
self.init_module()
@staticmethod

View File

@@ -84,7 +84,7 @@ class Plex:
logger.error(f"Authentication failed: {e}")
return None
@cached(maxsize=100, ttl=86400)
@cached(maxsize=32, ttl=86400)
def __get_library_images(self, library_key: str, mtype: int) -> Optional[List[str]]:
"""
获取媒体服务器最近添加的媒体的图片列表

View File

@@ -36,6 +36,7 @@ class QbittorrentModule(_ModuleBase, _DownloaderBase[Qbittorrent]):
event_data: schemas.ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.Downloaders.value]:
return
logger.info("配置变更重新加载Qbittorrent模块...")
self.init_module()
@staticmethod
@@ -165,19 +166,23 @@ class QbittorrentModule(_ModuleBase, _DownloaderBase[Qbittorrent]):
if error:
return None, None, None, "无法连接qbittorrent下载器"
if torrents:
for torrent in torrents:
# 名称与大小相等则认为是同一个种子
if torrent.get("name") == torrent_name and torrent.get("total_size") == torrent_size:
torrent_hash = torrent.get("hash")
torrent_tags = [str(tag).strip() for tag in torrent.get("tags").split(',')]
logger.warn(f"下载器中已存在该种子任务:{torrent_hash} - {torrent.get('name')}")
# 给种子打上标签
if "已整理" in torrent_tags:
server.remove_torrents_tag(ids=torrent_hash, tag=['已整理'])
if settings.TORRENT_TAG and settings.TORRENT_TAG not in torrent_tags:
logger.info(f"给种子 {torrent_hash} 打上标签:{settings.TORRENT_TAG}")
server.set_torrents_tag(ids=torrent_hash, tags=[settings.TORRENT_TAG])
return downloader or self.get_default_config_name(), torrent_hash, torrent_layout, f"下载任务已存在"
try:
for torrent in torrents:
# 名称与大小相等则认为是同一个种子
if torrent.get("name") == torrent_name and torrent.get("total_size") == torrent_size:
torrent_hash = torrent.get("hash")
torrent_tags = [str(tag).strip() for tag in torrent.get("tags").split(',')]
logger.warn(f"下载器中已存在该种子任务:{torrent_hash} - {torrent.get('name')}")
# 给种子打上标签
if "已整理" in torrent_tags:
server.remove_torrents_tag(ids=torrent_hash, tag=['已整理'])
if settings.TORRENT_TAG and settings.TORRENT_TAG not in torrent_tags:
logger.info(f"给种子 {torrent_hash} 打上标签:{settings.TORRENT_TAG}")
server.set_torrents_tag(ids=torrent_hash, tags=[settings.TORRENT_TAG])
return downloader or self.get_default_config_name(), torrent_hash, torrent_layout, f"下载任务已存在"
finally:
torrents.clear()
del torrents
return None, None, None, f"添加种子任务失败:{content}"
else:
# 获取种子Hash
@@ -195,16 +200,19 @@ class QbittorrentModule(_ModuleBase, _DownloaderBase[Qbittorrent]):
file_ids = []
# 需要的集清单
sucess_epidised = []
for torrent_file in torrent_files:
file_id = torrent_file.get("id")
file_name = torrent_file.get("name")
meta_info = MetaInfo(file_name)
if not meta_info.episode_list \
or not set(meta_info.episode_list).issubset(episodes):
file_ids.append(file_id)
else:
sucess_epidised = list(set(sucess_epidised).union(set(meta_info.episode_list)))
try:
for torrent_file in torrent_files:
file_id = torrent_file.get("id")
file_name = torrent_file.get("name")
meta_info = MetaInfo(file_name)
if not meta_info.episode_list \
or not set(meta_info.episode_list).issubset(episodes):
file_ids.append(file_id)
else:
sucess_epidised = list(set(sucess_epidised).union(set(meta_info.episode_list)))
finally:
torrent_files.clear()
del torrent_files
if sucess_epidised and file_ids:
# 选择文件
server.set_files(torrent_hash=torrent_hash, file_ids=file_ids, priority=0)
@@ -243,67 +251,79 @@ class QbittorrentModule(_ModuleBase, _DownloaderBase[Qbittorrent]):
if hashs:
# 按Hash获取
for name, server in servers.items():
torrents, _ = server.get_torrents(ids=hashs, tags=settings.TORRENT_TAG)
for torrent in torrents or []:
content_path = torrent.get("content_path")
if content_path:
torrent_path = Path(content_path)
else:
torrent_path = Path(torrent.get('save_path')) / torrent.get('name')
ret_torrents.append(TransferTorrent(
downloader=name,
title=torrent.get('name'),
path=torrent_path,
hash=torrent.get('hash'),
size=torrent.get('total_size'),
tags=torrent.get('tags'),
progress=torrent.get('progress') * 100,
state="paused" if torrent.get('state') in ("paused", "pausedDL") else "downloading",
))
torrents, _ = server.get_torrents(ids=hashs, tags=settings.TORRENT_TAG) or []
try:
for torrent in torrents:
content_path = torrent.get("content_path")
if content_path:
torrent_path = Path(content_path)
else:
torrent_path = Path(torrent.get('save_path')) / torrent.get('name')
ret_torrents.append(TransferTorrent(
downloader=name,
title=torrent.get('name'),
path=torrent_path,
hash=torrent.get('hash'),
size=torrent.get('total_size'),
tags=torrent.get('tags'),
progress=torrent.get('progress') * 100,
state="paused" if torrent.get('state') in ("paused", "pausedDL") else "downloading",
))
finally:
torrents.clear()
del torrents
elif status == TorrentStatus.TRANSFER:
# 获取已完成且未整理的
for name, server in servers.items():
torrents = server.get_completed_torrents(tags=settings.TORRENT_TAG)
for torrent in torrents or []:
tags = torrent.get("tags") or []
if "已整理" in tags:
continue
# 内容路径
content_path = torrent.get("content_path")
if content_path:
torrent_path = Path(content_path)
else:
torrent_path = torrent.get('save_path') / torrent.get('name')
ret_torrents.append(TransferTorrent(
downloader=name,
title=torrent.get('name'),
path=torrent_path,
hash=torrent.get('hash'),
tags=torrent.get('tags')
))
torrents = server.get_completed_torrents(tags=settings.TORRENT_TAG) or []
try:
for torrent in torrents:
tags = torrent.get("tags") or []
if "已整理" in tags:
continue
# 内容路径
content_path = torrent.get("content_path")
if content_path:
torrent_path = Path(content_path)
else:
torrent_path = torrent.get('save_path') / torrent.get('name')
ret_torrents.append(TransferTorrent(
downloader=name,
title=torrent.get('name'),
path=torrent_path,
hash=torrent.get('hash'),
tags=torrent.get('tags')
))
finally:
torrents.clear()
del torrents
elif status == TorrentStatus.DOWNLOADING:
# 获取正在下载的任务
for name, server in servers.items():
torrents = server.get_downloading_torrents(tags=settings.TORRENT_TAG)
for torrent in torrents or []:
meta = MetaInfo(torrent.get('name'))
ret_torrents.append(DownloadingTorrent(
downloader=name,
hash=torrent.get('hash'),
title=torrent.get('name'),
name=meta.name,
year=meta.year,
season_episode=meta.season_episode,
progress=torrent.get('progress') * 100,
size=torrent.get('total_size'),
state="paused" if torrent.get('state') in ("paused", "pausedDL") else "downloading",
dlspeed=StringUtils.str_filesize(torrent.get('dlspeed')),
upspeed=StringUtils.str_filesize(torrent.get('upspeed')),
left_time=StringUtils.str_secends(
(torrent.get('total_size') - torrent.get('completed')) / torrent.get(
'dlspeed')) if torrent.get(
'dlspeed') > 0 else ''
))
torrents = server.get_downloading_torrents(tags=settings.TORRENT_TAG) or []
try:
for torrent in torrents:
meta = MetaInfo(torrent.get('name'))
ret_torrents.append(DownloadingTorrent(
downloader=name,
hash=torrent.get('hash'),
title=torrent.get('name'),
name=meta.name,
year=meta.year,
season_episode=meta.season_episode,
progress=torrent.get('progress') * 100,
size=torrent.get('total_size'),
state="paused" if torrent.get('state') in ("paused", "pausedDL") else "downloading",
dlspeed=StringUtils.str_filesize(torrent.get('dlspeed')),
upspeed=StringUtils.str_filesize(torrent.get('upspeed')),
left_time=StringUtils.str_secends(
(torrent.get('total_size') - torrent.get('completed')) / torrent.get(
'dlspeed')) if torrent.get(
'dlspeed') > 0 else ''
))
finally:
torrents.clear()
del torrents
else:
return None
return ret_torrents # noqa

View File

@@ -12,16 +12,9 @@ from app.utils.string import StringUtils
class Qbittorrent:
_host: Optional[str] = None
_port: int = None
_username: Optional[str] = None
_password: Optional[str] = None
_category: Optional[bool] = False
_sequentail: Optional[bool] = False
_force_resume: Optional[bool] = False
qbc: Client = None
"""
qbittorrent下载器
"""
def __init__(self, host: Optional[str] = None, port: int = None,
username: Optional[str] = None, password: Optional[str] = None,
category: Optional[bool] = False, sequentail: Optional[bool] = False,
@@ -43,8 +36,7 @@ class Qbittorrent:
self._sequentail = sequentail
self._force_resume = force_resume
self._first_last_piece = first_last_piece
if self._host and self._port:
self.qbc = self.__login_qbittorrent()
self.qbc = self.__login_qbittorrent()
def is_inactive(self) -> bool:
"""
@@ -65,6 +57,8 @@ class Qbittorrent:
连接qbittorrent
:return: qbittorrent对象
"""
if not self._host or not self._port:
return None
try:
# 登录
logger.info(f"正在连接 qbittorrent{self._host}:{self._port}")
@@ -104,10 +98,14 @@ class Qbittorrent:
results = []
if not isinstance(tags, list):
tags = [tags]
for torrent in torrents:
torrent_tags = [str(tag).strip() for tag in torrent.get("tags").split(',')]
if set(tags).issubset(set(torrent_tags)):
results.append(torrent)
try:
for torrent in torrents:
torrent_tags = [str(tag).strip() for tag in torrent.get("tags").split(',')]
if set(tags).issubset(set(torrent_tags)):
results.append(torrent)
finally:
torrents.clear()
del torrents
return results, False
return torrents or [], False
except Exception as err:

View File

@@ -32,6 +32,7 @@ class SlackModule(_ModuleBase, _MessageBase[Slack]):
event_data: ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.Notifications.value]:
return
logger.info("配置变更重新加载Slack模块...")
self.init_module()
@staticmethod
@@ -81,8 +82,7 @@ class SlackModule(_ModuleBase, _MessageBase[Slack]):
def init_setting(self) -> Tuple[str, Union[str, bool]]:
pass
def message_parser(self, source: str, body: Any, form: Any,
args: Any) -> Optional[CommingMessage]:
def message_parser(self, source: str, body: Any, form: Any, args: Any) -> Optional[CommingMessage]:
"""
解析消息内容,返回字典,注意以下约定值:
userid: 用户ID
@@ -219,8 +219,32 @@ class SlackModule(_ModuleBase, _MessageBase[Slack]):
username = msg_json.get("user")
elif msg_json.get("type") == "block_actions":
userid = msg_json.get("user", {}).get("id")
text = msg_json.get("actions")[0].get("value")
callback_data = msg_json.get("actions")[0].get("value")
# 使用CALLBACK前缀标识按钮回调
text = f"CALLBACK:{callback_data}"
username = msg_json.get("user", {}).get("name")
# 获取原消息信息用于编辑
message_info = msg_json.get("message", {})
# Slack消息的时间戳作为消息ID
message_ts = message_info.get("ts")
channel_id = msg_json.get("channel", {}).get("id") or msg_json.get("container", {}).get("channel_id")
logger.info(f"收到来自 {client_config.name} 的Slack按钮回调"
f"userid={userid}, username={username}, callback_data={callback_data}")
# 创建包含回调信息的CommingMessage
return CommingMessage(
channel=MessageChannel.Slack,
source=client_config.name,
userid=userid,
username=username,
text=text,
is_callback=True,
callback_data=callback_data,
message_id=message_ts,
chat_id=channel_id
)
elif msg_json.get("type") == "event_callback":
userid = msg_json.get('event', {}).get('user')
text = re.sub(r"<@[0-9A-Z]+>", "", msg_json.get("event", {}).get("text"), flags=re.IGNORECASE).strip()
@@ -259,7 +283,10 @@ class SlackModule(_ModuleBase, _MessageBase[Slack]):
client: Slack = self.get_instance(conf.name)
if client:
client.send_msg(title=message.title, text=message.text,
image=message.image, userid=userid, link=message.link)
image=message.image, userid=userid, link=message.link,
buttons=message.buttons,
original_message_id=message.original_message_id,
original_chat_id=message.original_chat_id)
def post_medias_message(self, message: Notification, medias: List[MediaInfo]) -> None:
"""
@@ -273,7 +300,10 @@ class SlackModule(_ModuleBase, _MessageBase[Slack]):
continue
client: Slack = self.get_instance(conf.name)
if client:
client.send_medias_msg(title=message.title, medias=medias, userid=message.userid)
client.send_medias_msg(title=message.title, medias=medias, userid=message.userid,
buttons=message.buttons,
original_message_id=message.original_message_id,
original_chat_id=message.original_chat_id)
def post_torrents_message(self, message: Notification, torrents: List[Context]) -> None:
"""
@@ -288,4 +318,29 @@ class SlackModule(_ModuleBase, _MessageBase[Slack]):
client: Slack = self.get_instance(conf.name)
if client:
client.send_torrents_msg(title=message.title, torrents=torrents,
userid=message.userid)
userid=message.userid, buttons=message.buttons,
original_message_id=message.original_message_id,
original_chat_id=message.original_chat_id)
def delete_message(self, channel: MessageChannel, source: str,
message_id: str, chat_id: Optional[str] = None) -> bool:
"""
删除消息
:param channel: 消息渠道
:param source: 指定的消息源
:param message_id: 消息IDSlack中为时间戳
:param chat_id: 聊天ID频道ID
:return: 删除是否成功
"""
success = False
for conf in self.get_configs().values():
if channel != self._channel:
break
if source != conf.name:
continue
client: Slack = self.get_instance(conf.name)
if client:
result = client.delete_msg(message_id=message_id, chat_id=chat_id)
if result:
success = True
return success

View File

@@ -13,18 +13,16 @@ from app.core.metainfo import MetaInfo
from app.log import logger
from app.utils.string import StringUtils
lock = Lock()
class Slack:
_client: WebClient = None
_service: SocketModeHandler = None
_ds_url = f"http://127.0.0.1:{settings.PORT}/api/v1/message?token={settings.API_TOKEN}"
_channel = ""
def __init__(self, SLACK_OAUTH_TOKEN: Optional[str] = None, SLACK_APP_TOKEN: Optional[str] = None,
def __init__(self, SLACK_OAUTH_TOKEN: Optional[str] = None, SLACK_APP_TOKEN: Optional[str] = None,
SLACK_CHANNEL: Optional[str] = None, **kwargs):
if not SLACK_OAUTH_TOKEN or not SLACK_APP_TOKEN:
@@ -52,7 +50,7 @@ class Slack:
with requests.post(self._ds_url, json=message, timeout=10) as local_res:
logger.debug("message: %s processed, response is: %s" % (message, local_res.text))
@slack_app.action(re.compile(r"actionId-\d+"))
@slack_app.action(re.compile(r"actionId-.*"))
def slack_action(ack, body):
ack()
with requests.post(self._ds_url, json=body, timeout=60) as local_res:
@@ -101,15 +99,21 @@ class Slack:
"""
return True if self._client else False
def send_msg(self, title: str, text: Optional[str] = None, image: Optional[str] = None, link: Optional[str] = None, userid: Optional[str] = None):
def send_msg(self, title: str, text: Optional[str] = None,
image: Optional[str] = None, link: Optional[str] = None,
userid: Optional[str] = None, buttons: Optional[List[List[dict]]] = None,
original_message_id: Optional[str] = None,
original_chat_id: Optional[str] = None):
"""
发送Telegram消息
发送Slack消息
:param title: 消息标题
:param text: 消息内容
:param image: 消息图片地址
:param link: 点击消息转转的URL
:param userid: 用户ID如有则只发消息给该用户
:user_id: 发送消息的目标用户ID为空则发给管理员
:param buttons: 消息按钮列表,格式为 [[{"text": "按钮文本", "callback_data": "回调数据", "url": "链接"}]]
:param original_message_id: 原消息的时间戳,如果提供则编辑原消息
:param original_chat_id: 原消息的频道ID编辑消息时需要
"""
if not self._client:
return False, "消息客户端未就绪"
@@ -139,8 +143,42 @@ class Slack:
"image_url": f"{image}",
"alt_text": f"{title}"
}})
# 链接
if link:
# 自定义按钮
if buttons:
for button_row in buttons:
elements = []
for button in button_row:
if "url" in button:
# URL按钮
elements.append({
"type": "button",
"text": {
"type": "plain_text",
"text": button["text"],
"emoji": True
},
"url": button["url"],
"action_id": f"actionId-url-{button.get('text', 'url')}-{len(elements)}"
})
else:
# 回调按钮
elements.append({
"type": "button",
"text": {
"type": "plain_text",
"text": button["text"],
"emoji": True
},
"value": button["callback_data"],
"action_id": f"actionId-{button['callback_data']}"
})
if elements:
blocks.append({
"type": "actions",
"elements": elements
})
elif link:
# 默认链接按钮
blocks.append({
"type": "actions",
"elements": [
@@ -157,21 +195,41 @@ class Slack:
}
]
})
# 发送
result = self._client.chat_postMessage(
channel=channel,
text=message_text[:1000],
blocks=blocks,
mrkdwn=True
)
# 判断是编辑消息还是发送新消息
if original_message_id and original_chat_id:
# 编辑消息
result = self._client.chat_update(
channel=original_chat_id,
ts=original_message_id,
text=message_text[:1000],
blocks=blocks or []
)
else:
# 发送新消息
result = self._client.chat_postMessage(
channel=channel,
text=message_text[:1000],
blocks=blocks,
mrkdwn=True
)
return True, result
except Exception as msg_e:
logger.error(f"Slack消息发送失败: {msg_e}")
return False, str(msg_e)
def send_medias_msg(self, medias: List[MediaInfo], userid: Optional[str] = None, title: Optional[str] = None) -> Optional[bool]:
def send_medias_msg(self, medias: List[MediaInfo], userid: Optional[str] = None, title: Optional[str] = None,
buttons: Optional[List[List[dict]]] = None,
original_message_id: Optional[str] = None,
original_chat_id: Optional[str] = None) -> Optional[bool]:
"""
发送列表消息
发送媒体列表消息
:param medias: 媒体信息列表
:param userid: 用户ID如有则只发消息给该用户
:param title: 消息标题
:param buttons: 按钮列表,格式:[[{"text": "按钮文本", "callback_data": "回调数据"}]]
:param original_message_id: 原消息的时间戳,如果提供则编辑原消息
:param original_chat_id: 原消息的频道ID编辑消息时需要
"""
if not self._client:
return False
@@ -198,64 +256,148 @@ class Slack:
"type": "divider"
})
index = 1
for media in medias:
if media.get_poster_image():
if media.vote_star:
text = f"{index}. *<{media.detail_link}|{media.title_year}>*" \
f"\n类型:{media.type.value}" \
f"\n{media.vote_star}" \
f"\n{media.get_overview_string(50)}"
else:
text = f"{index}. *<{media.detail_link}|{media.title_year}>*" \
f"\n类型:{media.type.value}" \
f"\n{media.get_overview_string(50)}"
blocks.append(
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": text
},
"accessory": {
"type": "image",
"image_url": f"{media.get_poster_image()}",
"alt_text": f"{media.title_year}"
}
}
)
blocks.append(
{
"type": "actions",
"elements": [
{
"type": "button",
"text": {
"type": "plain_text",
"text": "选择",
"emoji": True
},
"value": f"{index}",
"action_id": f"actionId-{index}"
# 如果有自定义按钮,先添加所有媒体项,然后添加统一的按钮
if buttons:
# 添加媒体列表(不带单独的选择按钮)
for media in medias:
if media.get_poster_image():
if media.vote_star:
text = f"{index}. *<{media.detail_link}|{media.title_year}>*" \
f"\n类型:{media.type.value}" \
f"\n{media.vote_star}" \
f"\n{media.get_overview_string(50)}"
else:
text = f"{index}. *<{media.detail_link}|{media.title_year}>*" \
f"\n类型:{media.type.value}" \
f"\n{media.get_overview_string(50)}"
blocks.append(
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": text
},
"accessory": {
"type": "image",
"image_url": f"{media.get_poster_image()}",
"alt_text": f"{media.title_year}"
}
]
}
)
index += 1
# 发送
result = self._client.chat_postMessage(
channel=channel,
text=title,
blocks=blocks
)
}
)
index += 1
# 添加统一的自定义按钮(在所有媒体项之后)
for button_row in buttons:
elements = []
for button in button_row:
if "url" in button:
elements.append({
"type": "button",
"text": {
"type": "plain_text",
"text": button["text"],
"emoji": True
},
"url": button["url"],
"action_id": f"actionId-url-{button.get('text', 'url')}-{len(elements)}"
})
else:
elements.append({
"type": "button",
"text": {
"type": "plain_text",
"text": button["text"],
"emoji": True
},
"value": button["callback_data"],
"action_id": f"actionId-{button['callback_data']}"
})
if elements:
blocks.append({
"type": "actions",
"elements": elements
})
else:
# 使用默认的每个媒体项单独按钮
for media in medias:
if media.get_poster_image():
if media.vote_star:
text = f"{index}. *<{media.detail_link}|{media.title_year}>*" \
f"\n类型:{media.type.value}" \
f"\n{media.vote_star}" \
f"\n{media.get_overview_string(50)}"
else:
text = f"{index}. *<{media.detail_link}|{media.title_year}>*" \
f"\n类型:{media.type.value}" \
f"\n{media.get_overview_string(50)}"
blocks.append(
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": text
},
"accessory": {
"type": "image",
"image_url": f"{media.get_poster_image()}",
"alt_text": f"{media.title_year}"
}
}
)
# 使用默认选择按钮
blocks.append(
{
"type": "actions",
"elements": [
{
"type": "button",
"text": {
"type": "plain_text",
"text": "选择",
"emoji": True
},
"value": f"{index}",
"action_id": f"actionId-{index}"
}
]
}
)
index += 1
# 判断是编辑消息还是发送新消息
if original_message_id and original_chat_id:
# 编辑消息
result = self._client.chat_update(
channel=original_chat_id,
ts=original_message_id,
text=title,
blocks=blocks or []
)
else:
# 发送新消息
result = self._client.chat_postMessage(
channel=channel,
text=title,
blocks=blocks
)
return True if result else False
except Exception as msg_e:
logger.error(f"Slack消息发送失败: {msg_e}")
return False
def send_torrents_msg(self, torrents: List[Context],
userid: Optional[str] = None, title: Optional[str] = None) -> Optional[bool]:
def send_torrents_msg(self, torrents: List[Context], userid: Optional[str] = None, title: Optional[str] = None,
buttons: Optional[List[List[dict]]] = None,
original_message_id: Optional[str] = None,
original_chat_id: Optional[str] = None) -> Optional[bool]:
"""
发送列表消息
发送种子列表消息
:param torrents: 种子信息列表
:param userid: 用户ID如有则只发消息给该用户
:param title: 消息标题
:param buttons: 按钮列表,格式:[[{"text": "按钮文本", "callback_data": "回调数据"}]]
:param original_message_id: 原消息的时间戳,如果提供则编辑原消息
:param original_chat_id: 原消息的频道ID编辑消息时需要
"""
if not self._client:
return None
@@ -279,60 +421,172 @@ class Slack:
}]
# 列表
index = 1
for context in torrents:
torrent = context.torrent_info
site_name = torrent.site_name
meta = MetaInfo(torrent.title, torrent.description)
link = torrent.page_url
title = f"{meta.season_episode} " \
f"{meta.resource_term} " \
f"{meta.video_term} " \
f"{meta.release_group}"
title = re.sub(r"\s+", " ", title).strip()
free = torrent.volume_factor
seeder = f"{torrent.seeders}"
description = torrent.description
text = f"{index}. 【{site_name}】<{link}|{title}> " \
f"{StringUtils.str_filesize(torrent.size)} {free} {seeder}\n" \
f"{description}"
blocks.append(
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": text
# 如果有自定义按钮,先添加种子列表,然后添加统一的按钮
if buttons:
# 添加种子列表(不带单独的选择按钮)
for context in torrents:
torrent = context.torrent_info
site_name = torrent.site_name
meta = MetaInfo(torrent.title, torrent.description)
link = torrent.page_url
title_text = f"{meta.season_episode} " \
f"{meta.resource_term} " \
f"{meta.video_term} " \
f"{meta.release_group}"
title_text = re.sub(r"\s+", " ", title_text).strip()
free = torrent.volume_factor
seeder = f"{torrent.seeders}"
description = torrent.description
text = f"{index}. 【{site_name}】<{link}|{title_text}> " \
f"{StringUtils.str_filesize(torrent.size)} {free} {seeder}\n" \
f"{description}"
blocks.append(
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": text
}
}
}
)
blocks.append(
{
"type": "actions",
"elements": [
{
)
index += 1
# 添加统一的自定义按钮
for button_row in buttons:
elements = []
for button in button_row:
if "url" in button:
elements.append({
"type": "button",
"text": {
"type": "plain_text",
"text": "选择",
"text": button["text"],
"emoji": True
},
"value": f"{index}",
"action_id": f"actionId-{index}"
"url": button["url"],
"action_id": f"actionId-url-{button.get('text', 'url')}-{len(elements)}"
})
else:
elements.append({
"type": "button",
"text": {
"type": "plain_text",
"text": button["text"],
"emoji": True
},
"value": button["callback_data"],
"action_id": f"actionId-{button['callback_data']}"
})
if elements:
blocks.append({
"type": "actions",
"elements": elements
})
else:
# 使用默认的每个种子单独按钮
for context in torrents:
torrent = context.torrent_info
site_name = torrent.site_name
meta = MetaInfo(torrent.title, torrent.description)
link = torrent.page_url
title_text = f"{meta.season_episode} " \
f"{meta.resource_term} " \
f"{meta.video_term} " \
f"{meta.release_group}"
title_text = re.sub(r"\s+", " ", title_text).strip()
free = torrent.volume_factor
seeder = f"{torrent.seeders}"
description = torrent.description
text = f"{index}. 【{site_name}】<{link}|{title_text}> " \
f"{StringUtils.str_filesize(torrent.size)} {free} {seeder}\n" \
f"{description}"
blocks.append(
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": text
}
]
}
}
)
blocks.append(
{
"type": "actions",
"elements": [
{
"type": "button",
"text": {
"type": "plain_text",
"text": "选择",
"emoji": True
},
"value": f"{index}",
"action_id": f"actionId-{index}"
}
]
}
)
index += 1
# 判断是编辑消息还是发送新消息
if original_message_id and original_chat_id:
# 编辑消息
result = self._client.chat_update(
channel=original_chat_id,
ts=original_message_id,
text=title,
blocks=blocks or []
)
else:
# 发送新消息
result = self._client.chat_postMessage(
channel=channel,
text=title,
blocks=blocks
)
index += 1
# 发送
result = self._client.chat_postMessage(
channel=channel,
text=title,
blocks=blocks
)
return True if result else False
except Exception as msg_e:
logger.error(f"Slack消息发送失败: {msg_e}")
return False
def delete_msg(self, message_id: str, chat_id: Optional[str] = None) -> Optional[bool]:
"""
删除Slack消息
:param message_id: 消息时间戳Slack消息ID
:param chat_id: 频道ID
:return: 删除是否成功
"""
if not self._client:
return None
try:
# 确定要删除消息的频道ID
if chat_id:
target_channel = chat_id
else:
target_channel = self.__find_public_channel()
if not target_channel:
logger.error("无法确定要删除消息的Slack频道")
return False
# 删除消息
result = self._client.chat_delete(
channel=target_channel,
ts=message_id
)
if result.get("ok"):
logger.info(f"成功删除Slack消息: channel={target_channel}, ts={message_id}")
return True
else:
logger.error(f"删除Slack消息失败: {result.get('error', 'unknown error')}")
return False
except Exception as e:
logger.error(f"删除Slack消息异常: {str(e)}")
return False
def __find_public_channel(self):
"""
查找公共频道

View File

@@ -30,6 +30,7 @@ class SynologyChatModule(_ModuleBase, _MessageBase[SynologyChat]):
event_data: ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.Notifications.value]:
return
logger.info("配置变更重新加载SynologyChat模块...")
self.init_module()
@staticmethod

View File

@@ -9,7 +9,8 @@ from app.core.event import eventmanager
from app.log import logger
from app.modules import _ModuleBase, _MessageBase
from app.modules.telegram.telegram import Telegram
from app.schemas import MessageChannel, CommingMessage, Notification, CommandRegisterEventData, ConfigChangeEventData
from app.schemas import MessageChannel, CommingMessage, Notification, CommandRegisterEventData, ConfigChangeEventData, \
NotificationConf
from app.schemas.types import ModuleType, ChainEventType, SystemConfigKey, EventType
from app.utils.structures import DictUtils
@@ -35,6 +36,7 @@ class TelegramModule(_ModuleBase, _MessageBase[Telegram]):
event_data: ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.Notifications.value]:
return
logger.info("配置变更重新加载Telegram模块...")
self.init_module()
@staticmethod
@@ -98,6 +100,7 @@ class TelegramModule(_ModuleBase, _MessageBase[Telegram]):
:return: 渠道、消息体
"""
"""
普通消息格式:
{
'update_id': ,
'message': {
@@ -119,6 +122,16 @@ class TelegramModule(_ModuleBase, _MessageBase[Telegram]):
'text': ''
}
}
按钮回调格式:
{
'callback_query': {
'id': '',
'from': {...},
'message': {...},
'data': 'callback_data'
}
}
"""
# 获取服务配置
client_config = self.get_config(source)
@@ -130,32 +143,88 @@ class TelegramModule(_ModuleBase, _MessageBase[Telegram]):
except Exception as err:
logger.debug(f"解析Telegram消息失败{str(err)}")
return None
if message:
text = message.get("text")
user_id = message.get("from", {}).get("id")
# 获取用户名
user_name = message.get("from", {}).get("username")
if text:
logger.info(f"收到来自 {client_config.name} 的Telegram消息"
f"userid={user_id}, username={user_name}, text={text}")
# 检查权限
admin_users = client_config.config.get("TELEGRAM_ADMINS")
user_list = client_config.config.get("TELEGRAM_USERS")
chat_id = client_config.config.get("TELEGRAM_CHAT_ID")
if text.startswith("/"):
if admin_users \
and str(user_id) not in admin_users.split(',') \
and str(user_id) != chat_id:
client.send_msg(title="只有管理员才有权限执行此命令", userid=user_id)
return None
else:
if user_list \
and not str(user_id) in user_list.split(','):
logger.info(f"用户{user_id}不在用户白名单中,无法使用此机器人")
client.send_msg(title="你不在用户白名单中,无法使用此机器人", userid=user_id)
return None
return CommingMessage(channel=MessageChannel.Telegram, source=client_config.name,
userid=user_id, username=user_name, text=text)
# 处理按钮回调
if "callback_query" in message:
return self._handle_callback_query(message, client_config)
# 处理普通消息
return self._handle_text_message(message, client_config, client)
return None
@staticmethod
def _handle_callback_query(message: dict, client_config: NotificationConf) -> Optional[CommingMessage]:
"""
处理按钮回调查询
"""
callback_query = message.get("callback_query", {})
user_info = callback_query.get("from", {})
callback_data = callback_query.get("data", "")
user_id = user_info.get("id")
user_name = user_info.get("username")
if callback_data and user_id:
logger.info(f"收到来自 {client_config.name} 的Telegram按钮回调"
f"userid={user_id}, username={user_name}, callback_data={callback_data}")
# 将callback_data作为特殊格式的text返回以便主程序识别这是按钮回调
callback_text = f"CALLBACK:{callback_data}"
# 创建包含完整回调信息的CommingMessage
return CommingMessage(
channel=MessageChannel.Telegram,
source=client_config.name,
userid=user_id,
username=user_name,
text=callback_text,
is_callback=True,
callback_data=callback_data,
message_id=callback_query.get("message", {}).get("message_id"),
chat_id=str(callback_query.get("message", {}).get("chat", {}).get("id", "")),
callback_query=callback_query
)
return None
@staticmethod
def _handle_text_message(msg: dict, client_config: NotificationConf, client: Telegram) -> Optional[CommingMessage]:
"""
处理普通文本消息
"""
text = msg.get("text")
user_id = msg.get("from", {}).get("id")
user_name = msg.get("from", {}).get("username")
if text and user_id:
logger.info(f"收到来自 {client_config.name} 的Telegram消息"
f"userid={user_id}, username={user_name}, text={text}")
# 检查权限
admin_users = client_config.config.get("TELEGRAM_ADMINS")
user_list = client_config.config.get("TELEGRAM_USERS")
chat_id = client_config.config.get("TELEGRAM_CHAT_ID")
if text.startswith("/"):
if admin_users \
and str(user_id) not in admin_users.split(',') \
and str(user_id) != chat_id:
client.send_msg(title="只有管理员才有权限执行此命令", userid=user_id)
return None
else:
if user_list \
and str(user_id) not in user_list.split(','):
logger.info(f"用户{user_id}不在用户白名单中,无法使用此机器人")
client.send_msg(title="你不在用户白名单中,无法使用此机器人", userid=user_id)
return None
return CommingMessage(
channel=MessageChannel.Telegram,
source=client_config.name,
userid=user_id,
username=user_name,
text=text
)
return None
def post_message(self, message: Notification) -> None:
@@ -177,7 +246,10 @@ class TelegramModule(_ModuleBase, _MessageBase[Telegram]):
client: Telegram = self.get_instance(conf.name)
if client:
client.send_msg(title=message.title, text=message.text,
image=message.image, userid=userid, link=message.link)
image=message.image, userid=userid, link=message.link,
buttons=message.buttons,
original_message_id=message.original_message_id,
original_chat_id=message.original_chat_id)
def post_medias_message(self, message: Notification, medias: List[MediaInfo]) -> None:
"""
@@ -192,7 +264,10 @@ class TelegramModule(_ModuleBase, _MessageBase[Telegram]):
client: Telegram = self.get_instance(conf.name)
if client:
client.send_medias_msg(title=message.title, medias=medias,
userid=message.userid, link=message.link)
userid=message.userid, link=message.link,
buttons=message.buttons,
original_message_id=message.original_message_id,
original_chat_id=message.original_chat_id)
def post_torrents_message(self, message: Notification, torrents: List[Context]) -> None:
"""
@@ -207,7 +282,33 @@ class TelegramModule(_ModuleBase, _MessageBase[Telegram]):
client: Telegram = self.get_instance(conf.name)
if client:
client.send_torrents_msg(title=message.title, torrents=torrents,
userid=message.userid, link=message.link)
userid=message.userid, link=message.link,
buttons=message.buttons,
original_message_id=message.original_message_id,
original_chat_id=message.original_chat_id)
def delete_message(self, channel: MessageChannel, source: str,
message_id: int, chat_id: Optional[int] = None) -> bool:
"""
删除消息
:param channel: 消息渠道
:param source: 指定的消息源
:param message_id: 消息ID
:param chat_id: 聊天ID
:return: 删除是否成功
"""
success = False
for conf in self.get_configs().values():
if channel != self._channel:
break
if source != conf.name:
continue
client: Telegram = self.get_instance(conf.name)
if client:
result = client.delete_msg(message_id=message_id, chat_id=chat_id)
if result:
success = True
return success
def register_commands(self, commands: Dict[str, dict]):
"""

View File

@@ -3,12 +3,13 @@ import threading
import uuid
from pathlib import Path
from threading import Event
from typing import Optional, List, Dict
from typing import Optional, List, Dict, Callable
from urllib.parse import urljoin
import telebot
from telebot import apihelper
from telebot.types import InputFile
from telebot.types import InputFile, InlineKeyboardMarkup, InlineKeyboardButton
from telebot.types import InputMediaPhoto
from app.core.config import settings
from app.core.context import MediaInfo, Context
@@ -23,6 +24,7 @@ class Telegram:
_ds_url = f"http://127.0.0.1:{settings.PORT}/api/v1/message?token={settings.API_TOKEN}"
_event = Event()
_bot: telebot.TeleBot = None
_callback_handlers: Dict[str, Callable] = {} # 存储回调处理器
def __init__(self, TELEGRAM_TOKEN: Optional[str] = None, TELEGRAM_CHAT_ID: Optional[str] = None, **kwargs):
"""
@@ -57,7 +59,44 @@ class Telegram:
@_bot.message_handler(func=lambda message: True)
def echo_all(message):
RequestUtils(timeout=5).post_res(self._ds_url, json=message.json)
RequestUtils(timeout=15).post_res(self._ds_url, json=message.json)
@_bot.callback_query_handler(func=lambda call: True)
def callback_query(call):
"""
处理按钮点击回调
"""
try:
# 解析回调数据
callback_data = call.data
user_id = str(call.from_user.id)
logger.info(f"收到按钮回调:{callback_data},用户:{user_id}")
# 发送回调数据给主程序处理
callback_json = {
"callback_query": {
"id": call.id,
"from": call.from_user.to_dict(),
"message": {
"message_id": call.message.message_id,
"chat": {
"id": call.message.chat.id,
}
},
"data": callback_data
}
}
# 先确认回调避免用户看到loading状态
_bot.answer_callback_query(call.id)
# 发送给主程序处理
RequestUtils(timeout=15).post_res(self._ds_url, json=callback_json)
except Exception as e:
logger.error(f"处理按钮回调失败:{str(e)}")
_bot.answer_callback_query(call.id, "处理失败,请重试")
def run_polling():
"""
@@ -80,7 +119,10 @@ class Telegram:
return self._bot is not None
def send_msg(self, title: str, text: Optional[str] = None, image: Optional[str] = None,
userid: Optional[str] = None, link: Optional[str] = None) -> Optional[bool]:
userid: Optional[str] = None, link: Optional[str] = None,
buttons: Optional[List[List[dict]]] = None,
original_message_id: Optional[int] = None,
original_chat_id: Optional[str] = None) -> Optional[bool]:
"""
发送Telegram消息
:param title: 消息标题
@@ -88,6 +130,9 @@ class Telegram:
:param image: 消息图片地址
:param userid: 用户ID如有则只发消息给该用户
:param link: 跳转链接
:param buttons: 按钮列表,格式:[[{"text": "按钮文本", "callback_data": "回调数据"}]]
:param original_message_id: 原消息ID如果提供则编辑原消息
:param original_chat_id: 原消息的聊天ID编辑消息时需要
:userid: 发送消息的目标用户ID为空则发给管理员
"""
if not self._telegram_token or not self._telegram_chat_id:
@@ -113,16 +158,37 @@ class Telegram:
else:
chat_id = self._telegram_chat_id
return self.__send_request(userid=chat_id, image=image, caption=caption)
# 创建按钮键盘
reply_markup = None
if buttons:
reply_markup = self._create_inline_keyboard(buttons)
# 判断是编辑消息还是发送新消息
if original_message_id and original_chat_id:
# 编辑消息
return self.__edit_message(original_chat_id, original_message_id, caption, buttons, image)
else:
# 发送新消息
return self.__send_request(userid=chat_id, image=image, caption=caption, reply_markup=reply_markup)
except Exception as msg_e:
logger.error(f"发送消息失败:{msg_e}")
return False
def send_medias_msg(self, medias: List[MediaInfo], userid: Optional[str] = None,
title: Optional[str] = None, link: Optional[str] = None) -> Optional[bool]:
title: Optional[str] = None, link: Optional[str] = None,
buttons: Optional[List[List[Dict]]] = None,
original_message_id: Optional[int] = None,
original_chat_id: Optional[str] = None) -> Optional[bool]:
"""
发送媒体列表消息
:param medias: 媒体信息列表
:param userid: 用户ID如有则只发消息给该用户
:param title: 消息标题
:param link: 跳转链接
:param buttons: 按钮列表,格式:[[{"text": "按钮文本", "callback_data": "回调数据"}]]
:param original_message_id: 原消息ID如果提供则编辑原消息
:param original_chat_id: 原消息的聊天ID编辑消息时需要
"""
if not self._telegram_token or not self._telegram_chat_id:
return None
@@ -155,7 +221,18 @@ class Telegram:
else:
chat_id = self._telegram_chat_id
return self.__send_request(userid=chat_id, image=image, caption=caption)
# 创建按钮键盘
reply_markup = None
if buttons:
reply_markup = self._create_inline_keyboard(buttons)
# 判断是编辑消息还是发送新消息
if original_message_id and original_chat_id:
# 编辑消息
return self.__edit_message(original_chat_id, original_message_id, caption, buttons, image)
else:
# 发送新消息
return self.__send_request(userid=chat_id, image=image, caption=caption, reply_markup=reply_markup)
except Exception as msg_e:
logger.error(f"发送消息失败:{msg_e}")
@@ -163,19 +240,25 @@ class Telegram:
def send_torrents_msg(self, torrents: List[Context],
userid: Optional[str] = None, title: Optional[str] = None,
link: Optional[str] = None) -> Optional[bool]:
link: Optional[str] = None, buttons: Optional[List[List[Dict]]] = None,
original_message_id: Optional[int] = None,
original_chat_id: Optional[str] = None) -> Optional[bool]:
"""
发送列表消息
发送种子列表消息
:param torrents: 种子信息列表
:param userid: 用户ID如有则只发消息给该用户
:param title: 消息标题
:param link: 跳转链接
:param buttons: 按钮列表,格式:[[{"text": "按钮文本", "callback_data": "回调数据"}]]
:param original_message_id: 原消息ID如果提供则编辑原消息
:param original_chat_id: 原消息的聊天ID编辑消息时需要
"""
if not self._telegram_token or not self._telegram_chat_id:
return None
if not torrents:
return False
try:
index, caption = 1, "*%s*" % title
mediainfo = torrents[0].media_info
image = torrents[0].media_info.get_message_image()
for context in torrents:
torrent = context.torrent_info
site_name = torrent.site_name
@@ -200,20 +283,142 @@ class Telegram:
else:
chat_id = self._telegram_chat_id
return self.__send_request(userid=chat_id, caption=caption,
image=mediainfo.get_message_image())
# 创建按钮键盘
reply_markup = None
if buttons:
reply_markup = self._create_inline_keyboard(buttons)
# 判断是编辑消息还是发送新消息
if original_message_id and original_chat_id:
# 编辑消息(种子消息通常没有图片)
return self.__edit_message(original_chat_id, original_message_id, caption, buttons, image)
else:
# 发送新消息
return self.__send_request(userid=chat_id, image=image, caption=caption, reply_markup=reply_markup)
except Exception as msg_e:
logger.error(f"发送消息失败:{msg_e}")
return False
@staticmethod
def _create_inline_keyboard(buttons: List[List[Dict]]) -> InlineKeyboardMarkup:
"""
创建内联键盘
:param buttons: 按钮配置,格式:[[{"text": "按钮文本", "callback_data": "回调数据", "url": "链接"}]]
:return: InlineKeyboardMarkup对象
"""
keyboard = []
for row in buttons:
button_row = []
for button in row:
if "url" in button:
# URL按钮
btn = InlineKeyboardButton(text=button["text"], url=button["url"])
else:
# 回调按钮
btn = InlineKeyboardButton(text=button["text"], callback_data=button["callback_data"])
button_row.append(btn)
keyboard.append(button_row)
return InlineKeyboardMarkup(keyboard)
def answer_callback_query(self, callback_query_id: int, text: Optional[str] = None,
show_alert: bool = False) -> Optional[bool]:
"""
回应回调查询
"""
if not self._bot:
return None
try:
self._bot.answer_callback_query(callback_query_id, text=text, show_alert=show_alert)
return True
except Exception as e:
logger.error(f"回应回调查询失败:{str(e)}")
return False
def delete_msg(self, message_id: int, chat_id: Optional[int] = None) -> Optional[bool]:
"""
删除Telegram消息
:param message_id: 消息ID
:param chat_id: 聊天ID
:return: 删除是否成功
"""
if not self._telegram_token or not self._telegram_chat_id:
return None
try:
# 确定要删除消息的聊天ID
if chat_id:
target_chat_id = chat_id
else:
target_chat_id = self._telegram_chat_id
# 删除消息
result = self._bot.delete_message(chat_id=target_chat_id, message_id=int(message_id))
if result:
logger.info(f"成功删除Telegram消息: chat_id={target_chat_id}, message_id={message_id}")
return True
else:
logger.error(f"删除Telegram消息失败: chat_id={target_chat_id}, message_id={message_id}")
return False
except Exception as e:
logger.error(f"删除Telegram消息异常: {str(e)}")
return False
def __edit_message(self, chat_id: str, message_id: int, text: str,
buttons: Optional[List[List[dict]]] = None,
image: Optional[str] = None) -> Optional[bool]:
"""
编辑已发送的消息
:param chat_id: 聊天ID
:param message_id: 消息ID
:param text: 新的消息内容
:param buttons: 按钮列表
:param image: 图片URL或路径
:return: 编辑是否成功
"""
if not self._bot:
return None
try:
# 创建按钮键盘
reply_markup = None
if buttons:
reply_markup = self._create_inline_keyboard(buttons)
if image:
# 如果有图片使用edit_message_media
media = InputMediaPhoto(media=image, caption=text, parse_mode="Markdown")
self._bot.edit_message_media(
chat_id=chat_id,
message_id=message_id,
media=media,
reply_markup=reply_markup
)
else:
# 如果没有图片使用edit_message_text
self._bot.edit_message_text(
chat_id=chat_id,
message_id=message_id,
text=text,
parse_mode="Markdown",
reply_markup=reply_markup
)
return True
except Exception as e:
logger.error(f"编辑消息失败:{str(e)}")
return False
@retry(Exception, logger=logger)
def __send_request(self, userid: Optional[str] = None, image="", caption="") -> bool:
def __send_request(self, userid: Optional[str] = None, image="", caption="",
reply_markup: Optional[InlineKeyboardMarkup] = None) -> bool:
"""
向Telegram发送报文
:param reply_markup: 内联键盘
"""
if image:
res = RequestUtils(proxies=settings.PROXY).get_res(image)
res = RequestUtils(proxies=settings.PROXY, ua=settings.USER_AGENT).get_res(image)
if res is None:
raise Exception("获取图片失败")
if res.content:
@@ -227,7 +432,8 @@ class Telegram:
ret = self._bot.send_photo(chat_id=userid or self._telegram_chat_id,
photo=photo,
caption=caption,
parse_mode="Markdown")
parse_mode="Markdown",
reply_markup=reply_markup)
if ret is None:
raise Exception("发送图片消息失败")
return True
@@ -237,11 +443,13 @@ class Telegram:
for i in range(0, len(caption), 4095):
ret = self._bot.send_message(chat_id=userid or self._telegram_chat_id,
text=caption[i:i + 4095],
parse_mode="Markdown")
parse_mode="Markdown",
reply_markup=reply_markup if i == 0 else None)
else:
ret = self._bot.send_message(chat_id=userid or self._telegram_chat_id,
text=caption,
parse_mode="Markdown")
parse_mode="Markdown",
reply_markup=reply_markup)
if ret is None:
raise Exception("发送文本消息失败")
return True if ret else False

View File

@@ -15,7 +15,7 @@ from app.schemas.types import MediaType
lock = RLock()
CACHE_EXPIRE_TIMESTAMP_STR = "cache_expire_timestamp"
EXPIRE_TIMESTAMP = settings.CONF["meta"]
EXPIRE_TIMESTAMP = settings.CONF.meta
class TmdbCache(metaclass=Singleton):

View File

@@ -500,7 +500,7 @@ class TmdbApi:
return ret_info
@cached(maxsize=settings.CONF["tmdb"], ttl=settings.CONF["meta"])
@cached(maxsize=settings.CONF.tmdb, ttl=settings.CONF.meta)
@rate_limit_exponential(source="match_tmdb_web", base_wait=5, max_wait=1800, enable_logging=True)
def match_web(self, name: str, mtype: MediaType) -> Optional[dict]:
"""

View File

@@ -1,12 +1,9 @@
from app.core.cache import cached
from ..tmdb import TMDb
class Trending(TMDb):
_urls = {"trending": "/trending/%s/%s"}
@cached(maxsize=1024, ttl=43200)
def _trending(self, media_type="all", time_window="day", page=1):
"""
Get trending, TTLCache 12 hours

View File

@@ -124,7 +124,7 @@ class TMDb(object):
def cache(self, cache):
self._cache_enabled = bool(cache)
@cached(maxsize=settings.CONF["tmdb"], ttl=settings.CONF["meta"])
@cached(maxsize=settings.CONF.tmdb, ttl=settings.CONF.meta)
def cached_request(self, method, url, data, json,
_ts=datetime.strftime(datetime.now(), '%Y%m%d')):
"""

View File

@@ -36,6 +36,7 @@ class TransmissionModule(_ModuleBase, _DownloaderBase[Transmission]):
event_data: schemas.ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.Downloaders.value]:
return
logger.info("配置变更重新加载Transmission模块...")
self.init_module()
@staticmethod
@@ -162,24 +163,28 @@ class TransmissionModule(_ModuleBase, _DownloaderBase[Transmission]):
if error:
return None, None, None, "无法连接transmission下载器"
if torrents:
for torrent in torrents:
# 名称与大小相等则认为是同一个种子
if torrent.name == torrent_name and torrent.total_size == torrent_size:
torrent_hash = torrent.hashString
logger.warn(f"下载器中已存在该种子任务:{torrent_hash} - {torrent.name}")
# 给种子打上标签
if settings.TORRENT_TAG:
logger.info(f"给种子 {torrent_hash} 打上标签:{settings.TORRENT_TAG}")
# 种子标签
labels = [str(tag).strip()
for tag in torrent.labels] if hasattr(torrent, "labels") else []
if "已整理" in labels:
labels.remove("已整理")
server.set_torrent_tag(ids=torrent_hash, tags=labels)
if settings.TORRENT_TAG and settings.TORRENT_TAG not in labels:
labels.append(settings.TORRENT_TAG)
server.set_torrent_tag(ids=torrent_hash, tags=labels)
return downloader or self.get_default_config_name(), torrent_hash, torrent_layout, f"下载任务已存在"
try:
for torrent in torrents:
# 名称与大小相等则认为是同一个种子
if torrent.name == torrent_name and torrent.total_size == torrent_size:
torrent_hash = torrent.hashString
logger.warn(f"下载器中已存在该种子任务:{torrent_hash} - {torrent.name}")
# 给种子打上标签
if settings.TORRENT_TAG:
logger.info(f"给种子 {torrent_hash} 打上标签:{settings.TORRENT_TAG}")
# 种子标签
labels = [str(tag).strip()
for tag in torrent.labels] if hasattr(torrent, "labels") else []
if "已整理" in labels:
labels.remove("已整理")
server.set_torrent_tag(ids=torrent_hash, tags=labels)
if settings.TORRENT_TAG and settings.TORRENT_TAG not in labels:
labels.append(settings.TORRENT_TAG)
server.set_torrent_tag(ids=torrent_hash, tags=labels)
return downloader or self.get_default_config_name(), torrent_hash, torrent_layout, f"下载任务已存在"
finally:
torrents.clear()
del torrents
return None, None, None, f"添加种子任务失败:{content}"
else:
torrent_hash = torrent.hashString
@@ -191,23 +196,27 @@ class TransmissionModule(_ModuleBase, _DownloaderBase[Transmission]):
# 需要的文件信息
file_ids = []
unwanted_file_ids = []
for torrent_file in torrent_files:
file_id = torrent_file.id
file_name = torrent_file.name
meta_info = MetaInfo(file_name)
if not meta_info.episode_list:
unwanted_file_ids.append(file_id)
continue
selected = set(meta_info.episode_list).issubset(set(episodes))
if not selected:
unwanted_file_ids.append(file_id)
continue
file_ids.append(file_id)
# 选择文件
server.set_files(torrent_hash, file_ids)
server.set_unwanted_files(torrent_hash, unwanted_file_ids)
# 开始任务
server.start_torrents(torrent_hash)
try:
for torrent_file in torrent_files:
file_id = torrent_file.id
file_name = torrent_file.name
meta_info = MetaInfo(file_name)
if not meta_info.episode_list:
unwanted_file_ids.append(file_id)
continue
selected = set(meta_info.episode_list).issubset(set(episodes))
if not selected:
unwanted_file_ids.append(file_id)
continue
file_ids.append(file_id)
# 选择文件
server.set_files(torrent_hash, file_ids)
server.set_unwanted_files(torrent_hash, unwanted_file_ids)
# 开始任务
server.start_torrents(torrent_hash)
finally:
torrent_files.clear()
del torrent_files
return downloader or self.get_default_config_name(), torrent_hash, torrent_layout, "添加下载任务成功"
else:
return downloader or self.get_default_config_name(), torrent_hash, torrent_layout, "添加下载任务成功"
@@ -235,61 +244,73 @@ class TransmissionModule(_ModuleBase, _DownloaderBase[Transmission]):
if hashs:
# 按Hash获取
for name, server in servers.items():
torrents, _ = server.get_torrents(ids=hashs, tags=settings.TORRENT_TAG)
for torrent in torrents or []:
ret_torrents.append(TransferTorrent(
downloader=name,
title=torrent.name,
path=Path(torrent.download_dir) / torrent.name,
hash=torrent.hashString,
size=torrent.total_size,
tags=",".join(torrent.labels or [])
))
torrents, _ = server.get_torrents(ids=hashs, tags=settings.TORRENT_TAG) or []
try:
for torrent in torrents:
ret_torrents.append(TransferTorrent(
downloader=name,
title=torrent.name,
path=Path(torrent.download_dir) / torrent.name,
hash=torrent.hashString,
size=torrent.total_size,
tags=",".join(torrent.labels or [])
))
finally:
torrents.clear()
del torrents
elif status == TorrentStatus.TRANSFER:
# 获取已完成且未整理的
for name, server in servers.items():
torrents = server.get_completed_torrents(tags=settings.TORRENT_TAG)
for torrent in torrents or []:
# 含"已整理"tag的不处理
if "已整理" in torrent.labels or []:
continue
# 下载路径
path = torrent.download_dir
# 无法获取下载路径的不处理
if not path:
logger.debug(f"未获取到 {torrent.name} 下载保存路径")
continue
ret_torrents.append(TransferTorrent(
downloader=name,
title=torrent.name,
path=Path(torrent.download_dir) / torrent.name,
hash=torrent.hashString,
tags=",".join(torrent.labels or []),
progress=torrent.progress,
state="paused" if torrent.status == "stopped" else "downloading",
))
torrents = server.get_completed_torrents(tags=settings.TORRENT_TAG) or []
try:
for torrent in torrents:
# 含"已整理"tag的不处理
if "已整理" in torrent.labels or []:
continue
# 下载路径
path = torrent.download_dir
# 无法获取下载路径的不处理
if not path:
logger.debug(f"未获取到 {torrent.name} 下载保存路径")
continue
ret_torrents.append(TransferTorrent(
downloader=name,
title=torrent.name,
path=Path(torrent.download_dir) / torrent.name,
hash=torrent.hashString,
tags=",".join(torrent.labels or []),
progress=torrent.progress,
state="paused" if torrent.status == "stopped" else "downloading",
))
finally:
torrents.clear()
del torrents
elif status == TorrentStatus.DOWNLOADING:
# 获取正在下载的任务
for name, server in servers.items():
torrents = server.get_downloading_torrents(tags=settings.TORRENT_TAG)
for torrent in torrents or []:
meta = MetaInfo(torrent.name)
dlspeed = torrent.rate_download if hasattr(torrent, "rate_download") else torrent.rateDownload
upspeed = torrent.rate_upload if hasattr(torrent, "rate_upload") else torrent.rateUpload
ret_torrents.append(DownloadingTorrent(
downloader=name,
hash=torrent.hashString,
title=torrent.name,
name=meta.name,
year=meta.year,
season_episode=meta.season_episode,
progress=torrent.progress,
size=torrent.total_size,
state="paused" if torrent.status == "stopped" else "downloading",
dlspeed=StringUtils.str_filesize(dlspeed),
upspeed=StringUtils.str_filesize(upspeed),
left_time=StringUtils.str_secends(torrent.left_until_done / dlspeed) if dlspeed > 0 else ''
))
torrents = server.get_downloading_torrents(tags=settings.TORRENT_TAG) or []
try:
for torrent in torrents:
meta = MetaInfo(torrent.name)
dlspeed = torrent.rate_download if hasattr(torrent, "rate_download") else torrent.rateDownload
upspeed = torrent.rate_upload if hasattr(torrent, "rate_upload") else torrent.rateUpload
ret_torrents.append(DownloadingTorrent(
downloader=name,
hash=torrent.hashString,
title=torrent.name,
name=meta.name,
year=meta.year,
season_episode=meta.season_episode,
progress=torrent.progress,
size=torrent.total_size,
state="paused" if torrent.status == "stopped" else "downloading",
dlspeed=StringUtils.str_filesize(dlspeed),
upspeed=StringUtils.str_filesize(upspeed),
left_time=StringUtils.str_secends(torrent.left_until_done / dlspeed) if dlspeed > 0 else ''
))
finally:
torrents.clear()
del torrents
else:
return None
return ret_torrents # noqa

View File

@@ -1,4 +1,4 @@
from typing import Optional, Union, Tuple, List, Literal
from typing import Optional, Union, Tuple, List
import transmission_rpc
from transmission_rpc import Client, Torrent, File
@@ -9,14 +9,9 @@ from app.utils.url import UrlUtils
class Transmission:
_protocol: Literal["http", "https"] = "http"
_host: Optional[str] = None
_port: Optional[int] = None
_username: Optional[str] = None
_password: Optional[str] = None
trc: Optional[Client] = None
"""
Transmission下载器
"""
# 参考transmission web仅查询需要的参数加速种子搜索
_trarg = ["id", "name", "status", "labels", "hashString", "totalSize", "percentDone", "addedDate", "trackerList",
"trackerStats",
@@ -43,18 +38,19 @@ class Transmission:
return
self._username = username
self._password = password
if self._host and self._port:
self.trc = self.__login_transmission()
self.trc = self.__login_transmission()
def __login_transmission(self) -> Optional[Client]:
"""
连接transmission
:return: transmission对象
"""
if not self._host or not self._port:
return None
try:
# 登录
logger.info(f"正在连接 transmission{self._protocol}://{self._host}:{self._port}")
trt = transmission_rpc.Client(protocol=self._protocol,
trt = transmission_rpc.Client(protocol=self._protocol, # noqa
host=self._host,
port=self._port,
username=self._username,
@@ -97,16 +93,20 @@ class Transmission:
if tags and not isinstance(tags, list):
tags = [tags]
ret_torrents = []
for torrent in torrents:
# 状态过滤
if status and torrent.status not in status:
continue
# 种子标签
labels = [str(tag).strip()
for tag in torrent.labels] if hasattr(torrent, "labels") else []
if tags and not set(tags).issubset(set(labels)):
continue
ret_torrents.append(torrent)
try:
for torrent in torrents:
# 状态过滤
if status and torrent.status not in status:
continue
# 种子标签
labels = [str(tag).strip()
for tag in torrent.labels] if hasattr(torrent, "labels") else []
if tags and not set(tags).issubset(set(labels)):
continue
ret_torrents.append(torrent)
finally:
torrents.clear()
del torrents
return ret_torrents, False
def get_completed_torrents(self, ids: Union[str, list] = None,
@@ -134,7 +134,7 @@ class Transmission:
return None
try:
torrents, error = self.get_torrents(ids=ids,
status=["downloading", "download_pending", "stopped"],
status=["downloading", "download_pending"],
tags=tags)
return None if error else torrents or []
except Exception as err:

View File

@@ -34,6 +34,7 @@ class TrimeMediaModule(_ModuleBase, _MediaServerBase[TrimeMedia]):
event_data: schemas.ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.MediaServers.value]:
return
logger.info("配置变更,重新加载飞牛影视模块...")
self.init_module()
@staticmethod

View File

@@ -31,6 +31,7 @@ class VoceChatModule(_ModuleBase, _MessageBase[VoceChat]):
event_data: ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.Notifications.value]:
return
logger.info("配置变更重新加载VoceChat模块...")
self.init_module()
@staticmethod

View File

@@ -31,6 +31,7 @@ class WebPushModule(_ModuleBase, _MessageBase):
event_data: ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.Notifications.value]:
return
logger.info("配置变更重新加载WebPush模块...")
self.init_module()
@staticmethod

View File

@@ -35,6 +35,7 @@ class WechatModule(_ModuleBase, _MessageBase[WeChat]):
event_data: ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.Notifications.value]:
return
logger.info("配置变更重新加载Wechat模块...")
self.init_module()
@staticmethod

View File

@@ -91,6 +91,7 @@ class Monitor(metaclass=Singleton):
event_data: ConfigChangeEventData = event.event_data
if event_data.key not in [SystemConfigKey.Directories.value]:
return
logger.info("配置变更事件触发,重新初始化目录监控...")
self.init()
def init(self):

View File

@@ -68,6 +68,7 @@ class Scheduler(metaclass=Singleton):
if event_data.key not in ['DEV', 'COOKIECLOUD_INTERVAL', 'MEDIASERVER_SYNC_INTERVAL', 'SUBSCRIBE_SEARCH',
'SUBSCRIBE_MODE', 'SUBSCRIBE_RSS_INTERVAL', 'SITEDATA_REFRESH_INTERVAL']:
return
logger.info(f"配置项 {event_data.key} 变更,重新初始化定时服务...")
self.init()
def init(self):
@@ -166,7 +167,7 @@ class Scheduler(metaclass=Singleton):
# 创建定时服务
self._scheduler = BackgroundScheduler(timezone=settings.TZ,
executors={
'default': ThreadPoolExecutor(settings.CONF['scheduler'])
'default': ThreadPoolExecutor(settings.CONF.scheduler)
})
# CookieCloud定时同步
@@ -323,7 +324,7 @@ class Scheduler(metaclass=Singleton):
"interval",
id="clear_cache",
name="缓存清理",
hours=settings.CONF["meta"] / 3600,
hours=settings.CONF.meta / 3600,
kwargs={
'job_id': 'clear_cache'
}

View File

@@ -160,7 +160,7 @@ class WebhookEventInfo(BaseModel):
save_reason: Optional[str] = None
item_isvirtual: Optional[bool] = None
media_type: Optional[str] = None
json_object: Optional[dict] = {}
json_object: Optional[dict] = Field(default_factory=dict)
class MediaServerPlayItem(BaseModel):

View File

@@ -1,4 +1,6 @@
from typing import Optional, Union
from dataclasses import dataclass
from enum import Enum
from typing import Optional, Union, List, Dict, Set
from pydantic import BaseModel, Field
@@ -23,6 +25,16 @@ class CommingMessage(BaseModel):
date: Optional[str] = None
# 消息方向
action: Optional[int] = 0
# 是否为回调消息
is_callback: Optional[bool] = False
# 回调数据
callback_data: Optional[str] = None
# 消息ID用于回调时定位原消息
message_id: Optional[Union[str, int]] = None
# 聊天ID用于回调时定位聊天
chat_id: Optional[str] = None
# 完整的回调查询信息(原始数据)
callback_query: Optional[Dict] = None
def to_dict(self):
"""
@@ -65,6 +77,12 @@ class Notification(BaseModel):
action: Optional[int] = 1
# 消息目标用户ID字典未指定用户ID时使用
targets: Optional[dict] = None
# 按钮列表,格式:[[{"text": "按钮文本", "callback_data": "回调数据", "url": "链接"}]]
buttons: Optional[List[List[dict]]] = None
# 原消息ID用于编辑消息
original_message_id: Optional[Union[str, int]] = None
# 原消息的聊天ID用于编辑消息
original_chat_id: Optional[str] = None
def to_dict(self):
"""
@@ -115,3 +133,203 @@ class SubscriptionMessage(BaseModel):
icon: Optional[str] = None
url: Optional[str] = None
data: Optional[dict] = Field(default_factory=dict)
class ChannelCapability(Enum):
"""
渠道能力枚举
"""
# 支持内联按钮
INLINE_BUTTONS = "inline_buttons"
# 支持菜单命令
MENU_COMMANDS = "menu_commands"
# 支持消息编辑
MESSAGE_EDITING = "message_editing"
# 支持消息删除
MESSAGE_DELETION = "message_deletion"
# 支持回调查询
CALLBACK_QUERIES = "callback_queries"
# 支持富文本
RICH_TEXT = "rich_text"
# 支持图片
IMAGES = "images"
# 支持链接
LINKS = "links"
# 支持文件发送
FILE_SENDING = "file_sending"
@dataclass
class ChannelCapabilities:
"""
渠道能力配置
"""
channel: MessageChannel
capabilities: Set[ChannelCapability]
max_buttons_per_row: int = 5
max_button_rows: int = 10
max_button_text_length: int = 30
fallback_enabled: bool = True
class ChannelCapabilityManager:
"""
渠道能力管理器
"""
_capabilities: Dict[MessageChannel, ChannelCapabilities] = {
MessageChannel.Telegram: ChannelCapabilities(
channel=MessageChannel.Telegram,
capabilities={
ChannelCapability.INLINE_BUTTONS,
ChannelCapability.MENU_COMMANDS,
ChannelCapability.MESSAGE_EDITING,
ChannelCapability.MESSAGE_DELETION,
ChannelCapability.CALLBACK_QUERIES,
ChannelCapability.RICH_TEXT,
ChannelCapability.IMAGES,
ChannelCapability.LINKS,
ChannelCapability.FILE_SENDING
},
max_buttons_per_row=4,
max_button_rows=10,
max_button_text_length=30
),
MessageChannel.Wechat: ChannelCapabilities(
channel=MessageChannel.Wechat,
capabilities={
ChannelCapability.IMAGES,
ChannelCapability.LINKS,
ChannelCapability.MENU_COMMANDS
},
fallback_enabled=True
),
MessageChannel.Slack: ChannelCapabilities(
channel=MessageChannel.Slack,
capabilities={
ChannelCapability.INLINE_BUTTONS,
ChannelCapability.MESSAGE_EDITING,
ChannelCapability.MESSAGE_DELETION,
ChannelCapability.CALLBACK_QUERIES,
ChannelCapability.RICH_TEXT,
ChannelCapability.IMAGES,
ChannelCapability.LINKS,
ChannelCapability.MENU_COMMANDS
},
max_buttons_per_row=3,
max_button_rows=8,
max_button_text_length=25,
fallback_enabled=True
),
MessageChannel.SynologyChat: ChannelCapabilities(
channel=MessageChannel.SynologyChat,
capabilities={
ChannelCapability.RICH_TEXT,
ChannelCapability.IMAGES,
ChannelCapability.LINKS
},
fallback_enabled=True
),
MessageChannel.VoceChat: ChannelCapabilities(
channel=MessageChannel.VoceChat,
capabilities={
ChannelCapability.RICH_TEXT,
ChannelCapability.IMAGES,
ChannelCapability.LINKS
},
fallback_enabled=True
),
MessageChannel.WebPush: ChannelCapabilities(
channel=MessageChannel.WebPush,
capabilities={
ChannelCapability.LINKS
},
fallback_enabled=True
),
MessageChannel.Web: ChannelCapabilities(
channel=MessageChannel.Web,
capabilities={
ChannelCapability.RICH_TEXT,
ChannelCapability.IMAGES,
ChannelCapability.LINKS
},
fallback_enabled=True
)
}
@classmethod
def get_capabilities(cls, channel: MessageChannel) -> Optional[ChannelCapabilities]:
"""
获取渠道能力
"""
return cls._capabilities.get(channel)
@classmethod
def supports_capability(cls, channel: MessageChannel, capability: ChannelCapability) -> bool:
"""
检查渠道是否支持某项能力
"""
channel_caps = cls.get_capabilities(channel)
if not channel_caps:
return False
return capability in channel_caps.capabilities
@classmethod
def supports_buttons(cls, channel: MessageChannel) -> bool:
"""
检查渠道是否支持按钮
"""
return cls.supports_capability(channel, ChannelCapability.INLINE_BUTTONS)
@classmethod
def supports_callbacks(cls, channel: MessageChannel) -> bool:
"""
检查渠道是否支持回调
"""
return cls.supports_capability(channel, ChannelCapability.CALLBACK_QUERIES)
@classmethod
def supports_editing(cls, channel: MessageChannel) -> bool:
"""
检查渠道是否支持消息编辑
"""
return cls.supports_capability(channel, ChannelCapability.MESSAGE_EDITING)
@classmethod
def supports_deletion(cls, channel: MessageChannel) -> bool:
"""
检查渠道是否支持消息删除
"""
return cls.supports_capability(channel, ChannelCapability.MESSAGE_DELETION)
@classmethod
def get_max_buttons_per_row(cls, channel: MessageChannel) -> int:
"""
获取每行最大按钮数
"""
channel_caps = cls.get_capabilities(channel)
return channel_caps.max_buttons_per_row if channel_caps else 2
@classmethod
def get_max_button_rows(cls, channel: MessageChannel) -> int:
"""
获取最大按钮行数
"""
channel_caps = cls.get_capabilities(channel)
return channel_caps.max_button_rows if channel_caps else 5
@classmethod
def get_max_button_text_length(cls, channel: MessageChannel) -> int:
"""
获取按钮文本最大长度
"""
channel_caps = cls.get_capabilities(channel)
return channel_caps.max_button_text_length if channel_caps else 20
@classmethod
def should_use_fallback(cls, channel: MessageChannel) -> bool:
"""
是否应该使用降级策略
"""
channel_caps = cls.get_capabilities(channel)
return channel_caps.fallback_enabled if channel_caps else True

View File

@@ -63,6 +63,8 @@ class EventType(Enum):
ModuleReload = "module.reload"
# 配置项更新
ConfigChanged = "config.updated"
# 消息交互动作
MessageAction = "message.action"
# 同步链式事件

View File

@@ -8,7 +8,7 @@ from app.startup.command_initializer import init_command, stop_command, restart_
from app.startup.memory_initializer import init_memory_manager, stop_memory_manager
from app.startup.modules_initializer import init_modules, stop_modules
from app.startup.monitor_initializer import stop_monitor, init_monitor
from app.startup.plugins_initializer import init_plugins, stop_plugins, sync_plugins, backup_plugins, restore_plugins
from app.startup.plugins_initializer import init_plugins, stop_plugins, sync_plugins
from app.startup.routers_initializer import init_routers
from app.startup.scheduler_initializer import stop_scheduler, init_scheduler, init_plugin_scheduler
from app.startup.workflow_initializer import init_workflow, stop_workflow
@@ -41,7 +41,7 @@ async def lifespan(app: FastAPI):
# 初始化路由
init_routers(app)
# 恢复插件备份
restore_plugins()
SystemChain().restore_plugins()
# 初始化插件
init_plugins()
# 初始化定时器
@@ -70,7 +70,7 @@ async def lifespan(app: FastAPI):
except Exception as e:
print(str(e))
# 备份插件
backup_plugins()
SystemChain().backup_plugins()
# 停止内存管理器
stop_memory_manager()
# 停止工作流

View File

@@ -1,11 +1,7 @@
import asyncio
import shutil
from app.core.config import settings
from app.core.plugin import PluginManager
from app.log import logger
from app.utils.system import SystemUtils
from app.helper.system import SystemHelper
async def sync_plugins() -> bool:
@@ -79,105 +75,3 @@ def stop_plugins():
plugin_manager.stop_monitor()
except Exception as e:
logger.error(f"停止插件时发生错误:{e}", exc_info=True)
def backup_plugins():
"""
备份插件到用户配置目录仅docker环境
"""
# 非docker环境不处理
if not SystemUtils.is_docker():
return
try:
# 使用绝对路径确保准确性
plugins_dir = settings.ROOT_PATH / "app" / "plugins"
backup_dir = settings.CONFIG_PATH / "plugins_backup"
if not plugins_dir.exists():
logger.info("插件目录不存在,跳过备份")
return
# 确保备份目录存在
backup_dir.mkdir(parents=True, exist_ok=True)
# 需要排除的文件和目录
exclude_items = {"__init__.py", "__pycache__", ".DS_Store"}
# 遍历插件目录,备份除排除项外的所有内容
for item in plugins_dir.iterdir():
if item.name in exclude_items:
continue
target_path = backup_dir / item.name
# 如果是目录
if item.is_dir():
if target_path.exists():
shutil.rmtree(target_path)
shutil.copytree(item, target_path)
logger.info(f"已备份插件目录: {item.name}")
# 如果是文件
elif item.is_file():
shutil.copy2(item, target_path)
logger.info(f"已备份插件文件: {item.name}")
logger.info(f"插件备份完成,备份位置: {backup_dir}")
except Exception as e:
logger.error(f"插件备份失败: {str(e)}")
def restore_plugins():
"""
从备份恢复插件到app/plugins目录恢复完成后删除备份仅docker环境
"""
# 非docker环境不处理
if not SystemUtils.is_docker():
return
# 使用绝对路径确保准确性
plugins_dir = settings.ROOT_PATH / "app" / "plugins"
backup_dir = settings.CONFIG_PATH / "plugins_backup"
if not backup_dir.exists():
logger.info("插件备份目录不存在,跳过恢复")
return
# 系统被重置才恢复插件
if SystemHelper().is_system_reset():
# 确保插件目录存在
plugins_dir.mkdir(parents=True, exist_ok=True)
# 遍历备份目录,恢复所有内容
restored_count = 0
for item in backup_dir.iterdir():
target_path = plugins_dir / item.name
try:
# 如果是目录,且目录内有内容
if item.is_dir() and any(item.iterdir()):
if target_path.exists():
shutil.rmtree(target_path)
shutil.copytree(item, target_path)
logger.info(f"已恢复插件目录: {item.name}")
restored_count += 1
# 如果是文件
elif item.is_file():
shutil.copy2(item, target_path)
logger.info(f"已恢复插件文件: {item.name}")
restored_count += 1
except Exception as e:
logger.error(f"恢复插件 {item.name} 时发生错误: {str(e)}")
continue
logger.info(f"插件恢复完成,共恢复 {restored_count} 个项目")
# 删除备份目录
try:
shutil.rmtree(backup_dir)
logger.info(f"已删除插件备份目录: {backup_dir}")
except Exception as e:
logger.warning(f"删除备份目录失败: {str(e)}")

View File

@@ -13,12 +13,71 @@ from app.log import logger
urllib3.disable_warnings(InsecureRequestWarning)
class AutoCloseResponse:
"""
自动关闭连接的Response包装器
在访问常用属性后自动关闭连接
"""
def __init__(self, response: Response):
self._response = response
self._closed = False
def __getattr__(self, name):
"""
对于其他属性直接委托给原始response
"""
return getattr(self._response, name)
def _auto_close(self):
"""
自动关闭连接
"""
if not self._closed and self._response:
try:
self._response.close()
self._closed = True
except Exception as e:
logger.debug(f"自动关闭响应失败: {e}")
def json(self, **kwargs):
"""
获取JSON数据并自动关闭连接
"""
try:
data = self._response.json(**kwargs)
return data
finally:
self._auto_close()
@property
def text(self):
"""
获取文本内容并自动关闭连接
"""
try:
return self._response.text
finally:
self._auto_close()
@property
def content(self):
"""
获取二进制内容并自动关闭连接
"""
try:
return self._response.content
finally:
self._auto_close()
def close(self):
"""
手动关闭连接
"""
self._auto_close()
class RequestUtils:
_headers: dict = None
_cookies: Union[str, dict] = None
_proxies: dict = None
_timeout: int = 20
_session: Session = None
def __init__(self,
headers: dict = None,
@@ -30,6 +89,9 @@ class RequestUtils:
referer: str = None,
content_type: str = None,
accept_type: str = None):
self._proxies = proxies
self._session = session
self._timeout = timeout or 20
if not content_type:
content_type = "application/x-www-form-urlencoded; charset=UTF-8"
if headers:
@@ -46,12 +108,8 @@ class RequestUtils:
self._cookies = self.cookie_parse(cookies)
else:
self._cookies = cookies
if proxies:
self._proxies = proxies
if session:
self._session = session
if timeout:
self._timeout = timeout
else:
self._cookies = None
def request(self, method: str, url: str, raise_exception: bool = False, **kwargs) -> Optional[Response]:
"""
@@ -90,7 +148,16 @@ class RequestUtils:
:return: 响应的内容若发生RequestException则返回None
"""
response = self.request(method="get", url=url, params=params, **kwargs)
return str(response.content, "utf-8") if response else None
if response:
try:
content = str(response.content, "utf-8")
return content
except Exception as e:
logger.debug(f"处理响应内容失败: {e}")
return None
finally:
response.close() # 确保连接被关闭
return None
def post(self, url: str, data: Any = None, json: dict = None, **kwargs) -> Optional[Response]:
"""
@@ -122,7 +189,8 @@ class RequestUtils:
json: dict = None,
allow_redirects: bool = True,
raise_exception: bool = False,
**kwargs) -> Optional[Response]:
auto_close: bool = True,
**kwargs) -> Optional[AutoCloseResponse]:
"""
发送GET请求并返回响应对象
:param url: 请求的URL
@@ -131,18 +199,22 @@ class RequestUtils:
:param json: 请求的JSON数据
:param allow_redirects: 是否允许重定向
:param raise_exception: 是否在发生异常时抛出异常否则默认拦截异常返回None
:param auto_close: 是否自动关闭响应连接None时使用全局配置
:param kwargs: 其他请求参数如headers, cookies, proxies等
:return: HTTP响应对象若发生RequestException则返回None
:raises: requests.exceptions.RequestException 仅raise_exception为True时会抛出
"""
return self.request(method="get",
url=url,
params=params,
data=data,
json=json,
allow_redirects=allow_redirects,
raise_exception=raise_exception,
**kwargs)
response = self.request(method="get",
url=url,
params=params,
data=data,
json=json,
allow_redirects=allow_redirects,
raise_exception=raise_exception,
**kwargs)
if response is not None and auto_close:
return AutoCloseResponse(response)
return response
@contextmanager
def get_stream(self, url: str, params: dict = None, **kwargs):
@@ -168,7 +240,8 @@ class RequestUtils:
files: Any = None,
json: dict = None,
raise_exception: bool = False,
**kwargs) -> Optional[Response]:
auto_close: bool = True,
**kwargs) -> Optional[AutoCloseResponse]:
"""
发送POST请求并返回响应对象
:param url: 请求的URL
@@ -177,20 +250,24 @@ class RequestUtils:
:param allow_redirects: 是否允许重定向
:param files: 请求的文件
:param json: 请求的JSON数据
:param kwargs: 其他请求参数如headers, cookies, proxies等
:param raise_exception: 是否在发生异常时抛出异常否则默认拦截异常返回None
:param auto_close: 是否自动关闭响应连接None时使用全局配置
:param kwargs: 其他请求参数如headers, cookies, proxies等
:return: HTTP响应对象若发生RequestException则返回None
:raises: requests.exceptions.RequestException 仅raise_exception为True时会抛出
"""
return self.request(method="post",
url=url,
data=data,
params=params,
allow_redirects=allow_redirects,
files=files,
json=json,
raise_exception=raise_exception,
**kwargs)
response = self.request(method="post",
url=url,
data=data,
params=params,
allow_redirects=allow_redirects,
files=files,
json=json,
raise_exception=raise_exception,
**kwargs)
if response is not None and auto_close:
return AutoCloseResponse(response)
return response
def put_res(self,
url: str,
@@ -200,7 +277,8 @@ class RequestUtils:
files: Any = None,
json: dict = None,
raise_exception: bool = False,
**kwargs) -> Optional[Response]:
auto_close: bool = True,
**kwargs) -> Optional[AutoCloseResponse]:
"""
发送PUT请求并返回响应对象
:param url: 请求的URL
@@ -210,19 +288,23 @@ class RequestUtils:
:param files: 请求的文件
:param json: 请求的JSON数据
:param raise_exception: 是否在发生异常时抛出异常否则默认拦截异常返回None
:param auto_close: 是否自动关闭响应连接None时使用全局配置
:param kwargs: 其他请求参数如headers, cookies, proxies等
:return: HTTP响应对象若发生RequestException则返回None
:raises: requests.exceptions.RequestException 仅raise_exception为True时会抛出
"""
return self.request(method="put",
url=url,
data=data,
params=params,
allow_redirects=allow_redirects,
files=files,
json=json,
raise_exception=raise_exception,
**kwargs)
response = self.request(method="put",
url=url,
data=data,
params=params,
allow_redirects=allow_redirects,
files=files,
json=json,
raise_exception=raise_exception,
**kwargs)
if response is not None and auto_close:
return AutoCloseResponse(response)
return response
def delete_res(self,
url: str,
@@ -230,7 +312,8 @@ class RequestUtils:
params: dict = None,
allow_redirects: bool = True,
raise_exception: bool = False,
**kwargs) -> Optional[Response]:
auto_close: bool = True,
**kwargs) -> Optional[AutoCloseResponse]:
"""
发送DELETE请求并返回响应对象
:param url: 请求的URL
@@ -238,17 +321,21 @@ class RequestUtils:
:param params: 请求的参数
:param allow_redirects: 是否允许重定向
:param raise_exception: 是否在发生异常时抛出异常否则默认拦截异常返回None
:param auto_close: 是否自动关闭响应连接None时使用全局配置
:param kwargs: 其他请求参数如headers, cookies, proxies等
:return: HTTP响应对象若发生RequestException则返回None
:raises: requests.exceptions.RequestException 仅raise_exception为True时会抛出
"""
return self.request(method="delete",
url=url,
data=data,
params=params,
allow_redirects=allow_redirects,
raise_exception=raise_exception,
**kwargs)
response = self.request(method="delete",
url=url,
data=data,
params=params,
allow_redirects=allow_redirects,
raise_exception=raise_exception,
**kwargs)
if response is not None and auto_close:
return AutoCloseResponse(response)
return response
@staticmethod
def cookie_parse(cookies_str: str, array: bool = False) -> Union[list, dict]:
@@ -381,7 +468,7 @@ class RequestUtils:
return fallback_encoding or "utf-8"
@staticmethod
def get_decoded_html_content(response: Response,
def get_decoded_html_content(response: Union[Response, AutoCloseResponse],
performance_mode: bool = False, confidence_threshold: float = 0.8) -> str:
"""
获取HTML响应的解码文本内容
@@ -413,3 +500,65 @@ class RequestUtils:
except Exception as e:
logger.debug(f"Error when getting decoded content: {str(e)}")
return response.text
@contextmanager
def response_manager(self, method: str, url: str, **kwargs):
"""
响应管理器上下文管理器,确保响应对象被正确关闭
:param method: HTTP方法
:param url: 请求的URL
:param kwargs: 其他请求参数
"""
response = None
try:
response = self.request(method=method, url=url, **kwargs)
yield response
finally:
if response:
try:
response.close()
except Exception as e:
logger.debug(f"关闭响应失败: {e}")
def get_json(self, url: str, params: dict = None, **kwargs) -> Optional[dict]:
"""
发送GET请求并返回JSON数据自动关闭连接
:param url: 请求的URL
:param params: 请求的参数
:param kwargs: 其他请求参数
:return: JSON数据若发生异常则返回None
"""
response = self.request(method="get", url=url, params=params, **kwargs)
if response:
try:
data = response.json()
return data
except Exception as e:
logger.debug(f"解析JSON失败: {e}")
return None
finally:
response.close()
return None
def post_json(self, url: str, data: Any = None, json: dict = None, **kwargs) -> Optional[dict]:
"""
发送POST请求并返回JSON数据自动关闭连接
:param url: 请求的URL
:param data: 请求的数据
:param json: 请求的JSON数据
:param kwargs: 其他请求参数
:return: JSON数据若发生异常则返回None
"""
if json is None:
json = {}
response = self.request(method="post", url=url, data=data, json=json, **kwargs)
if response:
try:
data = response.json()
return data
except Exception as e:
logger.debug(f"解析JSON失败: {e}")
return None
finally:
response.close()
return None

View File

@@ -31,7 +31,7 @@ class SingletonClass(abc.ABCMeta, type):
def __call__(cls, *args, **kwargs):
if cls not in cls._instances:
cls._instances[cls] = super(SingletonClass, cls).__call__(*args, **kwargs)
cls._instances[cls] = super().__call__(*args, **kwargs)
return cls._instances[cls]

View File

@@ -36,3 +36,7 @@ class Tokens:
return None
else:
return self._tokens[index]
@property
def tokens(self):
return self._tokens

View File

@@ -9,15 +9,9 @@ SUPERUSER=admin
DEV=false
# 为指定字幕添加.default后缀设置为默认字幕支持为'zh-cn''zh-tw''eng'添加默认字幕未定义或设置为None则不添加
DEFAULT_SUB=zh-cn
# 数据库连接池的大小可适当降低如20-50以减少I/O压力
DB_POOL_SIZE=100
# 数据库连接池最大溢出连接数可适当降低如0以减少I/O压力
DB_MAX_OVERFLOW=500
# SQLite 的 busy_timeout 参数可适当增加如180以减少锁定错误
DB_TIMEOUT=60
# 是否启用内存监控,开启后将定期生成内存快照文件
MEMORY_ANALYSIS=false
# 内存快照间隔(分钟)
MEMORY_SNAPSHOT_INTERVAL=60
MEMORY_SNAPSHOT_INTERVAL=30
# 保留的内存快照文件数量
MEMORY_SNAPSHOT_KEEP_COUNT=20

View File

@@ -0,0 +1,37 @@
"""2.1.7
Revision ID: 3891a5e722a1
Revises: 3df653756eec
Create Date: 2025-06-28 08:40:14.516836
"""
from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects import sqlite
from app.db.systemconfig_oper import SystemConfigOper
from app.schemas.types import SystemConfigKey
# revision identifiers, used by Alembic.
revision = '3891a5e722a1'
down_revision = '3df653756eec'
branch_labels = None
depends_on = None
def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
# rename AList存储
_systemconfig = SystemConfigOper()
_storages = _systemconfig.get(SystemConfigKey.Storages)
if _storages:
for storage in _storages:
if storage["type"] == "alist":
storage["name"] = "OpenList"
break
_systemconfig.set(SystemConfigKey.Storages, _storages)
# ### end Alembic commands ###
def downgrade() -> None:
pass

View File

@@ -81,7 +81,7 @@ RUN cp -f /app/docker/nginx.common.conf /etc/nginx/common.conf \
&& cat /tmp/MoviePilot-Plugins-main/package.json | jq -r 'to_entries[] | select(.value.v2 == true) | .key' | awk '{print tolower($0)}' | \
while read -r i; do if [ ! -d "/app/app/plugins/$i" ]; then mv "/tmp/MoviePilot-Plugins-main/plugins/$i" "/app/app/plugins/"; else echo "跳过 $i"; fi; done \
&& curl -sL "https://github.com/jxxghp/MoviePilot-Resources/archive/refs/heads/main.zip" | busybox unzip -d /tmp - \
&& mv -f /tmp/MoviePilot-Resources-main/resources/* /app/app/helper/ \
&& mv -f /tmp/MoviePilot-Resources-main/resources.v2/* /app/app/helper/ \
&& rm -rf /tmp/*
EXPOSE 3000
VOLUME [ "${CONFIG_DIR}" ]

View File

@@ -103,7 +103,7 @@ function install_backend_and_download_resources() {
INFO "→ 正在备份站点资源目录..."
rm -rf /resources_bakcup
mkdir /resources_bakcup
cp -a /app/app/helper/user.sites.bin /resources_bakcup
cp -a /app/app/helper/user.sites.v2.bin /resources_bakcup
cp -a /app/app/helper/sites.cp* /resources_bakcup
# 清空程序目录
rm -rf /app
@@ -126,7 +126,7 @@ function install_backend_and_download_resources() {
return 1
fi
# 复制新站点资源
cp -a ${TMP_PATH}/Resources/resources/* /app/app/helper/
cp -a ${TMP_PATH}/Resources/resources.v2/* /app/app/helper/
INFO "站点资源更新成功"
# 清理临时目录
rm -rf "${TMP_PATH}"

View File

@@ -1,71 +1,71 @@
Cython~=3.0.12
pydantic~=1.10.13
SQLAlchemy~=2.0.15
uvicorn~=0.22.0
fastapi~=0.96.0
Cython~=3.1.2
pydantic~=1.10.22
SQLAlchemy~=2.0.41
uvicorn~=0.34.3
fastapi~=0.115.14
passlib~=1.7.4
PyJWT~=2.7.0
PyJWT~=2.10.1
python-multipart~=0.0.9
alembic~=1.11.1
alembic~=1.16.2
bcrypt~=4.0.1
regex~=2023.6.3
regex~=2024.11.6
cn2an~=0.5.19
dateparser~=1.1.8
dateparser~=1.2.2
python-dateutil~=2.8.2
zhconv~=1.4.3
anitopy~=2.1.1
requests[socks]~=2.32.3
urllib3~=2.2.2
lxml~=4.9.2
pyquery~=2.0.0
ruamel.yaml~=0.17.31
APScheduler~=3.10.1
cryptography~=43.0.0
pytz~=2023.3
pycryptodome~=3.20.0
qbittorrent-api==2024.11.70
plexapi~=4.16.0
transmission-rpc~=4.3.0
Jinja2~=3.1.4
pyparsing~=3.0.9
requests[socks]~=2.32.4
urllib3~=2.5.0
lxml~=6.0.0
pyquery~=2.0.1
ruamel.yaml~=0.18.14
APScheduler~=3.11.0
cryptography~=45.0.4
pytz~=2025.2
pycryptodome~=3.23.0
qbittorrent-api==2025.5.0
plexapi~=4.17.0
transmission-rpc~=7.0.11
Jinja2~=3.1.6
pyparsing~=3.2.3
func_timeout==4.3.5
bs4~=0.0.1
beautifulsoup4~=4.12.2
pillow~=10.4.0
pillow-avif-plugin~=1.4.6
pyTelegramBotAPI~=4.12.0
playwright~=1.49.1
cf-clearance~=0.31.0
bs4~=0.0.2
beautifulsoup4~=4.13.4
pillow~=11.2.1
pillow-avif-plugin~=1.5.2
pyTelegramBotAPI~=4.27.0
playwright~=1.53.0
cf_clearance~=0.31.0
torrentool~=1.2.0
slack-bolt~=1.18.0
slack-sdk~=3.21.3
chardet~=4.0.0
starlette~=0.27.0
slack-bolt~=1.23.0
slack-sdk~=3.35.0
chardet~=5.2.0
starlette~=0.46.2
PyVirtualDisplay~=3.0
psutil~=5.9.4
python-dotenv~=1.0.1
python-hosts~=1.0.7
watchdog~=3.0.0
openai~=0.27.2
cacheout~=0.14.1
click~=8.1.6
requests-cache~=0.5.2
parse~=1.19.0
psutil~=7.0.0
python-dotenv~=1.1.1
python-hosts~=1.1.2
watchdog~=6.0.0
openai~=1.92.2
cacheout~=0.16.0
click~=8.2.1
requests-cache~=1.2.1
parse~=1.20.2
docker~=7.1.0
pywin32==306; platform_system == "Windows"
cachetools~=5.3.1
fast-bencode~=1.1.3
pywin32==310; platform_system == "Windows"
cachetools~=6.1.0
fast-bencode~=1.1.7
pystray~=0.19.5
pyotp~=2.9.0
Pinyin2Hanzi~=0.1.1
pywebpush~=2.0.0
python-cookietools==0.0.2.1
pywebpush~=2.0.3
python-cookietools==0.0.4
aiofiles~=24.1.0
jieba~=0.42.1
rsa~=4.9
redis~=5.2.1
redis~=6.2.0
async_timeout~=5.0.1; python_full_version < "3.11.3"
packaging~=24.2
cf_clearance~=0.31.0
packaging~=25.0
oss2~=2.19.1
tqdm~=4.67.1
setuptools~=78.1.0

View File

@@ -153,7 +153,7 @@ meta_cases = [{
"part": "",
"season": "S01",
"episode": "E02",
"restype": "WEB-DL",
"restype": "B-Global WEB-DL",
"pix": "1080p",
"video_codec": "x264",
"audio_codec": "AAC"
@@ -569,7 +569,7 @@ meta_cases = [{
"part": "",
"season": "S02",
"episode": "E05",
"restype": "WEB-DL",
"restype": "Crunchyroll WEB-DL",
"pix": "1080p",
"video_codec": "x264",
"audio_codec": "AAC"
@@ -649,7 +649,7 @@ meta_cases = [{
"part": "",
"season": "",
"episode": "",
"restype": "WEBRip",
"restype": "Netflix WEBRip",
"pix": "1080p",
"video_codec": "H264",
"audio_codec": "DDP 5.1"
@@ -681,7 +681,7 @@ meta_cases = [{
"part": "",
"season": "S01",
"episode": "E16",
"restype": "WEB-DL",
"restype": "KKTV WEB-DL",
"pix": "1080p",
"video_codec": "x264",
"audio_codec": "AAC"
@@ -921,7 +921,7 @@ meta_cases = [{
"part": "",
"season": "S06",
"episode": "E06",
"restype": "WEBRip",
"restype": "Max WEBRip",
"pix": "1080p",
"video_codec": "x264",
"audio_codec": "DD 5.1"
@@ -937,7 +937,7 @@ meta_cases = [{
"part": "",
"season": "S06",
"episode": "E05",
"restype": "WEBRip",
"restype": "Max WEBRip",
"pix": "1080p",
"video_codec": "x264",
"audio_codec": "DD 5.1"
@@ -969,7 +969,7 @@ meta_cases = [{
"part": "",
"season": "S02",
"episode": "",
"restype": "WEB-DL",
"restype": "Netflix WEB-DL",
"pix": "2160p",
"video_codec": "H265",
"audio_codec": "DDP 5.1 Atmos"

View File

@@ -1,6 +1,5 @@
import unittest
from tests.test_bluray import BluRayTest
from tests.test_metainfo import MetaInfoTest
if __name__ == '__main__':
@@ -10,9 +9,6 @@ if __name__ == '__main__':
suite.addTest(MetaInfoTest('test_metainfo'))
suite.addTest(MetaInfoTest('test_emby_format_ids'))
# 测试蓝光目录识别
suite.addTest(BluRayTest())
# 运行测试
runner = unittest.TextTestRunner()
runner.run(suite)

View File

@@ -1,178 +0,0 @@
#!/usr/bin/env python
# -*- coding:utf-8 -*-
from pathlib import Path
from typing import List, Optional
from unittest import TestCase
from app import schemas
from app.chain.storage import StorageChain
from app.chain.transfer import TransferChain
from app.db.models.transferhistory import TransferHistory
from app.db.systemconfig_oper import SystemConfigOper
from app.db.transferhistory_oper import TransferHistoryOper
from tests.cases.files import bluray_files
class MockTransferHistoryOper(TransferHistoryOper):
def __init__(self):
# pylint: disable=super-init-not-called
self.history = []
def get_by_src(self, src, storage=None):
self.history.append(src)
return TransferHistory()
class MockStorage(StorageChain):
def __init__(self, files: list):
# pylint: disable=super-init-not-called
self.__root = schemas.FileItem(
path="/", name="", type="dir", extension="", size=0
)
self.__all = {self.__root.path: self.__root}
def __build_child(parent: schemas.FileItem, files: list[dict]):
parent.children = []
for item in files:
children = item.get("children")
sep = "" if parent.path.endswith("/") else "/"
name: str = item["name"]
file_item = schemas.FileItem(
path=f"{parent.path}{sep}{name}",
name=name,
extension=Path(name).suffix[1:],
basename=Path(name).stem,
type="file" if children is None else "dir",
size=item.get("size", 0),
)
parent.children.append(file_item)
self.__all[file_item.path] = file_item
if children is not None:
__build_child(file_item, children)
__build_child(self.__root, files)
def list_files(
self, fileitem: schemas.FileItem, recursion: bool = False
) -> Optional[List[schemas.FileItem]]:
if fileitem.type != "dir":
return None
if recursion:
result = []
file_path = f"{fileitem.path}/"
for path, item in self.__all.items():
if path.startswith(file_path):
result.append(item)
return result
else:
return fileitem.children
def get_file_item(self, storage: str, path: Path) -> Optional[schemas.FileItem]:
"""
根据路径获取文件项
"""
path_posix = path.as_posix()
return self.__all.get(path_posix)
class MockTransferChain(TransferChain):
def __init__(self, storage: MockStorage):
# pylint: disable=super-init-not-called
self.transferhis = MockTransferHistoryOper()
self.systemconfig = SystemConfigOper()
self.storagechain = storage
def test(self, path: str):
self.transferhis.history.clear()
self.do_transfer(
force=False,
background=False,
fileitem=self.storagechain.get_file_item(None, Path(path)),
)
return self.transferhis.history
class BluRayTest(TestCase):
def __init__(self, methodName="test"):
super().__init__(methodName)
def setUp(self) -> None:
pass
def tearDown(self) -> None:
pass
def test(self):
transfer = MockTransferChain(MockStorage(bluray_files))
self.assertEqual(
[
"/FOLDER/Digimon/Digimon (2055)",
"/FOLDER/Digimon/Digimon (2099)",
"/FOLDER/Digimon/Digimon (2199)/Digimon.2199.mp4",
],
transfer.test("/FOLDER/Digimon"),
)
self.assertEqual(
[
"/FOLDER/Digimon/Digimon (2055)",
],
transfer.test("/FOLDER/Digimon/Digimon (2055)"),
)
self.assertEqual(
[
"/FOLDER/Digimon/Digimon (2055)",
],
transfer.test("/FOLDER/Digimon/Digimon (2055)/BDMV"),
)
self.assertEqual(
[
"/FOLDER/Digimon/Digimon (2055)",
],
transfer.test("/FOLDER/Digimon/Digimon (2055)/BDMV/STREAM"),
)
self.assertEqual(
[
"/FOLDER/Digimon/Digimon (2055)",
],
transfer.test("/FOLDER/Digimon/Digimon (2055)/BDMV/STREAM/00001.m2ts"),
)
self.assertEqual(
[
"/FOLDER/Digimon/Digimon (2199)/Digimon.2199.mp4",
],
transfer.test("/FOLDER/Digimon/Digimon (2199)"),
)
self.assertEqual(
[
"/FOLDER/Digimon/Digimon (2199)/Digimon.2199.mp4",
],
transfer.test("/FOLDER/Digimon/Digimon (2199)/Digimon.2199.mp4"),
)
self.assertEqual(
[
"/FOLDER/Pokemon.2029.mp4",
],
transfer.test("/FOLDER/Pokemon.2029.mp4"),
)
self.assertEqual(
[
"/FOLDER/Digimon/Digimon (2055)",
"/FOLDER/Digimon/Digimon (2099)",
"/FOLDER/Digimon/Digimon (2199)/Digimon.2199.mp4",
"/FOLDER/Pokemon (2016)",
"/FOLDER/Pokemon (2021)",
"/FOLDER/Pokemon (2028)/Pokemon.2028.mkv",
"/FOLDER/Pokemon.2029.mp4",
],
transfer.test("/FOLDER"),
)

View File

@@ -1,2 +1,2 @@
APP_VERSION = 'v2.5.6'
FRONTEND_VERSION = 'v2.5.6'
APP_VERSION = 'v2.5.9'
FRONTEND_VERSION = 'v2.5.9'