Compare commits

...

84 Commits

Author SHA1 Message Date
jxxghp
825d9b768f 更新 version.py 2024-11-16 11:18:23 +08:00
jxxghp
f758a47f4f Merge pull request #3122 from DDS-Derek/fix_update 2024-11-16 11:02:04 +08:00
jxxghp
fc69d7e6c1 fix 2024-11-16 10:55:17 +08:00
DDSRem
edc30266c8 fix(update): clear tmp directory causes data loss
fix https://github.com/jxxghp/MoviePilot/issues/2996
2024-11-16 10:53:33 +08:00
jxxghp
665da9dad3 Merge pull request #3121 from DDS-Derek/fix_nginx 2024-11-16 10:37:23 +08:00
DDSRem
4048acf60e feat(docker): nginx client_max_body_size configuration
fix https://github.com/jxxghp/MoviePilot/issues/2951
fix https://github.com/jxxghp/MoviePilot/issues/2720
2024-11-16 10:23:28 +08:00
jxxghp
f116229ecc fix #3108 2024-11-16 09:50:55 +08:00
jxxghp
f6a2efb256 fix #3116 2024-11-16 09:25:46 +08:00
jxxghp
af3a50f7ea feat:订阅支持绑定下载器 2024-11-16 09:00:18 +08:00
jxxghp
44a0e5b4a7 fix #3120 2024-11-16 08:41:30 +08:00
jxxghp
f40a1246ff Merge pull request #3118 from wikrin/database 2024-11-16 07:54:53 +08:00
jxxghp
dd890c410c Merge pull request #3117 from wikrin/site 2024-11-16 07:54:42 +08:00
Attente
8fd7f2c875 fix 资源搜索下载时设置的下载器不生效的问题 2024-11-16 01:44:20 +08:00
Attente
8c09b3482f Upgrade the database 2024-11-16 00:28:13 +08:00
Attente
0066247a2b feat: 站点管理增加下载器选择 2024-11-16 00:22:04 +08:00
jxxghp
c7926fc575 Merge pull request #3113 from InfinityPacer/feature/module 2024-11-15 21:59:50 +08:00
InfinityPacer
ac5b9fd4e5 fix(rclone): specify UTF-8 encoding when save config 2024-11-15 17:42:11 +08:00
jxxghp
42dc539df6 fix #3013 2024-11-15 16:17:51 +08:00
jxxghp
e60d785a11 fix meta re 2024-11-15 13:50:33 +08:00
jxxghp
33558d6197 Merge pull request #3102 from InfinityPacer/feature/module 2024-11-15 12:01:21 +08:00
InfinityPacer
46d2ffeb75 fix #3100 2024-11-15 09:08:32 +08:00
jxxghp
8e4bce2f95 fix #3079 2024-11-15 08:03:23 +08:00
jxxghp
00f1f06e3d fix #3079 2024-11-15 08:00:22 +08:00
jxxghp
fe37bde993 fix offset ep 2024-11-14 22:29:14 +08:00
jxxghp
6c3bb8893f Merge pull request #3097 from wdmcheng/v2 2024-11-14 21:47:59 +08:00
wdmcheng
ca4d64819d fix 部分情况下Alist解析时间错误 2024-11-14 21:39:13 +08:00
jxxghp
0a53635d35 Merge pull request #3096 from rexshao/v2 2024-11-14 21:15:47 +08:00
rexshao
921e24b049 Update twofa.py
修复2fa使用secret无法正常生成code的BUG
2024-11-14 21:08:38 +08:00
jxxghp
24c21ed04e fix name 2024-11-14 19:58:37 +08:00
jxxghp
777785579e v2.0.4
- 修复了手动整理时找不到目录的问题
- 修复了白兔站点信息获取、登录状态检测
- 修复了一个索引报错问题
- 优化了资源下载对话框
- 目录设置增加了一个手动整理的选项
- 增加了QB无法连接时的日志打印
- 存储支持挂接AList
2024-11-14 19:48:16 +08:00
jxxghp
8061a06fe4 Merge remote-tracking branch 'origin/v2' into v2 2024-11-14 18:09:49 +08:00
jxxghp
438ce6ee3e fix SiteUserData schema 2024-11-14 18:09:40 +08:00
jxxghp
77e19c3de7 Merge pull request #3095 from InfinityPacer/feature/module 2024-11-14 17:25:31 +08:00
InfinityPacer
49881c9c54 fix #2952 2024-11-14 17:21:47 +08:00
jxxghp
5da28f702f fix alist 2024-11-14 14:54:22 +08:00
jxxghp
dfbd9f3b30 add alist storage card 2024-11-14 12:57:34 +08:00
jxxghp
d6c6ee9b4e fix #3092 2024-11-14 12:38:02 +08:00
jxxghp
4b27404ee5 Merge pull request #3091 from InfinityPacer/feature/cache 2024-11-14 11:57:26 +08:00
jxxghp
3a826b343a fix #3090 2024-11-14 11:52:56 +08:00
jxxghp
851aa5f9e2 fix #3031 2024-11-14 11:49:57 +08:00
InfinityPacer
9ef1f56ea1 feat(cache): add proxy support for specific domains in image caching 2024-11-14 10:21:00 +08:00
jxxghp
78d51b7621 Merge pull request #3031 from Akimio521/feat/filemanager-alist
feat: 增加 filemanager storages 类型:Alist
2024-11-14 08:12:31 +08:00
jxxghp
c12e2bdba7 fix 手动整理Bug 2024-11-14 08:04:52 +08:00
jxxghp
fda11f427c Merge pull request #3087 from amtoaer/fix_hares 2024-11-14 06:49:12 +08:00
amtoaer
d809330225 fix: 修复白兔俱乐部的站点信息获取、登录状态检测 2024-11-14 01:59:30 +08:00
jxxghp
ce4a2314d8 fix 手动整理时目录匹配Bug 2024-11-13 21:30:24 +08:00
amtoaer
c19e825e94 fix: 修复白兔俱乐部登录检测 2024-11-13 18:30:52 +08:00
jxxghp
c45d64b554 Merge pull request #3075 from wikrin/v2 2024-11-12 22:25:53 +08:00
Attente
0689b2e331 fix: episode_offset 2024-11-12 22:22:56 +08:00
jxxghp
e6105fdab5 **v2.0.3**
- 修复了最新版本号获取错误的问题
- 修复了文件管理重命名失败的问题
- 修复了整理多季时 season.nfo 刮削错误的问题
- 修复了Rclone存储容量检测错误的问题
- 优化了自定义规则,剧集文件大小规则按平均每集大小过滤
- 移动文件整理时,自动删除空的父目录
- 增加了自动阅读和发送站点消息的开关
- 增加了数据库WAL模式开关,开启后提升数据库性能
2024-11-12 18:48:15 +08:00
jxxghp
df34c7e2da Merge pull request #3074 from InfinityPacer/feature/db 2024-11-12 17:30:34 +08:00
InfinityPacer
24cc36033f feat(db): add support for SQLite WAL mode 2024-11-12 17:17:16 +08:00
jxxghp
aafb2bc269 fix #3071 增加站点消息开关 2024-11-12 13:59:13 +08:00
jxxghp
9dde56467a 更新 __init__.py 2024-11-12 12:24:05 +08:00
jxxghp
f9d62e7451 fix Rclone存储容量检测问题 2024-11-12 10:10:37 +08:00
jxxghp
f1f379966a fix 修复V2最新版本号获取 2024-11-12 08:37:07 +08:00
jxxghp
942c9ae545 Merge pull request #3058 from wikrin/fix-scrape_metadata 2024-11-10 14:02:31 +08:00
jxxghp
89be4f6200 Merge pull request #3054 from wikrin/fix-rename 2024-11-10 14:02:01 +08:00
Attente
bcbf729fd4 修复整理多季时season.nfo刮削错误的问题 2024-11-10 13:43:59 +08:00
Attente
7fc5b7678e 更改判断顺序 2024-11-10 07:47:49 +08:00
Attente
e20578685a fix: 修复重命名失败的问题 2024-11-09 23:59:58 +08:00
jxxghp
40b82d9cb6 fix #3042 移动模式删除空文件夹 2024-11-09 18:23:08 +08:00
jxxghp
9b2fccee01 feat:剧集文件大小过滤按平均每集大小 2024-11-09 18:01:50 +08:00
jxxghp
87bbee8c36 Merge pull request #3038 from InfinityPacer/feature/setup 2024-11-08 18:16:32 +08:00
InfinityPacer
4412ce9f17 fix(playwright): add check for HTTPS proxy 2024-11-08 18:08:45 +08:00
jxxghp
35b78b0e66 Merge pull request #3034 from lybtt/fix_update_bash 2024-11-08 16:44:55 +08:00
lvyb
d97fcc4a96 修复update脚本,版本号比较问题 2024-11-08 16:37:36 +08:00
Akimio521
c8e337440e feat(storages): add Alist storage type 2024-11-08 14:32:30 +08:00
Akimio521
726e7dfbd4 feat(StringUtils): add url_eqote method 2024-11-08 14:31:08 +08:00
jxxghp
a2096e8e0f v2.0.2 2024-11-08 13:26:05 +08:00
jxxghp
75e80158e5 Merge pull request #3030 from Akimio521/fix(tmdb/douban)-cache 2024-11-08 10:48:23 +08:00
Akimio521
d42bd14288 fix: 优先使用id作为cache key避免key冲突 2024-11-08 10:35:29 +08:00
jxxghp
28f6e7f9bb fix https://github.com/jxxghp/MoviePilot-Plugins/issues/540 2024-11-07 18:58:32 +08:00
jxxghp
2aadbeaed7 Merge pull request #3025 from amtoaer/feat_jellyfin_item_path 2024-11-07 18:48:15 +08:00
jxxghp
3f6b4bf3f2 Merge pull request #3022 from MMZOX/v2 2024-11-07 18:46:59 +08:00
amtoaer
f73750fcf7 feat: 为 jellyfin 的 webhook 事件填充 item_path 字段 2024-11-07 15:01:19 +08:00
MMZOX
59df673eb5 try to fix #2965 2024-11-07 13:45:06 +08:00
jxxghp
e29ab92cd1 fix #3008 2024-11-07 08:27:05 +08:00
jxxghp
3777045a17 fix #3012 2024-11-07 08:24:22 +08:00
jxxghp
16165c0fcc fix #3018 2024-11-07 08:20:11 +08:00
jxxghp
4d377d5e04 Merge pull request #3016 from InfinityPacer/feature/scheduler 2024-11-06 20:01:14 +08:00
InfinityPacer
76c84f9bac fix(scheduler): optimize job registration and removal logic 2024-11-06 19:37:22 +08:00
jxxghp
88f91152d6 Merge pull request #3009 from lybtt/fix_local_storage 2024-11-06 10:52:15 +08:00
lvyb
dfdb88c5ac fix softlink 2024-11-06 09:30:53 +08:00
59 changed files with 1520 additions and 283 deletions

2
.gitignore vendored
View File

@@ -12,7 +12,7 @@ app/helper/*.bin
app/plugins/**
!app/plugins/__init__.py
config/cookies/**
config/user.db
config/user.db*
config/sites/**
config/logs/
config/temp/

View File

@@ -149,8 +149,10 @@ def rename(fileitem: schemas.FileItem,
:param recursive: 是否递归修改
:param _: token
"""
if not fileitem.fileid or not new_name:
return schemas.Response(success=False)
if not new_name:
return schemas.Response(success=False, message="新名称为空")
if fileitem.storage != 'local' and not fileitem.fileid:
return schemas.Response(success=False, message="资源ID获取失败")
result = StorageChain().rename_file(fileitem, new_name)
if result:
if recursive:

View File

@@ -159,7 +159,8 @@ def cache_img(
本地缓存图片文件,支持 HTTP 缓存,如果启用全局图片缓存,则使用磁盘缓存
"""
# 如果没有启用全局图片缓存,则不使用磁盘缓存
return fetch_image(url=url, proxy=False, use_disk_cache=settings.GLOBAL_IMAGE_CACHE, if_none_match=if_none_match)
proxy = "doubanio.com" not in url
return fetch_image(url=url, proxy=proxy, use_disk_cache=settings.GLOBAL_IMAGE_CACHE, if_none_match=if_none_match)
@router.get("/global", summary="查询非敏感系统设置", response_model=schemas.Response)

View File

@@ -32,7 +32,7 @@ class ManualTransferItem(BaseModel):
episode_format: Optional[str] = None,
episode_detail: Optional[str] = None,
episode_part: Optional[str] = None,
episode_offset: Optional[int] = 0,
episode_offset: Optional[str] = None,
min_filesize: Optional[int] = 0,
scrape: bool = False,
from_history: bool = False

View File

@@ -204,10 +204,10 @@ class DownloadChain(ChainBase):
def download_single(self, context: Context, torrent_file: Path = None,
episodes: Set[int] = None,
channel: MessageChannel = None, source: str = None,
downloader: str = None,
save_path: str = None,
userid: Union[str, int] = None,
username: str = None,
downloader: str = None,
media_category: str = None) -> Optional[str]:
"""
下载及发送通知
@@ -216,15 +216,16 @@ class DownloadChain(ChainBase):
:param episodes: 需要下载的集数
:param channel: 通知渠道
:param source: 通知来源
:param downloader: 下载器
:param save_path: 保存路径
:param userid: 用户ID
:param username: 调用下载的用户名/插件名
:param downloader: 下载器
:param media_category: 自定义媒体类别
"""
_torrent = context.torrent_info
_media = context.media_info
_meta = context.meta_info
_site_downloader = _torrent.site_downloader
# 补充完整的media数据
if not _media.genre_ids:
@@ -252,10 +253,10 @@ class DownloadChain(ChainBase):
# 下载目录
if save_path:
# 有自定义下载目录时,尝试匹配目录配置
dir_info = self.directoryhelper.get_dir(_media, src_path=Path(save_path), local=True)
dir_info = self.directoryhelper.get_dir(_media, src_path=Path(save_path))
else:
# 根据媒体信息查询下载目录配置
dir_info = self.directoryhelper.get_dir(_media, local=True)
dir_info = self.directoryhelper.get_dir(_media)
# 拼装子目录
if dir_info:
@@ -287,7 +288,7 @@ class DownloadChain(ChainBase):
episodes=episodes,
download_dir=download_dir,
category=_media.category,
downloader=downloader)
downloader=downloader or _site_downloader)
if result:
_downloader, _hash, error_msg = result
else:
@@ -386,7 +387,8 @@ class DownloadChain(ChainBase):
source: str = None,
userid: str = None,
username: str = None,
media_category: str = None
media_category: str = None,
downloader: str = None
) -> Tuple[List[Context], Dict[Union[int, str], Dict[int, NotExistMediaInfo]]]:
"""
根据缺失数据,自动种子列表中组合择优下载
@@ -398,6 +400,7 @@ class DownloadChain(ChainBase):
:param userid: 用户ID
:param username: 调用下载的用户名/插件名
:param media_category: 自定义媒体类别
:param downloader: 下载器
:return: 已经下载的资源列表、剩余未下载到的剧集 no_exists[tmdb_id/douban_id] = {season: NotExistMediaInfo}
"""
# 已下载的项目
@@ -469,7 +472,7 @@ class DownloadChain(ChainBase):
logger.info(f"开始下载电影 {context.torrent_info.title} ...")
if self.download_single(context, save_path=save_path, channel=channel,
source=source, userid=userid, username=username,
media_category=media_category):
media_category=media_category, downloader=downloader):
# 下载成功
logger.info(f"{context.torrent_info.title} 添加下载成功")
downloaded_list.append(context)
@@ -554,7 +557,8 @@ class DownloadChain(ChainBase):
source=source,
userid=userid,
username=username,
media_category=media_category
media_category=media_category,
downloader=downloader,
)
else:
# 下载
@@ -562,7 +566,8 @@ class DownloadChain(ChainBase):
download_id = self.download_single(context, save_path=save_path,
channel=channel, source=source,
userid=userid, username=username,
media_category=media_category)
media_category=media_category,
downloader=downloader)
if download_id:
# 下载成功
@@ -633,7 +638,8 @@ class DownloadChain(ChainBase):
download_id = self.download_single(context, save_path=save_path,
channel=channel, source=source,
userid=userid, username=username,
media_category=media_category)
media_category=media_category,
downloader=downloader)
if download_id:
# 下载成功
logger.info(f"{meta.title} 添加下载成功")
@@ -722,7 +728,8 @@ class DownloadChain(ChainBase):
source=source,
userid=userid,
username=username,
media_category=media_category
media_category=media_category,
downloader=downloader
)
if not download_id:
continue

View File

@@ -335,11 +335,14 @@ class MediaChain(ChainBase, metaclass=Singleton):
:param _path: 元数据文件路径
:param _content: 文件内容
"""
if not _fileitem or not _content or not _path:
return
tmp_file = settings.TEMP_PATH / _path.name
tmp_file.write_bytes(_content)
logger.info(f"保存文件:【{_fileitem.storage}{_path}")
_fileitem.path = str(_path.parent)
self.storagechain.upload_file(fileitem=_fileitem, path=tmp_file)
item = self.storagechain.upload_file(fileitem=_fileitem, path=tmp_file)
if item:
logger.info(f"已保存文件:{item.path}")
if tmp_file.exists():
tmp_file.unlink()
@@ -356,6 +359,7 @@ class MediaChain(ChainBase, metaclass=Singleton):
logger.info(f"{_url} 图片下载失败,请检查网络连通性!")
except Exception as err:
logger.error(f"{_url} 图片下载失败:{str(err)}")
return None
# 当前文件路径
filepath = Path(fileitem.path)
@@ -410,7 +414,8 @@ class MediaChain(ChainBase, metaclass=Singleton):
# 下载图片
content = __download_image(_url=attr_value)
# 写入图片到当前目录
__save_file(_fileitem=fileitem, _path=image_path, _content=content)
if content:
__save_file(_fileitem=fileitem, _path=image_path, _content=content)
else:
# 电视剧
if fileitem.type == "file":
@@ -447,7 +452,8 @@ class MediaChain(ChainBase, metaclass=Singleton):
# 下载图片
content = __download_image(image_url)
# 保存图片文件到当前目录
__save_file(_fileitem=parent, _path=image_path, _content=content)
if content:
__save_file(_fileitem=parent, _path=image_path, _content=content)
else:
# 当前为目录,处理目录内的文件
@@ -463,7 +469,7 @@ class MediaChain(ChainBase, metaclass=Singleton):
season_meta = MetaInfo(filepath.name)
if season_meta.begin_season:
# 当前目录有季号生成季nfo
season_nfo = self.metadata_nfo(meta=meta, mediainfo=mediainfo, season=meta.begin_season)
season_nfo = self.metadata_nfo(meta=meta, mediainfo=mediainfo, season=season_meta.begin_season)
if not season_nfo:
logger.warn(f"无法生成电视剧季nfo文件{meta.name}")
return
@@ -485,7 +491,8 @@ class MediaChain(ChainBase, metaclass=Singleton):
# 下载图片
content = __download_image(image_url)
# 保存图片文件到当前目录
__save_file(_fileitem=fileitem, _path=image_path, _content=content)
if content:
__save_file(_fileitem=fileitem, _path=image_path, _content=content)
# 判断当前目录是不是剧集根目录
if season_meta.name:
# 当前目录有名称生成tvshow nfo 和 tv图片
@@ -511,6 +518,7 @@ class MediaChain(ChainBase, metaclass=Singleton):
# 下载图片
content = __download_image(image_url)
# 保存图片文件到当前目录
__save_file(_fileitem=fileitem, _path=image_path, _content=content)
if content:
__save_file(_fileitem=fileitem, _path=image_path, _content=content)
logger.info(f"{filepath.name} 刮削完成")

View File

@@ -3,12 +3,12 @@ from typing import List, Union, Optional, Generator
from cachetools import cached, TTLCache
from app import schemas
from app.chain import ChainBase
from app.core.config import global_vars
from app.db.mediaserver_oper import MediaServerOper
from app.helper.service import ServiceConfigHelper
from app.log import logger
from app.schemas import MediaServerLibrary, MediaServerItem, MediaServerSeasonInfo, MediaServerPlayItem
lock = threading.Lock()
@@ -22,7 +22,7 @@ class MediaServerChain(ChainBase):
super().__init__()
self.dboper = MediaServerOper()
def librarys(self, server: str, username: str = None, hidden: bool = False) -> List[schemas.MediaServerLibrary]:
def librarys(self, server: str, username: str = None, hidden: bool = False) -> List[MediaServerLibrary]:
"""
获取媒体服务器所有媒体库
"""
@@ -70,25 +70,25 @@ class MediaServerChain(ChainBase):
yield from self.run_module("mediaserver_items", server=server, library_id=library_id,
start_index=start_index, limit=limit)
def iteminfo(self, server: str, item_id: Union[str, int]) -> schemas.MediaServerItem:
def iteminfo(self, server: str, item_id: Union[str, int]) -> MediaServerItem:
"""
获取媒体服务器项目信息
"""
return self.run_module("mediaserver_iteminfo", server=server, item_id=item_id)
def episodes(self, server: str, item_id: Union[str, int]) -> List[schemas.MediaServerSeasonInfo]:
def episodes(self, server: str, item_id: Union[str, int]) -> List[MediaServerSeasonInfo]:
"""
获取媒体服务器剧集信息
"""
return self.run_module("mediaserver_tv_episodes", server=server, item_id=item_id)
def playing(self, server: str, count: int = 20, username: str = None) -> List[schemas.MediaServerPlayItem]:
def playing(self, server: str, count: int = 20, username: str = None) -> List[MediaServerPlayItem]:
"""
获取媒体服务器正在播放信息
"""
return self.run_module("mediaserver_playing", count=count, server=server, username=username)
def latest(self, server: str, count: int = 20, username: str = None) -> List[schemas.MediaServerPlayItem]:
def latest(self, server: str, count: int = 20, username: str = None) -> List[MediaServerPlayItem]:
"""
获取媒体服务器最新入库条目
"""

View File

@@ -37,6 +37,12 @@ class StorageChain(ChainBase):
"""
return self.run_module("list_files", fileitem=fileitem, recursion=recursion)
def any_files(self, fileitem: schemas.FileItem, extensions: list = None) -> Optional[bool]:
"""
查询当前目录下是否存在指定扩展名任意文件
"""
return self.run_module("any_files", fileitem=fileitem, extensions=extensions)
def create_folder(self, fileitem: schemas.FileItem, name: str) -> Optional[schemas.FileItem]:
"""
创建目录
@@ -51,13 +57,15 @@ class StorageChain(ChainBase):
"""
return self.run_module("download_file", fileitem=fileitem, path=path)
def upload_file(self, fileitem: schemas.FileItem, path: Path) -> Optional[bool]:
def upload_file(self, fileitem: schemas.FileItem, path: Path,
new_name: str = None) -> Optional[schemas.FileItem]:
"""
上传文件
:param fileitem: 保存目录项
:param path: 本地文件路径
:param new_name: 新文件名
"""
return self.run_module("upload_file", fileitem=fileitem, path=path)
return self.run_module("upload_file", fileitem=fileitem, path=path, new_name=new_name)
def delete_file(self, fileitem: schemas.FileItem) -> Optional[bool]:
"""
@@ -105,29 +113,27 @@ class StorageChain(ChainBase):
"""
删除媒体文件,以及不含媒体文件的目录
"""
if fileitem.path == "/" or len(Path(fileitem.path).parts) <= 2:
logger.warn(f"{fileitem.storage}{fileitem.path} 根目录或一级目录不允许删除")
return False
logger.warn(f"正在删除【{fileitem.storage}{fileitem.path}")
state = self.delete_file(fileitem)
if not state:
logger.warn(f"{fileitem.storage}{fileitem.path} 删除失败")
return False
if fileitem.type == "dir":
# 本身是目录不处理父目录
return True
# 上级目录
if mtype and mtype == MediaType.TV:
dir_path = Path(fileitem.path).parent.parent
dir_item = self.get_file_item(storage=fileitem.storage, path=dir_path)
dir_item = self.get_file_item(storage=fileitem.storage, path=Path(fileitem.path).parent.parent)
else:
dir_item = self.get_parent_item(fileitem)
if dir_item:
files = self.list_files(dir_item, recursion=True)
# 是否存在其他媒体文件
media_file_exist = False
if files:
for file in files:
if file.extension and f".{file.extension.lower()}" in settings.RMT_MEDIAEXT:
media_file_exist = True
break
if dir_item and len(Path(dir_item.path).parts) > 2:
# 不存在其他媒体文件,删除空目录
if not media_file_exist:
# 返回空目录删除状态
exts = settings.RMT_MEDIAEXT + settings.DOWNLOAD_TMPEXT
if self.any_files(dir_item, extensions=exts) is False:
logger.warn(f"{dir_item.storage}{dir_item.path} 不存在其它媒体文件,删除空目录")
return self.delete_file(dir_item)
# 存在媒体文件,返回文件删除状态

View File

@@ -159,6 +159,8 @@ class SubscribeChain(ChainBase):
"search_imdbid") else kwargs.get("search_imdbid"),
'sites': self.__get_default_subscribe_config(mediainfo.type, "sites") or None if not kwargs.get(
"sites") else kwargs.get("sites"),
'downloader': self.__get_default_subscribe_config(mediainfo.type, "downloader") if not kwargs.get(
"downloader") else kwargs.get("downloader"),
'save_path': self.__get_default_subscribe_config(mediainfo.type, "save_path") if not kwargs.get(
"save_path") else kwargs.get("save_path")
})
@@ -394,7 +396,8 @@ class SubscribeChain(ChainBase):
userid=subscribe.username,
username=subscribe.username,
save_path=subscribe.save_path,
media_category=subscribe.media_category
media_category=subscribe.media_category,
downloader=subscribe.downloader,
)
# 判断是否应完成订阅
@@ -773,7 +776,8 @@ class SubscribeChain(ChainBase):
userid=subscribe.username,
username=subscribe.username,
save_path=subscribe.save_path,
media_category=subscribe.media_category)
media_category=subscribe.media_category,
downloader=subscribe.downloader)
# 判断是否要完成订阅
self.finish_subscribe_or_not(subscribe=subscribe, meta=meta, mediainfo=mediainfo,
downloads=downloads, lefts=lefts)

View File

@@ -1,6 +1,5 @@
import json
import re
from pathlib import Path
from typing import Union
from app.chain import ChainBase
@@ -10,6 +9,7 @@ from app.schemas import Notification, MessageChannel
from app.utils.http import RequestUtils
from app.utils.singleton import Singleton
from app.utils.system import SystemUtils
from version import FRONTEND_VERSION, APP_VERSION
class SystemChain(ChainBase, metaclass=Singleton):
@@ -98,77 +98,67 @@ class SystemChain(ChainBase, metaclass=Singleton):
@staticmethod
def __get_server_release_version():
"""
获取后端最新版本
获取后端V2最新版本
"""
try:
version_res = RequestUtils(proxies=settings.PROXY, headers=settings.GITHUB_HEADERS).get_res(
"https://api.github.com/repos/jxxghp/MoviePilot/releases/latest")
if version_res:
ver_json = version_res.json()
version = f"{ver_json['tag_name']}"
return version
# 获取所有发布的版本列表
response = RequestUtils(
proxies=settings.PROXY,
headers=settings.GITHUB_HEADERS
).get_res("https://api.github.com/repos/jxxghp/MoviePilot/releases")
if response:
releases = [release['tag_name'] for release in response.json()]
v2_releases = [tag for tag in releases if re.match(r"^v2\.", tag)]
if not v2_releases:
logger.warn("获取v2后端最新版本版本出错")
else:
# 找到最新的v2版本
latest_v2 = sorted(v2_releases, key=lambda s: list(map(int, re.findall(r'\d+', s))))[-1]
logger.info(f"获取到后端最新版本:{latest_v2}")
return latest_v2
else:
return None
logger.error("无法获取后端版本信息请检查网络连接或GitHub API请求。")
except Exception as err:
logger.error(f"获取后端最新版本失败:{str(err)}")
return None
return None
@staticmethod
def __get_front_release_version():
"""
获取前端最新版本
获取前端V2最新版本
"""
try:
version_res = RequestUtils(proxies=settings.PROXY, headers=settings.GITHUB_HEADERS).get_res(
"https://api.github.com/repos/jxxghp/MoviePilot-Frontend/releases/latest")
if version_res:
ver_json = version_res.json()
version = f"{ver_json['tag_name']}"
return version
# 获取所有发布的版本列表
response = RequestUtils(
proxies=settings.PROXY,
headers=settings.GITHUB_HEADERS
).get_res("https://api.github.com/repos/jxxghp/MoviePilot-Frontend/releases")
if response:
releases = [release['tag_name'] for release in response.json()]
v2_releases = [tag for tag in releases if re.match(r"^v2\.", tag)]
if not v2_releases:
logger.warn("获取v2前端最新版本版本出错")
else:
# 找到最新的v2版本
latest_v2 = sorted(v2_releases, key=lambda s: list(map(int, re.findall(r'\d+', s))))[-1]
logger.info(f"获取到前端最新版本:{latest_v2}")
return latest_v2
else:
return None
logger.error("无法获取前端版本信息请检查网络连接或GitHub API请求。")
except Exception as err:
logger.error(f"获取前端最新版本失败:{str(err)}")
return None
return None
@staticmethod
def get_server_local_version():
"""
查看当前版本
"""
version_file = settings.ROOT_PATH / "version.py"
if version_file.exists():
try:
with open(version_file, 'rb') as f:
version = f.read()
pattern = r"'([^']*)'"
match = re.search(pattern, str(version))
if match:
version = match.group(1)
return version
else:
logger.warn("未找到版本号")
return None
except Exception as err:
logger.error(f"加载版本文件 {version_file} 出错:{str(err)}")
return APP_VERSION
@staticmethod
def get_frontend_version():
"""
获取前端版本
"""
if SystemUtils.is_frozen() and SystemUtils.is_windows():
version_file = settings.CONFIG_PATH.parent / "nginx" / "html" / "version.txt"
else:
version_file = Path(settings.FRONTEND_PATH) / "version.txt"
if version_file.exists():
try:
with open(version_file, 'r') as f:
version = str(f.read()).strip()
return version
except Exception as err:
logger.error(f"加载版本文件 {version_file} 出错:{str(err)}")
else:
logger.warn("未找到前端版本文件,请正确设置 FRONTEND_PATH")
return None
return FRONTEND_VERSION

View File

@@ -120,6 +120,7 @@ class TorrentsChain(ChainBase, metaclass=Singleton):
site_ua=site.get("ua") or settings.USER_AGENT,
site_proxy=site.get("proxy"),
site_order=site.get("pri"),
site_downloader=site.get("downloader"),
title=item.get("title"),
enclosure=item.get("enclosure"),
page_url=item.get("link"),

View File

@@ -120,7 +120,7 @@ class TransferChain(ChainBase):
# 非MoviePilot下载的任务按文件识别
mediainfo = None
# 执行整理
# 执行整理,匹配源目录
state, errmsg = self.__do_transfer(
fileitem=FileItem(
storage="local",
@@ -131,7 +131,8 @@ class TransferChain(ChainBase):
extension=file_path.suffix.lstrip('.'),
),
mediainfo=mediainfo,
download_hash=torrent.hash
download_hash=torrent.hash,
src_match=True
)
# 设置下载任务状态
@@ -148,7 +149,8 @@ class TransferChain(ChainBase):
target_storage: str = None, target_path: Path = None,
transfer_type: str = None, scrape: bool = None,
season: int = None, epformat: EpisodeFormat = None,
min_filesize: int = 0, download_hash: str = None, force: bool = False) -> Tuple[bool, str]:
min_filesize: int = 0, download_hash: str = None,
force: bool = False, src_match: bool = False) -> Tuple[bool, str]:
"""
执行一个复杂目录的整理操作
:param fileitem: 文件项
@@ -164,6 +166,7 @@ class TransferChain(ChainBase):
:param min_filesize: 最小文件大小(MB)
:param download_hash: 下载记录hash
:param force: 是否强制整理
:param src_match: 是否源目录匹配
返回:成功标识,错误信息
"""
@@ -202,6 +205,8 @@ class TransferChain(ChainBase):
skip_num = 0
# 本次整理方式
current_transfer_type = transfer_type
# 是否全部成功
all_success = True
# 获取待整理路径清单
trans_items = self.__get_trans_fileitems(fileitem)
@@ -281,6 +286,7 @@ class TransferChain(ChainBase):
# 计数
processed_num += 1
skip_num += 1
all_success = False
continue
# 整理成功的不再处理
@@ -291,6 +297,7 @@ class TransferChain(ChainBase):
# 计数
processed_num += 1
skip_num += 1
all_success = False
continue
# 更新进度
@@ -314,6 +321,7 @@ class TransferChain(ChainBase):
# 计数
processed_num += 1
fail_num += 1
all_success = False
continue
# 自定义识别
@@ -350,6 +358,7 @@ class TransferChain(ChainBase):
# 计数
processed_num += 1
fail_num += 1
all_success = False
continue
# 如果未开启新增已入库媒体是否跟随TMDB信息变化则根据tmdbid查询之前的title
@@ -379,6 +388,17 @@ class TransferChain(ChainBase):
if download_file:
download_hash = download_file.download_hash
# 查询整理目标目录
if not target_directory:
if target_path:
target_directory = self.directoryhelper.get_dir(file_mediainfo,
storage=target_storage, dest_path=target_path)
elif src_match:
target_directory = self.directoryhelper.get_dir(file_mediainfo,
storage=file_item.storage, src_path=file_path)
else:
target_directory = self.directoryhelper.get_dir(file_mediainfo)
# 执行整理
transferinfo: TransferInfo = self.transfer(fileitem=file_item,
meta=file_meta,
@@ -416,6 +436,7 @@ class TransferChain(ChainBase):
# 计数
processed_num += 1
fail_num += 1
all_success = False
continue
# 汇总信息
@@ -486,15 +507,21 @@ class TransferChain(ChainBase):
})
# 移动模式处理
if current_transfer_type in ["move"]:
if all_success and current_transfer_type in ["move"]:
# 下载器hash
if download_hash:
if self.remove_torrents(download_hash):
logger.info(f"移动模式删除种子成功:{download_hash} ")
# 删除残留文件
# 删除残留目录
if fileitem:
logger.warn(f"删除残留文件夹:【{fileitem.storage}{fileitem.path}")
self.storagechain.delete_file(fileitem)
if fileitem.type == "dir":
folder_item = fileitem
else:
folder_item = self.storagechain.get_parent_item(fileitem)
exts = settings.RMT_MEDIAEXT + settings.DOWNLOAD_TMPEXT
if folder_item and self.storagechain.any_files(folder_item, extensions=exts) is False:
logger.warn(f"删除残留空文件夹:【{folder_item.storage}{folder_item.path}")
self.storagechain.delete_file(folder_item)
# 结束进度
logger.info(f"{fileitem.path} 整理完成,共 {total_num} 个文件,"

View File

@@ -69,6 +69,8 @@ class ConfigModel(BaseModel):
DB_MAX_OVERFLOW: int = 500
# SQLite 的 busy_timeout 参数,默认为 60 秒
DB_TIMEOUT: int = 60
# SQLite 是否启用 WAL 模式,默认关闭
DB_WAL_ENABLE: bool = False
# 配置文件目录
CONFIG_DIR: Optional[str] = None
# 超级管理员
@@ -151,6 +153,8 @@ class ConfigModel(BaseModel):
SEARCH_MULTIPLE_NAME: bool = False
# 站点数据刷新间隔(小时)
SITEDATA_REFRESH_INTERVAL: int = 6
# 读取和发送站点消息
SITE_MESSAGE: bool = True
# 种子标签
TORRENT_TAG: str = "MOVIEPILOT"
# 下载站点字幕

View File

@@ -23,6 +23,8 @@ class TorrentInfo:
site_proxy: bool = False
# 站点优先级
site_order: int = 0
# 站点下载器
site_downloader: str = None
# 种子名称
title: str = None
# 种子副标题

View File

@@ -499,11 +499,18 @@ class EventManager(metaclass=Singleton):
def decorator(f: Callable):
# 将输入的事件类型统一转换为列表格式
if isinstance(etype, list):
event_list = etype # 传入的已经是列表,直接使用
# 传入的已经是列表,直接使用
event_list = etype
elif etype is EventType:
# 订阅所有事件
event_list = []
for et in etype:
event_list.append(et)
else:
event_list = [etype] # 不是列表则包裹成单一元素的列表
# 不是列表则包裹成单一元素的列表
event_list = [etype]
# 遍历列表,处理每个事件类型
# 遍历列表,处理每个事件类型
for event in event_list:
if isinstance(event, (EventType, ChainEventType)):
self.add_event_listener(event, f)

View File

@@ -30,8 +30,8 @@ class MetaVideo(MetaBase):
_episode_re = r"EP?(\d{2,4})$|^EP?(\d{1,4})$|^S\d{1,2}EP?(\d{1,4})$|S\d{2}EP?(\d{2,4})"
_part_re = r"(^PART[0-9ABI]{0,2}$|^CD[0-9]{0,2}$|^DVD[0-9]{0,2}$|^DISK[0-9]{0,2}$|^DISC[0-9]{0,2}$)"
_roman_numerals = r"^(?=[MDCLXVI])M*(C[MD]|D?C{0,3})(X[CL]|L?X{0,3})(I[XV]|V?I{0,3})$"
_source_re = r"^BLURAY$|^HDTV$|^UHDTV$|^HDDVD$|^WEBRIP$|^DVDRIP$|^BDRIP$|^BLU$|^WEB$|^BD$|^HDRip$"
_effect_re = r"^REMUX$|^UHD$|^SDR$|^HDR\d*$|^DOLBY$|^DOVI$|^DV$|^3D$|^REPACK$"
_source_re = r"^BLURAY$|^HDTV$|^UHDTV$|^HDDVD$|^WEBRIP$|^DVDRIP$|^BDRIP$|^BLU$|^WEB$|^BD$|^HDRip$|^REMUX$|^UHD$"
_effect_re = r"^SDR$|^HDR\d*$|^DOLBY$|^DOVI$|^DV$|^3D$|^REPACK$"
_resources_type_re = r"%s|%s" % (_source_re, _effect_re)
_name_no_begin_re = r"^[\[【].+?[\]】]"
_name_no_chinese_re = r".*版|.*字幕"
@@ -542,13 +542,27 @@ class MetaVideo(MetaBase):
elif token.upper() == "RAY" \
and self._last_token_type == "source" \
and self._last_token == "BLU":
self._source = "BluRay"
# UHD BluRay组合
if self._source == "UHD":
self._source = "UHD BluRay"
else:
self._source = "BluRay"
self._continue_flag = False
return
elif token.upper() == "WEBDL":
self._source = "WEB-DL"
self._continue_flag = False
return
# UHD REMUX组合
if token.upper() == "REMUX" \
and self._source == "BluRay":
self._source = "BluRay REMUX"
self._continue_flag = False
return
elif token.upper() == "BLURAY" \
and self._source == "UHD":
self._source = "UHD BluRay"
effect_res = re.search(r"(%s)" % self._effect_re, token, re.IGNORECASE)
if effect_res:
self._last_token_type = "effect"

View File

@@ -1,22 +1,25 @@
from typing import Any, Generator, List, Optional, Self, Tuple
from sqlalchemy import NullPool, QueuePool, and_, create_engine, inspect
from sqlalchemy import NullPool, QueuePool, and_, create_engine, inspect, text
from sqlalchemy.orm import Session, as_declarative, declared_attr, scoped_session, sessionmaker
from app.core.config import settings
# 根据池类型设置 poolclass 和相关参数
pool_class = NullPool if settings.DB_POOL_TYPE == "NullPool" else QueuePool
connect_args = {
"timeout": settings.DB_TIMEOUT
}
# 启用 WAL 模式时的额外配置
if settings.DB_WAL_ENABLE:
connect_args["check_same_thread"] = False
kwargs = {
"url": f"sqlite:///{settings.CONFIG_PATH}/user.db",
"pool_pre_ping": settings.DB_POOL_PRE_PING,
"echo": settings.DB_ECHO,
"poolclass": pool_class,
"pool_recycle": settings.DB_POOL_RECYCLE,
"connect_args": {
# "check_same_thread": False,
"timeout": settings.DB_TIMEOUT
}
"connect_args": connect_args
}
# 当使用 QueuePool 时,添加 QueuePool 特有的参数
if pool_class == QueuePool:
@@ -27,6 +30,11 @@ if pool_class == QueuePool:
})
# 创建数据库引擎
Engine = create_engine(**kwargs)
# 根据配置设置日志模式
journal_mode = "WAL" if settings.DB_WAL_ENABLE else "DELETE"
with Engine.connect() as connection:
current_mode = connection.execute(text(f"PRAGMA journal_mode={journal_mode};")).scalar()
print(f"Database journal mode set to: {current_mode}")
# 会话工厂
SessionFactory = sessionmaker(bind=Engine)
@@ -49,11 +57,34 @@ def get_db() -> Generator:
db.close()
def perform_checkpoint(mode: str = "PASSIVE"):
"""
执行 SQLite 的 checkpoint 操作,将 WAL 文件内容写回主数据库
:param mode: checkpoint 模式,可选值包括 "PASSIVE""FULL""RESTART""TRUNCATE"
默认为 "PASSIVE",即不锁定 WAL 文件的轻量级同步
"""
if not settings.DB_WAL_ENABLE:
return
valid_modes = {"PASSIVE", "FULL", "RESTART", "TRUNCATE"}
if mode.upper() not in valid_modes:
raise ValueError(f"Invalid checkpoint mode '{mode}'. Must be one of {valid_modes}")
try:
# 使用指定的 checkpoint 模式,确保 WAL 文件数据被正确写回主数据库
with Engine.connect() as conn:
conn.execute(text(f"PRAGMA wal_checkpoint({mode.upper()});"))
except Exception as e:
print(f"Error during WAL checkpoint: {e}")
def close_database():
"""
关闭所有数据库连接
关闭所有数据库连接并清理资源
"""
Engine.dispose()
try:
# 释放连接池SQLite 会自动清空 WAL 文件,这里不单独再调用 checkpoint
Engine.dispose()
except Exception as e:
print(f"Error while disposing database connections: {e}")
def get_args_db(args: tuple, kwargs: dict) -> Optional[Session]:

View File

@@ -51,6 +51,8 @@ class Site(Base):
is_active = Column(Boolean(), default=True)
# 创建时间
lst_mod_date = Column(String, default=datetime.now().strftime("%Y-%m-%d %H:%M:%S"))
# 下载器
downloader = Column(String)
@staticmethod
@db_query

View File

@@ -64,6 +64,8 @@ class Subscribe(Base):
username = Column(String)
# 订阅站点
sites = Column(JSON, default=list)
# 下载器
downloader = Column(String)
# 是否洗版
best_version = Column(Integer, default=0)
# 当前优先级

View File

@@ -4,7 +4,7 @@ from typing import List, Optional
from app import schemas
from app.core.context import MediaInfo
from app.db.systemconfig_oper import SystemConfigOper
from app.schemas.types import SystemConfigKey, MediaType
from app.schemas.types import SystemConfigKey
class DirectoryHelper:
@@ -48,40 +48,43 @@ class DirectoryHelper:
"""
return [d for d in self.get_library_dirs() if d.library_storage == "local"]
def get_dir(self, media: MediaInfo, src_path: Path = None, dest_path: Path = None,
fileitem: schemas.FileItem = None, local: bool = False) -> Optional[schemas.TransferDirectoryConf]:
def get_dir(self, media: MediaInfo, storage: str = "local",
src_path: Path = None, dest_path: Path = None, fileitem: schemas.FileItem = None
) -> Optional[schemas.TransferDirectoryConf]:
"""
根据媒体信息获取下载目录、媒体库目录配置
:param media: 媒体信息
:param storage: 存储类型
:param src_path: 源目录,有值时直接匹配
:param dest_path: 目标目录,有值时直接匹配
:param fileitem: 文件项,使用文件路径匹配
:param local: 是否本地目录
"""
# 处理类型
if media:
media_type = media.type.value
else:
media_type = MediaType.UNKNOWN.value
if not media:
return None
# 电影/电视剧
media_type = media.type.value
dirs = self.get_dirs()
# 按照配置顺序查找
for d in dirs:
# 没有启用整理的目录
if not d.monitor_type:
continue
# 存储类型不匹配
if storage and d.storage != storage:
continue
# 下载目录
download_path = Path(d.download_path)
# 媒体库目录
library_path = Path(d.library_path)
# 下载目录不匹配, 不符合条件, 通常处理`下载`匹配
if src_path and download_path != src_path:
# 有源目录时,源目录不匹配下载目录
if src_path and not src_path.is_relative_to(download_path):
continue
# 媒体库目录不匹配, 或监控方式为None(即不自动整理), 不符合条件, 通常处理`整理`匹配
if dest_path:
if library_path != dest_path or not d.monitor_type:
continue
# 没有目录配置时起作用, 通常处理`手动整理`未选择`目标目录`的情况
# 有文件项时,文件项不匹配下载目录
if fileitem and not Path(fileitem.path).is_relative_to(download_path):
continue
# 本地目录
if local and d.storage != "local":
# 有目标目录时,目标目录不匹配媒体库目录
if dest_path and not dest_path.is_relative_to(library_path):
continue
# 目录类型为全部的,符合条件
if not d.media_type:

View File

@@ -9,17 +9,27 @@ class FormatParser(object):
_split_chars = r"\.|\s+|\(|\)|\[|]|-|\+|【|】|/||;|&|\||#|_|「|」|~"
def __init__(self, eformat: str, details: str = None, part: str = None,
offset: int = None, key: str = "ep"):
offset: str = None, key: str = "ep"):
"""
:params eformat: 格式化字符串
:params details: 格式化详情
:params part: 分集
:params offset: 偏移量
:params offset: 偏移量 -10/EP*2
:prams key: EP关键字
"""
self._format = eformat
self._start_ep = None
self._end_ep = None
if not offset:
self.__offset = "EP"
elif "EP" in offset:
self.__offset = offset
else:
if offset.startswith("-") or offset.startswith("+"):
self.__offset = f"EP{offset}"
else:
self.__offset = f"EP+{offset}"
self._key = key
self._part = None
if part:
self._part = part
@@ -34,8 +44,6 @@ class FormatParser(object):
self._end_ep = int(tmp[0]) if int(tmp[0]) > int(tmp[1]) else int(tmp[1])
else:
self._start_ep = self._end_ep = int(tmp[0])
self.__offset = int(offset) if offset else 0
self._key = key
@property
def format(self):
@@ -77,15 +85,21 @@ class FormatParser(object):
if self._start_ep is not None and self._start_ep == self._end_ep:
if isinstance(self._start_ep, str):
s, e = self._start_ep.split("-")
start_ep = self.__offset.replace("EP", s)
end_ep = self.__offset.replace("EP", e)
if int(s) == int(e):
return int(s) + self.__offset, None, self.part
return int(s) + self.__offset, int(e) + self.__offset, self.part
return self._start_ep + self.__offset, None, self.part
return int(eval(start_ep)), None, self.part
return int(eval(start_ep)), int(eval(end_ep)), self.part
else:
start_ep = self.__offset.replace("EP", str(self._start_ep))
return int(eval(start_ep)), None, self.part
if not self._format:
return self._start_ep, self._end_ep, self.part
s, e = self.__handle_single(file_name)
return s + self.__offset if s is not None else None, \
e + self.__offset if e is not None else None, self.part
else:
s, e = self.__handle_single(file_name)
start_ep = self.__offset.replace("EP", str(s)) if s else None
end_ep = self.__offset.replace("EP", str(e)) if e else None
return int(eval(start_ep)) if start_ep else None, int(eval(end_ep)) if end_ep else None, self.part
def __handle_single(self, file: str) -> Tuple[Optional[int], Optional[int]]:
"""

View File

@@ -225,12 +225,13 @@ class RssHelper:
}
@staticmethod
def parse(url, proxy: bool = False, timeout: int = 15) -> Union[List[dict], None]:
def parse(url, proxy: bool = False, timeout: int = 15, headers: dict = None) -> Union[List[dict], None]:
"""
解析RSS订阅URL获取RSS中的种子信息
:param url: RSS地址
:param proxy: 是否使用代理
:param timeout: 请求超时
:param headers: 自定义请求头
:return: 种子信息列表如为None代表Rss过期
"""
# 开始处理
@@ -238,7 +239,8 @@ class RssHelper:
if not url:
return []
try:
ret = RequestUtils(proxies=settings.PROXY if proxy else None, timeout=timeout).get_res(url)
ret = RequestUtils(proxies=settings.PROXY if proxy else None,
timeout=timeout, headers=headers).get_res(url)
if not ret:
return []
except Exception as err:

View File

@@ -10,7 +10,7 @@ from app.log import logger
class TwoFactorAuth:
def __init__(self, code_or_secret: str):
if code_or_secret and len(code_or_secret) > 16:
if code_or_secret and len(code_or_secret) >= 16:
self.code = None
self.secret = code_or_secret
else:

View File

@@ -52,7 +52,7 @@ class DoubanCache(metaclass=Singleton):
获取缓存KEY
"""
return f"[{meta.type.value if meta.type else '未知'}]" \
f"{meta.name or meta.doubanid}-{meta.year}-{meta.begin_season}"
f"{meta.doubanid or meta.name}-{meta.year}-{meta.begin_season}"
def get(self, meta: MetaBase):
"""

View File

@@ -65,9 +65,8 @@ class FileManagerModule(_ModuleBase):
"""
测试模块连接性
"""
directoryhelper = DirectoryHelper()
# 检查目录
dirs = directoryhelper.get_dirs()
dirs = self.directoryhelper.get_dirs()
if not dirs:
return False, "未设置任何目录"
for d in dirs:
@@ -197,6 +196,35 @@ class FileManagerModule(_ModuleBase):
return result
def any_files(self, fileitem: FileItem, extensions: list = None) -> Optional[bool]:
"""
查询当前目录下是否存在指定扩展名任意文件
"""
storage_oper = self.__get_storage_oper(fileitem.storage)
if not storage_oper:
logger.error(f"不支持 {fileitem.storage} 的文件浏览")
return None
def __any_file(_item: FileItem):
"""
递归处理
"""
_items = storage_oper.list(_item)
if _items:
if not extensions:
return True
for t in _items:
if (t.type == "file"
and t.extension
and f".{t.extension.lower()}" in extensions):
return True
elif t.type == "dir":
return __any_file(t)
return False
# 返回结果
return __any_file(fileitem)
def create_folder(self, fileitem: FileItem, name: str) -> Optional[FileItem]:
"""
创建目录
@@ -240,7 +268,7 @@ class FileManagerModule(_ModuleBase):
return None
return storage_oper.download(fileitem, path=path)
def upload_file(self, fileitem: FileItem, path: Path) -> Optional[FileItem]:
def upload_file(self, fileitem: FileItem, path: Path, new_name: str = None) -> Optional[FileItem]:
"""
上传文件
"""
@@ -248,7 +276,7 @@ class FileManagerModule(_ModuleBase):
if not storage_oper:
logger.error(f"不支持 {fileitem.storage} 的上传处理")
return None
return storage_oper.upload(fileitem, path)
return storage_oper.upload(fileitem, path, new_name)
def get_file_item(self, storage: str, path: Path) -> Optional[FileItem]:
"""
@@ -320,14 +348,6 @@ class FileManagerModule(_ModuleBase):
fileitem=fileitem,
message=f"{target_path} 不是有效目录")
# 获取目标路径
directoryhelper = DirectoryHelper()
if not target_directory:
# 根据目的路径查找目录配置
if target_path:
target_directory = directoryhelper.get_dir(mediainfo, dest_path=target_path)
else:
target_directory = directoryhelper.get_dir(mediainfo, fileitem=fileitem)
if target_directory:
# 拼装媒体库一、二级子目录
target_path = self.__get_dest_dir(mediainfo=mediainfo, target_dir=target_directory)
@@ -337,6 +357,11 @@ class FileManagerModule(_ModuleBase):
# 整理方式
if not transfer_type:
transfer_type = target_directory.transfer_type
if not transfer_type:
logger.error(f"{target_directory.name} 未设置整理方式")
return TransferInfo(success=False,
fileitem=fileitem,
message=f"{target_directory.name} 未设置整理方式")
# 是否需要刮削
if scrape is None:
need_scrape = target_directory.scraping
@@ -349,7 +374,7 @@ class FileManagerModule(_ModuleBase):
# 覆盖模式
overwrite_mode = target_directory.overwrite_mode
elif target_path:
# 自定义目标路径,仅适用于手动整理的场景
# 手动整理的场景,有自定义目标路径
need_scrape = scrape or False
need_rename = True
need_notify = False
@@ -445,6 +470,8 @@ class FileManagerModule(_ModuleBase):
state = source_oper.link(fileitem, target_file)
elif transfer_type == "softlink":
state = source_oper.softlink(fileitem, target_file)
else:
return None, f"不支持的整理方式:{transfer_type}"
if state:
return __get_targetitem(target_file), ""
else:
@@ -460,7 +487,7 @@ class FileManagerModule(_ModuleBase):
target_fileitem = target_oper.get_folder(target_file.parent)
if target_fileitem:
# 上传文件
new_item = target_oper.upload(target_fileitem, filepath)
new_item = target_oper.upload(target_fileitem, filepath, target_file.name)
if new_item:
return new_item, ""
else:
@@ -473,7 +500,7 @@ class FileManagerModule(_ModuleBase):
target_fileitem = target_oper.get_folder(target_file.parent)
if target_fileitem:
# 上传文件
new_item = target_oper.upload(target_fileitem, filepath)
new_item = target_oper.upload(target_fileitem, filepath, target_file.name)
if new_item:
# 删除源文件
source_oper.delete(fileitem)
@@ -586,18 +613,16 @@ class FileManagerModule(_ModuleBase):
if not parent_item:
return False, f"{org_path} 上级目录获取失败"
# 字幕文件列表
file_list: List[FileItem] = storage_oper.list(parent_item)
file_list: List[FileItem] = storage_oper.list(parent_item) or []
file_list = [f for f in file_list if f.type == "file" and f.extension
and f".{f.extension.lower()}" in settings.RMT_SUBEXT]
if len(file_list) == 0:
logger.debug(f"{parent_item.path} 目录下没有找到字幕文件...")
logger.info(f"{parent_item.path} 目录下没有找到字幕文件...")
else:
logger.debug("字幕文件清单:" + str(file_list))
logger.info(f"字幕文件清单:{[f.name for f in file_list]}")
# 识别文件名
metainfo = MetaInfoPath(org_path)
for sub_item in file_list:
if sub_item.type == "dir" or not sub_item.extension:
continue
if f".{sub_item.extension.lower()}" not in settings.RMT_SUBEXT:
continue
# 识别字幕文件名
sub_file_name = re.sub(_zhtw_sub_re,
".",

View File

@@ -28,6 +28,13 @@ class StorageBase(metaclass=ABCMeta):
"""
return self.storagehelper.get_storage(self.schema.value)
def get_conf(self) -> dict:
"""
获取配置
"""
conf = self.get_config()
return conf.config if conf else {}
def set_config(self, conf: dict):
"""
设置配置
@@ -112,11 +119,12 @@ class StorageBase(metaclass=ABCMeta):
pass
@abstractmethod
def upload(self, fileitem: schemas.FileItem, path: Path) -> Optional[schemas.FileItem]:
def upload(self, fileitem: schemas.FileItem, path: Path, new_name: str = None) -> Optional[schemas.FileItem]:
"""
上传文件
:param fileitem: 上传目录项
:param path: 本地文件路径
:param new_name: 上传后文件名
"""
pass

View File

@@ -353,7 +353,7 @@ class AliPan(StorageBase):
return Path(local_path)
return None
def upload(self, fileitem: schemas.FileItem, path: Path) -> Optional[schemas.FileItem]:
def upload(self, fileitem: schemas.FileItem, path: Path, new_name: str = None) -> Optional[schemas.FileItem]:
"""
上传文件,并标记完成
"""
@@ -361,7 +361,7 @@ class AliPan(StorageBase):
return None
# 上传文件
result = self.aligo.upload_file(file_path=str(path), parent_file_id=fileitem.fileid,
drive_id=fileitem.drive_id, name=path.name,
drive_id=fileitem.drive_id, name=new_name or path.name,
check_name_mode="refuse")
if result:
item = self.aligo.get_file(file_id=result.file_id, drive_id=result.drive_id)

View File

@@ -0,0 +1,765 @@
import json
import logging
from datetime import datetime
from pathlib import Path
from typing import Optional, Tuple, List, Dict, Union
from requests import Response
from cachetools import cached, TTLCache
from app import schemas
from app.core.config import settings
from app.log import logger
from app.modules.filemanager.storages import StorageBase
from app.schemas.types import StorageSchema
from app.utils.http import RequestUtils
from app.utils.url import UrlUtils
class Alist(StorageBase):
"""
Alist相关操作
api文档https://alist.nn.ci/zh/guide/api
"""
# 存储类型
schema = StorageSchema.Alist
# 支持的整理方式
transtype = {
"copy": "复制",
"move": "移动",
}
def __init__(self):
super().__init__()
@property
def __get_base_url(self) -> str:
"""
获取基础URL
"""
url = self.get_conf().get("url")
if url is None:
return ""
return UrlUtils.standardize_base_url(self.get_conf().get("url"))
def __get_api_url(self, path: str) -> str:
"""
获取API URL
"""
return UrlUtils.adapt_request_url(self.__get_base_url, path)
@property
def __get_valuable_toke(self) -> str:
"""
获取一个可用的token
如果设置永久令牌则返回永久令牌
否则使用账号密码生成临时令牌
"""
token = self.get_conf().get("token")
if token:
return token
return self.__generate_token
@property
@cached(cache=TTLCache(maxsize=1, ttl=60 * 60 * 24 * 2 - 60 * 5))
def __generate_token(self) -> str:
"""
使用账号密码生成一个临时token
缓存2天提前5分钟更新
"""
conf = self.get_conf()
resp: Response = RequestUtils(headers={
'Content-Type': 'application/json'
}).post_res(
self.__get_api_url("/api/auth/login"),
data=json.dumps({
"username": conf.get("username"),
"password": conf.get("password"),
}),
)
"""
{
"username": "{{alist_username}}",
"password": "{{alist_password}}"
}
======================================
{
"code": 200,
"message": "success",
"data": {
"token": "abcd"
}
}
"""
if resp is None:
logger.warning("请求登录失败无法连接alist服务")
return ""
if resp.status_code != 200:
logger.warning(f"更新令牌请求发送失败,状态码:{resp.status_code}")
return ""
result = resp.json()
if result["code"] != 200:
logger.critical(f'更新令牌,错误信息:{result["message"]}')
return ""
logger.debug("AList获取令牌成功")
return result["data"]["token"]
def __get_header_with_token(self) -> dict:
"""
获取带有token的header
"""
return {"Authorization": self.__get_valuable_toke}
def check(self) -> bool:
"""
检查存储是否可用
"""
pass
def list(
self,
fileitem: schemas.FileItem,
password: str = "",
page: int = 1,
per_page: int = 0,
refresh: bool = False,
) -> Optional[List[schemas.FileItem]]:
"""
浏览文件
:param fileitem: 文件项
:param password: 路径密码
:param page: 页码
:param per_page: 每页数量
:param refresh: 是否刷新
"""
if fileitem.type == "file":
item = self.get_item(Path(fileitem.path))
if item:
return [item]
return None
resp: Response = RequestUtils(
headers=self.__get_header_with_token()
).post_res(
self.__get_api_url("/api/fs/list"),
json={
"path": fileitem.path,
"password": password,
"page": page,
"per_page": per_page,
"refresh": refresh,
},
)
"""
{
"path": "/t",
"password": "",
"page": 1,
"per_page": 0,
"refresh": false
}
======================================
{
"code": 200,
"message": "success",
"data": {
"content": [
{
"name": "Alist V3.md",
"size": 1592,
"is_dir": false,
"modified": "2024-05-17T13:47:55.4174917+08:00",
"created": "2024-05-17T13:47:47.5725906+08:00",
"sign": "",
"thumb": "",
"type": 4,
"hashinfo": "null",
"hash_info": null
}
],
"total": 1,
"readme": "",
"header": "",
"write": true,
"provider": "Local"
}
}
"""
if resp is None:
logging.warning(f"请求获取目录 {fileitem.path} 的文件列表失败无法连接alist服务")
return
if resp.status_code != 200:
logging.warning(
f"请求获取目录 {fileitem.path} 的文件列表失败,状态码:{resp.status_code}"
)
return
result = resp.json()
if result["code"] != 200:
logging.warning(
f'获取目录 {fileitem.path} 的文件列表失败,错误信息:{result["message"]}'
)
return
return [
schemas.FileItem(
storage=self.schema.value,
type="dir" if item["is_dir"] else "file",
path=(Path(fileitem.path) / item["name"]).as_posix() + ("/" if item["is_dir"] else ""),
name=item["name"],
basename=Path(item["name"]).stem,
extension=Path(item["name"]).suffix,
size=item["size"],
modify_time=self.__parse_timestamp(item["modified"]),
thumbnail=item["thumb"],
)
for item in result["data"]["content"] or []
]
def create_folder(
self, fileitem: schemas.FileItem, name: str
) -> Optional[schemas.FileItem]:
"""
创建目录
"""
path = Path(fileitem.path) / name
resp: Response = RequestUtils(
headers=self.__get_header_with_token()
).post_res(
self.__get_api_url("/api/fs/mkdir"),
json={"path": path.as_posix()},
)
"""
{
"path": "/tt"
}
======================================
{
"code": 200,
"message": "success",
"data": null
}
"""
if resp is None:
logging.warning(f"请求创建目录 {path} 失败无法连接alist服务")
return
if resp.status_code != 200:
logging.warning(f"请求创建目录 {path} 失败,状态码:{resp.status_code}")
return
result = resp.json()
if result["code"] != 200:
logging.warning(f'创建目录 {path} 失败,错误信息:{result["message"]}')
return
return self.get_item(path)
def get_folder(self, path: Path) -> Optional[schemas.FileItem]:
"""
获取目录,如目录不存在则创建
"""
folder = self.get_item(path)
if not folder:
folder = self.create_folder(self.get_parent(schemas.FileItem(
storage=self.schema.value,
type="dir",
path=path.as_posix() + "/",
name=path.name,
basename=path.stem
)), path.name)
return folder
def get_item(
self,
path: Path,
password: str = "",
page: int = 1,
per_page: int = 0,
refresh: bool = False,
) -> Optional[schemas.FileItem]:
"""
获取文件或目录不存在返回None
:param path: 文件路径
:param password: 路径密码
:param page: 页码
:param per_page: 每页数量
:param refresh: 是否刷新
"""
resp: Response = RequestUtils(
headers=self.__get_header_with_token()
).post_res(
self.__get_api_url("/api/fs/get"),
json={
"path": path.as_posix(),
"password": password,
"page": page,
"per_page": per_page,
"refresh": refresh,
},
)
"""
{
"path": "/t",
"password": "",
"page": 1,
"per_page": 0,
"refresh": false
}
======================================
{
"code": 200,
"message": "success",
"data": {
"name": "Alist V3.md",
"size": 2618,
"is_dir": false,
"modified": "2024-05-17T16:05:36.4651534+08:00",
"created": "2024-05-17T16:05:29.2001008+08:00",
"sign": "",
"thumb": "",
"type": 4,
"hashinfo": "null",
"hash_info": null,
"raw_url": "http://127.0.0.1:5244/p/local/Alist%20V3.md",
"readme": "",
"header": "",
"provider": "Local",
"related": null
}
}
"""
if resp is None:
logging.warning(f"请求获取文件 {path} 失败无法连接alist服务")
return
if resp.status_code != 200:
logging.warning(f"请求获取文件 {path} 失败,状态码:{resp.status_code}")
return
result = resp.json()
if result["code"] != 200:
logging.warning(f'获取文件 {path} 失败,错误信息:{result["message"]}')
return
return schemas.FileItem(
storage=self.schema.value,
type="dir" if result["data"]["is_dir"] else "file",
path=path.as_posix() + ("/" if result["data"]["is_dir"] else ""),
name=result["data"]["name"],
basename=Path(result["data"]["name"]).stem,
extension=Path(result["data"]["name"]).suffix,
size=result["data"]["size"],
modify_time=self.__parse_timestamp(result["data"]["modified"]),
thumbnail=result["data"]["thumb"],
)
def get_parent(self, fileitem: schemas.FileItem) -> Optional[schemas.FileItem]:
"""
获取父目录
"""
return self.get_folder(Path(fileitem.path).parent)
def delete(self, fileitem: schemas.FileItem) -> bool:
"""
删除文件
"""
resp: Response = RequestUtils(
headers=self.__get_header_with_token()
).post_res(
self.__get_api_url("/api/fs/delete"),
json={
"dir": Path(fileitem.path).parent.as_posix(),
"names": [fileitem.name],
},
)
"""
{
"names": [
"string"
],
"dir": "string"
}
======================================
{
"code": 200,
"message": "success",
"data": null
}
"""
if resp is None:
logging.warning(f"请求删除文件 {fileitem.path} 失败无法连接alist服务")
return False
if resp.status_code != 200:
logging.warning(
f"请求删除文件 {fileitem.path} 失败,状态码:{resp.status_code}"
)
return False
result = resp.json()
if result["code"] != 200:
logging.warning(
f'删除文件 {fileitem.path} 失败,错误信息:{result["message"]}'
)
return False
return True
def rename(self, fileitem: schemas.FileItem, name: str) -> bool:
"""
重命名文件
"""
resp: Response = RequestUtils(
headers=self.__get_header_with_token()
).post_res(
self.__get_api_url("/api/fs/rename"),
json={
"name": name,
"path": fileitem.path,
},
)
"""
{
"name": "test3",
"path": "/阿里云盘/test2"
}
======================================
{
"code": 200,
"message": "success",
"data": null
}
"""
if not resp:
logging.warning(f"请求重命名文件 {fileitem.path} 失败无法连接alist服务")
return False
if resp.status_code != 200:
logging.warning(
f"请求重命名文件 {fileitem.path} 失败,状态码:{resp.status_code}"
)
return False
result = resp.json()
if result["code"] != 200:
logging.warning(
f'重命名文件 {fileitem.path} 失败,错误信息:{result["message"]}'
)
return False
return True
def download(
self,
fileitem: schemas.FileItem,
path: Path = None,
password: str = "",
) -> Optional[Path]:
"""
下载文件,保存到本地,返回本地临时文件地址
:param fileitem: 文件项
:param path: 文件保存路径
:param password: 文件密码
"""
resp: Response = RequestUtils(
headers=self.__get_header_with_token()
).post_res(
self.__get_api_url("/api/fs/get"),
json={
"path": fileitem.path,
"password": password,
"page": 1,
"per_page": 0,
"refresh": False,
},
)
"""
{
"code": 200,
"message": "success",
"data": {
"name": "[ANi]輝夜姬想讓人告白~天才們的戀愛頭腦戰~[01][1080P][Baha][WEB-DL].mp4",
"size": 924933111,
"is_dir": false,
"modified": "1970-01-01T00:00:00Z",
"created": "1970-01-01T00:00:00Z",
"sign": "1v0xkMQz_uG8fkEOQ7-l58OnbB-g4GkdBlUBcrsApCQ=:0",
"thumb": "",
"type": 2,
"hashinfo": "null",
"hash_info": null,
"raw_url": "xxxxxx",
"readme": "",
"header": "",
"provider": "UrlTree",
"related": null
}
}
"""
if not resp:
logging.warning(f"请求获取文件 {path} 失败无法连接alist服务")
return
if resp.status_code != 200:
logging.warning(f"请求获取文件 {path} 失败,状态码:{resp.status_code}")
return
result = resp.json()
if result["code"] != 200:
logging.warning(f'获取文件 {path} 失败,错误信息:{result["message"]}')
return
if result["data"]["raw_url"]:
download_url = result["data"]["raw_url"]
else:
download_url = UrlUtils.adapt_request_url(self.__get_base_url, f"/d{fileitem.path}")
if result["data"]["sign"]:
download_url = download_url + "?sign=" + result["data"]["sign"]
resp = RequestUtils(
headers=self.__get_header_with_token()
).get_res(download_url)
if not path:
path = settings.TEMP_PATH / fileitem.name
with open(path, "wb") as f:
f.write(resp.content)
if path.exists():
return path
return None
def upload(
self, fileitem: schemas.FileItem, path: Path, new_name: str = None, task: bool = False
) -> Optional[schemas.FileItem]:
"""
上传文件
:param fileitem: 上传目录项
:param path: 本地文件路径
:param new_name: 上传后文件名
:param task: 是否为任务默认为False避免未完成上传时对文件进行操作
"""
encoded_path = UrlUtils.quote(fileitem.path)
headers = self.__get_header_with_token()
headers.setdefault("Content-Type", "multipart/form-data")
headers.setdefault("As-Task", str(task).lower())
headers.setdefault("File-Path", encoded_path)
with open(path, "rb") as f:
resp: Response = RequestUtils(headers=headers).put_res(
self.__get_api_url("/api/fs/form"),
data={"file": f},
)
if resp.status_code != 200:
logging.warning(f"请求上传文件 {path} 失败,状态码:{resp.status_code}")
return
if new_name and new_name != path.name:
if self.rename(fileitem, new_name):
return self.get_item(Path(fileitem.path).parent / new_name)
return self.get_item(Path(fileitem.path) / path.name)
def detail(self, fileitem: schemas.FileItem) -> Optional[schemas.FileItem]:
"""
获取文件详情
"""
return self.get_item(Path(fileitem.path))
@staticmethod
def __get_copy_and_move_data(
fileitem: schemas.FileItem, target: Union[schemas.FileItem, Path]
) -> Tuple[str, str, List[str], bool]:
"""
获取复制或移动文件需要的数据
:param fileitem: 文件项
:param target: 目标文件项或目标路径
:return: 源目录,目标目录,文件名列表,是否有效
"""
name = Path(target).name
if fileitem.name != name:
return "", "", [], False
src_dir = Path(fileitem.path).parent.as_posix()
if isinstance(target, schemas.FileItem):
traget_dir = Path(target.path).parent.as_posix()
else:
traget_dir = target.parent.as_posix()
return src_dir, traget_dir, [name], True
def copy(
self, fileitem: schemas.FileItem, target: Union[schemas.FileItem, Path]
) -> bool:
"""
复制文件
源文件名和目标文件名必须相同
"""
src_dir, dst_dir, names, is_valid = self.__get_copy_and_move_data(
fileitem, target
)
if not is_valid:
return False
resp: Response = RequestUtils(
headers=self.__get_header_with_token()
).post_res(
self.__get_api_url("/api/fs/copy"),
json={
"src_dir": src_dir,
"dst_dir": dst_dir,
"names": names,
},
)
"""
{
"src_dir": "string",
"dst_dir": "string",
"names": [
"string"
]
}
======================================
{
"code": 200,
"message": "success",
"data": null
}
"""
if resp is None:
logging.warning(
f"请求复制文件 {fileitem.path} 失败无法连接alist服务"
)
return False
if resp.status_code != 200:
logging.warning(
f"请求复制文件 {fileitem.path} 失败,状态码:{resp.status_code}"
)
return False
result = resp.json()
if result["code"] != 200:
logging.warning(
f'复制文件 {fileitem.path} 失败,错误信息:{result["message"]}'
)
return False
return True
def move(
self, fileitem: schemas.FileItem, target: Union[schemas.FileItem, Path]
) -> bool:
"""
移动文件
"""
src_dir, dst_dir, names, is_valid = self.__get_copy_and_move_data(
fileitem, target
)
if not is_valid:
return False
resp: Response = RequestUtils(
headers=self.__get_header_with_token()
).post_res(
self.__get_api_url("/api/fs/move"),
json={
"src_dir": src_dir,
"dst_dir": dst_dir,
"names": names,
},
)
"""
{
"src_dir": "string",
"dst_dir": "string",
"names": [
"string"
]
}
======================================
{
"code": 200,
"message": "success",
"data": null
}
"""
if resp is None:
logging.warning(
f"请求移动文件 {fileitem.path} 失败无法连接alist服务"
)
return False
if resp.status_code != 200:
logging.warning(
f"请求移动文件 {fileitem.path} 失败,状态码:{resp.status_code}"
)
return False
result = resp.json()
if result["code"] != 200:
logging.warning(
f'移动文件 {fileitem.path} 失败,错误信息:{result["message"]}'
)
return False
return True
def link(self, fileitem: schemas.FileItem, target_file: Path) -> bool:
"""
硬链接文件
"""
pass
def softlink(self, fileitem: schemas.FileItem, target_file: Path) -> bool:
"""
软链接文件
"""
pass
def usage(self) -> Optional[schemas.StorageUsage]:
"""
存储使用情况
"""
pass
def snapshot(self, path: Path) -> Dict[str, float]:
"""
快照文件系统,输出所有层级文件信息(不含目录)
"""
files_info = {}
def __snapshot_file(_fileitm: schemas.FileItem):
"""
递归获取文件信息
"""
if _fileitm.type == "dir":
for sub_file in self.list(_fileitm):
__snapshot_file(sub_file)
else:
files_info[_fileitm.path] = _fileitm.size
fileitem = self.get_item(path)
if not fileitem:
return {}
__snapshot_file(fileitem)
return files_info
@staticmethod
def __parse_timestamp(time_str: str) -> float:
# try:
# # 尝试解析带微秒的时间格式
# dt = datetime.strptime(time_str[:26], '%Y-%m-%dT%H:%M:%S.%f')
# except ValueError:
# # 如果失败,尝试解析不带微秒的时间格式
# dt = datetime.strptime(time_str, '%Y-%m-%dT%H:%M:%SZ')
# 直接使用 ISO 8601 格式解析时间
dt = datetime.fromisoformat(time_str)
# 返回时间戳
return dt.timestamp()

View File

@@ -183,12 +183,12 @@ class LocalStorage(StorageBase):
"""
return Path(fileitem.path)
def upload(self, fileitem: schemas.FileItem, path: Path) -> Optional[schemas.FileItem]:
def upload(self, fileitem: schemas.FileItem, path: Path, new_name: str = None) -> Optional[schemas.FileItem]:
"""
上传文件
"""
dir_path = Path(fileitem.path)
target_path = dir_path / path.name
target_path = dir_path / (new_name or path.name)
code, message = SystemUtils.move(path, target_path)
if code != 0:
logger.error(f"移动文件失败:{message}")
@@ -222,7 +222,7 @@ class LocalStorage(StorageBase):
软链接文件
"""
file_path = Path(fileitem.path)
code, message = SystemUtils.copy(file_path, target_file)
code, message = SystemUtils.softlink(file_path, target_file)
if code != 0:
logger.error(f"软链接文件失败:{message}")
return False

View File

@@ -39,7 +39,7 @@ class Rclone(StorageBase):
path = Path(filepath)
if not path.parent.exists():
path.parent.mkdir(parents=True)
path.write_text(conf.get('content'))
path.write_text(conf.get('content'), encoding='utf-8')
@staticmethod
def __get_hidden_shell():
@@ -76,7 +76,7 @@ class Rclone(StorageBase):
return schemas.FileItem(
storage=self.schema.value,
type="dir",
path=f"{parent}{item.get('Name')}",
path=f"{parent}{item.get('Name')}" + "/",
name=item.get("Name"),
basename=item.get("Name"),
modify_time=StringUtils.str_to_timestamp(item.get("ModTime"))
@@ -260,7 +260,7 @@ class Rclone(StorageBase):
logger.error(f"rclone复制文件失败{err}")
return None
def upload(self, fileitem: schemas.FileItem, path: Path) -> Optional[schemas.FileItem]:
def upload(self, fileitem: schemas.FileItem, path: Path, new_name: str = None) -> Optional[schemas.FileItem]:
"""
上传文件
"""
@@ -269,7 +269,7 @@ class Rclone(StorageBase):
[
'rclone', 'copyto',
str(path),
f'MP:{Path(fileitem.path) / path.name}'
f'MP:{Path(fileitem.path) / (new_name or path.name)}'
],
startupinfo=self.__get_hidden_shell()
).returncode
@@ -331,11 +331,24 @@ class Rclone(StorageBase):
"""
存储使用情况
"""
conf = self.get_config()
if not conf:
return None
file_path = conf.config.get("filepath")
if not file_path or not Path(file_path).exists():
return None
# 读取rclone文件检查是否有[MP]节点配置
with open(file_path, "r", encoding="utf-8") as f:
lines = f.readlines()
if not lines:
return None
if not any("[MP]" in line.strip() for line in lines):
return None
try:
ret = subprocess.run(
[
'rclone', 'about',
'/', '--json'
'MP:/', '--json'
],
capture_output=True,
startupinfo=self.__get_hidden_shell()

View File

@@ -328,7 +328,7 @@ class U115Pan(StorageBase, metaclass=Singleton):
logger.error(f"115下载失败{str(e)}")
return None
def upload(self, fileitem: schemas.FileItem, path: Path) -> Optional[schemas.FileItem]:
def upload(self, fileitem: schemas.FileItem, path: Path, new_name: str = None) -> Optional[schemas.FileItem]:
"""
上传文件
"""
@@ -359,7 +359,7 @@ class U115Pan(StorageBase, metaclass=Singleton):
if result:
result_data = result.get('data')
logger.info(f"115上传文件成功{result_data.get('file_name')}")
return schemas.FileItem(
item = schemas.FileItem(
storage=self.schema.value,
fileid=result_data.get('file_id'),
parent_fileid=fileitem.fileid,
@@ -371,6 +371,13 @@ class U115Pan(StorageBase, metaclass=Singleton):
extension=Path(result_data.get('file_name')).suffix[1:],
pickcode=result_data.get('pickcode')
)
if new_name and new_name != item.name:
if self.rename(item, new_name):
item.name = new_name
item.basename = Path(new_name).stem
item.path = f"{fileitem.path}{new_name}"
item.extension = Path(new_name).suffix[1:]
return item
else:
logger.warn(f"115上传文件失败{por.resp.response.text}")
return None

View File

@@ -432,26 +432,32 @@ class FilterModule(_ModuleBase):
@staticmethod
def __match_size(torrent: TorrentInfo, size_range: str) -> bool:
"""
判断种子是否匹配大小范围MB
判断种子是否匹配大小范围MB,剧集拆分为每集大小
"""
if not size_range:
return True
# 集数
meta = MetaInfo(title=torrent.title, subtitle=torrent.description)
episode_count = meta.total_episode or 1
# 每集大小
torrent_size = torrent.size / episode_count
# 大小范围
size_range = size_range.strip()
if size_range.find("-") != -1:
# 区间
size_min, size_max = size_range.split("-")
size_min = float(size_min.strip()) * 1024 * 1024
size_max = float(size_max.strip()) * 1024 * 1024
if size_min <= torrent.size <= size_max:
if size_min <= torrent_size <= size_max:
return True
elif size_range.startswith(">"):
# 大于
size_min = float(size_range[1:].strip()) * 1024 * 1024
if torrent.size >= size_min:
if torrent_size >= size_min:
return True
elif size_range.startswith("<"):
# 小于
size_max = float(size_range[1:].strip()) * 1024 * 1024
if torrent.size <= size_max:
if torrent_size <= size_max:
return True
return False

View File

@@ -191,6 +191,7 @@ class IndexerModule(_ModuleBase):
site_ua=site.get("ua"),
site_proxy=site.get("proxy"),
site_order=site.get("pri"),
site_downloader=site.get("downloader"),
**result) for result in result_array]
# 去重
return __remove_duplicate(torrents)
@@ -199,7 +200,7 @@ class IndexerModule(_ModuleBase):
def __spider_search(indexer: CommentedMap,
search_word: str = None,
mtype: MediaType = None,
page: int = 0) -> (bool, List[dict]):
page: int = 0) -> Tuple[bool, List[dict]]:
"""
根据关键字搜索单个站点
:param: indexer: 站点配置

View File

@@ -182,7 +182,8 @@ class SiteParserBase(metaclass=ABCMeta):
)
)
# 解析用户未读消息
self._pase_unread_msgs()
if settings.SITE_MESSAGE:
self._pase_unread_msgs()
# 解析用户上传、下载、分享率等信息
if self._user_traffic_page:
self._parse_user_traffic_info(

View File

@@ -1,21 +1,60 @@
# -*- coding: utf-8 -*-
import re
import json
from typing import Optional
from lxml import etree
from urllib.parse import urljoin
from app.log import logger
from app.modules.indexer.parser import SiteSchema
from app.modules.indexer.parser.nexus_php import NexusPhpSiteUserInfo
from app.modules.indexer.parser import SiteParserBase
from app.utils.string import StringUtils
class NexusRabbitSiteUserInfo(NexusPhpSiteUserInfo):
class NexusRabbitSiteUserInfo(SiteParserBase):
schema = SiteSchema.NexusRabbit
def _parse_site_page(self, html_text: str):
super()._parse_site_page(html_text)
self._torrent_seeding_page = f"getusertorrentlistajax.php?page=1&limit=5000000&type=seeding&uid={self.userid}"
self._torrent_seeding_headers = {"Accept": "application/json, text/javascript, */*; q=0.01"}
html_text = self._prepare_html_text(html_text)
def _parse_user_torrent_seeding_info(self, html_text: str, multi_page: bool = False) -> Optional[str]:
user_detail = re.search(r"user.php\?id=(\d+)", html_text)
if not (user_detail and user_detail.group().strip()):
return
self.userid = user_detail.group(1)
self._user_detail_page = f"user.php?id={self.userid}"
self._user_traffic_page = None
self._torrent_seeding_page = "api/general"
self._torrent_seeding_params = {
"page": 1,
"limit": 5000000,
"action": "userTorrentsList",
"data": {"type": "seeding", "id": int(self.userid)},
}
self._torrent_seeding_headers = {
"Content-Type": "application/json",
"Accept": "application/json, text/plain, */*",
"X-Requested-With": "XMLHttpRequest", # 必须要加上这一条,不然返回的是空数据
}
self._user_mail_unread_page = None
self._sys_mail_unread_page = "api/general"
self._mail_unread_params = {
"page": 1,
"limit": 5000000,
"action": "getMessageIn",
}
self._mail_unread_headers = {
"Content-Type": "application/json",
"Accept": "application/json, text/plain, */*",
"X-Requested-With": "XMLHttpRequest",
}
def _parse_user_torrent_seeding_info(
self, html_text: str, multi_page: bool = False
) -> Optional[str]:
"""
做种相关信息
:param html_text:
@@ -24,22 +63,112 @@ class NexusRabbitSiteUserInfo(NexusPhpSiteUserInfo):
"""
try:
torrents = json.loads(html_text).get('data')
torrents = json.loads(html_text).get("data", [])
except Exception as e:
logger.error(f"解析做种信息失败: {str(e)}")
return
page_seeding_size = 0
page_seeding_info = []
seeding_size = 0
seeding_info = []
page_seeding = len(torrents)
for torrent in torrents:
seeders = int(torrent.get('seeders', 0))
size = int(torrent.get('size', 0))
page_seeding_size += int(torrent.get('size', 0))
seeders = int(torrent.get("seeders", 0))
size = StringUtils.num_filesize(torrent.get("size"))
seeding_size += size
seeding_info.append([seeders, size])
page_seeding_info.append([seeders, size])
self.seeding = len(torrents)
self.seeding_size = seeding_size
self.seeding_info = seeding_info
self.seeding += page_seeding
self.seeding_size += page_seeding_size
self.seeding_info.extend(page_seeding_info)
def _parse_message_unread_links(
self, html_text: str, msg_links: list
) -> str | None:
unread_ids = []
try:
messages = json.loads(html_text).get("data", [])
except Exception as e:
logger.error(f"解析未读消息失败: {e}")
return
for msg in messages:
msg_id, msg_unread = msg.get("id"), msg.get("unread")
if not (msg_id and msg_unread) or msg_unread == "no":
continue
unread_ids.append(msg_id)
head, date, content = msg.get("subject"), msg.get("added"), msg.get("msg")
if head and date and content:
self.message_unread_contents.append((head, date, content))
self.message_unread = len(unread_ids)
if unread_ids:
self._get_page_content(
url=urljoin(self._base_url, "api/general?loading=true"),
params={"action": "readMessage", "data": {"ids": unread_ids}},
headers={
"Content-Type": "application/json",
"Accept": "application/json, text/plain, */*",
"X-Requested-With": "XMLHttpRequest",
},
)
return None
def _parse_user_base_info(self, html_text: str):
"""只有奶糖余额才需要在 base 中获取,其它均可以在详情页拿到"""
html = etree.HTML(html_text)
if not StringUtils.is_valid_html_element(html):
return
bonus = html.xpath(
'//div[contains(text(), "奶糖余额")]/following-sibling::div[1]/text()'
)
if bonus:
self.bonus = StringUtils.str_float(bonus[0].strip())
def _parse_user_detail_info(self, html_text: str):
html = etree.HTML(html_text)
if not StringUtils.is_valid_html_element(html):
return
# 缩小一下查找范围,所有的信息都在这个 div 里
user_info = html.xpath('//div[contains(@class, "layui-hares-user-info-right")]')
if not user_info:
return
user_info = user_info[0]
# 用户名
if username := user_info.xpath(
'.//span[contains(text(), "用户名")]/a/span/text()'
):
self.username = username[0].strip()
# 等级
if user_level := user_info.xpath('.//span[contains(text(), "等级")]/b/text()'):
self.user_level = user_level[0].strip()
# 加入日期
if join_date := user_info.xpath('.//span[contains(text(), "注册日期")]/text()'):
join_date = join_date[0].strip().split("\r")[0].removeprefix("注册日期:")
self.join_at = StringUtils.unify_datetime_str(join_date)
# 上传量
if upload := user_info.xpath('.//span[contains(text(), "上传量")]/text()'):
self.upload = StringUtils.num_filesize(
upload[0].strip().removeprefix("上传量:")
)
# 下载量
if download := user_info.xpath('.//span[contains(text(), "下载量")]/text()'):
self.download = StringUtils.num_filesize(
download[0].strip().removeprefix("下载量:")
)
# 分享率
if ratio := user_info.xpath('.//span[contains(text(), "分享率")]/em/text()'):
self.ratio = StringUtils.str_float(ratio[0].strip())
def _parse_message_content(self, html_text):
"""
解析短消息内容,已经在 _parse_message_unread_links 内实现,重载防止 abstractmethod 报错
:param html_text:
:return: head: message, date: time, content: message content
"""
pass
def _parse_user_traffic_info(self, html_text: str):
"""
解析用户的上传,下载,分享率等信息,已经在 _parse_user_detail_info 内实现,重载防止 abstractmethod 报错
:param html_text:
:return:
"""
pass

View File

@@ -36,7 +36,10 @@ class TNodeSiteUserInfo(SiteParserBase):
pass
def _parse_user_detail_info(self, html_text: str):
detail = json.loads(html_text)
try:
detail = json.loads(html_text)
except json.JSONDecodeError:
return
if detail.get("status") != 200:
return

View File

@@ -468,6 +468,30 @@ class Jellyfin:
return None
return None
def get_item_path_by_id(self, item_id: str) -> Optional[str]:
"""
根据ItemId查询所在的Path
:param item_id: 在Jellyfin中的ID
:return: Path
"""
if not self._host or not self._apikey:
return None
url = f"{self._host}Items/{item_id}/PlaybackInfo"
params = {"api_key": self._apikey}
try:
res = RequestUtils(timeout=10).get_res(url, params)
if res:
media_sources = res.json().get("MediaSources")
if media_sources:
return media_sources[0].get("Path")
else:
logger.error("Items/Id/PlaybackInfo 未获取到返回数据,不设置 Path")
return None
except Exception as e:
logger.error("连接Items/Id/PlaybackInfo出错" + str(e))
return None
return None
def generate_image_link(self, item_id: str, image_type: str, host_type: bool) -> Optional[str]:
"""
根据ItemId和imageType查询本地对应图片
@@ -662,6 +686,8 @@ class Jellyfin:
item_id=eventItem.item_id,
image_type="Backdrop"
)
# jellyfin 的 webhook 不含 item_path需要单独获取
eventItem.item_path = self.get_item_path_by_id(eventItem.item_id)
return eventItem

View File

@@ -162,26 +162,26 @@ class Plex:
def get_medias_count(self) -> schemas.Statistic:
"""
获得电影、电视剧、动漫媒体数量
:return: MovieCount SeriesCount SongCount
:return: movie_count tv_count episode_count
"""
if not self._plex:
return schemas.Statistic()
sections = self._plex.library.sections()
MovieCount = SeriesCount = EpisodeCount = 0
movie_count = tv_count = episode_count = 0
# 媒体库白名单
allow_library = [lib.id for lib in self.get_librarys(hidden=True)]
for sec in sections:
if str(sec.key) not in allow_library:
if sec.key not in allow_library:
continue
if sec.type == "movie":
MovieCount += sec.totalSize
movie_count += sec.totalSize
if sec.type == "show":
SeriesCount += sec.totalSize
EpisodeCount += sec.totalViewSize(libtype='episode')
tv_count += sec.totalSize
episode_count += sec.totalViewSize(libtype="episode")
return schemas.Statistic(
movie_count=MovieCount,
tv_count=SeriesCount,
episode_count=EpisodeCount
movie_count=movie_count,
tv_count=tv_count,
episode_count=episode_count
)
def get_movies(self,
@@ -721,7 +721,7 @@ class Plex:
if not self._plex:
return []
# 媒体库白名单
allow_library = ",".join([lib.id for lib in self.get_librarys(hidden=True)])
allow_library = ",".join(map(str, (lib.id for lib in self.get_librarys(hidden=True))))
params = {"contentDirectoryID": allow_library}
items = self._plex.fetchItems("/hubs/continueWatching/items",
container_start=0,
@@ -757,7 +757,7 @@ class Plex:
if not self._plex:
return None
# 请求参数(除黑名单)
allow_library = ",".join([lib.id for lib in self.get_librarys(hidden=True)])
allow_library = ",".join(map(str, (lib.id for lib in self.get_librarys(hidden=True))))
params = {
"contentDirectoryID": allow_library,
"count": num,

View File

@@ -1,4 +1,5 @@
import time
import traceback
from typing import Optional, Union, Tuple, List
import qbittorrentapi
@@ -75,8 +76,13 @@ class Qbittorrent:
REQUESTS_ARGS={'timeout': (15, 60)})
try:
qbt.auth_log_in()
except qbittorrentapi.LoginFailed as e:
except (qbittorrentapi.LoginFailed, qbittorrentapi.Forbidden403Error) as e:
logger.error(f"qbittorrent 登录失败:{str(e)}")
return None
except Exception as e:
stack_trace = "".join(traceback.format_exception(None, e, e.__traceback__))[:2000]
logger.error(f"qbittorrent 登录失败:{str(e)}\n{stack_trace}")
return None
return qbt
except Exception as err:
logger.error(f"qbittorrent 连接出错:{str(err)}")

View File

@@ -50,7 +50,7 @@ class TmdbCache(metaclass=Singleton):
"""
获取缓存KEY
"""
return f"[{meta.type.value if meta.type else '未知'}]{meta.name or meta.tmdbid}-{meta.year}-{meta.begin_season}"
return f"[{meta.type.value if meta.type else '未知'}]{meta.tmdbid or meta.name}-{meta.year}-{meta.begin_season}"
def get(self, meta: MetaBase):
"""

View File

@@ -263,19 +263,17 @@ class Monitor(metaclass=Singleton):
try:
item = self._queue.get(timeout=self._transfer_interval)
if item:
self.__handle_file(storage=item.get("storage"),
event_path=item.get("filepath"),
mon_path=item.get("mon_path"))
self.__handle_file(storage=item.get("storage"), event_path=item.get("filepath"))
except queue.Empty:
continue
except Exception as e:
logger.error(f"整理队列处理出现错误:{e}")
def __handle_file(self, storage: str, event_path: Path, mon_path: Path):
def __handle_file(self, storage: str, event_path: Path):
"""
整理一个文件
:param storage: 存储
:param event_path: 事件文件路径
:param mon_path: 监控目录
"""
def __get_bluray_dir(_path: Path):
@@ -386,7 +384,7 @@ class Monitor(metaclass=Singleton):
return
# 查询转移目的目录
dir_info = self.directoryhelper.get_dir(mediainfo, src_path=mon_path)
dir_info = self.directoryhelper.get_dir(mediainfo, storage=storage, src_path=event_path)
if not dir_info:
logger.warn(f"{event_path.name} 未找到对应的目标目录")
return
@@ -480,8 +478,14 @@ class Monitor(metaclass=Singleton):
# 移动模式删除空目录
if transferinfo.transfer_type in ["move"]:
logger.info(f"正在删除: {file_item.storage} {file_item.path}")
self.storagechain.delete_file(file_item)
if file_item.type == "dir":
folder_item = file_item
else:
folder_item = self.storagechain.get_parent_item(file_item)
exts = settings.RMT_MEDIAEXT + settings.DOWNLOAD_TMPEXT
if folder_item and self.storagechain.any_files(folder_item, extensions=exts) is False:
logger.warn(f"删除残留空文件夹:【{folder_item.storage}{folder_item.path}")
self.storagechain.delete_file(folder_item)
except Exception as e:
logger.error("目录监控发生错误:%s - %s" % (str(e), traceback.format_exc()))

View File

@@ -1,4 +1,3 @@
import logging
import threading
import traceback
from datetime import datetime, timedelta
@@ -27,12 +26,6 @@ from app.schemas.types import EventType
from app.utils.singleton import Singleton
from app.utils.timer import TimerUtils
# 获取 apscheduler 的日志记录器
scheduler_logger = logging.getLogger('apscheduler')
# 设置日志级别为 WARNING
scheduler_logger.setLevel(logging.WARNING)
class SchedulerChain(ChainBase):
pass
@@ -436,23 +429,23 @@ class Scheduler(metaclass=Singleton):
try:
sid = f"{service['id']}"
job_id = sid.split("|")[0]
if job_id not in self._jobs:
self._jobs[job_id] = {
"func": service["func"],
"name": service["name"],
"pid": pid,
"plugin_name": plugin_name,
"running": False,
}
self._scheduler.add_job(
self.start,
service["trigger"],
id=sid,
name=service["name"],
**service["kwargs"],
kwargs={"job_id": job_id}
)
logger.info(f"注册插件{plugin_name}服务:{service['name']} - {service['trigger']}")
self._jobs[job_id] = {
"func": service["func"],
"name": service["name"],
"pid": pid,
"plugin_name": plugin_name,
"running": False,
}
self._scheduler.add_job(
self.start,
service["trigger"],
id=sid,
name=service["name"],
**service["kwargs"],
kwargs={"job_id": job_id},
replace_existing=True
)
logger.info(f"注册插件{plugin_name}服务:{service['name']} - {service['trigger']}")
except Exception as e:
logger.error(f"注册插件{plugin_name}服务失败:{str(e)} - {service}")
SchedulerChain().messagehelper.put(title=f"插件 {plugin_name} 服务注册失败",
@@ -468,14 +461,25 @@ class Scheduler(metaclass=Singleton):
with self._lock:
# 获取插件名称
plugin_name = PluginManager().get_plugin_attr(pid, "plugin_name")
for job_id, service in self._jobs.copy().items():
# 先从 _jobs 中查找匹配的服务
jobs_to_remove = [(job_id, service) for job_id, service in self._jobs.items() if service.get("pid") == pid]
if not jobs_to_remove:
return
for job_id, service in jobs_to_remove:
try:
if service.get("pid") == pid:
self._jobs.pop(job_id, None)
try:
self._scheduler.remove_job(job_id)
except JobLookupError:
pass
# 移除服务
self._jobs.pop(job_id, None)
# 在调度器中查找并移除对应的 job
job_removed = False
for job in list(self._scheduler.get_jobs()):
job_id_from_service = job.id.split("|")[0]
if job_id == job_id_from_service:
try:
self._scheduler.remove_job(job.id)
job_removed = True
except JobLookupError:
pass
if job_removed:
logger.info(f"移除插件服务({plugin_name}){service.get('name')}")
except Exception as e:
logger.error(f"移除插件服务失败:{str(e)} - {job_id}: {service}")

View File

@@ -180,6 +180,8 @@ class TorrentInfo(BaseModel):
site_proxy: Optional[bool] = False
# 站点优先级
site_order: Optional[int] = 0
# 站点下载器
site_downloader: Optional[str] = None
# 种子名称
title: Optional[str] = None
# 种子副标题

View File

@@ -1,4 +1,4 @@
from typing import Optional, Any
from typing import Optional, Any, Union
from pydantic import BaseModel
@@ -44,6 +44,8 @@ class Site(BaseModel):
limit_seconds: Optional[int] = None
# 是否启用
is_active: Optional[bool] = True
# 下载器
downloader: Optional[str] = None
class Config:
orm_mode = True
@@ -75,7 +77,7 @@ class SiteUserData(BaseModel):
# 用户名
username: Optional[str]
# 用户ID
userid: Optional[int]
userid: Optional[Union[int, str]]
# 用户等级
user_level: Optional[str]
# 加入时间

View File

@@ -54,6 +54,8 @@ class Subscribe(BaseModel):
username: Optional[str] = None
# 订阅站点
sites: Optional[List[int]] = []
# 下载器
downloader: Optional[str] = None
# 是否洗版
best_version: Optional[int] = 0
# 当前优先级

View File

@@ -83,7 +83,7 @@ class StorageConf(BaseModel):
"""
存储配置
"""
# 类型 local/alipan/u115/rclone
# 类型 local/alipan/u115/rclone/alist
type: Optional[str] = None
# 名称
name: Optional[str] = None

View File

@@ -86,4 +86,4 @@ class EpisodeFormat(BaseModel):
format: Optional[str] = None
detail: Optional[str] = None
part: Optional[str] = None
offset: Optional[int] = None
offset: Optional[str] = None

View File

@@ -187,6 +187,7 @@ class StorageSchema(Enum):
Alipan = "alipan"
U115 = "u115"
Rclone = "rclone"
Alist = "alist"
# 模块类型

View File

@@ -24,7 +24,8 @@ class SiteUtils:
' or contains(@data-url, "logout")'
' or contains(@href, "mybonus") '
' or contains(@onclick, "logout")'
' or contains(@href, "usercp")]',
' or contains(@href, "usercp")'
' or contains(@lay-on, "logout")]',
'//form[contains(@action, "logout")]',
'//div[@class="user-info-side"]',
'//a[@id="myitem"]'

View File

@@ -275,6 +275,10 @@ class SystemUtils:
# 遍历目录
for path in directory.iterdir():
if path.is_dir():
if not SystemUtils.is_windows() and path.name.startswith("."):
continue
if path.name == "@eaDir":
continue
dirs.append(path)
return dirs

View File

@@ -1,6 +1,7 @@
import mimetypes
from pathlib import Path
from typing import Optional, Union
from urllib import parse
from urllib.parse import parse_qs, urlencode, urljoin, urlparse, urlunparse
from app.log import logger
@@ -95,3 +96,14 @@ class UrlUtils:
except Exception as e:
logger.debug(f"Error get_mime_type: {e}")
return default_type
@staticmethod
def quote(s: str) -> str:
"""
将字符串编码为 URL 安全的格式
这将确保路径中的特殊字符(如空格、中文字符等)被正确编码,以便在 URL 中传输
:param s: 要编码的字符串
:return: 编码后的字符串
"""
return parse.quote(s)

View File

@@ -15,6 +15,8 @@ DB_POOL_SIZE=100
DB_MAX_OVERFLOW=500
# SQLite 的 busy_timeout 参数可适当增加如180以减少锁定错误
DB_TIMEOUT=60
# SQLite 是否启用 WAL 模式,启用可提升读写并发性能,但可能在异常情况下增加数据丢失的风险
DB_WAL_ENABLE=false
# 【*】超级管理员,设置后一但重启将固化到数据库中,修改将无效(初始化超级管理员密码仅会生成一次,请在日志中查看并自行登录系统修改)
SUPERUSER=admin
# 辅助认证,允许通过外部服务进行认证、单点登录以及自动创建用户

View File

@@ -64,7 +64,7 @@ def upgrade() -> None:
},
{
"type": "rclone",
"name": "Rclone网盘",
"name": "RClone",
"config": {}
}
])

View File

@@ -0,0 +1,40 @@
"""2.0.6
Revision ID: a295e41830a6
Revises: ecf3c693fdf3
Create Date: 2024-11-14 12:49:13.838120
"""
from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects import sqlite
from app.db.systemconfig_oper import SystemConfigOper
from app.schemas.types import SystemConfigKey
# revision identifiers, used by Alembic.
revision = 'a295e41830a6'
down_revision = 'ecf3c693fdf3'
branch_labels = None
depends_on = None
def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
# 初始化AList存储
_systemconfig = SystemConfigOper()
_storages = _systemconfig.get(SystemConfigKey.Storages)
if _storages:
if "alist" not in [storage["type"] for storage in _storages]:
_storages.append({
"type": "alist",
"name": "AList",
"config": {}
})
_systemconfig.set(SystemConfigKey.Storages, _storages)
# ### end Alembic commands ###
def downgrade() -> None:
pass

View File

@@ -0,0 +1,31 @@
"""2.0.7
Revision ID: eaf9cbc49027
Revises: a295e41830a6
Create Date: 2024-11-16 00:26:09.505188
"""
import contextlib
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision = 'eaf9cbc49027'
down_revision = 'a295e41830a6'
branch_labels = None
depends_on = None
def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
# 站点管理、订阅增加下载器选项
with contextlib.suppress(Exception):
op.add_column('site', sa.Column('downloader', sa.String(), nullable=True))
op.add_column('subscribe', sa.Column('downloader', sa.String(), nullable=True))
# ### end Alembic commands ###
def downgrade() -> None:
pass

View File

@@ -3,7 +3,8 @@
# shellcheck disable=SC2016
# 使用 `envsubst` 将模板文件中的 ${NGINX_PORT} 替换为实际的环境变量值
envsubst '${NGINX_PORT}${PORT}' < /etc/nginx/nginx.template.conf > /etc/nginx/nginx.conf
export NGINX_CLIENT_MAX_BODY_SIZE=${NGINX_CLIENT_MAX_BODY_SIZE:-10m}
envsubst '${NGINX_PORT}${PORT}${NGINX_CLIENT_MAX_BODY_SIZE}' < /etc/nginx/nginx.template.conf > /etc/nginx/nginx.conf
# 自动更新
cd /
/usr/local/bin/mp_update
@@ -21,7 +22,11 @@ chown -R moviepilot:moviepilot \
/var/log/nginx
chown moviepilot:moviepilot /etc/hosts /tmp
# 下载浏览器内核
HTTPS_PROXY="${HTTPS_PROXY:-${https_proxy:-$PROXY_HOST}}" gosu moviepilot:moviepilot playwright install chromium
if [[ "$HTTPS_PROXY" =~ ^https?:// ]] || [[ "$https_proxy" =~ ^https?:// ]] || [[ "$PROXY_HOST" =~ ^https?:// ]]; then
HTTPS_PROXY="${HTTPS_PROXY:-${https_proxy:-$PROXY_HOST}}" gosu moviepilot:moviepilot playwright install chromium
else
gosu moviepilot:moviepilot playwright install chromium
fi
# 启动前端nginx服务
nginx
# 启动docker http proxy nginx

View File

@@ -17,6 +17,8 @@ http {
keepalive_timeout 3600;
client_max_body_size ${NGINX_CLIENT_MAX_BODY_SIZE};
gzip on;
gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;
gzip_proxied any;

36
update
View File

@@ -20,6 +20,16 @@ function WARN() {
echo -e "${WARN} ${1}"
}
TMP_PATH=$(mktemp -d)
if [ ! -d "${TMP_PATH}" ]; then
# 如果自动生成 tmp 文件夹失败则手动指定,避免出现数据丢失等情况
TMP_PATH=/tmp/mp_update_path
if [ -d /tmp/mp_update_path ]; then
rm -rf /tmp/mp_update_path
fi
mkdir -p /tmp/mp_update_path
fi
# 下载及解压
function download_and_unzip() {
local retries=0
@@ -28,9 +38,9 @@ function download_and_unzip() {
local target_dir="$2"
INFO "正在下载 ${url}..."
while [ $retries -lt $max_retries ]; do
if curl ${CURL_OPTIONS} "${url}" ${CURL_HEADERS} | busybox unzip -d /tmp - > /dev/null; then
if [ -e /tmp/MoviePilot-* ]; then
mv /tmp/MoviePilot-* /tmp/"${target_dir}"
if curl ${CURL_OPTIONS} "${url}" ${CURL_HEADERS} | busybox unzip -d ${TMP_PATH} - > /dev/null; then
if [ -e ${TMP_PATH}/MoviePilot-* ]; then
mv ${TMP_PATH}/MoviePilot-* ${TMP_PATH}/"${target_dir}"
fi
break
else
@@ -48,8 +58,6 @@ function download_and_unzip() {
# 下载程序资源,$1: 后端版本路径
function install_backend_and_download_resources() {
# 清理临时目录,上次安装失败可能有残留
rm -rf /tmp/*
# 更新后端程序
if ! download_and_unzip "${GITHUB_PROXY}https://github.com/jxxghp/MoviePilot/archive/refs/${1}" "App"; then
WARN "后端程序下载失败,继续使用旧的程序来启动..."
@@ -61,13 +69,13 @@ function install_backend_and_download_resources() {
ERROR "pip 更新失败,请重新拉取镜像"
return 1
fi
if ! pip install ${PIP_OPTIONS} --root-user-action=ignore -r /tmp/App/requirements.txt > /dev/null; then
if ! pip install ${PIP_OPTIONS} --root-user-action=ignore -r ${TMP_PATH}/App/requirements.txt > /dev/null; then
ERROR "安装依赖失败,请重新拉取镜像"
return 1
fi
INFO "安装依赖成功"
# 从后端文件中读取前端版本号
frontend_version=$(sed -n "s/^FRONTEND_VERSION\s*=\s*'\([^']*\)'/\1/p" /tmp/App/version.py)
frontend_version=$(sed -n "s/^FRONTEND_VERSION\s*=\s*'\([^']*\)'/\1/p" ${TMP_PATH}/App/version.py)
if [[ "${frontend_version}" != *v* ]]; then
WARN "前端最新版本号获取失败,继续启动..."
return 1
@@ -94,11 +102,11 @@ function install_backend_and_download_resources() {
rm -rf /app
mkdir -p /app
# 复制新后端程序
cp -a /tmp/App/* /app/
cp -a ${TMP_PATH}/App/* /app/
# 复制新前端程序
rm -rf /public
mkdir -p /public
cp -a /tmp/dist/* /public/
cp -a ${TMP_PATH}/dist/* /public/
INFO "程序部分更新成功,前端版本:${frontend_version},后端版本:${1}"
# 恢复插件目录
cp -a /plugins/* /app/app/plugins/
@@ -112,10 +120,10 @@ function install_backend_and_download_resources() {
fi
INFO "站点资源下载成功"
# 复制新站点资源
cp -a /tmp/Resources/resources/* /app/app/helper/
cp -a ${TMP_PATH}/Resources/resources/* /app/app/helper/
INFO "站点资源更新成功"
# 清理临时目录
rm -rf /tmp/*
rm -rf "${TMP_PATH}"
return 0
}
@@ -212,14 +220,14 @@ function compare_versions() {
return 1
elif (( current_ver < release_ver )); then
INFO "发现新版本,开始自动升级..."
install_backend_and_download_resources "tags/${release_ver}.zip"
install_backend_and_download_resources "tags/$2.zip"
return 0
else
WARN "当前版本已是最新版本,跳过更新步骤..."
return 1
continue
fi
fi
done
WARN "当前版本已是最新版本,跳过更新步骤..."
}
# 优先级转换

View File

@@ -1,2 +1,2 @@
APP_VERSION = 'v2.0.1'
FRONTEND_VERSION = 'v2.0.1'
APP_VERSION = 'v2.0.5'
FRONTEND_VERSION = 'v2.0.5'