mirror of
https://github.com/DrizzleTime/Foxel.git
synced 2026-05-09 09:32:40 +08:00
Compare commits
7 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6a52fa3fd5 | ||
|
|
219999914c | ||
|
|
1a3d9d41ec | ||
|
|
27ad49d8ed | ||
|
|
e230bf6661 | ||
|
|
50fb0b4977 | ||
|
|
b50f19bcb4 |
@@ -82,7 +82,24 @@
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
4. **启动开发服务器**
|
||||
4. **初始化环境**
|
||||
|
||||
在启动服务前,请进行以下准备:
|
||||
|
||||
- **创建数据目录**:
|
||||
在项目根目录执行 `mkdir -p data/db`。这将创建用于存放数据库等文件的目录。
|
||||
> [!IMPORTANT]
|
||||
> 请确保应用拥有对 `data/db` 目录的读写权限。
|
||||
|
||||
- **创建 `.env` 配置文件**:
|
||||
在项目根目录创建名为 `.env` 的文件,并填入以下内容。这些密钥用于保障应用安全,您可以按需修改。
|
||||
|
||||
```dotenv
|
||||
SECRET_KEY=EnsRhL9NFPxgFVc+7t96/y70DIOR+9SpntcIqQa90TU=
|
||||
TEMP_LINK_SECRET_KEY=EnsRhL9NFPxgFVc+7t96/y70DIOR+9SpntcIqQa90TU=
|
||||
```
|
||||
|
||||
5. **启动开发服务器**
|
||||
|
||||
```bash
|
||||
uvicorn main:app --reload --host 0.0.0.0 --port 8000
|
||||
|
||||
63
README.md
63
README.md
@@ -1,8 +1,12 @@
|
||||
<div align="right">
|
||||
<b>English</b> | <a href="./README_zh.md">简体中文</a>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
|
||||
# Foxel
|
||||
|
||||
**一个面向个人和团队的、高度可扩展的私有云盘解决方案,支持 AI 语义搜索。**
|
||||
**A highly extensible private cloud storage solution for individuals and teams, featuring AI-powered semantic search.**
|
||||
|
||||

|
||||

|
||||
@@ -11,32 +15,31 @@
|
||||
|
||||
---
|
||||
<blockquote>
|
||||
<em><strong>数据之洋浩瀚无涯,当以洞察之目引航,然其脉络深隐,非表象所能尽窥。</strong></em><br>
|
||||
<em><strong>The ocean of data is boundless, let the eye of insight guide the voyage, yet its intricate connections lie deep, not fully discernible from the surface.</strong></em>
|
||||
</blockquote>
|
||||
</div>
|
||||
|
||||
## 👀 在线体验
|
||||
## 👀 Online Demo
|
||||
|
||||
> [https://demo.foxel.cc](https://demo.foxel.cc)
|
||||
>
|
||||
> 账号/密码:`admin` / `admin`
|
||||
> Account/Password: `admin` / `admin`
|
||||
|
||||
## ✨ 核心功能
|
||||
## ✨ Core Features
|
||||
|
||||
- **统一文件管理**:集中管理分布于不同存储后端的文件。
|
||||
- **插件化存储后端**:采用可扩展的适配器模式,方便集成多种存储类型。
|
||||
- **语义搜索**:支持自然语言描述搜索图片、文档等非结构化数据内容。
|
||||
- **内置文件预览**:可直接预览图片、视频、PDF、Office 文档及文本、代码文件,无需下载。
|
||||
- **权限与分享**:支持公开或私密分享链接,便于文件共享。
|
||||
- **任务处理中心**:支持异步任务处理,如文件索引和数据备份,不影响主应用运行。
|
||||
- **Unified File Management**: Centralize management of files distributed across different storage backends.
|
||||
- **Pluggable Storage Backends**: Utilizes an extensible adapter pattern to easily integrate various storage types.
|
||||
- **Semantic Search**: Supports natural language search for content within unstructured data like images and documents.
|
||||
- **Built-in File Preview**: Preview images, videos, PDFs, Office documents, text, and code files directly without downloading.
|
||||
- **Permissions and Sharing**: Supports public or private sharing links for easy file distribution.
|
||||
- **Task Processing Center**: Supports asynchronous task processing, such as file indexing and data backups, without impacting the main application.
|
||||
|
||||
## 🚀 快速开始
|
||||
## 🚀 Quick Start
|
||||
|
||||
使用 Docker Compose 是启动 Foxel 最推荐的方式。
|
||||
Using Docker Compose is the most recommended way to start Foxel.
|
||||
|
||||
1. **创建数据目录**:
|
||||
新建 `data` 文件夹用于持久化数据:
|
||||
1. **Create Data Directories**:
|
||||
Create a `data` folder for persistent data:
|
||||
|
||||
```bash
|
||||
mkdir -p data/db
|
||||
@@ -44,40 +47,40 @@ mkdir -p data/mount
|
||||
chmod 777 data/db data/mount
|
||||
```
|
||||
|
||||
2. **下载 Docker Compose 文件**:
|
||||
2. **Download Docker Compose File**:
|
||||
|
||||
```bash
|
||||
curl -L -O https://github.com/DrizzleTime/Foxel/raw/main/compose.yaml
|
||||
```
|
||||
|
||||
下载完成后,**强烈建议**修改 `compose.yaml` 文件中的环境变量以确保安全:
|
||||
After downloading, it is **strongly recommended** to modify the environment variables in the `compose.yaml` file to ensure security:
|
||||
|
||||
- 修改 `SECRET_KEY` 和 `TEMP_LINK_SECRET_KEY`:将默认的密钥替换为随机生成的强密钥
|
||||
- Modify `SECRET_KEY` and `TEMP_LINK_SECRET_KEY`: Replace the default keys with randomly generated strong keys.
|
||||
|
||||
3. **启动服务**:
|
||||
3. **Start the Services**:
|
||||
|
||||
```bash
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
4. **访问应用**:
|
||||
4. **Access the Application**:
|
||||
|
||||
服务启动后,在浏览器中打开页面。
|
||||
Once the services are running, open the page in your browser.
|
||||
|
||||
> 首次启动,请根据引导页面完成管理员账号的初始化设置。
|
||||
> On the first launch, please follow the setup guide to initialize the administrator account.
|
||||
|
||||
## 🤝 如何贡献
|
||||
## 🤝 How to Contribute
|
||||
|
||||
我们非常欢迎来自社区的贡献!无论是提交 Bug、建议新功能还是直接贡献代码。
|
||||
We welcome contributions from the community! Whether it's submitting bugs, suggesting new features, or contributing code directly.
|
||||
|
||||
在开始之前,请先阅读我们的 [`CONTRIBUTING.md`](CONTRIBUTING.md) 文件,它会指导你如何设置开发环境以及提交流程。
|
||||
Before you start, please read our [`CONTRIBUTING.md`](CONTRIBUTING.md) file, which will guide you on how to set up your development environment and the submission process.
|
||||
|
||||
## 🌐 社区
|
||||
## 🌐 Community
|
||||
|
||||
加入我们的交流社区:[Telegram 群组](https://t.me/+thDsBfyqJxZkNTU1),与开发者和用户一起讨论!
|
||||
Join our community on [Telegram](https://t.me/+thDsBfyqJxZkNTU1) to discuss with developers and other users!
|
||||
|
||||
你也可以加入我们的微信群,获取更多实时交流与支持。请扫描下方二维码加入:
|
||||
You can also join our WeChat group for more real-time communication and support. Please scan the QR code below to join:
|
||||
|
||||
<img src="https://foxel.cc/image/wechat.png" alt="微信群二维码" width="180">
|
||||
<img src="https://foxel.cc/image/wechat.png" alt="WeChat Group QR Code" width="180">
|
||||
|
||||
> 如果二维码失效,请添加微信号 **drizzle2001**,我们会邀请你加入群聊。
|
||||
> If the QR code is invalid, please add WeChat ID **drizzle2001**, and we will invite you to the group.
|
||||
|
||||
87
README_zh.md
Normal file
87
README_zh.md
Normal file
@@ -0,0 +1,87 @@
|
||||
<div align="right">
|
||||
<a href="./README.md">English</a> | <b>简体中文</b>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
|
||||
# Foxel
|
||||
|
||||
**一个面向个人和团队的、高度可扩展的私有云盘解决方案,支持 AI 语义搜索。**
|
||||
|
||||

|
||||

|
||||

|
||||

|
||||
|
||||
---
|
||||
<blockquote>
|
||||
<em><strong>数据之洋浩瀚无涯,当以洞察之目引航,然其脉络深隐,非表象所能尽窥。</strong></em><br>
|
||||
<em><strong>The ocean of data is boundless, let the eye of insight guide the voyage, yet its intricate connections lie deep, not fully discernible from the surface.</strong></em>
|
||||
</blockquote>
|
||||
</div>
|
||||
|
||||
## 👀 在线体验
|
||||
|
||||
> [https://demo.foxel.cc](https://demo.foxel.cc)
|
||||
>
|
||||
> 账号/密码:`admin` / `admin`
|
||||
|
||||
## ✨ 核心功能
|
||||
|
||||
- **统一文件管理**:集中管理分布于不同存储后端的文件。
|
||||
- **插件化存储后端**:采用可扩展的适配器模式,方便集成多种存储类型。
|
||||
- **语义搜索**:支持自然语言描述搜索图片、文档等非结构化数据内容。
|
||||
- **内置文件预览**:可直接预览图片、视频、PDF、Office 文档及文本、代码文件,无需下载。
|
||||
- **权限与分享**:支持公开或私密分享链接,便于文件共享。
|
||||
- **任务处理中心**:支持异步任务处理,如文件索引和数据备份,不影响主应用运行。
|
||||
|
||||
## 🚀 快速开始
|
||||
|
||||
使用 Docker Compose 是启动 Foxel 最推荐的方式。
|
||||
|
||||
1. **创建数据目录**:
|
||||
新建 `data` 文件夹用于持久化数据:
|
||||
|
||||
```bash
|
||||
mkdir -p data/db
|
||||
mkdir -p data/mount
|
||||
chmod 777 data/db data/mount
|
||||
```
|
||||
|
||||
2. **下载 Docker Compose 文件**:
|
||||
|
||||
```bash
|
||||
curl -L -O https://github.com/DrizzleTime/Foxel/raw/main/compose.yaml
|
||||
```
|
||||
|
||||
下载完成后,**强烈建议**修改 `compose.yaml` 文件中的环境变量以确保安全:
|
||||
|
||||
- 修改 `SECRET_KEY` 和 `TEMP_LINK_SECRET_KEY`:将默认的密钥替换为随机生成的强密钥
|
||||
|
||||
3. **启动服务**:
|
||||
|
||||
```bash
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
4. **访问应用**:
|
||||
|
||||
服务启动后,在浏览器中打开页面。
|
||||
|
||||
> 首次启动,请根据引导页面完成管理员账号的初始化设置。
|
||||
|
||||
## 🤝 如何贡献
|
||||
|
||||
我们非常欢迎来自社区的贡献!无论是提交 Bug、建议新功能还是直接贡献代码。
|
||||
|
||||
在开始之前,请先阅读我们的 [`CONTRIBUTING.md`](CONTRIBUTING.md) 文件,它会指导你如何设置开发环境以及提交流程。
|
||||
|
||||
## 🌐 社区
|
||||
|
||||
加入我们的交流社区:[Telegram 群组](https://t.me/+thDsBfyqJxZkNTU1),与开发者和用户一起讨论!
|
||||
|
||||
你也可以加入我们的微信群,获取更多实时交流与支持。请扫描下方二维码加入:
|
||||
|
||||
<img src="https://foxel.cc/image/wechat.png" alt="微信群二维码" width="180">
|
||||
|
||||
> 如果二维码失效,请添加微信号 **drizzle2001**,我们会邀请你加入群聊。
|
||||
@@ -41,7 +41,9 @@ async def get_system_status():
|
||||
"version": VERSION,
|
||||
"title": await ConfigCenter.get("APP_NAME", "Foxel"),
|
||||
"logo": await ConfigCenter.get("APP_LOGO", "/logo.svg"),
|
||||
"is_initialized": await has_users()
|
||||
"is_initialized": await has_users(),
|
||||
"app_domain": await ConfigCenter.get("APP_DOMAIN"),
|
||||
"file_domain": await ConfigCenter.get("FILE_DOMAIN"),
|
||||
}
|
||||
return success(system_info)
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
from fastapi import APIRouter, Depends, Body
|
||||
from typing import Annotated
|
||||
from services.processors.registry import get_config_schemas
|
||||
from services.virtual_fs import process_file
|
||||
from services.task_queue import task_queue_service
|
||||
from services.auth import get_current_active_user, User
|
||||
from api.response import success
|
||||
from pydantic import BaseModel
|
||||
@@ -21,7 +21,7 @@ async def list_processors(
|
||||
"name": meta["name"],
|
||||
"supported_exts": meta.get("supported_exts", []),
|
||||
"config_schema": meta["config_schema"],
|
||||
"produces_file": meta.get("produces_file", False),
|
||||
"produces_file": meta.get("produces_file", False),
|
||||
})
|
||||
return success(out)
|
||||
|
||||
@@ -40,5 +40,13 @@ async def process_file_with_processor(
|
||||
req: ProcessRequest = Body(...)
|
||||
):
|
||||
save_to = req.path if req.overwrite else req.save_to
|
||||
result = await process_file(req.path, req.processor_type, req.config, save_to)
|
||||
return success(result)
|
||||
task = await task_queue_service.add_task(
|
||||
"process_file",
|
||||
{
|
||||
"path": req.path,
|
||||
"processor_type": req.processor_type,
|
||||
"config": req.config,
|
||||
"save_to": save_to,
|
||||
},
|
||||
)
|
||||
return success({"task_id": task.id})
|
||||
|
||||
@@ -6,6 +6,7 @@ from schemas.tasks import AutomationTaskCreate, AutomationTaskUpdate
|
||||
from api.response import success
|
||||
from services.auth import get_current_active_user, User
|
||||
from services.logging import LogService
|
||||
from services.task_queue import task_queue_service
|
||||
|
||||
router = APIRouter(
|
||||
prefix="/api/tasks",
|
||||
@@ -15,6 +16,25 @@ router = APIRouter(
|
||||
)
|
||||
|
||||
|
||||
@router.get("/queue")
|
||||
async def get_task_queue_status(
|
||||
current_user: Annotated[User, Depends(get_current_active_user)],
|
||||
):
|
||||
tasks = task_queue_service.get_all_tasks()
|
||||
return success([task.dict() for task in tasks])
|
||||
|
||||
|
||||
@router.get("/queue/{task_id}")
|
||||
async def get_task_status(
|
||||
task_id: str,
|
||||
current_user: Annotated[User, Depends(get_current_active_user)],
|
||||
):
|
||||
task = task_queue_service.get_task(task_id)
|
||||
if not task:
|
||||
raise HTTPException(status_code=404, detail="Task not found")
|
||||
return success(task.dict())
|
||||
|
||||
|
||||
@router.post("/")
|
||||
async def create_task(
|
||||
task_in: AutomationTaskCreate,
|
||||
|
||||
@@ -19,6 +19,7 @@ from services.virtual_fs import (
|
||||
from services.thumbnail import is_image_filename, get_or_create_thumb, is_raw_filename
|
||||
from schemas import MkdirRequest, MoveRequest
|
||||
from api.response import success
|
||||
from services.config import ConfigCenter
|
||||
|
||||
router = APIRouter(prefix='/api/fs', tags=["virtual-fs"])
|
||||
|
||||
@@ -151,7 +152,13 @@ async def get_temp_link(
|
||||
"""获取文件的临时公开访问令牌"""
|
||||
full_path = '/' + full_path if not full_path.startswith('/') else full_path
|
||||
token = await generate_temp_link_token(full_path, expires_in=expires_in)
|
||||
return success({"token": token, "path": full_path})
|
||||
file_domain = await ConfigCenter.get("FILE_DOMAIN")
|
||||
if file_domain:
|
||||
file_domain = file_domain.rstrip('/')
|
||||
url = f"{file_domain}/api/fs/public/{token}"
|
||||
else:
|
||||
url = f"/api/fs/public/{token}"
|
||||
return success({"token": token, "path": full_path, "url": url})
|
||||
|
||||
|
||||
@router.get("/public/{token}")
|
||||
|
||||
3
main.py
3
main.py
@@ -8,6 +8,7 @@ from fastapi import FastAPI
|
||||
from services.middleware.logging_middleware import LoggingMiddleware
|
||||
from services.middleware.exception_handler import global_exception_handler
|
||||
from dotenv import load_dotenv
|
||||
from services.task_queue import task_queue_service
|
||||
|
||||
load_dotenv()
|
||||
|
||||
@@ -17,9 +18,11 @@ async def lifespan(app: FastAPI):
|
||||
await init_db()
|
||||
await runtime_registry.refresh()
|
||||
await ConfigCenter.set("APP_VERSION", VERSION)
|
||||
await task_queue_service.start_worker()
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
await task_queue_service.stop_worker()
|
||||
await close_db()
|
||||
|
||||
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
from __future__ import annotations
|
||||
from typing import List, Dict, Tuple, AsyncIterator
|
||||
import io
|
||||
import os
|
||||
from models import StorageAdapter
|
||||
from telethon import TelegramClient
|
||||
from telethon.sessions import StringSession
|
||||
@@ -20,7 +22,7 @@ CONFIG_SCHEMA = [
|
||||
]
|
||||
|
||||
class TelegramAdapter:
|
||||
"""Telegram 存储适配器 (只读, 使用用户 Session)"""
|
||||
"""Telegram 存储适配器 (使用用户 Session)"""
|
||||
|
||||
def __init__(self, record: StorageAdapter):
|
||||
self.record = record
|
||||
@@ -70,30 +72,43 @@ class TelegramAdapter:
|
||||
await client.connect()
|
||||
messages = await client.get_messages(self.chat_id, limit=50)
|
||||
for message in messages:
|
||||
if message and (message.document or message.video):
|
||||
media = message.document or message.video
|
||||
filename = None
|
||||
if not message:
|
||||
continue
|
||||
|
||||
media = message.document or message.video or message.photo
|
||||
if not media:
|
||||
continue
|
||||
|
||||
filename = None
|
||||
size = 0
|
||||
|
||||
if message.photo:
|
||||
photo_size = message.photo.sizes[-1]
|
||||
size = photo_size.size if hasattr(photo_size, 'size') else 0
|
||||
filename = f"photo_{message.id}.jpg"
|
||||
|
||||
elif message.document or message.video:
|
||||
size = media.size
|
||||
if hasattr(media, 'attributes'):
|
||||
for attr in media.attributes:
|
||||
if hasattr(attr, 'file_name') and attr.file_name:
|
||||
filename = attr.file_name
|
||||
break
|
||||
|
||||
if not filename:
|
||||
if message.text and '.' in message.text:
|
||||
if len(message.text) < 256 and '\n' not in message.text:
|
||||
filename = message.text
|
||||
|
||||
if not filename:
|
||||
if message.text and '.' in message.text and len(message.text) < 256 and '\n' not in message.text:
|
||||
filename = message.text
|
||||
|
||||
if not filename:
|
||||
filename = f"unknown_{message.id}"
|
||||
|
||||
if not filename:
|
||||
filename = "Unknown"
|
||||
|
||||
entries.append({
|
||||
"name": f"{message.id}_{filename}",
|
||||
"is_dir": False,
|
||||
"size": media.size,
|
||||
"mtime": int(message.date.timestamp()),
|
||||
"type": "file",
|
||||
})
|
||||
entries.append({
|
||||
"name": f"{message.id}_{filename}",
|
||||
"is_dir": False,
|
||||
"size": size,
|
||||
"mtime": int(message.date.timestamp()),
|
||||
"type": "file",
|
||||
})
|
||||
finally:
|
||||
if client.is_connected():
|
||||
await client.disconnect()
|
||||
@@ -111,7 +126,7 @@ class TelegramAdapter:
|
||||
try:
|
||||
await client.connect()
|
||||
message = await client.get_messages(self.chat_id, ids=message_id)
|
||||
if not message or not (message.document or message.video):
|
||||
if not message or not (message.document or message.video or message.photo):
|
||||
raise FileNotFoundError(f"在频道 {self.chat_id} 中未找到消息ID为 {message_id} 的文件")
|
||||
|
||||
file_bytes = await client.download_media(message, file=bytes)
|
||||
@@ -121,25 +136,73 @@ class TelegramAdapter:
|
||||
await client.disconnect()
|
||||
|
||||
async def write_file(self, root: str, rel: str, data: bytes):
|
||||
raise NotImplementedError("Telegram 适配器是只读的,不支持写入文件。")
|
||||
"""将字节数据作为文件上传"""
|
||||
client = self._get_client()
|
||||
file_like = io.BytesIO(data)
|
||||
file_like.name = os.path.basename(rel) or "file"
|
||||
|
||||
try:
|
||||
await client.connect()
|
||||
await client.send_file(self.chat_id, file_like, caption=file_like.name)
|
||||
finally:
|
||||
if client.is_connected():
|
||||
await client.disconnect()
|
||||
|
||||
async def write_file_stream(self, root: str, rel: str, data_iter: AsyncIterator[bytes]):
|
||||
raise NotImplementedError("Telegram 适配器是只读的,不支持流式写入文件。")
|
||||
"""以流式方式上传文件"""
|
||||
client = self._get_client()
|
||||
filename = os.path.basename(rel) or "file"
|
||||
import tempfile
|
||||
temp_dir = tempfile.gettempdir()
|
||||
temp_path = os.path.join(temp_dir, filename)
|
||||
|
||||
total_size = 0
|
||||
try:
|
||||
with open(temp_path, "wb") as f:
|
||||
async for chunk in data_iter:
|
||||
if chunk:
|
||||
f.write(chunk)
|
||||
total_size += len(chunk)
|
||||
|
||||
await client.connect()
|
||||
await client.send_file(self.chat_id, temp_path, caption=filename)
|
||||
|
||||
finally:
|
||||
if os.path.exists(temp_path):
|
||||
os.remove(temp_path)
|
||||
if client.is_connected():
|
||||
await client.disconnect()
|
||||
return total_size
|
||||
|
||||
async def mkdir(self, root: str, rel: str):
|
||||
raise NotImplementedError("Telegram 适配器是只读的,不支持创建目录。")
|
||||
raise NotImplementedError("Telegram 适配器不支持创建目录。")
|
||||
|
||||
async def delete(self, root: str, rel: str):
|
||||
raise NotImplementedError("Telegram 适配器是只读的,不支持删除。")
|
||||
"""删除一个文件 (即一条消息)"""
|
||||
try:
|
||||
message_id_str, _ = rel.split('_', 1)
|
||||
message_id = int(message_id_str)
|
||||
except (ValueError, IndexError):
|
||||
raise FileNotFoundError(f"无效的文件路径格式,无法解析消息ID: {rel}")
|
||||
|
||||
client = self._get_client()
|
||||
try:
|
||||
await client.connect()
|
||||
result = await client.delete_messages(self.chat_id, [message_id])
|
||||
if not result or not result[0].pts:
|
||||
raise FileNotFoundError(f"在 {self.chat_id} 中删除消息 {message_id} 失败,可能消息不存在或无权限")
|
||||
finally:
|
||||
if client.is_connected():
|
||||
await client.disconnect()
|
||||
|
||||
async def move(self, root: str, src_rel: str, dst_rel: str):
|
||||
raise NotImplementedError("Telegram 适配器是只读的,不支持移动。")
|
||||
raise NotImplementedError("Telegram 适配器不支持移动。")
|
||||
|
||||
async def rename(self, root: str, src_rel: str, dst_rel: str):
|
||||
raise NotImplementedError("Telegram 适配器是只读的,不支持重命名。")
|
||||
raise NotImplementedError("Telegram 适配器不支持重命名。")
|
||||
|
||||
async def copy(self, root: str, src_rel: str, dst_rel: str, overwrite: bool = False):
|
||||
raise NotImplementedError("Telegram 适配器是只读的,不支持复制。")
|
||||
raise NotImplementedError("Telegram 适配器不支持复制。")
|
||||
|
||||
async def stream_file(self, root: str, rel: str, range_header: str | None):
|
||||
from fastapi.responses import StreamingResponse
|
||||
@@ -156,19 +219,25 @@ class TelegramAdapter:
|
||||
try:
|
||||
await client.connect()
|
||||
message = await client.get_messages(self.chat_id, ids=message_id)
|
||||
if not message or not (message.document or message.video):
|
||||
media = message.document or message.video or message.photo
|
||||
if not message or not media:
|
||||
raise FileNotFoundError(f"在频道 {self.chat_id} 中未找到消息ID为 {message_id} 的文件")
|
||||
|
||||
media = message.document or message.video
|
||||
file_size = media.size
|
||||
|
||||
|
||||
if message.photo:
|
||||
photo_size = media.sizes[-1]
|
||||
file_size = photo_size.size if hasattr(photo_size, 'size') else 0
|
||||
mime_type = "image/jpeg"
|
||||
else:
|
||||
file_size = media.size
|
||||
mime_type = media.mime_type or "application/octet-stream"
|
||||
|
||||
start = 0
|
||||
end = file_size - 1
|
||||
status = 200
|
||||
|
||||
headers = {
|
||||
"Accept-Ranges": "bytes",
|
||||
"Content-Type": media.mime_type or "application/octet-stream",
|
||||
"Content-Type": mime_type,
|
||||
"Content-Length": str(file_size),
|
||||
}
|
||||
|
||||
@@ -225,14 +294,20 @@ class TelegramAdapter:
|
||||
try:
|
||||
await client.connect()
|
||||
message = await client.get_messages(self.chat_id, ids=message_id)
|
||||
if not message or not (message.document or message.video):
|
||||
media = message.document or message.video or message.photo
|
||||
if not message or not media:
|
||||
raise FileNotFoundError(f"在频道 {self.chat_id} 中未找到消息ID为 {message_id} 的文件")
|
||||
|
||||
media = message.document or message.video
|
||||
|
||||
if message.photo:
|
||||
photo_size = media.sizes[-1]
|
||||
size = photo_size.size if hasattr(photo_size, 'size') else 0
|
||||
else:
|
||||
size = media.size
|
||||
|
||||
return {
|
||||
"name": rel,
|
||||
"is_dir": False,
|
||||
"size": media.size,
|
||||
"size": size,
|
||||
"mtime": int(message.date.timestamp()),
|
||||
"type": "file",
|
||||
}
|
||||
|
||||
@@ -4,7 +4,7 @@ from typing import Any, Optional, Dict
|
||||
from dotenv import load_dotenv
|
||||
from models.database import Configuration
|
||||
load_dotenv(dotenv_path=".env")
|
||||
VERSION = "v1.1.2"
|
||||
VERSION = "v1.1.3"
|
||||
|
||||
class ConfigCenter:
|
||||
_cache: Dict[str, Any] = {}
|
||||
|
||||
122
services/task_queue.py
Normal file
122
services/task_queue.py
Normal file
@@ -0,0 +1,122 @@
|
||||
import asyncio
|
||||
from typing import Dict, Any
|
||||
from pydantic import BaseModel, Field
|
||||
import uuid
|
||||
from services.logging import LogService
|
||||
from enum import Enum
|
||||
|
||||
|
||||
class TaskStatus(str, Enum):
|
||||
PENDING = "pending"
|
||||
RUNNING = "running"
|
||||
SUCCESS = "success"
|
||||
FAILED = "failed"
|
||||
|
||||
|
||||
class Task(BaseModel):
|
||||
id: str = Field(default_factory=lambda: uuid.uuid4().hex)
|
||||
name: str
|
||||
status: TaskStatus = TaskStatus.PENDING
|
||||
result: Any = None
|
||||
error: str | None = None
|
||||
task_info: Dict[str, Any] = {}
|
||||
|
||||
|
||||
class TaskQueueService:
|
||||
def __init__(self):
|
||||
self._queue = asyncio.Queue()
|
||||
self._tasks: Dict[str, Task] = {}
|
||||
self._worker_task: asyncio.Task | None = None
|
||||
|
||||
async def add_task(self, name: str, task_info: Dict[str, Any]) -> Task:
|
||||
task = Task(name=name, task_info=task_info)
|
||||
self._tasks[task.id] = task
|
||||
await self._queue.put(task)
|
||||
await LogService.info("task_queue", f"Task {name} ({task.id}) enqueued", {"task_id": task.id, "name": name})
|
||||
return task
|
||||
|
||||
def get_task(self, task_id: str) -> Task | None:
|
||||
return self._tasks.get(task_id)
|
||||
|
||||
def get_all_tasks(self) -> list[Task]:
|
||||
return list(self._tasks.values())
|
||||
|
||||
async def _execute_task(self, task: Task):
|
||||
from services.virtual_fs import process_file
|
||||
|
||||
task.status = TaskStatus.RUNNING
|
||||
await LogService.info("task_queue", f"Task {task.name} ({task.id}) started", {"task_id": task.id, "name": task.name})
|
||||
|
||||
try:
|
||||
if task.name == "process_file":
|
||||
params = task.task_info
|
||||
result = await process_file(
|
||||
path=params["path"],
|
||||
processor_type=params["processor_type"],
|
||||
config=params["config"],
|
||||
save_to=params["save_to"]
|
||||
)
|
||||
task.result = result
|
||||
elif task.name == "automation_task":
|
||||
from models.database import AutomationTask
|
||||
from services.processors.registry import get as get_processor
|
||||
from services.virtual_fs import read_file, write_file
|
||||
|
||||
params = task.task_info
|
||||
auto_task = await AutomationTask.get(id=params["task_id"])
|
||||
path = params["path"]
|
||||
|
||||
processor = get_processor(auto_task.processor_type)
|
||||
if not processor:
|
||||
raise ValueError(f"Processor {auto_task.processor_type} not found for task {auto_task.id}")
|
||||
|
||||
file_content = await read_file(path)
|
||||
result = await processor.process(file_content, path, auto_task.processor_config)
|
||||
|
||||
save_to = auto_task.processor_config.get("save_to")
|
||||
if save_to and getattr(processor, "produces_file", False):
|
||||
await write_file(save_to, result)
|
||||
task.result = "Automation task completed"
|
||||
else:
|
||||
raise ValueError(f"Unknown task name: {task.name}")
|
||||
|
||||
task.status = TaskStatus.SUCCESS
|
||||
await LogService.info("task_queue", f"Task {task.name} ({task.id}) succeeded", {"task_id": task.id, "name": task.name})
|
||||
|
||||
except Exception as e:
|
||||
task.status = TaskStatus.FAILED
|
||||
task.error = str(e)
|
||||
await LogService.error("task_queue", f"Task {task.name} ({task.id}) failed: {e}", {"task_id": task.id, "name": task.name})
|
||||
|
||||
async def worker(self):
|
||||
await LogService.info("task_queue", "Task worker started")
|
||||
while True:
|
||||
try:
|
||||
task = await self._queue.get()
|
||||
await self._execute_task(task)
|
||||
except asyncio.CancelledError:
|
||||
await LogService.info("task_queue", "Task worker stopped")
|
||||
break
|
||||
except Exception as e:
|
||||
await LogService.error("task_queue", f"Error in task worker: {e}", exc_info=True)
|
||||
finally:
|
||||
self._queue.task_done()
|
||||
|
||||
async def start_worker(self):
|
||||
if self._worker_task is None or self._worker_task.done():
|
||||
self._worker_task = asyncio.create_task(self.worker())
|
||||
await LogService.info("task_queue", "Task worker created.")
|
||||
|
||||
async def stop_worker(self):
|
||||
if self._worker_task and not self._worker_task.done():
|
||||
self._worker_task.cancel()
|
||||
try:
|
||||
await self._worker_task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
finally:
|
||||
self._worker_task = None
|
||||
await LogService.info("task_queue", "Task worker has been stopped.")
|
||||
|
||||
|
||||
task_queue_service = TaskQueueService()
|
||||
@@ -4,6 +4,9 @@ from models.database import AutomationTask
|
||||
from services.processors.registry import get as get_processor
|
||||
from services.logging import LogService
|
||||
|
||||
from services.task_queue import task_queue_service
|
||||
|
||||
|
||||
class TaskService:
|
||||
async def trigger_tasks(self, event: str, path: str):
|
||||
tasks = await AutomationTask.filter(event=event, enabled=True)
|
||||
@@ -21,28 +24,12 @@ class TaskService:
|
||||
return True
|
||||
|
||||
async def execute(self, task: AutomationTask, path: str):
|
||||
from services.virtual_fs import read_file, write_file
|
||||
|
||||
processor = get_processor(task.processor_type)
|
||||
if not processor:
|
||||
print(f"Processor {task.processor_type} not found for task {task.id}")
|
||||
return
|
||||
|
||||
try:
|
||||
file_content = await read_file(path)
|
||||
result = await processor.process(file_content, path, task.processor_config)
|
||||
|
||||
save_to = task.processor_config.get("save_to")
|
||||
if save_to and getattr(processor, "produces_file", False):
|
||||
await write_file(save_to, result)
|
||||
|
||||
except Exception as e:
|
||||
error_message = f"Error executing task {task.id} for path {path}: {e}"
|
||||
print(error_message)
|
||||
await LogService.error(
|
||||
source=f"task_executor:{task.id}",
|
||||
message=error_message,
|
||||
details={"task_name": task.name, "event": task.event, "path": path, "processor": task.processor_type}
|
||||
)
|
||||
await task_queue_service.add_task(
|
||||
"automation_task",
|
||||
{
|
||||
"task_id": task.id,
|
||||
"path": path,
|
||||
},
|
||||
)
|
||||
|
||||
task_service = TaskService()
|
||||
@@ -20,6 +20,8 @@ export interface SystemStatus {
|
||||
title: string;
|
||||
logo: string;
|
||||
is_initialized: boolean;
|
||||
app_domain?: string;
|
||||
file_domain?: string;
|
||||
}
|
||||
|
||||
export async function status() {
|
||||
|
||||
@@ -51,7 +51,7 @@ export const vfsApi = {
|
||||
streamUrl: (path: string) => `${API_BASE_URL}/fs/stream/${encodeURI(path.replace(/^\/+/, ''))}`,
|
||||
stat: (path: string) => request(`/fs/stat/${encodeURI(path.replace(/^\/+/, ''))}`),
|
||||
getTempLinkToken: (path: string, expiresIn: number = 3600) =>
|
||||
request<{token: string}>(`/fs/temp-link/${encodeURI(path.replace(/^\/+/, ''))}?expires_in=${expiresIn}`),
|
||||
request<{token: string, path: string, url: string}>(`/fs/temp-link/${encodeURI(path.replace(/^\/+/, ''))}?expires_in=${expiresIn}`),
|
||||
getTempPublicUrl: (token: string) => `${API_BASE_URL}/fs/public/${token}`,
|
||||
uploadStream: (fullPath: string, file: File, overwrite: boolean = true, onProgress?: (loaded: number, total: number) => void) => {
|
||||
const enc = encodeURI(fullPath.replace(/^\/+/, ''));
|
||||
|
||||
@@ -2,8 +2,10 @@ import React, { useEffect, useState } from 'react';
|
||||
import { vfsApi } from '../../api/client';
|
||||
import type { AppComponentProps } from '../types';
|
||||
import { Spin, Result, Button } from 'antd';
|
||||
import { useSystemStatus } from '../../contexts/SystemContext';
|
||||
|
||||
export const OfficeViewerApp: React.FC<AppComponentProps> = ({ filePath, onRequestClose }) => {
|
||||
const systemStatus = useSystemStatus();
|
||||
const [url, setUrl] = useState<string>();
|
||||
const [loading, setLoading] = useState(true);
|
||||
const [err, setErr] = useState<string>();
|
||||
@@ -17,8 +19,8 @@ export const OfficeViewerApp: React.FC<AppComponentProps> = ({ filePath, onReque
|
||||
vfsApi.getTempLinkToken(filePath.replace(/^\/+/, ''))
|
||||
.then(res => {
|
||||
if (cancelled) return;
|
||||
// 注意:vfsApi.getTempPublicUrl 返回的是相对路径,我们需要构建完整的 URL
|
||||
const fullUrl = new URL(vfsApi.getTempPublicUrl(res.token), window.location.origin).href;
|
||||
const baseUrl = systemStatus?.file_domain || window.location.origin;
|
||||
const fullUrl = new URL(res.url, baseUrl).href;
|
||||
const officeUrl = `https://view.officeapps.live.com/op/embed.aspx?src=${encodeURIComponent(fullUrl)}`;
|
||||
setUrl(officeUrl);
|
||||
})
|
||||
|
||||
@@ -29,8 +29,7 @@ export const DirectLinkModal = memo(function DirectLinkModal({ entry, path, open
|
||||
try {
|
||||
const fullPath = (path === '/' ? '' : path) + '/' + entry.name;
|
||||
const res = await vfsApi.getTempLinkToken(fullPath, expiresIn);
|
||||
const tempLink = `${window.location.origin}/api/fs/public/${res.token}`;
|
||||
setLink(tempLink);
|
||||
setLink(res.url);
|
||||
} catch (e: any) {
|
||||
message.error(e.message || '生成链接失败');
|
||||
} finally {
|
||||
|
||||
@@ -3,6 +3,7 @@ import { Modal, Form, Input, Radio, InputNumber, message, Button, Typography } f
|
||||
import { CopyOutlined } from '@ant-design/icons';
|
||||
import type { VfsEntry, ShareInfoWithPassword } from '../../../../api/client';
|
||||
import { shareApi } from '../../../../api/share';
|
||||
import { useSystemStatus } from '../../../../contexts/SystemContext';
|
||||
|
||||
interface ShareModalProps {
|
||||
entries: VfsEntry[];
|
||||
@@ -13,6 +14,7 @@ interface ShareModalProps {
|
||||
}
|
||||
|
||||
export const ShareModal = memo(function ShareModal({ entries, path, open, onOk, onCancel }: ShareModalProps) {
|
||||
const systemStatus = useSystemStatus();
|
||||
const [form] = Form.useForm();
|
||||
const [loading, setLoading] = useState(false);
|
||||
const [accessType, setAccessType] = useState('public');
|
||||
@@ -66,7 +68,8 @@ export const ShareModal = memo(function ShareModal({ entries, path, open, onOk,
|
||||
message.success('已复制到剪贴板');
|
||||
};
|
||||
|
||||
const shareUrl = createdShare ? `${window.location.origin}/share/${createdShare.token}` : '';
|
||||
const baseUrl = systemStatus?.app_domain || window.location.origin;
|
||||
const shareUrl = createdShare ? new URL(`/share/${createdShare.token}`, baseUrl).href : '';
|
||||
|
||||
const renderForm = () => (
|
||||
<Form form={form} layout="vertical" initialValues={{ name: defaultName, accessType: 'public', expiresInDays: 7 }}>
|
||||
|
||||
@@ -4,8 +4,10 @@ import PageCard from '../components/PageCard';
|
||||
import { shareApi, type ShareInfo } from '../api/share';
|
||||
import { format, parseISO } from 'date-fns';
|
||||
import { LinkOutlined, CopyOutlined, DeleteOutlined } from '@ant-design/icons';
|
||||
import { useSystemStatus } from '../contexts/SystemContext';
|
||||
|
||||
const SharePage = memo(function SharePage() {
|
||||
const systemStatus = useSystemStatus();
|
||||
const [loading, setLoading] = useState(false);
|
||||
const [data, setData] = useState<ShareInfo[]>([]);
|
||||
|
||||
@@ -24,7 +26,8 @@ const SharePage = memo(function SharePage() {
|
||||
useEffect(() => { fetchList(); }, [fetchList]);
|
||||
|
||||
const doCopy = (rec: ShareInfo) => {
|
||||
const shareUrl = `${window.location.origin}/share/${rec.token}`;
|
||||
const baseUrl = systemStatus?.app_domain || window.location.origin;
|
||||
const shareUrl = new URL(`/share/${rec.token}`, baseUrl).href;
|
||||
navigator.clipboard.writeText(shareUrl);
|
||||
message.success('链接已复制');
|
||||
};
|
||||
|
||||
@@ -2,13 +2,13 @@ import { Form, Input, Button, message, Tabs, Space, Card } from 'antd';
|
||||
import { useEffect, useState } from 'react';
|
||||
import PageCard from '../../components/PageCard';
|
||||
import { getAllConfig, setConfig } from '../../api/config';
|
||||
import { API_BASE_URL } from '../../api/client';
|
||||
import { AppstoreOutlined, RobotOutlined } from '@ant-design/icons';
|
||||
|
||||
const APP_CONFIG_KEYS = [
|
||||
const APP_CONFIG_KEYS: {key: string, label: string, default?: string}[] = [
|
||||
{ key: 'APP_NAME', label: '应用名称' },
|
||||
{ key: 'APP_LOGO', label: 'LOGO地址' },
|
||||
{ key: 'SERVER_URL', label: '服务端URL', default: API_BASE_URL },
|
||||
{ key: 'APP_DOMAIN', label: '应用域名' },
|
||||
{ key: 'FILE_DOMAIN', label: '文件域名' },
|
||||
];
|
||||
|
||||
const VISION_CONFIG_KEYS = [
|
||||
|
||||
Reference in New Issue
Block a user