Compare commits

...

7 Commits

Author SHA1 Message Date
shiyu
6a52fa3fd5 chore: Update version to v1.1.3 2025-08-31 19:52:30 +08:00
shiyu
219999914c docs: Update development environment initialization steps 2025-08-31 19:27:39 +08:00
shiyu
1a3d9d41ec feat: Update Telegram adapter to support uploads 2025-08-31 18:22:46 +08:00
shiyu
27ad49d8ed docs: Format badge display in README file 2025-08-31 12:56:15 +08:00
shiyu
e230bf6661 docs: Add English README file 2025-08-31 12:53:02 +08:00
shiyu
50fb0b4977 feat: Implement task queue service 2025-08-31 12:48:20 +08:00
shiyu
b50f19bcb4 feat: Add application domain and file domain configuration 2025-08-31 12:38:21 +08:00
19 changed files with 448 additions and 108 deletions

View File

@@ -82,7 +82,24 @@
pip install -r requirements.txt pip install -r requirements.txt
``` ```
4. **启动开发服务器** 4. **初始化环境**
在启动服务前,请进行以下准备:
- **创建数据目录**:
在项目根目录执行 `mkdir -p data/db`。这将创建用于存放数据库等文件的目录。
> [!IMPORTANT]
> 请确保应用拥有对 `data/db` 目录的读写权限。
- **创建 `.env` 配置文件**:
在项目根目录创建名为 `.env` 的文件,并填入以下内容。这些密钥用于保障应用安全,您可以按需修改。
```dotenv
SECRET_KEY=EnsRhL9NFPxgFVc+7t96/y70DIOR+9SpntcIqQa90TU=
TEMP_LINK_SECRET_KEY=EnsRhL9NFPxgFVc+7t96/y70DIOR+9SpntcIqQa90TU=
```
5. **启动开发服务器**
```bash ```bash
uvicorn main:app --reload --host 0.0.0.0 --port 8000 uvicorn main:app --reload --host 0.0.0.0 --port 8000

View File

@@ -1,8 +1,12 @@
<div align="right">
<b>English</b> | <a href="./README_zh.md">简体中文</a>
</div>
<div align="center"> <div align="center">
# Foxel # Foxel
**一个面向个人和团队的、高度可扩展的私有云盘解决方案,支持 AI 语义搜索。** **A highly extensible private cloud storage solution for individuals and teams, featuring AI-powered semantic search.**
![Python Version](https://img.shields.io/badge/Python-3.13+-blue.svg) ![Python Version](https://img.shields.io/badge/Python-3.13+-blue.svg)
![React](https://img.shields.io/badge/React-19.0-blue.svg) ![React](https://img.shields.io/badge/React-19.0-blue.svg)
@@ -11,32 +15,31 @@
--- ---
<blockquote> <blockquote>
<em><strong>数据之洋浩瀚无涯,当以洞察之目引航,然其脉络深隐,非表象所能尽窥。</strong></em><br>
<em><strong>The ocean of data is boundless, let the eye of insight guide the voyage, yet its intricate connections lie deep, not fully discernible from the surface.</strong></em> <em><strong>The ocean of data is boundless, let the eye of insight guide the voyage, yet its intricate connections lie deep, not fully discernible from the surface.</strong></em>
</blockquote> </blockquote>
</div> </div>
## 👀 在线体验 ## 👀 Online Demo
> [https://demo.foxel.cc](https://demo.foxel.cc) > [https://demo.foxel.cc](https://demo.foxel.cc)
> >
> 账号/密码:`admin` / `admin` > Account/Password: `admin` / `admin`
## ✨ 核心功能 ## ✨ Core Features
- **统一文件管理**:集中管理分布于不同存储后端的文件。 - **Unified File Management**: Centralize management of files distributed across different storage backends.
- **插件化存储后端**:采用可扩展的适配器模式,方便集成多种存储类型。 - **Pluggable Storage Backends**: Utilizes an extensible adapter pattern to easily integrate various storage types.
- **语义搜索**:支持自然语言描述搜索图片、文档等非结构化数据内容。 - **Semantic Search**: Supports natural language search for content within unstructured data like images and documents.
- **内置文件预览**可直接预览图片、视频、PDF、Office 文档及文本、代码文件,无需下载。 - **Built-in File Preview**: Preview images, videos, PDFs, Office documents, text, and code files directly without downloading.
- **权限与分享**:支持公开或私密分享链接,便于文件共享。 - **Permissions and Sharing**: Supports public or private sharing links for easy file distribution.
- **任务处理中心**:支持异步任务处理,如文件索引和数据备份,不影响主应用运行。 - **Task Processing Center**: Supports asynchronous task processing, such as file indexing and data backups, without impacting the main application.
## 🚀 快速开始 ## 🚀 Quick Start
使用 Docker Compose 是启动 Foxel 最推荐的方式。 Using Docker Compose is the most recommended way to start Foxel.
1. **创建数据目录**: 1. **Create Data Directories**:
新建 `data` 文件夹用于持久化数据: Create a `data` folder for persistent data:
```bash ```bash
mkdir -p data/db mkdir -p data/db
@@ -44,40 +47,40 @@ mkdir -p data/mount
chmod 777 data/db data/mount chmod 777 data/db data/mount
``` ```
2. **下载 Docker Compose 文件** 2. **Download Docker Compose File**:
```bash ```bash
curl -L -O https://github.com/DrizzleTime/Foxel/raw/main/compose.yaml curl -L -O https://github.com/DrizzleTime/Foxel/raw/main/compose.yaml
``` ```
下载完成后,**强烈建议**修改 `compose.yaml` 文件中的环境变量以确保安全: After downloading, it is **strongly recommended** to modify the environment variables in the `compose.yaml` file to ensure security:
- 修改 `SECRET_KEY` `TEMP_LINK_SECRET_KEY`:将默认的密钥替换为随机生成的强密钥 - Modify `SECRET_KEY` and `TEMP_LINK_SECRET_KEY`: Replace the default keys with randomly generated strong keys.
3. **启动服务**: 3. **Start the Services**:
```bash ```bash
docker-compose up -d docker-compose up -d
``` ```
4. **访问应用**: 4. **Access the Application**:
服务启动后,在浏览器中打开页面。 Once the services are running, open the page in your browser.
> 首次启动,请根据引导页面完成管理员账号的初始化设置。 > On the first launch, please follow the setup guide to initialize the administrator account.
## 🤝 如何贡献 ## 🤝 How to Contribute
我们非常欢迎来自社区的贡献!无论是提交 Bug、建议新功能还是直接贡献代码。 We welcome contributions from the community! Whether it's submitting bugs, suggesting new features, or contributing code directly.
在开始之前,请先阅读我们的 [`CONTRIBUTING.md`](CONTRIBUTING.md) 文件,它会指导你如何设置开发环境以及提交流程。 Before you start, please read our [`CONTRIBUTING.md`](CONTRIBUTING.md) file, which will guide you on how to set up your development environment and the submission process.
## 🌐 社区 ## 🌐 Community
加入我们的交流社区:[Telegram 群组](https://t.me/+thDsBfyqJxZkNTU1),与开发者和用户一起讨论! Join our community on [Telegram](https://t.me/+thDsBfyqJxZkNTU1) to discuss with developers and other users!
你也可以加入我们的微信群,获取更多实时交流与支持。请扫描下方二维码加入: You can also join our WeChat group for more real-time communication and support. Please scan the QR code below to join:
<img src="https://foxel.cc/image/wechat.png" alt="微信群二维码" width="180"> <img src="https://foxel.cc/image/wechat.png" alt="WeChat Group QR Code" width="180">
> 如果二维码失效,请添加微信号 **drizzle2001**,我们会邀请你加入群聊。 > If the QR code is invalid, please add WeChat ID **drizzle2001**, and we will invite you to the group.

87
README_zh.md Normal file
View File

@@ -0,0 +1,87 @@
<div align="right">
<a href="./README.md">English</a> | <b>简体中文</b>
</div>
<div align="center">
# Foxel
**一个面向个人和团队的、高度可扩展的私有云盘解决方案,支持 AI 语义搜索。**
![Python Version](https://img.shields.io/badge/Python-3.13+-blue.svg)
![React](https://img.shields.io/badge/React-19.0-blue.svg)
![License](https://img.shields.io/badge/license-MIT-green.svg)
![GitHub stars](https://img.shields.io/github/stars/DrizzleTime/foxel?style=social)
---
<blockquote>
<em><strong>数据之洋浩瀚无涯,当以洞察之目引航,然其脉络深隐,非表象所能尽窥。</strong></em><br>
<em><strong>The ocean of data is boundless, let the eye of insight guide the voyage, yet its intricate connections lie deep, not fully discernible from the surface.</strong></em>
</blockquote>
</div>
## 👀 在线体验
> [https://demo.foxel.cc](https://demo.foxel.cc)
>
> 账号/密码:`admin` / `admin`
## ✨ 核心功能
- **统一文件管理**:集中管理分布于不同存储后端的文件。
- **插件化存储后端**:采用可扩展的适配器模式,方便集成多种存储类型。
- **语义搜索**:支持自然语言描述搜索图片、文档等非结构化数据内容。
- **内置文件预览**可直接预览图片、视频、PDF、Office 文档及文本、代码文件,无需下载。
- **权限与分享**:支持公开或私密分享链接,便于文件共享。
- **任务处理中心**:支持异步任务处理,如文件索引和数据备份,不影响主应用运行。
## 🚀 快速开始
使用 Docker Compose 是启动 Foxel 最推荐的方式。
1. **创建数据目录**:
新建 `data` 文件夹用于持久化数据:
```bash
mkdir -p data/db
mkdir -p data/mount
chmod 777 data/db data/mount
```
2. **下载 Docker Compose 文件**
```bash
curl -L -O https://github.com/DrizzleTime/Foxel/raw/main/compose.yaml
```
下载完成后,**强烈建议**修改 `compose.yaml` 文件中的环境变量以确保安全:
- 修改 `SECRET_KEY` 和 `TEMP_LINK_SECRET_KEY`:将默认的密钥替换为随机生成的强密钥
3. **启动服务**:
```bash
docker-compose up -d
```
4. **访问应用**:
服务启动后,在浏览器中打开页面。
> 首次启动,请根据引导页面完成管理员账号的初始化设置。
## 🤝 如何贡献
我们非常欢迎来自社区的贡献!无论是提交 Bug、建议新功能还是直接贡献代码。
在开始之前,请先阅读我们的 [`CONTRIBUTING.md`](CONTRIBUTING.md) 文件,它会指导你如何设置开发环境以及提交流程。
## 🌐 社区
加入我们的交流社区:[Telegram 群组](https://t.me/+thDsBfyqJxZkNTU1),与开发者和用户一起讨论!
你也可以加入我们的微信群,获取更多实时交流与支持。请扫描下方二维码加入:
<img src="https://foxel.cc/image/wechat.png" alt="微信群二维码" width="180">
> 如果二维码失效,请添加微信号 **drizzle2001**,我们会邀请你加入群聊。

View File

@@ -41,7 +41,9 @@ async def get_system_status():
"version": VERSION, "version": VERSION,
"title": await ConfigCenter.get("APP_NAME", "Foxel"), "title": await ConfigCenter.get("APP_NAME", "Foxel"),
"logo": await ConfigCenter.get("APP_LOGO", "/logo.svg"), "logo": await ConfigCenter.get("APP_LOGO", "/logo.svg"),
"is_initialized": await has_users() "is_initialized": await has_users(),
"app_domain": await ConfigCenter.get("APP_DOMAIN"),
"file_domain": await ConfigCenter.get("FILE_DOMAIN"),
} }
return success(system_info) return success(system_info)

View File

@@ -1,7 +1,7 @@
from fastapi import APIRouter, Depends, Body from fastapi import APIRouter, Depends, Body
from typing import Annotated from typing import Annotated
from services.processors.registry import get_config_schemas from services.processors.registry import get_config_schemas
from services.virtual_fs import process_file from services.task_queue import task_queue_service
from services.auth import get_current_active_user, User from services.auth import get_current_active_user, User
from api.response import success from api.response import success
from pydantic import BaseModel from pydantic import BaseModel
@@ -21,7 +21,7 @@ async def list_processors(
"name": meta["name"], "name": meta["name"],
"supported_exts": meta.get("supported_exts", []), "supported_exts": meta.get("supported_exts", []),
"config_schema": meta["config_schema"], "config_schema": meta["config_schema"],
"produces_file": meta.get("produces_file", False), "produces_file": meta.get("produces_file", False),
}) })
return success(out) return success(out)
@@ -40,5 +40,13 @@ async def process_file_with_processor(
req: ProcessRequest = Body(...) req: ProcessRequest = Body(...)
): ):
save_to = req.path if req.overwrite else req.save_to save_to = req.path if req.overwrite else req.save_to
result = await process_file(req.path, req.processor_type, req.config, save_to) task = await task_queue_service.add_task(
return success(result) "process_file",
{
"path": req.path,
"processor_type": req.processor_type,
"config": req.config,
"save_to": save_to,
},
)
return success({"task_id": task.id})

View File

@@ -6,6 +6,7 @@ from schemas.tasks import AutomationTaskCreate, AutomationTaskUpdate
from api.response import success from api.response import success
from services.auth import get_current_active_user, User from services.auth import get_current_active_user, User
from services.logging import LogService from services.logging import LogService
from services.task_queue import task_queue_service
router = APIRouter( router = APIRouter(
prefix="/api/tasks", prefix="/api/tasks",
@@ -15,6 +16,25 @@ router = APIRouter(
) )
@router.get("/queue")
async def get_task_queue_status(
current_user: Annotated[User, Depends(get_current_active_user)],
):
tasks = task_queue_service.get_all_tasks()
return success([task.dict() for task in tasks])
@router.get("/queue/{task_id}")
async def get_task_status(
task_id: str,
current_user: Annotated[User, Depends(get_current_active_user)],
):
task = task_queue_service.get_task(task_id)
if not task:
raise HTTPException(status_code=404, detail="Task not found")
return success(task.dict())
@router.post("/") @router.post("/")
async def create_task( async def create_task(
task_in: AutomationTaskCreate, task_in: AutomationTaskCreate,

View File

@@ -19,6 +19,7 @@ from services.virtual_fs import (
from services.thumbnail import is_image_filename, get_or_create_thumb, is_raw_filename from services.thumbnail import is_image_filename, get_or_create_thumb, is_raw_filename
from schemas import MkdirRequest, MoveRequest from schemas import MkdirRequest, MoveRequest
from api.response import success from api.response import success
from services.config import ConfigCenter
router = APIRouter(prefix='/api/fs', tags=["virtual-fs"]) router = APIRouter(prefix='/api/fs', tags=["virtual-fs"])
@@ -151,7 +152,13 @@ async def get_temp_link(
"""获取文件的临时公开访问令牌""" """获取文件的临时公开访问令牌"""
full_path = '/' + full_path if not full_path.startswith('/') else full_path full_path = '/' + full_path if not full_path.startswith('/') else full_path
token = await generate_temp_link_token(full_path, expires_in=expires_in) token = await generate_temp_link_token(full_path, expires_in=expires_in)
return success({"token": token, "path": full_path}) file_domain = await ConfigCenter.get("FILE_DOMAIN")
if file_domain:
file_domain = file_domain.rstrip('/')
url = f"{file_domain}/api/fs/public/{token}"
else:
url = f"/api/fs/public/{token}"
return success({"token": token, "path": full_path, "url": url})
@router.get("/public/{token}") @router.get("/public/{token}")

View File

@@ -8,6 +8,7 @@ from fastapi import FastAPI
from services.middleware.logging_middleware import LoggingMiddleware from services.middleware.logging_middleware import LoggingMiddleware
from services.middleware.exception_handler import global_exception_handler from services.middleware.exception_handler import global_exception_handler
from dotenv import load_dotenv from dotenv import load_dotenv
from services.task_queue import task_queue_service
load_dotenv() load_dotenv()
@@ -17,9 +18,11 @@ async def lifespan(app: FastAPI):
await init_db() await init_db()
await runtime_registry.refresh() await runtime_registry.refresh()
await ConfigCenter.set("APP_VERSION", VERSION) await ConfigCenter.set("APP_VERSION", VERSION)
await task_queue_service.start_worker()
try: try:
yield yield
finally: finally:
await task_queue_service.stop_worker()
await close_db() await close_db()

View File

@@ -1,5 +1,7 @@
from __future__ import annotations from __future__ import annotations
from typing import List, Dict, Tuple, AsyncIterator from typing import List, Dict, Tuple, AsyncIterator
import io
import os
from models import StorageAdapter from models import StorageAdapter
from telethon import TelegramClient from telethon import TelegramClient
from telethon.sessions import StringSession from telethon.sessions import StringSession
@@ -20,7 +22,7 @@ CONFIG_SCHEMA = [
] ]
class TelegramAdapter: class TelegramAdapter:
"""Telegram 存储适配器 (只读, 使用用户 Session)""" """Telegram 存储适配器 (使用用户 Session)"""
def __init__(self, record: StorageAdapter): def __init__(self, record: StorageAdapter):
self.record = record self.record = record
@@ -70,30 +72,43 @@ class TelegramAdapter:
await client.connect() await client.connect()
messages = await client.get_messages(self.chat_id, limit=50) messages = await client.get_messages(self.chat_id, limit=50)
for message in messages: for message in messages:
if message and (message.document or message.video): if not message:
media = message.document or message.video continue
filename = None
media = message.document or message.video or message.photo
if not media:
continue
filename = None
size = 0
if message.photo:
photo_size = message.photo.sizes[-1]
size = photo_size.size if hasattr(photo_size, 'size') else 0
filename = f"photo_{message.id}.jpg"
elif message.document or message.video:
size = media.size
if hasattr(media, 'attributes'): if hasattr(media, 'attributes'):
for attr in media.attributes: for attr in media.attributes:
if hasattr(attr, 'file_name') and attr.file_name: if hasattr(attr, 'file_name') and attr.file_name:
filename = attr.file_name filename = attr.file_name
break break
if not filename: if not filename:
if message.text and '.' in message.text: if message.text and '.' in message.text and len(message.text) < 256 and '\n' not in message.text:
if len(message.text) < 256 and '\n' not in message.text: filename = message.text
filename = message.text
if not filename:
filename = f"unknown_{message.id}"
if not filename: entries.append({
filename = "Unknown" "name": f"{message.id}_{filename}",
"is_dir": False,
entries.append({ "size": size,
"name": f"{message.id}_{filename}", "mtime": int(message.date.timestamp()),
"is_dir": False, "type": "file",
"size": media.size, })
"mtime": int(message.date.timestamp()),
"type": "file",
})
finally: finally:
if client.is_connected(): if client.is_connected():
await client.disconnect() await client.disconnect()
@@ -111,7 +126,7 @@ class TelegramAdapter:
try: try:
await client.connect() await client.connect()
message = await client.get_messages(self.chat_id, ids=message_id) message = await client.get_messages(self.chat_id, ids=message_id)
if not message or not (message.document or message.video): if not message or not (message.document or message.video or message.photo):
raise FileNotFoundError(f"在频道 {self.chat_id} 中未找到消息ID为 {message_id} 的文件") raise FileNotFoundError(f"在频道 {self.chat_id} 中未找到消息ID为 {message_id} 的文件")
file_bytes = await client.download_media(message, file=bytes) file_bytes = await client.download_media(message, file=bytes)
@@ -121,25 +136,73 @@ class TelegramAdapter:
await client.disconnect() await client.disconnect()
async def write_file(self, root: str, rel: str, data: bytes): async def write_file(self, root: str, rel: str, data: bytes):
raise NotImplementedError("Telegram 适配器是只读的,不支持写入文件。") """将字节数据作为文件上传"""
client = self._get_client()
file_like = io.BytesIO(data)
file_like.name = os.path.basename(rel) or "file"
try:
await client.connect()
await client.send_file(self.chat_id, file_like, caption=file_like.name)
finally:
if client.is_connected():
await client.disconnect()
async def write_file_stream(self, root: str, rel: str, data_iter: AsyncIterator[bytes]): async def write_file_stream(self, root: str, rel: str, data_iter: AsyncIterator[bytes]):
raise NotImplementedError("Telegram 适配器是只读的,不支持流式写入文件") """以流式方式上传文件"""
client = self._get_client()
filename = os.path.basename(rel) or "file"
import tempfile
temp_dir = tempfile.gettempdir()
temp_path = os.path.join(temp_dir, filename)
total_size = 0
try:
with open(temp_path, "wb") as f:
async for chunk in data_iter:
if chunk:
f.write(chunk)
total_size += len(chunk)
await client.connect()
await client.send_file(self.chat_id, temp_path, caption=filename)
finally:
if os.path.exists(temp_path):
os.remove(temp_path)
if client.is_connected():
await client.disconnect()
return total_size
async def mkdir(self, root: str, rel: str): async def mkdir(self, root: str, rel: str):
raise NotImplementedError("Telegram 适配器是只读的,不支持创建目录。") raise NotImplementedError("Telegram 适配器不支持创建目录。")
async def delete(self, root: str, rel: str): async def delete(self, root: str, rel: str):
raise NotImplementedError("Telegram 适配器是只读的,不支持删除。") """删除一个文件 (即一条消息)"""
try:
message_id_str, _ = rel.split('_', 1)
message_id = int(message_id_str)
except (ValueError, IndexError):
raise FileNotFoundError(f"无效的文件路径格式无法解析消息ID: {rel}")
client = self._get_client()
try:
await client.connect()
result = await client.delete_messages(self.chat_id, [message_id])
if not result or not result[0].pts:
raise FileNotFoundError(f"{self.chat_id} 中删除消息 {message_id} 失败,可能消息不存在或无权限")
finally:
if client.is_connected():
await client.disconnect()
async def move(self, root: str, src_rel: str, dst_rel: str): async def move(self, root: str, src_rel: str, dst_rel: str):
raise NotImplementedError("Telegram 适配器是只读的,不支持移动。") raise NotImplementedError("Telegram 适配器不支持移动。")
async def rename(self, root: str, src_rel: str, dst_rel: str): async def rename(self, root: str, src_rel: str, dst_rel: str):
raise NotImplementedError("Telegram 适配器是只读的,不支持重命名。") raise NotImplementedError("Telegram 适配器不支持重命名。")
async def copy(self, root: str, src_rel: str, dst_rel: str, overwrite: bool = False): async def copy(self, root: str, src_rel: str, dst_rel: str, overwrite: bool = False):
raise NotImplementedError("Telegram 适配器是只读的,不支持复制。") raise NotImplementedError("Telegram 适配器不支持复制。")
async def stream_file(self, root: str, rel: str, range_header: str | None): async def stream_file(self, root: str, rel: str, range_header: str | None):
from fastapi.responses import StreamingResponse from fastapi.responses import StreamingResponse
@@ -156,19 +219,25 @@ class TelegramAdapter:
try: try:
await client.connect() await client.connect()
message = await client.get_messages(self.chat_id, ids=message_id) message = await client.get_messages(self.chat_id, ids=message_id)
if not message or not (message.document or message.video): media = message.document or message.video or message.photo
if not message or not media:
raise FileNotFoundError(f"在频道 {self.chat_id} 中未找到消息ID为 {message_id} 的文件") raise FileNotFoundError(f"在频道 {self.chat_id} 中未找到消息ID为 {message_id} 的文件")
media = message.document or message.video if message.photo:
file_size = media.size photo_size = media.sizes[-1]
file_size = photo_size.size if hasattr(photo_size, 'size') else 0
mime_type = "image/jpeg"
else:
file_size = media.size
mime_type = media.mime_type or "application/octet-stream"
start = 0 start = 0
end = file_size - 1 end = file_size - 1
status = 200 status = 200
headers = { headers = {
"Accept-Ranges": "bytes", "Accept-Ranges": "bytes",
"Content-Type": media.mime_type or "application/octet-stream", "Content-Type": mime_type,
"Content-Length": str(file_size), "Content-Length": str(file_size),
} }
@@ -225,14 +294,20 @@ class TelegramAdapter:
try: try:
await client.connect() await client.connect()
message = await client.get_messages(self.chat_id, ids=message_id) message = await client.get_messages(self.chat_id, ids=message_id)
if not message or not (message.document or message.video): media = message.document or message.video or message.photo
if not message or not media:
raise FileNotFoundError(f"在频道 {self.chat_id} 中未找到消息ID为 {message_id} 的文件") raise FileNotFoundError(f"在频道 {self.chat_id} 中未找到消息ID为 {message_id} 的文件")
media = message.document or message.video if message.photo:
photo_size = media.sizes[-1]
size = photo_size.size if hasattr(photo_size, 'size') else 0
else:
size = media.size
return { return {
"name": rel, "name": rel,
"is_dir": False, "is_dir": False,
"size": media.size, "size": size,
"mtime": int(message.date.timestamp()), "mtime": int(message.date.timestamp()),
"type": "file", "type": "file",
} }

View File

@@ -4,7 +4,7 @@ from typing import Any, Optional, Dict
from dotenv import load_dotenv from dotenv import load_dotenv
from models.database import Configuration from models.database import Configuration
load_dotenv(dotenv_path=".env") load_dotenv(dotenv_path=".env")
VERSION = "v1.1.2" VERSION = "v1.1.3"
class ConfigCenter: class ConfigCenter:
_cache: Dict[str, Any] = {} _cache: Dict[str, Any] = {}

122
services/task_queue.py Normal file
View File

@@ -0,0 +1,122 @@
import asyncio
from typing import Dict, Any
from pydantic import BaseModel, Field
import uuid
from services.logging import LogService
from enum import Enum
class TaskStatus(str, Enum):
PENDING = "pending"
RUNNING = "running"
SUCCESS = "success"
FAILED = "failed"
class Task(BaseModel):
id: str = Field(default_factory=lambda: uuid.uuid4().hex)
name: str
status: TaskStatus = TaskStatus.PENDING
result: Any = None
error: str | None = None
task_info: Dict[str, Any] = {}
class TaskQueueService:
def __init__(self):
self._queue = asyncio.Queue()
self._tasks: Dict[str, Task] = {}
self._worker_task: asyncio.Task | None = None
async def add_task(self, name: str, task_info: Dict[str, Any]) -> Task:
task = Task(name=name, task_info=task_info)
self._tasks[task.id] = task
await self._queue.put(task)
await LogService.info("task_queue", f"Task {name} ({task.id}) enqueued", {"task_id": task.id, "name": name})
return task
def get_task(self, task_id: str) -> Task | None:
return self._tasks.get(task_id)
def get_all_tasks(self) -> list[Task]:
return list(self._tasks.values())
async def _execute_task(self, task: Task):
from services.virtual_fs import process_file
task.status = TaskStatus.RUNNING
await LogService.info("task_queue", f"Task {task.name} ({task.id}) started", {"task_id": task.id, "name": task.name})
try:
if task.name == "process_file":
params = task.task_info
result = await process_file(
path=params["path"],
processor_type=params["processor_type"],
config=params["config"],
save_to=params["save_to"]
)
task.result = result
elif task.name == "automation_task":
from models.database import AutomationTask
from services.processors.registry import get as get_processor
from services.virtual_fs import read_file, write_file
params = task.task_info
auto_task = await AutomationTask.get(id=params["task_id"])
path = params["path"]
processor = get_processor(auto_task.processor_type)
if not processor:
raise ValueError(f"Processor {auto_task.processor_type} not found for task {auto_task.id}")
file_content = await read_file(path)
result = await processor.process(file_content, path, auto_task.processor_config)
save_to = auto_task.processor_config.get("save_to")
if save_to and getattr(processor, "produces_file", False):
await write_file(save_to, result)
task.result = "Automation task completed"
else:
raise ValueError(f"Unknown task name: {task.name}")
task.status = TaskStatus.SUCCESS
await LogService.info("task_queue", f"Task {task.name} ({task.id}) succeeded", {"task_id": task.id, "name": task.name})
except Exception as e:
task.status = TaskStatus.FAILED
task.error = str(e)
await LogService.error("task_queue", f"Task {task.name} ({task.id}) failed: {e}", {"task_id": task.id, "name": task.name})
async def worker(self):
await LogService.info("task_queue", "Task worker started")
while True:
try:
task = await self._queue.get()
await self._execute_task(task)
except asyncio.CancelledError:
await LogService.info("task_queue", "Task worker stopped")
break
except Exception as e:
await LogService.error("task_queue", f"Error in task worker: {e}", exc_info=True)
finally:
self._queue.task_done()
async def start_worker(self):
if self._worker_task is None or self._worker_task.done():
self._worker_task = asyncio.create_task(self.worker())
await LogService.info("task_queue", "Task worker created.")
async def stop_worker(self):
if self._worker_task and not self._worker_task.done():
self._worker_task.cancel()
try:
await self._worker_task
except asyncio.CancelledError:
pass
finally:
self._worker_task = None
await LogService.info("task_queue", "Task worker has been stopped.")
task_queue_service = TaskQueueService()

View File

@@ -4,6 +4,9 @@ from models.database import AutomationTask
from services.processors.registry import get as get_processor from services.processors.registry import get as get_processor
from services.logging import LogService from services.logging import LogService
from services.task_queue import task_queue_service
class TaskService: class TaskService:
async def trigger_tasks(self, event: str, path: str): async def trigger_tasks(self, event: str, path: str):
tasks = await AutomationTask.filter(event=event, enabled=True) tasks = await AutomationTask.filter(event=event, enabled=True)
@@ -21,28 +24,12 @@ class TaskService:
return True return True
async def execute(self, task: AutomationTask, path: str): async def execute(self, task: AutomationTask, path: str):
from services.virtual_fs import read_file, write_file await task_queue_service.add_task(
"automation_task",
processor = get_processor(task.processor_type) {
if not processor: "task_id": task.id,
print(f"Processor {task.processor_type} not found for task {task.id}") "path": path,
return },
)
try:
file_content = await read_file(path)
result = await processor.process(file_content, path, task.processor_config)
save_to = task.processor_config.get("save_to")
if save_to and getattr(processor, "produces_file", False):
await write_file(save_to, result)
except Exception as e:
error_message = f"Error executing task {task.id} for path {path}: {e}"
print(error_message)
await LogService.error(
source=f"task_executor:{task.id}",
message=error_message,
details={"task_name": task.name, "event": task.event, "path": path, "processor": task.processor_type}
)
task_service = TaskService() task_service = TaskService()

View File

@@ -20,6 +20,8 @@ export interface SystemStatus {
title: string; title: string;
logo: string; logo: string;
is_initialized: boolean; is_initialized: boolean;
app_domain?: string;
file_domain?: string;
} }
export async function status() { export async function status() {

View File

@@ -51,7 +51,7 @@ export const vfsApi = {
streamUrl: (path: string) => `${API_BASE_URL}/fs/stream/${encodeURI(path.replace(/^\/+/, ''))}`, streamUrl: (path: string) => `${API_BASE_URL}/fs/stream/${encodeURI(path.replace(/^\/+/, ''))}`,
stat: (path: string) => request(`/fs/stat/${encodeURI(path.replace(/^\/+/, ''))}`), stat: (path: string) => request(`/fs/stat/${encodeURI(path.replace(/^\/+/, ''))}`),
getTempLinkToken: (path: string, expiresIn: number = 3600) => getTempLinkToken: (path: string, expiresIn: number = 3600) =>
request<{token: string}>(`/fs/temp-link/${encodeURI(path.replace(/^\/+/, ''))}?expires_in=${expiresIn}`), request<{token: string, path: string, url: string}>(`/fs/temp-link/${encodeURI(path.replace(/^\/+/, ''))}?expires_in=${expiresIn}`),
getTempPublicUrl: (token: string) => `${API_BASE_URL}/fs/public/${token}`, getTempPublicUrl: (token: string) => `${API_BASE_URL}/fs/public/${token}`,
uploadStream: (fullPath: string, file: File, overwrite: boolean = true, onProgress?: (loaded: number, total: number) => void) => { uploadStream: (fullPath: string, file: File, overwrite: boolean = true, onProgress?: (loaded: number, total: number) => void) => {
const enc = encodeURI(fullPath.replace(/^\/+/, '')); const enc = encodeURI(fullPath.replace(/^\/+/, ''));

View File

@@ -2,8 +2,10 @@ import React, { useEffect, useState } from 'react';
import { vfsApi } from '../../api/client'; import { vfsApi } from '../../api/client';
import type { AppComponentProps } from '../types'; import type { AppComponentProps } from '../types';
import { Spin, Result, Button } from 'antd'; import { Spin, Result, Button } from 'antd';
import { useSystemStatus } from '../../contexts/SystemContext';
export const OfficeViewerApp: React.FC<AppComponentProps> = ({ filePath, onRequestClose }) => { export const OfficeViewerApp: React.FC<AppComponentProps> = ({ filePath, onRequestClose }) => {
const systemStatus = useSystemStatus();
const [url, setUrl] = useState<string>(); const [url, setUrl] = useState<string>();
const [loading, setLoading] = useState(true); const [loading, setLoading] = useState(true);
const [err, setErr] = useState<string>(); const [err, setErr] = useState<string>();
@@ -17,8 +19,8 @@ export const OfficeViewerApp: React.FC<AppComponentProps> = ({ filePath, onReque
vfsApi.getTempLinkToken(filePath.replace(/^\/+/, '')) vfsApi.getTempLinkToken(filePath.replace(/^\/+/, ''))
.then(res => { .then(res => {
if (cancelled) return; if (cancelled) return;
// 注意vfsApi.getTempPublicUrl 返回的是相对路径,我们需要构建完整的 URL const baseUrl = systemStatus?.file_domain || window.location.origin;
const fullUrl = new URL(vfsApi.getTempPublicUrl(res.token), window.location.origin).href; const fullUrl = new URL(res.url, baseUrl).href;
const officeUrl = `https://view.officeapps.live.com/op/embed.aspx?src=${encodeURIComponent(fullUrl)}`; const officeUrl = `https://view.officeapps.live.com/op/embed.aspx?src=${encodeURIComponent(fullUrl)}`;
setUrl(officeUrl); setUrl(officeUrl);
}) })

View File

@@ -29,8 +29,7 @@ export const DirectLinkModal = memo(function DirectLinkModal({ entry, path, open
try { try {
const fullPath = (path === '/' ? '' : path) + '/' + entry.name; const fullPath = (path === '/' ? '' : path) + '/' + entry.name;
const res = await vfsApi.getTempLinkToken(fullPath, expiresIn); const res = await vfsApi.getTempLinkToken(fullPath, expiresIn);
const tempLink = `${window.location.origin}/api/fs/public/${res.token}`; setLink(res.url);
setLink(tempLink);
} catch (e: any) { } catch (e: any) {
message.error(e.message || '生成链接失败'); message.error(e.message || '生成链接失败');
} finally { } finally {

View File

@@ -3,6 +3,7 @@ import { Modal, Form, Input, Radio, InputNumber, message, Button, Typography } f
import { CopyOutlined } from '@ant-design/icons'; import { CopyOutlined } from '@ant-design/icons';
import type { VfsEntry, ShareInfoWithPassword } from '../../../../api/client'; import type { VfsEntry, ShareInfoWithPassword } from '../../../../api/client';
import { shareApi } from '../../../../api/share'; import { shareApi } from '../../../../api/share';
import { useSystemStatus } from '../../../../contexts/SystemContext';
interface ShareModalProps { interface ShareModalProps {
entries: VfsEntry[]; entries: VfsEntry[];
@@ -13,6 +14,7 @@ interface ShareModalProps {
} }
export const ShareModal = memo(function ShareModal({ entries, path, open, onOk, onCancel }: ShareModalProps) { export const ShareModal = memo(function ShareModal({ entries, path, open, onOk, onCancel }: ShareModalProps) {
const systemStatus = useSystemStatus();
const [form] = Form.useForm(); const [form] = Form.useForm();
const [loading, setLoading] = useState(false); const [loading, setLoading] = useState(false);
const [accessType, setAccessType] = useState('public'); const [accessType, setAccessType] = useState('public');
@@ -66,7 +68,8 @@ export const ShareModal = memo(function ShareModal({ entries, path, open, onOk,
message.success('已复制到剪贴板'); message.success('已复制到剪贴板');
}; };
const shareUrl = createdShare ? `${window.location.origin}/share/${createdShare.token}` : ''; const baseUrl = systemStatus?.app_domain || window.location.origin;
const shareUrl = createdShare ? new URL(`/share/${createdShare.token}`, baseUrl).href : '';
const renderForm = () => ( const renderForm = () => (
<Form form={form} layout="vertical" initialValues={{ name: defaultName, accessType: 'public', expiresInDays: 7 }}> <Form form={form} layout="vertical" initialValues={{ name: defaultName, accessType: 'public', expiresInDays: 7 }}>

View File

@@ -4,8 +4,10 @@ import PageCard from '../components/PageCard';
import { shareApi, type ShareInfo } from '../api/share'; import { shareApi, type ShareInfo } from '../api/share';
import { format, parseISO } from 'date-fns'; import { format, parseISO } from 'date-fns';
import { LinkOutlined, CopyOutlined, DeleteOutlined } from '@ant-design/icons'; import { LinkOutlined, CopyOutlined, DeleteOutlined } from '@ant-design/icons';
import { useSystemStatus } from '../contexts/SystemContext';
const SharePage = memo(function SharePage() { const SharePage = memo(function SharePage() {
const systemStatus = useSystemStatus();
const [loading, setLoading] = useState(false); const [loading, setLoading] = useState(false);
const [data, setData] = useState<ShareInfo[]>([]); const [data, setData] = useState<ShareInfo[]>([]);
@@ -24,7 +26,8 @@ const SharePage = memo(function SharePage() {
useEffect(() => { fetchList(); }, [fetchList]); useEffect(() => { fetchList(); }, [fetchList]);
const doCopy = (rec: ShareInfo) => { const doCopy = (rec: ShareInfo) => {
const shareUrl = `${window.location.origin}/share/${rec.token}`; const baseUrl = systemStatus?.app_domain || window.location.origin;
const shareUrl = new URL(`/share/${rec.token}`, baseUrl).href;
navigator.clipboard.writeText(shareUrl); navigator.clipboard.writeText(shareUrl);
message.success('链接已复制'); message.success('链接已复制');
}; };

View File

@@ -2,13 +2,13 @@ import { Form, Input, Button, message, Tabs, Space, Card } from 'antd';
import { useEffect, useState } from 'react'; import { useEffect, useState } from 'react';
import PageCard from '../../components/PageCard'; import PageCard from '../../components/PageCard';
import { getAllConfig, setConfig } from '../../api/config'; import { getAllConfig, setConfig } from '../../api/config';
import { API_BASE_URL } from '../../api/client';
import { AppstoreOutlined, RobotOutlined } from '@ant-design/icons'; import { AppstoreOutlined, RobotOutlined } from '@ant-design/icons';
const APP_CONFIG_KEYS = [ const APP_CONFIG_KEYS: {key: string, label: string, default?: string}[] = [
{ key: 'APP_NAME', label: '应用名称' }, { key: 'APP_NAME', label: '应用名称' },
{ key: 'APP_LOGO', label: 'LOGO地址' }, { key: 'APP_LOGO', label: 'LOGO地址' },
{ key: 'SERVER_URL', label: '服务端URL', default: API_BASE_URL }, { key: 'APP_DOMAIN', label: '应用域名' },
{ key: 'FILE_DOMAIN', label: '文件域名' },
]; ];
const VISION_CONFIG_KEYS = [ const VISION_CONFIG_KEYS = [