mirror of
https://github.com/halfwaystudent/douyin-sparkflow.git
synced 2026-09-06 16:07:22 +08:00
fix: harden scheduling and deployment flow
This commit is contained in:
-41
@@ -1,41 +0,0 @@
|
||||
name: DouYin Spark Flow Schedule Run
|
||||
on:
|
||||
workflow_dispatch: # 允许手动触发
|
||||
schedule: # 定时任务
|
||||
- cron: "0 1 * * *" # 每天 1:00 UTC(对应北京时间 9:00)
|
||||
jobs:
|
||||
run-main:
|
||||
timeout-minutes: 20
|
||||
runs-on: ubuntu-latest
|
||||
environment: user-data
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: '3.11'
|
||||
- name: Test Douyin Accessibility
|
||||
run: |
|
||||
curl -I https://creator.douyin.com/
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install -r requirements.txt
|
||||
- name: Ensure browsers are installed
|
||||
run: playwright install chromium --with-deps --only-shell
|
||||
- name: Run DouYin Spark Flow
|
||||
env:
|
||||
USER_DATA: ${{ secrets.USER_DATA }}
|
||||
run: python main.py --doTask
|
||||
- uses: actions/upload-artifact@v4
|
||||
if: ${{ !cancelled() }}
|
||||
with:
|
||||
name: run-logs
|
||||
path: logs/
|
||||
workflow-keepalive:
|
||||
if: github.event_name == 'schedule'
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
actions: write
|
||||
steps:
|
||||
- uses: liskin/gh-workflow-keepalive@v1
|
||||
@@ -6,3 +6,4 @@ logs/
|
||||
.DS_Store
|
||||
usersData.json
|
||||
webui_settings.json
|
||||
config.json
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
FROM mcr.microsoft.com/playwright/python:v1.56.0-jammy
|
||||
ARG PLAYWRIGHT_BASE_IMAGE=mcr.microsoft.com/playwright/python:v1.56.0-jammy
|
||||
FROM ${PLAYWRIGHT_BASE_IMAGE}
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
@@ -32,6 +33,7 @@ RUN sed -i 's/archive.ubuntu.com/mirrors.aliyun.com/g' /etc/apt/sources.list \
|
||||
&& apt-get update && apt-get install -y \
|
||||
cron \
|
||||
curl \
|
||||
docker.io \
|
||||
fluxbox \
|
||||
fonts-wqy-zenhei \
|
||||
fonts-noto-cjk \
|
||||
@@ -39,14 +41,8 @@ RUN sed -i 's/archive.ubuntu.com/mirrors.aliyun.com/g' /etc/apt/sources.list \
|
||||
websockify \
|
||||
x11vnc \
|
||||
xfonts-intl-chinese \
|
||||
&& curl -fsSL ${HTTP_PROXY:+-x "$HTTP_PROXY"} https://download.docker.com/linux/static/stable/x86_64/docker-25.0.3.tgz -o docker.tgz \
|
||||
&& tar xzvf docker.tgz \
|
||||
&& mv docker/docker /usr/bin/docker \
|
||||
&& chmod +x /usr/bin/docker \
|
||||
&& rm -rf docker docker.tgz \
|
||||
&& mkdir -p /usr/local/lib/docker/cli-plugins \
|
||||
&& curl -SL ${HTTP_PROXY:+-x "$HTTP_PROXY"} https://github.com/docker/compose/releases/download/v2.24.5/docker-compose-linux-x86_64 -o /usr/local/lib/docker/cli-plugins/docker-compose \
|
||||
&& chmod +x /usr/local/lib/docker/cli-plugins/docker-compose \
|
||||
&& docker --version \
|
||||
&& node --version \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
COPY . .
|
||||
|
||||
+23
-24
@@ -33,7 +33,6 @@
|
||||
|------|------|
|
||||
| `app.py` | FastAPI 主应用,路由定义 |
|
||||
| `auth.py` | 用户认证和会话管理 |
|
||||
| `login_sessions.py` | 登录会话生命周期管理 |
|
||||
| `ops.py` | 操作接口(启动/停止任务、刷新好友等) |
|
||||
|
||||
#### 前端资源
|
||||
@@ -73,11 +72,10 @@ webui/
|
||||
| `cron_runner.py` | Cron 任务运行器 |
|
||||
| `start_login_desktop.sh` | 登录桌面启动脚本 |
|
||||
|
||||
### `docs/` - 文档和资源
|
||||
### `docs/` - 截图资源
|
||||
|
||||
| 文件/目录 | 说明 |
|
||||
| 目录 | 说明 |
|
||||
|----------|------|
|
||||
| `usage.md` | 详细使用教程(含截图) |
|
||||
| `images/` | 界面截图和示意图 |
|
||||
|
||||
---
|
||||
@@ -138,9 +136,9 @@ docker run -d \
|
||||
|
||||
## ⚙️ 配置文件
|
||||
|
||||
### `config.json` - 应用配置
|
||||
### `config.example.json` 与 `config.json` - 应用配置
|
||||
|
||||
主配置文件控制发送窗口、好友扫描、浏览器 Profile 和消息策略。仓库版本不包含真实账号标识:
|
||||
`config.example.json` 是公开模板;首次运行会生成不受 Git 跟踪的 `config.json`,用于保存 Web 中修改的发送窗口、好友扫描、浏览器 Profile 和消息策略:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -151,7 +149,7 @@ docker run -d \
|
||||
"enabled": true,
|
||||
"startHour": 10,
|
||||
"endHour": 18,
|
||||
"scheduleIntervalMinutes": 10
|
||||
"scheduleIntervalMinutes": 20
|
||||
},
|
||||
"friendListScan": {
|
||||
"maxScanSeconds": 300,
|
||||
@@ -185,17 +183,17 @@ docker run -d \
|
||||
|
||||
示例结构:
|
||||
```json
|
||||
{
|
||||
"accounts": [
|
||||
{
|
||||
"id": "user_123",
|
||||
"nickname": "用户昵称",
|
||||
"friends": [...],
|
||||
"last_send_time": "2026-06-20T10:30:00",
|
||||
"send_history": [...]
|
||||
}
|
||||
]
|
||||
}
|
||||
[
|
||||
{
|
||||
"unique_id": "123456789",
|
||||
"username": "账号显示名",
|
||||
"cookies": [],
|
||||
"targets": ["目标好友"],
|
||||
"enabled": true,
|
||||
"message_history": {},
|
||||
"failure_queue": {}
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
### `webui_settings.json` - Web UI 设置
|
||||
@@ -265,8 +263,8 @@ Web UI 提供以下 RESTful API 接口:
|
||||
定时触发 → tasks.py 检查发送条件
|
||||
→ 筛选需要发送的好友
|
||||
→ msg_builder.py 构建消息内容
|
||||
→ protocol_sender.mjs 发送消息
|
||||
→ 等待发送确认
|
||||
→ 按配置选择 Playwright 浏览器发送或 protocol_sender.mjs 协议发送
|
||||
→ 等待强证据发送确认
|
||||
→ 记录发送历史
|
||||
→ 更新下次发送时间
|
||||
```
|
||||
@@ -318,7 +316,7 @@ A: 检查 `login_desktop_server.py` 是否正常运行,端口 18090 是否被
|
||||
A: 确保已安装 Playwright:`playwright install chromium`
|
||||
|
||||
**Q: 消息发送失败?**
|
||||
A: 检查网络连接,查看 `logs/tasks.log` 中的错误信息。
|
||||
A: 检查网络连接,查看 `logs/app.log` 或 Web 运行日志中的错误信息。
|
||||
|
||||
**Q: Web UI 无法访问?**
|
||||
A: 检查端口 8787 是否被占用,防火墙是否允许该端口。
|
||||
@@ -331,7 +329,7 @@ A: 检查端口 8787 是否被占用,防火墙是否允许该端口。
|
||||
|
||||
```
|
||||
playwright>=1.40.0 # 浏览器自动化
|
||||
apscheduler>=3.10.0 # 任务调度
|
||||
自定义 cron_runner.py # 任务调度(无需额外 Python 依赖)
|
||||
```
|
||||
|
||||
### `requirements-web.txt` - Web 依赖
|
||||
@@ -352,6 +350,7 @@ jinja2>=3.1.0 # 模板引擎
|
||||
|
||||
以下文件**绝对不要**提交到 Git 仓库:
|
||||
|
||||
- ❌ `config.json` - 运行时发送配置
|
||||
- ❌ `usersData.json` - 包含账号数据和好友信息
|
||||
- ❌ `webui_settings.json` - 包含管理员密码
|
||||
- ❌ `.env` - 包含环境变量和密钥
|
||||
@@ -372,8 +371,8 @@ jinja2>=3.1.0 # 模板引擎
|
||||
## 📚 相关文档
|
||||
|
||||
- [项目主 README](../README.md) - 整体介绍和快速开始
|
||||
- [使用文档](docs/usage.md) - 详细使用教程
|
||||
- [更新日志](CHANGELOG.md) - 版本更新历史
|
||||
- [使用文档](../docs/usage.md) - 详细使用教程
|
||||
- [更新日志](../CHANGELOG.md) - 版本更新历史
|
||||
- [Docker 部署](../docker-compose.yml) - 容器编排配置
|
||||
|
||||
---
|
||||
|
||||
@@ -24,7 +24,7 @@
|
||||
"enabled": true,
|
||||
"startHour": 10,
|
||||
"endHour": 18,
|
||||
"scheduleIntervalMinutes": 10
|
||||
"scheduleIntervalMinutes": 20
|
||||
},
|
||||
"hitokotoTypes": [
|
||||
"文学",
|
||||
@@ -33,7 +33,7 @@
|
||||
"哲学"
|
||||
],
|
||||
"happyNewYear": {
|
||||
"enabled": true,
|
||||
"enabled": false,
|
||||
"messageTemplate": "\r\n"
|
||||
},
|
||||
"friendListScan": {
|
||||
@@ -1,6 +1,4 @@
|
||||
import asyncio
|
||||
from pathlib import Path
|
||||
|
||||
from core.browser import get_browser
|
||||
|
||||
|
||||
|
||||
@@ -120,7 +120,11 @@ def _normalize_persistent_profile_config(active_config):
|
||||
raw = active_config.get("persistentBrowserProfiles", {}) or {}
|
||||
return {
|
||||
"enabled": bool(raw.get("enabled", False)),
|
||||
"root": str(raw.get("root") or "/opt/douyin-sparkflow/state/browser-profiles"),
|
||||
"root": str(
|
||||
os.getenv("SPARKFLOW_BROWSER_PROFILE_ROOT")
|
||||
or raw.get("root")
|
||||
or "/opt/douyin-sparkflow/state/browser-profiles"
|
||||
),
|
||||
"seedCookiesWhenEmpty": bool(raw.get("seedCookiesWhenEmpty", True)),
|
||||
"syncStoredCookiesBeforeRun": bool(raw.get("syncStoredCookiesBeforeRun", True)),
|
||||
"refreshStoredCookiesAfterLogin": bool(raw.get("refreshStoredCookiesAfterLogin", True)),
|
||||
|
||||
@@ -1,56 +0,0 @@
|
||||
services:
|
||||
proxy:
|
||||
image: metacubex/mihomo:latest
|
||||
container_name: mihomo
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "7890:7890"
|
||||
- "9090:9090"
|
||||
volumes:
|
||||
- ./proxy/config.yaml:/root/.config/mihomo/config.yaml
|
||||
|
||||
web:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile.server
|
||||
network: host
|
||||
args:
|
||||
HTTP_PROXY: http://127.0.0.1:7890
|
||||
HTTPS_PROXY: http://127.0.0.1:7890
|
||||
ALL_PROXY: socks5://127.0.0.1:7890
|
||||
image: douyin-sparkflow:local
|
||||
container_name: douyin-web
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
- proxy
|
||||
environment:
|
||||
TZ: Asia/Shanghai
|
||||
HTTP_PROXY: http://proxy:7890
|
||||
HTTPS_PROXY: http://proxy:7890
|
||||
ALL_PROXY: socks5://proxy:7890
|
||||
NO_PROXY: localhost,127.0.0.1,douyin.com,amemv.com,snssdk.com,bytedance.com,pstatp.com,volccdn.com,bytescm.com,byted.net,douyinstatic.com,bytecdn.cn,byteimg.com,bytegoofy.com,toutiaostatic.com
|
||||
ports:
|
||||
- "8787:8787"
|
||||
command: python main.py --web --host 0.0.0.0 --port 8787
|
||||
volumes:
|
||||
- .:/app
|
||||
- ./logs:/app/logs
|
||||
- /var/run/docker.sock:/var/run/docker.sock
|
||||
- /var/spool/cron/root:/var/spool/cron/crontabs/root
|
||||
|
||||
task:
|
||||
image: douyin-sparkflow:local
|
||||
container_name: douyin-task
|
||||
depends_on:
|
||||
- proxy
|
||||
environment:
|
||||
TZ: Asia/Shanghai
|
||||
HTTP_PROXY: http://proxy:7890
|
||||
HTTPS_PROXY: http://proxy:7890
|
||||
ALL_PROXY: socks5://proxy:7890
|
||||
NO_PROXY: localhost,127.0.0.1,douyin.com,amemv.com,snssdk.com,bytedance.com,pstatp.com,volccdn.com,bytescm.com,byted.net,douyinstatic.com,bytecdn.cn,byteimg.com,bytegoofy.com,toutiaostatic.com
|
||||
command: python main.py --doTask
|
||||
volumes:
|
||||
- .:/app
|
||||
- ./logs:/app/logs
|
||||
restart: "no"
|
||||
@@ -1,261 +0,0 @@
|
||||
import argparse
|
||||
import asyncio
|
||||
import json
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def parse_args():
|
||||
parser = argparse.ArgumentParser(description="Run a headless Douyin relogin worker.")
|
||||
parser.add_argument("--repo-root", required=True)
|
||||
parser.add_argument("--account-index", type=int, required=True)
|
||||
parser.add_argument("--state-file", required=True)
|
||||
parser.add_argument("--screenshot-path", required=True)
|
||||
parser.add_argument("--poll-interval", type=float, default=2.0)
|
||||
parser.add_argument("--timeout-seconds", type=int, default=900)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def now_iso():
|
||||
return datetime.now().isoformat(timespec="seconds")
|
||||
|
||||
|
||||
def write_state(path: Path, **payload):
|
||||
base = {"updated_at": now_iso(), **payload}
|
||||
path.write_text(json.dumps(base, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
|
||||
|
||||
async def capture_login_screenshot(page, screenshot_path: Path, prefer_verification=False):
|
||||
verification_selectors = [
|
||||
".pc-login-verification-modal",
|
||||
".semi-modal-content",
|
||||
".semi-modal",
|
||||
'div[role="dialog"]',
|
||||
]
|
||||
qr_selectors = [
|
||||
".login-img-code-wrapper",
|
||||
'div[class*="qrcode"]',
|
||||
"canvas",
|
||||
".login-mask",
|
||||
".login-guide-container",
|
||||
]
|
||||
selectors = verification_selectors + qr_selectors if prefer_verification else qr_selectors + verification_selectors
|
||||
for selector in selectors:
|
||||
locator = page.locator(selector).first
|
||||
try:
|
||||
if await locator.count() > 0 and await locator.is_visible():
|
||||
await locator.scroll_into_view_if_needed()
|
||||
await locator.screenshot(path=str(screenshot_path))
|
||||
return selector
|
||||
except Exception:
|
||||
continue
|
||||
await page.screenshot(path=str(screenshot_path), full_page=True)
|
||||
return "page"
|
||||
|
||||
|
||||
async def is_verification_step(page):
|
||||
modal_selectors = [
|
||||
".pc-login-verification-modal",
|
||||
".semi-modal-content",
|
||||
".semi-modal",
|
||||
'div[role="dialog"]',
|
||||
]
|
||||
for selector in modal_selectors:
|
||||
locator = page.locator(selector).first
|
||||
try:
|
||||
if await locator.count() > 0 and await locator.is_visible():
|
||||
return True
|
||||
except Exception:
|
||||
continue
|
||||
|
||||
verification_texts = [
|
||||
"身份验证",
|
||||
"以确保为本人操作",
|
||||
"短信验证码",
|
||||
"安全验证",
|
||||
]
|
||||
for text in verification_texts:
|
||||
locator = page.get_by_text(text, exact=False).first
|
||||
try:
|
||||
if await locator.count() > 0 and await locator.is_visible():
|
||||
return True
|
||||
except Exception:
|
||||
continue
|
||||
return False
|
||||
|
||||
|
||||
async def refresh_expired_qr_if_needed(page):
|
||||
refresh_texts = ["点击刷新", "刷新", "刷新二维码"]
|
||||
expired_texts = ["二维码失效", "二维码已失效"]
|
||||
|
||||
for text in refresh_texts:
|
||||
locator = page.get_by_text(text, exact=False).first
|
||||
try:
|
||||
if await locator.count() > 0 and await locator.is_visible():
|
||||
await locator.scroll_into_view_if_needed()
|
||||
await locator.click(force=True, timeout=10000)
|
||||
await asyncio.sleep(1.5)
|
||||
return True
|
||||
except Exception:
|
||||
continue
|
||||
|
||||
for text in expired_texts:
|
||||
locator = page.get_by_text(text, exact=False).first
|
||||
try:
|
||||
if await locator.count() > 0 and await locator.is_visible():
|
||||
await locator.scroll_into_view_if_needed()
|
||||
await locator.click(force=True, timeout=10000)
|
||||
await asyncio.sleep(1.5)
|
||||
return True
|
||||
except Exception:
|
||||
continue
|
||||
return False
|
||||
|
||||
|
||||
async def main():
|
||||
args = parse_args()
|
||||
repo_root = Path(args.repo_root).resolve()
|
||||
state_file = Path(args.state_file).resolve()
|
||||
screenshot_path = Path(args.screenshot_path).resolve()
|
||||
screenshot_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
state_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, str(repo_root))
|
||||
|
||||
from core.browser import get_browser
|
||||
from core.login import collect_login_result
|
||||
from utils.config import get_userData, save_userData
|
||||
|
||||
accounts = get_userData(force_reload=True)
|
||||
account = accounts[args.account_index]
|
||||
write_state(
|
||||
state_file,
|
||||
status="starting",
|
||||
message=f"Preparing relogin session for {account.get('username', 'unknown')}",
|
||||
account_index=args.account_index,
|
||||
username=account.get("username", ""),
|
||||
screenshot_path=str(screenshot_path),
|
||||
)
|
||||
|
||||
playwright = browser = context = page = None
|
||||
started_at = asyncio.get_running_loop().time()
|
||||
timeout_seconds = max(args.timeout_seconds, 60)
|
||||
|
||||
try:
|
||||
playwright, browser = await get_browser(GUI=False)
|
||||
context = await browser.new_context(
|
||||
viewport={"width": 1600, "height": 1200},
|
||||
device_scale_factor=2,
|
||||
)
|
||||
page = await context.new_page()
|
||||
|
||||
await page.goto("https://creator.douyin.com/", wait_until="domcontentloaded", timeout=60000)
|
||||
await asyncio.sleep(3)
|
||||
|
||||
selectors = [
|
||||
"canvas",
|
||||
".login-img-code-wrapper",
|
||||
'div[class*="qrcode"]',
|
||||
".login-mask",
|
||||
".login-guide-container",
|
||||
".pc-login-verification-modal",
|
||||
".semi-modal-content",
|
||||
".semi-modal",
|
||||
'div[role="dialog"]',
|
||||
]
|
||||
|
||||
while True:
|
||||
if asyncio.get_running_loop().time() - started_at > timeout_seconds:
|
||||
write_state(
|
||||
state_file,
|
||||
status="timeout",
|
||||
message="Login session timed out before authentication completed",
|
||||
account_index=args.account_index,
|
||||
username=account.get("username", ""),
|
||||
screenshot_path=str(screenshot_path),
|
||||
)
|
||||
return
|
||||
|
||||
await refresh_expired_qr_if_needed(page)
|
||||
|
||||
unique_id_locator = page.locator(
|
||||
'xpath=//*[contains(@id, "garfish_app_for_douyin_creator_pc_home")]'
|
||||
'/div/div[2]/div/div[2]/div[1]/div[2]/div[1]/div[3]'
|
||||
).first
|
||||
name_locator = page.locator(
|
||||
'xpath=//*[contains(@id, "garfish_app_for_douyin_creator_pc_home")]'
|
||||
'/div/div[2]/div/div[2]/div[1]/div[2]/div[1]/div[1]/div[1]'
|
||||
).first
|
||||
|
||||
if await unique_id_locator.count() > 0 and await name_locator.count() > 0:
|
||||
result = await collect_login_result(page, context, timeout_ms=5000)
|
||||
refreshed_accounts = get_userData(force_reload=True)
|
||||
refreshed_accounts[args.account_index]["unique_id"] = result["unique_id"]
|
||||
refreshed_accounts[args.account_index]["username"] = result["username"]
|
||||
refreshed_accounts[args.account_index]["cookies"] = result["cookies"]
|
||||
save_userData(refreshed_accounts)
|
||||
await page.screenshot(path=str(screenshot_path), full_page=True, timeout=15000)
|
||||
write_state(
|
||||
state_file,
|
||||
status="authenticated",
|
||||
message=f"Authenticated as {result['username']}",
|
||||
account_index=args.account_index,
|
||||
username=result["username"],
|
||||
unique_id=result["unique_id"],
|
||||
screenshot_path=str(screenshot_path),
|
||||
)
|
||||
return
|
||||
|
||||
verification = await is_verification_step(page)
|
||||
await capture_login_screenshot(page, screenshot_path, prefer_verification=verification)
|
||||
|
||||
write_state(
|
||||
state_file,
|
||||
status="waiting_verify" if verification else "awaiting_scan",
|
||||
message="Identity verification is required" if verification else "Scan the QR code with the Douyin app",
|
||||
account_index=args.account_index,
|
||||
username=account.get("username", ""),
|
||||
screenshot_path=str(screenshot_path),
|
||||
)
|
||||
await asyncio.sleep(args.poll_interval)
|
||||
except Exception as exc:
|
||||
write_state(
|
||||
state_file,
|
||||
status="error",
|
||||
message=str(exc),
|
||||
account_index=args.account_index,
|
||||
username=account.get("username", ""),
|
||||
screenshot_path=str(screenshot_path),
|
||||
)
|
||||
raise
|
||||
finally:
|
||||
if page:
|
||||
try:
|
||||
await page.close()
|
||||
except Exception:
|
||||
pass
|
||||
if context:
|
||||
try:
|
||||
await context.close()
|
||||
except Exception:
|
||||
pass
|
||||
if browser:
|
||||
try:
|
||||
await browser.close()
|
||||
except Exception:
|
||||
pass
|
||||
if playwright:
|
||||
try:
|
||||
await playwright.stop()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
try:
|
||||
asyncio.run(main())
|
||||
except AttributeError:
|
||||
loop = asyncio.get_event_loop()
|
||||
loop.run_until_complete(main())
|
||||
@@ -58,7 +58,7 @@ def read_crontab(path):
|
||||
if not path.exists():
|
||||
return []
|
||||
lines = []
|
||||
for raw_line in path.read_text(encoding="utf-8", errors="replace").splitlines():
|
||||
for raw_line in path.read_text(encoding="utf-8-sig", errors="replace").splitlines():
|
||||
line = raw_line.strip()
|
||||
if not line or line.startswith("#") or "=" in line.split(maxsplit=1)[0]:
|
||||
continue
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
import json
|
||||
import os
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
from core import tasks
|
||||
from utils import config as config_module
|
||||
|
||||
|
||||
class ConfigContractTests(unittest.TestCase):
|
||||
def test_default_config_matches_public_example(self):
|
||||
example_path = Path(config_module.__file__).resolve().parents[1] / "config.example.json"
|
||||
example = json.loads(example_path.read_text(encoding="utf-8"))
|
||||
self.assertEqual(example, config_module.DEFAULT_CONFIG)
|
||||
|
||||
def test_missing_runtime_config_is_created_from_safe_defaults(self):
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
path = Path(temp_dir) / "config.json"
|
||||
loaded = config_module._load_json_file(path, config_module.DEFAULT_CONFIG)
|
||||
self.assertEqual(config_module.DEFAULT_CONFIG, loaded)
|
||||
self.assertEqual(
|
||||
config_module.DEFAULT_CONFIG,
|
||||
json.loads(path.read_text(encoding="utf-8")),
|
||||
)
|
||||
self.assertFalse(loaded["useProtocolSender"])
|
||||
self.assertTrue(loaded["persistentBrowserProfiles"]["enabled"])
|
||||
|
||||
def test_profile_root_environment_override_wins(self):
|
||||
with patch.dict(os.environ, {"SPARKFLOW_BROWSER_PROFILE_ROOT": "/tmp/sparkflow-profiles"}):
|
||||
normalized = tasks._normalize_persistent_profile_config(config_module.DEFAULT_CONFIG)
|
||||
self.assertEqual("/tmp/sparkflow-profiles", normalized["root"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,103 @@
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||
SOURCE_ROOT = REPO_ROOT / "DouYinSparkFlow"
|
||||
|
||||
|
||||
class DeploymentContractTests(unittest.TestCase):
|
||||
def test_github_workflow_is_at_repository_root(self):
|
||||
workflow = REPO_ROOT / ".github" / "workflows" / "schedule.yml"
|
||||
self.assertTrue(workflow.is_file())
|
||||
self.assertFalse((SOURCE_ROOT / ".github" / "workflows" / "schedule.yml").exists())
|
||||
text = workflow.read_text(encoding="utf-8")
|
||||
self.assertIn("working-directory: DouYinSparkFlow", text)
|
||||
self.assertIn("SPARKFLOW_BROWSER_PROFILE_ROOT", text)
|
||||
self.assertIn("SPARKFLOW_MANUAL_RUN", text)
|
||||
self.assertIn("path: DouYinSparkFlow/logs/", text)
|
||||
|
||||
def test_github_actions_are_pinned_to_commit_shas(self):
|
||||
import re
|
||||
|
||||
workflow = (REPO_ROOT / ".github" / "workflows" / "schedule.yml").read_text(encoding="utf-8")
|
||||
uses_values = re.findall(r"^\s*-?\s*uses:\s*([^#\s]+)", workflow, flags=re.MULTILINE)
|
||||
self.assertTrue(uses_values)
|
||||
for value in uses_values:
|
||||
self.assertRegex(value, r"^[^@]+@[0-9a-f]{40}$")
|
||||
|
||||
def test_runtime_config_is_not_tracked_as_the_template(self):
|
||||
self.assertTrue((SOURCE_ROOT / "config.example.json").is_file())
|
||||
self.assertIn("config.json", (SOURCE_ROOT / ".gitignore").read_text(encoding="utf-8"))
|
||||
|
||||
def test_compose_runtime_mounts_follow_least_privilege(self):
|
||||
text = (REPO_ROOT / "docker-compose.yml").read_text(encoding="utf-8")
|
||||
web = text.split(" web:", 1)[1].split("\n login-desktop:", 1)[0]
|
||||
scheduler = text.split(" scheduler:", 1)[1].split("\n task:", 1)[0]
|
||||
task = text.split(" task:", 1)[1]
|
||||
|
||||
self.assertIn("/var/run/docker.sock:/var/run/docker.sock", web)
|
||||
self.assertIn(".:/opt/douyin-sparkflow", web)
|
||||
self.assertIn("/var/run/docker.sock:/var/run/docker.sock", scheduler)
|
||||
self.assertNotIn(".:/opt/douyin-sparkflow", scheduler)
|
||||
self.assertNotIn("/var/run/docker.sock:/var/run/docker.sock", task)
|
||||
self.assertNotIn(".:/opt/douyin-sparkflow", task)
|
||||
for service in (scheduler, task):
|
||||
self.assertIn(
|
||||
"./state/browser-profiles:/opt/douyin-sparkflow/state/browser-profiles",
|
||||
service,
|
||||
)
|
||||
|
||||
def test_cron_reader_accepts_windows_utf8_bom(self):
|
||||
import tempfile
|
||||
|
||||
from scripts.cron_runner import read_crontab
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
path = Path(temp_dir) / "root"
|
||||
path.write_text(
|
||||
"*/20 10-17 * * * cd /app && python main.py --doTask\n",
|
||||
encoding="utf-8-sig",
|
||||
)
|
||||
lines = read_crontab(path)
|
||||
|
||||
self.assertEqual(len(lines), 1)
|
||||
self.assertTrue(lines[0].startswith("*/20 "))
|
||||
|
||||
def test_sensitive_ports_bind_to_loopback_by_default(self):
|
||||
text = (REPO_ROOT / "docker-compose.yml").read_text(encoding="utf-8")
|
||||
self.assertIn("${PROXY_BIND_ADDRESS:-127.0.0.1}:${PROXY_HTTP_PORT:-7890}:7890", text)
|
||||
self.assertIn(
|
||||
"${LOGIN_DESKTOP_BIND_ADDRESS:-127.0.0.1}:${LOGIN_DESKTOP_WEB_PORT:-8788}:6080",
|
||||
text,
|
||||
)
|
||||
|
||||
def test_container_login_api_and_public_url_are_wired(self):
|
||||
text = (REPO_ROOT / "docker-compose.yml").read_text(encoding="utf-8")
|
||||
self.assertIn("SPARKFLOW_LOGIN_DESKTOP_API_URL: http://login-desktop:18090", text)
|
||||
self.assertIn("SPARKFLOW_LOGIN_DESKTOP_PUBLIC_URL", text)
|
||||
|
||||
def test_installers_preserve_runtime_config_and_do_not_require_bash_on_windows(self):
|
||||
server = (REPO_ROOT / "deploy" / "install-server.sh").read_text(encoding="utf-8")
|
||||
windows = (REPO_ROOT / "deploy" / "install-local.ps1").read_text(encoding="utf-8")
|
||||
self.assertIn("runtime_config_backup", server)
|
||||
self.assertIn("Restored runtime config.json", server)
|
||||
self.assertNotIn("bash ./refresh_proxy.sh", windows)
|
||||
self.assertIn("Initialize-ProxyConfig", windows)
|
||||
|
||||
def test_playwright_base_image_argument_is_used(self):
|
||||
dockerfile = (SOURCE_ROOT / "Dockerfile.server").read_text(encoding="utf-8")
|
||||
self.assertTrue(dockerfile.startswith("ARG PLAYWRIGHT_BASE_IMAGE="))
|
||||
self.assertIn("FROM ${PLAYWRIGHT_BASE_IMAGE}", dockerfile)
|
||||
self.assertIn("docker.io", dockerfile)
|
||||
self.assertIn("node --version", dockerfile)
|
||||
self.assertNotIn("github.com/docker/compose", dockerfile)
|
||||
|
||||
def test_legacy_unused_entrypoints_are_removed(self):
|
||||
self.assertFalse((SOURCE_ROOT / "webui" / "login_sessions.py").exists())
|
||||
self.assertFalse((SOURCE_ROOT / "relogin_worker.py").exists())
|
||||
self.assertFalse((SOURCE_ROOT / "docker-compose.example.yml").exists())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -104,6 +104,53 @@ class WebUiSafetyTests(unittest.TestCase):
|
||||
self.assertEqual(200, response.status_code, path)
|
||||
self.assertEqual("no-store", response.headers["cache-control"])
|
||||
|
||||
def test_login_desktop_urls_honor_container_environment(self):
|
||||
with (
|
||||
patch.dict(
|
||||
os.environ,
|
||||
{
|
||||
"SPARKFLOW_LOGIN_DESKTOP_API_URL": "http://login-desktop:18090",
|
||||
"SPARKFLOW_LOGIN_DESKTOP_PUBLIC_URL": "http://127.0.0.1:8788/vnc.html",
|
||||
},
|
||||
),
|
||||
patch.object(app_module, "get_app_settings", return_value={}),
|
||||
):
|
||||
self.assertEqual("http://login-desktop:18090", app_module.login_desktop_api_url())
|
||||
request = type("Request", (), {"url": type("Url", (), {"hostname": "example", "scheme": "http"})()})()
|
||||
self.assertEqual(
|
||||
"http://127.0.0.1:8788/vnc.html",
|
||||
app_module.login_desktop_public_url(request),
|
||||
)
|
||||
|
||||
def test_schedule_sync_writes_configured_window_to_shared_spool(self):
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
cron_path = Path(temp_dir) / "root"
|
||||
with (
|
||||
patch.object(ops, "HOST_CRONTAB_PATH", cron_path),
|
||||
patch.object(ops, "running_in_container", return_value=True),
|
||||
patch.object(ops, "read_crontab", return_value=""),
|
||||
patch.object(
|
||||
ops,
|
||||
"get_config",
|
||||
return_value={
|
||||
"dailySendWindow": {
|
||||
"enabled": True,
|
||||
"startHour": 10,
|
||||
"endHour": 18,
|
||||
"scheduleIntervalMinutes": 20,
|
||||
}
|
||||
},
|
||||
),
|
||||
):
|
||||
result = ops.sync_daily_schedule_from_config()
|
||||
|
||||
self.assertEqual(0, result.returncode)
|
||||
text = cron_path.read_text(encoding="utf-8")
|
||||
self.assertIn("*/20 10-17 * * *", text)
|
||||
self.assertIn("0 18 * * *", text)
|
||||
self.assertIn("20 18 * * *", text)
|
||||
self.assertIn("docker exec", text)
|
||||
|
||||
def test_overview_api_requires_authentication_and_disables_cache(self):
|
||||
client = TestClient(app_module.app)
|
||||
response = client.get("/api/ops/overview")
|
||||
|
||||
@@ -19,39 +19,55 @@ APPSETTINGSFILE = "webui_settings.json"
|
||||
|
||||
DEFAULT_CONFIG = {
|
||||
"multiTask": True,
|
||||
"taskCount": 5,
|
||||
"taskCount": 1,
|
||||
"proxyAddress": "",
|
||||
"messageTemplate": "【AI续火花】",
|
||||
"messageTemplate": "🤩今日火花+1\r\n",
|
||||
"saveDebugArtifacts": False,
|
||||
"useProtocolSender": True,
|
||||
"useProtocolSender": False,
|
||||
"protocolDryRun": False,
|
||||
"browserSenderAccounts": [],
|
||||
"persistentBrowserProfiles": {
|
||||
"sendStrategy": {
|
||||
"shuffleTargets": True,
|
||||
"accountStartDelaySecondsMin": 15,
|
||||
"accountStartDelaySecondsMax": 60,
|
||||
"messageIntervalSecondsMin": 25,
|
||||
"messageIntervalSecondsMax": 70,
|
||||
"messageVariants": [
|
||||
"🤩今日火花+1",
|
||||
"今天来补个火花",
|
||||
"给你续一下今天的火花",
|
||||
"路过给你加个小火花"
|
||||
]
|
||||
},
|
||||
"dailySendWindow": {
|
||||
"enabled": True,
|
||||
"startHour": 10,
|
||||
"endHour": 18,
|
||||
"scheduleIntervalMinutes": 20
|
||||
},
|
||||
"hitokotoTypes": [
|
||||
"文学",
|
||||
"影视",
|
||||
"诗词",
|
||||
"哲学"
|
||||
],
|
||||
"happyNewYear": {
|
||||
"enabled": False,
|
||||
"messageTemplate": "\r\n"
|
||||
},
|
||||
"friendListScan": {
|
||||
"maxScanSeconds": 300,
|
||||
"idleScanSeconds": 120,
|
||||
"scrollStepPx": 400,
|
||||
"scrollDelaySeconds": 0.8
|
||||
},
|
||||
"persistentBrowserProfiles": {
|
||||
"enabled": True,
|
||||
"root": "/opt/douyin-sparkflow/state/browser-profiles",
|
||||
"seedCookiesWhenEmpty": True,
|
||||
"syncStoredCookiesBeforeRun": True,
|
||||
"refreshStoredCookiesAfterLogin": True,
|
||||
},
|
||||
"sendStrategy": {
|
||||
"shuffleTargets": True,
|
||||
"accountStartDelaySecondsMin": 0,
|
||||
"accountStartDelaySecondsMax": 20,
|
||||
"messageIntervalSecondsMin": 18,
|
||||
"messageIntervalSecondsMax": 45,
|
||||
"messageVariants": [],
|
||||
},
|
||||
"dailySendWindow": {
|
||||
"enabled": False,
|
||||
"startHour": 10,
|
||||
"endHour": 18,
|
||||
"scheduleIntervalMinutes": 10,
|
||||
},
|
||||
"hitokotoTypes": ["文学", "影视", "诗词", "哲学"],
|
||||
"happyNewYear": {
|
||||
"enabled": False,
|
||||
"messageTemplate": "【[data]|[data_lunar]】\n[API]",
|
||||
},
|
||||
"refreshStoredCookiesAfterLogin": True
|
||||
}
|
||||
}
|
||||
|
||||
DEFAULT_APP_SETTINGS = {
|
||||
|
||||
@@ -44,5 +44,5 @@ def request_hitokoto():
|
||||
if theFromWho is None or theFromWho.strip() == "":
|
||||
theFromWho = "未知作者"
|
||||
return f"{data['hitokoto']} —— {theFrom} ({theFromWho})"
|
||||
except Exception as e:
|
||||
except Exception:
|
||||
return "[error] 无法获取一言内容"
|
||||
|
||||
@@ -1,9 +1,11 @@
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from pathlib import Path
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
import uvicorn
|
||||
from fastapi import FastAPI, Request
|
||||
@@ -12,8 +14,6 @@ from fastapi.staticfiles import StaticFiles
|
||||
from fastapi.templating import Jinja2Templates
|
||||
from starlette.middleware.sessions import SessionMiddleware
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
from core.friends import fetch_account_friends
|
||||
from core.send_state import history_entry_is_strong_confirmed_today, parse_sent_at
|
||||
from core.tasks import run_browser_tasks, task_run_lock
|
||||
@@ -50,9 +50,12 @@ from webui.ops import (
|
||||
run_task_now,
|
||||
run_unsent_retry_now,
|
||||
task_run_lock_status,
|
||||
sync_daily_schedule_from_config,
|
||||
update_daily_schedule,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
BASE_DIR = Path(__file__).resolve().parent
|
||||
TEMPLATES_DIR = BASE_DIR / "templates"
|
||||
@@ -190,12 +193,17 @@ def mark_target_unconfirmed(account, target_name, *, reason="manual_reset_possib
|
||||
|
||||
def login_desktop_api_url():
|
||||
settings = get_app_settings(force_reload=True)
|
||||
return str(settings.get("login_desktop_api_url") or "http://127.0.0.1:18090").rstrip("/")
|
||||
configured = os.getenv("SPARKFLOW_LOGIN_DESKTOP_API_URL") or settings.get("login_desktop_api_url")
|
||||
return str(configured or "http://127.0.0.1:18090").rstrip("/")
|
||||
|
||||
|
||||
def login_desktop_public_url(request: Request) -> str:
|
||||
settings = get_app_settings(force_reload=True)
|
||||
configured_url = str(settings.get("login_desktop_public_url") or "").strip()
|
||||
configured_url = str(
|
||||
os.getenv("SPARKFLOW_LOGIN_DESKTOP_PUBLIC_URL")
|
||||
or settings.get("login_desktop_public_url")
|
||||
or ""
|
||||
).strip()
|
||||
if configured_url:
|
||||
return configured_url
|
||||
|
||||
@@ -274,13 +282,27 @@ def public_app_settings():
|
||||
|
||||
def create_app():
|
||||
settings = get_app_settings()
|
||||
app = FastAPI(title="DouYin Spark Flow Admin")
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(_app):
|
||||
result = sync_daily_schedule_from_config()
|
||||
if result.returncode != 0:
|
||||
logger.warning("Failed to synchronize the configured daily schedule: %s", result.stderr)
|
||||
yield
|
||||
|
||||
secure_cookie = str(os.getenv("SPARKFLOW_SESSION_COOKIE_SECURE") or "").strip().lower() in {
|
||||
"1",
|
||||
"true",
|
||||
"yes",
|
||||
"on",
|
||||
}
|
||||
app = FastAPI(title="DouYin Spark Flow Admin", lifespan=lifespan)
|
||||
app.add_middleware(
|
||||
SessionMiddleware,
|
||||
secret_key=settings["session_secret"],
|
||||
max_age=settings["session_max_age_seconds"],
|
||||
same_site="lax",
|
||||
https_only=False,
|
||||
https_only=secure_cookie,
|
||||
)
|
||||
app.mount("/static", StaticFiles(directory=str(STATIC_DIR)), name="static")
|
||||
DEBUG_ARTIFACTS_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -13,7 +13,7 @@ from pathlib import Path
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
from core.send_state import history_entry_is_strong_confirmed_today, parse_sent_at
|
||||
from utils.config import get_app_settings, get_config, get_userData, normalize_unique_id, repo_root, save_config
|
||||
from utils.config import get_app_settings, get_config, get_userData, repo_root, save_config
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -423,6 +423,8 @@ def refresh_proxy():
|
||||
|
||||
def restart_proxy():
|
||||
try:
|
||||
if running_in_container():
|
||||
return run_command(["docker", "restart", "mihomo"], timeout=120)
|
||||
return run_command(compose_command("restart", "proxy"), timeout=120)
|
||||
except Exception as exc:
|
||||
logger.error("restart_proxy failed: %s", exc)
|
||||
@@ -566,6 +568,43 @@ def update_daily_schedule(time_string):
|
||||
return _empty_result(stderr=str(exc))
|
||||
|
||||
|
||||
def sync_daily_schedule_from_config():
|
||||
config = get_config(force_reload=True)
|
||||
window = dict(config.get("dailySendWindow") or {})
|
||||
if not window.get("enabled"):
|
||||
return subprocess.CompletedProcess(
|
||||
args=["sync-daily-schedule"],
|
||||
returncode=0,
|
||||
stdout="schedule disabled; existing crontab left unchanged",
|
||||
stderr="",
|
||||
)
|
||||
|
||||
try:
|
||||
time_string = _format_window_schedule(window)
|
||||
current = read_crontab()
|
||||
updated = replace_douyin_cron_schedule(current, time_string)
|
||||
if updated == current:
|
||||
return subprocess.CompletedProcess(
|
||||
args=["sync-daily-schedule"], returncode=0, stdout="already synchronized", stderr=""
|
||||
)
|
||||
if running_in_container() and HOST_CRONTAB_PATH.parent.exists():
|
||||
HOST_CRONTAB_PATH.write_text(updated, encoding="utf-8")
|
||||
return subprocess.CompletedProcess(
|
||||
args=["sync-daily-schedule"], returncode=0, stdout="host spool updated", stderr=""
|
||||
)
|
||||
return subprocess.run(
|
||||
["crontab", "-"],
|
||||
input=updated,
|
||||
text=True,
|
||||
capture_output=True,
|
||||
check=False,
|
||||
timeout=10,
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.error("sync_daily_schedule_from_config failed: %s", exc)
|
||||
return _empty_result(stderr=str(exc))
|
||||
|
||||
|
||||
def current_daily_schedule():
|
||||
config = get_config(force_reload=True)
|
||||
window = dict(config.get("dailySendWindow") or {})
|
||||
|
||||
Reference in New Issue
Block a user