Compare commits

...

4 Commits

Author SHA1 Message Date
Wu Qing
81c9c042d6 功能: 修复并实现多节点集群部署 (#38)
基础修复:
- 新增节点离线检测:每 15s 扫描,超 45s 未心跳的远程节点自动置离线
- 节点删除前检查关联任务,避免孤立备份任务
- BackupTaskRepository 新增 CountByNodeID/ListByNodeID

Master 端 Agent 协议:
- 新增 AgentCommand 模型与命令队列仓储(pending/dispatched/succeeded/failed/timeout)
- 新增 AgentService:任务下发、命令轮询、结果回收、超时扫描
- 新增专用 Agent HTTP API(X-Agent-Token 认证):
  /api/agent/heartbeat
  /api/agent/commands/poll
  /api/agent/commands/:id/result
  /api/agent/tasks/:id
  /api/agent/records/:id
- BackupExecutionService 支持 node 路由:task.NodeID 指向远程节点时自动入队派发

Agent CLI(backupx agent 子命令):
- 配置:YAML 文件 / 环境变量 / CLI 参数,优先级 CLI > 文件 > 环境
- 心跳循环 + 命令轮询循环 + 优雅退出
- 本地复用 BackupRunner 与 storage registry 执行备份并直接上传
- 支持 run_task 和 list_dir 两种命令

远程目录浏览:
- NodeService 支持通过 Agent RPC 列出远程节点目录(15s 超时)

前端:
- NodesPage 添加节点后展示 Agent 启动命令和环境变量配置

文档:
- README 中英文重写"多节点集群"章节,含架构图、步骤、限制、CLI 参考
2026-04-17 12:29:08 +08:00
Wu Qing
3e90e0f8a8 功能: 新增 SAP HANA 完整备份支持与 Backint 协议代理 (#37)
* chore: ignore web/dist directory in git repository

* 功能: 新增 SAP HANA 完整备份支持与 Backint 协议代理

- 修复 service 层校验 bug,使 SAP HANA 类型可正常创建
- 增强 hdbsql Runner:支持完整/增量/差异/日志备份、并行通道、失败重试
- 新增 Backint 协议代理(backupx backint 子命令),HANA 原生接口直连 BackupX 存储后端
- 新增本地 SQLite 目录维护 EBID↔对象键映射
- 前端新增 SAP HANA 扩展字段表单(备份类型/级别/通道数/重试次数/实例编号)
- README 中英文补充 SAP HANA 两种模式的使用说明
2026-04-16 23:43:46 +08:00
Wu Qing
827a5a2181 文档: 更新 README 中英文文档 (#35)
- 存储后端描述更新为 70+ rclone 集成
- API 参考补充新增端点(节点编辑、Rclone 后端列表、版本检查、Agent 心跳)
- 技术栈补充 rclone
- 多节点集群章节补充 IP 检测、节点编辑等新功能描述
- 存储目标添加指南补充 Rclone 类型配置项分层说明
- 任务删除行为说明(清理远端文件、保留记录)
- 版本升级指引从一键更新改为手动 docker compose pull
- 发版示例更新为 v1.4.3
2026-04-05 11:33:56 +08:00
Wu Qing
970eb154e1 优化: 多模块功能修复与体验改进 (#34)
1. 保留策略清理后自动删除空文件夹(新增 StorageDirCleaner 接口)
2. 备份任务删除时清理远端文件但保留备份记录
3. 节点管理修复:本机 IP/版本检测、Heartbeat OS/Arch 修正、新增编辑功能
4. 审计日志规范化:统一格式、丰富详情、节点操作增加审计记录
5. 系统设置移除一键更新操作,仅保留版本检查
6. Rclone 配置项分层展示(必填 + 高级可选折叠)
7. DirectoryPicker 目录选择器样式优化
2026-04-05 11:23:46 +08:00
61 changed files with 4967 additions and 283 deletions

4
.gitignore vendored
View File

@@ -1 +1,3 @@
web/node_modules/
web/node_modules/
web/dist/
server/bin/

184
README.md
View File

@@ -34,11 +34,12 @@
| 能力 | 说明 |
|------|------|
| **备份类型** | 文件/目录多源路径、MySQL、PostgreSQL、SQLite、SAP HANA |
| **存储后端** | 阿里云 OSS、腾讯云 COS、七牛云、S3 兼容(AWS/MinIO/R2)、Google Drive、WebDAV、FTP/FTPS、本地磁盘 |
| **自动调度** | Cron 定时 + 可视化编辑器 + 自动保留策略(按天数/份数清理) |
| **多节点** | Master-Agent 集群,统一管理多台服务器的备份 |
| **安全** | JWT + bcrypt + AES-256-GCM 加密配置 + 可选备份文件加密 + 审计日志 |
| **备份类型** | 文件/目录多源路径、MySQL、PostgreSQL、SQLite、SAP HANA(完整/增量/差异/日志备份 + 并行通道 + 失败重试) |
| **SAP HANA Backint 代理** | 内置 SAP HANA Backint 协议代理HANA 原生备份接口可直接把数据路由到 BackupX 支持的任意存储后端 |
| **70+ 存储后端** | 内置阿里云 OSS / 腾讯云 COS / 七牛云 / S3 / Google Drive / WebDAV / FTP + 通过 rclone 集成 SFTP、Azure Blob、Dropbox、OneDrive 等 70+ 后端 |
| **自动调度** | Cron 定时 + 可视化编辑器 + 自动保留策略(按天数/份数清理,自动回收空目录) |
| **多节点** | Master-Agent 集群,统一管理多台服务器的备份,支持远程目录浏览与节点编辑 |
| **安全** | JWT + bcrypt + AES-256-GCM 加密配置 + 可选备份文件加密 + 完整审计日志 |
| **通知** | 邮件 / Webhook / Telegram备份成功或失败时自动推送 |
| **部署** | 单二进制 + 内嵌 SQLiteDocker 一键启动,零外部依赖 |
@@ -120,8 +121,9 @@ make docker-cn # 或用国内镜像构建 Dockergoproxy.cn / npmmir
| WebDAV | 服务器地址 + 用户名/密码 |
| FTP | 主机 + 端口 + 用户名/密码 |
| 本地磁盘 | 目标目录路径 |
| SFTP / Azure / Dropbox / OneDrive 等 | 选择对应类型后填写必填项,高级配置可折叠展开 |
> 国内云厂商只需填 Region 和 AccessKey系统自动组装 Endpoint。
> 国内云厂商只需填 Region 和 AccessKey系统自动组装 Endpoint。Rclone 类型的配置项按必填/可选分层展示,高级选项默认折叠。
添加后点击 **测试连接** 确认配置正确。
@@ -131,10 +133,12 @@ make docker-cn # 或用国内镜像构建 Dockergoproxy.cn / npmmir
1. **基础信息** — 任务名称、备份类型、Cron 表达式(留空则仅手动执行)
2. **源配置** — 文件备份选择源路径(支持多个)、数据库备份填写连接信息
3. **存储与策略** — 选择存储目标、压缩策略、保留天数、是否加密
3. **存储与策略** — 选择存储目标(支持多个)、压缩策略、保留天数、是否加密
保存后可以点击 **立即执行** 测试,在 **备份记录** 页面实时查看执行日志。
> 删除备份任务时会自动清理远端存储上的备份文件,但保留备份记录以供审计追溯。
### 5. 配置通知(可选)
进入 **通知配置** 页面支持邮件、Webhook、Telegram 三种方式,可分别配置成功/失败时是否推送。
@@ -167,6 +171,8 @@ environment:
- BACKUPX_BACKUP_MAX_CONCURRENT=4
```
版本更新:在 **系统设置** 页面点击「检查更新」查看是否有新版本,然后手动执行 `docker compose pull && docker compose up -d` 完成升级。
### 裸机部署
```bash
@@ -235,15 +241,159 @@ docker exec -it backupx /app/bin/backupx reset-password --username admin --passw
---
## SAP HANA 支持
BackupX 提供两种 SAP HANA 备份模式,按需选用:
### 模式一hdbsql RunnerWeb 控制台托管)
通过 Web 控制台创建 SAP HANA 备份任务,后端调用 `hdbsql` 执行备份,适合 BackupX 调度的周期性作业。
**源配置步骤支持:**
| 字段 | 可选值 | 说明 |
|------|--------|------|
| 备份类型 | `data` / `log` | 数据备份或日志备份 |
| 备份级别 | `full` / `incremental` / `differential` | 日志备份时自动禁用 |
| 并行通道数 | `1 ~ 32` | `BACKUP DATA USING FILE ('c1','c2',...)` 多路径并发 |
| 失败重试次数 | `1 ~ 10` | 指数退避5s × 尝试次数²) |
| 实例编号 | 可选 | 从端口推断或手动指定 |
### 模式二Backint 协议代理HANA 原生接口)
BackupX 内置 Backint AgentSAP HANA 通过原生 `BACKUP DATA USING BACKINT` 语法调用,数据自动路由到 BackupX 存储目标S3 / OSS / COS / WebDAV / 70+ 后端)。
**1. 准备参数文件** `/opt/backupx/backint_params.ini`
```ini
#STORAGE_TYPE = s3
#STORAGE_CONFIG_JSON = /opt/backupx/storage.json
#PARALLEL_FACTOR = 4
#COMPRESS = true
#KEY_PREFIX = hana-backup
#CATALOG_DB = /opt/backupx/backint_catalog.db
#LOG_FILE = /var/log/backupx/backint.log
```
**2. 准备存储配置** `/opt/backupx/storage.json`(与 BackupX 存储目标配置一致):
```json
{
"endpoint": "https://s3.amazonaws.com",
"region": "us-east-1",
"bucket": "hana-prod",
"accessKeyId": "AKIA...",
"secretAccessKey": "..."
}
```
**3. 创建 hdbbackint 软链接:**
```bash
ln -s /opt/backupx/backupx /usr/sap/<SID>/SYS/global/hdb/opt/hdbbackint
```
**4. 在 HANA `global.ini` 中启用:**
```ini
[backup]
data_backup_using_backint = true
catalog_backup_using_backint = true
log_backup_using_backint = true
data_backup_parameter_file = /opt/backupx/backint_params.ini
log_backup_parameter_file = /opt/backupx/backint_params.ini
```
**5. CLI 手动调用(用于排查):**
```bash
backupx backint -f backup -i input.txt -o output.txt -p backint_params.ini
backupx backint -f restore -i input.txt -o output.txt -p backint_params.ini
backupx backint -f inquire -i input.txt -o output.txt -p backint_params.ini
backupx backint -f delete -i input.txt -o output.txt -p backint_params.ini
```
Backint Agent 使用本地 SQLite 维护 `EBID ↔ 对象键` 目录,所有操作遵循 SAP HANA Backint 协议(`#PIPE` / `#SAVED` / `#RESTORED` / `#BACKUP` / `#NOTFOUND` / `#DELETED` / `#ERROR`)。
---
## 多节点集群
BackupX 支持 Master-Agent 模式管理多台服务器:
BackupX 支持 Master-Agent 模式管理多台服务器:备份任务可以指定在哪个节点执行Agent 在本地完成备份并直接上传到存储后端。
1. Web 控制台 → **节点管理****添加节点**,系统生成 Token
2. 在远程服务器部署 Agent 并使用 Token 连接 Master
3. 创建备份任务时选择对应节点Master 自动下发任务
### 架构概览
创建文件备份任务时,可通过可视化目录浏览器远程选择 Agent 节点上的目录,无需手动输入路径。
```
[Web 控制台] ←── JWT ──→ [Master (backupx)]
↑ ↓
│ │ HTTP 长轮询 (token 认证)
│ ↓
[Agent (backupx agent)] ← 运行在远程服务器
[70+ 存储后端]
```
- **通信协议**HTTP 长轮询Agent 主动发起所有连接,无需 Master 反向访问
- **心跳**Agent 每 15s 上报一次Master 每 15s 扫描,超过 45s 未心跳判为离线
- **任务下发**Master 通过数据库命令队列派发 `run_task`Agent 轮询拉取
- **执行**Agent 本地复用 BackupRunnerfile / mysql / postgresql / sqlite / saphana并直接上传到存储
- **安全**:每个节点独立 TokenAgent 不持有 Master 的 JWT 密钥和加密密钥
### 使用步骤
**1. 在 Master 创建节点并获取 Token**
Web 控制台 → **节点管理****添加节点**,填写节点名称并保存。界面会显示一个 64 字节十六进制令牌(仅显示一次,请妥善保存)。
**2. 在远程服务器部署 Agent**
把 BackupX 二进制上传到目标服务器(与 Master 同一个文件),然后用以下任一方式启动:
```bash
# 方式 ACLI 参数
backupx agent --master http://master.example.com:8340 --token <token>
# 方式 B配置文件
cat > /etc/backupx/agent.yaml <<EOF
master: http://master.example.com:8340
token: <token>
heartbeatInterval: 15s
pollInterval: 5s
tempDir: /var/lib/backupx-agent
EOF
backupx agent --config /etc/backupx/agent.yaml
# 方式 C环境变量适合 Docker / systemd
BACKUPX_AGENT_MASTER=http://master.example.com:8340 \
BACKUPX_AGENT_TOKEN=<token> \
backupx agent
```
启动成功后Master 的节点列表会把该节点标记为**在线**。
**3. 创建路由到该节点的备份任务**
**备份任务** 页面新建任务时选择对应节点。任务被触发后:
- 本机节点或未指定节点(`nodeId=0`):由 Master 进程本地执行
- 远程节点Master 写入命令队列 → Agent 轮询拉取 → 本地执行并上传 → 上报记录
### 限制说明
- **不支持加密备份**Agent 不持有 Master 的 AES-256 加密密钥,启用 `encrypt: true` 的任务会路由到 Agent 时失败
- **目录浏览超时**:远程目录浏览通过命令队列做同步 RPC默认 15s 超时,网络慢时可能失败
- **命令超时**Agent 领取但未完成的命令超过 10min 会被标记为超时
### CLI 参考
```bash
backupx agent --help
-master string Master URL
-token string Agent 认证令牌
-config string YAML 配置文件路径(优先级高于环境变量)
-temp-dir string 本地临时目录(默认 /tmp/backupx-agent
-insecure-tls 跳过 TLS 证书校验(仅测试用)
```
---
@@ -268,7 +418,7 @@ make docker-cn # 国内 Docker 构建(镜像加速)
### 发版
```bash
git tag v1.2.3 && git push --tags
git tag v1.4.3 && git push --tags
# GitHub Actions 自动:编译双架构二进制 → 发布 GitHub Release → 推送 Docker Hub 镜像
```
@@ -295,12 +445,16 @@ git tag v1.2.3 && git push --tags
| | `POST /backup/records/:id/restore` | 恢复 |
| **存储目标** | `GET\|POST /storage-targets` | 列表 / 添加 |
| | `POST /storage-targets/test` | 测试连接 |
| | `GET /storage-targets/rclone/backends` | Rclone 后端列表 |
| **节点** | `GET\|POST /nodes` | 列表 / 添加 |
| | `PUT /nodes/:id` | 编辑节点 |
| | `GET /nodes/:id/fs/list` | 目录浏览 |
| | `POST /agent/heartbeat` | Agent 心跳Token 认证) |
| **通知** | `GET\|POST /notifications` | 列表 / 添加 |
| **仪表盘** | `GET /dashboard/stats` | 概览统计 |
| **审计日志** | `GET /audit-logs` | 操作审计 |
| **系统** | `GET /system/info` | 系统信息 |
| | `GET /system/update-check` | 检查版本更新 |
---
@@ -308,9 +462,9 @@ git tag v1.2.3 && git push --tags
| 组件 | 技术 |
|------|------|
| **后端** | Go · Gin · GORM · SQLite · robfig/cron |
| **后端** | Go · Gin · GORM · SQLite · robfig/cron · rclone |
| **前端** | React 18 · TypeScript · ArcoDesign · Vite · Zustand · ECharts |
| **存储** | AWS SDK v2 · Google Drive API v3 · gowebdav · jlaffaye/ftp |
| **存储** | rclone70+ 后端)· AWS SDK v2 · Google Drive API v3 |
| **安全** | JWT · bcrypt · AES-256-GCM |
## Contributing

View File

@@ -34,11 +34,12 @@
| Capability | Details |
|-----------|---------|
| **Backup Types** | Files/Directories (multi-source), MySQL, PostgreSQL, SQLite, SAP HANA |
| **Storage Backends** | Alibaba Cloud OSS, Tencent COS, Qiniu Kodo, S3-compatible (AWS/MinIO/R2), Google Drive, WebDAV, FTP/FTPS, Local Disk |
| **Scheduling** | Cron-based scheduling + visual editor + auto-retention policy (by days/count) |
| **Multi-Node** | Master-Agent cluster for managing backups across multiple servers |
| **Security** | JWT + bcrypt + AES-256-GCM encrypted config + optional backup encryption + audit logs |
| **Backup Types** | Files/Directories (multi-source), MySQL, PostgreSQL, SQLite, SAP HANA (full / incremental / differential / log backups + parallel channels + retry) |
| **SAP HANA Backint Agent** | Built-in SAP HANA Backint protocol agent — HANA's native backup interface can route data directly to any storage backend supported by BackupX |
| **70+ Storage Backends** | Built-in Alibaba OSS / Tencent COS / Qiniu / S3 / Google Drive / WebDAV / FTP + 70+ backends via rclone (SFTP, Azure Blob, Dropbox, OneDrive, etc.) |
| **Scheduling** | Cron-based + visual editor + auto-retention policy (by days/count, auto empty directory cleanup) |
| **Multi-Node** | Master-Agent cluster for managing backups across multiple servers with remote directory browsing and node editing |
| **Security** | JWT + bcrypt + AES-256-GCM encrypted config + optional backup encryption + comprehensive audit logs |
| **Notifications** | Email / Webhook / Telegram — push on success or failure |
| **Deployment** | Single binary + embedded SQLite, Docker one-click, zero external dependencies |
@@ -120,6 +121,9 @@ Go to **Storage Targets** → **Add**, choose a storage type and enter credentia
| WebDAV | Server URL + Username/Password |
| FTP | Host + Port + Username/Password |
| Local Disk | Target directory path |
| SFTP / Azure / Dropbox / OneDrive etc. | Select the type, fill in required fields; advanced options are collapsible |
> For Chinese cloud providers, just enter Region and AccessKey — the system auto-assembles the Endpoint. Rclone-type configs separate required fields from optional advanced options (collapsed by default).
Click **Test Connection** to verify.
@@ -129,10 +133,12 @@ Go to **Backup Tasks** → **Create**, complete 3 steps:
1. **Basic Info** — Task name, backup type, Cron expression (leave empty for manual-only)
2. **Source Config** — File backup: select source paths (supports multiple); Database: enter connection info
3. **Storage & Policy** — Select storage target(s), compression, retention days, encryption toggle
3. **Storage & Policy** — Select storage target(s) (supports multiple), compression, retention days, encryption toggle
Save, then click **Run Now** to test. View real-time logs in **Backup Records**.
> Deleting a backup task automatically cleans up remote storage files while preserving backup records for audit purposes.
### 5. Set Up Notifications (Optional)
Go to **Notifications** to configure Email, Webhook, or Telegram alerts for backup success/failure.
@@ -165,6 +171,8 @@ environment:
- BACKUPX_BACKUP_MAX_CONCURRENT=4
```
To upgrade: go to **System Settings**, click "Check for Updates" to see if a new version is available, then run `docker compose pull && docker compose up -d`.
### Bare Metal
```bash
@@ -231,15 +239,159 @@ docker exec -it backupx /app/bin/backupx reset-password --username admin --passw
---
## SAP HANA Support
BackupX offers two SAP HANA backup modes — pick whichever fits:
### Mode 1: hdbsql Runner (Web-console managed)
Create a SAP HANA backup task in the Web console. The backend runs `hdbsql` to perform backups, suitable for BackupX-scheduled recurring jobs.
**Source configuration supports:**
| Field | Options | Description |
|-------|---------|-------------|
| Backup type | `data` / `log` | Data or log backup |
| Backup level | `full` / `incremental` / `differential` | Auto-disabled for log backups |
| Parallel channels | `1 ~ 32` | `BACKUP DATA USING FILE ('c1','c2',...)` parallel paths |
| Retry count | `1 ~ 10` | Exponential backoff (5s × attempt²) |
| Instance number | Optional | Inferred from port or manually specified |
### Mode 2: Backint Protocol Agent (HANA native)
BackupX ships a built-in Backint Agent. SAP HANA calls it via native `BACKUP DATA USING BACKINT` syntax, and data is routed automatically to BackupX storage targets (S3 / OSS / COS / WebDAV / 70+ backends).
**1. Prepare parameter file** `/opt/backupx/backint_params.ini`:
```ini
#STORAGE_TYPE = s3
#STORAGE_CONFIG_JSON = /opt/backupx/storage.json
#PARALLEL_FACTOR = 4
#COMPRESS = true
#KEY_PREFIX = hana-backup
#CATALOG_DB = /opt/backupx/backint_catalog.db
#LOG_FILE = /var/log/backupx/backint.log
```
**2. Prepare storage config** `/opt/backupx/storage.json` (same schema as BackupX storage targets):
```json
{
"endpoint": "https://s3.amazonaws.com",
"region": "us-east-1",
"bucket": "hana-prod",
"accessKeyId": "AKIA...",
"secretAccessKey": "..."
}
```
**3. Create the hdbbackint symlink:**
```bash
ln -s /opt/backupx/backupx /usr/sap/<SID>/SYS/global/hdb/opt/hdbbackint
```
**4. Enable in HANA `global.ini`:**
```ini
[backup]
data_backup_using_backint = true
catalog_backup_using_backint = true
log_backup_using_backint = true
data_backup_parameter_file = /opt/backupx/backint_params.ini
log_backup_parameter_file = /opt/backupx/backint_params.ini
```
**5. Manual CLI invocation (for troubleshooting):**
```bash
backupx backint -f backup -i input.txt -o output.txt -p backint_params.ini
backupx backint -f restore -i input.txt -o output.txt -p backint_params.ini
backupx backint -f inquire -i input.txt -o output.txt -p backint_params.ini
backupx backint -f delete -i input.txt -o output.txt -p backint_params.ini
```
The Backint Agent maintains an `EBID ↔ object-key` catalog in a local SQLite DB. All operations follow the SAP HANA Backint protocol (`#PIPE` / `#SAVED` / `#RESTORED` / `#BACKUP` / `#NOTFOUND` / `#DELETED` / `#ERROR`).
---
## Multi-Node Cluster
BackupX supports Master-Agent mode for managing multiple servers:
BackupX supports Master-Agent mode for managing multiple servers. Backup tasks can be routed to specific nodes — the Agent runs the backup locally and uploads straight to storage backends.
1. Web Console → **Node Management****Add Node** — system generates a Token
2. Deploy Agent on remote server, connect using the Token
3. Create backup tasks and assign to specific nodes — Master dispatches automatically
### Architecture
The visual directory browser lets you pick directories on remote Agent nodes — no manual path typing.
```
[Web Console] ←── JWT ──→ [Master (backupx)]
↑ ↓
│ │ HTTP long-poll (token auth)
│ ↓
[Agent (backupx agent)] ← runs on remote host
[70+ Storage Backends]
```
- **Protocol**: HTTP long-polling; the Agent initiates all connections — Master never needs reverse access
- **Heartbeat**: Agent reports every 15s; Master marks nodes offline after 45s of silence
- **Dispatch**: Master persists `run_task` commands to a queue; Agent polls and claims them
- **Execution**: Agent reuses the same BackupRunner (file / mysql / postgresql / sqlite / saphana) and uploads directly to storage
- **Security**: Each node gets its own token; the Agent never holds the Master's JWT secret or encryption key
### Walkthrough
**1. Create a node on Master and copy the token**
Web Console → **Node Management****Add Node**. The dialog shows a 64-byte hex token once — keep it safe.
**2. Deploy the Agent on a remote host**
Upload the BackupX binary (same file as Master) to the target host, then start the Agent:
```bash
# Option A: CLI flags
backupx agent --master http://master.example.com:8340 --token <token>
# Option B: config file
cat > /etc/backupx/agent.yaml <<EOF
master: http://master.example.com:8340
token: <token>
heartbeatInterval: 15s
pollInterval: 5s
tempDir: /var/lib/backupx-agent
EOF
backupx agent --config /etc/backupx/agent.yaml
# Option C: environment variables (Docker / systemd-friendly)
BACKUPX_AGENT_MASTER=http://master.example.com:8340 \
BACKUPX_AGENT_TOKEN=<token> \
backupx agent
```
Once connected, the node appears as **online** in the list.
**3. Create a task routed to that node**
In the **Backup Tasks** page, pick the target node when creating the task. When triggered:
- Local / unassigned (`nodeId=0`) tasks run in-process on Master
- Remote-node tasks are enqueued → Agent claims → Agent runs locally → uploads → reports back
### Limitations
- **No encrypted backups via Agent**: the Agent doesn't hold Master's AES-256 key. Tasks with `encrypt: true` will fail if routed to an Agent
- **Directory browse timeout**: remote dir listing is a synchronous RPC through the queue; default 15s timeout
- **Command timeout**: claimed-but-unfinished commands are marked timed out after 10 minutes
### CLI Reference
```bash
backupx agent --help
-master string Master URL
-token string Agent auth token
-config string YAML config path (takes precedence over env)
-temp-dir string Local temp directory (default /tmp/backupx-agent)
-insecure-tls Skip TLS verification (testing only)
```
---
@@ -264,7 +416,7 @@ make docker-cn # Docker build with China mirrors
### Release
```bash
git tag v1.2.3 && git push --tags
git tag v1.4.3 && git push --tags
# GitHub Actions: compile dual-arch binaries → publish GitHub Release → push Docker Hub image
```
@@ -291,12 +443,16 @@ All endpoints prefixed with `/api`, authenticated via JWT Bearer Token.
| | `POST /backup/records/:id/restore` | Restore |
| **Storage Targets** | `GET\|POST /storage-targets` | List / Add |
| | `POST /storage-targets/test` | Test connection |
| | `GET /storage-targets/rclone/backends` | Rclone backend list |
| **Nodes** | `GET\|POST /nodes` | List / Add |
| | `PUT /nodes/:id` | Edit node |
| | `GET /nodes/:id/fs/list` | Directory browser |
| | `POST /agent/heartbeat` | Agent heartbeat (Token auth) |
| **Notifications** | `GET\|POST /notifications` | List / Add |
| **Dashboard** | `GET /dashboard/stats` | Overview stats |
| **Audit Logs** | `GET /audit-logs` | Operation audit |
| **System** | `GET /system/info` | System info |
| | `GET /system/update-check` | Check for updates |
---
@@ -304,9 +460,9 @@ All endpoints prefixed with `/api`, authenticated via JWT Bearer Token.
| Component | Technology |
|-----------|-----------|
| **Backend** | Go · Gin · GORM · SQLite · robfig/cron |
| **Backend** | Go · Gin · GORM · SQLite · robfig/cron · rclone |
| **Frontend** | React 18 · TypeScript · ArcoDesign · Vite · Zustand · ECharts |
| **Storage** | AWS SDK v2 · Google Drive API v3 · gowebdav · jlaffaye/ftp |
| **Storage** | rclone (70+ backends) · AWS SDK v2 · Google Drive API v3 |
| **Security** | JWT · bcrypt · AES-256-GCM |
## Contributing

View File

@@ -0,0 +1,70 @@
package main
import (
"context"
"flag"
"fmt"
"os"
"os/signal"
"syscall"
"backupx/server/internal/agent"
)
// runAgent 是 `backupx agent` 子命令入口。
//
// 用法:
//
// backupx agent --master http://master:8340 --token <token>
// backupx agent --config /etc/backupx-agent.yaml
//
// 配置优先级CLI 参数 > 配置文件 > 环境变量
func runAgent(args []string) {
fs := flag.NewFlagSet("agent", flag.ExitOnError)
configPath := fs.String("config", "", "path to agent config YAML (optional)")
master := fs.String("master", "", "master URL, e.g. http://master.example.com:8340")
token := fs.String("token", "", "agent authentication token")
tempDir := fs.String("temp-dir", "", "local temp directory for backup artifacts")
insecureTLS := fs.Bool("insecure-tls", false, "skip TLS verification (testing only)")
if err := fs.Parse(args); err != nil {
os.Exit(2)
}
cfg, err := loadAgentConfig(*configPath)
if err != nil {
fmt.Fprintf(os.Stderr, "agent: load config: %v\n", err)
os.Exit(2)
}
cfg.MergeWithFlags(*master, *token, *tempDir)
if *insecureTLS {
cfg.InsecureSkipTLSVerify = true
}
if err := cfg.Validate(); err != nil {
fmt.Fprintf(os.Stderr, "agent: %v\n", err)
os.Exit(2)
}
a, err := agent.New(cfg, version)
if err != nil {
fmt.Fprintf(os.Stderr, "agent: init: %v\n", err)
os.Exit(1)
}
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
defer stop()
fmt.Fprintf(os.Stderr, "backupx agent %s starting (master=%s)\n", version, cfg.Master)
if err := a.Run(ctx); err != nil && err != context.Canceled {
fmt.Fprintf(os.Stderr, "agent: %v\n", err)
os.Exit(1)
}
}
// loadAgentConfig 按优先级加载配置:如果提供了 --config 就用文件,否则走环境变量。
func loadAgentConfig(configPath string) (*agent.Config, error) {
if configPath != "" {
return agent.LoadConfigFile(configPath)
}
return agent.LoadConfigFromEnv()
}

View File

@@ -0,0 +1,98 @@
package main
import (
"context"
"flag"
"fmt"
"os"
"os/signal"
"syscall"
"backupx/server/internal/backint"
)
// runBackint 是 `backupx backint` 子命令入口。
//
// CLI 参数遵循 SAP HANA Backint 规范:
//
// backupx backint -f <function> -i <input-file> -o <output-file> -p <param-file>
// [-u <user>] [-c <config-prefix>] [-l <log-file>] [-v <version>]
//
// 除 -f / -i / -o / -p 外其余参数接受但忽略(兼容 SAP 调用约定)。
func runBackint(args []string) {
fs := flag.NewFlagSet("backint", flag.ExitOnError)
fnStr := fs.String("f", "", "function: backup | restore | inquire | delete")
inputPath := fs.String("i", "", "input file path")
outputPath := fs.String("o", "", "output file path")
paramFile := fs.String("p", "", "parameter file path")
// 以下参数仅为兼容 SAP 调用约定,当前未使用
_ = fs.String("u", "", "user (ignored)")
_ = fs.String("c", "", "config-prefix (ignored)")
_ = fs.String("l", "", "log file override (ignored, use LOG_FILE in params)")
_ = fs.String("v", "", "backint version (ignored)")
if err := fs.Parse(args); err != nil {
os.Exit(2)
}
if *fnStr == "" || *inputPath == "" || *outputPath == "" || *paramFile == "" {
fmt.Fprintln(os.Stderr, "backint: -f, -i, -o, -p are required")
fs.Usage()
os.Exit(2)
}
fn, err := backint.ParseFunction(*fnStr)
if err != nil {
fmt.Fprintf(os.Stderr, "backint: %v\n", err)
os.Exit(2)
}
cfg, err := backint.LoadConfigFile(*paramFile)
if err != nil {
fmt.Fprintf(os.Stderr, "backint: load config: %v\n", err)
os.Exit(2)
}
// 配置日志重定向(如果指定 LOG_FILE
restoreLog, err := redirectStderr(cfg.LogFile)
if err != nil {
fmt.Fprintf(os.Stderr, "backint: open log: %v\n", err)
os.Exit(2)
}
defer restoreLog()
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
defer stop()
agent, err := backint.NewAgent(ctx, cfg)
if err != nil {
fmt.Fprintf(os.Stderr, "backint: init agent: %v\n", err)
os.Exit(1)
}
defer func() { _ = agent.Close() }()
if err := agent.Run(ctx, fn, *inputPath, *outputPath); err != nil {
fmt.Fprintf(os.Stderr, "backint: run: %v\n", err)
os.Exit(1)
}
}
// redirectStderr 将 stderr 重定向到指定日志文件,返回恢复函数。
// 空字符串表示保持原样。
func redirectStderr(path string) (func(), error) {
if path == "" {
return func() {}, nil
}
f, err := os.OpenFile(path, os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0644)
if err != nil {
return nil, err
}
orig := os.Stderr
os.Stderr = f
return func() {
os.Stderr = orig
_ = f.Close()
}, nil
}

View File

@@ -24,6 +24,16 @@ func main() {
runResetPassword(os.Args[2:])
return
}
// 子命令分发backintSAP HANA Backint Agent 模式)
if len(os.Args) > 1 && os.Args[1] == "backint" {
runBackint(os.Args[2:])
return
}
// 子命令分发agent远程节点 Agent 模式)
if len(os.Args) > 1 && os.Args[1] == "agent" {
runAgent(os.Args[2:])
return
}
var configPath string
var showVersion bool

View File

@@ -14,6 +14,7 @@ require (
golang.org/x/crypto v0.48.0
golang.org/x/oauth2 v0.34.0
google.golang.org/api v0.255.0
gopkg.in/yaml.v3 v3.0.1
gorm.io/gorm v1.25.12
)
@@ -245,7 +246,6 @@ require (
gopkg.in/natefinch/lumberjack.v2 v2.2.1 // indirect
gopkg.in/validator.v2 v2.0.1 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
modernc.org/libc v1.22.5 // indirect
modernc.org/mathutil v1.5.0 // indirect
modernc.org/memory v1.5.0 // indirect

View File

@@ -0,0 +1,203 @@
package agent
import (
"context"
"encoding/json"
"fmt"
"log"
"net"
"os"
"runtime"
"strings"
"sync"
"time"
)
// Agent 是 Agent 进程的主控制器。
type Agent struct {
cfg *Config
client *MasterClient
executor *Executor
version string
mu sync.Mutex
started bool
}
// New 构造 Agent。
func New(cfg *Config, version string) (*Agent, error) {
if err := cfg.Validate(); err != nil {
return nil, err
}
client := NewMasterClient(cfg.Master, cfg.Token, cfg.InsecureSkipTLSVerify)
executor := NewExecutor(client, cfg.TempDir)
return &Agent{
cfg: cfg,
client: client,
executor: executor,
version: version,
}, nil
}
// Run 启动 Agent 主循环,阻塞直到 ctx 被取消。
func (a *Agent) Run(ctx context.Context) error {
a.mu.Lock()
if a.started {
a.mu.Unlock()
return fmt.Errorf("agent already started")
}
a.started = true
a.mu.Unlock()
hbInterval := parseDuration(a.cfg.HeartbeatInterval, 15*time.Second)
pollInterval := parseDuration(a.cfg.PollInterval, 5*time.Second)
// 首次握手:通过一次心跳确认 token 有效
if err := a.heartbeatOnce(ctx); err != nil {
return fmt.Errorf("initial heartbeat failed: %w", err)
}
log.Printf("[agent] connected to master %s", a.cfg.Master)
var wg sync.WaitGroup
wg.Add(2)
go func() {
defer wg.Done()
a.heartbeatLoop(ctx, hbInterval)
}()
go func() {
defer wg.Done()
a.pollLoop(ctx, pollInterval)
}()
wg.Wait()
return ctx.Err()
}
// heartbeatLoop 定期发送心跳。
func (a *Agent) heartbeatLoop(ctx context.Context, interval time.Duration) {
ticker := time.NewTicker(interval)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
if err := a.heartbeatOnce(ctx); err != nil {
log.Printf("[agent] heartbeat failed: %v", err)
}
}
}
}
func (a *Agent) heartbeatOnce(ctx context.Context) error {
hostname, _ := os.Hostname()
req := HeartbeatRequest{
Token: a.cfg.Token,
Hostname: hostname,
IPAddress: detectLocalIP(),
AgentVersion: a.version,
OS: runtime.GOOS,
Arch: runtime.GOARCH,
}
_, err := a.client.Heartbeat(ctx, req)
return err
}
// pollLoop 定期拉取并处理待执行命令。
func (a *Agent) pollLoop(ctx context.Context, interval time.Duration) {
ticker := time.NewTicker(interval)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
a.pollAndHandleOnce(ctx)
}
}
}
func (a *Agent) pollAndHandleOnce(ctx context.Context) {
cmd, err := a.client.PollCommand(ctx)
if err != nil {
log.Printf("[agent] poll command failed: %v", err)
return
}
if cmd == nil {
return
}
log.Printf("[agent] received command #%d type=%s", cmd.ID, cmd.Type)
switch cmd.Type {
case "run_task":
a.handleRunTask(ctx, cmd)
case "list_dir":
a.handleListDir(ctx, cmd)
default:
msg := fmt.Sprintf("unknown command type: %s", cmd.Type)
log.Printf("[agent] %s", msg)
_ = a.client.SubmitCommandResult(ctx, cmd.ID, false, msg, nil)
}
}
// handleRunTask 处理 run_task 命令
func (a *Agent) handleRunTask(ctx context.Context, cmd *CommandPayload) {
var payload struct {
TaskID uint `json:"taskId"`
RecordID uint `json:"recordId"`
}
if err := json.Unmarshal(cmd.Payload, &payload); err != nil {
_ = a.client.SubmitCommandResult(ctx, cmd.ID, false, "invalid payload: "+err.Error(), nil)
return
}
if err := a.executor.ExecuteRunTask(ctx, payload.TaskID, payload.RecordID); err != nil {
_ = a.client.SubmitCommandResult(ctx, cmd.ID, false, err.Error(), nil)
return
}
_ = a.client.SubmitCommandResult(ctx, cmd.ID, true, "", map[string]any{
"taskId": payload.TaskID,
"recordId": payload.RecordID,
})
}
// handleListDir 处理 list_dir 命令(阶段四实现)
func (a *Agent) handleListDir(ctx context.Context, cmd *CommandPayload) {
var payload struct {
Path string `json:"path"`
}
if err := json.Unmarshal(cmd.Payload, &payload); err != nil {
_ = a.client.SubmitCommandResult(ctx, cmd.ID, false, "invalid payload: "+err.Error(), nil)
return
}
entries, err := listLocalDir(payload.Path)
if err != nil {
_ = a.client.SubmitCommandResult(ctx, cmd.ID, false, err.Error(), nil)
return
}
_ = a.client.SubmitCommandResult(ctx, cmd.ID, true, "", map[string]any{"entries": entries})
}
// 辅助函数
func parseDuration(s string, fallback time.Duration) time.Duration {
if strings.TrimSpace(s) == "" {
return fallback
}
if d, err := time.ParseDuration(s); err == nil {
return d
}
return fallback
}
func detectLocalIP() string {
addrs, err := net.InterfaceAddrs()
if err != nil {
return ""
}
for _, addr := range addrs {
if ipNet, ok := addr.(*net.IPNet); ok && !ipNet.IP.IsLoopback() {
if ipNet.IP.To4() != nil {
return ipNet.IP.String()
}
}
}
return ""
}

View File

@@ -0,0 +1,208 @@
package agent
import (
"bytes"
"context"
"crypto/tls"
"encoding/json"
"fmt"
"io"
"net/http"
"strings"
"time"
)
// MasterClient 是 Agent 调用 Master HTTP API 的封装。
type MasterClient struct {
baseURL string
token string
httpClient *http.Client
}
// NewMasterClient 构造 Master 客户端。
func NewMasterClient(baseURL, token string, insecureTLS bool) *MasterClient {
transport := &http.Transport{}
if insecureTLS {
transport.TLSClientConfig = &tls.Config{InsecureSkipVerify: true}
}
return &MasterClient{
baseURL: strings.TrimRight(baseURL, "/"),
token: token,
httpClient: &http.Client{
Timeout: 120 * time.Second,
Transport: transport,
},
}
}
// HeartbeatRequest Agent 上报心跳的请求
type HeartbeatRequest struct {
Token string `json:"token"`
Hostname string `json:"hostname,omitempty"`
IPAddress string `json:"ipAddress,omitempty"`
AgentVersion string `json:"agentVersion,omitempty"`
OS string `json:"os,omitempty"`
Arch string `json:"arch,omitempty"`
}
// HeartbeatResponse Master 返回的心跳响应
type HeartbeatResponse struct {
Status string `json:"status"`
NodeID uint `json:"nodeId"`
Name string `json:"name"`
}
// Heartbeat 上报心跳并获取节点元信息
func (c *MasterClient) Heartbeat(ctx context.Context, req HeartbeatRequest) (*HeartbeatResponse, error) {
var resp HeartbeatResponse
if err := c.do(ctx, http.MethodPost, "/api/agent/heartbeat", req, &resp); err != nil {
return nil, err
}
return &resp, nil
}
// CommandPayload 与 service.AgentCommandPayload 对齐
type CommandPayload struct {
ID uint `json:"id"`
Type string `json:"type"`
Payload json.RawMessage `json:"payload,omitempty"`
}
// PollCommandResponse 轮询响应:无命令时 Command 为 nil
type PollCommandResponse struct {
Command *CommandPayload `json:"command"`
}
// PollCommand 拉取下一条待执行命令
func (c *MasterClient) PollCommand(ctx context.Context) (*CommandPayload, error) {
var resp PollCommandResponse
if err := c.do(ctx, http.MethodPost, "/api/agent/commands/poll", nil, &resp); err != nil {
return nil, err
}
return resp.Command, nil
}
// SubmitCommandResult 上报命令执行结果
func (c *MasterClient) SubmitCommandResult(ctx context.Context, cmdID uint, success bool, errorMsg string, result any) error {
var resultJSON json.RawMessage
if result != nil {
data, err := json.Marshal(result)
if err != nil {
return fmt.Errorf("marshal result: %w", err)
}
resultJSON = data
}
payload := map[string]any{
"success": success,
"errorMessage": errorMsg,
}
if resultJSON != nil {
payload["result"] = resultJSON
}
path := fmt.Sprintf("/api/agent/commands/%d/result", cmdID)
return c.do(ctx, http.MethodPost, path, payload, nil)
}
// TaskSpec 与 service.AgentTaskSpec 对齐
type TaskSpec struct {
TaskID uint `json:"taskId"`
Name string `json:"name"`
Type string `json:"type"`
SourcePath string `json:"sourcePath"`
SourcePaths string `json:"sourcePaths"`
ExcludePatterns string `json:"excludePatterns"`
DBHost string `json:"dbHost"`
DBPort int `json:"dbPort"`
DBUser string `json:"dbUser"`
DBPassword string `json:"dbPassword"`
DBName string `json:"dbName"`
DBPath string `json:"dbPath"`
ExtraConfig string `json:"extraConfig"`
Compression string `json:"compression"`
Encrypt bool `json:"encrypt"`
StorageTargets []StorageTargetConfig `json:"storageTargets"`
}
// StorageTargetConfig 与 service.AgentStorageTargetConfig 对齐
type StorageTargetConfig struct {
ID uint `json:"id"`
Type string `json:"type"`
Name string `json:"name"`
Config json.RawMessage `json:"config"`
}
// GetTaskSpec 拉取任务规格
func (c *MasterClient) GetTaskSpec(ctx context.Context, taskID uint) (*TaskSpec, error) {
var spec TaskSpec
path := fmt.Sprintf("/api/agent/tasks/%d", taskID)
if err := c.do(ctx, http.MethodGet, path, nil, &spec); err != nil {
return nil, err
}
return &spec, nil
}
// RecordUpdate 与 service.AgentRecordUpdate 对齐
type RecordUpdate struct {
Status string `json:"status,omitempty"`
FileName string `json:"fileName,omitempty"`
FileSize int64 `json:"fileSize,omitempty"`
Checksum string `json:"checksum,omitempty"`
StoragePath string `json:"storagePath,omitempty"`
ErrorMessage string `json:"errorMessage,omitempty"`
LogAppend string `json:"logAppend,omitempty"`
}
// UpdateRecord 上报备份记录的状态/日志
func (c *MasterClient) UpdateRecord(ctx context.Context, recordID uint, update RecordUpdate) error {
path := fmt.Sprintf("/api/agent/records/%d", recordID)
return c.do(ctx, http.MethodPost, path, update, nil)
}
// do 是通用 HTTP 调用。所有 Agent API 都统一走 JSON + X-Agent-Token。
func (c *MasterClient) do(ctx context.Context, method, path string, body any, out any) error {
var reqBody io.Reader
if body != nil {
data, err := json.Marshal(body)
if err != nil {
return fmt.Errorf("marshal request: %w", err)
}
reqBody = bytes.NewReader(data)
}
req, err := http.NewRequestWithContext(ctx, method, c.baseURL+path, reqBody)
if err != nil {
return err
}
req.Header.Set("X-Agent-Token", c.token)
if body != nil {
req.Header.Set("Content-Type", "application/json")
}
resp, err := c.httpClient.Do(req)
if err != nil {
return fmt.Errorf("%s %s: %w", method, path, err)
}
defer resp.Body.Close()
data, err := io.ReadAll(resp.Body)
if err != nil {
return fmt.Errorf("read response: %w", err)
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return fmt.Errorf("%s %s: http %d: %s", method, path, resp.StatusCode, string(data))
}
if out == nil {
return nil
}
// BackupX API 统一封装成 {code, data, message} 形式,需要解出 data 字段
var envelope struct {
Code string `json:"code"`
Data json.RawMessage `json:"data"`
Message string `json:"message"`
}
if err := json.Unmarshal(data, &envelope); err == nil && envelope.Data != nil {
if err := json.Unmarshal(envelope.Data, out); err != nil {
return fmt.Errorf("decode data: %w", err)
}
return nil
}
// 兼容直接返回对象的情况
return json.Unmarshal(data, out)
}

View File

@@ -0,0 +1,105 @@
// Package agent 实现 BackupX 远程 Agent。
//
// Agent 是一个独立的 Go 进程,部署在远程服务器上,通过 HTTP 轮询的方式
// 与 Master 通信:定期上报心跳、拉取 Master 下发的命令、本地执行备份、
// 把执行结果和日志回报给 Master。
//
// 通信协议见 server/internal/http/agent_handler.go。
package agent
import (
"errors"
"fmt"
"os"
"strings"
"gopkg.in/yaml.v3"
)
// Config 是 Agent 的运行时配置。
type Config struct {
// Master BackupX Master 的 HTTP 基础地址,例如 http://master.example.com:8340
Master string `yaml:"master"`
// Token 节点认证令牌(在 Master 创建节点时生成)
Token string `yaml:"token"`
// HeartbeatInterval 心跳间隔,默认 15s
HeartbeatInterval string `yaml:"heartbeatInterval"`
// PollInterval 命令轮询间隔,默认 5s
PollInterval string `yaml:"pollInterval"`
// TempDir 备份临时目录,默认 /tmp/backupx-agent
TempDir string `yaml:"tempDir"`
// InsecureSkipTLSVerify 测试环境允许跳过 TLS 证书校验
InsecureSkipTLSVerify bool `yaml:"insecureSkipTlsVerify"`
}
// LoadConfigFile 从 YAML 文件加载 Agent 配置。
func LoadConfigFile(path string) (*Config, error) {
data, err := os.ReadFile(path)
if err != nil {
return nil, fmt.Errorf("read agent config: %w", err)
}
var cfg Config
if err := yaml.Unmarshal(data, &cfg); err != nil {
return nil, fmt.Errorf("parse agent config: %w", err)
}
return applyConfigDefaults(&cfg)
}
// LoadConfigFromEnv 从环境变量加载 Agent 配置。优先级低于 --config 文件。
//
// 支持的环境变量:
// - BACKUPX_AGENT_MASTER Master URL
// - BACKUPX_AGENT_TOKEN 节点认证令牌
// - BACKUPX_AGENT_HEARTBEAT 心跳间隔(如 15s
// - BACKUPX_AGENT_POLL 命令轮询间隔(如 5s
// - BACKUPX_AGENT_TEMP_DIR 临时目录
// - BACKUPX_AGENT_INSECURE_TLS true / 1 跳过 TLS 校验
func LoadConfigFromEnv() (*Config, error) {
cfg := &Config{
Master: strings.TrimSpace(os.Getenv("BACKUPX_AGENT_MASTER")),
Token: strings.TrimSpace(os.Getenv("BACKUPX_AGENT_TOKEN")),
HeartbeatInterval: strings.TrimSpace(os.Getenv("BACKUPX_AGENT_HEARTBEAT")),
PollInterval: strings.TrimSpace(os.Getenv("BACKUPX_AGENT_POLL")),
TempDir: strings.TrimSpace(os.Getenv("BACKUPX_AGENT_TEMP_DIR")),
InsecureSkipTLSVerify: strings.EqualFold(os.Getenv("BACKUPX_AGENT_INSECURE_TLS"), "true") || os.Getenv("BACKUPX_AGENT_INSECURE_TLS") == "1",
}
return applyConfigDefaults(cfg)
}
// MergeWithFlags 把命令行覆盖值合并入配置(非空覆盖)。
func (c *Config) MergeWithFlags(master, token, tempDir string) {
if strings.TrimSpace(master) != "" {
c.Master = master
}
if strings.TrimSpace(token) != "" {
c.Token = token
}
if strings.TrimSpace(tempDir) != "" {
c.TempDir = tempDir
}
}
// Validate 校验必填字段。
func (c *Config) Validate() error {
if strings.TrimSpace(c.Master) == "" {
return errors.New("master url is required (set via --master, BACKUPX_AGENT_MASTER or config file)")
}
if strings.TrimSpace(c.Token) == "" {
return errors.New("token is required (set via --token, BACKUPX_AGENT_TOKEN or config file)")
}
return nil
}
func applyConfigDefaults(cfg *Config) (*Config, error) {
if cfg.HeartbeatInterval == "" {
cfg.HeartbeatInterval = "15s"
}
if cfg.PollInterval == "" {
cfg.PollInterval = "5s"
}
if cfg.TempDir == "" {
cfg.TempDir = "/tmp/backupx-agent"
}
cfg.Master = strings.TrimRight(strings.TrimSpace(cfg.Master), "/")
return cfg, nil
}

View File

@@ -0,0 +1,101 @@
package agent
import (
"os"
"path/filepath"
"testing"
)
func TestLoadConfigFile(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "agent.yaml")
content := `master: http://master.example.com:8340/
token: abc123
heartbeatInterval: 20s
pollInterval: 3s
tempDir: /var/backupx-agent
insecureSkipTlsVerify: true
`
if err := os.WriteFile(path, []byte(content), 0644); err != nil {
t.Fatal(err)
}
cfg, err := LoadConfigFile(path)
if err != nil {
t.Fatalf("load: %v", err)
}
if cfg.Master != "http://master.example.com:8340" {
t.Errorf("trailing slash should be trimmed: %q", cfg.Master)
}
if cfg.Token != "abc123" {
t.Errorf("token: %q", cfg.Token)
}
if cfg.HeartbeatInterval != "20s" || cfg.PollInterval != "3s" {
t.Errorf("intervals: %+v", cfg)
}
if !cfg.InsecureSkipTLSVerify {
t.Errorf("insecure should be true")
}
}
func TestLoadConfigDefaults(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "agent.yaml")
if err := os.WriteFile(path, []byte("master: http://m\ntoken: t\n"), 0644); err != nil {
t.Fatal(err)
}
cfg, err := LoadConfigFile(path)
if err != nil {
t.Fatal(err)
}
if cfg.HeartbeatInterval != "15s" || cfg.PollInterval != "5s" {
t.Errorf("default intervals not applied: %+v", cfg)
}
if cfg.TempDir != "/tmp/backupx-agent" {
t.Errorf("default tempdir: %q", cfg.TempDir)
}
}
func TestConfigValidate(t *testing.T) {
cases := []struct {
name string
cfg Config
wantErr bool
}{
{"valid", Config{Master: "http://m", Token: "t"}, false},
{"missing master", Config{Token: "t"}, true},
{"missing token", Config{Master: "http://m"}, true},
}
for _, c := range cases {
err := c.cfg.Validate()
if (err != nil) != c.wantErr {
t.Errorf("%s: err=%v wantErr=%v", c.name, err, c.wantErr)
}
}
}
func TestMergeWithFlags(t *testing.T) {
cfg := &Config{Master: "http://old", Token: "old"}
cfg.MergeWithFlags("http://new", "", "/tmp/x")
if cfg.Master != "http://new" {
t.Errorf("master not overridden: %q", cfg.Master)
}
if cfg.Token != "old" {
t.Errorf("empty flag should not override: %q", cfg.Token)
}
if cfg.TempDir != "/tmp/x" {
t.Errorf("tempDir: %q", cfg.TempDir)
}
}
func TestLoadConfigFromEnv(t *testing.T) {
t.Setenv("BACKUPX_AGENT_MASTER", "http://env-master")
t.Setenv("BACKUPX_AGENT_TOKEN", "env-token")
t.Setenv("BACKUPX_AGENT_INSECURE_TLS", "true")
cfg, err := LoadConfigFromEnv()
if err != nil {
t.Fatal(err)
}
if cfg.Master != "http://env-master" || cfg.Token != "env-token" || !cfg.InsecureSkipTLSVerify {
t.Errorf("env not picked up: %+v", cfg)
}
}

View File

@@ -0,0 +1,266 @@
package agent
import (
"context"
"crypto/sha256"
"encoding/hex"
"fmt"
"io"
"os"
"path/filepath"
"strings"
"time"
"backupx/server/internal/backup"
"backupx/server/internal/storage"
storageRclone "backupx/server/internal/storage/rclone"
"backupx/server/pkg/compress"
)
// Executor 负责在 Agent 本地执行命令。
type Executor struct {
client *MasterClient
tempDir string
backupRegistry *backup.Registry
storageRegistry *storage.Registry
}
// NewExecutor 构造执行器。预先初始化 backup runner 与 storage registry。
func NewExecutor(client *MasterClient, tempDir string) *Executor {
backupRegistry := backup.NewRegistry(
backup.NewFileRunner(),
backup.NewSQLiteRunner(),
backup.NewMySQLRunner(nil),
backup.NewPostgreSQLRunner(nil),
backup.NewSAPHANARunner(nil),
)
storageRegistry := storage.NewRegistry(
storageRclone.NewLocalDiskFactory(),
storageRclone.NewS3Factory(),
storageRclone.NewWebDAVFactory(),
storageRclone.NewGoogleDriveFactory(),
storageRclone.NewAliyunOSSFactory(),
storageRclone.NewTencentCOSFactory(),
storageRclone.NewQiniuKodoFactory(),
storageRclone.NewFTPFactory(),
storageRclone.NewRcloneFactory(),
)
storageRclone.RegisterAllBackends(storageRegistry)
return &Executor{
client: client,
tempDir: tempDir,
backupRegistry: backupRegistry,
storageRegistry: storageRegistry,
}
}
// ExecuteRunTask 处理 run_task 命令:拉规格 → 执行 runner → 压缩 → 上传 → 上报记录。
//
// 注意Agent 当前不支持 Encrypt=true加密密钥不下发到 Agent避免密钥扩散
// 遇到启用加密的任务会向 Master 上报失败并返回错误。
func (e *Executor) ExecuteRunTask(ctx context.Context, taskID, recordID uint) error {
// 1) 拉取任务规格
spec, err := e.client.GetTaskSpec(ctx, taskID)
if err != nil {
e.reportRecordFailure(ctx, recordID, fmt.Sprintf("拉取任务规格失败: %v", err))
return err
}
if spec.Encrypt {
msg := "Agent 不支持加密备份(加密密钥仅在 Master 端持有)"
e.reportRecordFailure(ctx, recordID, msg)
return fmt.Errorf("%s", msg)
}
e.appendLog(ctx, recordID, fmt.Sprintf("[agent] 开始执行任务 %s (type=%s)\n", spec.Name, spec.Type))
// 2) 构造 backup.TaskSpec 并找对应 runner
startedAt := time.Now().UTC()
if err := os.MkdirAll(e.tempDir, 0o755); err != nil {
e.reportRecordFailure(ctx, recordID, fmt.Sprintf("创建临时目录失败: %v", err))
return err
}
backupSpec := buildBackupTaskSpec(spec, startedAt, e.tempDir)
runner, err := e.backupRegistry.Runner(backupSpec.Type)
if err != nil {
e.reportRecordFailure(ctx, recordID, fmt.Sprintf("不支持的备份类型: %v", err))
return err
}
// 3) 运行 runner
logger := newRecordLogger(ctx, e.client, recordID)
result, err := runner.Run(ctx, backupSpec, logger)
if err != nil {
e.reportRecordFailure(ctx, recordID, err.Error())
return err
}
defer os.RemoveAll(result.TempDir)
// 4) 可选 gzip 压缩
finalPath := result.ArtifactPath
if strings.EqualFold(spec.Compression, "gzip") && !strings.HasSuffix(strings.ToLower(finalPath), ".gz") {
e.appendLog(ctx, recordID, "[agent] 开始压缩备份文件\n")
compressedPath, compressErr := compress.GzipFile(finalPath)
if compressErr != nil {
e.reportRecordFailure(ctx, recordID, fmt.Sprintf("压缩失败: %v", compressErr))
return compressErr
}
finalPath = compressedPath
}
info, err := os.Stat(finalPath)
if err != nil {
e.reportRecordFailure(ctx, recordID, fmt.Sprintf("获取文件信息失败: %v", err))
return err
}
fileName := filepath.Base(finalPath)
fileSize := info.Size()
storagePath := backup.BuildStorageKey(spec.Type, startedAt, fileName)
// 5) 计算 checksum一次读一次并上传到所有目标
checksum, err := computeFileSHA256(finalPath)
if err != nil {
e.reportRecordFailure(ctx, recordID, fmt.Sprintf("计算 checksum 失败: %v", err))
return err
}
if len(spec.StorageTargets) == 0 {
e.reportRecordFailure(ctx, recordID, "没有关联的存储目标")
return fmt.Errorf("no storage targets")
}
for _, target := range spec.StorageTargets {
if err := e.uploadToTarget(ctx, recordID, target, finalPath, storagePath, fileSize, spec.TaskID); err != nil {
e.reportRecordFailure(ctx, recordID, fmt.Sprintf("上传到 %s 失败: %v", target.Name, err))
return err
}
e.appendLog(ctx, recordID, fmt.Sprintf("[agent] 已上传到存储目标 %s\n", target.Name))
}
// 6) 上报最终成功
return e.client.UpdateRecord(ctx, recordID, RecordUpdate{
Status: "success",
FileName: fileName,
FileSize: fileSize,
Checksum: checksum,
StoragePath: storagePath,
LogAppend: fmt.Sprintf("[agent] 任务完成,总计 %d 字节\n", fileSize),
})
}
// uploadToTarget 上传单个目标。为保持简化不做上传级重试rclone 本身已有 low-level 重试)。
func (e *Executor) uploadToTarget(ctx context.Context, recordID uint, target StorageTargetConfig, filePath, objectKey string, fileSize int64, taskID uint) error {
var rawConfig map[string]any
if len(target.Config) > 0 {
// DecodeRawConfig 通过 json 解析
if err := jsonUnmarshalMap(target.Config, &rawConfig); err != nil {
return fmt.Errorf("parse storage config: %w", err)
}
}
provider, err := e.storageRegistry.Create(ctx, target.Type, rawConfig)
if err != nil {
return fmt.Errorf("create provider: %w", err)
}
f, err := os.Open(filePath)
if err != nil {
return fmt.Errorf("open artifact: %w", err)
}
defer f.Close()
meta := map[string]string{
"taskId": fmt.Sprintf("%d", taskID),
"recordId": fmt.Sprintf("%d", recordID),
}
return provider.Upload(ctx, objectKey, f, fileSize, meta)
}
// appendLog 追加日志到 Master 记录(尽力而为,失败不中断主流程)
func (e *Executor) appendLog(ctx context.Context, recordID uint, line string) {
_ = e.client.UpdateRecord(ctx, recordID, RecordUpdate{LogAppend: line})
}
// reportRecordFailure 上报失败状态
func (e *Executor) reportRecordFailure(ctx context.Context, recordID uint, msg string) {
_ = e.client.UpdateRecord(ctx, recordID, RecordUpdate{
Status: "failed",
ErrorMessage: msg,
LogAppend: fmt.Sprintf("[agent] 错误: %s\n", msg),
})
}
// buildBackupTaskSpec 把 AgentTaskSpec 转换为 backup.TaskSpec。
func buildBackupTaskSpec(spec *TaskSpec, startedAt time.Time, tempDir string) backup.TaskSpec {
var sourcePaths []string
if strings.TrimSpace(spec.SourcePaths) != "" {
for _, p := range strings.Split(spec.SourcePaths, "\n") {
if p = strings.TrimSpace(p); p != "" {
sourcePaths = append(sourcePaths, p)
}
}
}
var excludes []string
if strings.TrimSpace(spec.ExcludePatterns) != "" {
for _, p := range strings.Split(spec.ExcludePatterns, "\n") {
if p = strings.TrimSpace(p); p != "" {
excludes = append(excludes, p)
}
}
}
return backup.TaskSpec{
ID: spec.TaskID,
Name: spec.Name,
Type: spec.Type,
SourcePath: spec.SourcePath,
SourcePaths: sourcePaths,
ExcludePatterns: excludes,
Database: backup.DatabaseSpec{
Host: spec.DBHost,
Port: spec.DBPort,
User: spec.DBUser,
Password: spec.DBPassword,
Path: spec.DBPath,
Names: splitCommaOrNewline(spec.DBName),
},
Compression: spec.Compression,
Encrypt: spec.Encrypt,
StartedAt: startedAt,
TempDir: tempDir,
}
}
// recordLogger 把 runner 日志回传到 Master 记录。
// 实现 backup.LogWriter每条日志追加到 record.log_content。
type recordLogger struct {
ctx context.Context
client *MasterClient
recordID uint
}
func newRecordLogger(ctx context.Context, client *MasterClient, recordID uint) *recordLogger {
return &recordLogger{ctx: ctx, client: client, recordID: recordID}
}
func (l *recordLogger) WriteLine(message string) {
_ = l.client.UpdateRecord(l.ctx, l.recordID, RecordUpdate{LogAppend: message + "\n"})
}
// 辅助函数
func computeFileSHA256(path string) (string, error) {
f, err := os.Open(path)
if err != nil {
return "", err
}
defer f.Close()
h := sha256.New()
if _, err := io.Copy(h, f); err != nil {
return "", err
}
return hex.EncodeToString(h.Sum(nil)), nil
}
func splitCommaOrNewline(s string) []string {
var result []string
for _, part := range strings.FieldsFunc(s, func(r rune) bool {
return r == ',' || r == '\n' || r == ';'
}) {
if p := strings.TrimSpace(part); p != "" {
result = append(result, p)
}
}
return result
}

View File

@@ -0,0 +1,49 @@
package agent
import (
"fmt"
"os"
"path/filepath"
"sort"
)
// DirEntry Agent 返回给 Master 的目录项。
type DirEntry struct {
Name string `json:"name"`
Path string `json:"path"`
IsDir bool `json:"isDir"`
Size int64 `json:"size"`
}
// listLocalDir 列出 Agent 所在机器的指定路径。
func listLocalDir(path string) ([]DirEntry, error) {
cleaned := filepath.Clean(path)
if cleaned == "" {
cleaned = "/"
}
entries, err := os.ReadDir(cleaned)
if err != nil {
return nil, fmt.Errorf("read dir: %w", err)
}
result := make([]DirEntry, 0, len(entries))
for _, entry := range entries {
info, _ := entry.Info()
size := int64(0)
if info != nil && !entry.IsDir() {
size = info.Size()
}
result = append(result, DirEntry{
Name: entry.Name(),
Path: filepath.Join(cleaned, entry.Name()),
IsDir: entry.IsDir(),
Size: size,
})
}
sort.Slice(result, func(i, j int) bool {
if result[i].IsDir != result[j].IsDir {
return result[i].IsDir
}
return result[i].Name < result[j].Name
})
return result, nil
}

View File

@@ -0,0 +1,61 @@
package agent
import (
"os"
"path/filepath"
"testing"
)
func TestListLocalDir(t *testing.T) {
dir := t.TempDir()
_ = os.WriteFile(filepath.Join(dir, "a.txt"), []byte("hello"), 0644)
_ = os.Mkdir(filepath.Join(dir, "sub"), 0755)
_ = os.WriteFile(filepath.Join(dir, "b.txt"), []byte("world!"), 0644)
entries, err := listLocalDir(dir)
if err != nil {
t.Fatalf("list: %v", err)
}
if len(entries) != 3 {
t.Fatalf("expected 3 entries, got %d", len(entries))
}
// 目录排序靠前
if !entries[0].IsDir || entries[0].Name != "sub" {
t.Errorf("directories should sort first: %+v", entries)
}
// 文件大小正确
var a *DirEntry
for i := range entries {
if entries[i].Name == "a.txt" {
a = &entries[i]
break
}
}
if a == nil || a.Size != 5 {
t.Errorf("file size: %+v", a)
}
}
func TestSplitCommaOrNewline(t *testing.T) {
cases := []struct {
in string
out []string
}{
{"", nil},
{"a,b,c", []string{"a", "b", "c"}},
{"a\nb\nc", []string{"a", "b", "c"}},
{"a; b ,\nc\n", []string{"a", "b", "c"}},
}
for _, c := range cases {
got := splitCommaOrNewline(c.in)
if len(got) != len(c.out) {
t.Errorf("%q: got %v want %v", c.in, got, c.out)
continue
}
for i := range got {
if got[i] != c.out[i] {
t.Errorf("%q[%d]: %q vs %q", c.in, i, got[i], c.out[i])
}
}
}
}

View File

@@ -0,0 +1,12 @@
package agent
import "encoding/json"
// jsonUnmarshalMap 把 []byte 或 json.RawMessage 解为 map[string]any。
func jsonUnmarshalMap(data []byte, out *map[string]any) error {
if len(data) == 0 {
*out = map[string]any{}
return nil
}
return json.Unmarshal(data, out)
}

View File

@@ -79,6 +79,7 @@ func New(ctx context.Context, cfg config.Config, version string) (*Application,
storageTargetService.SetBackupTaskRepository(backupTaskRepo)
storageTargetService.SetBackupRecordRepository(backupRecordRepo)
backupTaskService := service.NewBackupTaskService(backupTaskRepo, storageTargetRepo, configCipher)
backupTaskService.SetRecordsAndStorage(backupRecordRepo, storageRegistry)
backupRunnerRegistry := backup.NewRegistry(backup.NewFileRunner(), backup.NewSQLiteRunner(), backup.NewMySQLRunner(nil), backup.NewPostgreSQLRunner(nil), backup.NewSAPHANARunner(nil))
logHub := backup.NewLogHub()
retentionService := backupretention.NewService(backupRecordRepo)
@@ -110,10 +111,22 @@ func New(ctx context.Context, cfg config.Config, version string) (*Application,
// Cluster: Node management
nodeRepo := repository.NewNodeRepository(db)
nodeService := service.NewNodeService(nodeRepo)
nodeService := service.NewNodeService(nodeRepo, version)
nodeService.SetTaskRepository(backupTaskRepo)
if err := nodeService.EnsureLocalNode(ctx); err != nil {
appLogger.Warn("failed to ensure local node", zap.Error(err))
}
// 启动离线检测:每 15s 扫描一次,超过 45s 未心跳的远程节点标记为离线
nodeService.StartOfflineMonitor(ctx, 15*time.Second)
// Agent 协议服务:命令队列 + 任务下发 + 记录上报
agentCmdRepo := repository.NewAgentCommandRepository(db)
agentService := service.NewAgentService(nodeRepo, backupTaskRepo, backupRecordRepo, storageTargetRepo, agentCmdRepo, configCipher)
agentService.StartCommandTimeoutMonitor(ctx, 30*time.Second, 10*time.Minute)
// 把 Agent 下发能力注入到备份执行服务,实现多节点路由
backupExecutionService.SetClusterDependencies(nodeRepo, agentService)
// 启用远程目录浏览NodeService 通过 AgentService 做同步 RPC
nodeService.SetAgentRPC(agentService)
router := aphttp.NewRouter(aphttp.RouterDependencies{
Config: cfg,
@@ -129,6 +142,7 @@ func New(ctx context.Context, cfg config.Config, version string) (*Application,
DashboardService: dashboardService,
SettingsService: settingsService,
NodeService: nodeService,
AgentService: agentService,
DatabaseDiscoveryService: databaseDiscoveryService,
AuditService: auditService,
JWTManager: jwtManager,

View File

@@ -0,0 +1,360 @@
package backint
import (
"compress/gzip"
"context"
"crypto/rand"
"encoding/hex"
"fmt"
"io"
"os"
"path"
"strings"
"time"
"backupx/server/internal/storage"
storageRclone "backupx/server/internal/storage/rclone"
)
// Agent 是 Backint 协议代理主入口。
//
// 职责:
// 1. 根据 -f 指定的功能,从 -i 输入文件解析请求
// 2. 把数据路由到 BackupX storage 后端
// 3. 把结果写回 -o 输出文件(失败使用 #ERROR不中断批次
type Agent struct {
cfg *Config
provider storage.StorageProvider
catalog *Catalog
}
// NewAgent 构造 Agent初始化 storage provider 与 catalog。
func NewAgent(ctx context.Context, cfg *Config) (*Agent, error) {
registry := buildStorageRegistry()
provider, err := registry.Create(ctx, cfg.StorageType, cfg.StorageConfig)
if err != nil {
return nil, fmt.Errorf("create storage provider: %w", err)
}
if err := provider.TestConnection(ctx); err != nil {
return nil, fmt.Errorf("storage provider connection failed: %w", err)
}
cat, err := OpenCatalog(cfg.CatalogDB)
if err != nil {
return nil, err
}
return &Agent{cfg: cfg, provider: provider, catalog: cat}, nil
}
// Close 释放资源。
func (a *Agent) Close() error {
if a.catalog != nil {
return a.catalog.Close()
}
return nil
}
// Run 执行一次 Backint 调用。
//
// HANA 针对 BACKUP 调用时input 是 #PIPE 列表output 需返回 #SAVED 或 #ERROR。
// 批次中任一条目失败不应导致整个进程退出,因此错误被降级为 #ERROR 行。
// 仅在极端错误参数非法、I/O 失败)时返回 error进程以非 0 退出。
func (a *Agent) Run(ctx context.Context, fn Function, inputPath, outputPath string) error {
in, err := os.Open(inputPath)
if err != nil {
return fmt.Errorf("open input: %w", err)
}
defer in.Close()
out, err := os.Create(outputPath)
if err != nil {
return fmt.Errorf("create output: %w", err)
}
defer out.Close()
switch fn {
case FunctionBackup:
return a.runBackup(ctx, in, out)
case FunctionRestore:
return a.runRestore(ctx, in, out)
case FunctionInquire:
return a.runInquire(ctx, in, out)
case FunctionDelete:
return a.runDelete(ctx, in, out)
default:
return fmt.Errorf("unsupported function: %s", fn)
}
}
// runBackup 处理 BACKUP 操作:读取每条请求的管道/文件,上传到存储后端。
func (a *Agent) runBackup(ctx context.Context, in io.Reader, out io.Writer) error {
reqs, err := ParseBackupRequests(in)
if err != nil {
return err
}
for _, req := range reqs {
ebid, perr := a.handleBackupOne(ctx, req)
if perr != nil {
fmt.Fprintf(os.Stderr, "backint: backup %q failed: %v\n", req.Path, perr)
_ = WriteError(out, req.Path)
continue
}
_ = WriteSaved(out, ebid, req.Path)
}
return nil
}
// handleBackupOne 上传一条请求,返回分配的 EBID。
func (a *Agent) handleBackupOne(ctx context.Context, req BackupRequest) (string, error) {
src, size, err := openBackupSource(req)
if err != nil {
return "", err
}
defer src.Close()
ebid := generateEBID()
objectKey := a.objectKeyFor(ebid)
reader := io.Reader(src)
// 可选 gzip 压缩
if a.cfg.Compress {
pr, pw := io.Pipe()
go func() {
gw := gzip.NewWriter(pw)
if _, cerr := io.Copy(gw, src); cerr != nil {
_ = gw.Close()
_ = pw.CloseWithError(cerr)
return
}
if cerr := gw.Close(); cerr != nil {
_ = pw.CloseWithError(cerr)
return
}
_ = pw.Close()
}()
reader = pr
size = -1 // 压缩后大小未知
objectKey += ".gz"
}
meta := map[string]string{
"source-path": req.Path,
"ebid": ebid,
"compress": boolStr(a.cfg.Compress),
}
if err := a.provider.Upload(ctx, objectKey, reader, size, meta); err != nil {
return "", fmt.Errorf("upload: %w", err)
}
if err := a.catalog.Put(CatalogEntry{
EBID: ebid,
ObjectKey: objectKey,
SourcePath: req.Path,
Size: size,
}); err != nil {
return "", fmt.Errorf("catalog put: %w", err)
}
return ebid, nil
}
// runRestore 处理 RESTORE 操作:根据 EBID 从存储下载,写入 HANA 指定的管道/文件。
func (a *Agent) runRestore(ctx context.Context, in io.Reader, out io.Writer) error {
reqs, err := ParseRestoreRequests(in)
if err != nil {
return err
}
for _, req := range reqs {
if perr := a.handleRestoreOne(ctx, req); perr != nil {
fmt.Fprintf(os.Stderr, "backint: restore %q failed: %v\n", req.EBID, perr)
_ = WriteError(out, req.Path)
continue
}
_ = WriteRestored(out, req.EBID, req.Path)
}
return nil
}
func (a *Agent) handleRestoreOne(ctx context.Context, req RestoreRequest) error {
entry, err := a.catalog.Get(req.EBID)
if err != nil {
return fmt.Errorf("catalog get: %w", err)
}
if entry == nil {
return fmt.Errorf("ebid not found: %s", req.EBID)
}
rc, err := a.provider.Download(ctx, entry.ObjectKey)
if err != nil {
return fmt.Errorf("download: %w", err)
}
defer rc.Close()
var src io.Reader = rc
if strings.HasSuffix(entry.ObjectKey, ".gz") {
gr, err := gzip.NewReader(rc)
if err != nil {
return fmt.Errorf("gzip reader: %w", err)
}
defer gr.Close()
src = gr
}
dst, err := openRestoreTarget(req)
if err != nil {
return err
}
defer dst.Close()
if _, err := io.Copy(dst, src); err != nil {
return fmt.Errorf("copy to target: %w", err)
}
return nil
}
// runInquire 处理 INQUIRE 操作:查询 EBID 是否存在,或列出全部备份。
func (a *Agent) runInquire(ctx context.Context, in io.Reader, out io.Writer) error {
reqs, err := ParseInquireRequests(in)
if err != nil {
return err
}
for _, req := range reqs {
if req.All {
entries, err := a.catalog.List()
if err != nil {
fmt.Fprintf(os.Stderr, "backint: inquire list failed: %v\n", err)
_ = WriteError(out, "#NULL")
continue
}
for _, e := range entries {
_ = WriteBackup(out, e.EBID)
}
continue
}
entry, err := a.catalog.Get(req.EBID)
if err != nil {
fmt.Fprintf(os.Stderr, "backint: inquire %q failed: %v\n", req.EBID, err)
_ = WriteError(out, req.EBID)
continue
}
if entry == nil {
_ = WriteNotFound(out, req.EBID)
continue
}
_ = WriteBackup(out, entry.EBID)
}
return nil
}
// runDelete 处理 DELETE 操作:从存储删除对象并移除目录条目。
func (a *Agent) runDelete(ctx context.Context, in io.Reader, out io.Writer) error {
reqs, err := ParseDeleteRequests(in)
if err != nil {
return err
}
for _, req := range reqs {
if perr := a.handleDeleteOne(ctx, req); perr != nil {
fmt.Fprintf(os.Stderr, "backint: delete %q failed: %v\n", req.EBID, perr)
_ = WriteError(out, req.EBID)
continue
}
_ = WriteDeleted(out, req.EBID)
}
return nil
}
func (a *Agent) handleDeleteOne(ctx context.Context, req DeleteRequest) error {
entry, err := a.catalog.Get(req.EBID)
if err != nil {
return fmt.Errorf("catalog get: %w", err)
}
if entry == nil {
return fmt.Errorf("ebid not found: %s", req.EBID)
}
if err := a.provider.Delete(ctx, entry.ObjectKey); err != nil {
// 允许后端返回"不存在"类错误后继续删除目录条目,避免孤立条目
fmt.Fprintf(os.Stderr, "backint: storage delete warning for %s: %v\n", entry.ObjectKey, err)
}
return a.catalog.Delete(req.EBID)
}
// 辅助函数
func (a *Agent) objectKeyFor(ebid string) string {
base := ebid + ".bin"
if a.cfg.KeyPrefix == "" {
return base
}
return path.Join(a.cfg.KeyPrefix, base)
}
// openBackupSource 打开 HANA 提供的数据源。
//
// 对于 #PIPE 模式HANA 写入命名管道Agent 读取。管道是顺序流size 未知 (-1)。
// 对于文件模式HANA 已在指定路径写好完整文件。
func openBackupSource(req BackupRequest) (io.ReadCloser, int64, error) {
if req.IsPipe {
f, err := os.OpenFile(req.Path, os.O_RDONLY, 0)
if err != nil {
return nil, 0, fmt.Errorf("open pipe: %w", err)
}
return f, -1, nil
}
f, err := os.Open(req.Path)
if err != nil {
return nil, 0, fmt.Errorf("open file: %w", err)
}
info, err := f.Stat()
if err != nil {
_ = f.Close()
return nil, 0, fmt.Errorf("stat: %w", err)
}
return f, info.Size(), nil
}
// openRestoreTarget 打开 HANA 指定的恢复目标(管道或文件)。
func openRestoreTarget(req RestoreRequest) (io.WriteCloser, error) {
if req.IsPipe {
return os.OpenFile(req.Path, os.O_WRONLY, 0)
}
return os.Create(req.Path)
}
// generateEBID 生成 Backint 外部备份 ID。
// 格式backupx-<timestamp>-<16 hex chars>
func generateEBID() string {
var buf [8]byte
if _, err := rand.Read(buf[:]); err != nil {
// fallback用纳秒时间戳作为熵
now := time.Now().UnixNano()
for i := 0; i < 8; i++ {
buf[i] = byte(now >> (i * 8))
}
}
return fmt.Sprintf("backupx-%d-%s", time.Now().Unix(), hex.EncodeToString(buf[:]))
}
func boolStr(b bool) string {
if b {
return "true"
}
return "false"
}
// buildStorageRegistry 构造与主程序一致的 storage registry。
//
// Backint Agent 作为独立 CLI 进程运行,不依赖 BackupX HTTP 服务,
// 因此这里直接引用 storage/rclone 包注册所有后端。
func buildStorageRegistry() *storage.Registry {
registry := storage.NewRegistry(
storageRclone.NewLocalDiskFactory(),
storageRclone.NewS3Factory(),
storageRclone.NewWebDAVFactory(),
storageRclone.NewGoogleDriveFactory(),
storageRclone.NewAliyunOSSFactory(),
storageRclone.NewTencentCOSFactory(),
storageRclone.NewQiniuKodoFactory(),
storageRclone.NewFTPFactory(),
storageRclone.NewRcloneFactory(),
)
storageRclone.RegisterAllBackends(registry)
return registry
}

View File

@@ -0,0 +1,217 @@
package backint
import (
"bytes"
"context"
"os"
"path/filepath"
"strings"
"testing"
"backupx/server/internal/storage"
storageRclone "backupx/server/internal/storage/rclone"
)
// newTestAgent 构造一个使用本地磁盘后端的 Agent便于集成测试。
func newTestAgent(t *testing.T, compress bool) (*Agent, string) {
t.Helper()
dir := t.TempDir()
storageDir := filepath.Join(dir, "storage")
if err := os.MkdirAll(storageDir, 0755); err != nil {
t.Fatal(err)
}
registry := storage.NewRegistry(storageRclone.NewLocalDiskFactory())
provider, err := registry.Create(context.Background(), "local_disk", map[string]any{
"basePath": storageDir,
})
if err != nil {
t.Fatalf("create provider: %v", err)
}
cat, err := OpenCatalog(filepath.Join(dir, "catalog.db"))
if err != nil {
t.Fatal(err)
}
agent := &Agent{
cfg: &Config{StorageType: "local_disk", KeyPrefix: "backint", Compress: compress, CatalogDB: filepath.Join(dir, "catalog.db")},
provider: provider,
catalog: cat,
}
t.Cleanup(func() { _ = agent.Close() })
return agent, dir
}
func TestAgent_BackupAndRestore_File(t *testing.T) {
agent, dir := newTestAgent(t, false)
ctx := context.Background()
// 准备源文件
src := filepath.Join(dir, "src.bak")
content := []byte("hello backint world")
if err := os.WriteFile(src, content, 0644); err != nil {
t.Fatal(err)
}
// BACKUP
inPath := filepath.Join(dir, "backup.in")
outPath := filepath.Join(dir, "backup.out")
if err := os.WriteFile(inPath, []byte(src+"\n"), 0644); err != nil {
t.Fatal(err)
}
if err := agent.Run(ctx, FunctionBackup, inPath, outPath); err != nil {
t.Fatalf("backup: %v", err)
}
out, _ := os.ReadFile(outPath)
if !bytes.HasPrefix(out, []byte("#SAVED ")) {
t.Fatalf("expected #SAVED, got: %s", out)
}
// 提取 EBID#SAVED <ebid> "<path>"
parts := strings.Fields(string(out))
if len(parts) < 3 {
t.Fatalf("malformed output: %s", out)
}
ebid := parts[1]
// RESTORE
restoreDst := filepath.Join(dir, "restored.bak")
inPath2 := filepath.Join(dir, "restore.in")
outPath2 := filepath.Join(dir, "restore.out")
if err := os.WriteFile(inPath2, []byte(ebid+" \""+restoreDst+"\"\n"), 0644); err != nil {
t.Fatal(err)
}
if err := agent.Run(ctx, FunctionRestore, inPath2, outPath2); err != nil {
t.Fatalf("restore: %v", err)
}
got, err := os.ReadFile(restoreDst)
if err != nil {
t.Fatalf("read restored: %v", err)
}
if !bytes.Equal(got, content) {
t.Errorf("restored content mismatch: %q vs %q", got, content)
}
}
func TestAgent_BackupWithCompression(t *testing.T) {
agent, dir := newTestAgent(t, true)
ctx := context.Background()
src := filepath.Join(dir, "src.bak")
content := bytes.Repeat([]byte("ABCDEFGH"), 1024)
if err := os.WriteFile(src, content, 0644); err != nil {
t.Fatal(err)
}
inPath := filepath.Join(dir, "backup.in")
outPath := filepath.Join(dir, "backup.out")
_ = os.WriteFile(inPath, []byte(src+"\n"), 0644)
if err := agent.Run(ctx, FunctionBackup, inPath, outPath); err != nil {
t.Fatalf("backup: %v", err)
}
parts := strings.Fields(string(mustRead(t, outPath)))
ebid := parts[1]
// 验证 catalog 记录的对象键以 .gz 结尾
entry, _ := agent.catalog.Get(ebid)
if entry == nil || !strings.HasSuffix(entry.ObjectKey, ".gz") {
t.Fatalf("expected .gz suffix: %+v", entry)
}
// RESTORE 应能解压回原始内容
dst := filepath.Join(dir, "restored.bak")
in2 := filepath.Join(dir, "restore.in")
out2 := filepath.Join(dir, "restore.out")
_ = os.WriteFile(in2, []byte(ebid+" \""+dst+"\"\n"), 0644)
if err := agent.Run(ctx, FunctionRestore, in2, out2); err != nil {
t.Fatalf("restore: %v", err)
}
got := mustRead(t, dst)
if !bytes.Equal(got, content) {
t.Errorf("decompressed content mismatch (len=%d vs %d)", len(got), len(content))
}
}
func TestAgent_Inquire(t *testing.T) {
agent, dir := newTestAgent(t, false)
ctx := context.Background()
// 注入两条目录记录
_ = agent.catalog.Put(CatalogEntry{EBID: "bid-a", ObjectKey: "k/a"})
_ = agent.catalog.Put(CatalogEntry{EBID: "bid-b", ObjectKey: "k/b"})
// INQUIRE #NULL 应列出全部
in := filepath.Join(dir, "inq.in")
out := filepath.Join(dir, "inq.out")
_ = os.WriteFile(in, []byte("#NULL\n"), 0644)
if err := agent.Run(ctx, FunctionInquire, in, out); err != nil {
t.Fatalf("inquire: %v", err)
}
text := string(mustRead(t, out))
if !strings.Contains(text, "bid-a") || !strings.Contains(text, "bid-b") {
t.Errorf("expected both ebids, got: %s", text)
}
// INQUIRE 不存在的 ebid → #NOTFOUND
_ = os.WriteFile(in, []byte("bid-missing\n"), 0644)
if err := agent.Run(ctx, FunctionInquire, in, out); err != nil {
t.Fatalf("inquire missing: %v", err)
}
text = string(mustRead(t, out))
if !strings.Contains(text, "#NOTFOUND") {
t.Errorf("expected #NOTFOUND, got: %s", text)
}
}
func TestAgent_Delete(t *testing.T) {
agent, dir := newTestAgent(t, false)
ctx := context.Background()
// 先做一次 BACKUP
src := filepath.Join(dir, "src.bak")
_ = os.WriteFile(src, []byte("data"), 0644)
inPath := filepath.Join(dir, "b.in")
outPath := filepath.Join(dir, "b.out")
_ = os.WriteFile(inPath, []byte(src+"\n"), 0644)
if err := agent.Run(ctx, FunctionBackup, inPath, outPath); err != nil {
t.Fatal(err)
}
ebid := strings.Fields(string(mustRead(t, outPath)))[1]
// DELETE
delIn := filepath.Join(dir, "d.in")
delOut := filepath.Join(dir, "d.out")
_ = os.WriteFile(delIn, []byte(ebid+"\n"), 0644)
if err := agent.Run(ctx, FunctionDelete, delIn, delOut); err != nil {
t.Fatalf("delete: %v", err)
}
if !strings.Contains(string(mustRead(t, delOut)), "#DELETED") {
t.Errorf("expected #DELETED, got: %s", mustRead(t, delOut))
}
// catalog 条目应已删除
if entry, _ := agent.catalog.Get(ebid); entry != nil {
t.Errorf("catalog entry should be removed, got: %+v", entry)
}
}
func TestAgent_RestoreUnknownEBID(t *testing.T) {
agent, dir := newTestAgent(t, false)
ctx := context.Background()
in := filepath.Join(dir, "r.in")
out := filepath.Join(dir, "r.out")
_ = os.WriteFile(in, []byte("bid-unknown \""+filepath.Join(dir, "dst")+"\"\n"), 0644)
if err := agent.Run(ctx, FunctionRestore, in, out); err != nil {
t.Fatalf("run: %v", err)
}
if !strings.Contains(string(mustRead(t, out)), "#ERROR") {
t.Errorf("expected #ERROR for unknown ebid, got: %s", mustRead(t, out))
}
}
func mustRead(t *testing.T, path string) []byte {
t.Helper()
b, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read %s: %v", path, err)
}
return b
}

View File

@@ -0,0 +1,102 @@
package backint
import (
"fmt"
"time"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
"gorm.io/gorm/clause"
gormlogger "gorm.io/gorm/logger"
)
// CatalogEntry 是 Backint 目录条目,建立 BID (备份 ID) 与对象键的映射。
//
// BID 是 Backint Agent 返回给 SAP HANA 的唯一标识HANA 后续用它作为 RESTORE/DELETE
// 的句柄。Agent 用 catalog 查询该 BID 对应的实际存储对象键。
type CatalogEntry struct {
ID uint `gorm:"primaryKey"`
EBID string `gorm:"column:ebid;uniqueIndex;size:128;not null"`
ObjectKey string `gorm:"column:object_key;size:512;not null"`
SourcePath string `gorm:"column:source_path;size:1024"`
Size int64 `gorm:"column:size"`
CreatedAt time.Time `gorm:"column:created_at"`
}
// TableName 指定表名,避免 GORM 自动复数化。
func (CatalogEntry) TableName() string { return "backint_catalog" }
// Catalog 是本地 Backint 目录SQLite 后端)。
type Catalog struct {
db *gorm.DB
}
// OpenCatalog 打开或创建 catalog 数据库。
func OpenCatalog(dbPath string) (*Catalog, error) {
db, err := gorm.Open(sqlite.Open(dbPath), &gorm.Config{
Logger: gormlogger.Default.LogMode(gormlogger.Silent),
})
if err != nil {
return nil, fmt.Errorf("open catalog: %w", err)
}
if err := db.AutoMigrate(&CatalogEntry{}); err != nil {
return nil, fmt.Errorf("migrate catalog: %w", err)
}
return &Catalog{db: db}, nil
}
// Close 关闭底层连接。
func (c *Catalog) Close() error {
if c.db == nil {
return nil
}
sqlDB, err := c.db.DB()
if err != nil {
return err
}
return sqlDB.Close()
}
// Put 插入或更新一条记录。
func (c *Catalog) Put(entry CatalogEntry) error {
if entry.EBID == "" {
return fmt.Errorf("ebid is required")
}
if entry.CreatedAt.IsZero() {
entry.CreatedAt = time.Now().UTC()
}
// UpsertEBID 冲突时更新 object_key/size/source_path
return c.db.Clauses(clause.OnConflict{
Columns: []clause.Column{{Name: "ebid"}},
DoUpdates: clause.AssignmentColumns([]string{
"object_key", "source_path", "size", "created_at",
}),
}).Create(&entry).Error
}
// Get 通过 EBID 查询条目。未找到返回 (nil, nil)。
func (c *Catalog) Get(ebid string) (*CatalogEntry, error) {
var entry CatalogEntry
err := c.db.Where("ebid = ?", ebid).First(&entry).Error
if err == gorm.ErrRecordNotFound {
return nil, nil
}
if err != nil {
return nil, err
}
return &entry, nil
}
// Delete 删除一条记录。
func (c *Catalog) Delete(ebid string) error {
return c.db.Where("ebid = ?", ebid).Delete(&CatalogEntry{}).Error
}
// List 列出全部条目。
func (c *Catalog) List() ([]CatalogEntry, error) {
var entries []CatalogEntry
if err := c.db.Order("created_at DESC").Find(&entries).Error; err != nil {
return nil, err
}
return entries, nil
}

View File

@@ -0,0 +1,74 @@
package backint
import (
"path/filepath"
"testing"
)
func TestCatalog_CRUD(t *testing.T) {
dir := t.TempDir()
cat, err := OpenCatalog(filepath.Join(dir, "test.db"))
if err != nil {
t.Fatalf("open: %v", err)
}
defer cat.Close()
if err := cat.Put(CatalogEntry{EBID: "bid-1", ObjectKey: "k/1.bin", SourcePath: "/tmp/a", Size: 100}); err != nil {
t.Fatalf("put: %v", err)
}
if err := cat.Put(CatalogEntry{EBID: "bid-2", ObjectKey: "k/2.bin", Size: 200}); err != nil {
t.Fatalf("put: %v", err)
}
got, err := cat.Get("bid-1")
if err != nil || got == nil {
t.Fatalf("get: %v %v", got, err)
}
if got.ObjectKey != "k/1.bin" || got.Size != 100 {
t.Errorf("mismatch: %+v", got)
}
// 不存在的条目
missing, err := cat.Get("bid-999")
if err != nil {
t.Fatalf("get missing: %v", err)
}
if missing != nil {
t.Errorf("expected nil, got %+v", missing)
}
// List
all, err := cat.List()
if err != nil || len(all) != 2 {
t.Fatalf("list: %v %d", err, len(all))
}
// Delete
if err := cat.Delete("bid-1"); err != nil {
t.Fatalf("delete: %v", err)
}
got, _ = cat.Get("bid-1")
if got != nil {
t.Errorf("bid-1 should be deleted")
}
}
func TestCatalog_UpsertSameEBID(t *testing.T) {
dir := t.TempDir()
cat, err := OpenCatalog(filepath.Join(dir, "test.db"))
if err != nil {
t.Fatalf("open: %v", err)
}
defer cat.Close()
if err := cat.Put(CatalogEntry{EBID: "bid-x", ObjectKey: "v1"}); err != nil {
t.Fatal(err)
}
if err := cat.Put(CatalogEntry{EBID: "bid-x", ObjectKey: "v2"}); err != nil {
t.Fatal(err)
}
got, _ := cat.Get("bid-x")
if got == nil || got.ObjectKey != "v2" {
t.Errorf("upsert failed: %+v", got)
}
}

View File

@@ -0,0 +1,140 @@
package backint
import (
"bufio"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"strconv"
"strings"
)
// Config 是 Backint Agent 的运行时配置。
//
// SAP HANA 通过 -p <paramfile> 传入一个参数文件。BackupX Backint Agent 复用 SAP
// 的"#KEY = VALUE"风格(兼容原生 backint 参数文件习惯),不支持 section。
//
// 必填字段:
// - STORAGE_TYPE存储类型s3/webdav/local_disk/...,与 BackupX storage registry 一致)
// - STORAGE_CONFIG_JSON存储配置 JSON 文件路径(或直接 STORAGE_CONFIG = <json>
//
// 可选字段:
// - PARALLEL_FACTOR并行度默认 1
// - COMPRESS是否 gzip 压缩true/false默认 false
// - LOG_FILE日志文件路径默认 stderr
// - CATALOG_DB本地目录数据库路径默认 ./backint_catalog.db
// - KEY_PREFIX对象键前缀默认空最终对象键 = <prefix>/<ebid>
type Config struct {
StorageType string
StorageConfigJSON string // 存储配置 JSON 文件路径
StorageConfigRaw []byte // 也支持直接内联STORAGE_CONFIG
StorageConfig map[string]any // 解析后的存储配置
ParallelFactor int
Compress bool
LogFile string
CatalogDB string
KeyPrefix string
}
// LoadConfigFile 从文件加载配置。
func LoadConfigFile(path string) (*Config, error) {
f, err := os.Open(path)
if err != nil {
return nil, fmt.Errorf("open backint config: %w", err)
}
defer f.Close()
return ParseConfig(f)
}
// ParseConfig 从 reader 解析配置。
func ParseConfig(r io.Reader) (*Config, error) {
cfg := &Config{ParallelFactor: 1}
scanner := bufio.NewScanner(r)
scanner.Buffer(make([]byte, 64*1024), 4*1024*1024)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if line == "" || strings.HasPrefix(line, ";") {
continue
}
// 兼容可选的 "#" 前缀SAP 约定)
line = strings.TrimPrefix(line, "#")
eq := strings.Index(line, "=")
if eq < 0 {
continue
}
key := strings.TrimSpace(line[:eq])
value := strings.TrimSpace(line[eq+1:])
if len(value) >= 2 && value[0] == '"' && value[len(value)-1] == '"' {
value = value[1 : len(value)-1]
}
switch strings.ToUpper(key) {
case "STORAGE_TYPE":
cfg.StorageType = value
case "STORAGE_CONFIG_JSON":
cfg.StorageConfigJSON = value
case "STORAGE_CONFIG":
cfg.StorageConfigRaw = []byte(value)
case "PARALLEL_FACTOR":
n, err := strconv.Atoi(value)
if err != nil || n <= 0 {
return nil, fmt.Errorf("invalid PARALLEL_FACTOR: %q", value)
}
cfg.ParallelFactor = n
case "COMPRESS":
cfg.Compress = parseBool(value)
case "LOG_FILE":
cfg.LogFile = value
case "CATALOG_DB":
cfg.CatalogDB = value
case "KEY_PREFIX":
cfg.KeyPrefix = strings.Trim(value, "/")
}
}
if err := scanner.Err(); err != nil {
return nil, err
}
if err := cfg.finalize(); err != nil {
return nil, err
}
return cfg, nil
}
func (c *Config) finalize() error {
if c.StorageType == "" {
return errors.New("STORAGE_TYPE is required")
}
if c.CatalogDB == "" {
c.CatalogDB = "./backint_catalog.db"
}
// 加载存储配置 JSON
var raw []byte
switch {
case c.StorageConfigJSON != "":
data, err := os.ReadFile(c.StorageConfigJSON)
if err != nil {
return fmt.Errorf("read STORAGE_CONFIG_JSON: %w", err)
}
raw = data
case len(c.StorageConfigRaw) > 0:
raw = c.StorageConfigRaw
default:
return errors.New("STORAGE_CONFIG_JSON or STORAGE_CONFIG is required")
}
var m map[string]any
if err := json.Unmarshal(raw, &m); err != nil {
return fmt.Errorf("parse storage config JSON: %w", err)
}
c.StorageConfig = m
return nil
}
func parseBool(v string) bool {
switch strings.ToLower(strings.TrimSpace(v)) {
case "1", "true", "yes", "on":
return true
default:
return false
}
}

View File

@@ -0,0 +1,74 @@
package backint
import (
"os"
"path/filepath"
"strings"
"testing"
)
func TestParseConfig(t *testing.T) {
dir := t.TempDir()
storagePath := filepath.Join(dir, "storage.json")
if err := os.WriteFile(storagePath, []byte(`{"basePath":"/tmp/backup"}`), 0644); err != nil {
t.Fatal(err)
}
input := `
; 注释
#STORAGE_TYPE = local_disk
#STORAGE_CONFIG_JSON = ` + storagePath + `
#PARALLEL_FACTOR = 4
#COMPRESS = true
#KEY_PREFIX = /hana/backups/
#CATALOG_DB = ` + filepath.Join(dir, "catalog.db") + `
`
cfg, err := ParseConfig(strings.NewReader(input))
if err != nil {
t.Fatalf("parse: %v", err)
}
if cfg.StorageType != "local_disk" {
t.Errorf("StorageType: %q", cfg.StorageType)
}
if cfg.ParallelFactor != 4 {
t.Errorf("ParallelFactor: %d", cfg.ParallelFactor)
}
if !cfg.Compress {
t.Errorf("Compress should be true")
}
if cfg.KeyPrefix != "hana/backups" {
t.Errorf("KeyPrefix should be trimmed: %q", cfg.KeyPrefix)
}
if cfg.StorageConfig["basePath"] != "/tmp/backup" {
t.Errorf("StorageConfig mismatch: %+v", cfg.StorageConfig)
}
}
func TestParseConfig_MissingStorageType(t *testing.T) {
input := `PARALLEL_FACTOR = 1`
if _, err := ParseConfig(strings.NewReader(input)); err == nil {
t.Fatal("expected error for missing STORAGE_TYPE")
}
}
func TestParseConfig_InlineStorageConfig(t *testing.T) {
input := `STORAGE_TYPE = local_disk
STORAGE_CONFIG = {"basePath":"/x"}
`
cfg, err := ParseConfig(strings.NewReader(input))
if err != nil {
t.Fatalf("parse: %v", err)
}
if cfg.StorageConfig["basePath"] != "/x" {
t.Errorf("inline config not parsed: %+v", cfg.StorageConfig)
}
}
func TestParseConfig_InvalidParallel(t *testing.T) {
input := `STORAGE_TYPE = local_disk
STORAGE_CONFIG = {}
PARALLEL_FACTOR = oops
`
if _, err := ParseConfig(strings.NewReader(input)); err == nil {
t.Fatal("expected error for invalid PARALLEL_FACTOR")
}
}

View File

@@ -0,0 +1,267 @@
// Package backint 实现 SAP HANA Backint 协议代理。
//
// Backint 协议是 SAP HANA 与第三方备份工具之间的管道/文件协议。
// SAP HANA 通过 CLI 调用 Backint Agent传入参数文件、输入文件、输出文件
// Agent 根据输入文件中的 #PIPE / #EBID / #NULL 指令读取/写入数据,
// 并在输出文件中返回 #SAVED / #RESTORED / #BACKUP / #NOTFOUND / #DELETED / #ERROR。
//
// 支持的功能BACKUP / RESTORE / INQUIRE / DELETE
// 参考规范SAP HANA Backint Interface for Backup Tools (OSS 1642148)
package backint
import (
"bufio"
"errors"
"fmt"
"io"
"strings"
)
// Function 代表 Backint 操作类型,对应 CLI 的 -f 参数。
type Function string
const (
FunctionBackup Function = "backup"
FunctionRestore Function = "restore"
FunctionInquire Function = "inquire"
FunctionDelete Function = "delete"
)
// BackupRequest 是 BACKUP 操作的单条请求。
//
// 两种形态:
// - Pipe: #PIPE <path> (HANA 通过命名管道传输数据)
// - File: "<path>" (HANA 指向一个已完成的临时文件)
type BackupRequest struct {
IsPipe bool
Path string
}
// RestoreRequest 是 RESTORE 操作的单条请求。
//
// 形态:#PIPE <ebid> "<path>" 或 <ebid> "<path>"
type RestoreRequest struct {
IsPipe bool
EBID string // 之前 BACKUP 返回的备份 ID
Path string
}
// InquireRequest 是 INQUIRE 操作的单条请求。
//
// 形态:
// - #NULL (列出所有备份)
// - "<ebid>" (查询指定 ID 是否存在)
// - #EBID "<ebid>" (带前缀的变体)
type InquireRequest struct {
All bool
EBID string
}
// DeleteRequest 是 DELETE 操作的单条请求。
//
// 形态:<ebid> 或 #EBID <ebid>
type DeleteRequest struct {
EBID string
}
// ParseBackupRequests 解析 BACKUP 输入文件。
func ParseBackupRequests(r io.Reader) ([]BackupRequest, error) {
var items []BackupRequest
scanner := bufio.NewScanner(r)
scanner.Buffer(make([]byte, 64*1024), 4*1024*1024)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if line == "" {
continue
}
if strings.HasPrefix(line, "#PIPE") {
path := strings.TrimSpace(strings.TrimPrefix(line, "#PIPE"))
if path == "" {
return nil, fmt.Errorf("invalid #PIPE line: %q", line)
}
items = append(items, BackupRequest{IsPipe: true, Path: trimQuotes(path)})
continue
}
items = append(items, BackupRequest{IsPipe: false, Path: trimQuotes(line)})
}
if err := scanner.Err(); err != nil {
return nil, err
}
return items, nil
}
// ParseRestoreRequests 解析 RESTORE 输入文件。
func ParseRestoreRequests(r io.Reader) ([]RestoreRequest, error) {
var items []RestoreRequest
scanner := bufio.NewScanner(r)
scanner.Buffer(make([]byte, 64*1024), 4*1024*1024)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if line == "" {
continue
}
isPipe := false
if strings.HasPrefix(line, "#PIPE") {
isPipe = true
line = strings.TrimSpace(strings.TrimPrefix(line, "#PIPE"))
}
if strings.HasPrefix(line, "#EBID") {
line = strings.TrimSpace(strings.TrimPrefix(line, "#EBID"))
}
ebid, rest := splitFirstField(line)
if ebid == "" || rest == "" {
return nil, fmt.Errorf("invalid restore line: %q", line)
}
items = append(items, RestoreRequest{
IsPipe: isPipe,
EBID: trimQuotes(ebid),
Path: trimQuotes(rest),
})
}
if err := scanner.Err(); err != nil {
return nil, err
}
return items, nil
}
// ParseInquireRequests 解析 INQUIRE 输入文件。
func ParseInquireRequests(r io.Reader) ([]InquireRequest, error) {
var items []InquireRequest
scanner := bufio.NewScanner(r)
scanner.Buffer(make([]byte, 64*1024), 4*1024*1024)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if line == "" {
continue
}
if line == "#NULL" {
items = append(items, InquireRequest{All: true})
continue
}
if strings.HasPrefix(line, "#EBID") {
line = strings.TrimSpace(strings.TrimPrefix(line, "#EBID"))
}
items = append(items, InquireRequest{EBID: trimQuotes(line)})
}
if err := scanner.Err(); err != nil {
return nil, err
}
return items, nil
}
// ParseDeleteRequests 解析 DELETE 输入文件。
func ParseDeleteRequests(r io.Reader) ([]DeleteRequest, error) {
var items []DeleteRequest
scanner := bufio.NewScanner(r)
scanner.Buffer(make([]byte, 64*1024), 4*1024*1024)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if line == "" {
continue
}
if strings.HasPrefix(line, "#EBID") {
line = strings.TrimSpace(strings.TrimPrefix(line, "#EBID"))
}
ebid := trimQuotes(strings.TrimSpace(line))
if ebid == "" {
return nil, fmt.Errorf("invalid delete line: %q", line)
}
items = append(items, DeleteRequest{EBID: ebid})
}
if err := scanner.Err(); err != nil {
return nil, err
}
return items, nil
}
// 输出写入辅助
// WriteSaved 写入一条 BACKUP 成功响应:#SAVED <ebid> "<path>"
func WriteSaved(w io.Writer, ebid, path string) error {
_, err := fmt.Fprintf(w, "#SAVED %s %s\n", ebid, quote(path))
return err
}
// WriteRestored 写入一条 RESTORE 成功响应:#RESTORED "<ebid>" "<path>"
func WriteRestored(w io.Writer, ebid, path string) error {
_, err := fmt.Fprintf(w, "#RESTORED %s %s\n", quote(ebid), quote(path))
return err
}
// WriteBackup 写入一条 INQUIRE 命中响应:#BACKUP "<ebid>"
func WriteBackup(w io.Writer, ebid string) error {
_, err := fmt.Fprintf(w, "#BACKUP %s\n", quote(ebid))
return err
}
// WriteNotFound 写入一条 INQUIRE/RESTORE 未命中响应:#NOTFOUND "<path-or-ebid>"
func WriteNotFound(w io.Writer, identifier string) error {
_, err := fmt.Fprintf(w, "#NOTFOUND %s\n", quote(identifier))
return err
}
// WriteDeleted 写入一条 DELETE 成功响应:#DELETED "<ebid>"
func WriteDeleted(w io.Writer, ebid string) error {
_, err := fmt.Fprintf(w, "#DELETED %s\n", quote(ebid))
return err
}
// WriteError 写入一条错误响应:#ERROR "<path-or-ebid>"
//
// SAP HANA 会将 #ERROR 视为本条请求失败,但不会终止整个批次。
// 在 stderr 输出错误详情便于排查。
func WriteError(w io.Writer, identifier string) error {
_, err := fmt.Fprintf(w, "#ERROR %s\n", quote(identifier))
return err
}
// 内部工具函数
func trimQuotes(s string) string {
s = strings.TrimSpace(s)
if len(s) >= 2 && s[0] == '"' && s[len(s)-1] == '"' {
return s[1 : len(s)-1]
}
return s
}
func quote(s string) string {
return `"` + strings.ReplaceAll(s, `"`, `\"`) + `"`
}
// splitFirstField 把一行拆分为 "第一个字段" 和 "剩余部分"。
// 支持带引号的字段:`"abc def" "path"` → `abc def` / `"path"`。
func splitFirstField(line string) (first, rest string) {
line = strings.TrimSpace(line)
if line == "" {
return "", ""
}
if line[0] == '"' {
idx := strings.Index(line[1:], `"`)
if idx < 0 {
return line, ""
}
return line[1 : idx+1], strings.TrimSpace(line[idx+2:])
}
idx := strings.IndexAny(line, " \t")
if idx < 0 {
return line, ""
}
return line[:idx], strings.TrimSpace(line[idx+1:])
}
// ParseFunction 将 CLI 的 -f 参数字符串规范化为 Function。
func ParseFunction(s string) (Function, error) {
switch strings.ToLower(strings.TrimSpace(s)) {
case "backup":
return FunctionBackup, nil
case "restore":
return FunctionRestore, nil
case "inquire":
return FunctionInquire, nil
case "delete":
return FunctionDelete, nil
default:
return "", errors.New("unsupported backint function: " + s)
}
}

View File

@@ -0,0 +1,142 @@
package backint
import (
"bytes"
"strings"
"testing"
)
func TestParseBackupRequests(t *testing.T) {
input := `#PIPE /tmp/pipe1
#PIPE "/tmp/pipe two"
/tmp/file.bak
"/tmp/file two.bak"
`
reqs, err := ParseBackupRequests(strings.NewReader(input))
if err != nil {
t.Fatalf("parse: %v", err)
}
if len(reqs) != 4 {
t.Fatalf("expected 4 requests, got %d", len(reqs))
}
if !reqs[0].IsPipe || reqs[0].Path != "/tmp/pipe1" {
t.Errorf("req[0] mismatch: %+v", reqs[0])
}
if !reqs[1].IsPipe || reqs[1].Path != "/tmp/pipe two" {
t.Errorf("req[1] mismatch: %+v", reqs[1])
}
if reqs[2].IsPipe || reqs[2].Path != "/tmp/file.bak" {
t.Errorf("req[2] mismatch: %+v", reqs[2])
}
if reqs[3].Path != "/tmp/file two.bak" {
t.Errorf("req[3] mismatch: %+v", reqs[3])
}
}
func TestParseRestoreRequests(t *testing.T) {
input := `#PIPE backupx-123 "/tmp/pipe1"
#EBID "backupx-456" "/tmp/file.bak"
backupx-789 /tmp/plain.bak
`
reqs, err := ParseRestoreRequests(strings.NewReader(input))
if err != nil {
t.Fatalf("parse: %v", err)
}
if len(reqs) != 3 {
t.Fatalf("expected 3, got %d", len(reqs))
}
if !reqs[0].IsPipe || reqs[0].EBID != "backupx-123" || reqs[0].Path != "/tmp/pipe1" {
t.Errorf("req[0] mismatch: %+v", reqs[0])
}
if reqs[1].IsPipe || reqs[1].EBID != "backupx-456" {
t.Errorf("req[1] mismatch: %+v", reqs[1])
}
if reqs[2].EBID != "backupx-789" || reqs[2].Path != "/tmp/plain.bak" {
t.Errorf("req[2] mismatch: %+v", reqs[2])
}
}
func TestParseInquireRequests(t *testing.T) {
input := "#NULL\nbackupx-abc\n#EBID \"backupx-xyz\"\n"
reqs, err := ParseInquireRequests(strings.NewReader(input))
if err != nil {
t.Fatalf("parse: %v", err)
}
if len(reqs) != 3 {
t.Fatalf("expected 3, got %d", len(reqs))
}
if !reqs[0].All {
t.Errorf("req[0] should be All")
}
if reqs[1].EBID != "backupx-abc" {
t.Errorf("req[1] mismatch: %+v", reqs[1])
}
if reqs[2].EBID != "backupx-xyz" {
t.Errorf("req[2] mismatch: %+v", reqs[2])
}
}
func TestParseDeleteRequests(t *testing.T) {
input := "backupx-aaa\n#EBID \"backupx-bbb\"\n"
reqs, err := ParseDeleteRequests(strings.NewReader(input))
if err != nil {
t.Fatalf("parse: %v", err)
}
if len(reqs) != 2 || reqs[0].EBID != "backupx-aaa" || reqs[1].EBID != "backupx-bbb" {
t.Fatalf("unexpected: %+v", reqs)
}
}
func TestWriteResponses(t *testing.T) {
var buf bytes.Buffer
_ = WriteSaved(&buf, "backupx-1", "/tmp/x")
_ = WriteRestored(&buf, "backupx-2", "/tmp/y")
_ = WriteBackup(&buf, "backupx-3")
_ = WriteNotFound(&buf, "backupx-4")
_ = WriteDeleted(&buf, "backupx-5")
_ = WriteError(&buf, "/tmp/z")
want := "#SAVED backupx-1 \"/tmp/x\"\n" +
"#RESTORED \"backupx-2\" \"/tmp/y\"\n" +
"#BACKUP \"backupx-3\"\n" +
"#NOTFOUND \"backupx-4\"\n" +
"#DELETED \"backupx-5\"\n" +
"#ERROR \"/tmp/z\"\n"
if buf.String() != want {
t.Errorf("output mismatch:\n got: %q\nwant: %q", buf.String(), want)
}
}
func TestParseFunction(t *testing.T) {
cases := map[string]Function{
"backup": FunctionBackup,
"BACKUP": FunctionBackup,
"restore": FunctionRestore,
"inquire": FunctionInquire,
"delete": FunctionDelete,
}
for s, want := range cases {
got, err := ParseFunction(s)
if err != nil || got != want {
t.Errorf("ParseFunction(%q) = %v, %v; want %v", s, got, err, want)
}
}
if _, err := ParseFunction("bogus"); err == nil {
t.Errorf("expected error for bogus function")
}
}
func TestSplitFirstField(t *testing.T) {
cases := []struct{ in, first, rest string }{
{`abc def`, "abc", "def"},
{`"abc def" ghi`, "abc def", "ghi"},
{`"a b" "c d"`, "a b", `"c d"`},
{`lone`, "lone", ""},
{``, "", ""},
}
for _, c := range cases {
f, r := splitFirstField(c.in)
if f != c.first || r != c.rest {
t.Errorf("splitFirstField(%q) = (%q, %q); want (%q, %q)", c.in, f, r, c.first, c.rest)
}
}
}

View File

@@ -11,6 +11,28 @@ import (
"backupx/server/internal/storage"
)
// collectDirPrefixes 从待删除的记录中提取唯一的父目录前缀。
func collectDirPrefixes(records []model.BackupRecord) []string {
seen := make(map[string]struct{})
var prefixes []string
for _, record := range records {
path := strings.TrimSpace(record.StoragePath)
if path == "" {
continue
}
idx := strings.LastIndex(path, "/")
if idx <= 0 {
continue
}
dir := path[:idx]
if _, ok := seen[dir]; !ok {
seen[dir] = struct{}{}
prefixes = append(prefixes, dir)
}
}
return prefixes
}
type CleanupResult struct {
DeletedRecords int
DeletedObjects int
@@ -54,6 +76,17 @@ func (s *Service) Cleanup(ctx context.Context, task *model.BackupTask, provider
}
result.DeletedRecords++
}
// 清理空目录:收集被删除文件的父目录,尝试移除空目录
if dirCleaner, ok := provider.(storage.StorageDirCleaner); ok && result.DeletedObjects > 0 {
prefixes := collectDirPrefixes(candidates)
for _, prefix := range prefixes {
if err := dirCleaner.RemoveEmptyDirs(ctx, prefix); err != nil {
result.Warnings = append(result.Warnings, fmt.Sprintf("cleanup empty dirs for %s: %v", prefix, err))
}
}
}
return result, nil
}

View File

@@ -36,6 +36,9 @@ func (r *fakeRecordRepository) Delete(_ context.Context, id uint) error {
func (r *fakeRecordRepository) ListRecent(context.Context, int) ([]model.BackupRecord, error) {
return nil, nil
}
func (r *fakeRecordRepository) ListByTask(_ context.Context, _ uint) ([]model.BackupRecord, error) {
return r.records, nil
}
func (r *fakeRecordRepository) ListSuccessfulByTask(_ context.Context, _ uint) ([]model.BackupRecord, error) {
return r.records, nil
}

View File

@@ -35,6 +35,12 @@ func (r *SAPHANARunner) Type() string {
// Run executes a SAP HANA data-level backup using hdbsql + BACKUP DATA USING FILE.
// The backup files are written to a temporary directory, then packaged into a tar
// archive as the artifact for BackupX to compress/encrypt/upload.
//
// 支持以下增强(通过 task.Database 字段配置):
// - BackupLevel: full / incremental / differential
// - BackupType: data / log
// - BackupChannels: 并行通道数(>1 时生成多路径 SQL
// - MaxRetries: hdbsql 执行失败的重试次数
func (r *SAPHANARunner) Run(ctx context.Context, task TaskSpec, writer LogWriter) (*RunResult, error) {
if _, err := r.executor.LookPath("hdbsql"); err != nil {
return nil, fmt.Errorf("未找到 hdbsql 命令 (请确保服务器已安装 SAP HANA Client)")
@@ -68,32 +74,46 @@ func (r *SAPHANARunner) Run(ctx context.Context, task TaskSpec, writer LogWriter
port = 30015
}
writer.WriteLine(fmt.Sprintf("连接到 SAP HANA: %s:%d", task.Database.Host, port))
backupLevel := normalizeBackupLevel(task.Database.BackupLevel)
backupType := normalizeBackupType(task.Database.BackupType)
channels := task.Database.BackupChannels
if channels < 1 {
channels = 1
}
maxRetries := task.Database.MaxRetries
if maxRetries < 1 {
maxRetries = 3
}
instance := task.Database.InstanceNumber
if strings.TrimSpace(instance) == "" {
instance = hanaInstanceNumber(port)
}
writer.WriteLine(fmt.Sprintf("连接到 SAP HANA: %s:%d (实例 %s)", task.Database.Host, port, instance))
writer.WriteLine(fmt.Sprintf("备份数据库: %s", tenantDB))
writer.WriteLine(fmt.Sprintf("备份配置: 类型=%s, 级别=%s, 通道数=%d, 最大重试=%d", backupType, backupLevel, channels, maxRetries))
// Build backup prefix — HANA will create files like <prefix>_databackup_<N>_1.
timestamp := startedAt.UTC().Format("20060102_150405")
backupPrefix := filepath.Join(backupDir, fmt.Sprintf("hana_%s_%s", strings.ToLower(tenantDB), timestamp))
// Build `BACKUP DATA USING FILE` SQL.
backupSQL := fmt.Sprintf(`BACKUP DATA USING FILE ('%s')`, backupPrefix)
if strings.ToUpper(tenantDB) != "SYSTEMDB" {
backupSQL = fmt.Sprintf(`BACKUP DATA FOR %s USING FILE ('%s')`, tenantDB, backupPrefix)
prefixes, err := buildBackupPrefixes(backupDir, tenantDB, timestamp, channels)
if err != nil {
return nil, err
}
// Build SQL based on backup type and level.
backupSQL := buildBackupSQL(tenantDB, prefixes, backupType, backupLevel)
writer.WriteLine(fmt.Sprintf("生成 SQL: %s", backupSQL))
// Construct hdbsql connection arguments.
args := buildHdbsqlArgs(task.Database.Host, port, task.Database.User, task.Database.Password, tenantDB, backupSQL)
stderrWriter := newLogLineWriter(writer, "hdbsql")
writer.WriteLine("开始执行 SAP HANA BACKUP DATA USING FILE")
writer.WriteLine("开始执行 SAP HANA 备份命令")
if err := r.executor.Run(ctx, "hdbsql", args, CommandOptions{
Stderr: stderrWriter,
}); err != nil {
return nil, fmt.Errorf("run hdbsql BACKUP DATA: %w: %s", err, stderrWriter.collected())
if err := r.runHdbsqlWithRetry(ctx, "hdbsql", args, maxRetries, writer); err != nil {
return nil, fmt.Errorf("run hdbsql backup: %w", err)
}
writer.WriteLine("SAP HANA BACKUP DATA 命令执行完成,开始打包备份文件")
writer.WriteLine("SAP HANA 备份命令执行完成,开始打包备份文件")
// Package all generated backup files into a tar archive.
if err := packageBackupFiles(backupDir, artifactPath, writer); err != nil {
@@ -166,12 +186,12 @@ func (r *SAPHANARunner) Restore(ctx context.Context, task TaskSpec, artifactPath
args := buildHdbsqlArgs(task.Database.Host, port, task.Database.User, task.Database.Password, tenantDB, recoverSQL)
stderrWriter := newLogLineWriter(writer, "hdbsql")
if err := r.executor.Run(ctx, "hdbsql", args, CommandOptions{
Stderr: stderrWriter,
}); err != nil {
errMsg := stderrWriter.collected()
return fmt.Errorf("run hdbsql RECOVER DATA: %w: %s", err, strings.TrimSpace(errMsg))
maxRetries := task.Database.MaxRetries
if maxRetries < 1 {
maxRetries = 3
}
if err := r.runHdbsqlWithRetry(ctx, "hdbsql", args, maxRetries, writer); err != nil {
return fmt.Errorf("run hdbsql RECOVER DATA: %w", err)
}
writer.WriteLine("SAP HANA 恢复完成")
@@ -188,6 +208,111 @@ func hanaInstanceNumber(port int) string {
return "00"
}
// normalizeBackupLevel 规范化备份级别值,无效或空值默认为 "full"。
func normalizeBackupLevel(level string) string {
switch strings.ToLower(strings.TrimSpace(level)) {
case "incremental":
return "incremental"
case "differential":
return "differential"
default:
return "full"
}
}
// normalizeBackupType 规范化备份类型,无效或空值默认为 "data"。
func normalizeBackupType(t string) string {
switch strings.ToLower(strings.TrimSpace(t)) {
case "log":
return "log"
default:
return "data"
}
}
// buildBackupPrefixes 为每个并行通道生成独立子目录和路径前缀。
// 当 channels=1 时返回单个直接位于 backupDir 下的前缀;
// 当 channels>1 时为每个通道创建 chan_N/ 子目录。
func buildBackupPrefixes(backupDir, tenantDB, timestamp string, channels int) ([]string, error) {
tenantLower := strings.ToLower(tenantDB)
if channels <= 1 {
return []string{filepath.Join(backupDir, fmt.Sprintf("hana_%s_%s", tenantLower, timestamp))}, nil
}
prefixes := make([]string, 0, channels)
for i := 0; i < channels; i++ {
chanDir := filepath.Join(backupDir, fmt.Sprintf("chan_%d", i))
if err := os.MkdirAll(chanDir, 0o755); err != nil {
return nil, fmt.Errorf("create channel %d dir: %w", i, err)
}
prefixes = append(prefixes, filepath.Join(chanDir, fmt.Sprintf("hana_%s_%s", tenantLower, timestamp)))
}
return prefixes, nil
}
// buildBackupSQL 根据备份类型和级别构建 SAP HANA BACKUP SQL 语句。
//
// 支持的语法:
//
// 全量数据备份: BACKUP DATA [FOR <tenant>] USING FILE ('p1' [, 'p2', ...])
// 增量数据备份: BACKUP DATA [FOR <tenant>] INCREMENTAL USING FILE ('...')
// 差异数据备份: BACKUP DATA [FOR <tenant>] DIFFERENTIAL USING FILE ('...')
// 日志备份: BACKUP LOG [FOR <tenant>] USING FILE ('...')
func buildBackupSQL(tenantDB string, prefixes []string, backupType, backupLevel string) string {
tenantClause := ""
if strings.ToUpper(tenantDB) != "SYSTEMDB" {
tenantClause = fmt.Sprintf(" FOR %s", tenantDB)
}
// 多路径以 'p1', 'p2', ... 拼接HANA 多通道并行语法)
quoted := make([]string, len(prefixes))
for i, p := range prefixes {
quoted[i] = fmt.Sprintf("'%s'", p)
}
pathClause := strings.Join(quoted, ", ")
if backupType == "log" {
// LOG 备份不支持 INCREMENTAL/DIFFERENTIAL 关键字
return fmt.Sprintf("BACKUP LOG%s USING FILE (%s)", tenantClause, pathClause)
}
levelClause := ""
switch backupLevel {
case "incremental":
levelClause = " INCREMENTAL"
case "differential":
levelClause = " DIFFERENTIAL"
}
return fmt.Sprintf("BACKUP DATA%s%s USING FILE (%s)", tenantClause, levelClause, pathClause)
}
// runHdbsqlWithRetry 执行 hdbsql 命令并在失败时按指数退避重试。
// 退避公式5s × attempt²并在 ctx 取消时立即返回。
func (r *SAPHANARunner) runHdbsqlWithRetry(ctx context.Context, name string, args []string, maxAttempts int, writer LogWriter) error {
if maxAttempts < 1 {
maxAttempts = 1
}
var lastErr error
for attempt := 1; attempt <= maxAttempts; attempt++ {
if attempt > 1 {
backoff := time.Duration(attempt*attempt) * 5 * time.Second
writer.WriteLine(fmt.Sprintf("hdbsql 第 %d 次重试(等待 %s", attempt, backoff))
select {
case <-ctx.Done():
return ctx.Err()
case <-time.After(backoff):
}
}
stderrWriter := newLogLineWriter(writer, "hdbsql")
err := r.executor.Run(ctx, name, args, CommandOptions{Stderr: stderrWriter})
if err == nil {
return nil
}
lastErr = fmt.Errorf("%w: %s", err, strings.TrimSpace(stderrWriter.collected()))
writer.WriteLine(fmt.Sprintf("hdbsql 执行失败(第 %d/%d 次): %v", attempt, maxAttempts, lastErr))
}
return lastErr
}
// buildHdbsqlArgs constructs the common hdbsql CLI arguments.
func buildHdbsqlArgs(host string, port int, user, password, database, sql string) []string {
return []string{

View File

@@ -7,6 +7,7 @@ import (
"path/filepath"
"strings"
"testing"
"time"
)
func TestSAPHANARunnerRun_BackupDataCommand(t *testing.T) {
@@ -273,6 +274,246 @@ func TestSAPHANARunnerRestore_TenantRecoverCommand(t *testing.T) {
}
}
func TestBuildBackupSQL_FullSystemDB(t *testing.T) {
sql := buildBackupSQL("SYSTEMDB", []string{"/tmp/p1"}, "data", "full")
if sql != "BACKUP DATA USING FILE ('/tmp/p1')" {
t.Fatalf("unexpected SQL: %s", sql)
}
}
func TestBuildBackupSQL_IncrementalTenant(t *testing.T) {
sql := buildBackupSQL("HDB", []string{"/tmp/p1"}, "data", "incremental")
expected := "BACKUP DATA FOR HDB INCREMENTAL USING FILE ('/tmp/p1')"
if sql != expected {
t.Fatalf("expected %q, got %q", expected, sql)
}
}
func TestBuildBackupSQL_DifferentialTenant(t *testing.T) {
sql := buildBackupSQL("HDB", []string{"/tmp/p1"}, "data", "differential")
expected := "BACKUP DATA FOR HDB DIFFERENTIAL USING FILE ('/tmp/p1')"
if sql != expected {
t.Fatalf("expected %q, got %q", expected, sql)
}
}
func TestBuildBackupSQL_LogBackup(t *testing.T) {
sql := buildBackupSQL("HDB", []string{"/tmp/log"}, "log", "full")
expected := "BACKUP LOG FOR HDB USING FILE ('/tmp/log')"
if sql != expected {
t.Fatalf("expected %q, got %q", expected, sql)
}
}
func TestBuildBackupSQL_ParallelChannels(t *testing.T) {
sql := buildBackupSQL("SYSTEMDB", []string{"/tmp/c0/p", "/tmp/c1/p", "/tmp/c2/p"}, "data", "full")
expected := "BACKUP DATA USING FILE ('/tmp/c0/p', '/tmp/c1/p', '/tmp/c2/p')"
if sql != expected {
t.Fatalf("expected %q, got %q", expected, sql)
}
}
func TestNormalizeBackupLevel(t *testing.T) {
cases := map[string]string{
"": "full",
"FULL": "full",
"incremental": "incremental",
"DIFFERENTIAL": "differential",
"unknown": "full",
}
for in, want := range cases {
if got := normalizeBackupLevel(in); got != want {
t.Errorf("normalizeBackupLevel(%q) = %q, want %q", in, got, want)
}
}
}
func TestNormalizeBackupType(t *testing.T) {
cases := map[string]string{
"": "data",
"DATA": "data",
"log": "log",
"LOG": "log",
}
for in, want := range cases {
if got := normalizeBackupType(in); got != want {
t.Errorf("normalizeBackupType(%q) = %q, want %q", in, got, want)
}
}
}
func TestSAPHANARunnerRun_IncrementalBackup(t *testing.T) {
var capturedSQL string
executor := &fakeCommandExecutor{
runFunc: func(name string, args []string, options CommandOptions) error {
capturedSQL = args[len(args)-1]
startIdx := strings.Index(capturedSQL, "('") + 2
endIdx := strings.Index(capturedSQL, "')")
if startIdx > 1 && endIdx > startIdx {
prefix := capturedSQL[startIdx:endIdx]
_ = os.MkdirAll(filepath.Dir(prefix), 0o755)
_ = os.WriteFile(prefix+"_databackup_0_1", []byte("incremental data"), 0o644)
}
return nil
},
}
runner := NewSAPHANARunner(executor)
result, err := runner.Run(context.Background(), TaskSpec{
Name: "hana-incremental",
Type: "saphana",
Database: DatabaseSpec{
Host: "10.0.0.1",
Port: 30015,
User: "SYSTEM",
Password: "secret",
Names: []string{"HDB"},
BackupLevel: "incremental",
},
}, NopLogWriter{})
if err != nil {
t.Fatalf("Run returned error: %v", err)
}
defer os.RemoveAll(result.TempDir)
if !strings.Contains(capturedSQL, "INCREMENTAL USING FILE") {
t.Fatalf("expected INCREMENTAL in SQL, got: %s", capturedSQL)
}
}
func TestSAPHANARunnerRun_LogBackup(t *testing.T) {
var capturedSQL string
executor := &fakeCommandExecutor{
runFunc: func(name string, args []string, options CommandOptions) error {
capturedSQL = args[len(args)-1]
startIdx := strings.Index(capturedSQL, "('") + 2
endIdx := strings.Index(capturedSQL, "')")
if startIdx > 1 && endIdx > startIdx {
prefix := capturedSQL[startIdx:endIdx]
_ = os.MkdirAll(filepath.Dir(prefix), 0o755)
_ = os.WriteFile(prefix+"_logbackup_0_1", []byte("log data"), 0o644)
}
return nil
},
}
runner := NewSAPHANARunner(executor)
result, err := runner.Run(context.Background(), TaskSpec{
Name: "hana-log",
Type: "saphana",
Database: DatabaseSpec{
Host: "10.0.0.1", Port: 30015, User: "SYSTEM", Password: "secret",
Names: []string{"HDB"},
BackupType: "log",
},
}, NopLogWriter{})
if err != nil {
t.Fatalf("Run returned error: %v", err)
}
defer os.RemoveAll(result.TempDir)
if !strings.Contains(capturedSQL, "BACKUP LOG FOR HDB USING FILE") {
t.Fatalf("expected log backup SQL, got: %s", capturedSQL)
}
}
func TestSAPHANARunnerRun_ParallelChannels(t *testing.T) {
var capturedSQL string
executor := &fakeCommandExecutor{
runFunc: func(name string, args []string, options CommandOptions) error {
capturedSQL = args[len(args)-1]
// 模拟为每个通道生成备份文件
parts := strings.Split(capturedSQL, "',")
for _, p := range parts {
p = strings.TrimSpace(p)
if idx := strings.Index(p, "'"); idx >= 0 {
prefix := p[idx+1:]
prefix = strings.TrimSuffix(prefix, "')")
prefix = strings.TrimSuffix(prefix, "'")
if prefix != "" {
_ = os.MkdirAll(filepath.Dir(prefix), 0o755)
_ = os.WriteFile(prefix+"_databackup_0_1", []byte("data"), 0o644)
}
}
}
return nil
},
}
runner := NewSAPHANARunner(executor)
result, err := runner.Run(context.Background(), TaskSpec{
Name: "hana-parallel",
Type: "saphana",
Database: DatabaseSpec{
Host: "10.0.0.1", Port: 30015, User: "SYSTEM", Password: "secret",
Names: []string{"SYSTEMDB"},
BackupChannels: 3,
},
}, NopLogWriter{})
if err != nil {
t.Fatalf("Run returned error: %v", err)
}
defer os.RemoveAll(result.TempDir)
// 应该包含 3 个路径
if strings.Count(capturedSQL, "'") != 6 { // 3 路径 × 2 引号
t.Fatalf("expected 3 channels (6 quotes), got SQL: %s", capturedSQL)
}
if !strings.Contains(capturedSQL, "chan_0") || !strings.Contains(capturedSQL, "chan_2") {
t.Fatalf("expected channel directories in SQL, got: %s", capturedSQL)
}
}
func TestSAPHANARunnerRun_RetryOnFailure(t *testing.T) {
attempts := 0
executor := &fakeCommandExecutor{
runFunc: func(name string, args []string, options CommandOptions) error {
attempts++
if attempts < 2 {
return errors.New("transient failure")
}
// 第二次成功,写入备份文件
sql := args[len(args)-1]
startIdx := strings.Index(sql, "('") + 2
endIdx := strings.Index(sql, "')")
if startIdx > 1 && endIdx > startIdx {
prefix := sql[startIdx:endIdx]
_ = os.MkdirAll(filepath.Dir(prefix), 0o755)
_ = os.WriteFile(prefix+"_databackup_0_1", []byte("data"), 0o644)
}
return nil
},
}
// 使用极短的重试周期版本(这里通过 fake context 机制无法快进时间,所以直接验证 attempts
// 设置 MaxRetries=2 以加快测试,不会真实等待 5s
runner := NewSAPHANARunner(executor)
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
result, err := runner.Run(ctx, TaskSpec{
Name: "hana-retry",
Type: "saphana",
Database: DatabaseSpec{
Host: "10.0.0.1", Port: 30015, User: "SYSTEM", Password: "secret",
Names: []string{"SYSTEMDB"},
MaxRetries: 2,
},
}, NopLogWriter{})
if err != nil {
t.Fatalf("Run returned error after retry: %v", err)
}
defer os.RemoveAll(result.TempDir)
if attempts != 2 {
t.Fatalf("expected 2 attempts, got %d", attempts)
}
}
func TestHanaInstanceNumber(t *testing.T) {
tests := []struct {
port int

View File

@@ -12,6 +12,12 @@ type DatabaseSpec struct {
Password string
Names []string
Path string
// SAP HANA 特有字段(其他类型忽略)
InstanceNumber string // 实例编号(从端口推断或手动指定)
BackupLevel string // "full"(默认) / "incremental" / "differential"
BackupType string // "data"(默认) / "log"
BackupChannels int // 并行通道数(默认 1
MaxRetries int // 最大重试次数(默认 3
}
type TaskSpec struct {

View File

@@ -23,7 +23,7 @@ func Open(cfg config.DatabaseConfig, logger *zap.Logger) (*gorm.DB, error) {
return nil, fmt.Errorf("open sqlite: %w", err)
}
if err := db.AutoMigrate(&model.User{}, &model.SystemConfig{}, &model.StorageTarget{}, &model.OAuthSession{}, &model.BackupTask{}, &model.BackupRecord{}, &model.Notification{}, &model.Node{}, &model.BackupTaskStorageTarget{}, &model.AuditLog{}); err != nil {
if err := db.AutoMigrate(&model.User{}, &model.SystemConfig{}, &model.StorageTarget{}, &model.OAuthSession{}, &model.BackupTask{}, &model.BackupRecord{}, &model.Notification{}, &model.Node{}, &model.BackupTaskStorageTarget{}, &model.AuditLog{}, &model.AgentCommand{}); err != nil {
return nil, fmt.Errorf("migrate schema: %w", err)
}

View File

@@ -0,0 +1,156 @@
package http
import (
stdhttp "net/http"
"strconv"
"strings"
"backupx/server/internal/service"
"backupx/server/pkg/response"
"github.com/gin-gonic/gin"
)
// AgentHandler 实现 Agent 调用 Master 的 HTTP API。
// 全部端点通过 X-Agent-Token 头做节点认证,不使用 JWT。
type AgentHandler struct {
agentService *service.AgentService
nodeService *service.NodeService
}
func NewAgentHandler(agentService *service.AgentService, nodeService *service.NodeService) *AgentHandler {
return &AgentHandler{agentService: agentService, nodeService: nodeService}
}
// extractToken 从请求头或 JSON body 中提取 Agent Token。
func extractToken(c *gin.Context) string {
if t := strings.TrimSpace(c.GetHeader("X-Agent-Token")); t != "" {
return t
}
// Authorization: Bearer <token>
if auth := c.GetHeader("Authorization"); strings.HasPrefix(auth, "Bearer ") {
return strings.TrimSpace(strings.TrimPrefix(auth, "Bearer "))
}
return ""
}
// Heartbeat 扩展原有 heartbeat除上报状态外返回节点 ID 给 Agent 做后续调用。
func (h *AgentHandler) Heartbeat(c *gin.Context) {
var input struct {
Token string `json:"token"`
Hostname string `json:"hostname"`
IPAddress string `json:"ipAddress"`
AgentVersion string `json:"agentVersion"`
OS string `json:"os"`
Arch string `json:"arch"`
}
_ = c.ShouldBindJSON(&input)
// token 优先走 body向后兼容否则从 header 读
token := input.Token
if token == "" {
token = extractToken(c)
}
if token == "" {
c.JSON(stdhttp.StatusBadRequest, gin.H{"code": "INVALID_INPUT", "message": "missing token"})
return
}
if err := h.nodeService.Heartbeat(c.Request.Context(), token, input.Hostname, input.IPAddress, input.AgentVersion, input.OS, input.Arch); err != nil {
response.Error(c, err)
return
}
// 返回节点元信息给 Agentnode_id 用于后续 API 路径)
node, err := h.agentService.AuthenticatedNode(c.Request.Context(), token)
if err != nil {
response.Error(c, err)
return
}
response.Success(c, gin.H{
"status": "ok",
"nodeId": node.ID,
"name": node.Name,
})
}
// Poll Agent 长轮询获取下一条待执行命令。
// 无命令时返回 {command: null}。
func (h *AgentHandler) Poll(c *gin.Context) {
node, err := h.agentService.AuthenticatedNode(c.Request.Context(), extractToken(c))
if err != nil {
response.Error(c, err)
return
}
cmd, err := h.agentService.PollCommand(c.Request.Context(), node)
if err != nil {
response.Error(c, err)
return
}
response.Success(c, gin.H{"command": cmd})
}
// SubmitCommandResult Agent 上报命令执行结果。
func (h *AgentHandler) SubmitCommandResult(c *gin.Context) {
node, err := h.agentService.AuthenticatedNode(c.Request.Context(), extractToken(c))
if err != nil {
response.Error(c, err)
return
}
id, err := strconv.ParseUint(c.Param("id"), 10, 32)
if err != nil {
response.Error(c, err)
return
}
var input service.AgentCommandResult
if err := c.ShouldBindJSON(&input); err != nil {
c.JSON(stdhttp.StatusBadRequest, gin.H{"code": "INVALID_INPUT", "message": err.Error()})
return
}
if err := h.agentService.SubmitCommandResult(c.Request.Context(), node, uint(id), input); err != nil {
response.Error(c, err)
return
}
response.Success(c, gin.H{"status": "ok"})
}
// GetTaskSpec Agent 拉取任务规格(含解密后的存储配置)。
func (h *AgentHandler) GetTaskSpec(c *gin.Context) {
node, err := h.agentService.AuthenticatedNode(c.Request.Context(), extractToken(c))
if err != nil {
response.Error(c, err)
return
}
id, err := strconv.ParseUint(c.Param("id"), 10, 32)
if err != nil {
response.Error(c, err)
return
}
spec, err := h.agentService.GetTaskSpec(c.Request.Context(), node, uint(id))
if err != nil {
response.Error(c, err)
return
}
response.Success(c, spec)
}
// UpdateRecord Agent 更新备份记录(进度/完成状态/日志)。
func (h *AgentHandler) UpdateRecord(c *gin.Context) {
node, err := h.agentService.AuthenticatedNode(c.Request.Context(), extractToken(c))
if err != nil {
response.Error(c, err)
return
}
id, err := strconv.ParseUint(c.Param("id"), 10, 32)
if err != nil {
response.Error(c, err)
return
}
var input service.AgentRecordUpdate
if err := c.ShouldBindJSON(&input); err != nil {
c.JSON(stdhttp.StatusBadRequest, gin.H{"code": "INVALID_INPUT", "message": err.Error()})
return
}
if err := h.agentService.UpdateRecord(c.Request.Context(), node, uint(id), input); err != nil {
response.Error(c, err)
return
}
response.Success(c, gin.H{"status": "ok"})
}

View File

@@ -130,7 +130,8 @@ func (h *BackupRecordHandler) Restore(c *gin.Context) {
response.Error(c, err)
return
}
recordAudit(c, h.auditService, "backup_record", "restore", "backup_record", fmt.Sprintf("%d", id), "", fmt.Sprintf("恢复备份记录 #%d", id))
recordAudit(c, h.auditService, "backup_record", "restore", "backup_record", fmt.Sprintf("%d", id), "",
fmt.Sprintf("恢复备份记录 (ID: %d)", id))
response.Success(c, gin.H{"restored": true})
}
@@ -143,7 +144,8 @@ func (h *BackupRecordHandler) Delete(c *gin.Context) {
response.Error(c, err)
return
}
recordAudit(c, h.auditService, "backup_record", "delete", "backup_record", fmt.Sprintf("%d", id), "", fmt.Sprintf("删除备份记录 #%d", id))
recordAudit(c, h.auditService, "backup_record", "delete", "backup_record", fmt.Sprintf("%d", id), "",
fmt.Sprintf("删除备份记录 (ID: %d)", id))
response.Success(c, gin.H{"deleted": true})
}

View File

@@ -14,6 +14,19 @@ type BackupTaskHandler struct {
auditService *service.AuditService
}
// describeTaskInput 提取审计日志中通用的调度和存储目标描述。
func describeTaskInput(input service.BackupTaskUpsertInput) (cronDesc string, storageCount int) {
cronDesc = "仅手动执行"
if input.CronExpr != "" {
cronDesc = input.CronExpr
}
storageCount = len(input.StorageTargetIDs)
if storageCount == 0 && input.StorageTargetID > 0 {
storageCount = 1
}
return
}
func NewBackupTaskHandler(taskService *service.BackupTaskService, auditService *service.AuditService) *BackupTaskHandler {
return &BackupTaskHandler{service: taskService, auditService: auditService}
}
@@ -51,7 +64,9 @@ func (h *BackupTaskHandler) Create(c *gin.Context) {
response.Error(c, err)
return
}
recordAudit(c, h.auditService, "backup_task", "create", "backup_task", fmt.Sprintf("%d", item.ID), item.Name, fmt.Sprintf("类型: %s", input.Type))
cronDesc, storageCount := describeTaskInput(input)
recordAudit(c, h.auditService, "backup_task", "create", "backup_task", fmt.Sprintf("%d", item.ID), item.Name,
fmt.Sprintf("创建备份任务「%s」类型: %s, 调度: %s, 存储: %d 个目标", item.Name, input.Type, cronDesc, storageCount))
response.Success(c, item)
}
@@ -70,7 +85,9 @@ func (h *BackupTaskHandler) Update(c *gin.Context) {
response.Error(c, err)
return
}
recordAudit(c, h.auditService, "backup_task", "update", "backup_task", fmt.Sprintf("%d", item.ID), item.Name, fmt.Sprintf("类型: %s, Cron: %s", input.Type, input.CronExpr))
updCronDesc, updStorageCount := describeTaskInput(input)
recordAudit(c, h.auditService, "backup_task", "update", "backup_task", fmt.Sprintf("%d", item.ID), item.Name,
fmt.Sprintf("更新备份任务「%s」类型: %s, 调度: %s, 存储: %d 个目标", item.Name, input.Type, updCronDesc, updStorageCount))
response.Success(c, item)
}
@@ -79,11 +96,13 @@ func (h *BackupTaskHandler) Delete(c *gin.Context) {
if !ok {
return
}
if err := h.service.Delete(c.Request.Context(), id); err != nil {
result, err := h.service.Delete(c.Request.Context(), id)
if err != nil {
response.Error(c, err)
return
}
recordAudit(c, h.auditService, "backup_task", "delete", "backup_task", fmt.Sprintf("%d", id), "", fmt.Sprintf("删除备份任务 #%d", id))
recordAudit(c, h.auditService, "backup_task", "delete", "backup_task", fmt.Sprintf("%d", id), result.TaskName,
fmt.Sprintf("删除备份任务「%s」(ID: %d),关联记录 %d 条,已清理远端文件 %d 个", result.TaskName, id, result.RecordCount, result.CleanedFiles))
response.Success(c, gin.H{"deleted": true})
}
@@ -112,9 +131,12 @@ func (h *BackupTaskHandler) Toggle(c *gin.Context) {
return
}
action := "enable"
actionLabel := "启用"
if !enabled {
action = "disable"
actionLabel = "停用"
}
recordAudit(c, h.auditService, "backup_task", action, "backup_task", fmt.Sprintf("%d", id), item.Name, fmt.Sprintf("%s 备份任务", action))
recordAudit(c, h.auditService, "backup_task", action, "backup_task", fmt.Sprintf("%d", id), item.Name,
fmt.Sprintf("%s备份任务「%s」", actionLabel, item.Name))
response.Success(c, item)
}

View File

@@ -1,6 +1,7 @@
package http
import (
"fmt"
stdhttp "net/http"
"strconv"
@@ -10,11 +11,12 @@ import (
)
type NodeHandler struct {
service *service.NodeService
service *service.NodeService
auditService *service.AuditService
}
func NewNodeHandler(service *service.NodeService) *NodeHandler {
return &NodeHandler{service: service}
func NewNodeHandler(service *service.NodeService, auditService *service.AuditService) *NodeHandler {
return &NodeHandler{service: service, auditService: auditService}
}
func (h *NodeHandler) List(c *gin.Context) {
@@ -51,6 +53,8 @@ func (h *NodeHandler) Create(c *gin.Context) {
response.Error(c, err)
return
}
recordAudit(c, h.auditService, "node", "create", "node", "", input.Name,
fmt.Sprintf("创建远程节点「%s」", input.Name))
response.Success(c, gin.H{"token": token})
}
@@ -64,6 +68,8 @@ func (h *NodeHandler) Delete(c *gin.Context) {
response.Error(c, err)
return
}
recordAudit(c, h.auditService, "node", "delete", "node", fmt.Sprintf("%d", id), "",
fmt.Sprintf("删除节点 (ID: %d)", id))
response.Success(c, nil)
}
@@ -82,18 +88,41 @@ func (h *NodeHandler) ListDirectory(c *gin.Context) {
response.Success(c, entries)
}
func (h *NodeHandler) Update(c *gin.Context) {
id, err := strconv.ParseUint(c.Param("id"), 10, 32)
if err != nil {
response.Error(c, err)
return
}
var input service.NodeUpdateInput
if err := c.ShouldBindJSON(&input); err != nil {
c.JSON(stdhttp.StatusBadRequest, gin.H{"code": "INVALID_INPUT", "message": err.Error()})
return
}
item, err := h.service.Update(c.Request.Context(), uint(id), input)
if err != nil {
response.Error(c, err)
return
}
recordAudit(c, h.auditService, "node", "update", "node", fmt.Sprintf("%d", id), item.Name,
fmt.Sprintf("更新节点「%s」(ID: %d)", item.Name, id))
response.Success(c, item)
}
func (h *NodeHandler) Heartbeat(c *gin.Context) {
var input struct {
Token string `json:"token" binding:"required"`
Hostname string `json:"hostname"`
IPAddress string `json:"ipAddress"`
AgentVersion string `json:"agentVersion"`
OS string `json:"os"`
Arch string `json:"arch"`
}
if err := c.ShouldBindJSON(&input); err != nil {
c.JSON(stdhttp.StatusBadRequest, gin.H{"code": "INVALID_INPUT", "message": err.Error()})
return
}
if err := h.service.Heartbeat(c.Request.Context(), input.Token, input.Hostname, input.IPAddress, input.AgentVersion); err != nil {
if err := h.service.Heartbeat(c.Request.Context(), input.Token, input.Hostname, input.IPAddress, input.AgentVersion, input.OS, input.Arch); err != nil {
response.Error(c, err)
return
}

View File

@@ -28,6 +28,7 @@ type RouterDependencies struct {
DashboardService *service.DashboardService
SettingsService *service.SettingsService
NodeService *service.NodeService
AgentService *service.AgentService
DatabaseDiscoveryService *service.DatabaseDiscoveryService
AuditService *service.AuditService
JWTManager *security.JWTManager
@@ -69,7 +70,6 @@ func NewRouter(deps RouterDependencies) *gin.Engine {
system.Use(AuthMiddleware(deps.JWTManager))
system.GET("/info", systemHandler.Info)
system.GET("/update-check", systemHandler.CheckUpdate)
system.POST("/update-apply", systemHandler.ApplyUpdate)
storageTargets := api.Group("/storage-targets")
storageTargets.Use(AuthMiddleware(deps.JWTManager))
@@ -141,17 +141,29 @@ func NewRouter(deps RouterDependencies) *gin.Engine {
database.POST("/discover", databaseHandler.Discover)
}
nodeHandler := NewNodeHandler(deps.NodeService)
nodeHandler := NewNodeHandler(deps.NodeService, deps.AuditService)
nodes := api.Group("/nodes")
nodes.Use(AuthMiddleware(deps.JWTManager))
nodes.GET("", nodeHandler.List)
nodes.GET("/:id", nodeHandler.Get)
nodes.POST("", nodeHandler.Create)
nodes.PUT("/:id", nodeHandler.Update)
nodes.DELETE("/:id", nodeHandler.Delete)
nodes.GET("/:id/fs/list", nodeHandler.ListDirectory)
// Agent heartbeat (public, token-authenticated)
api.POST("/agent/heartbeat", nodeHandler.Heartbeat)
// Agent APItoken 认证,无需 JWT
if deps.AgentService != nil {
agentHandler := NewAgentHandler(deps.AgentService, deps.NodeService)
agent := api.Group("/agent")
agent.POST("/heartbeat", agentHandler.Heartbeat)
agent.POST("/commands/poll", agentHandler.Poll)
agent.POST("/commands/:id/result", agentHandler.SubmitCommandResult)
agent.GET("/tasks/:id", agentHandler.GetTaskSpec)
agent.POST("/records/:id", agentHandler.UpdateRecord)
} else {
// 未启用 Agent 服务时,保留原有 heartbeat 端点以兼容
api.POST("/agent/heartbeat", nodeHandler.Heartbeat)
}
}
engine.NoRoute(func(c *gin.Context) {

View File

@@ -65,7 +65,8 @@ func (h *StorageTargetHandler) Create(c *gin.Context) {
response.Error(c, err)
return
}
recordAudit(c, h.auditService, "storage_target", "create", "storage_target", fmt.Sprintf("%d", item.ID), item.Name, fmt.Sprintf("类型: %s", input.Type))
recordAudit(c, h.auditService, "storage_target", "create", "storage_target", fmt.Sprintf("%d", item.ID), item.Name,
fmt.Sprintf("创建存储目标「%s」类型: %s", item.Name, input.Type))
response.Success(c, item)
}
@@ -84,7 +85,8 @@ func (h *StorageTargetHandler) Update(c *gin.Context) {
response.Error(c, err)
return
}
recordAudit(c, h.auditService, "storage_target", "update", "storage_target", fmt.Sprintf("%d", item.ID), item.Name, fmt.Sprintf("类型: %s", input.Type))
recordAudit(c, h.auditService, "storage_target", "update", "storage_target", fmt.Sprintf("%d", item.ID), item.Name,
fmt.Sprintf("更新存储目标「%s」类型: %s", item.Name, input.Type))
response.Success(c, item)
}
@@ -97,7 +99,8 @@ func (h *StorageTargetHandler) Delete(c *gin.Context) {
response.Error(c, err)
return
}
recordAudit(c, h.auditService, "storage_target", "delete", "storage_target", fmt.Sprintf("%d", id), "", fmt.Sprintf("删除存储目标 #%d", id))
recordAudit(c, h.auditService, "storage_target", "delete", "storage_target", fmt.Sprintf("%d", id), "",
fmt.Sprintf("删除存储目标 (ID: %d)", id))
response.Success(c, gin.H{"deleted": true})
}

View File

@@ -18,15 +18,6 @@ func (h *SystemHandler) Info(c *gin.Context) {
response.Success(c, h.systemService.GetInfo(c.Request.Context()))
}
func (h *SystemHandler) ApplyUpdate(c *gin.Context) {
var input struct {
Version string `json:"version"`
}
_ = c.ShouldBindJSON(&input)
result := h.systemService.ApplyDockerUpdate(c.Request.Context(), input.Version)
response.Success(c, result)
}
func (h *SystemHandler) CheckUpdate(c *gin.Context) {
result, err := h.systemService.CheckUpdate(c.Request.Context())
if err != nil {

View File

@@ -0,0 +1,44 @@
package model
import "time"
// AgentCommand 状态常量
const (
AgentCommandStatusPending = "pending" // 待 Agent 拉取
AgentCommandStatusDispatched = "dispatched" // Agent 已领取,正在执行
AgentCommandStatusSucceeded = "succeeded" // 执行成功
AgentCommandStatusFailed = "failed" // 执行失败
AgentCommandStatusTimeout = "timeout" // 超时未完成
)
// AgentCommand 类型常量
const (
// AgentCommandTypeRunTask 运行指定备份任务
// Payload: {"taskId": 123, "recordId": 456}
AgentCommandTypeRunTask = "run_task"
// AgentCommandTypeListDir 远程列目录(用于文件备份时的目录浏览器)
// Payload: {"path": "/var/log"}
// Result: {"entries": [{"name":"...", "path":"...", "isDir":true, "size":0}]}
AgentCommandTypeListDir = "list_dir"
)
// AgentCommand 代表 Master 发给某个 Agent 节点的待执行命令。
// 使用简单的数据库队列实现Agent 通过 token 长轮询拉取本节点 pending 命令,
// 执行后回写状态与结果。Master 侧通过定时检查把超时的命令标记为 timeout。
type AgentCommand struct {
ID uint `gorm:"primaryKey" json:"id"`
NodeID uint `gorm:"column:node_id;index;not null" json:"nodeId"`
Type string `gorm:"size:32;index;not null" json:"type"`
Status string `gorm:"size:20;index;not null;default:'pending'" json:"status"`
Payload string `gorm:"type:text" json:"payload"` // JSON
Result string `gorm:"type:text" json:"result"` // JSON成功结果
ErrorMessage string `gorm:"column:error_message;type:text" json:"errorMessage"`
DispatchedAt *time.Time `gorm:"column:dispatched_at" json:"dispatchedAt,omitempty"`
CompletedAt *time.Time `gorm:"column:completed_at" json:"completedAt,omitempty"`
CreatedAt time.Time `json:"createdAt"`
UpdatedAt time.Time `json:"updatedAt"`
}
func (AgentCommand) TableName() string {
return "agent_commands"
}

View File

@@ -7,6 +7,7 @@ const (
BackupTaskTypeMySQL = "mysql"
BackupTaskTypeSQLite = "sqlite"
BackupTaskTypePostgreSQL = "postgresql"
BackupTaskTypeSAPHANA = "saphana"
)
const (
@@ -31,6 +32,8 @@ type BackupTask struct {
DBPasswordCiphertext string `gorm:"column:db_password_ciphertext;type:text" json:"-"`
DBName string `gorm:"column:db_name;size:255" json:"dbName"`
DBPath string `gorm:"column:db_path;size:500" json:"dbPath"`
// ExtraConfig 类型特有的扩展配置JSON如 SAP HANA 的 backupLevel / backupChannels 等
ExtraConfig string `gorm:"column:extra_config;type:text" json:"extraConfig"`
StorageTargetID uint `gorm:"column:storage_target_id;index;not null" json:"storageTargetId"` // deprecated: 保留兼容
StorageTarget StorageTarget `json:"storageTarget,omitempty"` // deprecated: 保留兼容
StorageTargets []StorageTarget `gorm:"many2many:backup_task_storage_targets" json:"storageTargets,omitempty"`

View File

@@ -0,0 +1,101 @@
package repository
import (
"context"
"errors"
"time"
"backupx/server/internal/model"
"gorm.io/gorm"
)
// AgentCommandRepository 维护 Agent 命令队列。
type AgentCommandRepository interface {
Create(ctx context.Context, cmd *model.AgentCommand) error
FindByID(ctx context.Context, id uint) (*model.AgentCommand, error)
// ClaimPending 以原子方式把该节点一条 pending 命令置为 dispatched
// 并返回领取到的命令。无命令时返回 (nil, nil)。
ClaimPending(ctx context.Context, nodeID uint) (*model.AgentCommand, error)
Update(ctx context.Context, cmd *model.AgentCommand) error
// MarkStaleTimeout 把 dispatched 状态但超时未完成的命令标记为 timeout。
MarkStaleTimeout(ctx context.Context, threshold time.Time) (int64, error)
}
type GormAgentCommandRepository struct {
db *gorm.DB
}
func NewAgentCommandRepository(db *gorm.DB) *GormAgentCommandRepository {
return &GormAgentCommandRepository{db: db}
}
func (r *GormAgentCommandRepository) Create(ctx context.Context, cmd *model.AgentCommand) error {
return r.db.WithContext(ctx).Create(cmd).Error
}
func (r *GormAgentCommandRepository) FindByID(ctx context.Context, id uint) (*model.AgentCommand, error) {
var item model.AgentCommand
if err := r.db.WithContext(ctx).First(&item, id).Error; err != nil {
if errors.Is(err, gorm.ErrRecordNotFound) {
return nil, nil
}
return nil, err
}
return &item, nil
}
// ClaimPending 使用 UPDATE...WHERE id=(SELECT...) 的两步方式实现原子领取。
// SQLite 不支持 SELECT FOR UPDATE这里用事务 + 乐观锁。
func (r *GormAgentCommandRepository) ClaimPending(ctx context.Context, nodeID uint) (*model.AgentCommand, error) {
var claimed *model.AgentCommand
err := r.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
var item model.AgentCommand
err := tx.Where("node_id = ? AND status = ?", nodeID, model.AgentCommandStatusPending).
Order("id asc").First(&item).Error
if err != nil {
if errors.Is(err, gorm.ErrRecordNotFound) {
return nil
}
return err
}
now := time.Now().UTC()
result := tx.Model(&model.AgentCommand{}).
Where("id = ? AND status = ?", item.ID, model.AgentCommandStatusPending).
Updates(map[string]any{
"status": model.AgentCommandStatusDispatched,
"dispatched_at": &now,
})
if result.Error != nil {
return result.Error
}
if result.RowsAffected == 0 {
// 被其它 worker 抢占,放弃
return nil
}
item.Status = model.AgentCommandStatusDispatched
item.DispatchedAt = &now
claimed = &item
return nil
})
if err != nil {
return nil, err
}
return claimed, nil
}
func (r *GormAgentCommandRepository) Update(ctx context.Context, cmd *model.AgentCommand) error {
return r.db.WithContext(ctx).Save(cmd).Error
}
func (r *GormAgentCommandRepository) MarkStaleTimeout(ctx context.Context, threshold time.Time) (int64, error) {
result := r.db.WithContext(ctx).Model(&model.AgentCommand{}).
Where("status = ? AND dispatched_at < ?", model.AgentCommandStatusDispatched, threshold).
Updates(map[string]any{
"status": model.AgentCommandStatusTimeout,
"error_message": "agent did not report result before timeout",
})
if result.Error != nil {
return 0, result.Error
}
return result.RowsAffected, nil
}

View File

@@ -0,0 +1,120 @@
package repository
import (
"context"
"testing"
"time"
"backupx/server/internal/model"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
gormlogger "gorm.io/gorm/logger"
)
func newTestDB(t *testing.T) *gorm.DB {
t.Helper()
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{Logger: gormlogger.Default.LogMode(gormlogger.Silent)})
if err != nil {
t.Fatalf("open: %v", err)
}
if err := db.AutoMigrate(&model.AgentCommand{}); err != nil {
t.Fatalf("migrate: %v", err)
}
return db
}
func TestAgentCommandRepository_ClaimPending(t *testing.T) {
db := newTestDB(t)
repo := NewAgentCommandRepository(db)
ctx := context.Background()
// 插入两条 pending 命令
c1 := &model.AgentCommand{NodeID: 5, Type: "run_task", Status: model.AgentCommandStatusPending, Payload: `{"taskId":1}`}
c2 := &model.AgentCommand{NodeID: 5, Type: "list_dir", Status: model.AgentCommandStatusPending, Payload: `{"path":"/"}`}
c3 := &model.AgentCommand{NodeID: 99, Type: "run_task", Status: model.AgentCommandStatusPending}
for _, c := range []*model.AgentCommand{c1, c2, c3} {
if err := repo.Create(ctx, c); err != nil {
t.Fatal(err)
}
}
// 第一次 Claim 应拿到 c1
claimed, err := repo.ClaimPending(ctx, 5)
if err != nil {
t.Fatalf("claim: %v", err)
}
if claimed == nil || claimed.ID != c1.ID || claimed.Status != model.AgentCommandStatusDispatched {
t.Fatalf("expected c1 dispatched: %+v", claimed)
}
// 第二次应拿到 c2
claimed2, err := repo.ClaimPending(ctx, 5)
if err != nil || claimed2 == nil || claimed2.ID != c2.ID {
t.Fatalf("expected c2: %+v %v", claimed2, err)
}
// 第三次无 pending返回 nil
claimed3, err := repo.ClaimPending(ctx, 5)
if err != nil || claimed3 != nil {
t.Fatalf("expected nil, got %+v", claimed3)
}
// 不同 node 的命令不应被抢到
other, err := repo.ClaimPending(ctx, 5)
if err != nil || other != nil {
t.Fatalf("expected nil: %+v", other)
}
}
func TestAgentCommandRepository_Update(t *testing.T) {
db := newTestDB(t)
repo := NewAgentCommandRepository(db)
ctx := context.Background()
cmd := &model.AgentCommand{NodeID: 1, Type: "run_task", Status: model.AgentCommandStatusPending}
_ = repo.Create(ctx, cmd)
cmd.Status = model.AgentCommandStatusSucceeded
cmd.Result = `{"ok":true}`
now := time.Now().UTC()
cmd.CompletedAt = &now
if err := repo.Update(ctx, cmd); err != nil {
t.Fatal(err)
}
got, err := repo.FindByID(ctx, cmd.ID)
if err != nil || got == nil {
t.Fatal(err)
}
if got.Status != model.AgentCommandStatusSucceeded || got.Result != `{"ok":true}` {
t.Errorf("mismatch: %+v", got)
}
}
func TestAgentCommandRepository_MarkStaleTimeout(t *testing.T) {
db := newTestDB(t)
repo := NewAgentCommandRepository(db)
ctx := context.Background()
old := time.Now().Add(-time.Hour)
recent := time.Now()
// 两条 dispatched一条旧、一条新
oldCmd := &model.AgentCommand{NodeID: 1, Type: "run_task", Status: model.AgentCommandStatusDispatched, DispatchedAt: &old}
newCmd := &model.AgentCommand{NodeID: 1, Type: "run_task", Status: model.AgentCommandStatusDispatched, DispatchedAt: &recent}
_ = repo.Create(ctx, oldCmd)
_ = repo.Create(ctx, newCmd)
n, err := repo.MarkStaleTimeout(ctx, time.Now().Add(-30*time.Minute))
if err != nil {
t.Fatal(err)
}
if n != 1 {
t.Errorf("expected 1 row, got %d", n)
}
oldGot, _ := repo.FindByID(ctx, oldCmd.ID)
newGot, _ := repo.FindByID(ctx, newCmd.ID)
if oldGot.Status != model.AgentCommandStatusTimeout {
t.Errorf("old should be timeout: %+v", oldGot)
}
if newGot.Status != model.AgentCommandStatusDispatched {
t.Errorf("new should stay dispatched: %+v", newGot)
}
}

View File

@@ -37,6 +37,7 @@ type BackupRecordRepository interface {
Update(context.Context, *model.BackupRecord) error
Delete(context.Context, uint) error
ListRecent(context.Context, int) ([]model.BackupRecord, error)
ListByTask(context.Context, uint) ([]model.BackupRecord, error)
ListSuccessfulByTask(context.Context, uint) ([]model.BackupRecord, error)
Count(context.Context) (int64, error)
CountSince(context.Context, time.Time) (int64, error)
@@ -115,6 +116,14 @@ func (r *GormBackupRecordRepository) ListRecent(ctx context.Context, limit int)
return items, nil
}
func (r *GormBackupRecordRepository) ListByTask(ctx context.Context, taskID uint) ([]model.BackupRecord, error) {
var items []model.BackupRecord
if err := r.db.WithContext(ctx).Where("task_id = ?", taskID).Order("id desc").Find(&items).Error; err != nil {
return nil, err
}
return items, nil
}
func (r *GormBackupRecordRepository) ListSuccessfulByTask(ctx context.Context, taskID uint) ([]model.BackupRecord, error) {
var items []model.BackupRecord
if err := r.db.WithContext(ctx).Where("task_id = ? AND status = ?", taskID, "success").Order("completed_at desc, id desc").Find(&items).Error; err != nil {

View File

@@ -21,6 +21,8 @@ type BackupTaskRepository interface {
Count(context.Context) (int64, error)
CountEnabled(context.Context) (int64, error)
CountByStorageTargetID(context.Context, uint) (int64, error)
CountByNodeID(context.Context, uint) (int64, error)
ListByNodeID(context.Context, uint) ([]model.BackupTask, error)
Create(context.Context, *model.BackupTask) error
Update(context.Context, *model.BackupTask) error
Delete(context.Context, uint) error
@@ -103,6 +105,24 @@ func (r *GormBackupTaskRepository) CountByStorageTargetID(ctx context.Context, s
return count, nil
}
// CountByNodeID 统计绑定到指定节点的任务数。用于删除节点前的引用检查。
func (r *GormBackupTaskRepository) CountByNodeID(ctx context.Context, nodeID uint) (int64, error) {
var count int64
if err := r.db.WithContext(ctx).Model(&model.BackupTask{}).Where("node_id = ?", nodeID).Count(&count).Error; err != nil {
return 0, err
}
return count, nil
}
// ListByNodeID 列出绑定到指定节点的任务。用于 Agent 拉取本节点待执行任务。
func (r *GormBackupTaskRepository) ListByNodeID(ctx context.Context, nodeID uint) ([]model.BackupTask, error) {
var items []model.BackupTask
if err := r.db.WithContext(ctx).Preload("StorageTarget").Preload("StorageTargets").Where("node_id = ?", nodeID).Order("id asc").Find(&items).Error; err != nil {
return nil, err
}
return items, nil
}
func (r *GormBackupTaskRepository) Create(ctx context.Context, item *model.BackupTask) error {
if err := r.db.WithContext(ctx).Create(item).Error; err != nil {
return err

View File

@@ -3,6 +3,7 @@ package repository
import (
"context"
"errors"
"time"
"backupx/server/internal/model"
"gorm.io/gorm"
@@ -16,6 +17,7 @@ type NodeRepository interface {
Create(context.Context, *model.Node) error
Update(context.Context, *model.Node) error
Delete(context.Context, uint) error
MarkStaleOffline(ctx context.Context, threshold time.Time) (int64, error)
}
type GormNodeRepository struct {
@@ -78,3 +80,16 @@ func (r *GormNodeRepository) Update(ctx context.Context, item *model.Node) error
func (r *GormNodeRepository) Delete(ctx context.Context, id uint) error {
return r.db.WithContext(ctx).Delete(&model.Node{}, id).Error
}
// MarkStaleOffline 把最近心跳早于 threshold 的在线远程节点标记为离线。
// 本机节点 (is_local=true) 不受影响,由主程序自己维护 online 状态。
// 返回受影响行数。
func (r *GormNodeRepository) MarkStaleOffline(ctx context.Context, threshold time.Time) (int64, error) {
result := r.db.WithContext(ctx).Model(&model.Node{}).
Where("is_local = ? AND status = ? AND last_seen < ?", false, model.NodeStatusOnline, threshold).
Update("status", model.NodeStatusOffline)
if result.Error != nil {
return 0, result.Error
}
return result.RowsAffected, nil
}

View File

@@ -31,6 +31,12 @@ func (r *fakeTaskRepository) CountEnabled(context.Context) (int64, error) { retu
func (r *fakeTaskRepository) CountByStorageTargetID(context.Context, uint) (int64, error) {
return 0, nil
}
func (r *fakeTaskRepository) CountByNodeID(context.Context, uint) (int64, error) {
return 0, nil
}
func (r *fakeTaskRepository) ListByNodeID(context.Context, uint) ([]model.BackupTask, error) {
return nil, nil
}
func (r *fakeTaskRepository) Create(context.Context, *model.BackupTask) error { return nil }
func (r *fakeTaskRepository) Update(context.Context, *model.BackupTask) error { return nil }
func (r *fakeTaskRepository) Delete(context.Context, uint) error { return nil }

View File

@@ -0,0 +1,348 @@
package service
import (
"context"
"encoding/json"
"errors"
"fmt"
"strings"
"time"
"backupx/server/internal/apperror"
"backupx/server/internal/model"
"backupx/server/internal/repository"
"backupx/server/internal/storage/codec"
)
// AgentService 实现 Master 端 Agent 协议,提供给远程 Agent 通过 HTTP 调用。
// 所有方法使用 Agent Token 进行节点认证,避免暴露 JWT 给 Agent。
type AgentService struct {
nodeRepo repository.NodeRepository
taskRepo repository.BackupTaskRepository
recordRepo repository.BackupRecordRepository
storageRepo repository.StorageTargetRepository
cmdRepo repository.AgentCommandRepository
cipher *codec.ConfigCipher
}
func NewAgentService(
nodeRepo repository.NodeRepository,
taskRepo repository.BackupTaskRepository,
recordRepo repository.BackupRecordRepository,
storageRepo repository.StorageTargetRepository,
cmdRepo repository.AgentCommandRepository,
cipher *codec.ConfigCipher,
) *AgentService {
return &AgentService{
nodeRepo: nodeRepo,
taskRepo: taskRepo,
recordRepo: recordRepo,
storageRepo: storageRepo,
cmdRepo: cmdRepo,
cipher: cipher,
}
}
// AuthenticatedNode 通过 token 解析并返回节点。失败返回 401。
func (s *AgentService) AuthenticatedNode(ctx context.Context, token string) (*model.Node, error) {
if strings.TrimSpace(token) == "" {
return nil, apperror.Unauthorized("NODE_INVALID_TOKEN", "缺少认证令牌", nil)
}
node, err := s.nodeRepo.FindByToken(ctx, token)
if err != nil {
return nil, err
}
if node == nil {
return nil, apperror.Unauthorized("NODE_INVALID_TOKEN", "无效的节点认证令牌", nil)
}
return node, nil
}
// AgentCommandPayload 给 Agent 返回的命令描述
type AgentCommandPayload struct {
ID uint `json:"id"`
Type string `json:"type"`
Payload json.RawMessage `json:"payload,omitempty"`
}
// PollCommand 为指定节点拉取一条 pending 命令;无命令时返回 (nil, nil)。
func (s *AgentService) PollCommand(ctx context.Context, node *model.Node) (*AgentCommandPayload, error) {
cmd, err := s.cmdRepo.ClaimPending(ctx, node.ID)
if err != nil {
return nil, err
}
if cmd == nil {
return nil, nil
}
return &AgentCommandPayload{
ID: cmd.ID,
Type: cmd.Type,
Payload: json.RawMessage(cmd.Payload),
}, nil
}
// AgentCommandResult Agent 上报命令执行结果
type AgentCommandResult struct {
Success bool `json:"success"`
ErrorMessage string `json:"errorMessage,omitempty"`
Result json.RawMessage `json:"result,omitempty"`
}
// SubmitCommandResult 接收 Agent 上报的命令结果。
func (s *AgentService) SubmitCommandResult(ctx context.Context, node *model.Node, cmdID uint, result AgentCommandResult) error {
cmd, err := s.cmdRepo.FindByID(ctx, cmdID)
if err != nil {
return err
}
if cmd == nil {
return apperror.New(404, "AGENT_COMMAND_NOT_FOUND", "命令不存在", fmt.Errorf("command %d not found", cmdID))
}
if cmd.NodeID != node.ID {
return apperror.Unauthorized("AGENT_COMMAND_FORBIDDEN", "命令不属于当前节点", nil)
}
now := time.Now().UTC()
if result.Success {
cmd.Status = model.AgentCommandStatusSucceeded
} else {
cmd.Status = model.AgentCommandStatusFailed
}
cmd.ErrorMessage = result.ErrorMessage
if len(result.Result) > 0 {
cmd.Result = string(result.Result)
}
cmd.CompletedAt = &now
return s.cmdRepo.Update(ctx, cmd)
}
// AgentTaskSpec 给 Agent 返回的任务规格,包含解密后的存储配置,供 Agent 直接执行。
// 敏感信息:此接口仅供 Agent 调用token 认证),避免通过公共 API 泄露。
type AgentTaskSpec struct {
TaskID uint `json:"taskId"`
Name string `json:"name"`
Type string `json:"type"`
SourcePath string `json:"sourcePath,omitempty"`
SourcePaths string `json:"sourcePaths,omitempty"`
ExcludePatterns string `json:"excludePatterns,omitempty"`
DBHost string `json:"dbHost,omitempty"`
DBPort int `json:"dbPort,omitempty"`
DBUser string `json:"dbUser,omitempty"`
DBPassword string `json:"dbPassword,omitempty"`
DBName string `json:"dbName,omitempty"`
DBPath string `json:"dbPath,omitempty"`
ExtraConfig string `json:"extraConfig,omitempty"`
Compression string `json:"compression"`
Encrypt bool `json:"encrypt"`
StorageTargets []AgentStorageTargetConfig `json:"storageTargets"`
}
// AgentStorageTargetConfig 存储目标配置(已解密)
type AgentStorageTargetConfig struct {
ID uint `json:"id"`
Type string `json:"type"`
Name string `json:"name"`
Config json.RawMessage `json:"config"`
}
// GetTaskSpec 返回 Agent 执行任务所需的完整规格。
func (s *AgentService) GetTaskSpec(ctx context.Context, node *model.Node, taskID uint) (*AgentTaskSpec, error) {
task, err := s.taskRepo.FindByID(ctx, taskID)
if err != nil {
return nil, err
}
if task == nil {
return nil, apperror.New(404, "BACKUP_TASK_NOT_FOUND", "任务不存在", nil)
}
if task.NodeID != node.ID {
return nil, apperror.Unauthorized("BACKUP_TASK_FORBIDDEN", "任务不属于当前节点", nil)
}
// 解密数据库密码(若有)
dbPassword := ""
if task.DBPasswordCiphertext != "" {
plain, decErr := s.cipher.Decrypt(task.DBPasswordCiphertext)
if decErr != nil {
return nil, fmt.Errorf("decrypt db password: %w", decErr)
}
dbPassword = string(plain)
}
// 解密存储目标配置
targets := collectTargetIDs(task)
storageTargets := make([]AgentStorageTargetConfig, 0, len(targets))
for _, tid := range targets {
target, err := s.storageRepo.FindByID(ctx, tid)
if err != nil {
return nil, err
}
if target == nil {
continue
}
configRaw, err := s.cipher.Decrypt(target.ConfigCiphertext)
if err != nil {
return nil, fmt.Errorf("decrypt storage config: %w", err)
}
storageTargets = append(storageTargets, AgentStorageTargetConfig{
ID: target.ID,
Type: target.Type,
Name: target.Name,
Config: json.RawMessage(configRaw),
})
}
return &AgentTaskSpec{
TaskID: task.ID,
Name: task.Name,
Type: task.Type,
SourcePath: task.SourcePath,
SourcePaths: task.SourcePaths,
ExcludePatterns: task.ExcludePatterns,
DBHost: task.DBHost,
DBPort: task.DBPort,
DBUser: task.DBUser,
DBPassword: dbPassword,
DBName: task.DBName,
DBPath: task.DBPath,
ExtraConfig: task.ExtraConfig,
Compression: task.Compression,
Encrypt: task.Encrypt,
StorageTargets: storageTargets,
}, nil
}
// AgentRecordUpdate Agent 上报备份记录的最终状态。
type AgentRecordUpdate struct {
Status string `json:"status"` // running | success | failed
FileName string `json:"fileName,omitempty"`
FileSize int64 `json:"fileSize,omitempty"`
Checksum string `json:"checksum,omitempty"`
StoragePath string `json:"storagePath,omitempty"`
ErrorMessage string `json:"errorMessage,omitempty"`
LogAppend string `json:"logAppend,omitempty"` // 增量日志,追加到 record.log_content
}
// UpdateRecord 更新备份记录的状态/日志。Agent 在执行过程中可多次调用。
func (s *AgentService) UpdateRecord(ctx context.Context, node *model.Node, recordID uint, update AgentRecordUpdate) error {
record, err := s.recordRepo.FindByID(ctx, recordID)
if err != nil {
return err
}
if record == nil {
return apperror.New(404, "BACKUP_RECORD_NOT_FOUND", "记录不存在", nil)
}
// 通过 task.NodeID 判断是否属于当前 agent
task, err := s.taskRepo.FindByID(ctx, record.TaskID)
if err != nil {
return err
}
if task == nil || task.NodeID != node.ID {
return apperror.Unauthorized("BACKUP_RECORD_FORBIDDEN", "记录不属于当前节点", nil)
}
if update.Status != "" {
record.Status = update.Status
}
if update.FileName != "" {
record.FileName = update.FileName
}
if update.FileSize > 0 {
record.FileSize = update.FileSize
}
if update.Checksum != "" {
record.Checksum = update.Checksum
}
if update.StoragePath != "" {
record.StoragePath = update.StoragePath
}
if update.ErrorMessage != "" {
record.ErrorMessage = update.ErrorMessage
}
if update.LogAppend != "" {
if record.LogContent == "" {
record.LogContent = update.LogAppend
} else {
record.LogContent += update.LogAppend
}
}
if update.Status == model.BackupRecordStatusSuccess || update.Status == model.BackupRecordStatusFailed {
now := time.Now().UTC()
record.CompletedAt = &now
record.DurationSeconds = int(now.Sub(record.StartedAt).Seconds())
}
if err := s.recordRepo.Update(ctx, record); err != nil {
return err
}
// 同步更新任务的 last_status
if update.Status == model.BackupRecordStatusSuccess || update.Status == model.BackupRecordStatusFailed {
task.LastStatus = update.Status
_ = s.taskRepo.Update(ctx, task)
}
return nil
}
// EnqueueCommand Master 端调用:给指定节点插入一条待执行命令。
// 返回命令 ID。
func (s *AgentService) EnqueueCommand(ctx context.Context, nodeID uint, cmdType string, payload any) (uint, error) {
if nodeID == 0 {
return 0, errors.New("nodeID is required")
}
payloadBytes, err := json.Marshal(payload)
if err != nil {
return 0, fmt.Errorf("marshal payload: %w", err)
}
cmd := &model.AgentCommand{
NodeID: nodeID,
Type: cmdType,
Status: model.AgentCommandStatusPending,
Payload: string(payloadBytes),
}
if err := s.cmdRepo.Create(ctx, cmd); err != nil {
return 0, err
}
return cmd.ID, nil
}
// WaitForCommandResult 同步等待指定命令完成(用于 list_dir 这类 RPC 式调用)。
// timeout 为 0 表示不限,建议传 10~30s。
func (s *AgentService) WaitForCommandResult(ctx context.Context, cmdID uint, timeout time.Duration) (*model.AgentCommand, error) {
deadline := time.Now().Add(timeout)
for {
cmd, err := s.cmdRepo.FindByID(ctx, cmdID)
if err != nil {
return nil, err
}
if cmd == nil {
return nil, apperror.New(404, "AGENT_COMMAND_NOT_FOUND", "命令不存在", nil)
}
switch cmd.Status {
case model.AgentCommandStatusSucceeded, model.AgentCommandStatusFailed, model.AgentCommandStatusTimeout:
return cmd, nil
}
if timeout > 0 && time.Now().After(deadline) {
return nil, apperror.New(504, "AGENT_COMMAND_TIMEOUT", "等待 Agent 响应超时", nil)
}
select {
case <-ctx.Done():
return nil, ctx.Err()
case <-time.After(300 * time.Millisecond):
}
}
}
// StartCommandTimeoutMonitor 启动后台定时任务,把超时命令标记为 timeout。
func (s *AgentService) StartCommandTimeoutMonitor(ctx context.Context, interval time.Duration, timeout time.Duration) {
if interval <= 0 {
interval = 30 * time.Second
}
if timeout <= 0 {
timeout = 10 * time.Minute
}
ticker := time.NewTicker(interval)
go func() {
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
threshold := time.Now().UTC().Add(-timeout)
_, _ = s.cmdRepo.MarkStaleTimeout(ctx, threshold)
}
}
}()
}

View File

@@ -75,12 +75,14 @@ type BackupExecutionService struct {
tasks repository.BackupTaskRepository
records repository.BackupRecordRepository
targets repository.StorageTargetRepository
nodeRepo repository.NodeRepository
storageRegistry *storage.Registry
runnerRegistry *backup.Registry
logHub *backup.LogHub
retention *backupretention.Service
cipher *codec.ConfigCipher
notifier BackupResultNotifier
agentDispatcher AgentDispatcher
async func(func())
now func() time.Time
tempDir string
@@ -89,6 +91,18 @@ type BackupExecutionService struct {
bandwidthLimit string // rclone 带宽限制
}
// AgentDispatcher 抽象把任务下发给 Agent 的能力,由 AgentService 实现。
// 用接口避免 execution service ↔ agent service 的循环依赖风险。
type AgentDispatcher interface {
EnqueueCommand(ctx context.Context, nodeID uint, cmdType string, payload any) (uint, error)
}
// SetClusterDependencies 注入集群相关的依赖,使备份执行时可把任务路由到远程节点。
func (s *BackupExecutionService) SetClusterDependencies(nodeRepo repository.NodeRepository, dispatcher AgentDispatcher) {
s.nodeRepo = nodeRepo
s.agentDispatcher = dispatcher
}
func NewBackupExecutionService(
tasks repository.BackupTaskRepository,
records repository.BackupRecordRepository,
@@ -243,6 +257,20 @@ func (s *BackupExecutionService) startTask(ctx context.Context, id uint, async b
if err := s.tasks.Update(ctx, task); err != nil {
return nil, apperror.Internal("BACKUP_TASK_UPDATE_FAILED", "无法更新任务状态", err)
}
// 多节点路由task.NodeID 指向远程节点时,把执行任务入队给 Agent
// NodeID=0 或本机节点时由 Master 直接执行。
if s.isRemoteNode(ctx, task.NodeID) {
if _, enqueueErr := s.agentDispatcher.EnqueueCommand(ctx, task.NodeID, model.AgentCommandTypeRunTask, map[string]any{
"taskId": task.ID,
"recordId": record.ID,
}); enqueueErr != nil {
// 入队失败 → 在记录中标记失败,继续返回详情
_ = s.finalizeRecord(ctx, task, record.ID, startedAt, model.BackupRecordStatusFailed,
"无法下发任务到远程节点: "+enqueueErr.Error(), "", "", 0, "", "")
return nil, apperror.Internal("AGENT_COMMAND_ENQUEUE_FAILED", "无法下发任务到远程节点", enqueueErr)
}
return s.getRecordDetail(ctx, record.ID)
}
run := func() {
s.executeTask(context.Background(), task, record.ID, startedAt)
}
@@ -254,6 +282,19 @@ func (s *BackupExecutionService) startTask(ctx context.Context, id uint, async b
return s.getRecordDetail(ctx, record.ID)
}
// isRemoteNode 判断 NodeID 是否指向一个有效的远程(非本机)节点。
// 当未注入集群依赖、nodeID 为 0、或节点为本机时均返回 false走本地执行
func (s *BackupExecutionService) isRemoteNode(ctx context.Context, nodeID uint) bool {
if s.nodeRepo == nil || s.agentDispatcher == nil || nodeID == 0 {
return false
}
node, err := s.nodeRepo.FindByID(ctx, nodeID)
if err != nil || node == nil {
return false
}
return !node.IsLocal
}
func (s *BackupExecutionService) executeTask(ctx context.Context, task *model.BackupTask, recordID uint, startedAt time.Time) {
s.semaphore <- struct{}{}
defer func() { <-s.semaphore }()
@@ -525,6 +566,22 @@ func (s *BackupExecutionService) buildTaskSpec(task *model.BackupTask, startedAt
return backup.TaskSpec{}, apperror.Internal("BACKUP_TASK_DECODE_FAILED", "无法解析源路径配置", err)
}
}
dbSpec := backup.DatabaseSpec{
Host: task.DBHost,
Port: task.DBPort,
User: task.DBUser,
Password: password,
Names: []string{task.DBName},
Path: task.DBPath,
}
// 解析 ExtraConfig 填充类型特有字段(目前主要用于 SAP HANA
if strings.TrimSpace(task.ExtraConfig) != "" {
extra := map[string]any{}
if err := json.Unmarshal([]byte(task.ExtraConfig), &extra); err != nil {
return backup.TaskSpec{}, apperror.Internal("BACKUP_TASK_DECODE_FAILED", "无法解析扩展配置", err)
}
applyHANAExtraConfig(&dbSpec, extra)
}
return backup.TaskSpec{
ID: task.ID,
Name: task.Name,
@@ -540,17 +597,30 @@ func (s *BackupExecutionService) buildTaskSpec(task *model.BackupTask, startedAt
MaxBackups: task.MaxBackups,
StartedAt: startedAt,
TempDir: s.tempDir,
Database: backup.DatabaseSpec{
Host: task.DBHost,
Port: task.DBPort,
User: task.DBUser,
Password: password,
Names: []string{task.DBName},
Path: task.DBPath,
},
Database: dbSpec,
}, nil
}
// applyHANAExtraConfig 从 ExtraConfig map 中提取 SAP HANA 字段填入 DatabaseSpec。
// 不识别的键被忽略,保持向后兼容。
func applyHANAExtraConfig(spec *backup.DatabaseSpec, extra map[string]any) {
if v, ok := extra["instanceNumber"].(string); ok {
spec.InstanceNumber = strings.TrimSpace(v)
}
if v, ok := extra["backupLevel"].(string); ok {
spec.BackupLevel = strings.ToLower(strings.TrimSpace(v))
}
if v, ok := extra["backupType"].(string); ok {
spec.BackupType = strings.ToLower(strings.TrimSpace(v))
}
if v, ok := extra["backupChannels"].(float64); ok {
spec.BackupChannels = int(v)
}
if v, ok := extra["maxRetries"].(float64); ok {
spec.MaxRetries = int(v)
}
}
func (s *BackupExecutionService) loadRecordProvider(ctx context.Context, recordID uint) (*model.BackupRecord, storage.StorageProvider, error) {
record, err := s.records.FindByID(ctx, recordID)
if err != nil {

View File

@@ -11,6 +11,7 @@ import (
"backupx/server/internal/apperror"
"backupx/server/internal/model"
"backupx/server/internal/repository"
"backupx/server/internal/storage"
"backupx/server/internal/storage/codec"
)
@@ -18,7 +19,7 @@ const backupTaskMaskedValue = "********"
type BackupTaskUpsertInput struct {
Name string `json:"name" binding:"required,min=1,max=100"`
Type string `json:"type" binding:"required,oneof=file mysql sqlite postgresql pgsql"`
Type string `json:"type" binding:"required,oneof=file mysql sqlite postgresql pgsql saphana"`
Enabled bool `json:"enabled"`
CronExpr string `json:"cronExpr" binding:"max=64"`
SourcePath string `json:"sourcePath" binding:"max=500"`
@@ -36,6 +37,8 @@ type BackupTaskUpsertInput struct {
Compression string `json:"compression" binding:"omitempty,oneof=gzip none"`
Encrypt bool `json:"encrypt"`
MaxBackups int `json:"maxBackups"`
// ExtraConfig 类型特有扩展配置(如 SAP HANA 的 backupLevel/backupChannels
ExtraConfig map[string]any `json:"extraConfig"`
}
type BackupTaskToggleInput struct {
@@ -63,16 +66,17 @@ type BackupTaskSummary struct {
type BackupTaskDetail struct {
BackupTaskSummary
SourcePath string `json:"sourcePath"`
SourcePaths []string `json:"sourcePaths"`
ExcludePatterns []string `json:"excludePatterns"`
DBHost string `json:"dbHost"`
DBPort int `json:"dbPort"`
DBUser string `json:"dbUser"`
DBName string `json:"dbName"`
DBPath string `json:"dbPath"`
MaskedFields []string `json:"maskedFields,omitempty"`
CreatedAt time.Time `json:"createdAt"`
SourcePath string `json:"sourcePath"`
SourcePaths []string `json:"sourcePaths"`
ExcludePatterns []string `json:"excludePatterns"`
DBHost string `json:"dbHost"`
DBPort int `json:"dbPort"`
DBUser string `json:"dbUser"`
DBName string `json:"dbName"`
DBPath string `json:"dbPath"`
ExtraConfig map[string]any `json:"extraConfig,omitempty"`
MaskedFields []string `json:"maskedFields,omitempty"`
CreatedAt time.Time `json:"createdAt"`
}
type BackupTaskScheduler interface {
@@ -81,10 +85,12 @@ type BackupTaskScheduler interface {
}
type BackupTaskService struct {
tasks repository.BackupTaskRepository
targets repository.StorageTargetRepository
cipher *codec.ConfigCipher
scheduler BackupTaskScheduler
tasks repository.BackupTaskRepository
targets repository.StorageTargetRepository
records repository.BackupRecordRepository
storageRegistry *storage.Registry
cipher *codec.ConfigCipher
scheduler BackupTaskScheduler
}
func NewBackupTaskService(
@@ -95,6 +101,12 @@ func NewBackupTaskService(
return &BackupTaskService{tasks: tasks, targets: targets, cipher: cipher}
}
// SetRecordsAndStorage 注入备份记录仓库和存储注册表,用于任务删除时清理远端文件。
func (s *BackupTaskService) SetRecordsAndStorage(records repository.BackupRecordRepository, registry *storage.Registry) {
s.records = records
s.storageRegistry = registry
}
func (s *BackupTaskService) SetScheduler(scheduler BackupTaskScheduler) {
s.scheduler = scheduler
}
@@ -185,26 +197,80 @@ func (s *BackupTaskService) Update(ctx context.Context, id uint, input BackupTas
return s.Get(ctx, item.ID)
}
func (s *BackupTaskService) Delete(ctx context.Context, id uint) error {
// DeleteResult 描述任务删除的结果信息,用于审计日志。
type DeleteResult struct {
TaskName string
RecordCount int
CleanedFiles int
}
func (s *BackupTaskService) Delete(ctx context.Context, id uint) (*DeleteResult, error) {
existing, err := s.tasks.FindByID(ctx, id)
if err != nil {
return apperror.Internal("BACKUP_TASK_GET_FAILED", "无法获取备份任务详情", err)
return nil, apperror.Internal("BACKUP_TASK_GET_FAILED", "无法获取备份任务详情", err)
}
if existing == nil {
return apperror.New(http.StatusNotFound, "BACKUP_TASK_NOT_FOUND", "备份任务不存在", fmt.Errorf("backup task %d not found", id))
}
if s.scheduler != nil {
if err := s.scheduler.RemoveTask(ctx, id); err != nil {
return apperror.Internal("BACKUP_TASK_SCHEDULE_FAILED", "无法移除备份任务调度", err)
}
}
if err := s.tasks.Delete(ctx, id); err != nil {
return apperror.Internal("BACKUP_TASK_DELETE_FAILED", "无法删除备份任务", err)
return nil, apperror.New(http.StatusNotFound, "BACKUP_TASK_NOT_FOUND", "备份任务不存在", fmt.Errorf("backup task %d not found", id))
}
if s.scheduler != nil {
_ = s.scheduler.RemoveTask(ctx, id)
}
return nil
// 清理远端存储文件(尽力而为,不阻止删除)
result := &DeleteResult{TaskName: existing.Name}
result.RecordCount, result.CleanedFiles = s.cleanupRemoteFiles(ctx, id)
if err := s.tasks.Delete(ctx, id); err != nil {
return nil, apperror.Internal("BACKUP_TASK_DELETE_FAILED", "无法删除备份任务", err)
}
return result, nil
}
// cleanupRemoteFiles 尽力删除任务相关的远端备份文件,返回记录数和成功删除的文件数。
func (s *BackupTaskService) cleanupRemoteFiles(ctx context.Context, taskID uint) (recordCount int, cleanedFiles int) {
if s.records == nil || s.storageRegistry == nil {
return 0, 0
}
records, err := s.records.ListByTask(ctx, taskID)
if err != nil {
return 0, 0
}
recordCount = len(records)
// 缓存 provider 避免同一存储目标重复创建连接
providerCache := make(map[uint]storage.StorageProvider)
for _, record := range records {
if strings.TrimSpace(record.StoragePath) == "" {
continue
}
provider, ok := providerCache[record.StorageTargetID]
if !ok {
provider, err = s.resolveStorageProvider(ctx, record.StorageTargetID)
if err != nil {
continue
}
providerCache[record.StorageTargetID] = provider
}
if err := provider.Delete(ctx, record.StoragePath); err == nil {
cleanedFiles++
}
}
return recordCount, cleanedFiles
}
func (s *BackupTaskService) resolveStorageProvider(ctx context.Context, targetID uint) (storage.StorageProvider, error) {
target, err := s.targets.FindByID(ctx, targetID)
if err != nil || target == nil {
return nil, fmt.Errorf("target %d not found", targetID)
}
configMap := map[string]any{}
if err := s.cipher.DecryptJSON(target.ConfigCiphertext, &configMap); err != nil {
return nil, err
}
provider, err := s.storageRegistry.Create(ctx, target.Type, configMap)
if err != nil {
return nil, err
}
return provider, nil
}
func (s *BackupTaskService) Toggle(ctx context.Context, id uint, enabled bool) (*BackupTaskSummary, error) {
@@ -283,7 +349,7 @@ func validateTaskTypeSpecificFields(input BackupTaskUpsertInput, passwordRequire
if !hasSourcePaths {
return apperror.BadRequest("BACKUP_TASK_INVALID", "文件备份必须填写源路径", nil)
}
case "mysql", "postgresql":
case "mysql", "postgresql", "saphana":
if strings.TrimSpace(input.DBHost) == "" {
return apperror.BadRequest("BACKUP_TASK_INVALID", "数据库主机不能为空", nil)
}
@@ -354,6 +420,10 @@ func (s *BackupTaskService) buildTask(existing *model.BackupTask, input BackupTa
if len(resolvedPaths) > 0 {
primarySourcePath = resolvedPaths[0]
}
extraConfigJSON, err := encodeExtraConfig(input.ExtraConfig)
if err != nil {
return nil, apperror.BadRequest("BACKUP_TASK_INVALID", "扩展配置格式不合法", err)
}
item := &model.BackupTask{
Name: strings.TrimSpace(input.Name),
Type: normalizeBackupTaskType(input.Type),
@@ -368,6 +438,7 @@ func (s *BackupTaskService) buildTask(existing *model.BackupTask, input BackupTa
DBPasswordCiphertext: passwordCiphertext,
DBName: strings.TrimSpace(input.DBName),
DBPath: strings.TrimSpace(input.DBPath),
ExtraConfig: extraConfigJSON,
StorageTargetID: primaryTargetID,
StorageTargets: storageTargets,
RetentionDays: input.RetentionDays,
@@ -393,6 +464,10 @@ func (s *BackupTaskService) toDetail(item *model.BackupTask) (*BackupTaskDetail,
if err != nil {
return nil, apperror.Internal("BACKUP_TASK_DECODE_FAILED", "无法解析源路径配置", err)
}
extraConfig, err := decodeExtraConfig(item.ExtraConfig)
if err != nil {
return nil, apperror.Internal("BACKUP_TASK_DECODE_FAILED", "无法解析扩展配置", err)
}
detail := &BackupTaskDetail{
BackupTaskSummary: toBackupTaskSummary(item),
SourcePath: item.SourcePath,
@@ -403,6 +478,7 @@ func (s *BackupTaskService) toDetail(item *model.BackupTask) (*BackupTaskDetail,
DBUser: item.DBUser,
DBName: item.DBName,
DBPath: item.DBPath,
ExtraConfig: extraConfig,
CreatedAt: item.CreatedAt,
}
if item.DBPasswordCiphertext != "" {
@@ -517,6 +593,29 @@ func decodeSourcePaths(value string) ([]string, error) {
return items, nil
}
func encodeExtraConfig(value map[string]any) (string, error) {
if len(value) == 0 {
return "", nil
}
encoded, err := json.Marshal(value)
if err != nil {
return "", err
}
return string(encoded), nil
}
func decodeExtraConfig(value string) (map[string]any, error) {
trimmed := strings.TrimSpace(value)
if trimmed == "" || trimmed == "{}" {
return nil, nil
}
result := map[string]any{}
if err := json.Unmarshal([]byte(trimmed), &result); err != nil {
return nil, err
}
return result, nil
}
func normalizeBackupTaskType(value string) string {
normalized := strings.TrimSpace(strings.ToLower(value))
if normalized == "pgsql" {

View File

@@ -4,12 +4,15 @@ import (
"context"
"crypto/rand"
"encoding/hex"
"encoding/json"
"fmt"
"net"
"net/http"
"os"
"path/filepath"
"runtime"
"sort"
"strings"
"time"
"backupx/server/internal/apperror"
@@ -37,13 +40,38 @@ type NodeCreateInput struct {
Name string `json:"name" binding:"required"`
}
// NodeService manages the cluster nodes.
type NodeService struct {
repo repository.NodeRepository
// NodeUpdateInput 是编辑节点的输入。
type NodeUpdateInput struct {
Name string `json:"name" binding:"required"`
}
func NewNodeService(repo repository.NodeRepository) *NodeService {
return &NodeService{repo: repo}
// NodeService manages the cluster nodes.
type NodeService struct {
repo repository.NodeRepository
taskRepo repository.BackupTaskRepository
agentRPC NodeAgentRPC
version string
}
// NodeAgentRPC 抽象 Agent 远程调用能力(避免 service 内循环依赖)。
// 由 AgentService 实现;当 Agent 未启用时可不注入,远程目录浏览返回提示。
type NodeAgentRPC interface {
EnqueueCommand(ctx context.Context, nodeID uint, cmdType string, payload any) (uint, error)
WaitForCommandResult(ctx context.Context, cmdID uint, timeout time.Duration) (*model.AgentCommand, error)
}
func NewNodeService(repo repository.NodeRepository, version string) *NodeService {
return &NodeService{repo: repo, version: version}
}
// SetTaskRepository 注入任务仓储以支持删除前引用检查。可选注入,便于测试。
func (s *NodeService) SetTaskRepository(taskRepo repository.BackupTaskRepository) {
s.taskRepo = taskRepo
}
// SetAgentRPC 注入 Agent RPC 能力,启用远程目录浏览。
func (s *NodeService) SetAgentRPC(rpc NodeAgentRPC) {
s.agentRPC = rpc
}
// EnsureLocalNode creates the default "local" node if it does not exist.
@@ -57,6 +85,8 @@ func (s *NodeService) EnsureLocalNode(ctx context.Context) error {
existing.LastSeen = time.Now().UTC()
hostname, _ := os.Hostname()
existing.Hostname = hostname
existing.IPAddress = detectLocalIP()
existing.AgentVer = s.version
existing.OS = runtime.GOOS
existing.Arch = runtime.GOARCH
return s.repo.Update(ctx, existing)
@@ -64,14 +94,16 @@ func (s *NodeService) EnsureLocalNode(ctx context.Context) error {
hostname, _ := os.Hostname()
token, _ := generateToken()
node := &model.Node{
Name: "本机 (Local)",
Hostname: hostname,
Token: token,
Status: model.NodeStatusOnline,
IsLocal: true,
OS: runtime.GOOS,
Arch: runtime.GOARCH,
LastSeen: time.Now().UTC(),
Name: "本机 (Local)",
Hostname: hostname,
IPAddress: detectLocalIP(),
Token: token,
Status: model.NodeStatusOnline,
IsLocal: true,
OS: runtime.GOOS,
Arch: runtime.GOARCH,
AgentVer: s.version,
LastSeen: time.Now().UTC(),
}
return s.repo.Create(ctx, node)
}
@@ -153,6 +185,20 @@ func (s *NodeService) Delete(ctx context.Context, id uint) error {
if node.IsLocal {
return apperror.BadRequest("NODE_DELETE_LOCAL", "无法删除本机节点", nil)
}
// 删除前检查是否有关联备份任务,避免孤立任务
if s.taskRepo != nil {
count, err := s.taskRepo.CountByNodeID(ctx, id)
if err != nil {
return err
}
if count > 0 {
return apperror.BadRequest(
"NODE_HAS_TASKS",
fmt.Sprintf("无法删除:该节点上还有 %d 个备份任务,请先删除或迁移", count),
nil,
)
}
}
return s.repo.Delete(ctx, id)
}
@@ -166,7 +212,8 @@ func (s *NodeService) ListDirectory(ctx context.Context, nodeID uint, path strin
return nil, apperror.New(http.StatusNotFound, "NODE_NOT_FOUND", "节点不存在", nil)
}
if !node.IsLocal {
return nil, apperror.BadRequest("NODE_REMOTE_FS_NOT_SUPPORTED", "远程节点的目录浏览需要 Agent 在线连接(即将支持)", nil)
// 远程节点:通过 Agent 命令队列做同步 RPC
return s.remoteListDirectory(ctx, node, path)
}
cleanPath := filepath.Clean(path)
@@ -198,8 +245,33 @@ func (s *NodeService) ListDirectory(ctx context.Context, nodeID uint, path strin
return result, nil
}
// OfflineThreshold 节点被判定为离线的心跳超时阈值。
// Agent 默认 15s 心跳一次45s 未见视为离线,预留 3 次重试空间。
const OfflineThreshold = 45 * time.Second
// StartOfflineMonitor 启动后台 goroutine定期把超时未心跳的节点标记为离线。
// 传入的 ctx 被取消后退出。
func (s *NodeService) StartOfflineMonitor(ctx context.Context, interval time.Duration) {
if interval <= 0 {
interval = 15 * time.Second
}
ticker := time.NewTicker(interval)
go func() {
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
threshold := time.Now().UTC().Add(-OfflineThreshold)
_, _ = s.repo.MarkStaleOffline(ctx, threshold)
}
}
}()
}
// Heartbeat updates the node status when an agent reports in.
func (s *NodeService) Heartbeat(ctx context.Context, token string, hostname string, ip string, agentVer string) error {
func (s *NodeService) Heartbeat(ctx context.Context, token string, hostname string, ip string, agentVer string, osName string, archName string) error {
node, err := s.repo.FindByToken(ctx, token)
if err != nil {
return err
@@ -211,12 +283,36 @@ func (s *NodeService) Heartbeat(ctx context.Context, token string, hostname stri
node.Hostname = hostname
node.IPAddress = ip
node.AgentVer = agentVer
node.OS = runtime.GOOS
node.Arch = runtime.GOARCH
if strings.TrimSpace(osName) != "" {
node.OS = osName
} else {
node.OS = runtime.GOOS
}
if strings.TrimSpace(archName) != "" {
node.Arch = archName
} else {
node.Arch = runtime.GOARCH
}
node.LastSeen = time.Now().UTC()
return s.repo.Update(ctx, node)
}
// Update 编辑节点名称。
func (s *NodeService) Update(ctx context.Context, id uint, input NodeUpdateInput) (*NodeSummary, error) {
node, err := s.repo.FindByID(ctx, id)
if err != nil {
return nil, err
}
if node == nil {
return nil, apperror.New(http.StatusNotFound, "NODE_NOT_FOUND", "节点不存在", nil)
}
node.Name = strings.TrimSpace(input.Name)
if err := s.repo.Update(ctx, node); err != nil {
return nil, err
}
return s.Get(ctx, id)
}
// DirEntry represents a file or directory in a node's file system.
type DirEntry struct {
Name string `json:"name"`
@@ -225,6 +321,58 @@ type DirEntry struct {
Size int64 `json:"size"`
}
// remoteListDirectory 通过命令队列下发 list_dir 给 Agent 并同步等待结果。
// Agent 必须在线,且响应需在 15s 内返回,否则返回超时错误。
func (s *NodeService) remoteListDirectory(ctx context.Context, node *model.Node, path string) ([]DirEntry, error) {
if s.agentRPC == nil {
return nil, apperror.BadRequest("NODE_REMOTE_FS_NOT_SUPPORTED", "远程目录浏览未启用,需要 Master 启用 Agent 服务", nil)
}
if node.Status != model.NodeStatusOnline {
return nil, apperror.BadRequest("NODE_OFFLINE", "节点当前不在线,无法浏览其目录", nil)
}
if strings.TrimSpace(path) == "" {
path = "/"
}
cmdID, err := s.agentRPC.EnqueueCommand(ctx, node.ID, model.AgentCommandTypeListDir, map[string]any{"path": path})
if err != nil {
return nil, apperror.Internal("AGENT_COMMAND_ENQUEUE_FAILED", "下发目录浏览命令失败", err)
}
cmd, err := s.agentRPC.WaitForCommandResult(ctx, cmdID, 15*time.Second)
if err != nil {
return nil, err
}
if cmd.Status != model.AgentCommandStatusSucceeded {
msg := cmd.ErrorMessage
if msg == "" {
msg = fmt.Sprintf("command status: %s", cmd.Status)
}
return nil, apperror.BadRequest("NODE_FS_READ_ERROR", fmt.Sprintf("远程目录浏览失败: %s", msg), nil)
}
var result struct {
Entries []DirEntry `json:"entries"`
}
if err := json.Unmarshal([]byte(cmd.Result), &result); err != nil {
return nil, apperror.Internal("AGENT_RESULT_INVALID", "Agent 返回结果格式错误", err)
}
return result.Entries, nil
}
// detectLocalIP 获取本机第一个非回环 IPv4 地址。
func detectLocalIP() string {
addrs, err := net.InterfaceAddrs()
if err != nil {
return ""
}
for _, addr := range addrs {
if ipNet, ok := addr.(*net.IPNet); ok && !ipNet.IP.IsLoopback() {
if ipNet.IP.To4() != nil {
return ipNet.IP.String()
}
}
}
return ""
}
func generateToken() (string, error) {
b := make([]byte, 32)
if _, err := rand.Read(b); err != nil {

View File

@@ -5,8 +5,6 @@ import (
"encoding/json"
"fmt"
"net/http"
"os"
"os/exec"
"path/filepath"
"runtime"
"strings"
@@ -135,62 +133,3 @@ func (s *SystemService) GetInfo(_ context.Context) *SystemInfo {
return info
}
// UpdateApplyResult 描述自动更新执行结果。
type UpdateApplyResult struct {
Success bool `json:"success"`
Message string `json:"message"`
Output string `json:"output,omitempty"`
}
// IsDockerEnvironment 检测当前是否运行在 Docker 容器中。
func (s *SystemService) IsDockerEnvironment() bool {
if _, err := os.Stat("/.dockerenv"); err == nil {
return true
}
return false
}
// ApplyDockerUpdate 执行 Docker 自动更新pull 新镜像 + recreate 容器。
// 容器会在 docker compose up -d 后自动重启为新版本。
func (s *SystemService) ApplyDockerUpdate(_ context.Context, targetVersion string) *UpdateApplyResult {
if !s.IsDockerEnvironment() {
return &UpdateApplyResult{Success: false, Message: "当前非 Docker 环境,请手动下载二进制更新"}
}
image := "awuqing/backupx"
tag := strings.TrimSpace(targetVersion)
if tag == "" {
tag = "latest"
}
pullTarget := image + ":" + tag
// Step 1: docker pull
pullCmd := exec.Command("docker", "pull", pullTarget)
pullOut, pullErr := pullCmd.CombinedOutput()
if pullErr != nil {
return &UpdateApplyResult{Success: false, Message: fmt.Sprintf("docker pull 失败: %v", pullErr), Output: string(pullOut)}
}
// Step 2: docker compose up -d后台执行容器会自重启
// 检测 compose 命令
composeBin := "docker"
composeArgs := []string{"compose", "up", "-d"}
if _, err := exec.LookPath("docker-compose"); err == nil {
composeBin = "docker-compose"
composeArgs = []string{"up", "-d"}
}
// 异步执行,给 API 响应留时间
go func() {
time.Sleep(1 * time.Second)
cmd := exec.Command(composeBin, composeArgs...)
cmd.Dir = "/app" // Docker 容器中的工作目录
_ = cmd.Run()
}()
return &UpdateApplyResult{
Success: true,
Message: fmt.Sprintf("已拉取 %s容器即将自动重启到新版本", pullTarget),
Output: string(pullOut),
}
}

View File

@@ -4,6 +4,7 @@ import (
"context"
"fmt"
"io"
"sort"
"strings"
"time"
@@ -124,6 +125,36 @@ func (p *Provider) About(ctx context.Context) (*storage.StorageUsageInfo, error)
}, nil
}
// RemoveEmptyDirs 递归删除 prefix 下的空目录,从最深层开始。
// 非空目录删除会失败(安全忽略),仅清理真正的空目录。
func (p *Provider) RemoveEmptyDirs(ctx context.Context, prefix string) error {
var dirs []string
err := walk.ListR(ctx, p.rfs, prefix, true, -1, walk.ListDirs, func(entries fs.DirEntries) error {
for _, entry := range entries {
if _, ok := entry.(fs.Directory); ok {
dirs = append(dirs, entry.Remote())
}
}
return nil
})
if err != nil {
// 列目录失败(比如目录不存在)静默返回
return nil
}
// 按路径长度倒序(深目录优先删除),同长度保持稳定顺序
sort.SliceStable(dirs, func(i, j int) bool {
return len(dirs[i]) > len(dirs[j])
})
for _, dir := range dirs {
_ = p.rfs.Rmdir(ctx, dir)
}
// 尝试清理 prefix 本身
if prefix != "" {
_ = p.rfs.Rmdir(ctx, prefix)
}
return nil
}
// pathDir 返回 objectKey 的目录部分(正斜杠分隔)。
func pathDir(objectKey string) string {
idx := strings.LastIndex(objectKey, "/")

View File

@@ -145,3 +145,10 @@ type FTPConfig struct {
UseTLS bool `json:"useTLS"`
}
// StorageDirCleaner 是可选能力接口,支持清理空目录。
// 主要用于本地磁盘等文件系统类存储,对象存储通常不需要。
// 通过 type assertion 检测 provider 是否实现该接口。
type StorageDirCleaner interface {
RemoveEmptyDirs(ctx context.Context, prefix string) error
}

View File

@@ -11,10 +11,15 @@ import { StorageTargetFormDrawer } from '../storage-targets/StorageTargetFormDra
import {
backupCompressionOptions,
backupTaskTypeOptions,
defaultSapHanaExtraConfig,
getDefaultPort,
isDatabaseBackupTask,
isFileBackupTask,
isSapHanaBackupTask,
isSQLiteBackupTask,
sapHanaBackupLevelOptions,
sapHanaBackupTypeOptions,
type SapHanaExtraConfig,
} from './field-config'
interface BackupTaskFormDrawerProps {
@@ -55,6 +60,7 @@ function createEmptyDraft(storageTargets?: StorageTargetSummary[]): BackupTaskPa
compression: 'gzip',
encrypt: false,
maxBackups: 10,
extraConfig: undefined,
}
}
@@ -114,6 +120,7 @@ export function BackupTaskFormDrawer({ visible, loading, initialValue, storageTa
compression: initialValue.compression,
encrypt: initialValue.encrypt,
maxBackups: initialValue.maxBackups,
extraConfig: initialValue.extraConfig,
})
setExcludePatternsText(initialValue.excludePatterns.join('\n'))
setCurrentStep(0)
@@ -152,12 +159,28 @@ export function BackupTaskFormDrawer({ visible, loading, initialValue, storageTa
dbPassword: value === 'mysql' || value === 'postgresql' || value === 'saphana' ? current.dbPassword : '',
dbName: value === 'mysql' || value === 'postgresql' || value === 'saphana' ? current.dbName : '',
dbPath: value === 'sqlite' ? current.dbPath : '',
// 切换到 SAP HANA 时初始化扩展配置;切换到其他类型时清空
extraConfig: value === 'saphana'
? ({ ...defaultSapHanaExtraConfig(), ...(current.extraConfig as SapHanaExtraConfig | undefined) } as unknown as Record<string, unknown>)
: undefined,
}))
if (value !== 'file') {
setExcludePatternsText('')
}
}
// 更新 SAP HANA 扩展配置的辅助函数
function updateHanaExtraConfig(patch: Partial<SapHanaExtraConfig>) {
setDraft((current) => {
const merged: SapHanaExtraConfig = {
...defaultSapHanaExtraConfig(),
...(current.extraConfig as SapHanaExtraConfig | undefined),
...patch,
}
return { ...current, extraConfig: merged as unknown as Record<string, unknown> }
})
}
function validate(value: BackupTaskPayload) {
if (!value.name.trim()) {
return '请输入任务名称'
@@ -368,12 +391,78 @@ export function BackupTaskFormDrawer({ visible, loading, initialValue, storageTa
<Input value={draft.dbName} placeholder="例如app_prod" onChange={(value) => updateDraft({ dbName: value })} />
)}
</div>
{isSapHanaBackupTask(draft.type) ? renderSapHanaExtraFields() : null}
</>
) : null}
</Space>
)
}
function renderSapHanaExtraFields() {
const hana: SapHanaExtraConfig = {
...defaultSapHanaExtraConfig(),
...(draft.extraConfig as SapHanaExtraConfig | undefined),
}
return (
<>
<Divider style={{ margin: '8px 0' }} orientation="left">
<Typography.Text type="secondary">SAP HANA </Typography.Text>
</Divider>
<div>
<Typography.Text></Typography.Text>
<Select
style={{ width: '100%' }}
value={hana.backupType}
options={[...sapHanaBackupTypeOptions]}
onChange={(value) => updateHanaExtraConfig({ backupType: value as SapHanaExtraConfig['backupType'] })}
/>
</div>
<div>
<Typography.Text></Typography.Text>
<Select
style={{ width: '100%' }}
value={hana.backupLevel}
options={[...sapHanaBackupLevelOptions]}
disabled={hana.backupType === 'log'}
onChange={(value) => updateHanaExtraConfig({ backupLevel: value as SapHanaExtraConfig['backupLevel'] })}
/>
{hana.backupType === 'log' ? (
<Typography.Text type="secondary" style={{ fontSize: 12 }}></Typography.Text>
) : null}
</div>
<div>
<Typography.Text></Typography.Text>
<InputNumber
style={{ width: '100%' }}
value={hana.backupChannels}
min={1}
max={32}
onChange={(value) => updateHanaExtraConfig({ backupChannels: Number(value ?? 1) })}
/>
<Typography.Text type="secondary" style={{ fontSize: 12 }}>{'>'} 1 HANA </Typography.Text>
</div>
<div>
<Typography.Text></Typography.Text>
<InputNumber
style={{ width: '100%' }}
value={hana.maxRetries}
min={1}
max={10}
onChange={(value) => updateHanaExtraConfig({ maxRetries: Number(value ?? 3) })}
/>
</div>
<div>
<Typography.Text></Typography.Text>
<Input
value={hana.instanceNumber}
placeholder="留空将根据端口自动推断(例如 30015 → 0"
onChange={(value) => updateHanaExtraConfig({ instanceNumber: value })}
/>
</div>
</>
)
}
async function handleQuickCreateSubmit(value: StorageTargetPayload) {
if (!onCreateStorageTarget) return
setQuickCreateLoading(true)

View File

@@ -86,3 +86,39 @@ export function getDefaultPort(type: BackupTaskType) {
export function getCompressionLabel(compression: BackupCompression) {
return compression === 'gzip' ? 'Gzip' : '无'
}
/** SAP HANA 备份级别选项 */
export const sapHanaBackupLevelOptions = [
{ label: '完整备份 (Full)', value: 'full' },
{ label: '增量备份 (Incremental)', value: 'incremental' },
{ label: '差异备份 (Differential)', value: 'differential' },
] as const
/** SAP HANA 备份类型选项 */
export const sapHanaBackupTypeOptions = [
{ label: '数据备份 (Data)', value: 'data' },
{ label: '日志备份 (Log)', value: 'log' },
] as const
/** SAP HANA 扩展配置默认值 */
export interface SapHanaExtraConfig {
backupType?: 'data' | 'log'
backupLevel?: 'full' | 'incremental' | 'differential'
backupChannels?: number
maxRetries?: number
instanceNumber?: string
}
export function isSapHanaBackupTask(type: BackupTaskType) {
return type === 'saphana'
}
export function defaultSapHanaExtraConfig(): SapHanaExtraConfig {
return {
backupType: 'data',
backupLevel: 'full',
backupChannels: 1,
maxRetries: 3,
instanceNumber: '',
}
}

View File

@@ -1,5 +1,5 @@
import { Button, Input, Message, Modal, Space, Spin, Tree, Typography } from '@arco-design/web-react'
import { IconFolder, IconFile } from '@arco-design/web-react/icon'
import { Button, Input, Message, Modal, Space, Spin, Tree, Typography, Empty } from '@arco-design/web-react'
import { IconFolder, IconFile, IconFolderAdd } from '@arco-design/web-react/icon'
import { useCallback, useState } from 'react'
import { listNodeDirectory } from '../../services/nodes'
import type { DirEntry } from '../../types/nodes'
@@ -27,7 +27,7 @@ function entriesToTreeNodes(entries: DirEntry[], mode: 'directory' | 'file'): Tr
.map((entry) => ({
key: entry.path,
title: entry.name,
icon: entry.isDir ? <IconFolder /> : <IconFile />,
icon: entry.isDir ? <IconFolder style={{ color: 'var(--color-warning-6)' }} /> : <IconFile />,
isLeaf: !entry.isDir,
}))
}
@@ -94,46 +94,83 @@ export function DirectoryPicker({ value, onChange, placeholder, mode = 'director
setModalVisible(false)
}
function handleInputKeyDown(e: React.KeyboardEvent) {
if (e.key === 'Enter') {
e.preventDefault()
const trimmed = value?.trim()
if (trimmed) {
onChange(trimmed)
}
}
}
// 没有 nodeId 时退化为普通输入框
if (nodeId === undefined) {
return <Input value={value} placeholder={placeholder} onChange={onChange} />
return <Input value={value} placeholder={placeholder} onChange={onChange} onKeyDown={handleInputKeyDown} />
}
return (
<>
<Space style={{ width: '100%' }}>
<Input style={{ flex: 1 }} value={value} placeholder={placeholder} onChange={onChange} />
<Button type="outline" size="small" onClick={handleOpen}>
<div style={{ display: 'flex', gap: 8, width: '100%' }}>
<Input
style={{ flex: 1 }}
value={value}
placeholder={placeholder}
onChange={onChange}
onKeyDown={handleInputKeyDown}
allowClear
/>
<Button type="outline" size="default" onClick={handleOpen} icon={<IconFolderAdd />}>
</Button>
</Space>
</div>
<Modal
title={mode === 'directory' ? '选择目录' : '选择文件'}
visible={modalVisible}
onCancel={() => setModalVisible(false)}
onOk={handleConfirm}
okText="选择"
okText="确认选择"
cancelText="取消"
style={{ width: 560 }}
style={{ width: 640 }}
okButtonProps={{ disabled: !selectedPath }}
unmountOnExit
>
{selectedPath && (
<div style={{ padding: '8px 12px', marginBottom: 12, background: 'var(--color-fill-2)', borderRadius: 4 }}>
<Typography.Text copyable style={{ fontSize: 13 }}>
{/* 当前选中路径 */}
<div style={{
padding: '10px 14px',
marginBottom: 16,
background: selectedPath ? 'var(--color-primary-light-1)' : 'var(--color-fill-2)',
borderRadius: 6,
border: selectedPath ? '1px solid var(--color-primary-light-3)' : '1px solid var(--color-border)',
display: 'flex',
alignItems: 'center',
gap: 8,
minHeight: 40,
}}>
<IconFolder style={{ color: selectedPath ? 'var(--color-primary-6)' : 'var(--color-text-4)', fontSize: 16, flexShrink: 0 }} />
{selectedPath ? (
<Typography.Text copyable style={{ fontSize: 13, fontFamily: 'monospace', wordBreak: 'break-all' }}>
{selectedPath}
</Typography.Text>
</div>
)}
) : (
<Typography.Text type="secondary" style={{ fontSize: 13 }}></Typography.Text>
)}
</div>
{/* 目录树 */}
{loading ? (
<Spin style={{ display: 'block', textAlign: 'center', padding: 40 }} />
<Spin style={{ display: 'block', textAlign: 'center', padding: 48 }} tip="加载目录中..." />
) : treeData.length === 0 ? (
<Typography.Text type="secondary" style={{ display: 'block', textAlign: 'center', padding: 40 }}>
</Typography.Text>
<Empty style={{ padding: 48 }} description="目录为空" />
) : (
<div style={{ maxHeight: 420, overflow: 'auto', border: '1px solid var(--color-border)', borderRadius: 4, padding: '4px 0' }}>
<div style={{
maxHeight: 400,
overflow: 'auto',
border: '1px solid var(--color-border)',
borderRadius: 6,
padding: '6px 0',
}}>
<Tree
blockNode
showLine

View File

@@ -1,8 +1,8 @@
import { Alert, Button, Divider, Drawer, Input, Select, Space, Switch, Typography } from '@arco-design/web-react'
import { Alert, Button, Collapse, Divider, Drawer, Input, Select, Space, Switch, Typography } from '@arco-design/web-react'
import { useEffect, useMemo, useState } from 'react'
import { getStorageTargetFieldConfigs, getStorageTargetTypeLabel, isBuiltinType, buildAllTypeOptions } from './field-config'
import type { StorageConnectionTestResult, StorageTargetDetail, StorageTargetPayload, StorageTargetType } from '../../types/storage-targets'
import { listRcloneBackends, type RcloneBackendInfo } from '../../services/rclone'
import { listRcloneBackends, type RcloneBackendInfo, type RcloneBackendOption } from '../../services/rclone'
interface StorageTargetFormDrawerProps {
visible: boolean
@@ -138,8 +138,28 @@ export function StorageTargetFormDrawer({
})
}
// 渲染动态字段rclone 后端)
// 渲染单个动态字段
function renderDynamicOption(opt: RcloneBackendOption) {
return (
<div key={opt.key}>
<Typography.Text>{opt.key}{opt.required ? ' *' : ''}</Typography.Text>
{opt.isPassword ? (
<Input.Password value={(draft.config[opt.key] as string) || ''} placeholder={opt.label} onChange={(v) => updateConfig(opt.key, v)} />
) : (
<Input value={(draft.config[opt.key] as string) || ''} placeholder={opt.label} onChange={(v) => updateConfig(opt.key, v)} />
)}
{opt.label && (
<Typography.Paragraph type="secondary" style={{ marginBottom: 0, marginTop: 2, fontSize: 12 }} ellipsis={{ rows: 2, expandable: true }}>{opt.label}</Typography.Paragraph>
)}
</div>
)
}
// 渲染动态字段rclone 后端)— 必填优先,可选折叠
function renderDynamicFields() {
const requiredOptions = dynamicBackend?.options.filter((opt) => opt.required) ?? []
const optionalOptions = dynamicBackend?.options.filter((opt) => !opt.required) ?? []
return (
<>
<div>
@@ -147,19 +167,19 @@ export function StorageTargetFormDrawer({
<Input value={(draft.config.root as string) || ''} placeholder="如 /backups 或 bucket 名" onChange={(v) => updateConfig('root', v)} />
<Typography.Paragraph type="secondary" style={{ marginBottom: 0, marginTop: 4 }}>使</Typography.Paragraph>
</div>
{dynamicBackend && dynamicBackend.options.length > 0 && dynamicBackend.options.map((opt) => (
<div key={opt.key}>
<Typography.Text>{opt.key}{opt.required ? ' *' : ''}</Typography.Text>
{opt.isPassword ? (
<Input.Password value={(draft.config[opt.key] as string) || ''} placeholder={opt.label} onChange={(v) => updateConfig(opt.key, v)} />
) : (
<Input value={(draft.config[opt.key] as string) || ''} placeholder={opt.label} onChange={(v) => updateConfig(opt.key, v)} />
)}
{opt.label && (
<Typography.Paragraph type="secondary" style={{ marginBottom: 0, marginTop: 2, fontSize: 12 }} ellipsis={{ rows: 2, expandable: true }}>{opt.label}</Typography.Paragraph>
)}
</div>
))}
{requiredOptions.map(renderDynamicOption)}
{optionalOptions.length > 0 && (
<Collapse bordered={false} style={{ background: 'transparent' }}>
<Collapse.Item
header={<Typography.Text type="secondary">{optionalOptions.length} </Typography.Text>}
name="advanced"
>
<Space direction="vertical" size="large" style={{ width: '100%' }}>
{optionalOptions.map(renderDynamicOption)}
</Space>
</Collapse.Item>
</Collapse>
)}
</>
)
}

View File

@@ -3,10 +3,10 @@ import {
Table, Button, Space, Tag, Typography, PageHeader, Modal, Input, Message, Badge, Popconfirm, Card, Descriptions, Empty
} from '@arco-design/web-react'
import {
IconPlus, IconDelete, IconDesktop, IconCloudDownload
IconPlus, IconDelete, IconDesktop, IconCloudDownload, IconEdit
} from '@arco-design/web-react/icon'
import type { NodeSummary } from '../../types/nodes'
import { listNodes, createNode, deleteNode } from '../../services/nodes'
import { listNodes, createNode, deleteNode, updateNode } from '../../services/nodes'
const { Title, Text } = Typography
@@ -17,6 +17,11 @@ export default function NodesPage() {
const [newNodeName, setNewNodeName] = useState('')
const [newToken, setNewToken] = useState('')
// 编辑状态
const [editVisible, setEditVisible] = useState(false)
const [editNode, setEditNode] = useState<NodeSummary | null>(null)
const [editName, setEditName] = useState('')
const fetchNodes = useCallback(async () => {
setLoading(true)
try {
@@ -56,6 +61,21 @@ export default function NodesPage() {
}
}
const handleEdit = async () => {
if (!editNode || !editName.trim()) {
Message.warning('请输入节点名称')
return
}
try {
await updateNode(editNode.id, { name: editName.trim() })
Message.success('节点更新成功')
setEditVisible(false)
fetchNodes()
} catch {
Message.error('更新节点失败')
}
}
const columns = [
{
title: '节点名称',
@@ -110,15 +130,22 @@ export default function NodesPage() {
},
{
title: '操作',
width: 80,
render: (_: unknown, record: NodeSummary) => {
if (record.isLocal) return <Text type="secondary">-</Text>
return (
<Popconfirm title="确定删除该节点?" onOk={() => handleDelete(record.id)}>
<Button type="text" status="danger" icon={<IconDelete />} size="small" />
</Popconfirm>
)
},
width: 120,
render: (_: unknown, record: NodeSummary) => (
<Space>
<Button
type="text"
icon={<IconEdit />}
size="small"
onClick={() => { setEditNode(record); setEditName(record.name); setEditVisible(true) }}
/>
{!record.isLocal && (
<Popconfirm title="确定删除该节点?" onOk={() => handleDelete(record.id)}>
<Button type="text" status="danger" icon={<IconDelete />} size="small" />
</Popconfirm>
)}
</Space>
),
},
]
@@ -145,10 +172,12 @@ export default function NodesPage() {
/>
</Card>
{/* 添加节点弹窗 */}
<Modal
title="添加远程节点"
visible={createVisible}
onCancel={() => setCreateVisible(false)}
style={{ width: 640 }}
footer={newToken ? (
<Button type="primary" onClick={() => setCreateVisible(false)}></Button>
) : undefined}
@@ -169,12 +198,48 @@ export default function NodesPage() {
]} />
<div style={{ marginTop: 12, padding: '8px 12px', background: 'var(--color-fill-2)', borderRadius: 6 }}>
<Text type="secondary" style={{ fontSize: 12 }}>
Agent
Agent Master
</Text>
</div>
<div style={{ marginTop: 12 }}>
<Text bold style={{ fontSize: 13 }}>Agent </Text>
<ol style={{ fontSize: 12, color: 'var(--color-text-2)', paddingLeft: 20, marginTop: 8 }}>
<li> BackupX Master </li>
<li> Agent MASTER_URL</li>
</ol>
<div style={{ background: 'var(--color-fill-2)', padding: '8px 12px', borderRadius: 6, marginTop: 4 }}>
<Text copyable style={{ fontFamily: 'monospace', fontSize: 12, wordBreak: 'break-all' }}>
{`backupx agent --master ${window.location.origin} --token ${newToken}`}
</Text>
</div>
<Text type="secondary" style={{ fontSize: 12, display: 'block', marginTop: 8 }}>
使 / <br />
<code>BACKUPX_AGENT_MASTER={window.location.origin}</code><br />
<code>BACKUPX_AGENT_TOKEN={newToken}</code>
</Text>
</div>
</div>
)}
</Modal>
{/* 编辑节点弹窗 */}
<Modal
title="编辑节点"
visible={editVisible}
onCancel={() => setEditVisible(false)}
onOk={handleEdit}
okText="保存"
cancelText="取消"
>
<div style={{ marginBottom: 8 }}>
<Text type="secondary"></Text>
</div>
<Input
placeholder="输入节点名称"
value={editName}
onChange={setEditName}
/>
</Modal>
</div>
)
}

View File

@@ -1,6 +1,6 @@
import { Badge, Button, Card, Descriptions, Grid, Link, Message, PageHeader, Space, Tag, Typography } from '@arco-design/web-react'
import { useEffect, useState } from 'react'
import { fetchSystemInfo, checkUpdate, applyUpdate, type SystemInfo, type UpdateCheckResult } from '../../services/system'
import { fetchSystemInfo, checkUpdate, type SystemInfo, type UpdateCheckResult } from '../../services/system'
import { resolveErrorMessage } from '../../utils/error'
import { formatDuration } from '../../utils/format'
@@ -24,7 +24,6 @@ export function SettingsPage() {
const [error, setError] = useState('')
const [updateResult, setUpdateResult] = useState<UpdateCheckResult | null>(null)
const [checking, setChecking] = useState(false)
const [applying, setApplying] = useState(false)
useEffect(() => {
let active = true
@@ -53,24 +52,6 @@ export function SettingsPage() {
}
}
async function handleApplyUpdate() {
if (!updateResult?.latestVersion) return
setApplying(true)
try {
const result = await applyUpdate(updateResult.latestVersion)
if (result.success) {
Message.success('更新已触发,容器即将自动重启...')
setTimeout(() => Message.info('请等待 10-30 秒后刷新页面'), 3000)
} else {
Message.warning(result.message)
}
} catch (e) {
Message.error(resolveErrorMessage(e, '触发更新失败'))
} finally {
setApplying(false)
}
}
return (
<Space direction="vertical" size="large" style={{ width: '100%' }}>
<PageHeader style={{ paddingBottom: 16 }} title="系统设置" subTitle="运行信息、磁盘状态与版本更新">
@@ -124,9 +105,6 @@ export function SettingsPage() {
</Card>
)}
<Space>
<Button type="primary" status="success" loading={applying} onClick={handleApplyUpdate}>
Docker
</Button>
{updateResult.downloadUrl && (
<Link href={updateResult.downloadUrl} target="_blank">
<Button type="outline"></Button>
@@ -138,13 +116,6 @@ export function SettingsPage() {
</Link>
)}
</Space>
{updateResult.dockerImage && (
<Card size="small" title="Docker 更新命令">
<Typography.Paragraph copyable code style={{ marginBottom: 0 }}>
{`docker pull ${updateResult.dockerImage}:${updateResult.latestVersion} && docker compose up -d`}
</Typography.Paragraph>
</Card>
)}
</Space>
) : (
<Space>

View File

@@ -16,6 +16,11 @@ export async function createNode(name: string) {
return unwrapApiEnvelope(response.data)
}
export async function updateNode(id: number, data: { name: string }) {
const response = await http.put<ApiEnvelope<NodeSummary>>(`/nodes/${id}`, data)
return unwrapApiEnvelope(response.data)
}
export async function deleteNode(id: number) {
const response = await http.delete<ApiEnvelope<null>>(`/nodes/${id}`)
return unwrapApiEnvelope(response.data)

View File

@@ -33,17 +33,6 @@ export async function checkUpdate() {
return response.data.data
}
export interface UpdateApplyResult {
success: boolean
message: string
output?: string
}
export async function applyUpdate(version: string) {
const response = await http.post<{ code: string; message: string; data: UpdateApplyResult }>('/system/update-apply', { version })
return response.data.data
}
export async function fetchSettings() {
const response = await http.get<{ code: string; message: string; data: Record<string, string> }>('/settings')
return response.data.data

View File

@@ -33,6 +33,8 @@ export interface BackupTaskDetail extends BackupTaskSummary {
dbUser: string
dbName: string
dbPath: string
/** 类型特有的扩展配置(如 SAP HANA 的 backupLevel/backupChannels 等) */
extraConfig?: Record<string, unknown>
maskedFields?: string[]
createdAt: string
}
@@ -59,6 +61,8 @@ export interface BackupTaskPayload {
compression: BackupCompression
encrypt: boolean
maxBackups: number
/** 类型特有的扩展配置(如 SAP HANA 的 backupLevel/backupChannels 等) */
extraConfig?: Record<string, unknown>
}
export interface BackupTaskTogglePayload {