mirror of
https://github.com/Awuqing/BackupX.git
synced 2026-09-05 15:37:03 +08:00
feat(cluster): 支持远程源服务器集中备份
为 Master 本地磁盘增加可选流式中转,远程 Agent 可直接把产物写入中央存储并通过反向通道恢复。 持久化并校验传输模式,兼容既有 Agent 本机磁盘目标,补齐鉴权、完整性、配额、访问保护、前端配置及双向链路测试。 Closes #101
This commit is contained in:
@@ -26,6 +26,27 @@ BackupX supports Master-Agent mode: backup tasks can be routed to specific nodes
|
||||
- **Execution** — Agent reuses the same BackupRunner (file / mysql / postgresql / sqlite / saphana) and uploads directly to storage
|
||||
- **Security** — Each node has its own token; the Agent never holds the Master's JWT secret or AES-256 key
|
||||
|
||||
## Centralize backups from servers B/C/D into storage M
|
||||
|
||||
Use the Master as the control plane and register every source server as an Agent. A task's **Source server** determines where paths and database tools are resolved; its **Storage targets** determine where the resulting artifact is retained.
|
||||
|
||||
BackupX chooses the data path per target:
|
||||
|
||||
| Destination | Data path |
|
||||
| --- | --- |
|
||||
| S3, WebDAV, FTP, cloud drive, or another network backend | Agent streams directly to the destination |
|
||||
| `local_disk` with **Relay remote backups through Master** enabled (for example storage server M mounted through NFS) | Agent streams through the authenticated Master API; Master writes to its configured local path |
|
||||
|
||||
The relay is streaming: the Master does not create a second temporary copy of the entire artifact. The reverse path is used when restoring a Master-local artifact back to its source Agent. Use HTTPS whenever Agent traffic crosses an untrusted network.
|
||||
|
||||
To configure the common `A → {B,C,D} → M` topology:
|
||||
|
||||
1. Run BackupX Master on A and mount M on A if M is exposed as NFS or another filesystem.
|
||||
2. Create a `local_disk` target for that mount and keep **Relay remote backups through Master** enabled, or create an S3/WebDAV target exposed by M. Existing local-disk targets keep their prior Agent-local behavior until this switch is enabled.
|
||||
3. Install one Agent on B, C, and D from **Node Management**.
|
||||
4. Create a backup task for each source, choose B/C/D under **Source server**, browse that server's paths, and select M as the storage target. A source-server pool label can route identical tasks dynamically.
|
||||
5. Verify the per-target result in the backup record. For a Master-local target, the record reports transfer mode `master_relay`; network backends remain `direct`.
|
||||
|
||||
## Walkthrough
|
||||
|
||||
### 0. Set the Master URL for production clusters
|
||||
@@ -78,7 +99,7 @@ In Step 1 choose "Batch" and paste node names (one per line, max 50). Step 3 sho
|
||||
|
||||
### 5. Route a task to the node
|
||||
|
||||
In the **Backup Tasks** page, pick the target node when creating the task. When the task runs:
|
||||
In the **Backup Tasks** page, pick the source server when creating the task. When the task runs:
|
||||
|
||||
- Local (`nodeId=0`) → Master executes in-process
|
||||
- Remote node → Master enqueues the command → Agent claims → Agent runs locally → uploads → reports back
|
||||
|
||||
@@ -19,7 +19,9 @@ BackupX aims to accept any place you'd want to drop a backup file.
|
||||
| **Google Drive** | Client ID/Secret + OAuth authorization |
|
||||
| **WebDAV** | URL + username/password |
|
||||
| **FTP / FTPS** | Host + port + username/password |
|
||||
| **Local disk** | Target directory (absolute path) |
|
||||
| **Local disk** | Target directory (absolute path) + optional Master relay for remote Agents |
|
||||
|
||||
New local-disk targets enable **Relay remote backups through Master** by default. This makes the configured path belong to the Master, so a storage server mounted there can collect backups from many source Agents. Turn the switch off when the path intentionally belongs to each Agent. Existing targets retain their previous Agent-local behavior until explicitly changed.
|
||||
|
||||
## Rclone backends
|
||||
|
||||
|
||||
@@ -26,6 +26,27 @@ BackupX 支持 Master-Agent 模式:备份任务可以指定在哪个节点执
|
||||
- **执行** — Agent 复用 BackupRunner(file / mysql / postgresql / sqlite / saphana)并直接上传到存储
|
||||
- **安全** — 每个节点独立 Token;Agent 不持有 Master 的 JWT 密钥或 AES-256 加密密钥
|
||||
|
||||
## 把 B/C/D 服务器集中备份到 M
|
||||
|
||||
Master 作为控制面,每台源服务器安装一个 Agent。任务里的 **源服务器** 决定源路径和数据库工具在哪台机器解析,**存储目标** 决定备份产物最终保留在哪里。
|
||||
|
||||
BackupX 会根据目标类型选择数据路径:
|
||||
|
||||
| 目标 | 数据路径 |
|
||||
| --- | --- |
|
||||
| S3、WebDAV、FTP、云盘或其他网络后端 | Agent 直接流式上传到目标 |
|
||||
| 启用 **远程备份经 Master 中转** 的 `local_disk`(例如通过 NFS 挂载的存储服务器 M) | Agent 通过认证后的 Master API 流式中转,由 Master 写入配置目录 |
|
||||
|
||||
中转过程不会在 Master 上额外落一份完整临时文件。把 Master 本地磁盘中的备份恢复到源 Agent 时会走反向流式通道。Agent 与 Master 之间跨越不可信网络时必须配置 HTTPS。
|
||||
|
||||
典型的 `A → {B,C,D} → M` 拓扑按以下步骤配置:
|
||||
|
||||
1. 在 A 运行 BackupX Master;如果 M 以 NFS 等文件系统提供存储,先把 M 挂载到 A。
|
||||
2. 为该挂载点创建 `local_disk` 目标并保持 **远程备份经 Master 中转** 开启;如果 M 提供 S3/WebDAV,也可直接创建对应网络目标。升级前已有的本地磁盘目标会继续沿用 Agent 本机落盘,手动开启该选项后才切换到中央目录。
|
||||
3. 从 **节点管理** 分别在 B、C、D 安装 Agent。
|
||||
4. 为每台源服务器创建任务,在 **源服务器** 选择 B/C/D,浏览该服务器的路径,再把 M 选为存储目标。相同任务也可用源服务器池标签动态调度。
|
||||
5. 在备份记录中检查逐目标结果。Master 本地磁盘目标会记录 `master_relay` 中转模式,网络后端仍为 `direct` 直传。
|
||||
|
||||
## 一键部署步骤
|
||||
|
||||
### 0. 为生产集群设置 Master 对外 URL
|
||||
@@ -78,7 +99,7 @@ Docker 模式使用同一组环境变量约定:`BACKUPX_AGENT_MASTER`、`BACKU
|
||||
|
||||
### 5. 把任务路由到该节点
|
||||
|
||||
在 **备份任务** 页面新建任务时选择对应节点。任务触发时:
|
||||
在 **备份任务** 页面新建任务时选择对应源服务器。任务触发时:
|
||||
|
||||
- 本机 / 未指定(`nodeId=0`):Master 进程内直接执行
|
||||
- 远程节点:Master 写入命令队列 → Agent 拉取 → Agent 本地执行 → 上传 → 回报
|
||||
|
||||
+3
-1
@@ -19,7 +19,9 @@ BackupX 的目标是接入任何你想放置备份文件的地方。
|
||||
| **Google Drive** | Client ID/Secret + OAuth 授权 |
|
||||
| **WebDAV** | 地址 + 用户名/密码 |
|
||||
| **FTP / FTPS** | 主机 + 端口 + 用户名/密码 |
|
||||
| **本地磁盘** | 目标目录(绝对路径) |
|
||||
| **本地磁盘** | 目标目录(绝对路径)+ 可选的远程 Agent 经 Master 中转 |
|
||||
|
||||
新建本地磁盘目标默认开启 **远程备份经 Master 中转**。开启时,配置目录属于 Master,挂载到 Master 的存储服务器可集中接收多台源 Agent 的备份;如果该路径本就属于各 Agent,请关闭此选项。升级前已有目标保持原来的 Agent 本机落盘行为,只有显式开启后才会切换。
|
||||
|
||||
## Rclone 后端
|
||||
|
||||
|
||||
Reference in New Issue
Block a user