mirror of
https://github.com/Awuqing/BackupX.git
synced 2026-08-13 08:24:05 +08:00
00151e466c
实现 CDC 内容寻址仓库、远程 Agent 中央中转备份与首次初始化体验，并补充安全校验、测试及双语文档。 Closes #94 Closes #101 Closes #104
156 lines
8.6 KiB
Markdown
156 lines
8.6 KiB
Markdown
---
|
|
sidebar_position: 4
|
|
title: Multi-Node Cluster
|
|
description: Master-Agent mode — route backups to remote servers via HTTP long-polling.
|
|
---
|
|
|
|
# Multi-Node Cluster
|
|
|
|
BackupX supports Master-Agent mode: backup tasks can be routed to specific nodes. The Agent runs the backup locally and uploads straight to storage. All connections are initiated by the Agent, so remote networks only need outbound HTTP access.
|
|
|
|
## Architecture
|
|
|
|
```
|
|
[Web Console] ─── JWT ──→ [Master (backupx)]
|
|
↑ ↓
|
|
│ │ HTTP long-poll (token auth)
|
|
│ ↓
|
|
[Agent (backupx agent)] ← runs on remote host
|
|
↓
|
|
[70+ Storage Backends]
|
|
```
|
|
|
|
- **Protocol** — HTTP long-polling; the Agent initiates every connection
|
|
- **Heartbeat** — Agent reports every 15s; Master marks nodes offline after 45s of silence
|
|
- **Dispatch** — Master persists `run_task` commands to a queue; Agent polls and claims them
|
|
- **Execution** — Agent reuses the same BackupRunner (file / mysql / postgresql / sqlite / saphana) and uploads directly to storage
|
|
- **Security** — Each node has its own token; the Agent never holds the Master's JWT secret or AES-256 key
|
|
|
|
## Centralize backups from servers B/C/D into storage M
|
|
|
|
Use the Master as the control plane and register every source server as an Agent. A task's **Source server** determines where paths and database tools are resolved; its **Storage targets** determine where the resulting artifact is retained.
|
|
|
|
BackupX chooses the data path per target:
|
|
|
|
| Destination | Data path |
|
|
| --- | --- |
|
|
| S3, WebDAV, FTP, cloud drive, or another network backend | Agent streams directly to the destination |
|
|
| `local_disk` with **Relay remote backups through Master** enabled (for example storage server M mounted through NFS) | Agent streams through the authenticated Master API; Master writes to its configured local path |
|
|
|
|
The relay is streaming: the Master does not create a second temporary copy of the entire artifact. The reverse path is used when restoring a Master-local artifact back to its source Agent. Use HTTPS whenever Agent traffic crosses an untrusted network.
|
|
|
|
To configure the common `A → {B,C,D} → M` topology:
|
|
|
|
1. Run BackupX Master on A and mount M on A if M is exposed as NFS or another filesystem.
|
|
2. Create a `local_disk` target for that mount and keep **Relay remote backups through Master** enabled, or create an S3/WebDAV target exposed by M. Existing local-disk targets keep their prior Agent-local behavior until this switch is enabled.
|
|
3. Install one Agent on B, C, and D from **Node Management**.
|
|
4. Create a backup task for each source, choose B/C/D under **Source server**, browse that server's paths, and select M as the storage target. A source-server pool label can route identical tasks dynamically.
|
|
5. Verify the per-target result in the backup record. For a Master-local target, the record reports transfer mode `master_relay`; network backends remain `direct`.
|
|
|
|
## Walkthrough
|
|
|
|
### 0. Set the Master URL for production clusters
|
|
|
|
Before generating Agent install commands, make sure the Master URL shown to Agents is stable and reachable from every target host.
|
|
|
|
If BackupX runs behind Docker, Nginx, a load balancer, or an outer reverse proxy, configure `server.external_url` or `BACKUPX_SERVER_EXTERNAL_URL` on the Master:
|
|
|
|
```yaml title="config.yaml"
|
|
server:
|
|
external_url: "https://backup.example.com"
|
|
```
|
|
|
|
This URL is baked into systemd units, foreground commands, and docker-compose snippets. If it is wrong, Agents will install successfully but stay offline because they keep polling an internal or browser-only address.
|
|
|
|
### 1. Open the install wizard
|
|
|
|
In the Web Console → **Node Management** → **Add Node**. You'll see a three-step wizard.
|
|
|
|
- **Step 1 — Node info.** Give the node a name, or switch to batch mode and paste multiple names (one per line, max 50).
|
|
- **Step 2 — Deploy options.** Pick install mode (`systemd` recommended, `docker`, or `foreground` for debugging), architecture (auto-detect by default), agent version (defaults to the master's version), TTL for the install link (5 min / 15 min / 1 h / 24 h), and download source (`github` direct, or the `ghproxy` mirror for mainland China).
|
|
- **Step 3 — Copy the command.** A one-line install command is shown with a live countdown. Click copy, paste into the target machine, and run with root privileges. The default command embeds the rendered installer, so the target host does not need to fetch `/api/install/:token` through your reverse proxy. The public install URL is still available as a fallback.
|
|
|
|
### 2. One-line install on the target host
|
|
|
|
Use the command generated by the Web Console. It writes the installer to a temporary file, validates the `BACKUPX_AGENT_INSTALL_V1` marker, then runs it with root privileges.
|
|
|
|
The script runs automatically and:
|
|
|
|
1. Detects OS and architecture (`uname -m`)
|
|
2. Downloads the matching `backupx` binary from GitHub Release (or the ghproxy mirror)
|
|
3. Installs to `/opt/backupx-agent` and creates a `backupx` system user
|
|
4. Writes `/etc/systemd/system/backupx-agent.service` with the token baked into environment variables
|
|
5. Runs `systemctl enable --now backupx-agent`
|
|
6. Polls `/api/v1/agent/self` until the master confirms `status: online` (up to 30 s)
|
|
|
|
Docker mode uses the same `BACKUPX_AGENT_MASTER`, `BACKUPX_AGENT_TOKEN`, and `BACKUPX_AGENT_TEMP_DIR=/var/lib/backupx-agent/tmp` environment contract. After starting the container, the installer also probes `/api/v1/agent/self`; if the node does not come online, it prints `docker ps` and `docker logs --tail=100 backupx-agent` diagnostics before exiting non-zero.
|
|
|
|
If you choose the URL-based fallback command and `curl` prints HTML or the shell reports `Syntax error: newline unexpected`, the install URL is being served by the web console instead of the backend. Ensure either `/api/install/` or `/install/` is forwarded to the BackupX backend, or use the embedded command generated by the console.
|
|
|
|
Reruns are idempotent — to upgrade or re-provision, simply generate a new install command and run it again. The one-time install link expires after its TTL or after first consumption, whichever is sooner.
|
|
|
|
### 3. Rotate agent tokens at any time
|
|
|
|
Go to the node's action menu (︙) → **Rotate Token**. The new token is shown once and the old token remains valid for 24 h, allowing rolling restarts without downtime. After 24 h, the old token is rejected.
|
|
|
|
### 4. Batch deployment
|
|
|
|
In Step 1 choose "Batch" and paste node names (one per line, max 50). Step 3 shows a table with one command per node plus a **Download .sh** button that bundles all commands into a shell script, convenient for SSH loops or Ansible tasks.
|
|
|
|
### 5. Route a task to the node
|
|
|
|
In the **Backup Tasks** page, pick the source server when creating the task. When the task runs:
|
|
|
|
- Local (`nodeId=0`) → Master executes in-process
|
|
- Remote node → Master enqueues the command → Agent claims → Agent runs locally → uploads → reports back
|
|
|
|
The node table shows the Agent health and command queue state: pending/dispatched depth, running long commands, timeouts, oldest active command age, and the latest Agent-side error. The same queue depth, running-command, and timeout snapshots are exported as Prometheus metrics:
|
|
|
|
- `backupx_agent_command_queue_depth`
|
|
- `backupx_agent_command_running`
|
|
- `backupx_agent_command_timeout_total`
|
|
|
|
## Known limitations
|
|
|
|
- **Encrypted backups are Master-only** — the Agent doesn't hold Master's AES-256 key. Creating or updating a task with `encrypt: true` and a remote node or node pool is rejected up front
|
|
- **Directory browser timeout** — remote dir listing is a synchronous RPC through the queue (15s default)
|
|
- **Dispatched command timeout** — claimed-but-unfinished commands are marked `timeout` after 10 minutes
|
|
|
|
## CLI reference
|
|
|
|
```
|
|
backupx agent --help
|
|
-master string Master URL
|
|
-token string Agent auth token
|
|
-config string YAML config path (takes precedence over env)
|
|
-temp-dir string Local temp directory (default /tmp/backupx-agent)
|
|
-insecure-tls Skip TLS verification (testing only)
|
|
```
|
|
|
|
## systemd unit
|
|
|
|
```ini title="/etc/systemd/system/backupx-agent.service"
|
|
[Unit]
|
|
Description=BackupX Agent
|
|
After=network.target
|
|
|
|
[Service]
|
|
Type=simple
|
|
User=backupx
|
|
Environment="BACKUPX_AGENT_MASTER=https://master.example.com"
|
|
Environment="BACKUPX_AGENT_TOKEN=your-token"
|
|
ExecStart=/opt/backupx/backupx agent
|
|
Restart=on-failure
|
|
RestartSec=10s
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|
|
```
|
|
|
|
Enable and start:
|
|
|
|
```bash
|
|
sudo systemctl enable --now backupx-agent
|
|
sudo journalctl -u backupx-agent -f
|
|
```
|