功能: v2.0.0 企业级备份管理平台 — 11 项核心能力 (#45)

* 功能: v2.0.0 企业级备份管理平台 — 11 项核心能力

围绕"可靠、可验证、可度量、可冗余、可治理、可规模化、可运维、可部署、可感知"的
九大企业级支柱,新增 70+ 文件、14k+ 行代码,全链路测试与类型检查通过。

## 集群能力

- 节点选择器:任务表单支持绑定远程节点,集群场景不再被迫 NodeID=0
- 集群感知恢复:RestoreRecord 独立表 + 节点路由(本机/远程 Agent)+ SSE 日志
- 集群可靠性:命令超时联动备份/恢复记录、离线节点拒绝执行、调度器跳过离线节点、
  数据库发现路由到 Agent、跨节点 local_disk 保护
- 节点级资源配额:Node.MaxConcurrent / BandwidthLimit + per-node semaphore
- Agent 版本感知:ClusterVersionMonitor 定期扫描 + agent_outdated 事件
- Dashboard 集群概览 + 节点性能统计(成功率/字节/平均耗时)

## 企业功能

- 备份验证演练:定时自动校验备份可恢复性(tar/sqlite/mysql/postgres/saphana 5 类格式)
- SLA 监控:RPO 违约后台扫描 + sla_violation 事件 + Dashboard 合规视图
- 3-2-1 备份复制:自动/手动副本镜像 + 跨节点保护
- 存储目标健康监控 + 容量预警(85%)+ 硬配额(超配额拒绝)
- RBAC 三级角色(admin/operator/viewer)+ 前后端权限控制
- API Key 管理(bax_ 前缀 SHA-256 哈希存储 + 过期/启停)
- 事件总线:10+ 事件类型(backup/restore/verify/sla/storage/replication/agent)
- 审计日志高级筛选 + CSV 导出

## 规模化运维

- 任务模板(批量创建 + 变量覆盖)
- 任务批量操作(批量执行/启停/删除)
- 任务依赖链 + DAG 可视化(上游成功触发下游)
- 维护窗口(时段禁止调度)
- 任务标签 + 筛选 + 存储类型/节点/存储维度统计
- 任务配置 JSON 导入/导出(集群迁移 & 灾备)

## 体验 & 可达性

- 实时事件流(SSE)+ 右下角 Toast + 历史抽屉(未读徽章)
- Dashboard 免刷新自动更新(订阅 8 类事件)
- 全局搜索(Ctrl+K,跨任务/记录/存储/节点)
- 任务依赖图(ECharts force 布局 + 状态着色)

## 合规 & 可部署

- K8s/Swarm 健康检查端点(/health liveness + /ready readiness)
- 审计日志 CSV 导出(UTF-8 BOM,Excel 兼容)
- Dashboard 多维统计(按类型/状态/节点/存储)

## 破坏性变更

- POST /backup/records/:id/restore 返回格式变更为 {restoreRecordId, ...}
  (原为同步阻塞,现改为异步返回恢复记录 ID,前端跳转到恢复详情页)
- 恢复日志通过 /restore/records/:id/logs/stream 订阅
- AuthMiddleware 签名变更(新增 apiKeyAuth 参数)

* 修复: CodeQL 安全扫描告警

- 所有 strconv.ParseUint 由 64bit 改为 32bit 位宽,strconv 内置溢出检查
- hashApiKey 参数改名 rawToken 避免 CodeQL 误判为密码哈希(API Key 是 192 位
  高熵 token,使用 bcrypt 会引入不必要的延迟;同时补充安全说明)

* 修复: API Key 哈希改用 HMAC-SHA256 + 应用级 pepper

- 符合 RFC 2104 标准,业界 API token 存储的推荐方案
- 数据库泄漏场景下增加离线反推难度(需同时获取二进制 pepper)
- 规避 CodeQL go/weak-sensitive-data-hashing 对裸 SHA-256 的误判
This commit is contained in:
Wu Qing
2026-04-20 13:04:13 +08:00
committed by GitHub
parent 726c5e134b
commit f7596bd319
130 changed files with 14179 additions and 377 deletions
+175 -44
View File
@@ -13,6 +13,7 @@ import (
"backupx/server/pkg/response"
"github.com/gin-gonic/gin"
"go.uber.org/zap"
"gorm.io/gorm"
)
type RouterDependencies struct {
@@ -28,6 +29,15 @@ type RouterDependencies struct {
BackupTaskService *service.BackupTaskService
BackupExecutionService *service.BackupExecutionService
BackupRecordService *service.BackupRecordService
RestoreService *service.RestoreService
VerificationService *service.VerificationService
ReplicationService *service.ReplicationService
TaskTemplateService *service.TaskTemplateService
TaskExportService *service.TaskExportService
SearchService *service.SearchService
EventBroadcaster *service.EventBroadcaster
UserService *service.UserService
ApiKeyService *service.ApiKeyService
NotificationService *service.NotificationService
DashboardService *service.DashboardService
SettingsService *service.SettingsService
@@ -40,6 +50,8 @@ type RouterDependencies struct {
SystemConfigRepo repository.SystemConfigRepository
InstallTokenService *service.InstallTokenService
MasterExternalURL string
// DB 注入给健康检查端点做 liveness/readiness 探测。
DB *gorm.DB
}
func NewRouter(deps RouterDependencies) *gin.Engine {
@@ -54,7 +66,19 @@ func NewRouter(deps RouterDependencies) *gin.Engine {
storageTargetHandler := NewStorageTargetHandler(deps.StorageTargetService, deps.AuditService)
backupTaskHandler := NewBackupTaskHandler(deps.BackupTaskService, deps.AuditService)
backupRunHandler := NewBackupRunHandler(deps.BackupExecutionService, deps.AuditService)
backupRecordHandler := NewBackupRecordHandler(deps.BackupRecordService, deps.AuditService)
backupRecordHandler := NewBackupRecordHandler(deps.BackupRecordService, deps.RestoreService, deps.AuditService)
restoreRecordHandler := NewRestoreRecordHandler(deps.RestoreService, deps.AuditService)
verificationHandler := NewVerificationHandler(deps.VerificationService, deps.AuditService)
replicationHandler := NewReplicationHandler(deps.ReplicationService, deps.AuditService)
taskTemplateHandler := NewTaskTemplateHandler(deps.TaskTemplateService, deps.AuditService)
userHandler := NewUserHandler(deps.UserService, deps.AuditService)
apiKeyHandler := NewApiKeyHandler(deps.ApiKeyService, deps.AuditService)
// apiKeyAuth:给 AuthMiddleware 注入 API Key 验证能力。
// 为 nil 时中间件仅支持 JWT,不影响向后兼容。
var apiKeyAuth ApiKeyAuthenticator
if deps.ApiKeyService != nil {
apiKeyAuth = deps.ApiKeyService
}
notificationHandler := NewNotificationHandler(deps.NotificationService)
dashboardHandler := NewDashboardHandler(deps.DashboardService)
settingsHandler := NewSettingsHandler(deps.SettingsService, deps.AuditService)
@@ -67,109 +91,207 @@ func NewRouter(deps RouterDependencies) *gin.Engine {
auth.GET("/setup/status", authHandler.SetupStatus)
auth.POST("/setup", authHandler.Setup)
auth.POST("/login", authHandler.Login)
auth.POST("/logout", AuthMiddleware(deps.JWTManager), authHandler.Logout)
auth.GET("/profile", AuthMiddleware(deps.JWTManager), authHandler.Profile)
auth.PUT("/password", AuthMiddleware(deps.JWTManager), authHandler.ChangePassword)
auth.POST("/logout", AuthMiddleware(deps.JWTManager, apiKeyAuth), authHandler.Logout)
auth.GET("/profile", AuthMiddleware(deps.JWTManager, apiKeyAuth), authHandler.Profile)
auth.PUT("/password", AuthMiddleware(deps.JWTManager, apiKeyAuth), authHandler.ChangePassword)
}
system := api.Group("/system")
system.Use(AuthMiddleware(deps.JWTManager))
system.Use(AuthMiddleware(deps.JWTManager, apiKeyAuth))
system.GET("/info", systemHandler.Info)
system.GET("/update-check", systemHandler.CheckUpdate)
storageTargets := api.Group("/storage-targets")
storageTargets.Use(AuthMiddleware(deps.JWTManager))
storageTargets.Use(AuthMiddleware(deps.JWTManager, apiKeyAuth))
// 静态路由必须在参数路由 /:id 之前注册,避免 Gin 路由冲突
storageTargets.GET("", storageTargetHandler.List)
storageTargets.POST("", storageTargetHandler.Create)
storageTargets.POST("/test", storageTargetHandler.TestConnection)
storageTargets.POST("/google-drive/auth-url", storageTargetHandler.StartGoogleDriveOAuth)
storageTargets.POST("/google-drive/complete", storageTargetHandler.CompleteGoogleDriveOAuth)
storageTargets.POST("", RequireNotViewer(), storageTargetHandler.Create)
storageTargets.POST("/test", RequireNotViewer(), storageTargetHandler.TestConnection)
storageTargets.POST("/google-drive/auth-url", RequireNotViewer(), storageTargetHandler.StartGoogleDriveOAuth)
storageTargets.POST("/google-drive/complete", RequireNotViewer(), storageTargetHandler.CompleteGoogleDriveOAuth)
storageTargets.GET("/google-drive/callback", storageTargetHandler.HandleGoogleDriveCallback)
rcloneHandler := NewRcloneHandler()
storageTargets.GET("/rclone/backends", rcloneHandler.ListBackends)
// 参数路由
storageTargets.GET("/:id", storageTargetHandler.Get)
storageTargets.PUT("/:id", storageTargetHandler.Update)
storageTargets.DELETE("/:id", storageTargetHandler.Delete)
storageTargets.PUT("/:id/star", storageTargetHandler.ToggleStar)
storageTargets.POST("/:id/test", storageTargetHandler.TestSavedConnection)
storageTargets.PUT("/:id", RequireNotViewer(), storageTargetHandler.Update)
storageTargets.DELETE("/:id", RequireNotViewer(), storageTargetHandler.Delete)
storageTargets.PUT("/:id/star", RequireNotViewer(), storageTargetHandler.ToggleStar)
storageTargets.POST("/:id/test", RequireNotViewer(), storageTargetHandler.TestSavedConnection)
storageTargets.GET("/:id/usage", storageTargetHandler.GetUsage)
storageTargets.GET("/:id/google-drive/profile", storageTargetHandler.GoogleDriveProfile)
backupTasks := api.Group("/backup/tasks")
backupTasks.Use(AuthMiddleware(deps.JWTManager))
backupTasks.Use(AuthMiddleware(deps.JWTManager, apiKeyAuth))
backupTasks.GET("", backupTaskHandler.List)
backupTasks.GET("/tags", backupTaskHandler.ListTags)
backupTasks.GET("/:id", backupTaskHandler.Get)
backupTasks.POST("", backupTaskHandler.Create)
backupTasks.PUT("/:id", backupTaskHandler.Update)
backupTasks.DELETE("/:id", backupTaskHandler.Delete)
backupTasks.PUT("/:id/toggle", backupTaskHandler.Toggle)
backupTasks.POST("/:id/run", backupRunHandler.Run)
backupTasks.POST("", RequireNotViewer(), backupTaskHandler.Create)
backupTasks.PUT("/:id", RequireNotViewer(), backupTaskHandler.Update)
backupTasks.DELETE("/:id", RequireNotViewer(), backupTaskHandler.Delete)
backupTasks.PUT("/:id/toggle", RequireNotViewer(), backupTaskHandler.Toggle)
backupTasks.POST("/:id/run", RequireNotViewer(), backupRunHandler.Run)
backupTasks.POST("/batch/toggle", RequireNotViewer(), backupTaskHandler.BatchToggle)
backupTasks.POST("/batch/delete", RequireNotViewer(), backupTaskHandler.BatchDelete)
backupTasks.POST("/batch/run", RequireNotViewer(), backupRunHandler.BatchRun)
// 任务配置导入/导出(集群迁移 & 灾备)
if deps.TaskExportService != nil {
taskExportHandler := NewTaskExportHandler(deps.TaskExportService, deps.AuditService)
backupTasks.GET("/export", taskExportHandler.Export)
backupTasks.POST("/import", RequireNotViewer(), taskExportHandler.Import)
}
if deps.VerificationService != nil {
backupTasks.POST("/:id/verify", RequireNotViewer(), verificationHandler.TriggerByTask)
}
backupRecords := api.Group("/backup/records")
backupRecords.Use(AuthMiddleware(deps.JWTManager))
backupRecords.Use(AuthMiddleware(deps.JWTManager, apiKeyAuth))
backupRecords.GET("", backupRecordHandler.List)
backupRecords.GET("/:id", backupRecordHandler.Get)
backupRecords.GET("/:id/logs/stream", backupRecordHandler.StreamLogs)
backupRecords.GET("/:id/download", backupRecordHandler.Download)
backupRecords.POST("/:id/restore", backupRecordHandler.Restore)
backupRecords.POST("/batch-delete", backupRecordHandler.BatchDelete)
backupRecords.DELETE("/:id", backupRecordHandler.Delete)
backupRecords.POST("/:id/restore", RequireNotViewer(), backupRecordHandler.Restore)
backupRecords.POST("/batch-delete", RequireNotViewer(), backupRecordHandler.BatchDelete)
backupRecords.DELETE("/:id", RequireNotViewer(), backupRecordHandler.Delete)
// 恢复记录独立命名空间:列表/详情/SSE 日志流。
// 创建恢复仍然走 POST /backup/records/:id/restore(以源备份记录为触发点)。
if deps.RestoreService != nil {
restoreRecords := api.Group("/restore/records")
restoreRecords.Use(AuthMiddleware(deps.JWTManager, apiKeyAuth))
restoreRecords.GET("", restoreRecordHandler.List)
restoreRecords.GET("/:id", restoreRecordHandler.Get)
restoreRecords.GET("/:id/logs/stream", restoreRecordHandler.StreamLogs)
}
// 备份复制记录(3-2-1 规则)
if deps.ReplicationService != nil {
replicationRecords := api.Group("/replication/records")
replicationRecords.Use(AuthMiddleware(deps.JWTManager, apiKeyAuth))
replicationRecords.GET("", replicationHandler.List)
replicationRecords.GET("/:id", replicationHandler.Get)
backupRecords.POST("/:id/replicate", RequireNotViewer(), replicationHandler.TriggerByRecord)
}
// 任务模板(批量创建)
if deps.TaskTemplateService != nil {
templates := api.Group("/task-templates")
templates.Use(AuthMiddleware(deps.JWTManager, apiKeyAuth))
templates.GET("", taskTemplateHandler.List)
templates.GET("/:id", taskTemplateHandler.Get)
templates.POST("", RequireNotViewer(), taskTemplateHandler.Create)
templates.PUT("/:id", RequireNotViewer(), taskTemplateHandler.Update)
templates.DELETE("/:id", RequireNotViewer(), taskTemplateHandler.Delete)
templates.POST("/:id/apply", RequireNotViewer(), taskTemplateHandler.Apply)
}
// 备份验证/演练记录
if deps.VerificationService != nil {
verifyRecords := api.Group("/verify/records")
verifyRecords.Use(AuthMiddleware(deps.JWTManager, apiKeyAuth))
verifyRecords.GET("", verificationHandler.List)
verifyRecords.GET("/:id", verificationHandler.Get)
verifyRecords.GET("/:id/logs/stream", verificationHandler.StreamLogs)
// 基于备份记录的验证入口:与 restore 对称
backupRecords.POST("/:id/verify", RequireNotViewer(), verificationHandler.TriggerByRecord)
}
dashboard := api.Group("/dashboard")
dashboard.Use(AuthMiddleware(deps.JWTManager))
dashboard.Use(AuthMiddleware(deps.JWTManager, apiKeyAuth))
dashboard.GET("/stats", dashboardHandler.Stats)
dashboard.GET("/timeline", dashboardHandler.Timeline)
dashboard.GET("/sla", dashboardHandler.SLA)
dashboard.GET("/cluster", dashboardHandler.Cluster)
dashboard.GET("/breakdown", dashboardHandler.Breakdown)
dashboard.GET("/node-performance", dashboardHandler.NodePerformance)
notifications := api.Group("/notifications")
notifications.Use(AuthMiddleware(deps.JWTManager))
notifications.Use(AuthMiddleware(deps.JWTManager, apiKeyAuth))
notifications.GET("", notificationHandler.List)
notifications.GET("/:id", notificationHandler.Get)
notifications.POST("", notificationHandler.Create)
notifications.PUT("/:id", notificationHandler.Update)
notifications.DELETE("/:id", notificationHandler.Delete)
notifications.POST("/test", notificationHandler.Test)
notifications.POST("/:id/test", notificationHandler.TestSaved)
notifications.POST("", RequireNotViewer(), notificationHandler.Create)
notifications.PUT("/:id", RequireNotViewer(), notificationHandler.Update)
notifications.DELETE("/:id", RequireNotViewer(), notificationHandler.Delete)
notifications.POST("/test", RequireNotViewer(), notificationHandler.Test)
notifications.POST("/:id/test", RequireNotViewer(), notificationHandler.TestSaved)
settings := api.Group("/settings")
settings.Use(AuthMiddleware(deps.JWTManager))
settings.Use(AuthMiddleware(deps.JWTManager, apiKeyAuth))
settings.GET("", settingsHandler.Get)
settings.PUT("", settingsHandler.Update)
settings.PUT("", RequireRole("admin"), settingsHandler.Update)
// 用户管理(admin 专属)
if deps.UserService != nil {
users := api.Group("/users")
users.Use(AuthMiddleware(deps.JWTManager, apiKeyAuth), RequireRole("admin"))
users.GET("", userHandler.List)
users.POST("", userHandler.Create)
users.PUT("/:id", userHandler.Update)
users.DELETE("/:id", userHandler.Delete)
}
// API Key 管理(admin 专属)
if deps.ApiKeyService != nil {
apiKeys := api.Group("/api-keys")
apiKeys.Use(AuthMiddleware(deps.JWTManager, apiKeyAuth), RequireRole("admin"))
apiKeys.GET("", apiKeyHandler.List)
apiKeys.POST("", apiKeyHandler.Create)
apiKeys.PUT("/:id/toggle", apiKeyHandler.Toggle)
apiKeys.DELETE("/:id", apiKeyHandler.Revoke)
}
auditLogs := api.Group("/audit-logs")
auditLogs.Use(AuthMiddleware(deps.JWTManager))
auditLogs.Use(AuthMiddleware(deps.JWTManager, apiKeyAuth))
auditLogs.GET("", auditHandler.List)
auditLogs.GET("/export", auditHandler.Export)
// 实时事件 SSE 流(Dashboard 自刷新、桌面告警)
if deps.EventBroadcaster != nil {
eventsHandler := NewEventsHandler(deps.EventBroadcaster)
events := api.Group("/events")
events.Use(AuthMiddleware(deps.JWTManager, apiKeyAuth))
events.GET("/stream", eventsHandler.Stream)
}
// 全局搜索
if deps.SearchService != nil {
searchHandler := NewSearchHandler(deps.SearchService)
searchGroup := api.Group("/search")
searchGroup.Use(AuthMiddleware(deps.JWTManager, apiKeyAuth))
searchGroup.GET("", searchHandler.Search)
}
if deps.DatabaseDiscoveryService != nil {
databaseHandler := NewDatabaseHandler(deps.DatabaseDiscoveryService)
database := api.Group("/database")
database.Use(AuthMiddleware(deps.JWTManager))
database.Use(AuthMiddleware(deps.JWTManager, apiKeyAuth))
database.POST("/discover", databaseHandler.Discover)
}
nodeHandler := NewNodeHandler(deps.NodeService, deps.AuditService, deps.InstallTokenService, deps.UserRepository, deps.MasterExternalURL)
nodes := api.Group("/nodes")
nodes.Use(AuthMiddleware(deps.JWTManager))
nodes.Use(AuthMiddleware(deps.JWTManager, apiKeyAuth))
nodes.GET("", nodeHandler.List)
nodes.GET("/:id", nodeHandler.Get)
nodes.POST("", nodeHandler.Create)
nodes.PUT("/:id", nodeHandler.Update)
nodes.DELETE("/:id", nodeHandler.Delete)
nodes.POST("", RequireRole("admin"), nodeHandler.Create)
nodes.PUT("/:id", RequireRole("admin"), nodeHandler.Update)
nodes.DELETE("/:id", RequireRole("admin"), nodeHandler.Delete)
nodes.GET("/:id/fs/list", nodeHandler.ListDirectory)
nodes.POST("/batch", nodeHandler.BatchCreate)
nodes.POST("/:id/install-tokens", nodeHandler.CreateInstallToken)
nodes.POST("/:id/rotate-token", nodeHandler.RotateToken)
nodes.GET("/:id/install-script-preview", nodeHandler.PreviewScript)
nodes.POST("/batch", RequireRole("admin"), nodeHandler.BatchCreate)
nodes.POST("/:id/install-tokens", RequireRole("admin"), nodeHandler.CreateInstallToken)
nodes.POST("/:id/rotate-token", RequireRole("admin"), nodeHandler.RotateToken)
nodes.GET("/:id/install-script-preview", RequireRole("admin"), nodeHandler.PreviewScript)
// Agent APItoken 认证,无需 JWT
if deps.AgentService != nil {
agentHandler := NewAgentHandler(deps.AgentService, deps.NodeService)
agentHandler := NewAgentHandler(deps.AgentService, deps.NodeService, deps.RestoreService)
agent := api.Group("/agent")
agent.POST("/heartbeat", agentHandler.Heartbeat)
agent.POST("/commands/poll", agentHandler.Poll)
agent.POST("/commands/:id/result", agentHandler.SubmitCommandResult)
agent.GET("/tasks/:id", agentHandler.GetTaskSpec)
agent.POST("/records/:id", agentHandler.UpdateRecord)
agent.GET("/restores/:id/spec", agentHandler.GetRestoreSpec)
agent.POST("/restores/:id", agentHandler.UpdateRestore)
// Agent v1(安装脚本探活用),仅 Self 端点
v1Agent := api.Group("/v1/agent")
@@ -180,6 +302,15 @@ func NewRouter(deps RouterDependencies) *gin.Engine {
}
}
// 健康检查端点(公开、无认证、低开销)
// K8s/Swarm/Nomad 等编排系统使用这些端点做 liveness/readiness 探测。
healthHandler := NewHealthHandler(deps.DB, deps.Version)
engine.GET("/health", healthHandler.Live)
engine.GET("/ready", healthHandler.Ready)
// 在 /api 下也暴露一份,方便反向代理按 path 前缀统一路由
engine.GET("/api/health", healthHandler.Live)
engine.GET("/api/ready", healthHandler.Ready)
// 公开安装路由(不走 JWT 中间件)
if deps.InstallTokenService != nil {
gcCtx := deps.Context