mirror of
https://github.com/isboyjc/GoProxy.git
synced 2026-05-07 05:12:42 +08:00
feat: ✨ init
This commit is contained in:
30
.gitignore
vendored
30
.gitignore
vendored
@@ -1,17 +1,41 @@
|
||||
# macOS
|
||||
.DS_Store
|
||||
|
||||
# Build artifacts
|
||||
# Windows
|
||||
Thumbs.db
|
||||
|
||||
# Go build artifacts
|
||||
goproxy
|
||||
proxy-pool
|
||||
*.exe
|
||||
*.out
|
||||
*.test
|
||||
vendor/
|
||||
|
||||
# Test binaries
|
||||
test/test_proxy
|
||||
test/test_proxy.exe
|
||||
|
||||
# Runtime data
|
||||
*.db
|
||||
*.db-shm
|
||||
*.db-wal
|
||||
*.log
|
||||
config.json
|
||||
data/*.db
|
||||
data/config.json
|
||||
data/
|
||||
!data/.gitkeep
|
||||
|
||||
# Temporary files
|
||||
*.tmp
|
||||
*.bak
|
||||
*.swp
|
||||
*~
|
||||
|
||||
# IDE files
|
||||
.idea/
|
||||
.vscode/
|
||||
.cursor/
|
||||
*.iml
|
||||
|
||||
# Docker volumes
|
||||
.docker-data/
|
||||
|
||||
827
README.md
827
README.md
@@ -1,42 +1,88 @@
|
||||
# ProxyGo
|
||||
# GoProxy
|
||||
|
||||
一个基于 Go 的轻量代理池服务。程序会从公开代理源抓取 HTTP/SOCKS5 代理,验证可用性后写入 SQLite,并对外暴露一个统一的本地 HTTP 代理入口,同时提供带登录的 Web 管理后台。
|
||||
> **智能代理池系统** — 基于 Go 的轻量级、低资源消耗、自适应的代理池服务
|
||||
|
||||
## 功能概览
|
||||
goproxy 从多个公开代理源自动抓取 HTTP/SOCKS5 代理,通过严格验证(出口 IP + 位置 + 延迟)后加入智能代理池,对外提供统一的 HTTP 代理服务。系统采用质量分级、智能补充、自动优化等机制,确保代理池始终保持高质量和稳定性。
|
||||
|
||||
- 启动时自动抓取并验证代理
|
||||
- 后台定时抓取新代理
|
||||
- 后台定时健康检查,自动清理不可用代理
|
||||
- 聚合 HTTP 和 SOCKS5 上游代理,对外统一提供 HTTP 代理端口
|
||||
- 支持普通 HTTP 请求和 HTTPS `CONNECT` 隧道转发
|
||||
- 内置 WebUI,支持查看统计、筛选代理、删除代理、手动触发抓取、查看日志、修改部分运行参数
|
||||
- 使用 SQLite 持久化代理池数据
|
||||
## ✨ 核心特性
|
||||
|
||||
## 项目结构
|
||||
### 🎯 智能池子机制
|
||||
- **固定容量管理**:可配置池子大小和 HTTP/SOCKS5 协议比例
|
||||
- **质量分级**:S/A/B/C 四级评分(基于延迟),智能选择高质量代理
|
||||
- **动态状态感知**:Healthy → Warning → Critical → Emergency 四级状态自适应
|
||||
- **严格准入标准**:必须通过出口 IP、地理位置、延迟三重验证才可入池
|
||||
- **智能替换**:新代理必须显著优于现有代理(默认快 30%)才触发替换
|
||||
|
||||
### 🚀 按需抓取
|
||||
- **源分组策略**:快更新源(5-30min)用于紧急补充,慢更新源(每天)用于优化轮换
|
||||
- **断路器保护**:连续失败的源自动降级/禁用,冷却后恢复
|
||||
- **多模式抓取**:
|
||||
- **Emergency**:单协议缺失或池子 <10%,使用所有可用源
|
||||
- **Refill**:池子 <80%,使用快更新源
|
||||
- **Optimize**:池子健康时,随机抽取少量慢源优化替换
|
||||
|
||||
### 🏥 分层健康管理
|
||||
- **轻量批次检查**:每次仅检查 20 个代理,避免资源浪费
|
||||
- **智能跳过 S 级**:池子健康时跳过 S 级代理检查
|
||||
- **定时优化轮换**:健康状态下,定期抓取优质代理替换池中延迟高的
|
||||
|
||||
### 🔄 智能重试机制
|
||||
- **自动故障切换**:代理请求失败时立即切换到另一个代理重试(最多 3 次)
|
||||
- **失败即删除**:连接失败或请求超时的代理立即从池子中移除
|
||||
- **用户无感知**:自动重试在服务端完成,用户只会收到成功响应或最终失败提示
|
||||
- **防重复尝试**:已尝试过的失败代理不会在同一请求中再次使用
|
||||
|
||||
### 🚪 双端口策略
|
||||
- **7777 端口(随机轮换)**:每次请求随机选择代理,IP 高度分散,适合爬虫和数据采集
|
||||
- **7776 端口(最低延迟)**:固定使用延迟最低的代理,除非失败才切换,适合长连接和流媒体
|
||||
- **自动切换**:两个端口都支持失败自动重试,7776 失败后切换到次优代理
|
||||
- **共享池子**:两个端口使用同一个代理池,统一管理和优化
|
||||
|
||||
### 🎨 黑客风格 WebUI
|
||||
- **Matrix 美学**:荧光绿 + 纯黑背景,CRT 扫描线效果,JetBrains Mono 等宽字体
|
||||
- **双角色权限**:访客模式(只读)+ 管理员模式(完全控制),可安全公网开放
|
||||
- **实时仪表盘**:池子状态、质量分布可视化、协议统计,带荧光光晕效果
|
||||
- **完整配置界面**:池子容量、延迟标准、验证参数、优化策略均可在线调整(管理员)
|
||||
- **代理注册表**:详细展示地址、出口 IP、位置、延迟、质量等级、使用统计
|
||||
- **中英文切换**:支持中文/英文界面切换,默认中文
|
||||
- **交互优化**:点击地址复制、单个代理刷新、实时日志倒计时
|
||||
|
||||
### 📊 适用场景
|
||||
- **小型 VPS**:低资源消耗(固定池子 + 按需抓取 + 限流查询)
|
||||
- **稳定需求**:自动剔除失败代理,始终保持健康池子
|
||||
- **质量优先**:S/A 级代理优先使用,自动优化延迟
|
||||
|
||||
## 📦 项目结构
|
||||
|
||||
```text
|
||||
.
|
||||
├── main.go # 程序入口
|
||||
├── config/ # 默认配置、配置加载与保存
|
||||
├── fetcher/ # 代理源抓取
|
||||
├── validator/ # 代理可用性验证
|
||||
├── checker/ # 周期健康检查
|
||||
├── storage/ # SQLite 存储
|
||||
├── main.go # 程序入口,协调所有模块
|
||||
├── config/ # 配置系统(池子容量、延迟标准、验证参数等)
|
||||
├── pool/ # 🆕 池子管理器(入池判断、替换逻辑、状态计算)
|
||||
├── fetcher/ # 🆕 智能抓取器(源分组、断路器、按需抓取)
|
||||
│ ├── fetcher.go # 多模式抓取逻辑
|
||||
│ ├── source_manager.go # 源状态管理和断路器
|
||||
│ └── ip_query.go # IP查询限流和多源降级
|
||||
├── validator/ # 代理验证(连接测试 + 出口IP检测)
|
||||
├── checker/ # 🆕 分批健康检查器
|
||||
├── optimizer/ # 🆕 优化轮换器(定时优化池子质量)
|
||||
├── storage/ # 🆕 扩展存储层(质量等级、使用统计、源状态表)
|
||||
├── proxy/ # 对外 HTTP 代理服务
|
||||
├── webui/ # 登录页、仪表盘、API
|
||||
├── logger/ # 内存日志 + stdout 输出
|
||||
├── Dockerfile
|
||||
└── docker-compose.yml
|
||||
├── webui/ # 🆕 黑客风格 WebUI(健康仪表盘、配置界面)
|
||||
├── logger/ # 内存日志收集
|
||||
├── test/ # 🧪 测试脚本(Bash/Go/Python)
|
||||
│ ├── test_proxy.sh # Bash 测试脚本
|
||||
│ ├── test_proxy.go # Go 测试脚本
|
||||
│ ├── test_proxy.py # Python 测试脚本
|
||||
│ └── TEST_GUIDE.md # 测试指南
|
||||
└── POOL_DESIGN.md # 🆕 完整架构设计文档
|
||||
```
|
||||
|
||||
## 运行要求
|
||||
## 🚀 快速开始
|
||||
|
||||
### 运行要求
|
||||
- Go `1.25`
|
||||
- 需要可用的 CGO 编译环境
|
||||
- 项目依赖 `github.com/mattn/go-sqlite3`
|
||||
- 本地构建通常需要 `gcc` / Xcode Command Line Tools
|
||||
|
||||
## 快速开始
|
||||
- CGO 编译环境(依赖 `github.com/mattn/go-sqlite3`)
|
||||
|
||||
### 本地运行
|
||||
|
||||
@@ -47,40 +93,100 @@ go run .
|
||||
或先编译再启动:
|
||||
|
||||
```bash
|
||||
go build -o proxy-pool .
|
||||
./proxy-pool
|
||||
go build -o proxygo .
|
||||
./proxygo
|
||||
```
|
||||
|
||||
程序启动后会:
|
||||
|
||||
1. 加载默认配置或读取 `config.json`
|
||||
2. 初始化 SQLite 数据库
|
||||
3. 启动 WebUI
|
||||
4. 立即抓取一次代理并开始验证
|
||||
5. 启动定时抓取和健康检查
|
||||
6. 在 `:7777` 启动统一代理服务
|
||||
1. 加载配置(优先 `config.json`)
|
||||
2. 初始化数据库和限流器
|
||||
3. 启动 WebUI(`:7778`)
|
||||
4. 立即执行智能填充(按需抓取 + 严格验证)
|
||||
5. 启动后台协程:
|
||||
- 状态监控(每分钟)
|
||||
- 健康检查(默认 5 分钟)
|
||||
- 优化轮换(默认 30 分钟)
|
||||
6. 启动两个代理服务:
|
||||
- `:7776` - 最低延迟模式(稳定连接)
|
||||
- `:7777` - 随机轮换模式(IP 多样性)
|
||||
|
||||
### 默认端口
|
||||
- **代理服务(随机轮换)**:`127.0.0.1:7777` - 每次请求随机选择代理,IP 多样性高
|
||||
- **代理服务(最低延迟)**:`127.0.0.1:7776` - 固定使用延迟最低的代理,性能优先
|
||||
- **WebUI**:`http://127.0.0.1:7778`
|
||||
- **默认密码**:`goproxy`(可通过 `WEBUI_PASSWORD` 环境变量自定义)
|
||||
|
||||
- 代理服务:`127.0.0.1:7777` 或 `:7777`
|
||||
- WebUI:`http://127.0.0.1:7778`
|
||||
### 使用代理
|
||||
|
||||
### 使用聚合代理
|
||||
GoProxy 提供**两个代理端口**,满足不同场景需求:
|
||||
|
||||
例如:
|
||||
#### 🎲 7777 端口 - 随机轮换模式
|
||||
|
||||
适合需要 **IP 多样性** 的场景(爬虫、数据采集、负载均衡):
|
||||
|
||||
```bash
|
||||
curl -x http://127.0.0.1:7777 https://httpbin.org/ip
|
||||
```
|
||||
|
||||
也可以给命令行程序设置环境变量:
|
||||
**特点**:
|
||||
- 每次请求随机选择一个代理
|
||||
- 优先使用高质量(S/A 级)代理
|
||||
- IP 地址高度分散
|
||||
|
||||
#### ⚡ 7776 端口 - 最低延迟模式
|
||||
|
||||
适合需要 **稳定连接** 的场景(长连接、流媒体、实时通信):
|
||||
|
||||
```bash
|
||||
export http_proxy=http://127.0.0.1:7777
|
||||
export https_proxy=http://127.0.0.1:7777
|
||||
curl -x http://127.0.0.1:7776 https://httpbin.org/ip
|
||||
```
|
||||
|
||||
## Docker
|
||||
**特点**:
|
||||
- 固定使用池中延迟最低的代理
|
||||
- 除非该代理失败,否则不会切换
|
||||
- 失败时自动删除并切换到下一个最低延迟代理
|
||||
- 性能和稳定性最优
|
||||
|
||||
#### 环境变量配置
|
||||
|
||||
```bash
|
||||
# 使用随机模式
|
||||
export http_proxy=http://127.0.0.1:7777
|
||||
export https_proxy=http://127.0.0.1:7777
|
||||
|
||||
# 或使用稳定模式
|
||||
export http_proxy=http://127.0.0.1:7776
|
||||
export https_proxy=http://127.0.0.1:7776
|
||||
```
|
||||
|
||||
#### 端口对比
|
||||
|
||||
| 特性 | 7777(随机轮换) | 7776(最低延迟) |
|
||||
|------|-----------------|-----------------|
|
||||
| **选择策略** | 随机选择(优先高质量) | 固定使用延迟最低的 |
|
||||
| **IP 多样性** | ⭐⭐⭐⭐⭐ 高度分散 | ⭐ 基本固定 |
|
||||
| **连接稳定性** | ⭐⭐⭐ 每次切换 | ⭐⭐⭐⭐⭐ 固定不变 |
|
||||
| **性能表现** | ⭐⭐⭐⭐ 平均延迟 | ⭐⭐⭐⭐⭐ 最优延迟 |
|
||||
| **适用场景** | 爬虫、数据采集、防封禁 | 长连接、流媒体、下载 |
|
||||
| **失败切换** | 自动重试 3 次 | 失败后切换到次优代理 |
|
||||
|
||||
#### 自动重试机制说明
|
||||
|
||||
当你通过 GoProxy 发送请求时,如果上游代理失败,系统会**自动处理**:
|
||||
|
||||
1. **立即删除失败代理**:从池子中移除不可用的代理
|
||||
2. **自动切换重试**:随机选择另一个可用代理重新发送请求(最多重试 3 次)
|
||||
3. **用户完全无感知**:整个过程在服务端完成,你的应用只会收到成功响应或最终失败提示
|
||||
4. **防止重复尝试**:同一请求中不会重复使用已失败的代理
|
||||
|
||||
**示例流程**:
|
||||
```
|
||||
用户请求 → 代理A失败(删除) → 自动切换代理B → 代理B成功 → 返回响应
|
||||
```
|
||||
|
||||
这意味着即使池子中有部分失效代理,你的应用依然可以正常工作,系统会自动保持池子质量。
|
||||
|
||||
## 🐳 Docker 部署
|
||||
|
||||
### 使用 Dockerfile
|
||||
|
||||
@@ -88,170 +194,597 @@ export https_proxy=http://127.0.0.1:7777
|
||||
docker build -t proxygo .
|
||||
docker run -d \
|
||||
--name proxygo \
|
||||
-p 127.0.0.1:7776:7776 \
|
||||
-p 127.0.0.1:7777:7777 \
|
||||
-p 7778:7778 \
|
||||
-e TZ=Asia/Shanghai \
|
||||
-e WEBUI_PASSWORD=your_password \
|
||||
-e DATA_DIR=/app/data \
|
||||
-v "$(pwd)/data:/app/data" \
|
||||
proxygo
|
||||
```
|
||||
|
||||
### 使用 docker-compose.yml
|
||||
### 使用 docker-compose
|
||||
|
||||
```bash
|
||||
docker compose up -d --build
|
||||
```
|
||||
|
||||
当前仓库中的 `docker-compose.yml` 有两个前提:
|
||||
### WebUI 公网访问配置
|
||||
|
||||
- 它会将 `./data` 挂载到容器内 `/app/data`
|
||||
- 它依赖一个已存在的外部网络 `cursor2api_default`
|
||||
得益于**双角色权限系统**,WebUI 可以安全地对外开放:
|
||||
|
||||
如果宿主机没有这个网络,需要先创建,或者直接修改 `docker-compose.yml` 中的网络配置。
|
||||
```bash
|
||||
docker run -d \
|
||||
--name proxygo \
|
||||
-p 127.0.0.1:7776:7776 \
|
||||
-p 127.0.0.1:7777:7777 \
|
||||
-p 0.0.0.0:7778:7778 \ # 公网访问 WebUI
|
||||
-e TZ=Asia/Shanghai \
|
||||
-e WEBUI_PASSWORD=strong_password \
|
||||
-e DATA_DIR=/app/data \
|
||||
-v "$(pwd)/data:/app/data" \
|
||||
proxygo
|
||||
```
|
||||
|
||||
## 数据目录
|
||||
**安全说明**:
|
||||
- 访客(未登录)只能查看数据,无法执行任何操作
|
||||
- 所有写操作(抓取、删除、配置修改)都需要管理员密码
|
||||
- 建议设置强密码(通过 `WEBUI_PASSWORD` 环境变量)
|
||||
- 代理服务端口(7776、7777)建议仅绑定内网(`127.0.0.1`)
|
||||
|
||||
程序支持通过 `DATA_DIR` 指定数据目录。
|
||||
## ⚙️ 配置说明
|
||||
|
||||
- 未设置 `DATA_DIR` 时:
|
||||
- 数据库默认写到项目根目录 `proxy.db`
|
||||
- 配置文件默认读取/写入项目根目录 `config.json`
|
||||
- 设置 `DATA_DIR=/app/data` 时:
|
||||
- 数据库路径变为 `/app/data/proxy.db`
|
||||
- 配置文件路径变为 `/app/data/config.json`
|
||||
### 配置文件示例
|
||||
|
||||
## 配置说明
|
||||
|
||||
### 可持久化配置
|
||||
|
||||
当前版本只会从 `config.json` 读取并保存以下 4 个字段:
|
||||
所有配置均可通过 WebUI 的 **Configure Pool** 界面在线调整,也可以手动编辑 `config.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"fetch_interval": 30,
|
||||
"check_interval": 10,
|
||||
"pool_max_size": 100,
|
||||
"pool_http_ratio": 0.5,
|
||||
"pool_min_per_protocol": 10,
|
||||
"max_latency_ms": 2000,
|
||||
"max_latency_healthy": 1500,
|
||||
"max_latency_emergency": 3000,
|
||||
"validate_concurrency": 300,
|
||||
"validate_timeout": 3
|
||||
"validate_timeout": 8,
|
||||
"health_check_interval": 5,
|
||||
"health_check_batch_size": 20,
|
||||
"optimize_interval": 30,
|
||||
"replace_threshold": 0.7
|
||||
}
|
||||
```
|
||||
|
||||
字段含义:
|
||||
### 配置参数详解
|
||||
|
||||
- `fetch_interval`:定时抓取间隔,单位分钟
|
||||
- `check_interval`:健康检查间隔,单位分钟
|
||||
- `validate_concurrency`:并发验证数量
|
||||
- `validate_timeout`:单个代理验证超时,单位秒
|
||||
**服务端口配置**
|
||||
|
||||
这些参数既可以通过编辑 `config.json` 修改,也可以在 WebUI 的“系统设置”中在线保存。
|
||||
| 参数 | 默认值 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `proxy_port` | `:7777` | 随机轮换代理端口 |
|
||||
| `stable_proxy_port` | `:7776` | 最低延迟代理端口 |
|
||||
| `webui_port` | `:7778` | WebUI 端口 |
|
||||
|
||||
### 当前代码中的默认值
|
||||
**池子容量配置**
|
||||
|
||||
除上面 4 项外,其余配置目前来自代码默认值:
|
||||
| 参数 | 默认值 | 说明 | 推荐范围 |
|
||||
| --- | --- | --- | --- |
|
||||
| `pool_max_size` | `100` | 代理池总容量 | 50-500 |
|
||||
| `pool_http_ratio` | `0.5` | HTTP 协议占比 | 0.3-0.8 |
|
||||
| `pool_min_per_protocol` | `10` | 每协议最少保证数量 | 5-50 |
|
||||
|
||||
| 配置项 | 默认值 | 说明 |
|
||||
**延迟标准配置**
|
||||
|
||||
| 参数 | 默认值 | 说明 | 推荐范围 |
|
||||
| --- | --- | --- | --- |
|
||||
| `max_latency_ms` | `2000` | 标准模式最大延迟(毫秒) | 1000-3000 |
|
||||
| `max_latency_healthy` | `1500` | 健康模式严格延迟(毫秒) | 800-2000 |
|
||||
| `max_latency_emergency` | `3000` | 紧急模式放宽延迟(毫秒) | 2000-5000 |
|
||||
|
||||
**验证与健康检查配置**
|
||||
|
||||
| 参数 | 默认值 | 说明 | 推荐范围 |
|
||||
| --- | --- | --- | --- |
|
||||
| `validate_concurrency` | `300` | 并发验证数量 | 100-500 |
|
||||
| `validate_timeout` | `8` | 验证超时(秒) | 5-15 |
|
||||
| `health_check_interval` | `5` | 检查间隔(分钟) | 3-15 |
|
||||
| `health_check_batch_size` | `20` | 每批检查数量 | 10-50 |
|
||||
|
||||
**优化配置**
|
||||
|
||||
| 参数 | 默认值 | 说明 | 推荐范围 |
|
||||
| --- | --- | --- | --- |
|
||||
| `optimize_interval` | `30` | 优化轮换间隔(分钟) | 15-120 |
|
||||
| `replace_threshold` | `0.7` | 替换阈值(新代理需快 30%) | 0.5-0.9 |
|
||||
|
||||
### 不同场景配置建议
|
||||
|
||||
**小型 VPS(1C2G)**
|
||||
```json
|
||||
{
|
||||
"pool_max_size": 50,
|
||||
"pool_http_ratio": 0.5,
|
||||
"validate_concurrency": 100,
|
||||
"health_check_interval": 10,
|
||||
"health_check_batch_size": 10,
|
||||
"optimize_interval": 60
|
||||
}
|
||||
```
|
||||
|
||||
**中型服务器(2C4G+)**
|
||||
```json
|
||||
{
|
||||
"pool_max_size": 200,
|
||||
"pool_http_ratio": 0.6,
|
||||
"validate_concurrency": 300,
|
||||
"health_check_interval": 5,
|
||||
"health_check_batch_size": 30,
|
||||
"optimize_interval": 30
|
||||
}
|
||||
```
|
||||
|
||||
**低延迟优先**
|
||||
```json
|
||||
{
|
||||
"pool_max_size": 100,
|
||||
"max_latency_ms": 1000,
|
||||
"max_latency_healthy": 800,
|
||||
"optimize_interval": 15,
|
||||
"replace_threshold": 0.8
|
||||
}
|
||||
```
|
||||
|
||||
**高可用优先(需要更多代理)**
|
||||
```json
|
||||
{
|
||||
"pool_max_size": 300,
|
||||
"pool_http_ratio": 0.7,
|
||||
"pool_min_per_protocol": 20,
|
||||
"max_latency_ms": 3000
|
||||
}
|
||||
```
|
||||
|
||||
### 固定配置
|
||||
|
||||
以下配置在代码中固定,无需调整:
|
||||
|
||||
| 配置项 | 值 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `WebUIPort` | `:7778` | Web 管理后台端口 |
|
||||
| `ProxyPort` | `:7777` | 对外统一代理端口 |
|
||||
| `DBPath` | `proxy.db` 或 `${DATA_DIR}/proxy.db` | SQLite 数据库路径 |
|
||||
| `ValidateURL` | `https://cursor.com/api/auth/me` | 验证目标地址 |
|
||||
| `MaxResponseMs` | `2500` | 最大可接受延迟,毫秒 |
|
||||
| `MaxFailCount` | `3` | 失败阈值字段已定义,但当前运行逻辑未完整使用 |
|
||||
| `MaxRetry` | `3` | 请求失败后的重试次数 |
|
||||
| `ValidateURL` | `http://www.gstatic.com/generate_204` | 验证目标地址 |
|
||||
| `IPQueryRateLimit` | `10` | IP 查询限流(次/秒) |
|
||||
| `SourceFailThreshold` | `3` | 源降级阈值 |
|
||||
| `SourceDisableThreshold` | `5` | 源禁用阈值 |
|
||||
| `SourceCooldownMinutes` | `30` | 源禁用冷却时间(分钟) |
|
||||
|
||||
## WebUI
|
||||
## 🎨 WebUI 使用指南
|
||||
|
||||
访问地址:
|
||||
访问地址:`http://127.0.0.1:7778`
|
||||
|
||||
- `http://127.0.0.1:7778`
|
||||
### 👥 双角色权限系统
|
||||
|
||||
提供的功能:
|
||||
GoProxy WebUI 支持**访客模式**和**管理员模式**:
|
||||
|
||||
- 登录鉴权
|
||||
- 展示代理总数、HTTP 数量、SOCKS5 数量
|
||||
- 按协议筛选代理
|
||||
- 删除单个代理
|
||||
- 手动触发抓取
|
||||
- 查看最近日志
|
||||
- 在线修改抓取/校验参数
|
||||
#### 访客模式(只读)
|
||||
|
||||
### 登录密码说明
|
||||
**无需登录**即可访问,可以查看所有数据但不能操作:
|
||||
|
||||
默认密码为 `proxygo`,程序启动时会在日志中提示。
|
||||
- ✅ 查看池子状态和质量分布
|
||||
- ✅ 查看代理列表和详细信息
|
||||
- ✅ 查看系统日志
|
||||
- ✅ 点击复制代理地址
|
||||
- ❌ 不能抓取代理
|
||||
- ❌ 不能刷新延迟
|
||||
- ❌ 不能删除代理
|
||||
- ❌ 不能修改配置
|
||||
|
||||
可通过环境变量 `WEBUI_PASSWORD` 自定义密码:
|
||||
**适用场景**:
|
||||
- 团队成员监控代理池状态
|
||||
- 展示给客户或第三方查看
|
||||
- 公网开放访问(只读数据安全)
|
||||
|
||||
#### ⚡ 管理员模式(完全控制)
|
||||
|
||||
**登录后**拥有所有操作权限:
|
||||
|
||||
- ✅ 所有访客模式的查看功能
|
||||
- ✅ 手动触发代理抓取
|
||||
- ✅ 刷新所有代理延迟
|
||||
- ✅ 刷新单个代理信息
|
||||
- ✅ 删除指定代理
|
||||
- ✅ 修改池子配置(容量、延迟标准、检查间隔等)
|
||||
|
||||
**默认密码**:`goproxy`(通过环境变量 `WEBUI_PASSWORD` 自定义)
|
||||
|
||||
### 健康仪表盘
|
||||
|
||||
**四宫格指标卡**
|
||||
- **Pool Status**:当前池子状态(HEALTHY/WARNING/CRITICAL/EMERGENCY)
|
||||
- **Total Proxies**:总代理数 / 池子容量
|
||||
- **HTTP**:HTTP 代理数 / HTTP 槽位数 + 平均延迟
|
||||
- **SOCKS5**:SOCKS5 代理数 / SOCKS5 槽位数 + 平均延迟
|
||||
|
||||
**质量分布可视化**
|
||||
- 横向条形图展示 S/A/B/C 四级质量分布
|
||||
- 实时显示各级别代理数量
|
||||
|
||||
### 代理注册表
|
||||
|
||||
**表格字段**
|
||||
- **Grade**:质量等级(S/A/B/C)
|
||||
- **Protocol**:协议类型(HTTP/SOCKS5)
|
||||
- **Address**:代理地址(host:port)
|
||||
- **Exit IP**:出口 IP
|
||||
- **Location**:出口地理位置(国旗 + 国家代码 + 城市)
|
||||
- **Latency**:延迟(毫秒,颜色编码)
|
||||
- **Usage**:使用次数 / 成功次数
|
||||
- **Action**:删除按钮
|
||||
|
||||
**操作功能**
|
||||
- **筛选**:All / HTTP / SOCKS5(所有用户可用)
|
||||
- **点击复制地址**:点击代理地址单元格复制到剪贴板(所有用户可用)
|
||||
- **Fetch Proxies**:手动触发智能抓取(⚡ 管理员专属)
|
||||
- **Refresh Latency**:重新验证所有代理并更新延迟(⚡ 管理员专属)
|
||||
- **刷新单个代理**:点击行内刷新按钮验证单个代理(⚡ 管理员专属)
|
||||
- **删除代理**:点击行内删除按钮移除指定代理(⚡ 管理员专属)
|
||||
- **Configure Pool**:打开配置界面(⚡ 管理员专属)
|
||||
|
||||
### 配置界面(⚡ 管理员专属)
|
||||
|
||||
点击 **Configure Pool** 打开配置模态框,包含:
|
||||
|
||||
**Pool Capacity 部分**
|
||||
- Max Size:池子总容量
|
||||
- HTTP Ratio:HTTP 协议占比(0.5 = 50%)
|
||||
- Min Per Protocol:每协议最小保证
|
||||
|
||||
**Latency Standards 部分**
|
||||
- Standard:标准模式延迟阈值
|
||||
- Healthy:健康模式严格延迟
|
||||
- Emergency:紧急模式放宽延迟
|
||||
|
||||
**Validation & Health Check 部分**
|
||||
- Validate Concurrency:并发验证数
|
||||
- Validate Timeout:验证超时
|
||||
- Health Check Interval:健康检查间隔
|
||||
- Health Check Batch Size:每批检查数量
|
||||
|
||||
**Optimization 部分**
|
||||
- Optimize Interval:优化轮换间隔
|
||||
- Replace Threshold:替换阈值(0.7 = 新代理需快 30%)
|
||||
|
||||
保存后立即生效,系统会自动调整池子策略。
|
||||
|
||||
## 🏗️ 核心架构
|
||||
|
||||
### 智能池子生命周期
|
||||
|
||||
```text
|
||||
[启动] → 状态监控 → 判断池子健康度
|
||||
↓
|
||||
需要补充?
|
||||
↙ ↘
|
||||
是 否
|
||||
↓ ↓
|
||||
智能抓取 保持监控
|
||||
(多模式) ↓
|
||||
↓ 优化轮换
|
||||
严格验证 (定时执行)
|
||||
↓ ↓
|
||||
智能入池 替换劣质代理
|
||||
(替换逻辑) ↓
|
||||
↓ 分批健康检查
|
||||
↓ (剔除失败)
|
||||
↓ ↓
|
||||
└─────────┘
|
||||
↓
|
||||
持续优化循环
|
||||
```
|
||||
|
||||
### 状态转换机制
|
||||
|
||||
```text
|
||||
Healthy (总数≥80% 且 各协议≥80%槽位)
|
||||
↓ 代理失效
|
||||
Warning (总数<80% 或 任一协议<80%)
|
||||
↓ 继续失效
|
||||
Critical (总数<50% 或 任一协议<20%槽位)
|
||||
↓ 继续失效
|
||||
Emergency (总数<10% 或 单协议缺失)
|
||||
↑
|
||||
└─ 自动触发紧急抓取 ─┘
|
||||
```
|
||||
|
||||
### 抓取模式选择
|
||||
|
||||
| 池子状态 | 抓取模式 | 使用源 | 触发条件 |
|
||||
| --- | --- | --- | --- |
|
||||
| Emergency | 紧急模式 | 所有可用源 | 单协议缺失或总数<10% |
|
||||
| Critical/Warning | 补充模式 | 快更新源 | 总数<80%或协议不均 |
|
||||
| Healthy | 优化模式 | 慢更新源(随机2-3个) | 定时触发(30分钟) |
|
||||
|
||||
### 质量分级标准
|
||||
|
||||
| 等级 | 延迟范围 | 说明 | 权重 |
|
||||
| --- | --- | --- | --- |
|
||||
| S | ≤500ms | 超快,优先使用,健康状态跳过检查 | 最高 |
|
||||
| A | 501-1000ms | 良好,稳定可用 | 高 |
|
||||
| B | 1001-2000ms | 可用,会被优化替换 | 中 |
|
||||
| C | >2000ms | 淘汰候选,优先替换 | 低 |
|
||||
|
||||
## 🔧 数据库 Schema
|
||||
|
||||
### proxies 表
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `id` | INTEGER | 主键 |
|
||||
| `address` | TEXT | 代理地址(UNIQUE) |
|
||||
| `protocol` | TEXT | 协议类型(http/socks5) |
|
||||
| `exit_ip` | TEXT | 出口 IP |
|
||||
| `exit_location` | TEXT | 出口位置 |
|
||||
| `latency` | INTEGER | 延迟(毫秒) |
|
||||
| `quality_grade` | TEXT | 质量等级(S/A/B/C) |
|
||||
| `use_count` | INTEGER | 使用次数 |
|
||||
| `success_count` | INTEGER | 成功次数 |
|
||||
| `fail_count` | INTEGER | 失败次数 |
|
||||
| `last_used` | DATETIME | 最后使用时间 |
|
||||
| `last_check` | DATETIME | 最后检查时间 |
|
||||
| `created_at` | DATETIME | 创建时间 |
|
||||
| `status` | TEXT | 状态(active/degraded/candidate_replace) |
|
||||
|
||||
### source_status 表
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `id` | INTEGER | 主键 |
|
||||
| `url` | TEXT | 源地址(UNIQUE) |
|
||||
| `success_count` | INTEGER | 成功次数 |
|
||||
| `fail_count` | INTEGER | 失败次数 |
|
||||
| `consecutive_fails` | INTEGER | 连续失败次数 |
|
||||
| `last_success` | DATETIME | 最后成功时间 |
|
||||
| `last_fail` | DATETIME | 最后失败时间 |
|
||||
| `status` | TEXT | 状态(active/degraded/disabled) |
|
||||
| `disabled_until` | DATETIME | 禁用到期时间 |
|
||||
|
||||
## 🔍 代理源
|
||||
|
||||
系统内置 16 个代理源,分为快更新和慢更新两组:
|
||||
|
||||
**快更新源(5-30分钟更新)**
|
||||
- proxifly/free-proxy-list (HTTP/SOCKS4/SOCKS5)
|
||||
- ProxyScraper/ProxyScraper (HTTP/SOCKS4/SOCKS5)
|
||||
- monosans/proxy-list (HTTP)
|
||||
|
||||
**慢更新源(每天更新)**
|
||||
- TheSpeedX/SOCKS-List (HTTP/SOCKS4/SOCKS5)
|
||||
- monosans/proxy-list (SOCKS4/SOCKS5)
|
||||
- databay-labs/free-proxy-list (HTTP/SOCKS5)
|
||||
|
||||
系统会根据池子状态自动选择合适的源组:
|
||||
- 紧急/补充模式:使用快更新源,快速填充
|
||||
- 优化模式:随机选择慢更新源,精细优化
|
||||
|
||||
## 🚦 核心机制详解
|
||||
|
||||
### 1. 智能入池机制
|
||||
|
||||
每个代理在入池前需通过:
|
||||
1. **连接验证**:能否成功连接 `http://www.gstatic.com/generate_204`
|
||||
2. **出口 IP 检测**:获取代理的出口 IP
|
||||
3. **地理位置查询**:获取出口 IP 的国家/城市
|
||||
4. **延迟测试**:测量连接延迟
|
||||
5. **质量评估**:根据延迟计算质量等级
|
||||
|
||||
**入池判断逻辑**
|
||||
- ✅ 协议槽位未满:直接加入
|
||||
- ✅ 槽位满但总量允许10%浮动:浮动加入
|
||||
- 🔄 池子满且质量更优:替换延迟最高的现有代理(需快30%+)
|
||||
- ❌ 池子满且质量不足:拒绝
|
||||
|
||||
### 2. 健康检查机制
|
||||
|
||||
**批次检查策略**
|
||||
- 每次检查 20 个代理(可配置)
|
||||
- 优先检查长时间未检查的
|
||||
- 池子健康时跳过 S 级代理(降低资源消耗)
|
||||
|
||||
**检查结果处理**
|
||||
- ✅ 验证通过:更新延迟、出口 IP、质量等级
|
||||
- ❌ 验证失败:失败计数 +1,≥3次自动删除
|
||||
|
||||
### 3. 优化轮换机制
|
||||
|
||||
**触发条件**
|
||||
- 池子状态:Healthy
|
||||
- 定时触发:默认 30 分钟
|
||||
|
||||
**优化流程**
|
||||
1. 从慢更新源随机抽取 2-3 个源
|
||||
2. 抓取候选代理并验证
|
||||
3. 筛选出延迟 ≤1500ms 的优质代理
|
||||
4. 尝试替换池中 B/C 级代理(需快30%+)
|
||||
|
||||
**资源控制**
|
||||
- 仅在池子健康时执行
|
||||
- 抽取少量源,避免浪费
|
||||
- 严格质量标准(≤1500ms)
|
||||
|
||||
### 4. 源管理与断路器
|
||||
|
||||
**状态跟踪**
|
||||
- 记录每个源的成功/失败次数
|
||||
- 连续失败 3 次:降级(Degraded)
|
||||
- 连续失败 5 次:禁用 30 分钟(Disabled)
|
||||
- 冷却期结束:自动恢复为 Active
|
||||
|
||||
**好处**
|
||||
- 避免浪费资源在失效源上
|
||||
- 自动恢复,无需人工干预
|
||||
- 保护系统免受源故障影响
|
||||
|
||||
## 📖 常见问题
|
||||
|
||||
### Q: 为什么池子容量是固定的?
|
||||
A: 固定容量可以:
|
||||
- **可预测资源消耗**:内存、CPU、网络带宽均可控
|
||||
- **提升代理质量**:通过严格准入和替换保持高质量
|
||||
- **简化管理逻辑**:避免无限增长和复杂的淘汰策略
|
||||
|
||||
### Q: 如何调整池子大小和协议比例?
|
||||
A:
|
||||
1. 访问 WebUI → 点击 **Configure Pool**
|
||||
2. 修改 **Max Size** 和 **HTTP Ratio**
|
||||
3. 点击 **Save Configuration**
|
||||
4. 系统会自动调整槽位分配
|
||||
|
||||
示例:
|
||||
- 池子大小 200,HTTP 比例 0.7 → HTTP 槽位 140,SOCKS5 槽位 60
|
||||
- 池子大小 50,HTTP 比例 0.3 → HTTP 槽位 15,SOCKS5 槽位 35
|
||||
|
||||
### Q: 池子状态如何计算?
|
||||
A:
|
||||
- **Healthy**:总数 ≥80% 且各协议 ≥80% 槽位
|
||||
- **Warning**:总数 <80% 或任一协议 <80% 槽位
|
||||
- **Critical**:总数 <50% 或任一协议 <20% 槽位
|
||||
- **Emergency**:总数 <10% 或单协议缺失
|
||||
|
||||
### Q: 如何优化延迟?
|
||||
A: 系统会自动优化,也可以手动调整:
|
||||
1. 降低 `max_latency_healthy`(严格模式)
|
||||
2. 增加 `optimize_interval` 频率(更频繁优化)
|
||||
3. 调高 `replace_threshold`(要求新代理更快)
|
||||
4. 点击 **Refresh Latency** 立即重新验证
|
||||
|
||||
### Q: 为什么有的代理没有出口 IP?
|
||||
A:
|
||||
- IP 查询有限流(10 次/秒)
|
||||
- 部分代理可能不支持 IP 查询
|
||||
- 系统会在后续健康检查中补全信息
|
||||
|
||||
### Q: 资源消耗如何?
|
||||
A:
|
||||
- **内存**:池子 100 个约 50MB,200 个约 100MB
|
||||
- **CPU**:空闲时 <1%,验证时 10-30%(取决于并发数)
|
||||
- **网络**:
|
||||
- IP 查询限流 10 次/秒
|
||||
- 按需抓取,避免无效流量
|
||||
- 健康检查批次小(20 个)
|
||||
|
||||
## 📚 详细设计文档
|
||||
|
||||
完整的架构设计、模块说明、配置策略、资源优化方案,请查看:
|
||||
|
||||
👉 [POOL_DESIGN.md](./POOL_DESIGN.md)
|
||||
|
||||
## 🛠️ 开发与调试
|
||||
|
||||
### 查看日志
|
||||
|
||||
日志会输出到 stdout,同时在 WebUI 的 **System Log** 部分实时展示。
|
||||
|
||||
关键日志标识:
|
||||
- `[pool]`:池子管理器
|
||||
- `[fetch]`:抓取器
|
||||
- `[source]`:源管理器
|
||||
- `[health]`:健康检查器
|
||||
- `[optimize]`:优化器
|
||||
- `[monitor]`:状态监控器
|
||||
|
||||
### 数据库操作
|
||||
|
||||
```bash
|
||||
# 本地运行
|
||||
WEBUI_PASSWORD=your-password go run .
|
||||
# 查看当前代理
|
||||
sqlite3 data/proxy.db "SELECT address, protocol, latency, quality_grade, status FROM proxies LIMIT 10;"
|
||||
|
||||
# Docker
|
||||
docker run -e WEBUI_PASSWORD=your-password ...
|
||||
# 查看质量分布
|
||||
sqlite3 data/proxy.db "SELECT quality_grade, COUNT(*) FROM proxies WHERE status='active' GROUP BY quality_grade;"
|
||||
|
||||
# 查看源状态
|
||||
sqlite3 data/proxy.db "SELECT url, status, consecutive_fails FROM source_status;"
|
||||
|
||||
# 清空池子(慎用)
|
||||
sqlite3 data/proxy.db "DELETE FROM proxies;"
|
||||
```
|
||||
|
||||
也可以在 `docker-compose.yml` 的 `environment` 中添加:
|
||||
## 🧪 测试代理服务
|
||||
|
||||
```yaml
|
||||
- WEBUI_PASSWORD=your-password
|
||||
项目提供了三种测试脚本,用于验证代理服务功能和性能(位于 `test/` 目录)。
|
||||
|
||||
### 快速测试
|
||||
|
||||
```bash
|
||||
# 测试随机轮换模式(默认 7777 端口)
|
||||
./test/test_proxy.sh
|
||||
|
||||
# 测试最低延迟模式(7776 端口)
|
||||
./test/test_proxy.sh 7776
|
||||
|
||||
# 使用 Go 脚本
|
||||
go run test/test_proxy.go # 默认 7777
|
||||
go run test/test_proxy.go 7776 # 测试 7776
|
||||
|
||||
# 使用 Python 脚本
|
||||
python test/test_proxy.py # 默认 7777
|
||||
python test/test_proxy.py 7776 # 测试 7776
|
||||
|
||||
# 按 Ctrl+C 停止测试并查看统计
|
||||
```
|
||||
|
||||
## API 概览
|
||||
测试脚本特点:
|
||||
- **持续运行模式**:类似 `ping` 命令,持续发送请求
|
||||
- 实时显示每次请求的出口 IP 和延迟
|
||||
- 动态更新成功率统计
|
||||
- 验证代理轮换机制
|
||||
- 按 `Ctrl+C` 停止并显示完整统计报告
|
||||
|
||||
除 `/login` 和 `/logout` 外,其余管理 API 都要求已登录会话。
|
||||
### 测试输出示例
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `GET` | `/` | 仪表盘页面 |
|
||||
| `GET/POST` | `/login` | 登录页面 / 提交登录 |
|
||||
| `GET` | `/logout` | 退出登录 |
|
||||
| `GET` | `/api/stats` | 代理统计信息 |
|
||||
| `GET` | `/api/proxies?protocol=http` | 查询代理列表,可按协议筛选 |
|
||||
| `POST` | `/api/proxy/delete` | 删除指定代理 |
|
||||
| `POST` | `/api/fetch` | 手动触发一次抓取 |
|
||||
| `GET` | `/api/logs` | 获取最近 200 条日志 |
|
||||
| `GET/POST` | `/api/config` | 读取/保存运行参数 |
|
||||
```
|
||||
PROXY 127.0.0.1:7777 (http://ip-api.com/json/?fields=countryCode,query): continuous mode
|
||||
|
||||
## 代理抓取与校验逻辑
|
||||
proxy from 🇺🇸 203.0.113.45: seq=1 time=1234ms
|
||||
proxy from 🇩🇪 198.51.100.78: seq=2 time=987ms
|
||||
proxy from 🇬🇧 192.0.2.123: seq=3 time=1567ms
|
||||
proxy #4: request failed (timeout)
|
||||
proxy from 🇯🇵 198.51.100.12: seq=5 time=890ms
|
||||
...(持续运行,按 Ctrl+C 停止)
|
||||
|
||||
当前实现会并发抓取内置代理源,然后做去重与验证:
|
||||
^C
|
||||
---
|
||||
50 requests transmitted, 47 received, 3 failed, 6.0% packet loss
|
||||
```
|
||||
|
||||
- HTTP 源:`https://cdn.jsdelivr.net/gh/databay-labs/free-proxy-list/http.txt`
|
||||
- SOCKS5 源:`https://cdn.jsdelivr.net/gh/databay-labs/free-proxy-list/socks5.txt`
|
||||
- 混合源:`https://cdn.jsdelivr.net/gh/proxifly/free-proxy-list@main/proxies/all/data.txt`
|
||||
详细测试指南请查看:👉 [test/TEST_GUIDE.md](./test/TEST_GUIDE.md)
|
||||
|
||||
验证规则:
|
||||
## 🙏 致谢与声明
|
||||
|
||||
- 仅接受 HTTP `200` 或 `204`
|
||||
- 响应超时或延迟超过 `MaxResponseMs` 的代理会被丢弃
|
||||
- 默认验证目标为 `https://cursor.com/api/auth/me`
|
||||
本项目基于 [jonasen1988/proxygo](https://github.com/jonasen1988/proxygo) 进行魔改和增强。
|
||||
|
||||
## 日志
|
||||
### 原项目
|
||||
- **项目地址**:https://github.com/jonasen1988/proxygo
|
||||
- **作者**:jonasen1988
|
||||
- **基础功能**:代理抓取、验证、存储、HTTP代理服务、WebUI管理
|
||||
|
||||
- 日志会输出到进程标准输出
|
||||
- 同时会保留最近 500 条在内存中供 WebUI 展示
|
||||
- `/api/logs` 当前返回最近 200 条日志
|
||||
### 本项目增强功能
|
||||
在原项目基础上,我们进行了大量改进和功能增强:
|
||||
|
||||
## 当前实现限制
|
||||
- 🆕 **智能池子机制**:固定容量管理、质量分级(S/A/B/C)、智能替换逻辑
|
||||
- 🆕 **按需抓取策略**:源分组、断路器保护、Emergency/Refill/Optimize 多模式
|
||||
- 🆕 **分层健康管理**:批次检查、智能跳过 S 级、定时优化轮换
|
||||
- 🆕 **智能重试机制**:自动故障切换、失败即删除、防重复尝试
|
||||
- 🆕 **双端口服务**:7777 随机轮换(IP 多样性)+ 7776 最低延迟(稳定连接)
|
||||
- 🆕 **黑客风格 WebUI**:Matrix 美学、实时仪表盘、完整配置界面、中英文切换
|
||||
- 🆕 **双角色权限**:访客模式(只读)+ 管理员模式(完全控制),可安全公网开放
|
||||
- 🆕 **扩展存储层**:质量等级、使用统计、源状态管理
|
||||
- 🆕 **测试套件**:Bash/Go/Python 三种测试脚本,持续运行模式,显示国旗 emoji
|
||||
|
||||
- `config.Config` 中虽然定义了 `HTTPSourceURL` 和 `SOCKS5SourceURL`,但抓取器当前实际使用的是 `fetcher/defaultSources` 内置来源
|
||||
- `config.json` 目前只持久化 4 个字段,不包含端口、密码哈希、验证 URL 等配置
|
||||
- WebUI 登录密码不能在线修改
|
||||
- 代理请求失败时,运行逻辑倾向于直接删除上游代理,`MaxFailCount` 目前没有完整参与主流程
|
||||
- 日志没有单独写文件,管理端看到的是内存中的最近日志窗口
|
||||
感谢原作者提供的基础实现,让我们能够在此之上构建更强大的代理池系统。
|
||||
|
||||
## 适用场景
|
||||
同时感谢 [LINUX DO](https://linux.do/) 社区的支持。
|
||||
|
||||
- 在本机快速聚合一批公开代理,提供给命令行或程序统一使用
|
||||
- 临时验证免费 HTTP / SOCKS5 代理的可用性
|
||||
- 通过简单 Web 面板查看当前代理池状态
|
||||
## 📝 License
|
||||
|
||||
如果后续要继续完善,优先建议补这几项:
|
||||
|
||||
- 支持通过配置文件完整覆盖所有默认参数
|
||||
- 支持自定义代理源并真正接入抓取器
|
||||
- 支持 WebUI 密码初始化和修改
|
||||
- 为失败计数、重试和删除策略补齐一致的状态流转
|
||||
- 增加自动化测试
|
||||
|
||||
## 致谢
|
||||
|
||||
认可 [LINUX DO](https://linux.do/)
|
||||
MIT License
|
||||
@@ -4,9 +4,9 @@ import (
|
||||
"log"
|
||||
"time"
|
||||
|
||||
"proxy-pool/config"
|
||||
"proxy-pool/storage"
|
||||
"proxy-pool/validator"
|
||||
"goproxy/config"
|
||||
"goproxy/storage"
|
||||
"goproxy/validator"
|
||||
)
|
||||
|
||||
type Checker struct {
|
||||
@@ -52,6 +52,19 @@ func (c *Checker) run() {
|
||||
for _, r := range results {
|
||||
if r.Valid {
|
||||
valid++
|
||||
// 更新出口 IP、位置和延迟信息
|
||||
latencyMs := int(r.Latency.Milliseconds())
|
||||
if r.ExitIP != "" && r.Proxy.ExitIP == "" {
|
||||
// 如果之前没有出口 IP 信息,更新完整信息
|
||||
if err := c.storage.UpdateExitInfo(r.Proxy.Address, r.ExitIP, r.ExitLocation, latencyMs); err != nil {
|
||||
log.Printf("[checker] update exit info error: %v", err)
|
||||
}
|
||||
} else if r.Latency > 0 {
|
||||
// 否则只更新延迟
|
||||
if err := c.storage.UpdateLatency(r.Proxy.Address, latencyMs); err != nil {
|
||||
log.Printf("[checker] update latency error: %v", err)
|
||||
}
|
||||
}
|
||||
} else {
|
||||
invalid++
|
||||
if err := c.storage.Delete(r.Proxy.Address); err != nil {
|
||||
|
||||
103
checker/health_checker.go
Normal file
103
checker/health_checker.go
Normal file
@@ -0,0 +1,103 @@
|
||||
package checker
|
||||
|
||||
import (
|
||||
"log"
|
||||
"time"
|
||||
|
||||
"goproxy/config"
|
||||
"goproxy/pool"
|
||||
"goproxy/storage"
|
||||
"goproxy/validator"
|
||||
)
|
||||
|
||||
// HealthChecker 健康检查器
|
||||
type HealthChecker struct {
|
||||
storage *storage.Storage
|
||||
validator *validator.Validator
|
||||
cfg *config.Config
|
||||
poolMgr *pool.Manager
|
||||
}
|
||||
|
||||
func NewHealthChecker(s *storage.Storage, v *validator.Validator, cfg *config.Config, pm *pool.Manager) *HealthChecker {
|
||||
return &HealthChecker{
|
||||
storage: s,
|
||||
validator: v,
|
||||
cfg: cfg,
|
||||
poolMgr: pm,
|
||||
}
|
||||
}
|
||||
|
||||
// RunOnce 执行一次健康检查
|
||||
func (hc *HealthChecker) RunOnce() {
|
||||
start := time.Now()
|
||||
log.Println("[health] 开始健康检查...")
|
||||
|
||||
// 获取池子状态
|
||||
status, err := hc.poolMgr.GetStatus()
|
||||
if err != nil {
|
||||
log.Printf("[health] 获取状态失败: %v", err)
|
||||
return
|
||||
}
|
||||
|
||||
// 健康状态且S级占比高时,跳过S级代理检查
|
||||
skipSGrade := status.State == "healthy"
|
||||
dist, _ := hc.storage.GetQualityDistribution()
|
||||
sGradeCount := dist["S"]
|
||||
totalCount := status.Total
|
||||
if totalCount > 0 && float64(sGradeCount)/float64(totalCount) > 0.3 {
|
||||
skipSGrade = true
|
||||
}
|
||||
|
||||
// 批量获取需要检查的代理
|
||||
proxies, err := hc.storage.GetBatchForHealthCheck(hc.cfg.HealthCheckBatchSize, skipSGrade)
|
||||
if err != nil {
|
||||
log.Printf("[health] 获取检查批次失败: %v", err)
|
||||
return
|
||||
}
|
||||
|
||||
if len(proxies) == 0 {
|
||||
log.Println("[health] 无需检查的代理")
|
||||
return
|
||||
}
|
||||
|
||||
log.Printf("[health] 检查 %d 个代理(跳过S级=%v)", len(proxies), skipSGrade)
|
||||
|
||||
// 执行验证
|
||||
validCount := 0
|
||||
removeCount := 0
|
||||
updateCount := 0
|
||||
|
||||
for result := range hc.validator.ValidateStream(proxies) {
|
||||
if result.Valid {
|
||||
validCount++
|
||||
// 更新延迟和质量等级
|
||||
latencyMs := int(result.Latency.Milliseconds())
|
||||
if err := hc.storage.UpdateExitInfo(result.Proxy.Address, result.ExitIP, result.ExitLocation, latencyMs); err == nil {
|
||||
updateCount++
|
||||
}
|
||||
} else {
|
||||
// 失败次数+1
|
||||
hc.storage.IncrementFailCount(result.Proxy.Address)
|
||||
// 如果失败次数 >= 3,删除
|
||||
if result.Proxy.FailCount+1 >= 3 {
|
||||
hc.storage.Delete(result.Proxy.Address)
|
||||
removeCount++
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
elapsed := time.Since(start)
|
||||
log.Printf("[health] 完成: 验证%d 有效%d 更新%d 移除%d 耗时%v",
|
||||
len(proxies), validCount, updateCount, removeCount, elapsed)
|
||||
}
|
||||
|
||||
// StartBackground 后台定时健康检查
|
||||
func (hc *HealthChecker) StartBackground() {
|
||||
ticker := time.NewTicker(time.Duration(hc.cfg.HealthCheckInterval) * time.Minute)
|
||||
go func() {
|
||||
for range ticker.C {
|
||||
hc.RunOnce()
|
||||
}
|
||||
}()
|
||||
log.Printf("[health] 健康检查器已启动,间隔 %d 分钟", hc.cfg.HealthCheckInterval)
|
||||
}
|
||||
255
config/config.go
255
config/config.go
@@ -8,7 +8,7 @@ import (
|
||||
"sync"
|
||||
)
|
||||
|
||||
const DefaultPassword = "proxygo"
|
||||
const DefaultPassword = "goproxy"
|
||||
|
||||
func dataDir() string {
|
||||
if d := os.Getenv("DATA_DIR"); d != "" {
|
||||
@@ -27,37 +27,57 @@ type Config struct {
|
||||
// WebUI 密码 SHA256 哈希
|
||||
WebUIPasswordHash string
|
||||
|
||||
// 代理池本地监听端口
|
||||
// 代理池本地监听端口(随机轮换模式)
|
||||
ProxyPort string
|
||||
|
||||
// 稳定代理端口(最低延迟模式)
|
||||
StableProxyPort string
|
||||
|
||||
// SQLite 数据库路径
|
||||
DBPath string
|
||||
|
||||
// 验证并发数
|
||||
ValidateConcurrency int
|
||||
// ========== 池子容量配置 ==========
|
||||
PoolMaxSize int // 代理池总容量(默认100)
|
||||
PoolHTTPRatio float64 // HTTP协议占比(默认0.5)
|
||||
PoolMinPerProtocol int // 每协议最小保证(默认10)
|
||||
|
||||
// 验证超时(秒)
|
||||
ValidateTimeout int
|
||||
// ========== 延迟标准配置 ==========
|
||||
MaxLatencyMs int // 标准模式最大延迟(默认2000ms)
|
||||
MaxLatencyEmergency int // 紧急模式放宽延迟(默认3000ms)
|
||||
MaxLatencyHealthy int // 健康模式严格延迟(默认1500ms)
|
||||
MaxLatencyDegradation int // 降级模式超宽松延迟(默认5000ms)
|
||||
|
||||
// 验证目标 URL
|
||||
ValidateURL string
|
||||
// ========== 验证配置 ==========
|
||||
ValidateConcurrency int // 验证并发数(默认300)
|
||||
ValidateTimeout int // 验证超时(秒)(默认8)
|
||||
ValidateURL string // 验证目标 URL
|
||||
|
||||
// 最大响应时间(毫秒),超过则丢弃
|
||||
MaxResponseMs int
|
||||
// ========== 健康检查配置 ==========
|
||||
HealthCheckInterval int // 状态监控间隔(分钟)(默认5)
|
||||
HealthCheckBatchSize int // 每批验证数量(默认20)
|
||||
HealthCheckConcurrency int // 批次内并发数(默认50)
|
||||
|
||||
// 代理失败次数阈值,超过后删除
|
||||
MaxFailCount int
|
||||
// ========== 优化配置 ==========
|
||||
OptimizeInterval int // 优化轮换间隔(分钟)(默认30)
|
||||
OptimizeConcurrency int // 优化时并发数(默认100)
|
||||
ReplaceThreshold float64 // 替换阈值(默认0.7,新代理需快30%)
|
||||
|
||||
// 自动重试次数
|
||||
MaxRetry int
|
||||
// ========== IP查询配置 ==========
|
||||
IPQueryRateLimit int // IP查询限流(次/秒)(默认10)
|
||||
|
||||
// 定时抓取间隔(分钟)
|
||||
FetchInterval int
|
||||
// ========== 源管理配置 ==========
|
||||
SourceFailThreshold int // 源降级阈值(默认3)
|
||||
SourceDisableThreshold int // 源禁用阈值(默认5)
|
||||
SourceCooldownMinutes int // 源禁用冷却时间(默认30)
|
||||
|
||||
// 定时健康检查间隔(分钟)
|
||||
CheckInterval int
|
||||
// ========== 兼容旧配置 ==========
|
||||
MaxResponseMs int // 已废弃,使用 MaxLatencyMs 替代
|
||||
MaxFailCount int // 代理失败次数阈值
|
||||
MaxRetry int // 请求失败后的重试次数
|
||||
FetchInterval int // 已废弃,由智能抓取器管理
|
||||
CheckInterval int // 已废弃,由 HealthCheckInterval 替代
|
||||
|
||||
// 代理来源 URL
|
||||
// 代理来源 URL(已废弃,内置多源)
|
||||
HTTPSourceURL string
|
||||
SOCKS5SourceURL string
|
||||
}
|
||||
@@ -78,20 +98,55 @@ func DefaultConfig() *Config {
|
||||
password = DefaultPassword
|
||||
}
|
||||
return &Config{
|
||||
WebUIPort: ":7778",
|
||||
WebUIPasswordHash: passwordHash(password),
|
||||
ProxyPort: ":7777",
|
||||
DBPath: dataDir() + "proxy.db",
|
||||
// 基础服务配置
|
||||
WebUIPort: ":7778",
|
||||
WebUIPasswordHash: passwordHash(password),
|
||||
ProxyPort: ":7777",
|
||||
StableProxyPort: ":7776",
|
||||
DBPath: dataDir() + "proxy.db",
|
||||
|
||||
// 池子容量配置
|
||||
PoolMaxSize: 100, // 总容量
|
||||
PoolHTTPRatio: 0.5, // HTTP占50%
|
||||
PoolMinPerProtocol: 10, // 每协议最少10个
|
||||
|
||||
// 延迟标准配置
|
||||
MaxLatencyMs: 2000, // 标准2秒
|
||||
MaxLatencyEmergency: 3000, // 紧急3秒
|
||||
MaxLatencyHealthy: 1500, // 健康1.5秒
|
||||
MaxLatencyDegradation: 5000, // 降级5秒
|
||||
|
||||
// 验证配置
|
||||
ValidateConcurrency: 300,
|
||||
ValidateTimeout: 3,
|
||||
ValidateURL: "https://cursor.com/api/auth/me",
|
||||
MaxResponseMs: 2500,
|
||||
MaxFailCount: 3,
|
||||
MaxRetry: 3,
|
||||
FetchInterval: 30,
|
||||
CheckInterval: 10,
|
||||
HTTPSourceURL: "https://cdn.jsdelivr.net/gh/databay-labs/free-proxy-list/http.txt",
|
||||
SOCKS5SourceURL: "https://cdn.jsdelivr.net/gh/databay-labs/free-proxy-list/socks5.txt",
|
||||
ValidateTimeout: 8,
|
||||
ValidateURL: "http://www.gstatic.com/generate_204",
|
||||
|
||||
// 健康检查配置
|
||||
HealthCheckInterval: 5, // 5分钟
|
||||
HealthCheckBatchSize: 20, // 每批20个
|
||||
HealthCheckConcurrency: 50, // 批次并发50
|
||||
|
||||
// 优化配置
|
||||
OptimizeInterval: 30, // 30分钟
|
||||
OptimizeConcurrency: 100, // 并发100
|
||||
ReplaceThreshold: 0.7, // 新代理需快30%
|
||||
|
||||
// IP查询配置
|
||||
IPQueryRateLimit: 10, // 10次/秒
|
||||
|
||||
// 源管理配置
|
||||
SourceFailThreshold: 3, // 失败3次降级
|
||||
SourceDisableThreshold: 5, // 失败5次禁用
|
||||
SourceCooldownMinutes: 30, // 禁用30分钟
|
||||
|
||||
// 兼容旧配置
|
||||
MaxResponseMs: 5000,
|
||||
MaxFailCount: 3,
|
||||
MaxRetry: 3,
|
||||
FetchInterval: 30,
|
||||
CheckInterval: 10,
|
||||
HTTPSourceURL: "https://cdn.jsdelivr.net/gh/databay-labs/free-proxy-list/http.txt",
|
||||
SOCKS5SourceURL: "https://cdn.jsdelivr.net/gh/databay-labs/free-proxy-list/socks5.txt",
|
||||
}
|
||||
}
|
||||
|
||||
@@ -100,21 +155,61 @@ func Load() *Config {
|
||||
cfg := DefaultConfig()
|
||||
data, err := os.ReadFile(ConfigFile())
|
||||
if err == nil {
|
||||
// 只覆盖可调整的4个字段
|
||||
var saved savedConfig
|
||||
if json.Unmarshal(data, &saved) == nil {
|
||||
if saved.FetchInterval > 0 {
|
||||
cfg.FetchInterval = saved.FetchInterval
|
||||
// 池子配置
|
||||
if saved.PoolMaxSize > 0 {
|
||||
cfg.PoolMaxSize = saved.PoolMaxSize
|
||||
}
|
||||
if saved.CheckInterval > 0 {
|
||||
cfg.CheckInterval = saved.CheckInterval
|
||||
if saved.PoolHTTPRatio > 0 && saved.PoolHTTPRatio <= 1 {
|
||||
cfg.PoolHTTPRatio = saved.PoolHTTPRatio
|
||||
}
|
||||
if saved.PoolMinPerProtocol > 0 {
|
||||
cfg.PoolMinPerProtocol = saved.PoolMinPerProtocol
|
||||
}
|
||||
|
||||
// 延迟配置
|
||||
if saved.MaxLatencyMs > 0 {
|
||||
cfg.MaxLatencyMs = saved.MaxLatencyMs
|
||||
}
|
||||
if saved.MaxLatencyEmergency > 0 {
|
||||
cfg.MaxLatencyEmergency = saved.MaxLatencyEmergency
|
||||
}
|
||||
if saved.MaxLatencyHealthy > 0 {
|
||||
cfg.MaxLatencyHealthy = saved.MaxLatencyHealthy
|
||||
}
|
||||
|
||||
// 验证配置
|
||||
if saved.ValidateConcurrency > 0 {
|
||||
cfg.ValidateConcurrency = saved.ValidateConcurrency
|
||||
}
|
||||
if saved.ValidateTimeout > 0 {
|
||||
cfg.ValidateTimeout = saved.ValidateTimeout
|
||||
}
|
||||
|
||||
// 健康检查配置
|
||||
if saved.HealthCheckInterval > 0 {
|
||||
cfg.HealthCheckInterval = saved.HealthCheckInterval
|
||||
}
|
||||
if saved.HealthCheckBatchSize > 0 {
|
||||
cfg.HealthCheckBatchSize = saved.HealthCheckBatchSize
|
||||
}
|
||||
|
||||
// 优化配置
|
||||
if saved.OptimizeInterval > 0 {
|
||||
cfg.OptimizeInterval = saved.OptimizeInterval
|
||||
}
|
||||
if saved.ReplaceThreshold > 0 && saved.ReplaceThreshold <= 1 {
|
||||
cfg.ReplaceThreshold = saved.ReplaceThreshold
|
||||
}
|
||||
|
||||
// 兼容旧配置
|
||||
if saved.FetchInterval > 0 {
|
||||
cfg.FetchInterval = saved.FetchInterval
|
||||
}
|
||||
if saved.CheckInterval > 0 {
|
||||
cfg.CheckInterval = saved.CheckInterval
|
||||
}
|
||||
}
|
||||
}
|
||||
cfgMu.Lock()
|
||||
@@ -130,31 +225,91 @@ func Get() *Config {
|
||||
return globalCfg
|
||||
}
|
||||
|
||||
// savedConfig 只持久化可调整的字段
|
||||
// savedConfig 持久化可调整的字段
|
||||
type savedConfig struct {
|
||||
FetchInterval int `json:"fetch_interval"`
|
||||
CheckInterval int `json:"check_interval"`
|
||||
// 池子配置
|
||||
PoolMaxSize int `json:"pool_max_size"`
|
||||
PoolHTTPRatio float64 `json:"pool_http_ratio"`
|
||||
PoolMinPerProtocol int `json:"pool_min_per_protocol"`
|
||||
|
||||
// 延迟配置
|
||||
MaxLatencyMs int `json:"max_latency_ms"`
|
||||
MaxLatencyEmergency int `json:"max_latency_emergency"`
|
||||
MaxLatencyHealthy int `json:"max_latency_healthy"`
|
||||
|
||||
// 验证配置
|
||||
ValidateConcurrency int `json:"validate_concurrency"`
|
||||
ValidateTimeout int `json:"validate_timeout"`
|
||||
|
||||
// 健康检查配置
|
||||
HealthCheckInterval int `json:"health_check_interval"`
|
||||
HealthCheckBatchSize int `json:"health_check_batch_size"`
|
||||
|
||||
// 优化配置
|
||||
OptimizeInterval int `json:"optimize_interval"`
|
||||
ReplaceThreshold float64 `json:"replace_threshold"`
|
||||
|
||||
// 兼容旧配置
|
||||
FetchInterval int `json:"fetch_interval,omitempty"`
|
||||
CheckInterval int `json:"check_interval,omitempty"`
|
||||
}
|
||||
|
||||
// Save 保存可调整字段到文件,并更新内存配置
|
||||
func Save(fetchInterval, checkInterval, validateConcurrency, validateTimeout int) error {
|
||||
// Save 保存配置到文件,并更新内存配置
|
||||
func Save(cfg *Config) error {
|
||||
cfgMu.Lock()
|
||||
globalCfg.FetchInterval = fetchInterval
|
||||
globalCfg.CheckInterval = checkInterval
|
||||
globalCfg.ValidateConcurrency = validateConcurrency
|
||||
globalCfg.ValidateTimeout = validateTimeout
|
||||
*globalCfg = *cfg
|
||||
cfgMu.Unlock()
|
||||
|
||||
data, err := json.MarshalIndent(savedConfig{
|
||||
FetchInterval: fetchInterval,
|
||||
CheckInterval: checkInterval,
|
||||
ValidateConcurrency: validateConcurrency,
|
||||
ValidateTimeout: validateTimeout,
|
||||
PoolMaxSize: cfg.PoolMaxSize,
|
||||
PoolHTTPRatio: cfg.PoolHTTPRatio,
|
||||
PoolMinPerProtocol: cfg.PoolMinPerProtocol,
|
||||
MaxLatencyMs: cfg.MaxLatencyMs,
|
||||
MaxLatencyEmergency: cfg.MaxLatencyEmergency,
|
||||
MaxLatencyHealthy: cfg.MaxLatencyHealthy,
|
||||
ValidateConcurrency: cfg.ValidateConcurrency,
|
||||
ValidateTimeout: cfg.ValidateTimeout,
|
||||
HealthCheckInterval: cfg.HealthCheckInterval,
|
||||
HealthCheckBatchSize: cfg.HealthCheckBatchSize,
|
||||
OptimizeInterval: cfg.OptimizeInterval,
|
||||
ReplaceThreshold: cfg.ReplaceThreshold,
|
||||
FetchInterval: cfg.FetchInterval,
|
||||
CheckInterval: cfg.CheckInterval,
|
||||
}, "", " ")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return os.WriteFile(ConfigFile(), data, 0644)
|
||||
}
|
||||
|
||||
// CalculateSlots 根据配置计算各协议的槽位数
|
||||
func (c *Config) CalculateSlots() (httpSlots, socks5Slots int) {
|
||||
httpSlots = int(float64(c.PoolMaxSize) * c.PoolHTTPRatio)
|
||||
socks5Slots = c.PoolMaxSize - httpSlots
|
||||
|
||||
// 保证最小值
|
||||
if httpSlots < c.PoolMinPerProtocol {
|
||||
httpSlots = c.PoolMinPerProtocol
|
||||
}
|
||||
if socks5Slots < c.PoolMinPerProtocol {
|
||||
socks5Slots = c.PoolMinPerProtocol
|
||||
}
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
// GetLatencyThreshold 根据池子状态返回合适的延迟阈值
|
||||
func (c *Config) GetLatencyThreshold(poolStatus string) int {
|
||||
switch poolStatus {
|
||||
case "emergency":
|
||||
return c.MaxLatencyEmergency
|
||||
case "critical":
|
||||
return c.MaxLatencyEmergency
|
||||
case "warning":
|
||||
return c.MaxLatencyMs
|
||||
case "healthy":
|
||||
return c.MaxLatencyHealthy
|
||||
default:
|
||||
return c.MaxLatencyMs
|
||||
}
|
||||
}
|
||||
|
||||
@@ -4,7 +4,8 @@ services:
|
||||
container_name: proxygo
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "127.0.0.1:7777:7777" # HTTP 代理端口(仅内网)
|
||||
- "127.0.0.1:7776:7776" # 稳定代理端口(最低延迟)
|
||||
- "127.0.0.1:7777:7777" # 随机代理端口(轮换模式)
|
||||
- "7778:7778" # WebUI 端口(外网可访问)
|
||||
volumes:
|
||||
- ./data:/app/data
|
||||
|
||||
@@ -9,7 +9,7 @@ import (
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"proxy-pool/storage"
|
||||
"goproxy/storage"
|
||||
)
|
||||
|
||||
// 代理来源定义
|
||||
@@ -18,27 +18,171 @@ type Source struct {
|
||||
Protocol string // http 或 socks5
|
||||
}
|
||||
|
||||
// 内置多个免费代理来源
|
||||
var defaultSources = []Source{
|
||||
// 快速更新源(5-30分钟更新)- 用于紧急和补充模式
|
||||
var fastUpdateSources = []Source{
|
||||
// proxifly - 每5分钟更新
|
||||
{"https://cdn.jsdelivr.net/gh/proxifly/free-proxy-list@main/proxies/http/data.txt", "http"},
|
||||
{"https://cdn.jsdelivr.net/gh/proxifly/free-proxy-list@main/proxies/socks4/data.txt", "socks5"},
|
||||
{"https://cdn.jsdelivr.net/gh/proxifly/free-proxy-list@main/proxies/socks5/data.txt", "socks5"},
|
||||
// ProxyScraper - 每30分钟更新
|
||||
{"https://raw.githubusercontent.com/ProxyScraper/ProxyScraper/main/http.txt", "http"},
|
||||
{"https://raw.githubusercontent.com/ProxyScraper/ProxyScraper/main/socks4.txt", "socks5"},
|
||||
{"https://raw.githubusercontent.com/ProxyScraper/ProxyScraper/main/socks5.txt", "socks5"},
|
||||
// monosans - 每小时更新
|
||||
{"https://raw.githubusercontent.com/monosans/proxy-list/main/proxies/http.txt", "http"},
|
||||
}
|
||||
|
||||
// 慢速更新源(每天更新)- 用于优化轮换模式
|
||||
var slowUpdateSources = []Source{
|
||||
// TheSpeedX - 每天更新
|
||||
{"https://raw.githubusercontent.com/TheSpeedX/SOCKS-List/master/http.txt", "http"},
|
||||
{"https://raw.githubusercontent.com/TheSpeedX/SOCKS-List/master/socks4.txt", "socks5"},
|
||||
{"https://raw.githubusercontent.com/TheSpeedX/SOCKS-List/master/socks5.txt", "socks5"},
|
||||
// monosans SOCKS
|
||||
{"https://raw.githubusercontent.com/monosans/proxy-list/main/proxies/socks4.txt", "socks5"},
|
||||
{"https://raw.githubusercontent.com/monosans/proxy-list/main/proxies/socks5.txt", "socks5"},
|
||||
// databay-labs - 备用源
|
||||
{"https://cdn.jsdelivr.net/gh/databay-labs/free-proxy-list/http.txt", "http"},
|
||||
{"https://cdn.jsdelivr.net/gh/databay-labs/free-proxy-list/socks5.txt", "socks5"},
|
||||
{"https://cdn.jsdelivr.net/gh/proxifly/free-proxy-list@main/proxies/all/data.txt", ""},
|
||||
}
|
||||
|
||||
// 所有源
|
||||
var allSources = append(fastUpdateSources, slowUpdateSources...)
|
||||
|
||||
type Fetcher struct {
|
||||
sources []Source
|
||||
client *http.Client
|
||||
sources []Source
|
||||
client *http.Client
|
||||
sourceManager *SourceManager
|
||||
}
|
||||
|
||||
func New(httpURL, socks5URL string) *Fetcher {
|
||||
func New(httpURL, socks5URL string, sourceManager *SourceManager) *Fetcher {
|
||||
return &Fetcher{
|
||||
sources: defaultSources,
|
||||
sources: allSources,
|
||||
sourceManager: sourceManager,
|
||||
client: &http.Client{
|
||||
Timeout: 30 * time.Second,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// FetchSmart 智能抓取:根据模式和协议需求选择源
|
||||
func (f *Fetcher) FetchSmart(mode string, preferredProtocol string) ([]storage.Proxy, error) {
|
||||
var sources []Source
|
||||
|
||||
switch mode {
|
||||
case "emergency":
|
||||
// 紧急模式:使用所有可用源
|
||||
sources = f.filterAvailableSources(allSources, preferredProtocol)
|
||||
log.Printf("[fetch] 🚨 紧急模式: 使用 %d 个源", len(sources))
|
||||
|
||||
case "refill":
|
||||
// 补充模式:使用快更新源
|
||||
sources = f.filterAvailableSources(fastUpdateSources, preferredProtocol)
|
||||
log.Printf("[fetch] 🔄 补充模式: 使用 %d 个快更新源", len(sources))
|
||||
|
||||
case "optimize":
|
||||
// 优化模式:随机选择2-3个慢更新源
|
||||
sources = f.selectRandomSources(slowUpdateSources, 3, preferredProtocol)
|
||||
log.Printf("[fetch] ⚡ 优化模式: 使用 %d 个源", len(sources))
|
||||
|
||||
default:
|
||||
sources = f.filterAvailableSources(fastUpdateSources, preferredProtocol)
|
||||
}
|
||||
|
||||
if len(sources) == 0 {
|
||||
return nil, fmt.Errorf("no available sources")
|
||||
}
|
||||
|
||||
return f.fetchFromSources(sources)
|
||||
}
|
||||
|
||||
// filterAvailableSources 过滤可用的源(通过断路器)
|
||||
func (f *Fetcher) filterAvailableSources(sources []Source, preferredProtocol string) []Source {
|
||||
var available []Source
|
||||
for _, src := range sources {
|
||||
// 检查断路器
|
||||
if f.sourceManager != nil && !f.sourceManager.CanUseSource(src.URL) {
|
||||
continue
|
||||
}
|
||||
// 如果指定了协议偏好,优先该协议的源
|
||||
if preferredProtocol != "" && src.Protocol != "" && src.Protocol != preferredProtocol {
|
||||
continue
|
||||
}
|
||||
available = append(available, src)
|
||||
}
|
||||
return available
|
||||
}
|
||||
|
||||
// selectRandomSources 随机选择N个源
|
||||
func (f *Fetcher) selectRandomSources(sources []Source, count int, preferredProtocol string) []Source {
|
||||
available := f.filterAvailableSources(sources, preferredProtocol)
|
||||
if len(available) <= count {
|
||||
return available
|
||||
}
|
||||
|
||||
// 随机打乱
|
||||
shuffled := make([]Source, len(available))
|
||||
copy(shuffled, available)
|
||||
for i := range shuffled {
|
||||
j := i + int(time.Now().UnixNano())%(len(shuffled)-i)
|
||||
shuffled[i], shuffled[j] = shuffled[j], shuffled[i]
|
||||
}
|
||||
|
||||
return shuffled[:count]
|
||||
}
|
||||
|
||||
// fetchFromSources 从指定源列表抓取
|
||||
func (f *Fetcher) fetchFromSources(sources []Source) ([]storage.Proxy, error) {
|
||||
type result struct {
|
||||
proxies []storage.Proxy
|
||||
source Source
|
||||
err error
|
||||
}
|
||||
|
||||
ch := make(chan result, len(sources))
|
||||
for _, src := range sources {
|
||||
go func(s Source) {
|
||||
proxies, err := f.fetchFromURL(s.URL, s.Protocol)
|
||||
ch <- result{proxies: proxies, source: s, err: err}
|
||||
}(src)
|
||||
}
|
||||
|
||||
var all []storage.Proxy
|
||||
seen := make(map[string]bool)
|
||||
for range sources {
|
||||
r := <-ch
|
||||
if r.err != nil {
|
||||
log.Printf("[fetch] ❌ %s error: %v", r.source.URL, r.err)
|
||||
if f.sourceManager != nil {
|
||||
f.sourceManager.RecordFail(r.source.URL, 3, 5, 30)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
// 记录成功
|
||||
if f.sourceManager != nil {
|
||||
f.sourceManager.RecordSuccess(r.source.URL)
|
||||
}
|
||||
|
||||
// 去重
|
||||
var deduped []storage.Proxy
|
||||
for _, p := range r.proxies {
|
||||
if !seen[p.Address] {
|
||||
seen[p.Address] = true
|
||||
deduped = append(deduped, p)
|
||||
}
|
||||
}
|
||||
log.Printf("[fetch] ✅ %d 个 %s 代理 from %s", len(deduped), r.source.Protocol, r.source.URL)
|
||||
all = append(all, deduped...)
|
||||
}
|
||||
|
||||
if len(all) == 0 {
|
||||
return nil, fmt.Errorf("no proxies fetched")
|
||||
}
|
||||
log.Printf("[fetch] 总共抓取: %d 个代理(去重后)", len(all))
|
||||
return all, nil
|
||||
}
|
||||
|
||||
// Fetch 从所有来源并发抓取代理
|
||||
func (f *Fetcher) Fetch() ([]storage.Proxy, error) {
|
||||
type result struct {
|
||||
|
||||
152
fetcher/ip_query.go
Normal file
152
fetcher/ip_query.go
Normal file
@@ -0,0 +1,152 @@
|
||||
package fetcher
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"golang.org/x/time/rate"
|
||||
)
|
||||
|
||||
// IPQueryLimiter 全局IP查询限流器
|
||||
var IPQueryLimiter *rate.Limiter
|
||||
|
||||
// InitIPQueryLimiter 初始化限流器
|
||||
func InitIPQueryLimiter(rps int) {
|
||||
IPQueryLimiter = rate.NewLimiter(rate.Limit(rps), rps*2)
|
||||
}
|
||||
|
||||
// GetExitIPInfo 通过代理获取出口 IP 和地理位置(多源降级)
|
||||
func GetExitIPInfo(client *http.Client) (string, string) {
|
||||
// 等待限流令牌
|
||||
if IPQueryLimiter != nil {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
if err := IPQueryLimiter.Wait(ctx); err != nil {
|
||||
return "", ""
|
||||
}
|
||||
}
|
||||
|
||||
// 优先级1:ip-api.com
|
||||
if ip, loc := tryIPAPI(client); ip != "" {
|
||||
return ip, loc
|
||||
}
|
||||
|
||||
// 优先级2:ipapi.co
|
||||
if ip, loc := tryIPAPICo(client); ip != "" {
|
||||
return ip, loc
|
||||
}
|
||||
|
||||
// 优先级3:ipinfo.io
|
||||
if ip, loc := tryIPInfo(client); ip != "" {
|
||||
return ip, loc
|
||||
}
|
||||
|
||||
// 优先级4:仅获取IP
|
||||
if ip := tryHTTPBinIP(client); ip != "" {
|
||||
return ip, "UNKNOWN"
|
||||
}
|
||||
|
||||
return "", ""
|
||||
}
|
||||
|
||||
// tryIPAPI 尝试 ip-api.com
|
||||
func tryIPAPI(client *http.Client) (string, string) {
|
||||
resp, err := client.Get("http://ip-api.com/json/?fields=status,country,countryCode,city,query")
|
||||
if err != nil {
|
||||
return "", ""
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
var result struct {
|
||||
Status string `json:"status"`
|
||||
Query string `json:"query"`
|
||||
Country string `json:"country"`
|
||||
CountryCode string `json:"countryCode"`
|
||||
City string `json:"city"`
|
||||
}
|
||||
|
||||
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil || result.Status != "success" {
|
||||
return "", ""
|
||||
}
|
||||
|
||||
location := result.CountryCode
|
||||
if result.City != "" {
|
||||
location = fmt.Sprintf("%s %s", result.CountryCode, result.City)
|
||||
}
|
||||
|
||||
return result.Query, location
|
||||
}
|
||||
|
||||
// tryIPAPICo 尝试 ipapi.co
|
||||
func tryIPAPICo(client *http.Client) (string, string) {
|
||||
resp, err := client.Get("https://ipapi.co/json/")
|
||||
if err != nil {
|
||||
return "", ""
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
var result struct {
|
||||
IP string `json:"ip"`
|
||||
City string `json:"city"`
|
||||
CountryCode string `json:"country_code"`
|
||||
}
|
||||
|
||||
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
|
||||
return "", ""
|
||||
}
|
||||
|
||||
location := result.CountryCode
|
||||
if result.City != "" {
|
||||
location = fmt.Sprintf("%s %s", result.CountryCode, result.City)
|
||||
}
|
||||
|
||||
return result.IP, location
|
||||
}
|
||||
|
||||
// tryIPInfo 尝试 ipinfo.io
|
||||
func tryIPInfo(client *http.Client) (string, string) {
|
||||
resp, err := client.Get("https://ipinfo.io/json")
|
||||
if err != nil {
|
||||
return "", ""
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
var result struct {
|
||||
IP string `json:"ip"`
|
||||
City string `json:"city"`
|
||||
Country string `json:"country"`
|
||||
}
|
||||
|
||||
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
|
||||
return "", ""
|
||||
}
|
||||
|
||||
location := result.Country
|
||||
if result.City != "" {
|
||||
location = fmt.Sprintf("%s %s", result.Country, result.City)
|
||||
}
|
||||
|
||||
return result.IP, location
|
||||
}
|
||||
|
||||
// tryHTTPBinIP 尝试 httpbin(仅获取IP)
|
||||
func tryHTTPBinIP(client *http.Client) string {
|
||||
resp, err := client.Get("https://httpbin.org/ip")
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
var result struct {
|
||||
Origin string `json:"origin"`
|
||||
}
|
||||
|
||||
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
|
||||
return ""
|
||||
}
|
||||
|
||||
return result.Origin
|
||||
}
|
||||
132
fetcher/source_manager.go
Normal file
132
fetcher/source_manager.go
Normal file
@@ -0,0 +1,132 @@
|
||||
package fetcher
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"log"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
// SourceManager 代理源管理器(断路器)
|
||||
type SourceManager struct {
|
||||
db *sql.DB
|
||||
mu sync.RWMutex
|
||||
}
|
||||
|
||||
func NewSourceManager(db *sql.DB) *SourceManager {
|
||||
return &SourceManager{db: db}
|
||||
}
|
||||
|
||||
// CanUseSource 判断源是否可用
|
||||
func (sm *SourceManager) CanUseSource(url string) bool {
|
||||
sm.mu.RLock()
|
||||
defer sm.mu.RUnlock()
|
||||
|
||||
var status string
|
||||
var disabledUntil sql.NullTime
|
||||
err := sm.db.QueryRow(
|
||||
`SELECT status, disabled_until FROM source_status WHERE url = ?`,
|
||||
url,
|
||||
).Scan(&status, &disabledUntil)
|
||||
|
||||
// 源不存在,默认可用
|
||||
if err != nil {
|
||||
return true
|
||||
}
|
||||
|
||||
// 检查是否被禁用且还在冷却期
|
||||
if status == "disabled" && disabledUntil.Valid {
|
||||
if time.Now().Before(disabledUntil.Time) {
|
||||
return false
|
||||
}
|
||||
// 冷却期结束,重置状态
|
||||
sm.db.Exec(`UPDATE source_status SET status = 'active', consecutive_fails = 0 WHERE url = ?`, url)
|
||||
return true
|
||||
}
|
||||
|
||||
return status != "disabled"
|
||||
}
|
||||
|
||||
// RecordSuccess 记录源抓取成功
|
||||
func (sm *SourceManager) RecordSuccess(url string) {
|
||||
sm.mu.Lock()
|
||||
defer sm.mu.Unlock()
|
||||
|
||||
sm.db.Exec(`
|
||||
INSERT INTO source_status (url, success_count, consecutive_fails, last_success, status)
|
||||
VALUES (?, 1, 0, CURRENT_TIMESTAMP, 'active')
|
||||
ON CONFLICT(url) DO UPDATE SET
|
||||
success_count = success_count + 1,
|
||||
consecutive_fails = 0,
|
||||
last_success = CURRENT_TIMESTAMP,
|
||||
status = 'active'
|
||||
`, url)
|
||||
}
|
||||
|
||||
// RecordFail 记录源抓取失败
|
||||
func (sm *SourceManager) RecordFail(url string, failThreshold, disableThreshold, cooldownMinutes int) {
|
||||
sm.mu.Lock()
|
||||
defer sm.mu.Unlock()
|
||||
|
||||
// 增加失败计数
|
||||
sm.db.Exec(`
|
||||
INSERT INTO source_status (url, fail_count, consecutive_fails, last_fail)
|
||||
VALUES (?, 1, 1, CURRENT_TIMESTAMP)
|
||||
ON CONFLICT(url) DO UPDATE SET
|
||||
fail_count = fail_count + 1,
|
||||
consecutive_fails = consecutive_fails + 1,
|
||||
last_fail = CURRENT_TIMESTAMP
|
||||
`, url)
|
||||
|
||||
// 检查是否需要降级或禁用
|
||||
var consecutiveFails int
|
||||
sm.db.QueryRow(`SELECT consecutive_fails FROM source_status WHERE url = ?`, url).Scan(&consecutiveFails)
|
||||
|
||||
if consecutiveFails >= disableThreshold {
|
||||
// 禁用源
|
||||
disabledUntil := time.Now().Add(time.Duration(cooldownMinutes) * time.Minute)
|
||||
sm.db.Exec(
|
||||
`UPDATE source_status SET status = 'disabled', disabled_until = ? WHERE url = ?`,
|
||||
disabledUntil, url,
|
||||
)
|
||||
log.Printf("[source] ⛔ 禁用源(连续失败%d次): %s (冷却%d分钟)", consecutiveFails, url, cooldownMinutes)
|
||||
} else if consecutiveFails >= failThreshold {
|
||||
// 降级源
|
||||
sm.db.Exec(`UPDATE source_status SET status = 'degraded' WHERE url = ?`, url)
|
||||
log.Printf("[source] ⚠️ 降级源(连续失败%d次): %s", consecutiveFails, url)
|
||||
}
|
||||
}
|
||||
|
||||
// GetSourceStats 获取所有源的统计信息
|
||||
func (sm *SourceManager) GetSourceStats() ([]map[string]interface{}, error) {
|
||||
rows, err := sm.db.Query(`
|
||||
SELECT url, success_count, fail_count, consecutive_fails,
|
||||
last_success, last_fail, status
|
||||
FROM source_status
|
||||
ORDER BY success_count DESC
|
||||
`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var stats []map[string]interface{}
|
||||
for rows.Next() {
|
||||
var url, status string
|
||||
var successCount, failCount, consecutiveFails int
|
||||
var lastSuccess, lastFail sql.NullTime
|
||||
|
||||
rows.Scan(&url, &successCount, &failCount, &consecutiveFails, &lastSuccess, &lastFail, &status)
|
||||
|
||||
stats = append(stats, map[string]interface{}{
|
||||
"url": url,
|
||||
"success_count": successCount,
|
||||
"fail_count": failCount,
|
||||
"consecutive_fails": consecutiveFails,
|
||||
"last_success": lastSuccess,
|
||||
"last_fail": lastFail,
|
||||
"status": status,
|
||||
})
|
||||
}
|
||||
return stats, nil
|
||||
}
|
||||
4
go.mod
4
go.mod
@@ -1,4 +1,4 @@
|
||||
module proxy-pool
|
||||
module goproxy
|
||||
|
||||
go 1.25.0
|
||||
|
||||
@@ -6,3 +6,5 @@ require (
|
||||
github.com/mattn/go-sqlite3 v1.14.37
|
||||
golang.org/x/net v0.38.0
|
||||
)
|
||||
|
||||
require golang.org/x/time v0.15.0
|
||||
|
||||
2
go.sum
2
go.sum
@@ -2,3 +2,5 @@ github.com/mattn/go-sqlite3 v1.14.37 h1:3DOZp4cXis1cUIpCfXLtmlGolNLp2VEqhiB/PARN
|
||||
github.com/mattn/go-sqlite3 v1.14.37/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
|
||||
golang.org/x/net v0.38.0 h1:vRMAPTMaeGqVhG5QyLJHqNDwecKTomGeqbnfZyKlBI8=
|
||||
golang.org/x/net v0.38.0/go.mod h1:ivrbrMbzFq5J41QOQh0siUuly180yBYtLp+CKbEaFx8=
|
||||
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
|
||||
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
|
||||
|
||||
@@ -25,8 +25,8 @@ type writer struct{}
|
||||
|
||||
func (w *writer) Write(p []byte) (n int, err error) {
|
||||
line := strings.TrimRight(string(p), "\n")
|
||||
ts := time.Now().Format("2006/01/02 15:04:05")
|
||||
formatted := fmt.Sprintf("%s %s", ts, line)
|
||||
ts := time.Now().Format("15:04:05")
|
||||
formatted := fmt.Sprintf("[%s] %s", ts, line)
|
||||
|
||||
mu.Lock()
|
||||
lines = append(lines, formatted)
|
||||
|
||||
263
main.go
263
main.go
@@ -7,14 +7,16 @@ import (
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"proxy-pool/checker"
|
||||
"proxy-pool/config"
|
||||
"proxy-pool/fetcher"
|
||||
"proxy-pool/logger"
|
||||
"proxy-pool/proxy"
|
||||
"proxy-pool/storage"
|
||||
"proxy-pool/validator"
|
||||
"proxy-pool/webui"
|
||||
"goproxy/checker"
|
||||
"goproxy/config"
|
||||
"goproxy/fetcher"
|
||||
"goproxy/logger"
|
||||
"goproxy/optimizer"
|
||||
"goproxy/pool"
|
||||
"goproxy/proxy"
|
||||
"goproxy/storage"
|
||||
"goproxy/validator"
|
||||
"goproxy/webui"
|
||||
)
|
||||
|
||||
var fetchRunning atomic.Bool
|
||||
@@ -24,7 +26,7 @@ func main() {
|
||||
// 初始化日志收集器
|
||||
logger.Init()
|
||||
|
||||
// 加载配置(优先读取 config.json)
|
||||
// 加载配置
|
||||
cfg := config.Load()
|
||||
|
||||
// 提示密码信息
|
||||
@@ -34,6 +36,9 @@ func main() {
|
||||
log.Println("[main] WebUI 密码已通过环境变量 WEBUI_PASSWORD 设置")
|
||||
}
|
||||
|
||||
log.Printf("[main] 🎯 智能代理池配置: 容量=%d HTTP=%.0f%% SOCKS5=%.0f%% 延迟标准=%dms",
|
||||
cfg.PoolMaxSize, cfg.PoolHTTPRatio*100, (1-cfg.PoolHTTPRatio)*100, cfg.MaxLatencyMs)
|
||||
|
||||
// 初始化存储
|
||||
store, err := storage.New(cfg.DBPath)
|
||||
if err != nil {
|
||||
@@ -41,102 +46,210 @@ func main() {
|
||||
}
|
||||
defer store.Close()
|
||||
|
||||
// 初始化模块
|
||||
fetch := fetcher.New(cfg.HTTPSourceURL, cfg.SOCKS5SourceURL)
|
||||
check := checker.New(store, validator.New(cfg.ValidateConcurrency, cfg.ValidateTimeout, cfg.ValidateURL), cfg)
|
||||
server := proxy.New(store, cfg)
|
||||
// 初始化限流器
|
||||
fetcher.InitIPQueryLimiter(cfg.IPQueryRateLimit)
|
||||
|
||||
// 初始化核心模块
|
||||
sourceMgr := fetcher.NewSourceManager(store.GetDB())
|
||||
fetch := fetcher.New(cfg.HTTPSourceURL, cfg.SOCKS5SourceURL, sourceMgr)
|
||||
validate := validator.New(cfg.ValidateConcurrency, cfg.ValidateTimeout, cfg.ValidateURL)
|
||||
poolMgr := pool.NewManager(store, cfg)
|
||||
healthChecker := checker.NewHealthChecker(store, validate, cfg, poolMgr)
|
||||
opt := optimizer.NewOptimizer(store, fetch, validate, poolMgr, cfg)
|
||||
|
||||
// 清理无效代理
|
||||
totalDeleted := 0
|
||||
if deleted, err := store.DeleteChinaMainland(); err == nil && deleted > 0 {
|
||||
log.Printf("[main] 🧹 已清理 %d 个中国大陆出口代理", deleted)
|
||||
totalDeleted += int(deleted)
|
||||
}
|
||||
if deleted, err := store.DeleteWithoutExitInfo(); err == nil && deleted > 0 {
|
||||
log.Printf("[main] 🧹 已清理 %d 个无出口信息的代理", deleted)
|
||||
totalDeleted += int(deleted)
|
||||
}
|
||||
|
||||
// 创建两个代理服务器:随机轮换 + 最低延迟
|
||||
randomServer := proxy.New(store, cfg, "random", cfg.ProxyPort)
|
||||
stableServer := proxy.New(store, cfg, "lowest-latency", cfg.StableProxyPort)
|
||||
|
||||
// 配置变更通知 channel
|
||||
configChanged := make(chan struct{}, 1)
|
||||
|
||||
// 启动 WebUI
|
||||
ui := webui.New(store, cfg, func() {
|
||||
go func() {
|
||||
if err := fetchAndValidate(fetch, store); err != nil {
|
||||
log.Printf("[webui] fetch error: %v", err)
|
||||
}
|
||||
}()
|
||||
// 启动 WebUI(传递池子管理器)
|
||||
ui := webui.New(store, cfg, poolMgr, func() {
|
||||
go smartFetchAndFill(fetch, validate, store, poolMgr)
|
||||
}, configChanged)
|
||||
ui.Start()
|
||||
|
||||
// 后台启动:首次抓取验证
|
||||
// 首次智能填充(清理后立即触发)
|
||||
go func() {
|
||||
log.Println("[main] fetching proxies on startup...")
|
||||
if err := fetchAndValidate(fetch, store); err != nil {
|
||||
log.Printf("[main] initial fetch error: %v", err)
|
||||
if totalDeleted > 0 {
|
||||
log.Printf("[main] 🚀 清理后立即启动补充填充...")
|
||||
} else {
|
||||
log.Println("[main] 🚀 启动初始化填充...")
|
||||
}
|
||||
smartFetchAndFill(fetch, validate, store, poolMgr)
|
||||
}()
|
||||
|
||||
// 启动状态监控协程
|
||||
go startStatusMonitor(poolMgr, fetch, validate, store)
|
||||
|
||||
// 启动健康检查器
|
||||
healthChecker.StartBackground()
|
||||
|
||||
// 启动优化轮换器
|
||||
opt.StartBackground()
|
||||
|
||||
// 监听配置变更
|
||||
go watchConfigChanges(configChanged, poolMgr)
|
||||
|
||||
// 启动稳定代理服务(最低延迟模式)
|
||||
go func() {
|
||||
if err := stableServer.Start(); err != nil {
|
||||
log.Fatalf("stable proxy server: %v", err)
|
||||
}
|
||||
}()
|
||||
|
||||
// 启动动态定时抓取
|
||||
go startFetchLoop(fetch, store, configChanged)
|
||||
|
||||
// 启动定时健康检查
|
||||
check.Start()
|
||||
|
||||
// 启动代理服务(阻塞)
|
||||
if err := server.Start(); err != nil {
|
||||
log.Fatalf("proxy server: %v", err)
|
||||
// 启动随机代理服务(阻塞)
|
||||
if err := randomServer.Start(); err != nil {
|
||||
log.Fatalf("random proxy server: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func fetchAndValidate(fetch *fetcher.Fetcher, store *storage.Storage) error {
|
||||
// smartFetchAndFill 智能抓取和填充
|
||||
func smartFetchAndFill(fetch *fetcher.Fetcher, validate *validator.Validator, store *storage.Storage, poolMgr *pool.Manager) {
|
||||
// 防止并发执行
|
||||
if !fetchRunning.CompareAndSwap(false, true) {
|
||||
log.Println("[main] fetch already running, skipping")
|
||||
return nil
|
||||
log.Println("[main] 抓取已在运行,跳过")
|
||||
return
|
||||
}
|
||||
defer fetchRunning.Store(false)
|
||||
|
||||
proxies, err := fetch.Fetch()
|
||||
// 获取池子状态
|
||||
status, err := poolMgr.GetStatus()
|
||||
if err != nil {
|
||||
return err
|
||||
log.Printf("[main] 获取池子状态失败: %v", err)
|
||||
return
|
||||
}
|
||||
log.Printf("[main] validating %d proxies (streaming)...", len(proxies))
|
||||
|
||||
// 每次用最新配置创建 validator
|
||||
cfg := config.Get()
|
||||
validate := validator.New(cfg.ValidateConcurrency, cfg.ValidateTimeout, cfg.ValidateURL)
|
||||
log.Printf("[main] 📊 池子状态: %s | HTTP=%d/%d SOCKS5=%d/%d 总计=%d/%d",
|
||||
status.State, status.HTTP, status.HTTPSlots, status.SOCKS5, status.SOCKS5Slots,
|
||||
status.Total, config.Get().PoolMaxSize)
|
||||
|
||||
var valid, total int
|
||||
for r := range validate.ValidateStream(proxies) {
|
||||
total++
|
||||
if total%1000 == 0 {
|
||||
log.Printf("[main] checked=%d/%d valid=%d", total, len(proxies), valid)
|
||||
// 判断是否需要抓取
|
||||
needFetch, mode, preferredProtocol := poolMgr.NeedsFetch(status)
|
||||
if !needFetch {
|
||||
log.Println("[main] 池子健康,无需抓取")
|
||||
return
|
||||
}
|
||||
|
||||
log.Printf("[main] 🔍 智能抓取: 模式=%s 协议偏好=%s", mode, preferredProtocol)
|
||||
|
||||
// 智能抓取
|
||||
candidates, err := fetch.FetchSmart(mode, preferredProtocol)
|
||||
if err != nil {
|
||||
log.Printf("[main] 抓取失败: %v", err)
|
||||
return
|
||||
}
|
||||
|
||||
log.Printf("[main] 抓取到 %d 个候选代理,开始严格验证...", len(candidates))
|
||||
|
||||
// 严格验证并尝试入池
|
||||
addedCount := 0
|
||||
validCount := 0
|
||||
|
||||
for result := range validate.ValidateStream(candidates) {
|
||||
if !result.Valid {
|
||||
continue
|
||||
}
|
||||
if r.Valid {
|
||||
if valid == 0 {
|
||||
log.Printf("[main] first valid proxy: %s (%s) latency=%v", r.Proxy.Address, r.Proxy.Protocol, r.Latency)
|
||||
}
|
||||
valid++
|
||||
if err := store.AddProxy(r.Proxy.Address, r.Proxy.Protocol); err != nil {
|
||||
log.Printf("[main] addProxy error: %v", err)
|
||||
}
|
||||
if valid%10 == 0 {
|
||||
log.Printf("[main] progress: valid=%d checked=%d/%d", valid, total, len(proxies))
|
||||
|
||||
validCount++
|
||||
latencyMs := int(result.Latency.Milliseconds())
|
||||
|
||||
// 根据池子状态动态调整延迟标准
|
||||
cfg := config.Get()
|
||||
maxLatency := cfg.GetLatencyThreshold(status.State)
|
||||
|
||||
// 必须满足:有出口IP、有位置、延迟达标
|
||||
if result.ExitIP == "" || result.ExitLocation == "" || latencyMs > maxLatency {
|
||||
continue
|
||||
}
|
||||
|
||||
// 尝试加入池子
|
||||
proxyToAdd := storage.Proxy{
|
||||
Address: result.Proxy.Address,
|
||||
Protocol: result.Proxy.Protocol,
|
||||
ExitIP: result.ExitIP,
|
||||
ExitLocation: result.ExitLocation,
|
||||
Latency: latencyMs,
|
||||
}
|
||||
|
||||
if added, _ := poolMgr.TryAddProxy(proxyToAdd); added {
|
||||
addedCount++
|
||||
}
|
||||
|
||||
// 如果是紧急模式且已达到最小要求,停止验证
|
||||
if mode == "emergency" && status.HTTP >= cfg.PoolMinPerProtocol && status.SOCKS5 >= cfg.PoolMinPerProtocol {
|
||||
log.Println("[main] 🎉 紧急模式:达到最小要求,停止验证")
|
||||
break
|
||||
}
|
||||
|
||||
// 动态检查是否已经填满
|
||||
if addedCount > 0 && addedCount%20 == 0 {
|
||||
currentStatus, _ := poolMgr.GetStatus()
|
||||
if !poolMgr.NeedsFetchQuick(currentStatus) {
|
||||
log.Println("[main] ✅ 池子已填满,停止验证")
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
count, _ := store.Count()
|
||||
log.Printf("[main] validated: valid=%d/%d, pool size=%d", valid, len(proxies), count)
|
||||
return nil
|
||||
// 最终状态
|
||||
finalStatus, _ := poolMgr.GetStatus()
|
||||
log.Printf("[main] 填充完成: 验证%d 通过%d 入池%d | 最终状态: %s HTTP=%d SOCKS5=%d",
|
||||
len(candidates), validCount, addedCount,
|
||||
finalStatus.State, finalStatus.HTTP, finalStatus.SOCKS5)
|
||||
}
|
||||
|
||||
func startFetchLoop(fetch *fetcher.Fetcher, store *storage.Storage, configChanged <-chan struct{}) {
|
||||
cfg := config.Get()
|
||||
ticker := time.NewTicker(time.Duration(cfg.FetchInterval) * time.Minute)
|
||||
log.Printf("[main] fetch loop started, interval: %d min", cfg.FetchInterval)
|
||||
for {
|
||||
select {
|
||||
case <-ticker.C:
|
||||
log.Println("[main] scheduled fetch started...")
|
||||
if err := fetchAndValidate(fetch, store); err != nil {
|
||||
log.Printf("[main] scheduled fetch error: %v", err)
|
||||
}
|
||||
case <-configChanged:
|
||||
newCfg := config.Get()
|
||||
ticker.Reset(time.Duration(newCfg.FetchInterval) * time.Minute)
|
||||
log.Printf("[main] fetch interval updated to %d min", newCfg.FetchInterval)
|
||||
// startStatusMonitor 状态监控协程
|
||||
func startStatusMonitor(poolMgr *pool.Manager, fetch *fetcher.Fetcher, validate *validator.Validator, store *storage.Storage) {
|
||||
ticker := time.NewTicker(30 * time.Second)
|
||||
log.Println("[monitor] 📡 状态监控器已启动(每30秒检查)")
|
||||
|
||||
for range ticker.C {
|
||||
status, err := poolMgr.GetStatus()
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
|
||||
// 每分钟检查池子状态
|
||||
needFetch, mode, preferredProtocol := poolMgr.NeedsFetch(status)
|
||||
if needFetch {
|
||||
log.Printf("[monitor] ⚠️ 检测到池子需求: 状态=%s 模式=%s 协议=%s",
|
||||
status.State, mode, preferredProtocol)
|
||||
// 触发智能填充
|
||||
go smartFetchAndFill(fetch, validate, store, poolMgr)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// watchConfigChanges 监听配置变更
|
||||
func watchConfigChanges(configChanged <-chan struct{}, poolMgr *pool.Manager) {
|
||||
var oldSize int
|
||||
var oldRatio float64
|
||||
|
||||
cfg := config.Get()
|
||||
oldSize = cfg.PoolMaxSize
|
||||
oldRatio = cfg.PoolHTTPRatio
|
||||
|
||||
for range configChanged {
|
||||
newCfg := config.Get()
|
||||
if newCfg.PoolMaxSize != oldSize || newCfg.PoolHTTPRatio != oldRatio {
|
||||
log.Printf("[config] 🔧 配置变更检测: 容量 %d→%d 比例 %.2f→%.2f",
|
||||
oldSize, newCfg.PoolMaxSize, oldRatio, newCfg.PoolHTTPRatio)
|
||||
poolMgr.AdjustForConfigChange(oldSize, oldRatio)
|
||||
oldSize = newCfg.PoolMaxSize
|
||||
oldRatio = newCfg.PoolHTTPRatio
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
108
optimizer/optimizer.go
Normal file
108
optimizer/optimizer.go
Normal file
@@ -0,0 +1,108 @@
|
||||
package optimizer
|
||||
|
||||
import (
|
||||
"log"
|
||||
"time"
|
||||
|
||||
"goproxy/config"
|
||||
"goproxy/fetcher"
|
||||
"goproxy/pool"
|
||||
"goproxy/storage"
|
||||
"goproxy/validator"
|
||||
)
|
||||
|
||||
// Optimizer 优化轮换器
|
||||
type Optimizer struct {
|
||||
storage *storage.Storage
|
||||
fetcher *fetcher.Fetcher
|
||||
validator *validator.Validator
|
||||
poolMgr *pool.Manager
|
||||
cfg *config.Config
|
||||
}
|
||||
|
||||
func NewOptimizer(s *storage.Storage, f *fetcher.Fetcher, v *validator.Validator, pm *pool.Manager, cfg *config.Config) *Optimizer {
|
||||
return &Optimizer{
|
||||
storage: s,
|
||||
fetcher: f,
|
||||
validator: v,
|
||||
poolMgr: pm,
|
||||
cfg: cfg,
|
||||
}
|
||||
}
|
||||
|
||||
// RunOnce 执行一次优化轮换
|
||||
func (o *Optimizer) RunOnce() {
|
||||
start := time.Now()
|
||||
log.Println("[optimize] 🎯 开始优化轮换...")
|
||||
|
||||
// 获取池子状态
|
||||
status, err := o.poolMgr.GetStatus()
|
||||
if err != nil {
|
||||
log.Printf("[optimize] 获取状态失败: %v", err)
|
||||
return
|
||||
}
|
||||
|
||||
// 只有健康状态才执行优化
|
||||
if status.State != "healthy" {
|
||||
log.Printf("[optimize] 池子状态 %s,跳过优化", status.State)
|
||||
return
|
||||
}
|
||||
|
||||
// 抓取新的候选代理(优化模式)
|
||||
log.Println("[optimize] 抓取新候选代理...")
|
||||
candidates, err := o.fetcher.FetchSmart("optimize", "")
|
||||
if err != nil {
|
||||
log.Printf("[optimize] 抓取失败: %v", err)
|
||||
return
|
||||
}
|
||||
|
||||
log.Printf("[optimize] 抓取到 %d 个候选代理", len(candidates))
|
||||
|
||||
// 验证候选代理
|
||||
validCandidates := []storage.Proxy{}
|
||||
for result := range o.validator.ValidateStream(candidates) {
|
||||
if result.Valid {
|
||||
latencyMs := int(result.Latency.Milliseconds())
|
||||
// 只保留延迟在健康标准内的
|
||||
if latencyMs <= o.cfg.MaxLatencyHealthy {
|
||||
validCandidates = append(validCandidates, storage.Proxy{
|
||||
Address: result.Proxy.Address,
|
||||
Protocol: result.Proxy.Protocol,
|
||||
ExitIP: result.ExitIP,
|
||||
ExitLocation: result.ExitLocation,
|
||||
Latency: latencyMs,
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
log.Printf("[optimize] 验证通过 %d 个优质候选(延迟<%dms)", len(validCandidates), o.cfg.MaxLatencyHealthy)
|
||||
|
||||
if len(validCandidates) == 0 {
|
||||
log.Println("[optimize] 无优质候选,跳过优化")
|
||||
return
|
||||
}
|
||||
|
||||
// 尝试用优质候选替换延迟高的代理
|
||||
replacedCount := 0
|
||||
for _, candidate := range validCandidates {
|
||||
added, reason := o.poolMgr.TryAddProxy(candidate)
|
||||
if added && reason == "replaced" {
|
||||
replacedCount++
|
||||
}
|
||||
}
|
||||
|
||||
elapsed := time.Since(start)
|
||||
log.Printf("[optimize] ✅ 完成: 替换 %d 个代理 耗时%v", replacedCount, elapsed)
|
||||
}
|
||||
|
||||
// StartBackground 后台定时优化
|
||||
func (o *Optimizer) StartBackground() {
|
||||
ticker := time.NewTicker(time.Duration(o.cfg.OptimizeInterval) * time.Minute)
|
||||
go func() {
|
||||
for range ticker.C {
|
||||
o.RunOnce()
|
||||
}
|
||||
}()
|
||||
log.Printf("[optimize] 优化轮换器已启动,间隔 %d 分钟", o.cfg.OptimizeInterval)
|
||||
}
|
||||
229
pool/manager.go
Normal file
229
pool/manager.go
Normal file
@@ -0,0 +1,229 @@
|
||||
package pool
|
||||
|
||||
import (
|
||||
"log"
|
||||
|
||||
"goproxy/config"
|
||||
"goproxy/storage"
|
||||
)
|
||||
|
||||
// Manager 池子管理器
|
||||
type Manager struct {
|
||||
storage *storage.Storage
|
||||
cfg *config.Config
|
||||
}
|
||||
|
||||
func NewManager(s *storage.Storage, cfg *config.Config) *Manager {
|
||||
return &Manager{
|
||||
storage: s,
|
||||
cfg: cfg,
|
||||
}
|
||||
}
|
||||
|
||||
// PoolStatus 池子状态
|
||||
type PoolStatus struct {
|
||||
Total int
|
||||
HTTP int
|
||||
SOCKS5 int
|
||||
HTTPSlots int
|
||||
SOCKS5Slots int
|
||||
State string // healthy/warning/critical/emergency
|
||||
AvgLatencyHTTP int
|
||||
AvgLatencySocks5 int
|
||||
}
|
||||
|
||||
// GetStatus 获取当前池子状态
|
||||
func (m *Manager) GetStatus() (*PoolStatus, error) {
|
||||
total, _ := m.storage.Count()
|
||||
httpCount, _ := m.storage.CountByProtocol("http")
|
||||
socks5Count, _ := m.storage.CountByProtocol("socks5")
|
||||
|
||||
httpSlots, socks5Slots := m.cfg.CalculateSlots()
|
||||
|
||||
// 计算平均延迟
|
||||
avgHTTP, _ := m.storage.GetAverageLatency("http")
|
||||
avgSOCKS5, _ := m.storage.GetAverageLatency("socks5")
|
||||
|
||||
// 判断状态
|
||||
state := m.determineState(total, httpCount, socks5Count)
|
||||
|
||||
return &PoolStatus{
|
||||
Total: total,
|
||||
HTTP: httpCount,
|
||||
SOCKS5: socks5Count,
|
||||
HTTPSlots: httpSlots,
|
||||
SOCKS5Slots: socks5Slots,
|
||||
State: state,
|
||||
AvgLatencyHTTP: avgHTTP,
|
||||
AvgLatencySocks5: avgSOCKS5,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// determineState 判断池子状态
|
||||
func (m *Manager) determineState(total, httpCount, socks5Count int) string {
|
||||
httpSlots, socks5Slots := m.cfg.CalculateSlots()
|
||||
|
||||
// 单协议缺失
|
||||
if httpCount == 0 || socks5Count == 0 {
|
||||
return "emergency"
|
||||
}
|
||||
|
||||
// 紧急:总数<10%
|
||||
emergencyThreshold := int(float64(m.cfg.PoolMaxSize) * 0.1)
|
||||
if total < emergencyThreshold {
|
||||
return "emergency"
|
||||
}
|
||||
|
||||
// 危急:任一协议<20%槽位
|
||||
if httpCount < int(float64(httpSlots)*0.2) || socks5Count < int(float64(socks5Slots)*0.2) {
|
||||
return "critical"
|
||||
}
|
||||
|
||||
// 警告:总数<95%
|
||||
healthyThreshold := int(float64(m.cfg.PoolMaxSize) * 0.95)
|
||||
if total < healthyThreshold {
|
||||
return "warning"
|
||||
}
|
||||
|
||||
// 健康
|
||||
return "healthy"
|
||||
}
|
||||
|
||||
// NeedsFetch 判断是否需要抓取以及抓取模式
|
||||
func (m *Manager) NeedsFetch(status *PoolStatus) (bool, string, string) {
|
||||
// 单协议缺失:紧急模式,指定协议
|
||||
if status.HTTP == 0 {
|
||||
return true, "emergency", "http"
|
||||
}
|
||||
if status.SOCKS5 == 0 {
|
||||
return true, "emergency", "socks5"
|
||||
}
|
||||
|
||||
// 紧急状态:紧急模式
|
||||
if status.State == "emergency" {
|
||||
return true, "emergency", ""
|
||||
}
|
||||
|
||||
// 危急或警告:补充模式
|
||||
if status.State == "critical" || status.State == "warning" {
|
||||
// 判断哪个协议更缺
|
||||
httpPct := float64(status.HTTP) / float64(status.HTTPSlots)
|
||||
socks5Pct := float64(status.SOCKS5) / float64(status.SOCKS5Slots)
|
||||
|
||||
if httpPct < 0.5 {
|
||||
return true, "refill", "http"
|
||||
}
|
||||
if socks5Pct < 0.5 {
|
||||
return true, "refill", "socks5"
|
||||
}
|
||||
return true, "refill", ""
|
||||
}
|
||||
|
||||
// 健康状态:不需要补充抓取
|
||||
return false, "", ""
|
||||
}
|
||||
|
||||
// NeedsFetchQuick 快速判断是否还需要抓取(用于提前终止)
|
||||
func (m *Manager) NeedsFetchQuick(status *PoolStatus) bool {
|
||||
need, _, _ := m.NeedsFetch(status)
|
||||
return need
|
||||
}
|
||||
|
||||
// TryAddProxy 尝试将代理加入池子
|
||||
func (m *Manager) TryAddProxy(p storage.Proxy) (bool, string) {
|
||||
httpSlots, socks5Slots := m.cfg.CalculateSlots()
|
||||
httpCount, _ := m.storage.CountByProtocol("http")
|
||||
socks5Count, _ := m.storage.CountByProtocol("socks5")
|
||||
total, _ := m.storage.Count()
|
||||
|
||||
var maxSlots int
|
||||
var currentCount int
|
||||
if p.Protocol == "http" {
|
||||
maxSlots = httpSlots
|
||||
currentCount = httpCount
|
||||
} else {
|
||||
maxSlots = socks5Slots
|
||||
currentCount = socks5Count
|
||||
}
|
||||
|
||||
// 情况1:该协议槽位未满,直接加入
|
||||
if currentCount < maxSlots {
|
||||
if err := m.storage.AddProxy(p.Address, p.Protocol); err != nil {
|
||||
return false, "db_error"
|
||||
}
|
||||
// 更新完整信息
|
||||
m.storage.UpdateExitInfo(p.Address, p.ExitIP, p.ExitLocation, p.Latency)
|
||||
log.Printf("[pool] ✅ 直接入池: %s (%s %d/%d) %dms %s %s",
|
||||
p.Address, p.Protocol, currentCount+1, maxSlots, p.Latency, p.ExitIP, p.ExitLocation)
|
||||
return true, "added"
|
||||
}
|
||||
|
||||
// 情况2:槽位满,但允许10%浮动
|
||||
allowedFloat := int(float64(maxSlots) * 0.1)
|
||||
if total < m.cfg.PoolMaxSize && currentCount < maxSlots+allowedFloat {
|
||||
if err := m.storage.AddProxy(p.Address, p.Protocol); err != nil {
|
||||
return false, "db_error"
|
||||
}
|
||||
m.storage.UpdateExitInfo(p.Address, p.ExitIP, p.ExitLocation, p.Latency)
|
||||
log.Printf("[pool] ✅ 浮动入池: %s (%s %d/%d+%d) %dms",
|
||||
p.Address, p.Protocol, currentCount+1, maxSlots, allowedFloat, p.Latency)
|
||||
return true, "added_float"
|
||||
}
|
||||
|
||||
// 情况3:池子满了,尝试替换
|
||||
if currentCount >= maxSlots || total >= m.cfg.PoolMaxSize {
|
||||
return m.tryReplace(p)
|
||||
}
|
||||
|
||||
return false, "slots_full"
|
||||
}
|
||||
|
||||
// tryReplace 尝试替换现有代理
|
||||
func (m *Manager) tryReplace(newProxy storage.Proxy) (bool, string) {
|
||||
// 获取同协议中可替换的代理(延迟最高的前10个)
|
||||
candidates, err := m.storage.GetWorstProxies(newProxy.Protocol, 10)
|
||||
if err != nil || len(candidates) == 0 {
|
||||
return false, "no_candidates"
|
||||
}
|
||||
|
||||
worst := candidates[0]
|
||||
|
||||
// 判断是否值得替换:新代理需要显著更快
|
||||
threshold := m.cfg.ReplaceThreshold
|
||||
if float64(newProxy.Latency) < float64(worst.Latency)*threshold {
|
||||
if err := m.storage.ReplaceProxy(worst.Address, newProxy); err != nil {
|
||||
return false, "replace_error"
|
||||
}
|
||||
log.Printf("[pool] 🔄 替换: %s(%dms) → %s(%dms) 提升%.0f%%",
|
||||
worst.Address, worst.Latency, newProxy.Address, newProxy.Latency,
|
||||
(1-float64(newProxy.Latency)/float64(worst.Latency))*100)
|
||||
return true, "replaced"
|
||||
}
|
||||
|
||||
return false, "not_better"
|
||||
}
|
||||
|
||||
// AdjustForConfigChange 配置变更后调整池子
|
||||
func (m *Manager) AdjustForConfigChange(oldSize int, oldRatio float64) {
|
||||
newHTTP, newSOCKS5 := m.cfg.CalculateSlots()
|
||||
oldHTTP := int(float64(oldSize) * oldRatio)
|
||||
oldSOCKS5 := oldSize - oldHTTP
|
||||
|
||||
log.Printf("[pool] 配置变更: 容量 %d→%d, HTTP槽位 %d→%d, SOCKS5槽位 %d→%d",
|
||||
oldSize, m.cfg.PoolMaxSize, oldHTTP, newHTTP, oldSOCKS5, newSOCKS5)
|
||||
|
||||
// 如果槽位减少且当前超标,标记超标的为替换候选
|
||||
httpCount, _ := m.storage.CountByProtocol("http")
|
||||
socks5Count, _ := m.storage.CountByProtocol("socks5")
|
||||
|
||||
if httpCount > newHTTP {
|
||||
excess := httpCount - newHTTP
|
||||
log.Printf("[pool] HTTP 超标 %d 个,标记为替换候选", excess)
|
||||
// 这里可以标记延迟高的为候选
|
||||
}
|
||||
|
||||
if socks5Count > newSOCKS5 {
|
||||
excess := socks5Count - newSOCKS5
|
||||
log.Printf("[pool] SOCKS5 超标 %d 个,标记为替换候选", excess)
|
||||
}
|
||||
}
|
||||
@@ -10,25 +10,33 @@ import (
|
||||
"time"
|
||||
|
||||
"golang.org/x/net/proxy"
|
||||
"proxy-pool/config"
|
||||
"proxy-pool/storage"
|
||||
"goproxy/config"
|
||||
"goproxy/storage"
|
||||
)
|
||||
|
||||
type Server struct {
|
||||
storage *storage.Storage
|
||||
cfg *config.Config
|
||||
mode string // "random" 或 "lowest-latency"
|
||||
port string
|
||||
}
|
||||
|
||||
func New(s *storage.Storage, cfg *config.Config) *Server {
|
||||
func New(s *storage.Storage, cfg *config.Config, mode string, port string) *Server {
|
||||
return &Server{
|
||||
storage: s,
|
||||
cfg: cfg,
|
||||
mode: mode,
|
||||
port: port,
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Server) Start() error {
|
||||
log.Printf("proxy server listening on %s", s.cfg.ProxyPort)
|
||||
return http.ListenAndServe(s.cfg.ProxyPort, s)
|
||||
modeDesc := "随机轮换"
|
||||
if s.mode == "lowest-latency" {
|
||||
modeDesc = "最低延迟"
|
||||
}
|
||||
log.Printf("proxy server listening on %s [%s]", s.port, modeDesc)
|
||||
return http.ListenAndServe(s.port, s)
|
||||
}
|
||||
|
||||
func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
@@ -43,11 +51,22 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
func (s *Server) handleHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
var tried []string
|
||||
for attempt := 0; attempt <= s.cfg.MaxRetry; attempt++ {
|
||||
p, err := s.storage.GetRandomExclude(tried)
|
||||
var p *storage.Proxy
|
||||
var err error
|
||||
|
||||
// 根据模式选择代理
|
||||
if s.mode == "lowest-latency" {
|
||||
p, err = s.storage.GetLowestLatencyExclude(tried)
|
||||
} else {
|
||||
p, err = s.storage.GetRandomExclude(tried)
|
||||
}
|
||||
|
||||
if err != nil {
|
||||
http.Error(w, "no available proxy", http.StatusServiceUnavailable)
|
||||
return
|
||||
}
|
||||
|
||||
tried = append(tried, p.Address)
|
||||
|
||||
client, err := s.buildClient(p)
|
||||
if err != nil {
|
||||
@@ -94,11 +113,22 @@ func (s *Server) handleHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
func (s *Server) handleTunnel(w http.ResponseWriter, r *http.Request) {
|
||||
var tried []string
|
||||
for attempt := 0; attempt <= s.cfg.MaxRetry; attempt++ {
|
||||
p, err := s.storage.GetRandomExclude(tried)
|
||||
var p *storage.Proxy
|
||||
var err error
|
||||
|
||||
// 根据模式选择代理
|
||||
if s.mode == "lowest-latency" {
|
||||
p, err = s.storage.GetLowestLatencyExclude(tried)
|
||||
} else {
|
||||
p, err = s.storage.GetRandomExclude(tried)
|
||||
}
|
||||
|
||||
if err != nil {
|
||||
http.Error(w, "no available proxy", http.StatusServiceUnavailable)
|
||||
return
|
||||
}
|
||||
|
||||
tried = append(tried, p.Address)
|
||||
|
||||
conn, err := s.dialViaProxy(p, r.Host)
|
||||
if err != nil {
|
||||
|
||||
@@ -11,12 +11,33 @@ import (
|
||||
)
|
||||
|
||||
type Proxy struct {
|
||||
ID int64
|
||||
Address string // host:port
|
||||
Protocol string // http, socks5
|
||||
FailCount int
|
||||
LastCheck time.Time
|
||||
CreatedAt time.Time
|
||||
ID int64 `json:"id"`
|
||||
Address string `json:"address"`
|
||||
Protocol string `json:"protocol"`
|
||||
ExitIP string `json:"exit_ip"`
|
||||
ExitLocation string `json:"exit_location"`
|
||||
Latency int `json:"latency"`
|
||||
QualityGrade string `json:"quality_grade"`
|
||||
UseCount int `json:"use_count"`
|
||||
SuccessCount int `json:"success_count"`
|
||||
FailCount int `json:"fail_count"`
|
||||
LastUsed time.Time `json:"last_used"`
|
||||
LastCheck time.Time `json:"last_check"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
Status string `json:"status"`
|
||||
}
|
||||
|
||||
// SourceStatus 代理源状态
|
||||
type SourceStatus struct {
|
||||
ID int64
|
||||
URL string
|
||||
SuccessCount int
|
||||
FailCount int
|
||||
ConsecutiveFails int
|
||||
LastSuccess time.Time
|
||||
LastFail time.Time
|
||||
Status string // active/degraded/disabled
|
||||
DisabledUntil time.Time
|
||||
}
|
||||
|
||||
type Storage struct {
|
||||
@@ -39,17 +60,119 @@ func New(dbPath string) (*Storage, error) {
|
||||
}
|
||||
|
||||
func (s *Storage) initSchema() error {
|
||||
// 创建代理表
|
||||
_, err := s.db.Exec(`
|
||||
CREATE TABLE IF NOT EXISTS proxies (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
address TEXT NOT NULL UNIQUE,
|
||||
protocol TEXT NOT NULL,
|
||||
fail_count INTEGER NOT NULL DEFAULT 0,
|
||||
last_check DATETIME,
|
||||
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
address TEXT NOT NULL UNIQUE,
|
||||
protocol TEXT NOT NULL,
|
||||
exit_ip TEXT NOT NULL DEFAULT '',
|
||||
exit_location TEXT NOT NULL DEFAULT '',
|
||||
latency INTEGER NOT NULL DEFAULT 0,
|
||||
quality_grade TEXT NOT NULL DEFAULT 'C',
|
||||
use_count INTEGER NOT NULL DEFAULT 0,
|
||||
success_count INTEGER NOT NULL DEFAULT 0,
|
||||
fail_count INTEGER NOT NULL DEFAULT 0,
|
||||
last_used DATETIME,
|
||||
last_check DATETIME,
|
||||
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
status TEXT NOT NULL DEFAULT 'active'
|
||||
)
|
||||
`)
|
||||
return err
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// 创建索引
|
||||
s.db.Exec(`CREATE INDEX IF NOT EXISTS idx_protocol_latency ON proxies(protocol, latency)`)
|
||||
s.db.Exec(`CREATE INDEX IF NOT EXISTS idx_quality_grade ON proxies(quality_grade, latency)`)
|
||||
s.db.Exec(`CREATE INDEX IF NOT EXISTS idx_status ON proxies(status)`)
|
||||
|
||||
// 创建源状态表
|
||||
_, err = s.db.Exec(`
|
||||
CREATE TABLE IF NOT EXISTS source_status (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
url TEXT NOT NULL UNIQUE,
|
||||
success_count INTEGER NOT NULL DEFAULT 0,
|
||||
fail_count INTEGER NOT NULL DEFAULT 0,
|
||||
consecutive_fails INTEGER NOT NULL DEFAULT 0,
|
||||
last_success DATETIME,
|
||||
last_fail DATETIME,
|
||||
status TEXT NOT NULL DEFAULT 'active',
|
||||
disabled_until DATETIME
|
||||
)
|
||||
`)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// 迁移:处理旧的 location 字段(如果存在)
|
||||
var hasOldLocation int
|
||||
err = s.db.QueryRow(`SELECT COUNT(*) FROM pragma_table_info('proxies') WHERE name='location'`).Scan(&hasOldLocation)
|
||||
if err == nil && hasOldLocation > 0 {
|
||||
log.Println("[storage] migrating: renaming location to exit_location")
|
||||
// 如果有旧的 location 字段,先添加新字段再复制数据
|
||||
s.db.Exec(`ALTER TABLE proxies ADD COLUMN exit_location TEXT NOT NULL DEFAULT ''`)
|
||||
s.db.Exec(`UPDATE proxies SET exit_location = location WHERE location != ''`)
|
||||
}
|
||||
|
||||
// 迁移:添加 exit_ip 字段
|
||||
var hasExitIP int
|
||||
err = s.db.QueryRow(`SELECT COUNT(*) FROM pragma_table_info('proxies') WHERE name='exit_ip'`).Scan(&hasExitIP)
|
||||
if err == nil && hasExitIP == 0 {
|
||||
log.Println("[storage] migrating: adding exit_ip column")
|
||||
_, err = s.db.Exec(`ALTER TABLE proxies ADD COLUMN exit_ip TEXT NOT NULL DEFAULT ''`)
|
||||
if err != nil {
|
||||
return fmt.Errorf("migrate exit_ip column: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
// 迁移:添加 exit_location 字段
|
||||
var hasExitLocation int
|
||||
err = s.db.QueryRow(`SELECT COUNT(*) FROM pragma_table_info('proxies') WHERE name='exit_location'`).Scan(&hasExitLocation)
|
||||
if err == nil && hasExitLocation == 0 {
|
||||
log.Println("[storage] migrating: adding exit_location column")
|
||||
_, err = s.db.Exec(`ALTER TABLE proxies ADD COLUMN exit_location TEXT NOT NULL DEFAULT ''`)
|
||||
if err != nil {
|
||||
return fmt.Errorf("migrate exit_location column: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
// 迁移:添加 latency 字段
|
||||
var hasLatency int
|
||||
err = s.db.QueryRow(`SELECT COUNT(*) FROM pragma_table_info('proxies') WHERE name='latency'`).Scan(&hasLatency)
|
||||
if err == nil && hasLatency == 0 {
|
||||
log.Println("[storage] migrating: adding latency column")
|
||||
s.db.Exec(`ALTER TABLE proxies ADD COLUMN latency INTEGER NOT NULL DEFAULT 0`)
|
||||
}
|
||||
|
||||
// 迁移:添加质量等级字段
|
||||
var hasQuality int
|
||||
s.db.QueryRow(`SELECT COUNT(*) FROM pragma_table_info('proxies') WHERE name='quality_grade'`).Scan(&hasQuality)
|
||||
if hasQuality == 0 {
|
||||
log.Println("[storage] migrating: adding quality_grade column")
|
||||
s.db.Exec(`ALTER TABLE proxies ADD COLUMN quality_grade TEXT NOT NULL DEFAULT 'C'`)
|
||||
}
|
||||
|
||||
// 迁移:添加使用统计字段
|
||||
var hasUseCount int
|
||||
s.db.QueryRow(`SELECT COUNT(*) FROM pragma_table_info('proxies') WHERE name='use_count'`).Scan(&hasUseCount)
|
||||
if hasUseCount == 0 {
|
||||
log.Println("[storage] migrating: adding usage tracking columns")
|
||||
s.db.Exec(`ALTER TABLE proxies ADD COLUMN use_count INTEGER NOT NULL DEFAULT 0`)
|
||||
s.db.Exec(`ALTER TABLE proxies ADD COLUMN success_count INTEGER NOT NULL DEFAULT 0`)
|
||||
s.db.Exec(`ALTER TABLE proxies ADD COLUMN last_used DATETIME`)
|
||||
}
|
||||
|
||||
// 迁移:添加状态字段
|
||||
var hasStatus int
|
||||
s.db.QueryRow(`SELECT COUNT(*) FROM pragma_table_info('proxies') WHERE name='status'`).Scan(&hasStatus)
|
||||
if hasStatus == 0 {
|
||||
log.Println("[storage] migrating: adding status column")
|
||||
s.db.Exec(`ALTER TABLE proxies ADD COLUMN status TEXT NOT NULL DEFAULT 'active'`)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// AddProxy 新增代理,已存在则忽略
|
||||
@@ -82,12 +205,23 @@ func (s *Storage) AddProxies(proxies []Proxy) error {
|
||||
return tx.Commit()
|
||||
}
|
||||
|
||||
// GetRandom 随机取一个可用代理
|
||||
// GetRandom 随机取一个可用代理(优先选择质量高的)
|
||||
func (s *Storage) GetRandom() (*Proxy, error) {
|
||||
// 优先从 S/A 级代理中随机选择
|
||||
rows, err := s.db.Query(
|
||||
`SELECT id, address, protocol, fail_count, last_check, created_at
|
||||
FROM proxies WHERE fail_count < 3
|
||||
ORDER BY RANDOM() LIMIT 1`,
|
||||
`SELECT id, address, protocol, exit_ip, exit_location, latency, quality_grade,
|
||||
use_count, success_count, fail_count, last_used, last_check, created_at, status
|
||||
FROM proxies
|
||||
WHERE status = 'active' AND fail_count < 3
|
||||
ORDER BY
|
||||
CASE quality_grade
|
||||
WHEN 'S' THEN 1
|
||||
WHEN 'A' THEN 2
|
||||
WHEN 'B' THEN 3
|
||||
ELSE 4
|
||||
END,
|
||||
RANDOM()
|
||||
LIMIT 1`,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -95,24 +229,37 @@ func (s *Storage) GetRandom() (*Proxy, error) {
|
||||
defer rows.Close()
|
||||
|
||||
if rows.Next() {
|
||||
p := &Proxy{}
|
||||
var lastCheck sql.NullTime
|
||||
if err := rows.Scan(&p.ID, &p.Address, &p.Protocol, &p.FailCount, &lastCheck, &p.CreatedAt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if lastCheck.Valid {
|
||||
p.LastCheck = lastCheck.Time
|
||||
}
|
||||
return p, nil
|
||||
return scanProxy(rows)
|
||||
}
|
||||
return nil, fmt.Errorf("no available proxy")
|
||||
}
|
||||
|
||||
// scanProxy 扫描代理行数据
|
||||
func scanProxy(rows *sql.Rows) (*Proxy, error) {
|
||||
p := &Proxy{}
|
||||
var lastUsed, lastCheck sql.NullTime
|
||||
if err := rows.Scan(&p.ID, &p.Address, &p.Protocol, &p.ExitIP, &p.ExitLocation,
|
||||
&p.Latency, &p.QualityGrade, &p.UseCount, &p.SuccessCount, &p.FailCount,
|
||||
&lastUsed, &lastCheck, &p.CreatedAt, &p.Status); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if lastUsed.Valid {
|
||||
p.LastUsed = lastUsed.Time
|
||||
}
|
||||
if lastCheck.Valid {
|
||||
p.LastCheck = lastCheck.Time
|
||||
}
|
||||
return p, nil
|
||||
}
|
||||
|
||||
// GetAll 获取所有可用代理
|
||||
func (s *Storage) GetAll() ([]Proxy, error) {
|
||||
rows, err := s.db.Query(
|
||||
`SELECT id, address, protocol, fail_count, last_check, created_at
|
||||
FROM proxies WHERE fail_count < 3`,
|
||||
`SELECT id, address, protocol, exit_ip, exit_location, latency, quality_grade,
|
||||
use_count, success_count, fail_count, last_used, last_check, created_at, status
|
||||
FROM proxies
|
||||
WHERE status IN ('active', 'degraded') AND fail_count < 3
|
||||
ORDER BY latency ASC`,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -121,15 +268,11 @@ func (s *Storage) GetAll() ([]Proxy, error) {
|
||||
|
||||
var proxies []Proxy
|
||||
for rows.Next() {
|
||||
p := Proxy{}
|
||||
var lastCheck sql.NullTime
|
||||
if err := rows.Scan(&p.ID, &p.Address, &p.Protocol, &p.FailCount, &lastCheck, &p.CreatedAt); err != nil {
|
||||
p, err := scanProxy(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if lastCheck.Valid {
|
||||
p.LastCheck = lastCheck.Time
|
||||
}
|
||||
proxies = append(proxies, p)
|
||||
proxies = append(proxies, *p)
|
||||
}
|
||||
return proxies, nil
|
||||
}
|
||||
@@ -162,6 +305,29 @@ func (s *Storage) GetRandomExclude(excludes []string) (*Proxy, error) {
|
||||
return &p, nil
|
||||
}
|
||||
|
||||
// GetLowestLatencyExclude 排除指定地址后获取延迟最低的代理
|
||||
func (s *Storage) GetLowestLatencyExclude(excludes []string) (*Proxy, error) {
|
||||
proxies, err := s.GetAll()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
excludeMap := make(map[string]bool)
|
||||
for _, e := range excludes {
|
||||
excludeMap[e] = true
|
||||
}
|
||||
|
||||
// GetAll() 已经按 latency ASC 排序,找到第一个不在排除列表中的
|
||||
for _, p := range proxies {
|
||||
if !excludeMap[p.Address] {
|
||||
proxy := p
|
||||
return &proxy, nil
|
||||
}
|
||||
}
|
||||
|
||||
return nil, fmt.Errorf("no available proxy")
|
||||
}
|
||||
|
||||
// Delete 立即删除指定代理
|
||||
func (s *Storage) Delete(address string) error {
|
||||
_, err := s.db.Exec(`DELETE FROM proxies WHERE address = ?`, address)
|
||||
@@ -186,6 +352,201 @@ func (s *Storage) ResetFail(address string) error {
|
||||
return err
|
||||
}
|
||||
|
||||
// UpdateLatency 更新代理的延迟信息(毫秒)
|
||||
func (s *Storage) UpdateLatency(address string, latencyMs int) error {
|
||||
_, err := s.db.Exec(
|
||||
`UPDATE proxies SET latency = ? WHERE address = ?`,
|
||||
latencyMs, address,
|
||||
)
|
||||
return err
|
||||
}
|
||||
|
||||
// UpdateExitInfo 更新代理的出口 IP、位置和质量等级
|
||||
func (s *Storage) UpdateExitInfo(address, exitIP, exitLocation string, latencyMs int) error {
|
||||
grade := CalculateQualityGrade(latencyMs)
|
||||
_, err := s.db.Exec(
|
||||
`UPDATE proxies SET exit_ip = ?, exit_location = ?, latency = ?, quality_grade = ? WHERE address = ?`,
|
||||
exitIP, exitLocation, latencyMs, grade, address,
|
||||
)
|
||||
return err
|
||||
}
|
||||
|
||||
// RecordProxyUse 记录代理使用(成功)
|
||||
func (s *Storage) RecordProxyUse(address string, success bool) error {
|
||||
if success {
|
||||
_, err := s.db.Exec(
|
||||
`UPDATE proxies SET use_count = use_count + 1, success_count = success_count + 1,
|
||||
last_used = CURRENT_TIMESTAMP WHERE address = ?`,
|
||||
address,
|
||||
)
|
||||
return err
|
||||
}
|
||||
_, err := s.db.Exec(
|
||||
`UPDATE proxies SET use_count = use_count + 1, fail_count = fail_count + 1,
|
||||
last_used = CURRENT_TIMESTAMP WHERE address = ?`,
|
||||
address,
|
||||
)
|
||||
return err
|
||||
}
|
||||
|
||||
// GetWorstProxies 获取指定协议中延迟最高的N个代理
|
||||
func (s *Storage) GetWorstProxies(protocol string, limit int) ([]Proxy, error) {
|
||||
rows, err := s.db.Query(
|
||||
`SELECT id, address, protocol, exit_ip, exit_location, latency, quality_grade,
|
||||
use_count, success_count, fail_count, last_used, last_check, created_at, status
|
||||
FROM proxies
|
||||
WHERE protocol = ? AND status = 'active'
|
||||
AND quality_grade != 'S'
|
||||
AND (JULIANDAY('now') - JULIANDAY(created_at)) * 1440 > 60
|
||||
ORDER BY latency DESC, fail_count DESC
|
||||
LIMIT ?`, protocol, limit,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var proxies []Proxy
|
||||
for rows.Next() {
|
||||
p, err := scanProxy(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
proxies = append(proxies, *p)
|
||||
}
|
||||
return proxies, nil
|
||||
}
|
||||
|
||||
// ReplaceProxy 替换代理(删除旧的,添加新的)
|
||||
func (s *Storage) ReplaceProxy(oldAddress string, newProxy Proxy) error {
|
||||
tx, err := s.db.Begin()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer tx.Rollback()
|
||||
|
||||
// 删除旧代理
|
||||
_, err = tx.Exec(`DELETE FROM proxies WHERE address = ?`, oldAddress)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// 添加新代理(带完整信息)
|
||||
grade := CalculateQualityGrade(newProxy.Latency)
|
||||
_, err = tx.Exec(
|
||||
`INSERT INTO proxies (address, protocol, exit_ip, exit_location, latency, quality_grade, status)
|
||||
VALUES (?, ?, ?, ?, ?, ?, 'active')`,
|
||||
newProxy.Address, newProxy.Protocol, newProxy.ExitIP, newProxy.ExitLocation, newProxy.Latency, grade,
|
||||
)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return tx.Commit()
|
||||
}
|
||||
|
||||
// MarkAsReplacementCandidate 标记代理为替换候选
|
||||
func (s *Storage) MarkAsReplacementCandidate(addresses []string) error {
|
||||
if len(addresses) == 0 {
|
||||
return nil
|
||||
}
|
||||
placeholders := make([]string, len(addresses))
|
||||
args := make([]interface{}, len(addresses))
|
||||
for i, addr := range addresses {
|
||||
placeholders[i] = "?"
|
||||
args[i] = addr
|
||||
}
|
||||
query := fmt.Sprintf(`UPDATE proxies SET status = 'candidate_replace' WHERE address IN (%s)`,
|
||||
fmt.Sprintf("%s", placeholders))
|
||||
_, err := s.db.Exec(query, args...)
|
||||
return err
|
||||
}
|
||||
|
||||
// GetAverageLatency 获取指定协议的平均延迟
|
||||
func (s *Storage) GetAverageLatency(protocol string) (int, error) {
|
||||
var avg sql.NullFloat64
|
||||
err := s.db.QueryRow(
|
||||
`SELECT AVG(latency) FROM proxies WHERE protocol = ? AND status = 'active' AND latency > 0`,
|
||||
protocol,
|
||||
).Scan(&avg)
|
||||
if err != nil || !avg.Valid {
|
||||
return 0, err
|
||||
}
|
||||
return int(avg.Float64), nil
|
||||
}
|
||||
|
||||
// GetQualityDistribution 获取质量分布统计
|
||||
func (s *Storage) GetQualityDistribution() (map[string]int, error) {
|
||||
rows, err := s.db.Query(
|
||||
`SELECT quality_grade, COUNT(*) as count
|
||||
FROM proxies
|
||||
WHERE status = 'active' AND fail_count < 3
|
||||
GROUP BY quality_grade`,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
dist := make(map[string]int)
|
||||
for rows.Next() {
|
||||
var grade string
|
||||
var count int
|
||||
if err := rows.Scan(&grade, &count); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
dist[grade] = count
|
||||
}
|
||||
return dist, nil
|
||||
}
|
||||
|
||||
// GetBatchForHealthCheck 获取一批需要健康检查的代理
|
||||
func (s *Storage) GetBatchForHealthCheck(batchSize int, skipSGrade bool) ([]Proxy, error) {
|
||||
query := `SELECT id, address, protocol, exit_ip, exit_location, latency, quality_grade,
|
||||
use_count, success_count, fail_count, last_used, last_check, created_at, status
|
||||
FROM proxies
|
||||
WHERE status IN ('active', 'degraded') AND fail_count < 3`
|
||||
|
||||
if skipSGrade {
|
||||
query += ` AND quality_grade != 'S'`
|
||||
}
|
||||
|
||||
query += ` ORDER BY
|
||||
COALESCE(last_check, '1970-01-01') ASC,
|
||||
quality_grade DESC
|
||||
LIMIT ?`
|
||||
|
||||
rows, err := s.db.Query(query, batchSize)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var proxies []Proxy
|
||||
for rows.Next() {
|
||||
p, err := scanProxy(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
proxies = append(proxies, *p)
|
||||
}
|
||||
return proxies, nil
|
||||
}
|
||||
|
||||
// CalculateQualityGrade 根据延迟计算质量等级
|
||||
func CalculateQualityGrade(latencyMs int) string {
|
||||
switch {
|
||||
case latencyMs <= 500:
|
||||
return "S" // 超快
|
||||
case latencyMs <= 1000:
|
||||
return "A" // 良好
|
||||
case latencyMs <= 2000:
|
||||
return "B" // 可用
|
||||
default:
|
||||
return "C" // 淘汰候选
|
||||
}
|
||||
}
|
||||
|
||||
// DeleteInvalid 删除失败次数超过阈值的代理
|
||||
func (s *Storage) DeleteInvalid(maxFailCount int) (int64, error) {
|
||||
res, err := s.db.Exec(`DELETE FROM proxies WHERE fail_count >= ?`, maxFailCount)
|
||||
@@ -195,26 +556,62 @@ func (s *Storage) DeleteInvalid(maxFailCount int) (int64, error) {
|
||||
return res.RowsAffected()
|
||||
}
|
||||
|
||||
// DeleteChinaMainland 删除中国大陆出口的代理(保留香港)
|
||||
func (s *Storage) DeleteChinaMainland() (int64, error) {
|
||||
// 删除 exit_location 以 "CN " 开头的代理(CN后面有空格表示是城市)
|
||||
// 香港是 "HK Hong Kong" 或 "HK" 开头,不会被删除
|
||||
res, err := s.db.Exec(`DELETE FROM proxies WHERE exit_location LIKE 'CN %'`)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return res.RowsAffected()
|
||||
}
|
||||
|
||||
// DeleteWithoutExitInfo 删除没有出口信息的代理
|
||||
func (s *Storage) DeleteWithoutExitInfo() (int64, error) {
|
||||
res, err := s.db.Exec(`DELETE FROM proxies WHERE exit_ip = '' OR exit_location = ''`)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return res.RowsAffected()
|
||||
}
|
||||
|
||||
// Count 返回可用代理数量
|
||||
func (s *Storage) Count() (int, error) {
|
||||
var count int
|
||||
err := s.db.QueryRow(`SELECT COUNT(*) FROM proxies WHERE fail_count < 3`).Scan(&count)
|
||||
err := s.db.QueryRow(
|
||||
`SELECT COUNT(*) FROM proxies WHERE status IN ('active', 'degraded') AND fail_count < 3`,
|
||||
).Scan(&count)
|
||||
return count, err
|
||||
}
|
||||
|
||||
// CountByProtocol 按协议统计数量
|
||||
func (s *Storage) CountByProtocol(protocol string) (int, error) {
|
||||
var count int
|
||||
err := s.db.QueryRow(`SELECT COUNT(*) FROM proxies WHERE fail_count < 3 AND protocol = ?`, protocol).Scan(&count)
|
||||
err := s.db.QueryRow(
|
||||
`SELECT COUNT(*) FROM proxies WHERE status IN ('active', 'degraded') AND fail_count < 3 AND protocol = ?`,
|
||||
protocol,
|
||||
).Scan(&count)
|
||||
return count, err
|
||||
}
|
||||
|
||||
// IncrementFailCount 增加失败次数
|
||||
func (s *Storage) IncrementFailCount(address string) error {
|
||||
_, err := s.db.Exec(
|
||||
`UPDATE proxies SET fail_count = fail_count + 1 WHERE address = ?`,
|
||||
address,
|
||||
)
|
||||
return err
|
||||
}
|
||||
|
||||
// GetByProtocol 按协议获取代理列表
|
||||
func (s *Storage) GetByProtocol(protocol string) ([]Proxy, error) {
|
||||
rows, err := s.db.Query(
|
||||
`SELECT id, address, protocol, fail_count, last_check, created_at
|
||||
FROM proxies WHERE fail_count < 3 AND protocol = ?
|
||||
ORDER BY created_at DESC`, protocol,
|
||||
`SELECT id, address, protocol, exit_ip, exit_location, latency, quality_grade,
|
||||
use_count, success_count, fail_count, last_used, last_check, created_at, status
|
||||
FROM proxies
|
||||
WHERE status IN ('active', 'degraded') AND fail_count < 3 AND protocol = ?
|
||||
ORDER BY latency ASC`, protocol,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -223,15 +620,11 @@ func (s *Storage) GetByProtocol(protocol string) ([]Proxy, error) {
|
||||
|
||||
var proxies []Proxy
|
||||
for rows.Next() {
|
||||
p := Proxy{}
|
||||
var lastCheck sql.NullTime
|
||||
if err := rows.Scan(&p.ID, &p.Address, &p.Protocol, &p.FailCount, &lastCheck, &p.CreatedAt); err != nil {
|
||||
p, err := scanProxy(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if lastCheck.Valid {
|
||||
p.LastCheck = lastCheck.Time
|
||||
}
|
||||
proxies = append(proxies, p)
|
||||
proxies = append(proxies, *p)
|
||||
}
|
||||
return proxies, nil
|
||||
}
|
||||
@@ -240,3 +633,8 @@ func (s *Storage) GetByProtocol(protocol string) ([]Proxy, error) {
|
||||
func (s *Storage) Close() error {
|
||||
return s.db.Close()
|
||||
}
|
||||
|
||||
// GetDB 获取数据库实例(供其他模块使用)
|
||||
func (s *Storage) GetDB() *sql.DB {
|
||||
return s.db
|
||||
}
|
||||
|
||||
121
test/README.md
Normal file
121
test/README.md
Normal file
@@ -0,0 +1,121 @@
|
||||
# GoProxy 测试脚本
|
||||
|
||||
本目录包含用于测试 GoProxy 代理服务的脚本。所有脚本都采用**持续运行模式**(类似 `ping` 命令),按 `Ctrl+C` 停止并显示统计。
|
||||
|
||||
## 📝 脚本列表
|
||||
|
||||
| 脚本 | 语言 | 依赖 | 运行模式 | 推荐度 |
|
||||
|------|------|------|----------|--------|
|
||||
| `test_proxy.sh` | Bash | curl + Python3 | 持续运行 | ⭐⭐⭐ |
|
||||
| `test_proxy.go` | Go | `golang.org/x/net/proxy` | 持续运行 | ⭐⭐ |
|
||||
| `test_proxy.py` | Python | `requests`, `pysocks` | 持续运行 | ⭐⭐ |
|
||||
|
||||
## 🚀 快速使用
|
||||
|
||||
### Bash 脚本(推荐)
|
||||
|
||||
```bash
|
||||
# 从项目根目录运行(持续测试 HTTP)
|
||||
./test/test_proxy.sh
|
||||
|
||||
# 测试 HTTP 协议
|
||||
./test/test_proxy.sh http
|
||||
|
||||
# 测试 SOCKS5 协议
|
||||
./test/test_proxy.sh socks5
|
||||
|
||||
# 按 Ctrl+C 停止并查看统计
|
||||
```
|
||||
|
||||
### Go 脚本
|
||||
|
||||
```bash
|
||||
# 安装依赖
|
||||
go get golang.org/x/net/proxy
|
||||
|
||||
# 运行测试
|
||||
go run test/test_proxy.go
|
||||
|
||||
# 或编译后运行
|
||||
cd test
|
||||
go build -o test_proxy test_proxy.go
|
||||
./test_proxy
|
||||
```
|
||||
|
||||
### Python 脚本
|
||||
|
||||
```bash
|
||||
# 安装依赖
|
||||
pip install requests pysocks
|
||||
|
||||
# 运行测试
|
||||
python test/test_proxy.py
|
||||
```
|
||||
|
||||
## 📊 测试内容
|
||||
|
||||
所有脚本都会:
|
||||
1. 通过指定端口代理发送请求(默认 `127.0.0.1:7777`)
|
||||
2. 访问 `http://ip-api.com/json` 获取出口 IP 和国家信息
|
||||
3. **持续发送请求**,间隔 1 秒(类似 `ping` 命令)
|
||||
4. 实时显示国旗 emoji、出口 IP 和延迟
|
||||
5. 按 `Ctrl+C` 停止并显示统计摘要
|
||||
|
||||
## 📖 详细文档
|
||||
|
||||
完整的测试指南、故障排查、高级用法,请查看:
|
||||
|
||||
👉 [TEST_GUIDE.md](./TEST_GUIDE.md)
|
||||
|
||||
## 🔀 测试不同端口策略
|
||||
|
||||
```bash
|
||||
# 对比两个端口的行为差异:
|
||||
|
||||
# 随机轮换模式 - IP 高度分散
|
||||
./test/test_proxy.sh 7777
|
||||
|
||||
# 最低延迟模式 - 固定使用最快代理
|
||||
./test/test_proxy.sh 7776
|
||||
```
|
||||
|
||||
**观察要点**:
|
||||
- **7777 端口**:每次请求的出口 IP 应该不同(证明在轮换)
|
||||
- **7776 端口**:连续多次请求的出口 IP 基本相同(证明固定使用最优代理)
|
||||
|
||||
## 🔍 预期输出
|
||||
|
||||
```
|
||||
PROXY 127.0.0.1:7777 (http://ip-api.com/json/?fields=countryCode,query): continuous mode
|
||||
|
||||
proxy from 🇺🇸 203.0.113.45: seq=1 time=1234ms
|
||||
proxy from 🇩🇪 198.51.100.78: seq=2 time=987ms
|
||||
proxy from 🇬🇧 192.0.2.123: seq=3 time=1567ms
|
||||
proxy #4: request failed (timeout)
|
||||
proxy from 🇯🇵 198.51.100.12: seq=5 time=890ms
|
||||
proxy from 🇫🇷 192.0.2.234: seq=6 time=1456ms
|
||||
...
|
||||
(持续运行,按 Ctrl+C 停止)
|
||||
|
||||
^C
|
||||
---
|
||||
50 requests transmitted, 47 received, 3 failed, 6.0% packet loss
|
||||
```
|
||||
|
||||
**输出风格**:
|
||||
- 简洁清晰,类似 `ping` 命令
|
||||
- 一行一个结果
|
||||
- 显示国旗 emoji、出口 IP、序号、延迟
|
||||
- 统计信息简洁明了
|
||||
|
||||
**观察要点**:
|
||||
- 每次请求的出口 IP 应该不同(证明代理轮换)
|
||||
- 延迟应该在合理范围(< 2000ms)
|
||||
- 丢包率应该 < 10%
|
||||
- 可以长时间运行观察稳定性
|
||||
|
||||
## 📝 注意事项
|
||||
|
||||
1. 确保 GoProxy 服务已启动:`./goproxy`
|
||||
2. 首次启动需等待代理池就绪(约 30-60 秒)
|
||||
3. 可配合 WebUI (http://localhost:7778) 查看实时状态
|
||||
245
test/TEST_GUIDE.md
Normal file
245
test/TEST_GUIDE.md
Normal file
@@ -0,0 +1,245 @@
|
||||
# GoProxy 测试指南
|
||||
|
||||
本项目提供了三种测试脚本,用于验证代理服务的功能和性能。所有脚本都采用**持续运行模式**(类似 `ping` 命令),按 `Ctrl+C` 停止并显示统计信息。
|
||||
|
||||
## 📋 测试脚本说明
|
||||
|
||||
所有测试脚本位于 `test/` 目录下。
|
||||
|
||||
### 1. Bash 脚本(推荐)- `test_proxy.sh`
|
||||
|
||||
最简单直接,使用 `curl` 命令持续测试代理服务。
|
||||
|
||||
**使用方法:**
|
||||
|
||||
```bash
|
||||
# 测试随机轮换模式(默认 7777 端口)
|
||||
./test/test_proxy.sh
|
||||
|
||||
# 测试最低延迟模式(7776 端口)
|
||||
./test/test_proxy.sh 7776
|
||||
|
||||
# 自定义端口
|
||||
./test/test_proxy.sh 8080
|
||||
```
|
||||
|
||||
**特点:**
|
||||
- 无需安装依赖(仅需 curl 和 Python3)
|
||||
- 兼容 macOS 和 Linux
|
||||
- 持续运行,按 Ctrl+C 停止
|
||||
- 显示国旗 emoji 和出口 IP
|
||||
|
||||
---
|
||||
|
||||
### 2. Go 脚本 - `test_proxy.go`
|
||||
|
||||
使用 Go 语言编写,与项目本身技术栈一致。
|
||||
|
||||
**使用方法:**
|
||||
|
||||
```bash
|
||||
# 测试随机轮换模式(默认 7777 端口)
|
||||
go run test/test_proxy.go
|
||||
|
||||
# 测试最低延迟模式(7776 端口)
|
||||
go run test/test_proxy.go 7776
|
||||
|
||||
# 自定义端口
|
||||
go run test/test_proxy.go 8080
|
||||
|
||||
# 或编译后运行
|
||||
cd test && go build -o test_proxy test_proxy.go
|
||||
./test_proxy 7776
|
||||
```
|
||||
|
||||
**特点:**
|
||||
- 与项目技术栈统一
|
||||
- 可编译为独立二进制
|
||||
- 显示国旗 emoji 和出口 IP
|
||||
- 持续运行,按 Ctrl+C 停止
|
||||
- 彩色输出
|
||||
|
||||
---
|
||||
|
||||
### 3. Python 脚本 - `test_proxy.py`
|
||||
|
||||
使用 Python 编写,语法简洁易读。
|
||||
|
||||
**使用方法:**
|
||||
|
||||
```bash
|
||||
# 安装依赖
|
||||
pip install requests
|
||||
|
||||
# 测试随机轮换模式(默认 7777 端口)
|
||||
python test/test_proxy.py
|
||||
|
||||
# 测试最低延迟模式(7776 端口)
|
||||
python test/test_proxy.py 7776
|
||||
|
||||
# 自定义端口
|
||||
python test/test_proxy.py 8080
|
||||
```
|
||||
|
||||
**特点:**
|
||||
- Python 生态丰富
|
||||
- 易于扩展和修改
|
||||
- 持续运行,按 Ctrl+C 停止
|
||||
- 显示国旗 emoji 和出口 IP
|
||||
|
||||
---
|
||||
|
||||
## 🎯 测试输出示例
|
||||
|
||||
```
|
||||
PROXY 127.0.0.1:7777 (http://ip-api.com/json/?fields=countryCode,query): continuous mode
|
||||
|
||||
proxy from 🇺🇸 203.0.113.45: seq=1 time=1234ms
|
||||
proxy from 🇩🇪 198.51.100.78: seq=2 time=987ms
|
||||
proxy from 🇬🇧 192.0.2.123: seq=3 time=1567ms
|
||||
proxy from 🇺🇸 203.0.113.45: seq=4 time=1123ms
|
||||
proxy #5: request failed (timeout)
|
||||
proxy from 🇯🇵 198.51.100.12: seq=6 time=890ms
|
||||
proxy from 🇫🇷 192.0.2.234: seq=7 time=1456ms
|
||||
proxy from 🇨🇳 68.71.249.153: seq=8 time=1102ms
|
||||
...
|
||||
^C
|
||||
---
|
||||
50 requests transmitted, 47 received, 3 failed, 6.0% packet loss
|
||||
```
|
||||
|
||||
**特点:**
|
||||
- ✅ 简洁输出,类似 `ping` 命令
|
||||
- ✅ 一行一个结果,清晰易读
|
||||
- ✅ 显示国旗 emoji、出口 IP 和延迟
|
||||
- ✅ 按 Ctrl+C 停止并显示统计摘要
|
||||
- ✅ 显示丢包率(失败率)
|
||||
|
||||
## 🔍 观察要点
|
||||
|
||||
### 1. 出口 IP 变化
|
||||
- 每次请求的出口 IP **应该不同**,证明代理池在轮换
|
||||
- 如果连续多次是同一个 IP,说明池子中可用代理较少
|
||||
- 持续运行可以观察到代理的循环使用模式
|
||||
|
||||
### 2. 延迟表现
|
||||
- 延迟应该在合理范围内(通常 < 2000ms)
|
||||
- 如果延迟过高或超时,说明代理质量下降
|
||||
- 观察延迟趋势:是否稳定在某个范围
|
||||
|
||||
### 3. 成功率(动态显示)
|
||||
- 正常情况下成功率应该 > 90%
|
||||
- 持续运行时成功率会动态更新
|
||||
- 如果成功率持续下降,需要检查:
|
||||
- 代理池是否有足够的健康代理
|
||||
- 网络连接是否正常
|
||||
- 代理源质量是否下降
|
||||
|
||||
### 4. 协议对比
|
||||
- HTTP 代理:速度通常较快,兼容性好
|
||||
- SOCKS5 代理:更底层,可传输各种协议数据
|
||||
|
||||
### 5. 持续测试优势
|
||||
- 可以长时间运行,观察代理池的稳定性
|
||||
- 实时监控成功率变化
|
||||
- 按 Ctrl+C 随时查看统计结果
|
||||
- 类似 `ping` 命令的使用体验
|
||||
|
||||
## 📊 配合 WebUI 监控
|
||||
|
||||
测试时可以同时打开 WebUI (http://localhost:7778):
|
||||
|
||||
1. **代理列表**:查看正在使用的代理
|
||||
2. **使用统计**:观察 `使用次数` 列的变化
|
||||
3. **系统日志**:实时查看代理请求日志
|
||||
4. **质量分布**:查看当前池子的质量分布
|
||||
|
||||
## ⚙️ 自定义配置
|
||||
|
||||
修改脚本顶部的配置变量:
|
||||
|
||||
```bash
|
||||
PROXY_HOST="127.0.0.1" # 代理服务地址
|
||||
PROXY_PORT="7777" # 代理服务端口
|
||||
TEST_URL="http://httpbin.org/ip" # 测试目标 URL
|
||||
DELAY=1 # 每次请求间隔(秒)
|
||||
```
|
||||
|
||||
**注意**:所有脚本现在都是持续运行模式,按 `Ctrl+C` 停止,无需配置请求次数。
|
||||
|
||||
## 🛠️ 高级测试
|
||||
|
||||
### 测试特定网站
|
||||
|
||||
```bash
|
||||
# 测试访问 Google
|
||||
curl -x "http://127.0.0.1:7777" https://www.google.com
|
||||
|
||||
# 测试访问 GitHub API
|
||||
curl -x "http://127.0.0.1:7777" https://api.github.com/users/github
|
||||
|
||||
# 测试中国网站
|
||||
curl -x "http://127.0.0.1:7777" https://www.baidu.com
|
||||
```
|
||||
|
||||
### 并发压力测试
|
||||
|
||||
```bash
|
||||
# 使用 ab (Apache Bench) 进行压力测试
|
||||
ab -n 100 -c 10 -X 127.0.0.1:7777 http://httpbin.org/ip
|
||||
|
||||
# 使用 wrk 进行压力测试
|
||||
wrk -t4 -c100 -d30s --proxy http://127.0.0.1:7777 http://httpbin.org/ip
|
||||
```
|
||||
|
||||
### 查看当前使用的代理
|
||||
|
||||
```bash
|
||||
# 通过 httpbin 获取当前出口 IP
|
||||
curl -x "http://127.0.0.1:7777" http://httpbin.org/ip
|
||||
|
||||
# 获取更详细的信息(包括 headers)
|
||||
curl -x "http://127.0.0.1:7777" http://httpbin.org/anything
|
||||
```
|
||||
|
||||
## 📝 注意事项
|
||||
|
||||
1. **启动代理服务**:测试前确保 `goproxy` 已启动
|
||||
```bash
|
||||
./goproxy
|
||||
```
|
||||
|
||||
2. **等待代理池就绪**:首次启动需要等待代理抓取和验证(约 30-60 秒)
|
||||
|
||||
3. **网络环境**:确保服务器可以访问外部代理源和测试 URL
|
||||
|
||||
4. **防火墙**:确保 7777 端口没有被防火墙阻止
|
||||
|
||||
5. **代理协议**:不同协议可能表现不同,建议都测试一遍
|
||||
|
||||
## 🐛 故障排查
|
||||
|
||||
### 全部请求失败
|
||||
- 检查 `goproxy` 服务是否正在运行
|
||||
- 检查代理池是否有可用代理(访问 WebUI 查看)
|
||||
- 检查端口 7777 是否被占用
|
||||
|
||||
### 成功率低
|
||||
- 查看 WebUI 日志,确认代理质量
|
||||
- 可能需要等待池子优化(系统会自动轮换低质量代理)
|
||||
- 检查网络连接是否稳定
|
||||
|
||||
### SOCKS5 测试失败
|
||||
- 确认 curl 版本支持 SOCKS5(`curl --version`)
|
||||
- Python 脚本需要安装 `pysocks`:`pip install pysocks`
|
||||
- Go 脚本需要安装依赖:`go get golang.org/x/net/proxy`
|
||||
|
||||
### Bash 脚本时间戳错误(macOS)
|
||||
如果看到 `value too great for base` 错误:
|
||||
- 脚本已自动使用 Python3 获取毫秒时间戳(macOS 兼容)
|
||||
- 确保系统已安装 Python3(macOS 自带)
|
||||
- 或安装 GNU coreutils:`brew install coreutils`
|
||||
|
||||
---
|
||||
|
||||
**Happy Testing! 🚀**
|
||||
122
test/test_proxy.go
Normal file
122
test/test_proxy.go
Normal file
@@ -0,0 +1,122 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"os/signal"
|
||||
"strings"
|
||||
"syscall"
|
||||
"time"
|
||||
)
|
||||
|
||||
const (
|
||||
proxyHost = "127.0.0.1"
|
||||
testURL = "http://ip-api.com/json/?fields=countryCode,query"
|
||||
delaySeconds = 1
|
||||
)
|
||||
|
||||
var proxyPort = "7777"
|
||||
|
||||
type IPResponse struct {
|
||||
Query string `json:"query"`
|
||||
CountryCode string `json:"countryCode"`
|
||||
}
|
||||
|
||||
var (
|
||||
totalCount = 0
|
||||
successCount = 0
|
||||
)
|
||||
|
||||
// 国家代码转 emoji 旗帜
|
||||
func countryToEmoji(countryCode string) string {
|
||||
if countryCode == "" {
|
||||
return "🌐"
|
||||
}
|
||||
|
||||
countryCode = strings.ToUpper(countryCode)
|
||||
if len(countryCode) != 2 {
|
||||
return "🌐"
|
||||
}
|
||||
|
||||
// 将国家代码转换为 emoji
|
||||
// A=127462, 所以 'US' -> 🇺🇸
|
||||
first := rune(countryCode[0]) - 'A' + 127462
|
||||
second := rune(countryCode[1]) - 'A' + 127462
|
||||
|
||||
return string([]rune{first, second})
|
||||
}
|
||||
|
||||
func printStats() {
|
||||
fmt.Println()
|
||||
fmt.Println("---")
|
||||
lossCount := totalCount - successCount
|
||||
lossRate := 0.0
|
||||
if totalCount > 0 {
|
||||
lossRate = float64(lossCount) / float64(totalCount) * 100
|
||||
}
|
||||
fmt.Printf("%d requests transmitted, %d received, %d failed, %.1f%% packet loss\n",
|
||||
totalCount, successCount, lossCount, lossRate)
|
||||
os.Exit(0)
|
||||
}
|
||||
|
||||
func testHTTPProxyContinuous() {
|
||||
fmt.Printf("PROXY %s:%s (%s): continuous mode\n", proxyHost, proxyPort, testURL)
|
||||
fmt.Println()
|
||||
|
||||
proxyURL, _ := url.Parse(fmt.Sprintf("http://%s:%s", proxyHost, proxyPort))
|
||||
client := &http.Client{
|
||||
Transport: &http.Transport{
|
||||
Proxy: http.ProxyURL(proxyURL),
|
||||
},
|
||||
Timeout: 15 * time.Second,
|
||||
}
|
||||
|
||||
// 捕获 Ctrl+C 信号
|
||||
sigChan := make(chan os.Signal, 1)
|
||||
signal.Notify(sigChan, os.Interrupt, syscall.SIGTERM)
|
||||
go func() {
|
||||
<-sigChan
|
||||
printStats()
|
||||
}()
|
||||
|
||||
for {
|
||||
totalCount++
|
||||
|
||||
start := time.Now()
|
||||
resp, err := client.Get(testURL)
|
||||
elapsed := time.Since(start).Milliseconds()
|
||||
|
||||
if err != nil {
|
||||
fmt.Printf("proxy #%d: request failed (%v)\n", totalCount, err)
|
||||
time.Sleep(delaySeconds * time.Second)
|
||||
continue
|
||||
}
|
||||
|
||||
body, _ := io.ReadAll(resp.Body)
|
||||
resp.Body.Close()
|
||||
|
||||
var ipResp IPResponse
|
||||
if err := json.Unmarshal(body, &ipResp); err == nil {
|
||||
flag := countryToEmoji(ipResp.CountryCode)
|
||||
fmt.Printf("proxy from %s %s: seq=%d time=%dms\n", flag, ipResp.Query, totalCount, elapsed)
|
||||
successCount++
|
||||
} else {
|
||||
fmt.Printf("proxy #%d: parse error\n", totalCount)
|
||||
}
|
||||
|
||||
time.Sleep(delaySeconds * time.Second)
|
||||
}
|
||||
}
|
||||
|
||||
func main() {
|
||||
// 支持指定端口号
|
||||
if len(os.Args) > 1 {
|
||||
proxyPort = os.Args[1]
|
||||
}
|
||||
|
||||
testHTTPProxyContinuous()
|
||||
}
|
||||
97
test/test_proxy.py
Normal file
97
test/test_proxy.py
Normal file
@@ -0,0 +1,97 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
GoProxy 持续测试脚本 - 类似 ping 命令的简洁输出
|
||||
按 Ctrl+C 停止测试
|
||||
"""
|
||||
|
||||
import requests
|
||||
import time
|
||||
import sys
|
||||
import signal
|
||||
from requests.exceptions import RequestException
|
||||
|
||||
# 配置
|
||||
PROXY_HOST = "127.0.0.1"
|
||||
PROXY_PORT = int(sys.argv[1]) if len(sys.argv) > 1 and sys.argv[1].isdigit() else 7777
|
||||
TEST_URL = "http://ip-api.com/json/?fields=countryCode,query"
|
||||
DELAY_SECONDS = 1
|
||||
|
||||
# 统计变量
|
||||
total_count = 0
|
||||
success_count = 0
|
||||
|
||||
|
||||
def country_to_emoji(country_code):
|
||||
"""将国家代码转换为 emoji 旗帜"""
|
||||
if not country_code or country_code == "null":
|
||||
return "🌐"
|
||||
|
||||
# 将国家代码转换为区域指示符号
|
||||
# A=127462, 所以 'US' -> 🇺🇸
|
||||
try:
|
||||
first = ord(country_code[0].upper()) - ord('A') + 127462
|
||||
second = ord(country_code[1].upper()) - ord('A') + 127462
|
||||
return chr(first) + chr(second)
|
||||
except:
|
||||
return "🌐"
|
||||
|
||||
|
||||
def signal_handler(sig, frame):
|
||||
"""处理 Ctrl+C 信号"""
|
||||
print()
|
||||
print("---")
|
||||
loss_count = total_count - success_count
|
||||
loss_rate = 0.0
|
||||
if total_count > 0:
|
||||
loss_rate = loss_count / total_count * 100
|
||||
print(f"{total_count} requests transmitted, {success_count} received, {loss_count} failed, {loss_rate:.1f}% packet loss")
|
||||
sys.exit(0)
|
||||
|
||||
|
||||
def test_http_proxy_continuous():
|
||||
"""持续测试 HTTP 代理"""
|
||||
global total_count, success_count
|
||||
|
||||
proxy_url = f"http://{PROXY_HOST}:{PROXY_PORT}"
|
||||
proxies = {
|
||||
"http": proxy_url,
|
||||
"https": proxy_url,
|
||||
}
|
||||
|
||||
print(f"PROXY {PROXY_HOST}:{PROXY_PORT} ({TEST_URL}): continuous mode")
|
||||
print()
|
||||
|
||||
# 注册信号处理
|
||||
signal.signal(signal.SIGINT, signal_handler)
|
||||
|
||||
while True:
|
||||
total_count += 1
|
||||
|
||||
try:
|
||||
start_time = time.time()
|
||||
response = requests.get(
|
||||
TEST_URL,
|
||||
proxies=proxies,
|
||||
timeout=15,
|
||||
)
|
||||
elapsed = int((time.time() - start_time) * 1000)
|
||||
|
||||
if response.status_code == 200:
|
||||
data = response.json()
|
||||
exit_ip = data.get("query", "Unknown")
|
||||
country_code = data.get("countryCode", "")
|
||||
flag = country_to_emoji(country_code)
|
||||
print(f"proxy from {flag} {exit_ip}: seq={total_count} time={elapsed}ms")
|
||||
success_count += 1
|
||||
else:
|
||||
print(f"proxy #{total_count}: request failed (HTTP {response.status_code})")
|
||||
|
||||
except RequestException as e:
|
||||
error_msg = str(e).split(':')[0]
|
||||
print(f"proxy #{total_count}: {error_msg}")
|
||||
|
||||
time.sleep(DELAY_SECONDS)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
test_http_proxy_continuous()
|
||||
95
test/test_proxy.sh
Executable file
95
test/test_proxy.sh
Executable file
@@ -0,0 +1,95 @@
|
||||
#!/bin/bash
|
||||
|
||||
# GoProxy 持续测试脚本 - 类似 ping 命令的简洁输出
|
||||
# 按 Ctrl+C 停止测试
|
||||
# 用法: ./test_proxy.sh [端口号,默认7777]
|
||||
|
||||
PROXY_HOST="127.0.0.1"
|
||||
PROXY_PORT="${1:-7777}"
|
||||
TEST_URL="http://ip-api.com/json/?fields=countryCode,query"
|
||||
DELAY=1
|
||||
|
||||
# 统计变量
|
||||
total=0
|
||||
success=0
|
||||
fail=0
|
||||
|
||||
# 获取毫秒时间戳(兼容 macOS 和 Linux)
|
||||
get_ms_time() {
|
||||
python3 -c 'import time; print(int(time.time() * 1000))'
|
||||
}
|
||||
|
||||
# 国家代码转 emoji 旗帜
|
||||
country_to_emoji() {
|
||||
local country_code="$1"
|
||||
if [ -z "$country_code" ] || [ "$country_code" = "null" ]; then
|
||||
echo "🌐"
|
||||
return
|
||||
fi
|
||||
|
||||
# 将国家代码转换为 emoji(使用 Unicode 区域指示符)
|
||||
# 每个字母转换为对应的区域指示符号字符
|
||||
local first="${country_code:0:1}"
|
||||
local second="${country_code:1:1}"
|
||||
|
||||
# A=127462, 所以 A->🇦 就是 127462,B->🇧 就是 127463
|
||||
# 使用 printf 和 Unicode 编码
|
||||
python3 -c "print(chr(127462 + ord('$first') - ord('A')) + chr(127462 + ord('$second') - ord('A')))"
|
||||
}
|
||||
|
||||
# 捕获 Ctrl+C 信号
|
||||
trap ctrl_c INT
|
||||
function ctrl_c() {
|
||||
echo ""
|
||||
echo "---"
|
||||
loss_rate=$(awk "BEGIN {printf \"%.1f\", ($total - $success)/$total*100}")
|
||||
echo "$total requests transmitted, $success received, $((total - success)) failed, ${loss_rate}% packet loss"
|
||||
exit 0
|
||||
}
|
||||
|
||||
# 测试 HTTP 代理
|
||||
test_http_proxy() {
|
||||
echo "PROXY $PROXY_HOST:$PROXY_PORT ($TEST_URL): continuous mode"
|
||||
echo ""
|
||||
|
||||
while true; do
|
||||
total=$((total + 1))
|
||||
|
||||
# 使用 HTTP 代理发送请求
|
||||
start_time=$(get_ms_time)
|
||||
response=$(curl -x "http://${PROXY_HOST}:${PROXY_PORT}" \
|
||||
-s \
|
||||
-w "\n%{http_code}" \
|
||||
--connect-timeout 10 \
|
||||
--max-time 15 \
|
||||
"${TEST_URL}" 2>&1)
|
||||
end_time=$(get_ms_time)
|
||||
elapsed=$((end_time - start_time))
|
||||
|
||||
# 分离响应体和状态码
|
||||
http_code=$(echo "$response" | tail -n 1)
|
||||
body=$(echo "$response" | sed '$d')
|
||||
|
||||
if [ "$http_code" = "200" ]; then
|
||||
exit_ip=$(echo "$body" | grep -o '"query":"[^"]*"' | cut -d'"' -f4)
|
||||
country_code=$(echo "$body" | grep -o '"countryCode":"[^"]*"' | cut -d'"' -f4)
|
||||
|
||||
if [ -n "$exit_ip" ]; then
|
||||
flag=$(country_to_emoji "$country_code")
|
||||
echo "proxy from $flag $exit_ip: seq=$total time=${elapsed}ms"
|
||||
success=$((success + 1))
|
||||
else
|
||||
echo "proxy #$total: parse error"
|
||||
fail=$((fail + 1))
|
||||
fi
|
||||
else
|
||||
echo "proxy #$total: request failed (HTTP $http_code)"
|
||||
fail=$((fail + 1))
|
||||
fi
|
||||
|
||||
sleep $DELAY
|
||||
done
|
||||
}
|
||||
|
||||
# 主函数
|
||||
test_http_proxy
|
||||
@@ -1,6 +1,7 @@
|
||||
package validator
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"log"
|
||||
@@ -10,8 +11,8 @@ import (
|
||||
"time"
|
||||
|
||||
"golang.org/x/net/proxy"
|
||||
"proxy-pool/config"
|
||||
"proxy-pool/storage"
|
||||
"goproxy/config"
|
||||
"goproxy/storage"
|
||||
)
|
||||
|
||||
type Validator struct {
|
||||
@@ -43,9 +44,41 @@ func New(concurrency, timeoutSec int, validateURL string) *Validator {
|
||||
}
|
||||
|
||||
type Result struct {
|
||||
Proxy storage.Proxy
|
||||
Valid bool
|
||||
Latency time.Duration
|
||||
Proxy storage.Proxy
|
||||
Valid bool
|
||||
Latency time.Duration
|
||||
ExitIP string
|
||||
ExitLocation string
|
||||
}
|
||||
|
||||
// getExitIPInfo 通过代理获取出口 IP 和地理位置
|
||||
func getExitIPInfo(client *http.Client) (string, string) {
|
||||
// 使用 ip-api.com 返回 JSON 格式的 IP 信息
|
||||
resp, err := client.Get("http://ip-api.com/json/?fields=status,country,countryCode,city,query")
|
||||
if err != nil {
|
||||
return "", ""
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
var result struct {
|
||||
Status string `json:"status"`
|
||||
Query string `json:"query"` // IP 地址
|
||||
Country string `json:"country"`
|
||||
CountryCode string `json:"countryCode"`
|
||||
City string `json:"city"`
|
||||
}
|
||||
|
||||
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil || result.Status != "success" {
|
||||
return "", ""
|
||||
}
|
||||
|
||||
// 返回格式:IP, "国家代码 城市"
|
||||
location := result.CountryCode
|
||||
if result.City != "" {
|
||||
location = fmt.Sprintf("%s %s", result.CountryCode, result.City)
|
||||
}
|
||||
|
||||
return result.Query, location
|
||||
}
|
||||
|
||||
// ValidateAll 并发验证所有代理,返回验证结果
|
||||
@@ -70,8 +103,8 @@ func (v *Validator) ValidateStream(proxies []storage.Proxy) <-chan Result {
|
||||
go func(px storage.Proxy) {
|
||||
defer wg.Done()
|
||||
defer func() { <-sem }()
|
||||
valid, latency := v.ValidateOne(px)
|
||||
ch <- Result{Proxy: px, Valid: valid, Latency: latency}
|
||||
valid, latency, exitIP, exitLocation := v.ValidateOne(px)
|
||||
ch <- Result{Proxy: px, Valid: valid, Latency: latency, ExitIP: exitIP, ExitLocation: exitLocation}
|
||||
}(p)
|
||||
}
|
||||
wg.Wait()
|
||||
@@ -81,8 +114,8 @@ func (v *Validator) ValidateStream(proxies []storage.Proxy) <-chan Result {
|
||||
return ch
|
||||
}
|
||||
|
||||
// ValidateOne 验证单个代理是否可用,返回是否有效和延迟
|
||||
func (v *Validator) ValidateOne(p storage.Proxy) (bool, time.Duration) {
|
||||
// ValidateOne 验证单个代理是否可用,返回是否有效、延迟、出口IP和地理位置
|
||||
func (v *Validator) ValidateOne(p storage.Proxy) (bool, time.Duration, string, string) {
|
||||
var client *http.Client
|
||||
var err error
|
||||
|
||||
@@ -93,33 +126,50 @@ func (v *Validator) ValidateOne(p storage.Proxy) (bool, time.Duration) {
|
||||
client, err = newSOCKS5Client(p.Address, v.timeout)
|
||||
default:
|
||||
log.Printf("unknown protocol %s for %s", p.Protocol, p.Address)
|
||||
return false, 0
|
||||
return false, 0, "", ""
|
||||
}
|
||||
|
||||
if err != nil {
|
||||
return false, 0
|
||||
return false, 0, "", ""
|
||||
}
|
||||
|
||||
start := time.Now()
|
||||
resp, err := client.Get(v.validateURL)
|
||||
latency := time.Since(start)
|
||||
if err != nil {
|
||||
return false, 0
|
||||
return false, 0, "", ""
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
io.Copy(io.Discard, resp.Body)
|
||||
|
||||
// 验证状态码(200 或 204 都接受)
|
||||
if resp.StatusCode != http.StatusOK && resp.StatusCode != http.StatusNoContent {
|
||||
return false, latency
|
||||
return false, latency, "", ""
|
||||
}
|
||||
|
||||
// 响应时间过滤
|
||||
if v.maxResponseMs > 0 && latency > time.Duration(v.maxResponseMs)*time.Millisecond {
|
||||
return false, latency
|
||||
return false, latency, "", ""
|
||||
}
|
||||
|
||||
return true, latency
|
||||
// 获取出口 IP 和地理位置(仅在验证通过时)
|
||||
exitIP, exitLocation := getExitIPInfo(client)
|
||||
|
||||
// 必须能获取到出口信息
|
||||
if exitIP == "" || exitLocation == "" {
|
||||
return false, latency, exitIP, exitLocation
|
||||
}
|
||||
|
||||
// 过滤中国大陆出口(香港的countryCode是HK,不是CN)
|
||||
if len(exitLocation) >= 2 {
|
||||
countryCode := exitLocation[:2]
|
||||
if countryCode == "CN" {
|
||||
// 中国大陆出口,直接拒绝
|
||||
return false, latency, exitIP, exitLocation
|
||||
}
|
||||
}
|
||||
|
||||
return true, latency, exitIP, exitLocation
|
||||
}
|
||||
|
||||
func newHTTPClient(address string, timeout time.Duration) (*http.Client, error) {
|
||||
|
||||
1109
webui/dashboard.go
1109
webui/dashboard.go
File diff suppressed because it is too large
Load Diff
100
webui/html.go
100
webui/html.go
@@ -5,31 +5,45 @@ const loginHTML = `<!DOCTYPE html>
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>ProxyGo - 登录</title>
|
||||
<title>GoProxy — 身份验证</title>
|
||||
<link href="https://fonts.googleapis.com/css2?family=JetBrains+Mono:wght@400;600;700&family=Share+Tech+Mono&display=swap" rel="stylesheet">
|
||||
<style>
|
||||
*{box-sizing:border-box;margin:0;padding:0}
|
||||
body{background:#0f172a;color:#e2e8f0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',sans-serif;display:flex;align-items:center;justify-content:center;min-height:100vh}
|
||||
.card{background:#1e293b;border:1px solid #334155;border-radius:12px;padding:40px;width:360px;box-shadow:0 20px 60px rgba(0,0,0,0.5)}
|
||||
h1{font-size:24px;font-weight:700;margin-bottom:8px;color:#f1f5f9}
|
||||
.sub{color:#94a3b8;font-size:14px;margin-bottom:32px}
|
||||
label{display:block;font-size:13px;color:#94a3b8;margin-bottom:6px}
|
||||
input[type=password]{width:100%;padding:10px 14px;background:#0f172a;border:1px solid #334155;border-radius:8px;color:#f1f5f9;font-size:15px;outline:none;transition:border 0.2s}
|
||||
input[type=password]:focus{border-color:#6366f1}
|
||||
button{width:100%;margin-top:20px;padding:11px;background:#6366f1;color:#fff;border:none;border-radius:8px;font-size:15px;font-weight:600;cursor:pointer;transition:background 0.2s}
|
||||
button:hover{background:#4f46e5}
|
||||
.logo{font-size:32px;margin-bottom:16px}
|
||||
body{background:#0a0a0a;color:#00ff41;font-family:JetBrains Mono,monospace;display:flex;align-items:center;justify-content:center;min-height:100vh;position:relative}
|
||||
body::before{content:'';position:fixed;top:0;left:0;width:100%;height:100%;background:repeating-linear-gradient(0deg,rgba(0,255,65,0.03) 0px,transparent 1px,transparent 2px,rgba(0,255,65,0.03) 3px);pointer-events:none;z-index:9999}
|
||||
body::after{content:'';position:fixed;top:0;left:0;width:100%;height:100%;background:radial-gradient(ellipse at center,rgba(0,255,65,0.08) 0%,transparent 70%);pointer-events:none;z-index:9998}
|
||||
.card{border:1px solid #00ff41;padding:64px;width:440px;background:#111;box-shadow:0 0 40px rgba(0,255,65,0.3);position:relative;z-index:1}
|
||||
h1{font-size:32px;font-weight:700;margin-bottom:8px;letter-spacing:0.15em;text-transform:uppercase;color:#00ff41;text-shadow:0 0 15px #00ff41}
|
||||
.sub{color:#00cc33;font-size:12px;margin-bottom:48px;font-family:JetBrains Mono,monospace;letter-spacing:0.08em;text-transform:uppercase}
|
||||
label{display:block;font-size:10px;text-transform:uppercase;letter-spacing:0.1em;color:#00cc33;margin-bottom:8px;font-weight:600}
|
||||
input[type=password]{width:100%;padding:16px;background:#0d0d0d;border:1px solid #1a3a1a;color:#00ff41;font-size:16px;font-family:JetBrains Mono,monospace;outline:none;transition:all 0.2s}
|
||||
input[type=password]:focus{border-color:#00ff41;background:#111;box-shadow:0 0 10px rgba(0,255,65,0.3)}
|
||||
button{width:100%;margin-top:24px;padding:16px;background:#00ff41;color:#000;border:1px solid #00ff41;font-size:12px;font-weight:700;cursor:pointer;transition:all 0.2s;text-transform:uppercase;letter-spacing:0.1em;font-family:JetBrains Mono,monospace;box-shadow:0 0 15px rgba(0,255,65,0.5)}
|
||||
button:hover{box-shadow:0 0 25px rgba(0,255,65,0.8);transform:translateY(-1px)}
|
||||
.logo{font-size:64px;margin-bottom:24px;line-height:1;font-weight:700;letter-spacing:0.1em;color:#00ff41;text-shadow:0 0 20px #00ff41}
|
||||
.tip{color:#888;font-size:10px;margin-top:24px;line-height:1.6;letter-spacing:0.05em;text-align:center}
|
||||
.tip a{color:#00ff41;text-decoration:none;border-bottom:1px solid transparent;transition:all 0.2s}
|
||||
.tip a:hover{border-bottom-color:#00ff41;text-shadow:0 0 8px #00ff41}
|
||||
.github{position:absolute;top:20px;right:20px;color:#00ff41;opacity:0.6;transition:all 0.3s}
|
||||
.github:hover{opacity:1;transform:scale(1.1);filter:drop-shadow(0 0 8px #00ff41)}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<a href="https://github.com/isboyjc/ProxyGo" target="_blank" class="github" title="GitHub">
|
||||
<svg width="32" height="32" viewBox="0 0 16 16" fill="currentColor">
|
||||
<path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"/>
|
||||
</svg>
|
||||
</a>
|
||||
<div class="card">
|
||||
<div class="logo">⚡</div>
|
||||
<h1>ProxyGo</h1>
|
||||
<p class="sub">代理池管理系统</p>
|
||||
<div class="logo">[GP]</div>
|
||||
<h1>GoProxy</h1>
|
||||
<p class="sub">// Intelligent Proxy Pool</p>
|
||||
<form method="POST" action="/login">
|
||||
<label>管理密码</label>
|
||||
<input type="password" name="password" placeholder="请输入密码" autofocus>
|
||||
<button type="submit">登录</button>
|
||||
<label>> Password</label>
|
||||
<input type="password" name="password" placeholder="****************" autofocus>
|
||||
<button type="submit">[ AUTHENTICATE ]</button>
|
||||
</form>
|
||||
<p class="tip">访客模式可<a href="/">查看数据</a>,管理员登录后可完全控制</p>
|
||||
</div>
|
||||
</body>
|
||||
</html>`
|
||||
@@ -39,33 +53,47 @@ const loginHTMLWithError = `<!DOCTYPE html>
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>ProxyGo - 登录</title>
|
||||
<title>GoProxy — 身份验证</title>
|
||||
<link href="https://fonts.googleapis.com/css2?family=JetBrains+Mono:wght@400;600;700&family=Share+Tech+Mono&display=swap" rel="stylesheet">
|
||||
<style>
|
||||
*{box-sizing:border-box;margin:0;padding:0}
|
||||
body{background:#0f172a;color:#e2e8f0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',sans-serif;display:flex;align-items:center;justify-content:center;min-height:100vh}
|
||||
.card{background:#1e293b;border:1px solid #334155;border-radius:12px;padding:40px;width:360px;box-shadow:0 20px 60px rgba(0,0,0,0.5)}
|
||||
h1{font-size:24px;font-weight:700;margin-bottom:8px;color:#f1f5f9}
|
||||
.sub{color:#94a3b8;font-size:14px;margin-bottom:32px}
|
||||
label{display:block;font-size:13px;color:#94a3b8;margin-bottom:6px}
|
||||
input[type=password]{width:100%;padding:10px 14px;background:#0f172a;border:1px solid #334155;border-radius:8px;color:#f1f5f9;font-size:15px;outline:none;transition:border 0.2s}
|
||||
input[type=password]:focus{border-color:#6366f1}
|
||||
button{width:100%;margin-top:20px;padding:11px;background:#6366f1;color:#fff;border:none;border-radius:8px;font-size:15px;font-weight:600;cursor:pointer;transition:background 0.2s}
|
||||
button:hover{background:#4f46e5}
|
||||
.logo{font-size:32px;margin-bottom:16px}
|
||||
.error{background:#450a0a;border:1px solid #7f1d1d;color:#fca5a5;padding:10px 14px;border-radius:8px;font-size:13px;margin-bottom:16px}
|
||||
body{background:#0a0a0a;color:#00ff41;font-family:JetBrains Mono,monospace;display:flex;align-items:center;justify-content:center;min-height:100vh;position:relative}
|
||||
body::before{content:'';position:fixed;top:0;left:0;width:100%;height:100%;background:repeating-linear-gradient(0deg,rgba(0,255,65,0.03) 0px,transparent 1px,transparent 2px,rgba(0,255,65,0.03) 3px);pointer-events:none;z-index:9999}
|
||||
body::after{content:'';position:fixed;top:0;left:0;width:100%;height:100%;background:radial-gradient(ellipse at center,rgba(0,255,65,0.08) 0%,transparent 70%);pointer-events:none;z-index:9998}
|
||||
.card{border:1px solid #00ff41;padding:64px;width:440px;background:#111;box-shadow:0 0 40px rgba(0,255,65,0.3);position:relative;z-index:1}
|
||||
h1{font-size:32px;font-weight:700;margin-bottom:8px;letter-spacing:0.15em;text-transform:uppercase;color:#00ff41;text-shadow:0 0 15px #00ff41}
|
||||
.sub{color:#00cc33;font-size:12px;margin-bottom:48px;font-family:JetBrains Mono,monospace;letter-spacing:0.08em;text-transform:uppercase}
|
||||
label{display:block;font-size:10px;text-transform:uppercase;letter-spacing:0.1em;color:#00cc33;margin-bottom:8px;font-weight:600}
|
||||
input[type=password]{width:100%;padding:16px;background:#0d0d0d;border:1px solid #1a3a1a;color:#00ff41;font-size:16px;font-family:JetBrains Mono,monospace;outline:none;transition:all 0.2s}
|
||||
input[type=password]:focus{border-color:#00ff41;background:#111;box-shadow:0 0 10px rgba(0,255,65,0.3)}
|
||||
button{width:100%;margin-top:24px;padding:16px;background:#00ff41;color:#000;border:1px solid #00ff41;font-size:12px;font-weight:700;cursor:pointer;transition:all 0.2s;text-transform:uppercase;letter-spacing:0.1em;font-family:JetBrains Mono,monospace;box-shadow:0 0 15px rgba(0,255,65,0.5)}
|
||||
button:hover{box-shadow:0 0 25px rgba(0,255,65,0.8);transform:translateY(-1px)}
|
||||
.logo{font-size:64px;margin-bottom:24px;line-height:1;font-weight:700;letter-spacing:0.1em;color:#00ff41;text-shadow:0 0 20px #00ff41}
|
||||
.error{background:#ff0033;color:#fff;padding:16px;font-size:11px;margin-bottom:24px;font-family:JetBrains Mono,monospace;font-weight:600;border:1px solid #ff0033;box-shadow:0 0 15px rgba(255,0,51,0.5);text-transform:uppercase;letter-spacing:0.05em}
|
||||
.tip{color:#888;font-size:10px;margin-top:24px;line-height:1.6;letter-spacing:0.05em;text-align:center}
|
||||
.tip a{color:#00ff41;text-decoration:none;border-bottom:1px solid transparent;transition:all 0.2s}
|
||||
.tip a:hover{border-bottom-color:#00ff41;text-shadow:0 0 8px #00ff41}
|
||||
.github{position:absolute;top:20px;right:20px;color:#00ff41;opacity:0.6;transition:all 0.3s}
|
||||
.github:hover{opacity:1;transform:scale(1.1);filter:drop-shadow(0 0 8px #00ff41)}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<a href="https://github.com/isboyjc/ProxyGo" target="_blank" class="github" title="GitHub">
|
||||
<svg width="32" height="32" viewBox="0 0 16 16" fill="currentColor">
|
||||
<path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"/>
|
||||
</svg>
|
||||
</a>
|
||||
<div class="card">
|
||||
<div class="logo">⚡</div>
|
||||
<h1>ProxyGo</h1>
|
||||
<p class="sub">代理池管理系统</p>
|
||||
<div class="error">密码错误,请重试</div>
|
||||
<div class="logo">[GP]</div>
|
||||
<h1>GoProxy</h1>
|
||||
<p class="sub">// Intelligent Proxy Pool</p>
|
||||
<div class="error">[!] ACCESS DENIED - INVALID PASSWORD</div>
|
||||
<form method="POST" action="/login">
|
||||
<label>管理密码</label>
|
||||
<input type="password" name="password" placeholder="请输入密码" autofocus>
|
||||
<button type="submit">登录</button>
|
||||
<label>> Password</label>
|
||||
<input type="password" name="password" placeholder="****************" autofocus>
|
||||
<button type="submit">[ AUTHENTICATE ]</button>
|
||||
</form>
|
||||
<p class="tip">访客模式可<a href="/">查看数据</a>,管理员登录后可完全控制</p>
|
||||
</div>
|
||||
</body>
|
||||
</html>`
|
||||
|
||||
292
webui/server.go
292
webui/server.go
@@ -9,9 +9,11 @@ import (
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"proxy-pool/config"
|
||||
"proxy-pool/logger"
|
||||
"proxy-pool/storage"
|
||||
"goproxy/config"
|
||||
"goproxy/logger"
|
||||
"goproxy/pool"
|
||||
"goproxy/storage"
|
||||
"goproxy/validator"
|
||||
)
|
||||
|
||||
// 简单内存 session
|
||||
@@ -44,12 +46,19 @@ type FetchTrigger func()
|
||||
type Server struct {
|
||||
storage *storage.Storage
|
||||
cfg *config.Config
|
||||
poolMgr *pool.Manager
|
||||
fetchTrigger FetchTrigger
|
||||
configChanged chan<- struct{}
|
||||
}
|
||||
|
||||
func New(s *storage.Storage, cfg *config.Config, ft FetchTrigger, cc chan<- struct{}) *Server {
|
||||
return &Server{storage: s, cfg: cfg, fetchTrigger: ft, configChanged: cc}
|
||||
func New(s *storage.Storage, cfg *config.Config, pm *pool.Manager, ft FetchTrigger, cc chan<- struct{}) *Server {
|
||||
return &Server{
|
||||
storage: s,
|
||||
cfg: cfg,
|
||||
poolMgr: pm,
|
||||
fetchTrigger: ft,
|
||||
configChanged: cc,
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Server) Start() {
|
||||
@@ -57,12 +66,22 @@ func (s *Server) Start() {
|
||||
mux.HandleFunc("/", s.handleIndex)
|
||||
mux.HandleFunc("/login", s.handleLogin)
|
||||
mux.HandleFunc("/logout", s.handleLogout)
|
||||
mux.HandleFunc("/api/stats", s.authMiddleware(s.apiStats))
|
||||
mux.HandleFunc("/api/proxies", s.authMiddleware(s.apiProxies))
|
||||
|
||||
// 只读 API(访客可访问)
|
||||
mux.HandleFunc("/api/stats", s.readOnlyMiddleware(s.apiStats))
|
||||
mux.HandleFunc("/api/proxies", s.readOnlyMiddleware(s.apiProxies))
|
||||
mux.HandleFunc("/api/logs", s.readOnlyMiddleware(s.apiLogs))
|
||||
mux.HandleFunc("/api/pool/status", s.readOnlyMiddleware(s.apiPoolStatus))
|
||||
mux.HandleFunc("/api/pool/quality", s.readOnlyMiddleware(s.apiQualityDistribution))
|
||||
mux.HandleFunc("/api/config", s.readOnlyMiddleware(s.apiConfig))
|
||||
mux.HandleFunc("/api/auth/check", s.apiAuthCheck) // 检查登录状态
|
||||
|
||||
// 管理员 API(需要登录)
|
||||
mux.HandleFunc("/api/proxy/delete", s.authMiddleware(s.apiDeleteProxy))
|
||||
mux.HandleFunc("/api/proxy/refresh", s.authMiddleware(s.apiRefreshProxy))
|
||||
mux.HandleFunc("/api/fetch", s.authMiddleware(s.apiFetch))
|
||||
mux.HandleFunc("/api/logs", s.authMiddleware(s.apiLogs))
|
||||
mux.HandleFunc("/api/config", s.authMiddleware(s.apiConfig))
|
||||
mux.HandleFunc("/api/refresh-latency", s.authMiddleware(s.apiRefreshLatency))
|
||||
mux.HandleFunc("/api/config/save", s.authMiddleware(s.apiConfigSave))
|
||||
|
||||
log.Printf("WebUI listening on %s", s.cfg.WebUIPort)
|
||||
go func() {
|
||||
@@ -72,6 +91,7 @@ func (s *Server) Start() {
|
||||
}()
|
||||
}
|
||||
|
||||
// authMiddleware 管理员权限中间件(必须登录)
|
||||
func (s *Server) authMiddleware(next http.HandlerFunc) http.HandlerFunc {
|
||||
return func(w http.ResponseWriter, r *http.Request) {
|
||||
if !validSession(r) {
|
||||
@@ -86,11 +106,16 @@ func (s *Server) authMiddleware(next http.HandlerFunc) http.HandlerFunc {
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Server) handleIndex(w http.ResponseWriter, r *http.Request) {
|
||||
if !validSession(r) {
|
||||
http.Redirect(w, r, "/login", http.StatusFound)
|
||||
return
|
||||
// readOnlyMiddleware 只读中间件(访客可访问,但会标记是否为管理员)
|
||||
func (s *Server) readOnlyMiddleware(next http.HandlerFunc) http.HandlerFunc {
|
||||
return func(w http.ResponseWriter, r *http.Request) {
|
||||
// 访客和管理员都可以访问,通过 validSession 判断权限
|
||||
next(w, r)
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Server) handleIndex(w http.ResponseWriter, r *http.Request) {
|
||||
// 允许访客访问(只读模式),管理员登录后有完整权限
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
fmt.Fprint(w, dashboardHTML)
|
||||
}
|
||||
@@ -129,6 +154,20 @@ func (s *Server) handleLogout(w http.ResponseWriter, r *http.Request) {
|
||||
http.Redirect(w, r, "/login", http.StatusFound)
|
||||
}
|
||||
|
||||
// apiAuthCheck 检查当前用户是否为管理员
|
||||
func (s *Server) apiAuthCheck(w http.ResponseWriter, r *http.Request) {
|
||||
isAdmin := validSession(r)
|
||||
jsonOK(w, map[string]interface{}{
|
||||
"isAdmin": isAdmin,
|
||||
"mode": func() string {
|
||||
if isAdmin {
|
||||
return "admin"
|
||||
}
|
||||
return "guest"
|
||||
}(),
|
||||
})
|
||||
}
|
||||
|
||||
func (s *Server) apiStats(w http.ResponseWriter, r *http.Request) {
|
||||
total, _ := s.storage.Count()
|
||||
httpCount, _ := s.storage.CountByProtocol("http")
|
||||
@@ -173,6 +212,60 @@ func (s *Server) apiDeleteProxy(w http.ResponseWriter, r *http.Request) {
|
||||
jsonOK(w, map[string]string{"status": "deleted"})
|
||||
}
|
||||
|
||||
func (s *Server) apiRefreshProxy(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodPost {
|
||||
jsonError(w, "method not allowed", http.StatusMethodNotAllowed)
|
||||
return
|
||||
}
|
||||
var req struct {
|
||||
Address string `json:"address"`
|
||||
}
|
||||
if err := json.NewDecoder(r.Body).Decode(&req); err != nil || req.Address == "" {
|
||||
jsonError(w, "invalid request", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
// 从数据库获取代理信息
|
||||
proxies, err := s.storage.GetAll()
|
||||
if err != nil {
|
||||
jsonError(w, "failed to get proxy", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
var targetProxy *storage.Proxy
|
||||
for i := range proxies {
|
||||
if proxies[i].Address == req.Address {
|
||||
targetProxy = &proxies[i]
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
if targetProxy == nil {
|
||||
jsonError(w, "proxy not found", http.StatusNotFound)
|
||||
return
|
||||
}
|
||||
|
||||
// 异步验证并更新
|
||||
go func() {
|
||||
cfg := config.Get()
|
||||
v := validator.New(1, cfg.ValidateTimeout, cfg.ValidateURL)
|
||||
|
||||
log.Printf("[webui] refreshing proxy: %s", req.Address)
|
||||
valid, latency, exitIP, exitLocation := v.ValidateOne(*targetProxy)
|
||||
|
||||
if valid {
|
||||
latencyMs := int(latency.Milliseconds())
|
||||
s.storage.UpdateExitInfo(req.Address, exitIP, exitLocation, latencyMs)
|
||||
log.Printf("[webui] proxy refreshed: %s latency=%dms grade=%s", req.Address, latencyMs, storage.CalculateQualityGrade(latencyMs))
|
||||
} else {
|
||||
s.storage.Delete(req.Address)
|
||||
log.Printf("[webui] proxy validation failed, removed: %s", req.Address)
|
||||
}
|
||||
}()
|
||||
|
||||
jsonOK(w, map[string]string{"status": "refresh started"})
|
||||
}
|
||||
|
||||
func (s *Server) apiFetch(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodPost {
|
||||
jsonError(w, "method not allowed", http.StatusMethodNotAllowed)
|
||||
@@ -182,54 +275,173 @@ func (s *Server) apiFetch(w http.ResponseWriter, r *http.Request) {
|
||||
jsonOK(w, map[string]string{"status": "fetch started"})
|
||||
}
|
||||
|
||||
func (s *Server) apiLogs(w http.ResponseWriter, r *http.Request) {
|
||||
lines := logger.GetLines(200)
|
||||
jsonOK(w, map[string]interface{}{"lines": lines})
|
||||
}
|
||||
|
||||
func (s *Server) apiConfig(w http.ResponseWriter, r *http.Request) {
|
||||
cfg := config.Get()
|
||||
if r.Method == http.MethodGet {
|
||||
jsonOK(w, map[string]interface{}{
|
||||
"fetch_interval": cfg.FetchInterval,
|
||||
"check_interval": cfg.CheckInterval,
|
||||
"validate_concurrency": cfg.ValidateConcurrency,
|
||||
"validate_timeout": cfg.ValidateTimeout,
|
||||
})
|
||||
return
|
||||
}
|
||||
func (s *Server) apiRefreshLatency(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodPost {
|
||||
jsonError(w, "method not allowed", http.StatusMethodNotAllowed)
|
||||
return
|
||||
}
|
||||
var req struct {
|
||||
FetchInterval int `json:"fetch_interval"`
|
||||
CheckInterval int `json:"check_interval"`
|
||||
ValidateConcurrency int `json:"validate_concurrency"`
|
||||
ValidateTimeout int `json:"validate_timeout"`
|
||||
go func() {
|
||||
log.Println("[webui] refreshing latency for all proxies...")
|
||||
proxies, err := s.storage.GetAll()
|
||||
if err != nil {
|
||||
log.Printf("[webui] get proxies error: %v", err)
|
||||
return
|
||||
}
|
||||
if len(proxies) == 0 {
|
||||
log.Println("[webui] no proxies to refresh")
|
||||
return
|
||||
}
|
||||
|
||||
cfg := config.Get()
|
||||
validate := validator.New(cfg.ValidateConcurrency, cfg.ValidateTimeout, cfg.ValidateURL)
|
||||
|
||||
log.Printf("[webui] refreshing latency for %d proxies...", len(proxies))
|
||||
updated := 0
|
||||
for r := range validate.ValidateStream(proxies) {
|
||||
if r.Valid {
|
||||
latencyMs := int(r.Latency.Milliseconds())
|
||||
s.storage.UpdateExitInfo(r.Proxy.Address, r.ExitIP, r.ExitLocation, latencyMs)
|
||||
updated++
|
||||
} else {
|
||||
s.storage.Delete(r.Proxy.Address)
|
||||
}
|
||||
}
|
||||
log.Printf("[webui] latency refresh done: updated=%d", updated)
|
||||
}()
|
||||
jsonOK(w, map[string]string{"status": "refresh started"})
|
||||
}
|
||||
|
||||
func (s *Server) apiLogs(w http.ResponseWriter, r *http.Request) {
|
||||
lines := logger.GetLines(100)
|
||||
jsonOK(w, map[string]interface{}{"lines": lines})
|
||||
}
|
||||
|
||||
// apiConfig 获取配置
|
||||
func (s *Server) apiConfig(w http.ResponseWriter, r *http.Request) {
|
||||
cfg := config.Get()
|
||||
httpSlots, socks5Slots := cfg.CalculateSlots()
|
||||
|
||||
jsonOK(w, map[string]interface{}{
|
||||
// 池子配置
|
||||
"pool_max_size": cfg.PoolMaxSize,
|
||||
"pool_http_ratio": cfg.PoolHTTPRatio,
|
||||
"pool_min_per_protocol": cfg.PoolMinPerProtocol,
|
||||
"pool_http_slots": httpSlots,
|
||||
"pool_socks5_slots": socks5Slots,
|
||||
|
||||
// 延迟配置
|
||||
"max_latency_ms": cfg.MaxLatencyMs,
|
||||
"max_latency_emergency": cfg.MaxLatencyEmergency,
|
||||
"max_latency_healthy": cfg.MaxLatencyHealthy,
|
||||
|
||||
// 验证配置
|
||||
"validate_concurrency": cfg.ValidateConcurrency,
|
||||
"validate_timeout": cfg.ValidateTimeout,
|
||||
|
||||
// 健康检查配置
|
||||
"health_check_interval": cfg.HealthCheckInterval,
|
||||
"health_check_batch_size": cfg.HealthCheckBatchSize,
|
||||
|
||||
// 优化配置
|
||||
"optimize_interval": cfg.OptimizeInterval,
|
||||
"replace_threshold": cfg.ReplaceThreshold,
|
||||
})
|
||||
}
|
||||
|
||||
// apiConfigSave 保存配置
|
||||
func (s *Server) apiConfigSave(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodPost {
|
||||
jsonError(w, "method not allowed", http.StatusMethodNotAllowed)
|
||||
return
|
||||
}
|
||||
|
||||
var req struct {
|
||||
PoolMaxSize int `json:"pool_max_size"`
|
||||
PoolHTTPRatio float64 `json:"pool_http_ratio"`
|
||||
PoolMinPerProtocol int `json:"pool_min_per_protocol"`
|
||||
MaxLatencyMs int `json:"max_latency_ms"`
|
||||
MaxLatencyEmergency int `json:"max_latency_emergency"`
|
||||
MaxLatencyHealthy int `json:"max_latency_healthy"`
|
||||
ValidateConcurrency int `json:"validate_concurrency"`
|
||||
ValidateTimeout int `json:"validate_timeout"`
|
||||
HealthCheckInterval int `json:"health_check_interval"`
|
||||
HealthCheckBatchSize int `json:"health_check_batch_size"`
|
||||
OptimizeInterval int `json:"optimize_interval"`
|
||||
ReplaceThreshold float64 `json:"replace_threshold"`
|
||||
}
|
||||
|
||||
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
||||
jsonError(w, "invalid request", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
if req.FetchInterval <= 0 || req.CheckInterval <= 0 || req.ValidateConcurrency <= 0 || req.ValidateTimeout <= 0 {
|
||||
jsonError(w, "all values must be positive", http.StatusBadRequest)
|
||||
|
||||
// 验证配置有效性
|
||||
if req.PoolMaxSize <= 0 || req.PoolHTTPRatio <= 0 || req.PoolHTTPRatio > 1 {
|
||||
jsonError(w, "invalid pool config", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
if err := config.Save(req.FetchInterval, req.CheckInterval, req.ValidateConcurrency, req.ValidateTimeout); err != nil {
|
||||
|
||||
// 记录旧配置
|
||||
oldCfg := config.Get()
|
||||
oldSize := oldCfg.PoolMaxSize
|
||||
oldRatio := oldCfg.PoolHTTPRatio
|
||||
|
||||
// 更新配置
|
||||
newCfg := *oldCfg
|
||||
newCfg.PoolMaxSize = req.PoolMaxSize
|
||||
newCfg.PoolHTTPRatio = req.PoolHTTPRatio
|
||||
newCfg.PoolMinPerProtocol = req.PoolMinPerProtocol
|
||||
newCfg.MaxLatencyMs = req.MaxLatencyMs
|
||||
newCfg.MaxLatencyEmergency = req.MaxLatencyEmergency
|
||||
newCfg.MaxLatencyHealthy = req.MaxLatencyHealthy
|
||||
newCfg.ValidateConcurrency = req.ValidateConcurrency
|
||||
newCfg.ValidateTimeout = req.ValidateTimeout
|
||||
newCfg.HealthCheckInterval = req.HealthCheckInterval
|
||||
newCfg.HealthCheckBatchSize = req.HealthCheckBatchSize
|
||||
newCfg.OptimizeInterval = req.OptimizeInterval
|
||||
newCfg.ReplaceThreshold = req.ReplaceThreshold
|
||||
|
||||
if err := config.Save(&newCfg); err != nil {
|
||||
jsonError(w, "save config error: "+err.Error(), http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
// 通知定时器重置
|
||||
|
||||
// 通知配置变更
|
||||
select {
|
||||
case s.configChanged <- struct{}{}:
|
||||
default:
|
||||
}
|
||||
log.Printf("[config] updated: fetch=%dm check=%dm concurrency=%d timeout=%ds",
|
||||
req.FetchInterval, req.CheckInterval, req.ValidateConcurrency, req.ValidateTimeout)
|
||||
|
||||
// 如果池子大小或比例变更,调整池子
|
||||
if oldSize != req.PoolMaxSize || oldRatio != req.PoolHTTPRatio {
|
||||
go s.poolMgr.AdjustForConfigChange(oldSize, oldRatio)
|
||||
}
|
||||
|
||||
log.Printf("[config] 配置已更新: 池子=%d HTTP=%.0f%% 延迟=%dms",
|
||||
req.PoolMaxSize, req.PoolHTTPRatio*100, req.MaxLatencyMs)
|
||||
jsonOK(w, map[string]string{"status": "saved"})
|
||||
}
|
||||
|
||||
// apiPoolStatus 获取池子状态
|
||||
func (s *Server) apiPoolStatus(w http.ResponseWriter, r *http.Request) {
|
||||
status, err := s.poolMgr.GetStatus()
|
||||
if err != nil {
|
||||
jsonError(w, err.Error(), http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
jsonOK(w, status)
|
||||
}
|
||||
|
||||
// apiQualityDistribution 获取质量分布
|
||||
func (s *Server) apiQualityDistribution(w http.ResponseWriter, r *http.Request) {
|
||||
dist, err := s.storage.GetQualityDistribution()
|
||||
if err != nil {
|
||||
jsonError(w, err.Error(), http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
jsonOK(w, dist)
|
||||
}
|
||||
|
||||
func jsonOK(w http.ResponseWriter, data interface{}) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(data)
|
||||
|
||||
Reference in New Issue
Block a user