feat: init

This commit is contained in:
isboyjc
2026-03-29 03:31:59 +08:00
parent f2c3fbac24
commit f55209d8d3
27 changed files with 4424 additions and 664 deletions

30
.gitignore vendored
View File

@@ -1,17 +1,41 @@
# macOS
.DS_Store
# Build artifacts
# Windows
Thumbs.db
# Go build artifacts
goproxy
proxy-pool
*.exe
*.out
*.test
vendor/
# Test binaries
test/test_proxy
test/test_proxy.exe
# Runtime data
*.db
*.db-shm
*.db-wal
*.log
config.json
data/*.db
data/config.json
data/
!data/.gitkeep
# Temporary files
*.tmp
*.bak
*.swp
*~
# IDE files
.idea/
.vscode/
.cursor/
*.iml
# Docker volumes
.docker-data/

827
README.md
View File

@@ -1,42 +1,88 @@
# ProxyGo
# GoProxy
一个基于 Go 的轻量代理池服务。程序会从公开代理源抓取 HTTP/SOCKS5 代理,验证可用性后写入 SQLite并对外暴露一个统一的本地 HTTP 代理入口,同时提供带登录的 Web 管理后台。
> **智能代理池系统** — 基于 Go 的轻量级、低资源消耗、自适应的代理池服务
## 功能概览
goproxy 从多个公开代理源自动抓取 HTTP/SOCKS5 代理,通过严格验证(出口 IP + 位置 + 延迟)后加入智能代理池,对外提供统一的 HTTP 代理服务。系统采用质量分级、智能补充、自动优化等机制,确保代理池始终保持高质量和稳定性。
- 启动时自动抓取并验证代理
- 后台定时抓取新代理
- 后台定时健康检查,自动清理不可用代理
- 聚合 HTTP 和 SOCKS5 上游代理,对外统一提供 HTTP 代理端口
- 支持普通 HTTP 请求和 HTTPS `CONNECT` 隧道转发
- 内置 WebUI支持查看统计、筛选代理、删除代理、手动触发抓取、查看日志、修改部分运行参数
- 使用 SQLite 持久化代理池数据
## ✨ 核心特性
## 项目结构
### 🎯 智能池子机制
- **固定容量管理**:可配置池子大小和 HTTP/SOCKS5 协议比例
- **质量分级**S/A/B/C 四级评分(基于延迟),智能选择高质量代理
- **动态状态感知**Healthy → Warning → Critical → Emergency 四级状态自适应
- **严格准入标准**:必须通过出口 IP、地理位置、延迟三重验证才可入池
- **智能替换**:新代理必须显著优于现有代理(默认快 30%)才触发替换
### 🚀 按需抓取
- **源分组策略**快更新源5-30min用于紧急补充慢更新源每天用于优化轮换
- **断路器保护**:连续失败的源自动降级/禁用,冷却后恢复
- **多模式抓取**
- **Emergency**:单协议缺失或池子 <10%,使用所有可用源
- **Refill**:池子 <80%,使用快更新源
- **Optimize**:池子健康时,随机抽取少量慢源优化替换
### 🏥 分层健康管理
- **轻量批次检查**:每次仅检查 20 个代理,避免资源浪费
- **智能跳过 S 级**:池子健康时跳过 S 级代理检查
- **定时优化轮换**:健康状态下,定期抓取优质代理替换池中延迟高的
### 🔄 智能重试机制
- **自动故障切换**:代理请求失败时立即切换到另一个代理重试(最多 3 次)
- **失败即删除**:连接失败或请求超时的代理立即从池子中移除
- **用户无感知**:自动重试在服务端完成,用户只会收到成功响应或最终失败提示
- **防重复尝试**:已尝试过的失败代理不会在同一请求中再次使用
### 🚪 双端口策略
- **7777 端口(随机轮换)**每次请求随机选择代理IP 高度分散,适合爬虫和数据采集
- **7776 端口(最低延迟)**:固定使用延迟最低的代理,除非失败才切换,适合长连接和流媒体
- **自动切换**两个端口都支持失败自动重试7776 失败后切换到次优代理
- **共享池子**:两个端口使用同一个代理池,统一管理和优化
### 🎨 黑客风格 WebUI
- **Matrix 美学**:荧光绿 + 纯黑背景CRT 扫描线效果JetBrains Mono 等宽字体
- **双角色权限**:访客模式(只读)+ 管理员模式(完全控制),可安全公网开放
- **实时仪表盘**:池子状态、质量分布可视化、协议统计,带荧光光晕效果
- **完整配置界面**:池子容量、延迟标准、验证参数、优化策略均可在线调整(管理员)
- **代理注册表**:详细展示地址、出口 IP、位置、延迟、质量等级、使用统计
- **中英文切换**:支持中文/英文界面切换,默认中文
- **交互优化**:点击地址复制、单个代理刷新、实时日志倒计时
### 📊 适用场景
- **小型 VPS**:低资源消耗(固定池子 + 按需抓取 + 限流查询)
- **稳定需求**:自动剔除失败代理,始终保持健康池子
- **质量优先**S/A 级代理优先使用,自动优化延迟
## 📦 项目结构
```text
.
├── main.go # 程序入口
├── config/ # 默认配置、配置加载与保存
├── fetcher/ # 代理源抓取
├── validator/ # 代理可用性验证
├── checker/ # 周期健康检查
├── storage/ # SQLite 存储
├── main.go # 程序入口,协调所有模块
├── config/ # 配置系统(池子容量、延迟标准、验证参数等)
├── pool/ # 🆕 池子管理器(入池判断、替换逻辑、状态计算)
├── fetcher/ # 🆕 智能抓取器(源分组、断路器、按需抓取)
│ ├── fetcher.go # 多模式抓取逻辑
├── source_manager.go # 源状态管理和断路器
│ └── ip_query.go # IP查询限流和多源降级
├── validator/ # 代理验证(连接测试 + 出口IP检测
├── checker/ # 🆕 分批健康检查器
├── optimizer/ # 🆕 优化轮换器(定时优化池子质量)
├── storage/ # 🆕 扩展存储层(质量等级、使用统计、源状态表)
├── proxy/ # 对外 HTTP 代理服务
├── webui/ # 登录页、仪表盘、API
├── logger/ # 内存日志 + stdout 输出
├── Dockerfile
└── docker-compose.yml
├── webui/ # 🆕 黑客风格 WebUI健康仪表盘、配置界面
├── logger/ # 内存日志收集
├── test/ # 🧪 测试脚本Bash/Go/Python
│ ├── test_proxy.sh # Bash 测试脚本
│ ├── test_proxy.go # Go 测试脚本
│ ├── test_proxy.py # Python 测试脚本
│ └── TEST_GUIDE.md # 测试指南
└── POOL_DESIGN.md # 🆕 完整架构设计文档
```
## 运行要求
## 🚀 快速开始
### 运行要求
- Go `1.25`
- 需要可用的 CGO 编译环境
- 项目依赖 `github.com/mattn/go-sqlite3`
- 本地构建通常需要 `gcc` / Xcode Command Line Tools
## 快速开始
- CGO 编译环境(依赖 `github.com/mattn/go-sqlite3`
### 本地运行
@@ -47,40 +93,100 @@ go run .
或先编译再启动:
```bash
go build -o proxy-pool .
./proxy-pool
go build -o proxygo .
./proxygo
```
程序启动后会:
1. 加载默认配置或读取 `config.json`
2. 初始化 SQLite 数据库
3. 启动 WebUI
4. 立即抓取一次代理并开始验证
5. 启动定时抓取和健康检查
6.`:7777` 启动统一代理服务
1. 加载配置(优先 `config.json`
2. 初始化数据库和限流器
3. 启动 WebUI`:7778`
4. 立即执行智能填充(按需抓取 + 严格验证)
5. 启动后台协程:
- 状态监控(每分钟)
- 健康检查(默认 5 分钟)
- 优化轮换(默认 30 分钟)
6. 启动两个代理服务:
- `:7776` - 最低延迟模式(稳定连接)
- `:7777` - 随机轮换模式IP 多样性)
### 默认端口
- **代理服务(随机轮换)**`127.0.0.1:7777` - 每次请求随机选择代理IP 多样性高
- **代理服务(最低延迟)**`127.0.0.1:7776` - 固定使用延迟最低的代理,性能优先
- **WebUI**`http://127.0.0.1:7778`
- **默认密码**`goproxy`(可通过 `WEBUI_PASSWORD` 环境变量自定义)
- 代理服务:`127.0.0.1:7777``:7777`
- WebUI`http://127.0.0.1:7778`
### 使用代理
### 使用聚合代理
GoProxy 提供**两个代理端口**,满足不同场景需求:
例如:
#### 🎲 7777 端口 - 随机轮换模式
适合需要 **IP 多样性** 的场景(爬虫、数据采集、负载均衡):
```bash
curl -x http://127.0.0.1:7777 https://httpbin.org/ip
```
也可以给命令行程序设置环境变量
**特点**
- 每次请求随机选择一个代理
- 优先使用高质量S/A 级)代理
- IP 地址高度分散
#### ⚡ 7776 端口 - 最低延迟模式
适合需要 **稳定连接** 的场景(长连接、流媒体、实时通信):
```bash
export http_proxy=http://127.0.0.1:7777
export https_proxy=http://127.0.0.1:7777
curl -x http://127.0.0.1:7776 https://httpbin.org/ip
```
## Docker
**特点**
- 固定使用池中延迟最低的代理
- 除非该代理失败,否则不会切换
- 失败时自动删除并切换到下一个最低延迟代理
- 性能和稳定性最优
#### 环境变量配置
```bash
# 使用随机模式
export http_proxy=http://127.0.0.1:7777
export https_proxy=http://127.0.0.1:7777
# 或使用稳定模式
export http_proxy=http://127.0.0.1:7776
export https_proxy=http://127.0.0.1:7776
```
#### 端口对比
| 特性 | 7777随机轮换 | 7776最低延迟 |
|------|-----------------|-----------------|
| **选择策略** | 随机选择(优先高质量) | 固定使用延迟最低的 |
| **IP 多样性** | ⭐⭐⭐⭐⭐ 高度分散 | ⭐ 基本固定 |
| **连接稳定性** | ⭐⭐⭐ 每次切换 | ⭐⭐⭐⭐⭐ 固定不变 |
| **性能表现** | ⭐⭐⭐⭐ 平均延迟 | ⭐⭐⭐⭐⭐ 最优延迟 |
| **适用场景** | 爬虫、数据采集、防封禁 | 长连接、流媒体、下载 |
| **失败切换** | 自动重试 3 次 | 失败后切换到次优代理 |
#### 自动重试机制说明
当你通过 GoProxy 发送请求时,如果上游代理失败,系统会**自动处理**
1. **立即删除失败代理**:从池子中移除不可用的代理
2. **自动切换重试**:随机选择另一个可用代理重新发送请求(最多重试 3 次)
3. **用户完全无感知**:整个过程在服务端完成,你的应用只会收到成功响应或最终失败提示
4. **防止重复尝试**:同一请求中不会重复使用已失败的代理
**示例流程**
```
用户请求 → 代理A失败(删除) → 自动切换代理B → 代理B成功 → 返回响应
```
这意味着即使池子中有部分失效代理,你的应用依然可以正常工作,系统会自动保持池子质量。
## 🐳 Docker 部署
### 使用 Dockerfile
@@ -88,170 +194,597 @@ export https_proxy=http://127.0.0.1:7777
docker build -t proxygo .
docker run -d \
--name proxygo \
-p 127.0.0.1:7776:7776 \
-p 127.0.0.1:7777:7777 \
-p 7778:7778 \
-e TZ=Asia/Shanghai \
-e WEBUI_PASSWORD=your_password \
-e DATA_DIR=/app/data \
-v "$(pwd)/data:/app/data" \
proxygo
```
### 使用 docker-compose.yml
### 使用 docker-compose
```bash
docker compose up -d --build
```
当前仓库中的 `docker-compose.yml` 有两个前提:
### WebUI 公网访问配置
- 它会将 `./data` 挂载到容器内 `/app/data`
- 它依赖一个已存在的外部网络 `cursor2api_default`
得益于**双角色权限系统**WebUI 可以安全地对外开放:
如果宿主机没有这个网络,需要先创建,或者直接修改 `docker-compose.yml` 中的网络配置。
```bash
docker run -d \
--name proxygo \
-p 127.0.0.1:7776:7776 \
-p 127.0.0.1:7777:7777 \
-p 0.0.0.0:7778:7778 \ # 公网访问 WebUI
-e TZ=Asia/Shanghai \
-e WEBUI_PASSWORD=strong_password \
-e DATA_DIR=/app/data \
-v "$(pwd)/data:/app/data" \
proxygo
```
## 数据目录
**安全说明**
- 访客(未登录)只能查看数据,无法执行任何操作
- 所有写操作(抓取、删除、配置修改)都需要管理员密码
- 建议设置强密码(通过 `WEBUI_PASSWORD` 环境变量)
- 代理服务端口7776、7777建议仅绑定内网`127.0.0.1`
程序支持通过 `DATA_DIR` 指定数据目录。
## ⚙️ 配置说明
- 未设置 `DATA_DIR` 时:
- 数据库默认写到项目根目录 `proxy.db`
- 配置文件默认读取/写入项目根目录 `config.json`
- 设置 `DATA_DIR=/app/data` 时:
- 数据库路径变为 `/app/data/proxy.db`
- 配置文件路径变为 `/app/data/config.json`
### 配置文件示例
## 配置说明
### 可持久化配置
当前版本只会从 `config.json` 读取并保存以下 4 个字段:
所有配置均可通过 WebUI 的 **Configure Pool** 界面在线调整,也可以手动编辑 `config.json`
```json
{
"fetch_interval": 30,
"check_interval": 10,
"pool_max_size": 100,
"pool_http_ratio": 0.5,
"pool_min_per_protocol": 10,
"max_latency_ms": 2000,
"max_latency_healthy": 1500,
"max_latency_emergency": 3000,
"validate_concurrency": 300,
"validate_timeout": 3
"validate_timeout": 8,
"health_check_interval": 5,
"health_check_batch_size": 20,
"optimize_interval": 30,
"replace_threshold": 0.7
}
```
字段含义:
### 配置参数详解
- `fetch_interval`:定时抓取间隔,单位分钟
- `check_interval`:健康检查间隔,单位分钟
- `validate_concurrency`:并发验证数量
- `validate_timeout`:单个代理验证超时,单位秒
**服务端口配置**
这些参数既可以通过编辑 `config.json` 修改,也可以在 WebUI 的“系统设置”中在线保存。
| 参数 | 默认值 | 说明 |
| --- | --- | --- |
| `proxy_port` | `:7777` | 随机轮换代理端口 |
| `stable_proxy_port` | `:7776` | 最低延迟代理端口 |
| `webui_port` | `:7778` | WebUI 端口 |
### 当前代码中的默认值
**池子容量配置**
除上面 4 项外,其余配置目前来自代码默认值:
| 参数 | 默认值 | 说明 | 推荐范围 |
| --- | --- | --- | --- |
| `pool_max_size` | `100` | 代理池总容量 | 50-500 |
| `pool_http_ratio` | `0.5` | HTTP 协议占比 | 0.3-0.8 |
| `pool_min_per_protocol` | `10` | 每协议最少保证数量 | 5-50 |
| 配置项 | 默认值 | 说明 |
**延迟标准配置**
| 参数 | 默认值 | 说明 | 推荐范围 |
| --- | --- | --- | --- |
| `max_latency_ms` | `2000` | 标准模式最大延迟(毫秒) | 1000-3000 |
| `max_latency_healthy` | `1500` | 健康模式严格延迟(毫秒) | 800-2000 |
| `max_latency_emergency` | `3000` | 紧急模式放宽延迟(毫秒) | 2000-5000 |
**验证与健康检查配置**
| 参数 | 默认值 | 说明 | 推荐范围 |
| --- | --- | --- | --- |
| `validate_concurrency` | `300` | 并发验证数量 | 100-500 |
| `validate_timeout` | `8` | 验证超时(秒) | 5-15 |
| `health_check_interval` | `5` | 检查间隔(分钟) | 3-15 |
| `health_check_batch_size` | `20` | 每批检查数量 | 10-50 |
**优化配置**
| 参数 | 默认值 | 说明 | 推荐范围 |
| --- | --- | --- | --- |
| `optimize_interval` | `30` | 优化轮换间隔(分钟) | 15-120 |
| `replace_threshold` | `0.7` | 替换阈值(新代理需快 30% | 0.5-0.9 |
### 不同场景配置建议
**小型 VPS1C2G**
```json
{
"pool_max_size": 50,
"pool_http_ratio": 0.5,
"validate_concurrency": 100,
"health_check_interval": 10,
"health_check_batch_size": 10,
"optimize_interval": 60
}
```
**中型服务器2C4G+**
```json
{
"pool_max_size": 200,
"pool_http_ratio": 0.6,
"validate_concurrency": 300,
"health_check_interval": 5,
"health_check_batch_size": 30,
"optimize_interval": 30
}
```
**低延迟优先**
```json
{
"pool_max_size": 100,
"max_latency_ms": 1000,
"max_latency_healthy": 800,
"optimize_interval": 15,
"replace_threshold": 0.8
}
```
**高可用优先(需要更多代理)**
```json
{
"pool_max_size": 300,
"pool_http_ratio": 0.7,
"pool_min_per_protocol": 20,
"max_latency_ms": 3000
}
```
### 固定配置
以下配置在代码中固定,无需调整:
| 配置项 | 值 | 说明 |
| --- | --- | --- |
| `WebUIPort` | `:7778` | Web 管理后台端口 |
| `ProxyPort` | `:7777` | 对外统一代理端口 |
| `DBPath` | `proxy.db``${DATA_DIR}/proxy.db` | SQLite 数据库路径 |
| `ValidateURL` | `https://cursor.com/api/auth/me` | 验证目标地址 |
| `MaxResponseMs` | `2500` | 最大可接受延迟,毫秒 |
| `MaxFailCount` | `3` | 失败阈值字段已定义,但当前运行逻辑未完整使用 |
| `MaxRetry` | `3` | 请求失败后的重试次数 |
| `ValidateURL` | `http://www.gstatic.com/generate_204` | 验证目标地址 |
| `IPQueryRateLimit` | `10` | IP 查询限流(次/秒) |
| `SourceFailThreshold` | `3` | 源降级阈值 |
| `SourceDisableThreshold` | `5` | 源禁用阈值 |
| `SourceCooldownMinutes` | `30` | 源禁用冷却时间(分钟) |
## WebUI
## 🎨 WebUI 使用指南
访问地址:
访问地址:`http://127.0.0.1:7778`
- `http://127.0.0.1:7778`
### 👥 双角色权限系统
提供的功能
GoProxy WebUI 支持**访客模式**和**管理员模式**
- 登录鉴权
- 展示代理总数、HTTP 数量、SOCKS5 数量
- 按协议筛选代理
- 删除单个代理
- 手动触发抓取
- 查看最近日志
- 在线修改抓取/校验参数
#### 访客模式(只读)
### 登录密码说明
**无需登录**即可访问,可以查看所有数据但不能操作:
默认密码为 `proxygo`,程序启动时会在日志中提示。
- ✅ 查看池子状态和质量分布
- ✅ 查看代理列表和详细信息
- ✅ 查看系统日志
- ✅ 点击复制代理地址
- ❌ 不能抓取代理
- ❌ 不能刷新延迟
- ❌ 不能删除代理
- ❌ 不能修改配置
可通过环境变量 `WEBUI_PASSWORD` 自定义密码
**适用场景**
- 团队成员监控代理池状态
- 展示给客户或第三方查看
- 公网开放访问(只读数据安全)
#### ⚡ 管理员模式(完全控制)
**登录后**拥有所有操作权限:
- ✅ 所有访客模式的查看功能
- ✅ 手动触发代理抓取
- ✅ 刷新所有代理延迟
- ✅ 刷新单个代理信息
- ✅ 删除指定代理
- ✅ 修改池子配置(容量、延迟标准、检查间隔等)
**默认密码**`goproxy`(通过环境变量 `WEBUI_PASSWORD` 自定义)
### 健康仪表盘
**四宫格指标卡**
- **Pool Status**当前池子状态HEALTHY/WARNING/CRITICAL/EMERGENCY
- **Total Proxies**:总代理数 / 池子容量
- **HTTP**HTTP 代理数 / HTTP 槽位数 + 平均延迟
- **SOCKS5**SOCKS5 代理数 / SOCKS5 槽位数 + 平均延迟
**质量分布可视化**
- 横向条形图展示 S/A/B/C 四级质量分布
- 实时显示各级别代理数量
### 代理注册表
**表格字段**
- **Grade**质量等级S/A/B/C
- **Protocol**协议类型HTTP/SOCKS5
- **Address**代理地址host:port
- **Exit IP**:出口 IP
- **Location**:出口地理位置(国旗 + 国家代码 + 城市)
- **Latency**:延迟(毫秒,颜色编码)
- **Usage**:使用次数 / 成功次数
- **Action**:删除按钮
**操作功能**
- **筛选**All / HTTP / SOCKS5所有用户可用
- **点击复制地址**:点击代理地址单元格复制到剪贴板(所有用户可用)
- **Fetch Proxies**:手动触发智能抓取(⚡ 管理员专属)
- **Refresh Latency**:重新验证所有代理并更新延迟(⚡ 管理员专属)
- **刷新单个代理**:点击行内刷新按钮验证单个代理(⚡ 管理员专属)
- **删除代理**:点击行内删除按钮移除指定代理(⚡ 管理员专属)
- **Configure Pool**:打开配置界面(⚡ 管理员专属)
### 配置界面(⚡ 管理员专属)
点击 **Configure Pool** 打开配置模态框,包含:
**Pool Capacity 部分**
- Max Size池子总容量
- HTTP RatioHTTP 协议占比0.5 = 50%
- Min Per Protocol每协议最小保证
**Latency Standards 部分**
- Standard标准模式延迟阈值
- Healthy健康模式严格延迟
- Emergency紧急模式放宽延迟
**Validation & Health Check 部分**
- Validate Concurrency并发验证数
- Validate Timeout验证超时
- Health Check Interval健康检查间隔
- Health Check Batch Size每批检查数量
**Optimization 部分**
- Optimize Interval优化轮换间隔
- Replace Threshold替换阈值0.7 = 新代理需快 30%
保存后立即生效,系统会自动调整池子策略。
## 🏗️ 核心架构
### 智能池子生命周期
```text
[启动] → 状态监控 → 判断池子健康度
需要补充?
↙ ↘
是 否
↓ ↓
智能抓取 保持监控
(多模式) ↓
↓ 优化轮换
严格验证 (定时执行)
↓ ↓
智能入池 替换劣质代理
(替换逻辑) ↓
↓ 分批健康检查
↓ (剔除失败)
↓ ↓
└─────────┘
持续优化循环
```
### 状态转换机制
```text
Healthy (总数≥80% 且 各协议≥80%槽位)
↓ 代理失效
Warning (总数<80% 或 任一协议<80%)
↓ 继续失效
Critical (总数<50% 或 任一协议<20%槽位)
↓ 继续失效
Emergency (总数<10% 或 单协议缺失)
└─ 自动触发紧急抓取 ─┘
```
### 抓取模式选择
| 池子状态 | 抓取模式 | 使用源 | 触发条件 |
| --- | --- | --- | --- |
| Emergency | 紧急模式 | 所有可用源 | 单协议缺失或总数<10% |
| Critical/Warning | 补充模式 | 快更新源 | 总数<80%或协议不均 |
| Healthy | 优化模式 | 慢更新源随机2-3个 | 定时触发30分钟 |
### 质量分级标准
| 等级 | 延迟范围 | 说明 | 权重 |
| --- | --- | --- | --- |
| S | ≤500ms | 超快,优先使用,健康状态跳过检查 | 最高 |
| A | 501-1000ms | 良好,稳定可用 | 高 |
| B | 1001-2000ms | 可用,会被优化替换 | 中 |
| C | >2000ms | 淘汰候选,优先替换 | 低 |
## 🔧 数据库 Schema
### proxies 表
| 字段 | 类型 | 说明 |
| --- | --- | --- |
| `id` | INTEGER | 主键 |
| `address` | TEXT | 代理地址UNIQUE |
| `protocol` | TEXT | 协议类型http/socks5 |
| `exit_ip` | TEXT | 出口 IP |
| `exit_location` | TEXT | 出口位置 |
| `latency` | INTEGER | 延迟(毫秒) |
| `quality_grade` | TEXT | 质量等级S/A/B/C |
| `use_count` | INTEGER | 使用次数 |
| `success_count` | INTEGER | 成功次数 |
| `fail_count` | INTEGER | 失败次数 |
| `last_used` | DATETIME | 最后使用时间 |
| `last_check` | DATETIME | 最后检查时间 |
| `created_at` | DATETIME | 创建时间 |
| `status` | TEXT | 状态active/degraded/candidate_replace |
### source_status 表
| 字段 | 类型 | 说明 |
| --- | --- | --- |
| `id` | INTEGER | 主键 |
| `url` | TEXT | 源地址UNIQUE |
| `success_count` | INTEGER | 成功次数 |
| `fail_count` | INTEGER | 失败次数 |
| `consecutive_fails` | INTEGER | 连续失败次数 |
| `last_success` | DATETIME | 最后成功时间 |
| `last_fail` | DATETIME | 最后失败时间 |
| `status` | TEXT | 状态active/degraded/disabled |
| `disabled_until` | DATETIME | 禁用到期时间 |
## 🔍 代理源
系统内置 16 个代理源,分为快更新和慢更新两组:
**快更新源5-30分钟更新**
- proxifly/free-proxy-list (HTTP/SOCKS4/SOCKS5)
- ProxyScraper/ProxyScraper (HTTP/SOCKS4/SOCKS5)
- monosans/proxy-list (HTTP)
**慢更新源(每天更新)**
- TheSpeedX/SOCKS-List (HTTP/SOCKS4/SOCKS5)
- monosans/proxy-list (SOCKS4/SOCKS5)
- databay-labs/free-proxy-list (HTTP/SOCKS5)
系统会根据池子状态自动选择合适的源组:
- 紧急/补充模式:使用快更新源,快速填充
- 优化模式:随机选择慢更新源,精细优化
## 🚦 核心机制详解
### 1. 智能入池机制
每个代理在入池前需通过:
1. **连接验证**:能否成功连接 `http://www.gstatic.com/generate_204`
2. **出口 IP 检测**:获取代理的出口 IP
3. **地理位置查询**:获取出口 IP 的国家/城市
4. **延迟测试**:测量连接延迟
5. **质量评估**:根据延迟计算质量等级
**入池判断逻辑**
- ✅ 协议槽位未满:直接加入
- ✅ 槽位满但总量允许10%浮动:浮动加入
- 🔄 池子满且质量更优替换延迟最高的现有代理需快30%+
- ❌ 池子满且质量不足:拒绝
### 2. 健康检查机制
**批次检查策略**
- 每次检查 20 个代理(可配置)
- 优先检查长时间未检查的
- 池子健康时跳过 S 级代理(降低资源消耗)
**检查结果处理**
- ✅ 验证通过:更新延迟、出口 IP、质量等级
- ❌ 验证失败:失败计数 +1≥3次自动删除
### 3. 优化轮换机制
**触发条件**
- 池子状态Healthy
- 定时触发:默认 30 分钟
**优化流程**
1. 从慢更新源随机抽取 2-3 个源
2. 抓取候选代理并验证
3. 筛选出延迟 ≤1500ms 的优质代理
4. 尝试替换池中 B/C 级代理需快30%+
**资源控制**
- 仅在池子健康时执行
- 抽取少量源,避免浪费
- 严格质量标准≤1500ms
### 4. 源管理与断路器
**状态跟踪**
- 记录每个源的成功/失败次数
- 连续失败 3 次降级Degraded
- 连续失败 5 次:禁用 30 分钟Disabled
- 冷却期结束:自动恢复为 Active
**好处**
- 避免浪费资源在失效源上
- 自动恢复,无需人工干预
- 保护系统免受源故障影响
## 📖 常见问题
### Q: 为什么池子容量是固定的?
A: 固定容量可以:
- **可预测资源消耗**内存、CPU、网络带宽均可控
- **提升代理质量**:通过严格准入和替换保持高质量
- **简化管理逻辑**:避免无限增长和复杂的淘汰策略
### Q: 如何调整池子大小和协议比例?
A:
1. 访问 WebUI → 点击 **Configure Pool**
2. 修改 **Max Size****HTTP Ratio**
3. 点击 **Save Configuration**
4. 系统会自动调整槽位分配
示例:
- 池子大小 200HTTP 比例 0.7 → HTTP 槽位 140SOCKS5 槽位 60
- 池子大小 50HTTP 比例 0.3 → HTTP 槽位 15SOCKS5 槽位 35
### Q: 池子状态如何计算?
A:
- **Healthy**:总数 ≥80% 且各协议 ≥80% 槽位
- **Warning**:总数 <80% 或任一协议 <80% 槽位
- **Critical**:总数 <50% 或任一协议 <20% 槽位
- **Emergency**:总数 <10% 或单协议缺失
### Q: 如何优化延迟?
A: 系统会自动优化,也可以手动调整:
1. 降低 `max_latency_healthy`(严格模式)
2. 增加 `optimize_interval` 频率(更频繁优化)
3. 调高 `replace_threshold`(要求新代理更快)
4. 点击 **Refresh Latency** 立即重新验证
### Q: 为什么有的代理没有出口 IP
A:
- IP 查询有限流10 次/秒)
- 部分代理可能不支持 IP 查询
- 系统会在后续健康检查中补全信息
### Q: 资源消耗如何?
A:
- **内存**:池子 100 个约 50MB200 个约 100MB
- **CPU**:空闲时 <1%,验证时 10-30%(取决于并发数)
- **网络**
- IP 查询限流 10 次/秒
- 按需抓取,避免无效流量
- 健康检查批次小20 个)
## 📚 详细设计文档
完整的架构设计、模块说明、配置策略、资源优化方案,请查看:
👉 [POOL_DESIGN.md](./POOL_DESIGN.md)
## 🛠️ 开发与调试
### 查看日志
日志会输出到 stdout同时在 WebUI 的 **System Log** 部分实时展示。
关键日志标识:
- `[pool]`:池子管理器
- `[fetch]`:抓取器
- `[source]`:源管理器
- `[health]`:健康检查器
- `[optimize]`:优化器
- `[monitor]`:状态监控器
### 数据库操作
```bash
# 本地运行
WEBUI_PASSWORD=your-password go run .
# 查看当前代理
sqlite3 data/proxy.db "SELECT address, protocol, latency, quality_grade, status FROM proxies LIMIT 10;"
# Docker
docker run -e WEBUI_PASSWORD=your-password ...
# 查看质量分布
sqlite3 data/proxy.db "SELECT quality_grade, COUNT(*) FROM proxies WHERE status='active' GROUP BY quality_grade;"
# 查看源状态
sqlite3 data/proxy.db "SELECT url, status, consecutive_fails FROM source_status;"
# 清空池子(慎用)
sqlite3 data/proxy.db "DELETE FROM proxies;"
```
也可以在 `docker-compose.yml``environment` 中添加:
## 🧪 测试代理服务
```yaml
- WEBUI_PASSWORD=your-password
项目提供了三种测试脚本,用于验证代理服务功能和性能(位于 `test/` 目录)。
### 快速测试
```bash
# 测试随机轮换模式(默认 7777 端口)
./test/test_proxy.sh
# 测试最低延迟模式7776 端口)
./test/test_proxy.sh 7776
# 使用 Go 脚本
go run test/test_proxy.go # 默认 7777
go run test/test_proxy.go 7776 # 测试 7776
# 使用 Python 脚本
python test/test_proxy.py # 默认 7777
python test/test_proxy.py 7776 # 测试 7776
# 按 Ctrl+C 停止测试并查看统计
```
## API 概览
测试脚本特点:
- **持续运行模式**:类似 `ping` 命令,持续发送请求
- 实时显示每次请求的出口 IP 和延迟
- 动态更新成功率统计
- 验证代理轮换机制
-`Ctrl+C` 停止并显示完整统计报告
`/login``/logout` 外,其余管理 API 都要求已登录会话。
### 测试输出示例
| 方法 | 路径 | 说明 |
| --- | --- | --- |
| `GET` | `/` | 仪表盘页面 |
| `GET/POST` | `/login` | 登录页面 / 提交登录 |
| `GET` | `/logout` | 退出登录 |
| `GET` | `/api/stats` | 代理统计信息 |
| `GET` | `/api/proxies?protocol=http` | 查询代理列表,可按协议筛选 |
| `POST` | `/api/proxy/delete` | 删除指定代理 |
| `POST` | `/api/fetch` | 手动触发一次抓取 |
| `GET` | `/api/logs` | 获取最近 200 条日志 |
| `GET/POST` | `/api/config` | 读取/保存运行参数 |
```
PROXY 127.0.0.1:7777 (http://ip-api.com/json/?fields=countryCode,query): continuous mode
## 代理抓取与校验逻辑
proxy from 🇺🇸 203.0.113.45: seq=1 time=1234ms
proxy from 🇩🇪 198.51.100.78: seq=2 time=987ms
proxy from 🇬🇧 192.0.2.123: seq=3 time=1567ms
proxy #4: request failed (timeout)
proxy from 🇯🇵 198.51.100.12: seq=5 time=890ms
...(持续运行,按 Ctrl+C 停止)
当前实现会并发抓取内置代理源,然后做去重与验证:
^C
---
50 requests transmitted, 47 received, 3 failed, 6.0% packet loss
```
- HTTP 源:`https://cdn.jsdelivr.net/gh/databay-labs/free-proxy-list/http.txt`
- SOCKS5 源:`https://cdn.jsdelivr.net/gh/databay-labs/free-proxy-list/socks5.txt`
- 混合源:`https://cdn.jsdelivr.net/gh/proxifly/free-proxy-list@main/proxies/all/data.txt`
详细测试指南请查看:👉 [test/TEST_GUIDE.md](./test/TEST_GUIDE.md)
验证规则:
## 🙏 致谢与声明
- 仅接受 HTTP `200``204`
- 响应超时或延迟超过 `MaxResponseMs` 的代理会被丢弃
- 默认验证目标为 `https://cursor.com/api/auth/me`
本项目基于 [jonasen1988/proxygo](https://github.com/jonasen1988/proxygo) 进行魔改和增强。
## 日志
### 原项目
- **项目地址**https://github.com/jonasen1988/proxygo
- **作者**jonasen1988
- **基础功能**代理抓取、验证、存储、HTTP代理服务、WebUI管理
- 日志会输出到进程标准输出
- 同时会保留最近 500 条在内存中供 WebUI 展示
- `/api/logs` 当前返回最近 200 条日志
### 本项目增强功能
在原项目基础上,我们进行了大量改进和功能增强:
## 当前实现限制
- 🆕 **智能池子机制**固定容量管理、质量分级S/A/B/C、智能替换逻辑
- 🆕 **按需抓取策略**源分组、断路器保护、Emergency/Refill/Optimize 多模式
- 🆕 **分层健康管理**:批次检查、智能跳过 S 级、定时优化轮换
- 🆕 **智能重试机制**:自动故障切换、失败即删除、防重复尝试
- 🆕 **双端口服务**7777 随机轮换IP 多样性)+ 7776 最低延迟(稳定连接)
- 🆕 **黑客风格 WebUI**Matrix 美学、实时仪表盘、完整配置界面、中英文切换
- 🆕 **双角色权限**:访客模式(只读)+ 管理员模式(完全控制),可安全公网开放
- 🆕 **扩展存储层**:质量等级、使用统计、源状态管理
- 🆕 **测试套件**Bash/Go/Python 三种测试脚本,持续运行模式,显示国旗 emoji
- `config.Config` 中虽然定义了 `HTTPSourceURL``SOCKS5SourceURL`,但抓取器当前实际使用的是 `fetcher/defaultSources` 内置来源
- `config.json` 目前只持久化 4 个字段,不包含端口、密码哈希、验证 URL 等配置
- WebUI 登录密码不能在线修改
- 代理请求失败时,运行逻辑倾向于直接删除上游代理,`MaxFailCount` 目前没有完整参与主流程
- 日志没有单独写文件,管理端看到的是内存中的最近日志窗口
感谢原作者提供的基础实现,让我们能够在此之上构建更强大的代理池系统。
## 适用场景
同时感谢 [LINUX DO](https://linux.do/) 社区的支持。
- 在本机快速聚合一批公开代理,提供给命令行或程序统一使用
- 临时验证免费 HTTP / SOCKS5 代理的可用性
- 通过简单 Web 面板查看当前代理池状态
## 📝 License
如果后续要继续完善,优先建议补这几项:
- 支持通过配置文件完整覆盖所有默认参数
- 支持自定义代理源并真正接入抓取器
- 支持 WebUI 密码初始化和修改
- 为失败计数、重试和删除策略补齐一致的状态流转
- 增加自动化测试
## 致谢
认可 [LINUX DO](https://linux.do/)
MIT License

View File

@@ -4,9 +4,9 @@ import (
"log"
"time"
"proxy-pool/config"
"proxy-pool/storage"
"proxy-pool/validator"
"goproxy/config"
"goproxy/storage"
"goproxy/validator"
)
type Checker struct {
@@ -52,6 +52,19 @@ func (c *Checker) run() {
for _, r := range results {
if r.Valid {
valid++
// 更新出口 IP、位置和延迟信息
latencyMs := int(r.Latency.Milliseconds())
if r.ExitIP != "" && r.Proxy.ExitIP == "" {
// 如果之前没有出口 IP 信息,更新完整信息
if err := c.storage.UpdateExitInfo(r.Proxy.Address, r.ExitIP, r.ExitLocation, latencyMs); err != nil {
log.Printf("[checker] update exit info error: %v", err)
}
} else if r.Latency > 0 {
// 否则只更新延迟
if err := c.storage.UpdateLatency(r.Proxy.Address, latencyMs); err != nil {
log.Printf("[checker] update latency error: %v", err)
}
}
} else {
invalid++
if err := c.storage.Delete(r.Proxy.Address); err != nil {

103
checker/health_checker.go Normal file
View File

@@ -0,0 +1,103 @@
package checker
import (
"log"
"time"
"goproxy/config"
"goproxy/pool"
"goproxy/storage"
"goproxy/validator"
)
// HealthChecker 健康检查器
type HealthChecker struct {
storage *storage.Storage
validator *validator.Validator
cfg *config.Config
poolMgr *pool.Manager
}
func NewHealthChecker(s *storage.Storage, v *validator.Validator, cfg *config.Config, pm *pool.Manager) *HealthChecker {
return &HealthChecker{
storage: s,
validator: v,
cfg: cfg,
poolMgr: pm,
}
}
// RunOnce 执行一次健康检查
func (hc *HealthChecker) RunOnce() {
start := time.Now()
log.Println("[health] 开始健康检查...")
// 获取池子状态
status, err := hc.poolMgr.GetStatus()
if err != nil {
log.Printf("[health] 获取状态失败: %v", err)
return
}
// 健康状态且S级占比高时跳过S级代理检查
skipSGrade := status.State == "healthy"
dist, _ := hc.storage.GetQualityDistribution()
sGradeCount := dist["S"]
totalCount := status.Total
if totalCount > 0 && float64(sGradeCount)/float64(totalCount) > 0.3 {
skipSGrade = true
}
// 批量获取需要检查的代理
proxies, err := hc.storage.GetBatchForHealthCheck(hc.cfg.HealthCheckBatchSize, skipSGrade)
if err != nil {
log.Printf("[health] 获取检查批次失败: %v", err)
return
}
if len(proxies) == 0 {
log.Println("[health] 无需检查的代理")
return
}
log.Printf("[health] 检查 %d 个代理跳过S级=%v", len(proxies), skipSGrade)
// 执行验证
validCount := 0
removeCount := 0
updateCount := 0
for result := range hc.validator.ValidateStream(proxies) {
if result.Valid {
validCount++
// 更新延迟和质量等级
latencyMs := int(result.Latency.Milliseconds())
if err := hc.storage.UpdateExitInfo(result.Proxy.Address, result.ExitIP, result.ExitLocation, latencyMs); err == nil {
updateCount++
}
} else {
// 失败次数+1
hc.storage.IncrementFailCount(result.Proxy.Address)
// 如果失败次数 >= 3删除
if result.Proxy.FailCount+1 >= 3 {
hc.storage.Delete(result.Proxy.Address)
removeCount++
}
}
}
elapsed := time.Since(start)
log.Printf("[health] 完成: 验证%d 有效%d 更新%d 移除%d 耗时%v",
len(proxies), validCount, updateCount, removeCount, elapsed)
}
// StartBackground 后台定时健康检查
func (hc *HealthChecker) StartBackground() {
ticker := time.NewTicker(time.Duration(hc.cfg.HealthCheckInterval) * time.Minute)
go func() {
for range ticker.C {
hc.RunOnce()
}
}()
log.Printf("[health] 健康检查器已启动,间隔 %d 分钟", hc.cfg.HealthCheckInterval)
}

View File

@@ -8,7 +8,7 @@ import (
"sync"
)
const DefaultPassword = "proxygo"
const DefaultPassword = "goproxy"
func dataDir() string {
if d := os.Getenv("DATA_DIR"); d != "" {
@@ -27,37 +27,57 @@ type Config struct {
// WebUI 密码 SHA256 哈希
WebUIPasswordHash string
// 代理池本地监听端口
// 代理池本地监听端口(随机轮换模式)
ProxyPort string
// 稳定代理端口(最低延迟模式)
StableProxyPort string
// SQLite 数据库路径
DBPath string
// 验证并发数
ValidateConcurrency int
// ========== 池子容量配置 ==========
PoolMaxSize int // 代理池总容量默认100
PoolHTTPRatio float64 // HTTP协议占比默认0.5
PoolMinPerProtocol int // 每协议最小保证默认10
// 验证超时(秒)
ValidateTimeout int
// ========== 延迟标准配置 ==========
MaxLatencyMs int // 标准模式最大延迟默认2000ms
MaxLatencyEmergency int // 紧急模式放宽延迟默认3000ms
MaxLatencyHealthy int // 健康模式严格延迟默认1500ms
MaxLatencyDegradation int // 降级模式超宽松延迟默认5000ms
// 验证目标 URL
ValidateURL string
// ========== 验证配置 ==========
ValidateConcurrency int // 验证并发数默认300
ValidateTimeout int // 验证超时默认8
ValidateURL string // 验证目标 URL
// 最大响应时间(毫秒),超过则丢弃
MaxResponseMs int
// ========== 健康检查配置 ==========
HealthCheckInterval int // 状态监控间隔分钟默认5
HealthCheckBatchSize int // 每批验证数量默认20
HealthCheckConcurrency int // 批次内并发数默认50
// 代理失败次数阈值,超过后删除
MaxFailCount int
// ========== 优化配置 ==========
OptimizeInterval int // 优化轮换间隔分钟默认30
OptimizeConcurrency int // 优化时并发数默认100
ReplaceThreshold float64 // 替换阈值默认0.7新代理需快30%
// 自动重试次数
MaxRetry int
// ========== IP查询配置 ==========
IPQueryRateLimit int // IP查询限流次/秒默认10
// 定时抓取间隔(分钟)
FetchInterval int
// ========== 源管理配置 ==========
SourceFailThreshold int // 源降级阈值默认3
SourceDisableThreshold int // 源禁用阈值默认5
SourceCooldownMinutes int // 源禁用冷却时间默认30
// 定时健康检查间隔(分钟)
CheckInterval int
// ========== 兼容旧配置 ==========
MaxResponseMs int // 已废弃,使用 MaxLatencyMs 替代
MaxFailCount int // 代理失败次数阈值
MaxRetry int // 请求失败后的重试次数
FetchInterval int // 已废弃,由智能抓取器管理
CheckInterval int // 已废弃,由 HealthCheckInterval 替代
// 代理来源 URL
// 代理来源 URL(已废弃,内置多源)
HTTPSourceURL string
SOCKS5SourceURL string
}
@@ -78,20 +98,55 @@ func DefaultConfig() *Config {
password = DefaultPassword
}
return &Config{
WebUIPort: ":7778",
WebUIPasswordHash: passwordHash(password),
ProxyPort: ":7777",
DBPath: dataDir() + "proxy.db",
// 基础服务配置
WebUIPort: ":7778",
WebUIPasswordHash: passwordHash(password),
ProxyPort: ":7777",
StableProxyPort: ":7776",
DBPath: dataDir() + "proxy.db",
// 池子容量配置
PoolMaxSize: 100, // 总容量
PoolHTTPRatio: 0.5, // HTTP占50%
PoolMinPerProtocol: 10, // 每协议最少10个
// 延迟标准配置
MaxLatencyMs: 2000, // 标准2秒
MaxLatencyEmergency: 3000, // 紧急3秒
MaxLatencyHealthy: 1500, // 健康1.5秒
MaxLatencyDegradation: 5000, // 降级5秒
// 验证配置
ValidateConcurrency: 300,
ValidateTimeout: 3,
ValidateURL: "https://cursor.com/api/auth/me",
MaxResponseMs: 2500,
MaxFailCount: 3,
MaxRetry: 3,
FetchInterval: 30,
CheckInterval: 10,
HTTPSourceURL: "https://cdn.jsdelivr.net/gh/databay-labs/free-proxy-list/http.txt",
SOCKS5SourceURL: "https://cdn.jsdelivr.net/gh/databay-labs/free-proxy-list/socks5.txt",
ValidateTimeout: 8,
ValidateURL: "http://www.gstatic.com/generate_204",
// 健康检查配置
HealthCheckInterval: 5, // 5分钟
HealthCheckBatchSize: 20, // 每批20个
HealthCheckConcurrency: 50, // 批次并发50
// 优化配置
OptimizeInterval: 30, // 30分钟
OptimizeConcurrency: 100, // 并发100
ReplaceThreshold: 0.7, // 新代理需快30%
// IP查询配置
IPQueryRateLimit: 10, // 10次/秒
// 源管理配置
SourceFailThreshold: 3, // 失败3次降级
SourceDisableThreshold: 5, // 失败5次禁用
SourceCooldownMinutes: 30, // 禁用30分钟
// 兼容旧配置
MaxResponseMs: 5000,
MaxFailCount: 3,
MaxRetry: 3,
FetchInterval: 30,
CheckInterval: 10,
HTTPSourceURL: "https://cdn.jsdelivr.net/gh/databay-labs/free-proxy-list/http.txt",
SOCKS5SourceURL: "https://cdn.jsdelivr.net/gh/databay-labs/free-proxy-list/socks5.txt",
}
}
@@ -100,21 +155,61 @@ func Load() *Config {
cfg := DefaultConfig()
data, err := os.ReadFile(ConfigFile())
if err == nil {
// 只覆盖可调整的4个字段
var saved savedConfig
if json.Unmarshal(data, &saved) == nil {
if saved.FetchInterval > 0 {
cfg.FetchInterval = saved.FetchInterval
// 池子配置
if saved.PoolMaxSize > 0 {
cfg.PoolMaxSize = saved.PoolMaxSize
}
if saved.CheckInterval > 0 {
cfg.CheckInterval = saved.CheckInterval
if saved.PoolHTTPRatio > 0 && saved.PoolHTTPRatio <= 1 {
cfg.PoolHTTPRatio = saved.PoolHTTPRatio
}
if saved.PoolMinPerProtocol > 0 {
cfg.PoolMinPerProtocol = saved.PoolMinPerProtocol
}
// 延迟配置
if saved.MaxLatencyMs > 0 {
cfg.MaxLatencyMs = saved.MaxLatencyMs
}
if saved.MaxLatencyEmergency > 0 {
cfg.MaxLatencyEmergency = saved.MaxLatencyEmergency
}
if saved.MaxLatencyHealthy > 0 {
cfg.MaxLatencyHealthy = saved.MaxLatencyHealthy
}
// 验证配置
if saved.ValidateConcurrency > 0 {
cfg.ValidateConcurrency = saved.ValidateConcurrency
}
if saved.ValidateTimeout > 0 {
cfg.ValidateTimeout = saved.ValidateTimeout
}
// 健康检查配置
if saved.HealthCheckInterval > 0 {
cfg.HealthCheckInterval = saved.HealthCheckInterval
}
if saved.HealthCheckBatchSize > 0 {
cfg.HealthCheckBatchSize = saved.HealthCheckBatchSize
}
// 优化配置
if saved.OptimizeInterval > 0 {
cfg.OptimizeInterval = saved.OptimizeInterval
}
if saved.ReplaceThreshold > 0 && saved.ReplaceThreshold <= 1 {
cfg.ReplaceThreshold = saved.ReplaceThreshold
}
// 兼容旧配置
if saved.FetchInterval > 0 {
cfg.FetchInterval = saved.FetchInterval
}
if saved.CheckInterval > 0 {
cfg.CheckInterval = saved.CheckInterval
}
}
}
cfgMu.Lock()
@@ -130,31 +225,91 @@ func Get() *Config {
return globalCfg
}
// savedConfig 持久化可调整的字段
// savedConfig 持久化可调整的字段
type savedConfig struct {
FetchInterval int `json:"fetch_interval"`
CheckInterval int `json:"check_interval"`
// 池子配置
PoolMaxSize int `json:"pool_max_size"`
PoolHTTPRatio float64 `json:"pool_http_ratio"`
PoolMinPerProtocol int `json:"pool_min_per_protocol"`
// 延迟配置
MaxLatencyMs int `json:"max_latency_ms"`
MaxLatencyEmergency int `json:"max_latency_emergency"`
MaxLatencyHealthy int `json:"max_latency_healthy"`
// 验证配置
ValidateConcurrency int `json:"validate_concurrency"`
ValidateTimeout int `json:"validate_timeout"`
// 健康检查配置
HealthCheckInterval int `json:"health_check_interval"`
HealthCheckBatchSize int `json:"health_check_batch_size"`
// 优化配置
OptimizeInterval int `json:"optimize_interval"`
ReplaceThreshold float64 `json:"replace_threshold"`
// 兼容旧配置
FetchInterval int `json:"fetch_interval,omitempty"`
CheckInterval int `json:"check_interval,omitempty"`
}
// Save 保存可调整字段到文件,并更新内存配置
func Save(fetchInterval, checkInterval, validateConcurrency, validateTimeout int) error {
// Save 保存配置到文件,并更新内存配置
func Save(cfg *Config) error {
cfgMu.Lock()
globalCfg.FetchInterval = fetchInterval
globalCfg.CheckInterval = checkInterval
globalCfg.ValidateConcurrency = validateConcurrency
globalCfg.ValidateTimeout = validateTimeout
*globalCfg = *cfg
cfgMu.Unlock()
data, err := json.MarshalIndent(savedConfig{
FetchInterval: fetchInterval,
CheckInterval: checkInterval,
ValidateConcurrency: validateConcurrency,
ValidateTimeout: validateTimeout,
PoolMaxSize: cfg.PoolMaxSize,
PoolHTTPRatio: cfg.PoolHTTPRatio,
PoolMinPerProtocol: cfg.PoolMinPerProtocol,
MaxLatencyMs: cfg.MaxLatencyMs,
MaxLatencyEmergency: cfg.MaxLatencyEmergency,
MaxLatencyHealthy: cfg.MaxLatencyHealthy,
ValidateConcurrency: cfg.ValidateConcurrency,
ValidateTimeout: cfg.ValidateTimeout,
HealthCheckInterval: cfg.HealthCheckInterval,
HealthCheckBatchSize: cfg.HealthCheckBatchSize,
OptimizeInterval: cfg.OptimizeInterval,
ReplaceThreshold: cfg.ReplaceThreshold,
FetchInterval: cfg.FetchInterval,
CheckInterval: cfg.CheckInterval,
}, "", " ")
if err != nil {
return err
}
return os.WriteFile(ConfigFile(), data, 0644)
}
// CalculateSlots 根据配置计算各协议的槽位数
func (c *Config) CalculateSlots() (httpSlots, socks5Slots int) {
httpSlots = int(float64(c.PoolMaxSize) * c.PoolHTTPRatio)
socks5Slots = c.PoolMaxSize - httpSlots
// 保证最小值
if httpSlots < c.PoolMinPerProtocol {
httpSlots = c.PoolMinPerProtocol
}
if socks5Slots < c.PoolMinPerProtocol {
socks5Slots = c.PoolMinPerProtocol
}
return
}
// GetLatencyThreshold 根据池子状态返回合适的延迟阈值
func (c *Config) GetLatencyThreshold(poolStatus string) int {
switch poolStatus {
case "emergency":
return c.MaxLatencyEmergency
case "critical":
return c.MaxLatencyEmergency
case "warning":
return c.MaxLatencyMs
case "healthy":
return c.MaxLatencyHealthy
default:
return c.MaxLatencyMs
}
}

View File

@@ -4,7 +4,8 @@ services:
container_name: proxygo
restart: unless-stopped
ports:
- "127.0.0.1:7777:7777" # HTTP 代理端口(仅内网
- "127.0.0.1:7776:7776" # 稳定代理端口(最低延迟
- "127.0.0.1:7777:7777" # 随机代理端口(轮换模式)
- "7778:7778" # WebUI 端口(外网可访问)
volumes:
- ./data:/app/data

View File

@@ -9,7 +9,7 @@ import (
"strings"
"time"
"proxy-pool/storage"
"goproxy/storage"
)
// 代理来源定义
@@ -18,27 +18,171 @@ type Source struct {
Protocol string // http 或 socks5
}
// 内置多个免费代理来源
var defaultSources = []Source{
// 快速更新源5-30分钟更新- 用于紧急和补充模式
var fastUpdateSources = []Source{
// proxifly - 每5分钟更新
{"https://cdn.jsdelivr.net/gh/proxifly/free-proxy-list@main/proxies/http/data.txt", "http"},
{"https://cdn.jsdelivr.net/gh/proxifly/free-proxy-list@main/proxies/socks4/data.txt", "socks5"},
{"https://cdn.jsdelivr.net/gh/proxifly/free-proxy-list@main/proxies/socks5/data.txt", "socks5"},
// ProxyScraper - 每30分钟更新
{"https://raw.githubusercontent.com/ProxyScraper/ProxyScraper/main/http.txt", "http"},
{"https://raw.githubusercontent.com/ProxyScraper/ProxyScraper/main/socks4.txt", "socks5"},
{"https://raw.githubusercontent.com/ProxyScraper/ProxyScraper/main/socks5.txt", "socks5"},
// monosans - 每小时更新
{"https://raw.githubusercontent.com/monosans/proxy-list/main/proxies/http.txt", "http"},
}
// 慢速更新源(每天更新)- 用于优化轮换模式
var slowUpdateSources = []Source{
// TheSpeedX - 每天更新
{"https://raw.githubusercontent.com/TheSpeedX/SOCKS-List/master/http.txt", "http"},
{"https://raw.githubusercontent.com/TheSpeedX/SOCKS-List/master/socks4.txt", "socks5"},
{"https://raw.githubusercontent.com/TheSpeedX/SOCKS-List/master/socks5.txt", "socks5"},
// monosans SOCKS
{"https://raw.githubusercontent.com/monosans/proxy-list/main/proxies/socks4.txt", "socks5"},
{"https://raw.githubusercontent.com/monosans/proxy-list/main/proxies/socks5.txt", "socks5"},
// databay-labs - 备用源
{"https://cdn.jsdelivr.net/gh/databay-labs/free-proxy-list/http.txt", "http"},
{"https://cdn.jsdelivr.net/gh/databay-labs/free-proxy-list/socks5.txt", "socks5"},
{"https://cdn.jsdelivr.net/gh/proxifly/free-proxy-list@main/proxies/all/data.txt", ""},
}
// 所有源
var allSources = append(fastUpdateSources, slowUpdateSources...)
type Fetcher struct {
sources []Source
client *http.Client
sources []Source
client *http.Client
sourceManager *SourceManager
}
func New(httpURL, socks5URL string) *Fetcher {
func New(httpURL, socks5URL string, sourceManager *SourceManager) *Fetcher {
return &Fetcher{
sources: defaultSources,
sources: allSources,
sourceManager: sourceManager,
client: &http.Client{
Timeout: 30 * time.Second,
},
}
}
// FetchSmart 智能抓取:根据模式和协议需求选择源
func (f *Fetcher) FetchSmart(mode string, preferredProtocol string) ([]storage.Proxy, error) {
var sources []Source
switch mode {
case "emergency":
// 紧急模式:使用所有可用源
sources = f.filterAvailableSources(allSources, preferredProtocol)
log.Printf("[fetch] 🚨 紧急模式: 使用 %d 个源", len(sources))
case "refill":
// 补充模式:使用快更新源
sources = f.filterAvailableSources(fastUpdateSources, preferredProtocol)
log.Printf("[fetch] 🔄 补充模式: 使用 %d 个快更新源", len(sources))
case "optimize":
// 优化模式随机选择2-3个慢更新源
sources = f.selectRandomSources(slowUpdateSources, 3, preferredProtocol)
log.Printf("[fetch] ⚡ 优化模式: 使用 %d 个源", len(sources))
default:
sources = f.filterAvailableSources(fastUpdateSources, preferredProtocol)
}
if len(sources) == 0 {
return nil, fmt.Errorf("no available sources")
}
return f.fetchFromSources(sources)
}
// filterAvailableSources 过滤可用的源(通过断路器)
func (f *Fetcher) filterAvailableSources(sources []Source, preferredProtocol string) []Source {
var available []Source
for _, src := range sources {
// 检查断路器
if f.sourceManager != nil && !f.sourceManager.CanUseSource(src.URL) {
continue
}
// 如果指定了协议偏好,优先该协议的源
if preferredProtocol != "" && src.Protocol != "" && src.Protocol != preferredProtocol {
continue
}
available = append(available, src)
}
return available
}
// selectRandomSources 随机选择N个源
func (f *Fetcher) selectRandomSources(sources []Source, count int, preferredProtocol string) []Source {
available := f.filterAvailableSources(sources, preferredProtocol)
if len(available) <= count {
return available
}
// 随机打乱
shuffled := make([]Source, len(available))
copy(shuffled, available)
for i := range shuffled {
j := i + int(time.Now().UnixNano())%(len(shuffled)-i)
shuffled[i], shuffled[j] = shuffled[j], shuffled[i]
}
return shuffled[:count]
}
// fetchFromSources 从指定源列表抓取
func (f *Fetcher) fetchFromSources(sources []Source) ([]storage.Proxy, error) {
type result struct {
proxies []storage.Proxy
source Source
err error
}
ch := make(chan result, len(sources))
for _, src := range sources {
go func(s Source) {
proxies, err := f.fetchFromURL(s.URL, s.Protocol)
ch <- result{proxies: proxies, source: s, err: err}
}(src)
}
var all []storage.Proxy
seen := make(map[string]bool)
for range sources {
r := <-ch
if r.err != nil {
log.Printf("[fetch] ❌ %s error: %v", r.source.URL, r.err)
if f.sourceManager != nil {
f.sourceManager.RecordFail(r.source.URL, 3, 5, 30)
}
continue
}
// 记录成功
if f.sourceManager != nil {
f.sourceManager.RecordSuccess(r.source.URL)
}
// 去重
var deduped []storage.Proxy
for _, p := range r.proxies {
if !seen[p.Address] {
seen[p.Address] = true
deduped = append(deduped, p)
}
}
log.Printf("[fetch] ✅ %d 个 %s 代理 from %s", len(deduped), r.source.Protocol, r.source.URL)
all = append(all, deduped...)
}
if len(all) == 0 {
return nil, fmt.Errorf("no proxies fetched")
}
log.Printf("[fetch] 总共抓取: %d 个代理(去重后)", len(all))
return all, nil
}
// Fetch 从所有来源并发抓取代理
func (f *Fetcher) Fetch() ([]storage.Proxy, error) {
type result struct {

152
fetcher/ip_query.go Normal file
View File

@@ -0,0 +1,152 @@
package fetcher
import (
"context"
"encoding/json"
"fmt"
"net/http"
"time"
"golang.org/x/time/rate"
)
// IPQueryLimiter 全局IP查询限流器
var IPQueryLimiter *rate.Limiter
// InitIPQueryLimiter 初始化限流器
func InitIPQueryLimiter(rps int) {
IPQueryLimiter = rate.NewLimiter(rate.Limit(rps), rps*2)
}
// GetExitIPInfo 通过代理获取出口 IP 和地理位置(多源降级)
func GetExitIPInfo(client *http.Client) (string, string) {
// 等待限流令牌
if IPQueryLimiter != nil {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := IPQueryLimiter.Wait(ctx); err != nil {
return "", ""
}
}
// 优先级1ip-api.com
if ip, loc := tryIPAPI(client); ip != "" {
return ip, loc
}
// 优先级2ipapi.co
if ip, loc := tryIPAPICo(client); ip != "" {
return ip, loc
}
// 优先级3ipinfo.io
if ip, loc := tryIPInfo(client); ip != "" {
return ip, loc
}
// 优先级4仅获取IP
if ip := tryHTTPBinIP(client); ip != "" {
return ip, "UNKNOWN"
}
return "", ""
}
// tryIPAPI 尝试 ip-api.com
func tryIPAPI(client *http.Client) (string, string) {
resp, err := client.Get("http://ip-api.com/json/?fields=status,country,countryCode,city,query")
if err != nil {
return "", ""
}
defer resp.Body.Close()
var result struct {
Status string `json:"status"`
Query string `json:"query"`
Country string `json:"country"`
CountryCode string `json:"countryCode"`
City string `json:"city"`
}
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil || result.Status != "success" {
return "", ""
}
location := result.CountryCode
if result.City != "" {
location = fmt.Sprintf("%s %s", result.CountryCode, result.City)
}
return result.Query, location
}
// tryIPAPICo 尝试 ipapi.co
func tryIPAPICo(client *http.Client) (string, string) {
resp, err := client.Get("https://ipapi.co/json/")
if err != nil {
return "", ""
}
defer resp.Body.Close()
var result struct {
IP string `json:"ip"`
City string `json:"city"`
CountryCode string `json:"country_code"`
}
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
return "", ""
}
location := result.CountryCode
if result.City != "" {
location = fmt.Sprintf("%s %s", result.CountryCode, result.City)
}
return result.IP, location
}
// tryIPInfo 尝试 ipinfo.io
func tryIPInfo(client *http.Client) (string, string) {
resp, err := client.Get("https://ipinfo.io/json")
if err != nil {
return "", ""
}
defer resp.Body.Close()
var result struct {
IP string `json:"ip"`
City string `json:"city"`
Country string `json:"country"`
}
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
return "", ""
}
location := result.Country
if result.City != "" {
location = fmt.Sprintf("%s %s", result.Country, result.City)
}
return result.IP, location
}
// tryHTTPBinIP 尝试 httpbin仅获取IP
func tryHTTPBinIP(client *http.Client) string {
resp, err := client.Get("https://httpbin.org/ip")
if err != nil {
return ""
}
defer resp.Body.Close()
var result struct {
Origin string `json:"origin"`
}
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
return ""
}
return result.Origin
}

132
fetcher/source_manager.go Normal file
View File

@@ -0,0 +1,132 @@
package fetcher
import (
"database/sql"
"log"
"sync"
"time"
)
// SourceManager 代理源管理器(断路器)
type SourceManager struct {
db *sql.DB
mu sync.RWMutex
}
func NewSourceManager(db *sql.DB) *SourceManager {
return &SourceManager{db: db}
}
// CanUseSource 判断源是否可用
func (sm *SourceManager) CanUseSource(url string) bool {
sm.mu.RLock()
defer sm.mu.RUnlock()
var status string
var disabledUntil sql.NullTime
err := sm.db.QueryRow(
`SELECT status, disabled_until FROM source_status WHERE url = ?`,
url,
).Scan(&status, &disabledUntil)
// 源不存在,默认可用
if err != nil {
return true
}
// 检查是否被禁用且还在冷却期
if status == "disabled" && disabledUntil.Valid {
if time.Now().Before(disabledUntil.Time) {
return false
}
// 冷却期结束,重置状态
sm.db.Exec(`UPDATE source_status SET status = 'active', consecutive_fails = 0 WHERE url = ?`, url)
return true
}
return status != "disabled"
}
// RecordSuccess 记录源抓取成功
func (sm *SourceManager) RecordSuccess(url string) {
sm.mu.Lock()
defer sm.mu.Unlock()
sm.db.Exec(`
INSERT INTO source_status (url, success_count, consecutive_fails, last_success, status)
VALUES (?, 1, 0, CURRENT_TIMESTAMP, 'active')
ON CONFLICT(url) DO UPDATE SET
success_count = success_count + 1,
consecutive_fails = 0,
last_success = CURRENT_TIMESTAMP,
status = 'active'
`, url)
}
// RecordFail 记录源抓取失败
func (sm *SourceManager) RecordFail(url string, failThreshold, disableThreshold, cooldownMinutes int) {
sm.mu.Lock()
defer sm.mu.Unlock()
// 增加失败计数
sm.db.Exec(`
INSERT INTO source_status (url, fail_count, consecutive_fails, last_fail)
VALUES (?, 1, 1, CURRENT_TIMESTAMP)
ON CONFLICT(url) DO UPDATE SET
fail_count = fail_count + 1,
consecutive_fails = consecutive_fails + 1,
last_fail = CURRENT_TIMESTAMP
`, url)
// 检查是否需要降级或禁用
var consecutiveFails int
sm.db.QueryRow(`SELECT consecutive_fails FROM source_status WHERE url = ?`, url).Scan(&consecutiveFails)
if consecutiveFails >= disableThreshold {
// 禁用源
disabledUntil := time.Now().Add(time.Duration(cooldownMinutes) * time.Minute)
sm.db.Exec(
`UPDATE source_status SET status = 'disabled', disabled_until = ? WHERE url = ?`,
disabledUntil, url,
)
log.Printf("[source] ⛔ 禁用源(连续失败%d次: %s (冷却%d分钟)", consecutiveFails, url, cooldownMinutes)
} else if consecutiveFails >= failThreshold {
// 降级源
sm.db.Exec(`UPDATE source_status SET status = 'degraded' WHERE url = ?`, url)
log.Printf("[source] ⚠️ 降级源(连续失败%d次: %s", consecutiveFails, url)
}
}
// GetSourceStats 获取所有源的统计信息
func (sm *SourceManager) GetSourceStats() ([]map[string]interface{}, error) {
rows, err := sm.db.Query(`
SELECT url, success_count, fail_count, consecutive_fails,
last_success, last_fail, status
FROM source_status
ORDER BY success_count DESC
`)
if err != nil {
return nil, err
}
defer rows.Close()
var stats []map[string]interface{}
for rows.Next() {
var url, status string
var successCount, failCount, consecutiveFails int
var lastSuccess, lastFail sql.NullTime
rows.Scan(&url, &successCount, &failCount, &consecutiveFails, &lastSuccess, &lastFail, &status)
stats = append(stats, map[string]interface{}{
"url": url,
"success_count": successCount,
"fail_count": failCount,
"consecutive_fails": consecutiveFails,
"last_success": lastSuccess,
"last_fail": lastFail,
"status": status,
})
}
return stats, nil
}

4
go.mod
View File

@@ -1,4 +1,4 @@
module proxy-pool
module goproxy
go 1.25.0
@@ -6,3 +6,5 @@ require (
github.com/mattn/go-sqlite3 v1.14.37
golang.org/x/net v0.38.0
)
require golang.org/x/time v0.15.0

2
go.sum
View File

@@ -2,3 +2,5 @@ github.com/mattn/go-sqlite3 v1.14.37 h1:3DOZp4cXis1cUIpCfXLtmlGolNLp2VEqhiB/PARN
github.com/mattn/go-sqlite3 v1.14.37/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
golang.org/x/net v0.38.0 h1:vRMAPTMaeGqVhG5QyLJHqNDwecKTomGeqbnfZyKlBI8=
golang.org/x/net v0.38.0/go.mod h1:ivrbrMbzFq5J41QOQh0siUuly180yBYtLp+CKbEaFx8=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=

View File

@@ -25,8 +25,8 @@ type writer struct{}
func (w *writer) Write(p []byte) (n int, err error) {
line := strings.TrimRight(string(p), "\n")
ts := time.Now().Format("2006/01/02 15:04:05")
formatted := fmt.Sprintf("%s %s", ts, line)
ts := time.Now().Format("15:04:05")
formatted := fmt.Sprintf("[%s] %s", ts, line)
mu.Lock()
lines = append(lines, formatted)

263
main.go
View File

@@ -7,14 +7,16 @@ import (
"sync/atomic"
"time"
"proxy-pool/checker"
"proxy-pool/config"
"proxy-pool/fetcher"
"proxy-pool/logger"
"proxy-pool/proxy"
"proxy-pool/storage"
"proxy-pool/validator"
"proxy-pool/webui"
"goproxy/checker"
"goproxy/config"
"goproxy/fetcher"
"goproxy/logger"
"goproxy/optimizer"
"goproxy/pool"
"goproxy/proxy"
"goproxy/storage"
"goproxy/validator"
"goproxy/webui"
)
var fetchRunning atomic.Bool
@@ -24,7 +26,7 @@ func main() {
// 初始化日志收集器
logger.Init()
// 加载配置(优先读取 config.json
// 加载配置
cfg := config.Load()
// 提示密码信息
@@ -34,6 +36,9 @@ func main() {
log.Println("[main] WebUI 密码已通过环境变量 WEBUI_PASSWORD 设置")
}
log.Printf("[main] 🎯 智能代理池配置: 容量=%d HTTP=%.0f%% SOCKS5=%.0f%% 延迟标准=%dms",
cfg.PoolMaxSize, cfg.PoolHTTPRatio*100, (1-cfg.PoolHTTPRatio)*100, cfg.MaxLatencyMs)
// 初始化存储
store, err := storage.New(cfg.DBPath)
if err != nil {
@@ -41,102 +46,210 @@ func main() {
}
defer store.Close()
// 初始化模块
fetch := fetcher.New(cfg.HTTPSourceURL, cfg.SOCKS5SourceURL)
check := checker.New(store, validator.New(cfg.ValidateConcurrency, cfg.ValidateTimeout, cfg.ValidateURL), cfg)
server := proxy.New(store, cfg)
// 初始化限流器
fetcher.InitIPQueryLimiter(cfg.IPQueryRateLimit)
// 初始化核心模块
sourceMgr := fetcher.NewSourceManager(store.GetDB())
fetch := fetcher.New(cfg.HTTPSourceURL, cfg.SOCKS5SourceURL, sourceMgr)
validate := validator.New(cfg.ValidateConcurrency, cfg.ValidateTimeout, cfg.ValidateURL)
poolMgr := pool.NewManager(store, cfg)
healthChecker := checker.NewHealthChecker(store, validate, cfg, poolMgr)
opt := optimizer.NewOptimizer(store, fetch, validate, poolMgr, cfg)
// 清理无效代理
totalDeleted := 0
if deleted, err := store.DeleteChinaMainland(); err == nil && deleted > 0 {
log.Printf("[main] 🧹 已清理 %d 个中国大陆出口代理", deleted)
totalDeleted += int(deleted)
}
if deleted, err := store.DeleteWithoutExitInfo(); err == nil && deleted > 0 {
log.Printf("[main] 🧹 已清理 %d 个无出口信息的代理", deleted)
totalDeleted += int(deleted)
}
// 创建两个代理服务器:随机轮换 + 最低延迟
randomServer := proxy.New(store, cfg, "random", cfg.ProxyPort)
stableServer := proxy.New(store, cfg, "lowest-latency", cfg.StableProxyPort)
// 配置变更通知 channel
configChanged := make(chan struct{}, 1)
// 启动 WebUI
ui := webui.New(store, cfg, func() {
go func() {
if err := fetchAndValidate(fetch, store); err != nil {
log.Printf("[webui] fetch error: %v", err)
}
}()
// 启动 WebUI(传递池子管理器)
ui := webui.New(store, cfg, poolMgr, func() {
go smartFetchAndFill(fetch, validate, store, poolMgr)
}, configChanged)
ui.Start()
// 后台启动:首次抓取验证
// 首次智能填充(清理后立即触发)
go func() {
log.Println("[main] fetching proxies on startup...")
if err := fetchAndValidate(fetch, store); err != nil {
log.Printf("[main] initial fetch error: %v", err)
if totalDeleted > 0 {
log.Printf("[main] 🚀 清理后立即启动补充填充...")
} else {
log.Println("[main] 🚀 启动初始化填充...")
}
smartFetchAndFill(fetch, validate, store, poolMgr)
}()
// 启动状态监控协程
go startStatusMonitor(poolMgr, fetch, validate, store)
// 启动健康检查器
healthChecker.StartBackground()
// 启动优化轮换器
opt.StartBackground()
// 监听配置变更
go watchConfigChanges(configChanged, poolMgr)
// 启动稳定代理服务(最低延迟模式)
go func() {
if err := stableServer.Start(); err != nil {
log.Fatalf("stable proxy server: %v", err)
}
}()
// 启动动态定时抓取
go startFetchLoop(fetch, store, configChanged)
// 启动定时健康检查
check.Start()
// 启动代理服务(阻塞)
if err := server.Start(); err != nil {
log.Fatalf("proxy server: %v", err)
// 启动随机代理服务(阻塞)
if err := randomServer.Start(); err != nil {
log.Fatalf("random proxy server: %v", err)
}
}
func fetchAndValidate(fetch *fetcher.Fetcher, store *storage.Storage) error {
// smartFetchAndFill 智能抓取和填充
func smartFetchAndFill(fetch *fetcher.Fetcher, validate *validator.Validator, store *storage.Storage, poolMgr *pool.Manager) {
// 防止并发执行
if !fetchRunning.CompareAndSwap(false, true) {
log.Println("[main] fetch already running, skipping")
return nil
log.Println("[main] 抓取已在运行,跳过")
return
}
defer fetchRunning.Store(false)
proxies, err := fetch.Fetch()
// 获取池子状态
status, err := poolMgr.GetStatus()
if err != nil {
return err
log.Printf("[main] 获取池子状态失败: %v", err)
return
}
log.Printf("[main] validating %d proxies (streaming)...", len(proxies))
// 每次用最新配置创建 validator
cfg := config.Get()
validate := validator.New(cfg.ValidateConcurrency, cfg.ValidateTimeout, cfg.ValidateURL)
log.Printf("[main] 📊 池子状态: %s | HTTP=%d/%d SOCKS5=%d/%d 总计=%d/%d",
status.State, status.HTTP, status.HTTPSlots, status.SOCKS5, status.SOCKS5Slots,
status.Total, config.Get().PoolMaxSize)
var valid, total int
for r := range validate.ValidateStream(proxies) {
total++
if total%1000 == 0 {
log.Printf("[main] checked=%d/%d valid=%d", total, len(proxies), valid)
// 判断是否需要抓取
needFetch, mode, preferredProtocol := poolMgr.NeedsFetch(status)
if !needFetch {
log.Println("[main] 池子健康,无需抓取")
return
}
log.Printf("[main] 🔍 智能抓取: 模式=%s 协议偏好=%s", mode, preferredProtocol)
// 智能抓取
candidates, err := fetch.FetchSmart(mode, preferredProtocol)
if err != nil {
log.Printf("[main] 抓取失败: %v", err)
return
}
log.Printf("[main] 抓取到 %d 个候选代理,开始严格验证...", len(candidates))
// 严格验证并尝试入池
addedCount := 0
validCount := 0
for result := range validate.ValidateStream(candidates) {
if !result.Valid {
continue
}
if r.Valid {
if valid == 0 {
log.Printf("[main] first valid proxy: %s (%s) latency=%v", r.Proxy.Address, r.Proxy.Protocol, r.Latency)
}
valid++
if err := store.AddProxy(r.Proxy.Address, r.Proxy.Protocol); err != nil {
log.Printf("[main] addProxy error: %v", err)
}
if valid%10 == 0 {
log.Printf("[main] progress: valid=%d checked=%d/%d", valid, total, len(proxies))
validCount++
latencyMs := int(result.Latency.Milliseconds())
// 根据池子状态动态调整延迟标准
cfg := config.Get()
maxLatency := cfg.GetLatencyThreshold(status.State)
// 必须满足有出口IP、有位置、延迟达标
if result.ExitIP == "" || result.ExitLocation == "" || latencyMs > maxLatency {
continue
}
// 尝试加入池子
proxyToAdd := storage.Proxy{
Address: result.Proxy.Address,
Protocol: result.Proxy.Protocol,
ExitIP: result.ExitIP,
ExitLocation: result.ExitLocation,
Latency: latencyMs,
}
if added, _ := poolMgr.TryAddProxy(proxyToAdd); added {
addedCount++
}
// 如果是紧急模式且已达到最小要求,停止验证
if mode == "emergency" && status.HTTP >= cfg.PoolMinPerProtocol && status.SOCKS5 >= cfg.PoolMinPerProtocol {
log.Println("[main] 🎉 紧急模式:达到最小要求,停止验证")
break
}
// 动态检查是否已经填满
if addedCount > 0 && addedCount%20 == 0 {
currentStatus, _ := poolMgr.GetStatus()
if !poolMgr.NeedsFetchQuick(currentStatus) {
log.Println("[main] ✅ 池子已填满,停止验证")
break
}
}
}
count, _ := store.Count()
log.Printf("[main] validated: valid=%d/%d, pool size=%d", valid, len(proxies), count)
return nil
// 最终状态
finalStatus, _ := poolMgr.GetStatus()
log.Printf("[main] 填充完成: 验证%d 通过%d 入池%d | 最终状态: %s HTTP=%d SOCKS5=%d",
len(candidates), validCount, addedCount,
finalStatus.State, finalStatus.HTTP, finalStatus.SOCKS5)
}
func startFetchLoop(fetch *fetcher.Fetcher, store *storage.Storage, configChanged <-chan struct{}) {
cfg := config.Get()
ticker := time.NewTicker(time.Duration(cfg.FetchInterval) * time.Minute)
log.Printf("[main] fetch loop started, interval: %d min", cfg.FetchInterval)
for {
select {
case <-ticker.C:
log.Println("[main] scheduled fetch started...")
if err := fetchAndValidate(fetch, store); err != nil {
log.Printf("[main] scheduled fetch error: %v", err)
}
case <-configChanged:
newCfg := config.Get()
ticker.Reset(time.Duration(newCfg.FetchInterval) * time.Minute)
log.Printf("[main] fetch interval updated to %d min", newCfg.FetchInterval)
// startStatusMonitor 状态监控协程
func startStatusMonitor(poolMgr *pool.Manager, fetch *fetcher.Fetcher, validate *validator.Validator, store *storage.Storage) {
ticker := time.NewTicker(30 * time.Second)
log.Println("[monitor] 📡 状态监控器已启动每30秒检查")
for range ticker.C {
status, err := poolMgr.GetStatus()
if err != nil {
continue
}
// 每分钟检查池子状态
needFetch, mode, preferredProtocol := poolMgr.NeedsFetch(status)
if needFetch {
log.Printf("[monitor] ⚠️ 检测到池子需求: 状态=%s 模式=%s 协议=%s",
status.State, mode, preferredProtocol)
// 触发智能填充
go smartFetchAndFill(fetch, validate, store, poolMgr)
}
}
}
// watchConfigChanges 监听配置变更
func watchConfigChanges(configChanged <-chan struct{}, poolMgr *pool.Manager) {
var oldSize int
var oldRatio float64
cfg := config.Get()
oldSize = cfg.PoolMaxSize
oldRatio = cfg.PoolHTTPRatio
for range configChanged {
newCfg := config.Get()
if newCfg.PoolMaxSize != oldSize || newCfg.PoolHTTPRatio != oldRatio {
log.Printf("[config] 🔧 配置变更检测: 容量 %d→%d 比例 %.2f→%.2f",
oldSize, newCfg.PoolMaxSize, oldRatio, newCfg.PoolHTTPRatio)
poolMgr.AdjustForConfigChange(oldSize, oldRatio)
oldSize = newCfg.PoolMaxSize
oldRatio = newCfg.PoolHTTPRatio
}
}
}

108
optimizer/optimizer.go Normal file
View File

@@ -0,0 +1,108 @@
package optimizer
import (
"log"
"time"
"goproxy/config"
"goproxy/fetcher"
"goproxy/pool"
"goproxy/storage"
"goproxy/validator"
)
// Optimizer 优化轮换器
type Optimizer struct {
storage *storage.Storage
fetcher *fetcher.Fetcher
validator *validator.Validator
poolMgr *pool.Manager
cfg *config.Config
}
func NewOptimizer(s *storage.Storage, f *fetcher.Fetcher, v *validator.Validator, pm *pool.Manager, cfg *config.Config) *Optimizer {
return &Optimizer{
storage: s,
fetcher: f,
validator: v,
poolMgr: pm,
cfg: cfg,
}
}
// RunOnce 执行一次优化轮换
func (o *Optimizer) RunOnce() {
start := time.Now()
log.Println("[optimize] 🎯 开始优化轮换...")
// 获取池子状态
status, err := o.poolMgr.GetStatus()
if err != nil {
log.Printf("[optimize] 获取状态失败: %v", err)
return
}
// 只有健康状态才执行优化
if status.State != "healthy" {
log.Printf("[optimize] 池子状态 %s跳过优化", status.State)
return
}
// 抓取新的候选代理(优化模式)
log.Println("[optimize] 抓取新候选代理...")
candidates, err := o.fetcher.FetchSmart("optimize", "")
if err != nil {
log.Printf("[optimize] 抓取失败: %v", err)
return
}
log.Printf("[optimize] 抓取到 %d 个候选代理", len(candidates))
// 验证候选代理
validCandidates := []storage.Proxy{}
for result := range o.validator.ValidateStream(candidates) {
if result.Valid {
latencyMs := int(result.Latency.Milliseconds())
// 只保留延迟在健康标准内的
if latencyMs <= o.cfg.MaxLatencyHealthy {
validCandidates = append(validCandidates, storage.Proxy{
Address: result.Proxy.Address,
Protocol: result.Proxy.Protocol,
ExitIP: result.ExitIP,
ExitLocation: result.ExitLocation,
Latency: latencyMs,
})
}
}
}
log.Printf("[optimize] 验证通过 %d 个优质候选(延迟<%dms", len(validCandidates), o.cfg.MaxLatencyHealthy)
if len(validCandidates) == 0 {
log.Println("[optimize] 无优质候选,跳过优化")
return
}
// 尝试用优质候选替换延迟高的代理
replacedCount := 0
for _, candidate := range validCandidates {
added, reason := o.poolMgr.TryAddProxy(candidate)
if added && reason == "replaced" {
replacedCount++
}
}
elapsed := time.Since(start)
log.Printf("[optimize] ✅ 完成: 替换 %d 个代理 耗时%v", replacedCount, elapsed)
}
// StartBackground 后台定时优化
func (o *Optimizer) StartBackground() {
ticker := time.NewTicker(time.Duration(o.cfg.OptimizeInterval) * time.Minute)
go func() {
for range ticker.C {
o.RunOnce()
}
}()
log.Printf("[optimize] 优化轮换器已启动,间隔 %d 分钟", o.cfg.OptimizeInterval)
}

229
pool/manager.go Normal file
View File

@@ -0,0 +1,229 @@
package pool
import (
"log"
"goproxy/config"
"goproxy/storage"
)
// Manager 池子管理器
type Manager struct {
storage *storage.Storage
cfg *config.Config
}
func NewManager(s *storage.Storage, cfg *config.Config) *Manager {
return &Manager{
storage: s,
cfg: cfg,
}
}
// PoolStatus 池子状态
type PoolStatus struct {
Total int
HTTP int
SOCKS5 int
HTTPSlots int
SOCKS5Slots int
State string // healthy/warning/critical/emergency
AvgLatencyHTTP int
AvgLatencySocks5 int
}
// GetStatus 获取当前池子状态
func (m *Manager) GetStatus() (*PoolStatus, error) {
total, _ := m.storage.Count()
httpCount, _ := m.storage.CountByProtocol("http")
socks5Count, _ := m.storage.CountByProtocol("socks5")
httpSlots, socks5Slots := m.cfg.CalculateSlots()
// 计算平均延迟
avgHTTP, _ := m.storage.GetAverageLatency("http")
avgSOCKS5, _ := m.storage.GetAverageLatency("socks5")
// 判断状态
state := m.determineState(total, httpCount, socks5Count)
return &PoolStatus{
Total: total,
HTTP: httpCount,
SOCKS5: socks5Count,
HTTPSlots: httpSlots,
SOCKS5Slots: socks5Slots,
State: state,
AvgLatencyHTTP: avgHTTP,
AvgLatencySocks5: avgSOCKS5,
}, nil
}
// determineState 判断池子状态
func (m *Manager) determineState(total, httpCount, socks5Count int) string {
httpSlots, socks5Slots := m.cfg.CalculateSlots()
// 单协议缺失
if httpCount == 0 || socks5Count == 0 {
return "emergency"
}
// 紧急:总数<10%
emergencyThreshold := int(float64(m.cfg.PoolMaxSize) * 0.1)
if total < emergencyThreshold {
return "emergency"
}
// 危急:任一协议<20%槽位
if httpCount < int(float64(httpSlots)*0.2) || socks5Count < int(float64(socks5Slots)*0.2) {
return "critical"
}
// 警告:总数<95%
healthyThreshold := int(float64(m.cfg.PoolMaxSize) * 0.95)
if total < healthyThreshold {
return "warning"
}
// 健康
return "healthy"
}
// NeedsFetch 判断是否需要抓取以及抓取模式
func (m *Manager) NeedsFetch(status *PoolStatus) (bool, string, string) {
// 单协议缺失:紧急模式,指定协议
if status.HTTP == 0 {
return true, "emergency", "http"
}
if status.SOCKS5 == 0 {
return true, "emergency", "socks5"
}
// 紧急状态:紧急模式
if status.State == "emergency" {
return true, "emergency", ""
}
// 危急或警告:补充模式
if status.State == "critical" || status.State == "warning" {
// 判断哪个协议更缺
httpPct := float64(status.HTTP) / float64(status.HTTPSlots)
socks5Pct := float64(status.SOCKS5) / float64(status.SOCKS5Slots)
if httpPct < 0.5 {
return true, "refill", "http"
}
if socks5Pct < 0.5 {
return true, "refill", "socks5"
}
return true, "refill", ""
}
// 健康状态:不需要补充抓取
return false, "", ""
}
// NeedsFetchQuick 快速判断是否还需要抓取(用于提前终止)
func (m *Manager) NeedsFetchQuick(status *PoolStatus) bool {
need, _, _ := m.NeedsFetch(status)
return need
}
// TryAddProxy 尝试将代理加入池子
func (m *Manager) TryAddProxy(p storage.Proxy) (bool, string) {
httpSlots, socks5Slots := m.cfg.CalculateSlots()
httpCount, _ := m.storage.CountByProtocol("http")
socks5Count, _ := m.storage.CountByProtocol("socks5")
total, _ := m.storage.Count()
var maxSlots int
var currentCount int
if p.Protocol == "http" {
maxSlots = httpSlots
currentCount = httpCount
} else {
maxSlots = socks5Slots
currentCount = socks5Count
}
// 情况1该协议槽位未满直接加入
if currentCount < maxSlots {
if err := m.storage.AddProxy(p.Address, p.Protocol); err != nil {
return false, "db_error"
}
// 更新完整信息
m.storage.UpdateExitInfo(p.Address, p.ExitIP, p.ExitLocation, p.Latency)
log.Printf("[pool] ✅ 直接入池: %s (%s %d/%d) %dms %s %s",
p.Address, p.Protocol, currentCount+1, maxSlots, p.Latency, p.ExitIP, p.ExitLocation)
return true, "added"
}
// 情况2槽位满但允许10%浮动
allowedFloat := int(float64(maxSlots) * 0.1)
if total < m.cfg.PoolMaxSize && currentCount < maxSlots+allowedFloat {
if err := m.storage.AddProxy(p.Address, p.Protocol); err != nil {
return false, "db_error"
}
m.storage.UpdateExitInfo(p.Address, p.ExitIP, p.ExitLocation, p.Latency)
log.Printf("[pool] ✅ 浮动入池: %s (%s %d/%d+%d) %dms",
p.Address, p.Protocol, currentCount+1, maxSlots, allowedFloat, p.Latency)
return true, "added_float"
}
// 情况3池子满了尝试替换
if currentCount >= maxSlots || total >= m.cfg.PoolMaxSize {
return m.tryReplace(p)
}
return false, "slots_full"
}
// tryReplace 尝试替换现有代理
func (m *Manager) tryReplace(newProxy storage.Proxy) (bool, string) {
// 获取同协议中可替换的代理延迟最高的前10个
candidates, err := m.storage.GetWorstProxies(newProxy.Protocol, 10)
if err != nil || len(candidates) == 0 {
return false, "no_candidates"
}
worst := candidates[0]
// 判断是否值得替换:新代理需要显著更快
threshold := m.cfg.ReplaceThreshold
if float64(newProxy.Latency) < float64(worst.Latency)*threshold {
if err := m.storage.ReplaceProxy(worst.Address, newProxy); err != nil {
return false, "replace_error"
}
log.Printf("[pool] 🔄 替换: %s(%dms) → %s(%dms) 提升%.0f%%",
worst.Address, worst.Latency, newProxy.Address, newProxy.Latency,
(1-float64(newProxy.Latency)/float64(worst.Latency))*100)
return true, "replaced"
}
return false, "not_better"
}
// AdjustForConfigChange 配置变更后调整池子
func (m *Manager) AdjustForConfigChange(oldSize int, oldRatio float64) {
newHTTP, newSOCKS5 := m.cfg.CalculateSlots()
oldHTTP := int(float64(oldSize) * oldRatio)
oldSOCKS5 := oldSize - oldHTTP
log.Printf("[pool] 配置变更: 容量 %d→%d, HTTP槽位 %d→%d, SOCKS5槽位 %d→%d",
oldSize, m.cfg.PoolMaxSize, oldHTTP, newHTTP, oldSOCKS5, newSOCKS5)
// 如果槽位减少且当前超标,标记超标的为替换候选
httpCount, _ := m.storage.CountByProtocol("http")
socks5Count, _ := m.storage.CountByProtocol("socks5")
if httpCount > newHTTP {
excess := httpCount - newHTTP
log.Printf("[pool] HTTP 超标 %d 个,标记为替换候选", excess)
// 这里可以标记延迟高的为候选
}
if socks5Count > newSOCKS5 {
excess := socks5Count - newSOCKS5
log.Printf("[pool] SOCKS5 超标 %d 个,标记为替换候选", excess)
}
}

View File

@@ -10,25 +10,33 @@ import (
"time"
"golang.org/x/net/proxy"
"proxy-pool/config"
"proxy-pool/storage"
"goproxy/config"
"goproxy/storage"
)
type Server struct {
storage *storage.Storage
cfg *config.Config
mode string // "random" 或 "lowest-latency"
port string
}
func New(s *storage.Storage, cfg *config.Config) *Server {
func New(s *storage.Storage, cfg *config.Config, mode string, port string) *Server {
return &Server{
storage: s,
cfg: cfg,
mode: mode,
port: port,
}
}
func (s *Server) Start() error {
log.Printf("proxy server listening on %s", s.cfg.ProxyPort)
return http.ListenAndServe(s.cfg.ProxyPort, s)
modeDesc := "随机轮换"
if s.mode == "lowest-latency" {
modeDesc = "最低延迟"
}
log.Printf("proxy server listening on %s [%s]", s.port, modeDesc)
return http.ListenAndServe(s.port, s)
}
func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
@@ -43,11 +51,22 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
func (s *Server) handleHTTP(w http.ResponseWriter, r *http.Request) {
var tried []string
for attempt := 0; attempt <= s.cfg.MaxRetry; attempt++ {
p, err := s.storage.GetRandomExclude(tried)
var p *storage.Proxy
var err error
// 根据模式选择代理
if s.mode == "lowest-latency" {
p, err = s.storage.GetLowestLatencyExclude(tried)
} else {
p, err = s.storage.GetRandomExclude(tried)
}
if err != nil {
http.Error(w, "no available proxy", http.StatusServiceUnavailable)
return
}
tried = append(tried, p.Address)
client, err := s.buildClient(p)
if err != nil {
@@ -94,11 +113,22 @@ func (s *Server) handleHTTP(w http.ResponseWriter, r *http.Request) {
func (s *Server) handleTunnel(w http.ResponseWriter, r *http.Request) {
var tried []string
for attempt := 0; attempt <= s.cfg.MaxRetry; attempt++ {
p, err := s.storage.GetRandomExclude(tried)
var p *storage.Proxy
var err error
// 根据模式选择代理
if s.mode == "lowest-latency" {
p, err = s.storage.GetLowestLatencyExclude(tried)
} else {
p, err = s.storage.GetRandomExclude(tried)
}
if err != nil {
http.Error(w, "no available proxy", http.StatusServiceUnavailable)
return
}
tried = append(tried, p.Address)
conn, err := s.dialViaProxy(p, r.Host)
if err != nil {

BIN
proxygo Executable file

Binary file not shown.

View File

@@ -11,12 +11,33 @@ import (
)
type Proxy struct {
ID int64
Address string // host:port
Protocol string // http, socks5
FailCount int
LastCheck time.Time
CreatedAt time.Time
ID int64 `json:"id"`
Address string `json:"address"`
Protocol string `json:"protocol"`
ExitIP string `json:"exit_ip"`
ExitLocation string `json:"exit_location"`
Latency int `json:"latency"`
QualityGrade string `json:"quality_grade"`
UseCount int `json:"use_count"`
SuccessCount int `json:"success_count"`
FailCount int `json:"fail_count"`
LastUsed time.Time `json:"last_used"`
LastCheck time.Time `json:"last_check"`
CreatedAt time.Time `json:"created_at"`
Status string `json:"status"`
}
// SourceStatus 代理源状态
type SourceStatus struct {
ID int64
URL string
SuccessCount int
FailCount int
ConsecutiveFails int
LastSuccess time.Time
LastFail time.Time
Status string // active/degraded/disabled
DisabledUntil time.Time
}
type Storage struct {
@@ -39,17 +60,119 @@ func New(dbPath string) (*Storage, error) {
}
func (s *Storage) initSchema() error {
// 创建代理表
_, err := s.db.Exec(`
CREATE TABLE IF NOT EXISTS proxies (
id INTEGER PRIMARY KEY AUTOINCREMENT,
address TEXT NOT NULL UNIQUE,
protocol TEXT NOT NULL,
fail_count INTEGER NOT NULL DEFAULT 0,
last_check DATETIME,
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
id INTEGER PRIMARY KEY AUTOINCREMENT,
address TEXT NOT NULL UNIQUE,
protocol TEXT NOT NULL,
exit_ip TEXT NOT NULL DEFAULT '',
exit_location TEXT NOT NULL DEFAULT '',
latency INTEGER NOT NULL DEFAULT 0,
quality_grade TEXT NOT NULL DEFAULT 'C',
use_count INTEGER NOT NULL DEFAULT 0,
success_count INTEGER NOT NULL DEFAULT 0,
fail_count INTEGER NOT NULL DEFAULT 0,
last_used DATETIME,
last_check DATETIME,
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
status TEXT NOT NULL DEFAULT 'active'
)
`)
return err
if err != nil {
return err
}
// 创建索引
s.db.Exec(`CREATE INDEX IF NOT EXISTS idx_protocol_latency ON proxies(protocol, latency)`)
s.db.Exec(`CREATE INDEX IF NOT EXISTS idx_quality_grade ON proxies(quality_grade, latency)`)
s.db.Exec(`CREATE INDEX IF NOT EXISTS idx_status ON proxies(status)`)
// 创建源状态表
_, err = s.db.Exec(`
CREATE TABLE IF NOT EXISTS source_status (
id INTEGER PRIMARY KEY AUTOINCREMENT,
url TEXT NOT NULL UNIQUE,
success_count INTEGER NOT NULL DEFAULT 0,
fail_count INTEGER NOT NULL DEFAULT 0,
consecutive_fails INTEGER NOT NULL DEFAULT 0,
last_success DATETIME,
last_fail DATETIME,
status TEXT NOT NULL DEFAULT 'active',
disabled_until DATETIME
)
`)
if err != nil {
return err
}
// 迁移:处理旧的 location 字段(如果存在)
var hasOldLocation int
err = s.db.QueryRow(`SELECT COUNT(*) FROM pragma_table_info('proxies') WHERE name='location'`).Scan(&hasOldLocation)
if err == nil && hasOldLocation > 0 {
log.Println("[storage] migrating: renaming location to exit_location")
// 如果有旧的 location 字段,先添加新字段再复制数据
s.db.Exec(`ALTER TABLE proxies ADD COLUMN exit_location TEXT NOT NULL DEFAULT ''`)
s.db.Exec(`UPDATE proxies SET exit_location = location WHERE location != ''`)
}
// 迁移:添加 exit_ip 字段
var hasExitIP int
err = s.db.QueryRow(`SELECT COUNT(*) FROM pragma_table_info('proxies') WHERE name='exit_ip'`).Scan(&hasExitIP)
if err == nil && hasExitIP == 0 {
log.Println("[storage] migrating: adding exit_ip column")
_, err = s.db.Exec(`ALTER TABLE proxies ADD COLUMN exit_ip TEXT NOT NULL DEFAULT ''`)
if err != nil {
return fmt.Errorf("migrate exit_ip column: %w", err)
}
}
// 迁移:添加 exit_location 字段
var hasExitLocation int
err = s.db.QueryRow(`SELECT COUNT(*) FROM pragma_table_info('proxies') WHERE name='exit_location'`).Scan(&hasExitLocation)
if err == nil && hasExitLocation == 0 {
log.Println("[storage] migrating: adding exit_location column")
_, err = s.db.Exec(`ALTER TABLE proxies ADD COLUMN exit_location TEXT NOT NULL DEFAULT ''`)
if err != nil {
return fmt.Errorf("migrate exit_location column: %w", err)
}
}
// 迁移:添加 latency 字段
var hasLatency int
err = s.db.QueryRow(`SELECT COUNT(*) FROM pragma_table_info('proxies') WHERE name='latency'`).Scan(&hasLatency)
if err == nil && hasLatency == 0 {
log.Println("[storage] migrating: adding latency column")
s.db.Exec(`ALTER TABLE proxies ADD COLUMN latency INTEGER NOT NULL DEFAULT 0`)
}
// 迁移:添加质量等级字段
var hasQuality int
s.db.QueryRow(`SELECT COUNT(*) FROM pragma_table_info('proxies') WHERE name='quality_grade'`).Scan(&hasQuality)
if hasQuality == 0 {
log.Println("[storage] migrating: adding quality_grade column")
s.db.Exec(`ALTER TABLE proxies ADD COLUMN quality_grade TEXT NOT NULL DEFAULT 'C'`)
}
// 迁移:添加使用统计字段
var hasUseCount int
s.db.QueryRow(`SELECT COUNT(*) FROM pragma_table_info('proxies') WHERE name='use_count'`).Scan(&hasUseCount)
if hasUseCount == 0 {
log.Println("[storage] migrating: adding usage tracking columns")
s.db.Exec(`ALTER TABLE proxies ADD COLUMN use_count INTEGER NOT NULL DEFAULT 0`)
s.db.Exec(`ALTER TABLE proxies ADD COLUMN success_count INTEGER NOT NULL DEFAULT 0`)
s.db.Exec(`ALTER TABLE proxies ADD COLUMN last_used DATETIME`)
}
// 迁移:添加状态字段
var hasStatus int
s.db.QueryRow(`SELECT COUNT(*) FROM pragma_table_info('proxies') WHERE name='status'`).Scan(&hasStatus)
if hasStatus == 0 {
log.Println("[storage] migrating: adding status column")
s.db.Exec(`ALTER TABLE proxies ADD COLUMN status TEXT NOT NULL DEFAULT 'active'`)
}
return nil
}
// AddProxy 新增代理,已存在则忽略
@@ -82,12 +205,23 @@ func (s *Storage) AddProxies(proxies []Proxy) error {
return tx.Commit()
}
// GetRandom 随机取一个可用代理
// GetRandom 随机取一个可用代理(优先选择质量高的)
func (s *Storage) GetRandom() (*Proxy, error) {
// 优先从 S/A 级代理中随机选择
rows, err := s.db.Query(
`SELECT id, address, protocol, fail_count, last_check, created_at
FROM proxies WHERE fail_count < 3
ORDER BY RANDOM() LIMIT 1`,
`SELECT id, address, protocol, exit_ip, exit_location, latency, quality_grade,
use_count, success_count, fail_count, last_used, last_check, created_at, status
FROM proxies
WHERE status = 'active' AND fail_count < 3
ORDER BY
CASE quality_grade
WHEN 'S' THEN 1
WHEN 'A' THEN 2
WHEN 'B' THEN 3
ELSE 4
END,
RANDOM()
LIMIT 1`,
)
if err != nil {
return nil, err
@@ -95,24 +229,37 @@ func (s *Storage) GetRandom() (*Proxy, error) {
defer rows.Close()
if rows.Next() {
p := &Proxy{}
var lastCheck sql.NullTime
if err := rows.Scan(&p.ID, &p.Address, &p.Protocol, &p.FailCount, &lastCheck, &p.CreatedAt); err != nil {
return nil, err
}
if lastCheck.Valid {
p.LastCheck = lastCheck.Time
}
return p, nil
return scanProxy(rows)
}
return nil, fmt.Errorf("no available proxy")
}
// scanProxy 扫描代理行数据
func scanProxy(rows *sql.Rows) (*Proxy, error) {
p := &Proxy{}
var lastUsed, lastCheck sql.NullTime
if err := rows.Scan(&p.ID, &p.Address, &p.Protocol, &p.ExitIP, &p.ExitLocation,
&p.Latency, &p.QualityGrade, &p.UseCount, &p.SuccessCount, &p.FailCount,
&lastUsed, &lastCheck, &p.CreatedAt, &p.Status); err != nil {
return nil, err
}
if lastUsed.Valid {
p.LastUsed = lastUsed.Time
}
if lastCheck.Valid {
p.LastCheck = lastCheck.Time
}
return p, nil
}
// GetAll 获取所有可用代理
func (s *Storage) GetAll() ([]Proxy, error) {
rows, err := s.db.Query(
`SELECT id, address, protocol, fail_count, last_check, created_at
FROM proxies WHERE fail_count < 3`,
`SELECT id, address, protocol, exit_ip, exit_location, latency, quality_grade,
use_count, success_count, fail_count, last_used, last_check, created_at, status
FROM proxies
WHERE status IN ('active', 'degraded') AND fail_count < 3
ORDER BY latency ASC`,
)
if err != nil {
return nil, err
@@ -121,15 +268,11 @@ func (s *Storage) GetAll() ([]Proxy, error) {
var proxies []Proxy
for rows.Next() {
p := Proxy{}
var lastCheck sql.NullTime
if err := rows.Scan(&p.ID, &p.Address, &p.Protocol, &p.FailCount, &lastCheck, &p.CreatedAt); err != nil {
p, err := scanProxy(rows)
if err != nil {
return nil, err
}
if lastCheck.Valid {
p.LastCheck = lastCheck.Time
}
proxies = append(proxies, p)
proxies = append(proxies, *p)
}
return proxies, nil
}
@@ -162,6 +305,29 @@ func (s *Storage) GetRandomExclude(excludes []string) (*Proxy, error) {
return &p, nil
}
// GetLowestLatencyExclude 排除指定地址后获取延迟最低的代理
func (s *Storage) GetLowestLatencyExclude(excludes []string) (*Proxy, error) {
proxies, err := s.GetAll()
if err != nil {
return nil, err
}
excludeMap := make(map[string]bool)
for _, e := range excludes {
excludeMap[e] = true
}
// GetAll() 已经按 latency ASC 排序,找到第一个不在排除列表中的
for _, p := range proxies {
if !excludeMap[p.Address] {
proxy := p
return &proxy, nil
}
}
return nil, fmt.Errorf("no available proxy")
}
// Delete 立即删除指定代理
func (s *Storage) Delete(address string) error {
_, err := s.db.Exec(`DELETE FROM proxies WHERE address = ?`, address)
@@ -186,6 +352,201 @@ func (s *Storage) ResetFail(address string) error {
return err
}
// UpdateLatency 更新代理的延迟信息(毫秒)
func (s *Storage) UpdateLatency(address string, latencyMs int) error {
_, err := s.db.Exec(
`UPDATE proxies SET latency = ? WHERE address = ?`,
latencyMs, address,
)
return err
}
// UpdateExitInfo 更新代理的出口 IP、位置和质量等级
func (s *Storage) UpdateExitInfo(address, exitIP, exitLocation string, latencyMs int) error {
grade := CalculateQualityGrade(latencyMs)
_, err := s.db.Exec(
`UPDATE proxies SET exit_ip = ?, exit_location = ?, latency = ?, quality_grade = ? WHERE address = ?`,
exitIP, exitLocation, latencyMs, grade, address,
)
return err
}
// RecordProxyUse 记录代理使用(成功)
func (s *Storage) RecordProxyUse(address string, success bool) error {
if success {
_, err := s.db.Exec(
`UPDATE proxies SET use_count = use_count + 1, success_count = success_count + 1,
last_used = CURRENT_TIMESTAMP WHERE address = ?`,
address,
)
return err
}
_, err := s.db.Exec(
`UPDATE proxies SET use_count = use_count + 1, fail_count = fail_count + 1,
last_used = CURRENT_TIMESTAMP WHERE address = ?`,
address,
)
return err
}
// GetWorstProxies 获取指定协议中延迟最高的N个代理
func (s *Storage) GetWorstProxies(protocol string, limit int) ([]Proxy, error) {
rows, err := s.db.Query(
`SELECT id, address, protocol, exit_ip, exit_location, latency, quality_grade,
use_count, success_count, fail_count, last_used, last_check, created_at, status
FROM proxies
WHERE protocol = ? AND status = 'active'
AND quality_grade != 'S'
AND (JULIANDAY('now') - JULIANDAY(created_at)) * 1440 > 60
ORDER BY latency DESC, fail_count DESC
LIMIT ?`, protocol, limit,
)
if err != nil {
return nil, err
}
defer rows.Close()
var proxies []Proxy
for rows.Next() {
p, err := scanProxy(rows)
if err != nil {
return nil, err
}
proxies = append(proxies, *p)
}
return proxies, nil
}
// ReplaceProxy 替换代理(删除旧的,添加新的)
func (s *Storage) ReplaceProxy(oldAddress string, newProxy Proxy) error {
tx, err := s.db.Begin()
if err != nil {
return err
}
defer tx.Rollback()
// 删除旧代理
_, err = tx.Exec(`DELETE FROM proxies WHERE address = ?`, oldAddress)
if err != nil {
return err
}
// 添加新代理(带完整信息)
grade := CalculateQualityGrade(newProxy.Latency)
_, err = tx.Exec(
`INSERT INTO proxies (address, protocol, exit_ip, exit_location, latency, quality_grade, status)
VALUES (?, ?, ?, ?, ?, ?, 'active')`,
newProxy.Address, newProxy.Protocol, newProxy.ExitIP, newProxy.ExitLocation, newProxy.Latency, grade,
)
if err != nil {
return err
}
return tx.Commit()
}
// MarkAsReplacementCandidate 标记代理为替换候选
func (s *Storage) MarkAsReplacementCandidate(addresses []string) error {
if len(addresses) == 0 {
return nil
}
placeholders := make([]string, len(addresses))
args := make([]interface{}, len(addresses))
for i, addr := range addresses {
placeholders[i] = "?"
args[i] = addr
}
query := fmt.Sprintf(`UPDATE proxies SET status = 'candidate_replace' WHERE address IN (%s)`,
fmt.Sprintf("%s", placeholders))
_, err := s.db.Exec(query, args...)
return err
}
// GetAverageLatency 获取指定协议的平均延迟
func (s *Storage) GetAverageLatency(protocol string) (int, error) {
var avg sql.NullFloat64
err := s.db.QueryRow(
`SELECT AVG(latency) FROM proxies WHERE protocol = ? AND status = 'active' AND latency > 0`,
protocol,
).Scan(&avg)
if err != nil || !avg.Valid {
return 0, err
}
return int(avg.Float64), nil
}
// GetQualityDistribution 获取质量分布统计
func (s *Storage) GetQualityDistribution() (map[string]int, error) {
rows, err := s.db.Query(
`SELECT quality_grade, COUNT(*) as count
FROM proxies
WHERE status = 'active' AND fail_count < 3
GROUP BY quality_grade`,
)
if err != nil {
return nil, err
}
defer rows.Close()
dist := make(map[string]int)
for rows.Next() {
var grade string
var count int
if err := rows.Scan(&grade, &count); err != nil {
return nil, err
}
dist[grade] = count
}
return dist, nil
}
// GetBatchForHealthCheck 获取一批需要健康检查的代理
func (s *Storage) GetBatchForHealthCheck(batchSize int, skipSGrade bool) ([]Proxy, error) {
query := `SELECT id, address, protocol, exit_ip, exit_location, latency, quality_grade,
use_count, success_count, fail_count, last_used, last_check, created_at, status
FROM proxies
WHERE status IN ('active', 'degraded') AND fail_count < 3`
if skipSGrade {
query += ` AND quality_grade != 'S'`
}
query += ` ORDER BY
COALESCE(last_check, '1970-01-01') ASC,
quality_grade DESC
LIMIT ?`
rows, err := s.db.Query(query, batchSize)
if err != nil {
return nil, err
}
defer rows.Close()
var proxies []Proxy
for rows.Next() {
p, err := scanProxy(rows)
if err != nil {
return nil, err
}
proxies = append(proxies, *p)
}
return proxies, nil
}
// CalculateQualityGrade 根据延迟计算质量等级
func CalculateQualityGrade(latencyMs int) string {
switch {
case latencyMs <= 500:
return "S" // 超快
case latencyMs <= 1000:
return "A" // 良好
case latencyMs <= 2000:
return "B" // 可用
default:
return "C" // 淘汰候选
}
}
// DeleteInvalid 删除失败次数超过阈值的代理
func (s *Storage) DeleteInvalid(maxFailCount int) (int64, error) {
res, err := s.db.Exec(`DELETE FROM proxies WHERE fail_count >= ?`, maxFailCount)
@@ -195,26 +556,62 @@ func (s *Storage) DeleteInvalid(maxFailCount int) (int64, error) {
return res.RowsAffected()
}
// DeleteChinaMainland 删除中国大陆出口的代理(保留香港)
func (s *Storage) DeleteChinaMainland() (int64, error) {
// 删除 exit_location 以 "CN " 开头的代理CN后面有空格表示是城市
// 香港是 "HK Hong Kong" 或 "HK" 开头,不会被删除
res, err := s.db.Exec(`DELETE FROM proxies WHERE exit_location LIKE 'CN %'`)
if err != nil {
return 0, err
}
return res.RowsAffected()
}
// DeleteWithoutExitInfo 删除没有出口信息的代理
func (s *Storage) DeleteWithoutExitInfo() (int64, error) {
res, err := s.db.Exec(`DELETE FROM proxies WHERE exit_ip = '' OR exit_location = ''`)
if err != nil {
return 0, err
}
return res.RowsAffected()
}
// Count 返回可用代理数量
func (s *Storage) Count() (int, error) {
var count int
err := s.db.QueryRow(`SELECT COUNT(*) FROM proxies WHERE fail_count < 3`).Scan(&count)
err := s.db.QueryRow(
`SELECT COUNT(*) FROM proxies WHERE status IN ('active', 'degraded') AND fail_count < 3`,
).Scan(&count)
return count, err
}
// CountByProtocol 按协议统计数量
func (s *Storage) CountByProtocol(protocol string) (int, error) {
var count int
err := s.db.QueryRow(`SELECT COUNT(*) FROM proxies WHERE fail_count < 3 AND protocol = ?`, protocol).Scan(&count)
err := s.db.QueryRow(
`SELECT COUNT(*) FROM proxies WHERE status IN ('active', 'degraded') AND fail_count < 3 AND protocol = ?`,
protocol,
).Scan(&count)
return count, err
}
// IncrementFailCount 增加失败次数
func (s *Storage) IncrementFailCount(address string) error {
_, err := s.db.Exec(
`UPDATE proxies SET fail_count = fail_count + 1 WHERE address = ?`,
address,
)
return err
}
// GetByProtocol 按协议获取代理列表
func (s *Storage) GetByProtocol(protocol string) ([]Proxy, error) {
rows, err := s.db.Query(
`SELECT id, address, protocol, fail_count, last_check, created_at
FROM proxies WHERE fail_count < 3 AND protocol = ?
ORDER BY created_at DESC`, protocol,
`SELECT id, address, protocol, exit_ip, exit_location, latency, quality_grade,
use_count, success_count, fail_count, last_used, last_check, created_at, status
FROM proxies
WHERE status IN ('active', 'degraded') AND fail_count < 3 AND protocol = ?
ORDER BY latency ASC`, protocol,
)
if err != nil {
return nil, err
@@ -223,15 +620,11 @@ func (s *Storage) GetByProtocol(protocol string) ([]Proxy, error) {
var proxies []Proxy
for rows.Next() {
p := Proxy{}
var lastCheck sql.NullTime
if err := rows.Scan(&p.ID, &p.Address, &p.Protocol, &p.FailCount, &lastCheck, &p.CreatedAt); err != nil {
p, err := scanProxy(rows)
if err != nil {
return nil, err
}
if lastCheck.Valid {
p.LastCheck = lastCheck.Time
}
proxies = append(proxies, p)
proxies = append(proxies, *p)
}
return proxies, nil
}
@@ -240,3 +633,8 @@ func (s *Storage) GetByProtocol(protocol string) ([]Proxy, error) {
func (s *Storage) Close() error {
return s.db.Close()
}
// GetDB 获取数据库实例(供其他模块使用)
func (s *Storage) GetDB() *sql.DB {
return s.db
}

121
test/README.md Normal file
View File

@@ -0,0 +1,121 @@
# GoProxy 测试脚本
本目录包含用于测试 GoProxy 代理服务的脚本。所有脚本都采用**持续运行模式**(类似 `ping` 命令),按 `Ctrl+C` 停止并显示统计。
## 📝 脚本列表
| 脚本 | 语言 | 依赖 | 运行模式 | 推荐度 |
|------|------|------|----------|--------|
| `test_proxy.sh` | Bash | curl + Python3 | 持续运行 | ⭐⭐⭐ |
| `test_proxy.go` | Go | `golang.org/x/net/proxy` | 持续运行 | ⭐⭐ |
| `test_proxy.py` | Python | `requests`, `pysocks` | 持续运行 | ⭐⭐ |
## 🚀 快速使用
### Bash 脚本(推荐)
```bash
# 从项目根目录运行(持续测试 HTTP
./test/test_proxy.sh
# 测试 HTTP 协议
./test/test_proxy.sh http
# 测试 SOCKS5 协议
./test/test_proxy.sh socks5
# 按 Ctrl+C 停止并查看统计
```
### Go 脚本
```bash
# 安装依赖
go get golang.org/x/net/proxy
# 运行测试
go run test/test_proxy.go
# 或编译后运行
cd test
go build -o test_proxy test_proxy.go
./test_proxy
```
### Python 脚本
```bash
# 安装依赖
pip install requests pysocks
# 运行测试
python test/test_proxy.py
```
## 📊 测试内容
所有脚本都会:
1. 通过指定端口代理发送请求(默认 `127.0.0.1:7777`
2. 访问 `http://ip-api.com/json` 获取出口 IP 和国家信息
3. **持续发送请求**,间隔 1 秒(类似 `ping` 命令)
4. 实时显示国旗 emoji、出口 IP 和延迟
5.`Ctrl+C` 停止并显示统计摘要
## 📖 详细文档
完整的测试指南、故障排查、高级用法,请查看:
👉 [TEST_GUIDE.md](./TEST_GUIDE.md)
## 🔀 测试不同端口策略
```bash
# 对比两个端口的行为差异:
# 随机轮换模式 - IP 高度分散
./test/test_proxy.sh 7777
# 最低延迟模式 - 固定使用最快代理
./test/test_proxy.sh 7776
```
**观察要点**
- **7777 端口**:每次请求的出口 IP 应该不同(证明在轮换)
- **7776 端口**:连续多次请求的出口 IP 基本相同(证明固定使用最优代理)
## 🔍 预期输出
```
PROXY 127.0.0.1:7777 (http://ip-api.com/json/?fields=countryCode,query): continuous mode
proxy from 🇺🇸 203.0.113.45: seq=1 time=1234ms
proxy from 🇩🇪 198.51.100.78: seq=2 time=987ms
proxy from 🇬🇧 192.0.2.123: seq=3 time=1567ms
proxy #4: request failed (timeout)
proxy from 🇯🇵 198.51.100.12: seq=5 time=890ms
proxy from 🇫🇷 192.0.2.234: seq=6 time=1456ms
...
(持续运行,按 Ctrl+C 停止)
^C
---
50 requests transmitted, 47 received, 3 failed, 6.0% packet loss
```
**输出风格**
- 简洁清晰,类似 `ping` 命令
- 一行一个结果
- 显示国旗 emoji、出口 IP、序号、延迟
- 统计信息简洁明了
**观察要点**
- 每次请求的出口 IP 应该不同(证明代理轮换)
- 延迟应该在合理范围(< 2000ms
- 丢包率应该 < 10%
- 可以长时间运行观察稳定性
## 📝 注意事项
1. 确保 GoProxy 服务已启动:`./goproxy`
2. 首次启动需等待代理池就绪(约 30-60 秒)
3. 可配合 WebUI (http://localhost:7778) 查看实时状态

245
test/TEST_GUIDE.md Normal file
View File

@@ -0,0 +1,245 @@
# GoProxy 测试指南
本项目提供了三种测试脚本,用于验证代理服务的功能和性能。所有脚本都采用**持续运行模式**(类似 `ping` 命令),按 `Ctrl+C` 停止并显示统计信息。
## 📋 测试脚本说明
所有测试脚本位于 `test/` 目录下。
### 1. Bash 脚本(推荐)- `test_proxy.sh`
最简单直接,使用 `curl` 命令持续测试代理服务。
**使用方法:**
```bash
# 测试随机轮换模式(默认 7777 端口)
./test/test_proxy.sh
# 测试最低延迟模式7776 端口)
./test/test_proxy.sh 7776
# 自定义端口
./test/test_proxy.sh 8080
```
**特点:**
- 无需安装依赖(仅需 curl 和 Python3
- 兼容 macOS 和 Linux
- 持续运行,按 Ctrl+C 停止
- 显示国旗 emoji 和出口 IP
---
### 2. Go 脚本 - `test_proxy.go`
使用 Go 语言编写,与项目本身技术栈一致。
**使用方法:**
```bash
# 测试随机轮换模式(默认 7777 端口)
go run test/test_proxy.go
# 测试最低延迟模式7776 端口)
go run test/test_proxy.go 7776
# 自定义端口
go run test/test_proxy.go 8080
# 或编译后运行
cd test && go build -o test_proxy test_proxy.go
./test_proxy 7776
```
**特点:**
- 与项目技术栈统一
- 可编译为独立二进制
- 显示国旗 emoji 和出口 IP
- 持续运行,按 Ctrl+C 停止
- 彩色输出
---
### 3. Python 脚本 - `test_proxy.py`
使用 Python 编写,语法简洁易读。
**使用方法:**
```bash
# 安装依赖
pip install requests
# 测试随机轮换模式(默认 7777 端口)
python test/test_proxy.py
# 测试最低延迟模式7776 端口)
python test/test_proxy.py 7776
# 自定义端口
python test/test_proxy.py 8080
```
**特点:**
- Python 生态丰富
- 易于扩展和修改
- 持续运行,按 Ctrl+C 停止
- 显示国旗 emoji 和出口 IP
---
## 🎯 测试输出示例
```
PROXY 127.0.0.1:7777 (http://ip-api.com/json/?fields=countryCode,query): continuous mode
proxy from 🇺🇸 203.0.113.45: seq=1 time=1234ms
proxy from 🇩🇪 198.51.100.78: seq=2 time=987ms
proxy from 🇬🇧 192.0.2.123: seq=3 time=1567ms
proxy from 🇺🇸 203.0.113.45: seq=4 time=1123ms
proxy #5: request failed (timeout)
proxy from 🇯🇵 198.51.100.12: seq=6 time=890ms
proxy from 🇫🇷 192.0.2.234: seq=7 time=1456ms
proxy from 🇨🇳 68.71.249.153: seq=8 time=1102ms
...
^C
---
50 requests transmitted, 47 received, 3 failed, 6.0% packet loss
```
**特点:**
- ✅ 简洁输出,类似 `ping` 命令
- ✅ 一行一个结果,清晰易读
- ✅ 显示国旗 emoji、出口 IP 和延迟
- ✅ 按 Ctrl+C 停止并显示统计摘要
- ✅ 显示丢包率(失败率)
## 🔍 观察要点
### 1. 出口 IP 变化
- 每次请求的出口 IP **应该不同**,证明代理池在轮换
- 如果连续多次是同一个 IP说明池子中可用代理较少
- 持续运行可以观察到代理的循环使用模式
### 2. 延迟表现
- 延迟应该在合理范围内(通常 < 2000ms
- 如果延迟过高或超时,说明代理质量下降
- 观察延迟趋势:是否稳定在某个范围
### 3. 成功率(动态显示)
- 正常情况下成功率应该 > 90%
- 持续运行时成功率会动态更新
- 如果成功率持续下降,需要检查:
- 代理池是否有足够的健康代理
- 网络连接是否正常
- 代理源质量是否下降
### 4. 协议对比
- HTTP 代理:速度通常较快,兼容性好
- SOCKS5 代理:更底层,可传输各种协议数据
### 5. 持续测试优势
- 可以长时间运行,观察代理池的稳定性
- 实时监控成功率变化
- 按 Ctrl+C 随时查看统计结果
- 类似 `ping` 命令的使用体验
## 📊 配合 WebUI 监控
测试时可以同时打开 WebUI (http://localhost:7778)
1. **代理列表**:查看正在使用的代理
2. **使用统计**:观察 `使用次数` 列的变化
3. **系统日志**:实时查看代理请求日志
4. **质量分布**:查看当前池子的质量分布
## ⚙️ 自定义配置
修改脚本顶部的配置变量:
```bash
PROXY_HOST="127.0.0.1" # 代理服务地址
PROXY_PORT="7777" # 代理服务端口
TEST_URL="http://httpbin.org/ip" # 测试目标 URL
DELAY=1 # 每次请求间隔(秒)
```
**注意**:所有脚本现在都是持续运行模式,按 `Ctrl+C` 停止,无需配置请求次数。
## 🛠️ 高级测试
### 测试特定网站
```bash
# 测试访问 Google
curl -x "http://127.0.0.1:7777" https://www.google.com
# 测试访问 GitHub API
curl -x "http://127.0.0.1:7777" https://api.github.com/users/github
# 测试中国网站
curl -x "http://127.0.0.1:7777" https://www.baidu.com
```
### 并发压力测试
```bash
# 使用 ab (Apache Bench) 进行压力测试
ab -n 100 -c 10 -X 127.0.0.1:7777 http://httpbin.org/ip
# 使用 wrk 进行压力测试
wrk -t4 -c100 -d30s --proxy http://127.0.0.1:7777 http://httpbin.org/ip
```
### 查看当前使用的代理
```bash
# 通过 httpbin 获取当前出口 IP
curl -x "http://127.0.0.1:7777" http://httpbin.org/ip
# 获取更详细的信息(包括 headers
curl -x "http://127.0.0.1:7777" http://httpbin.org/anything
```
## 📝 注意事项
1. **启动代理服务**:测试前确保 `goproxy` 已启动
```bash
./goproxy
```
2. **等待代理池就绪**:首次启动需要等待代理抓取和验证(约 30-60 秒)
3. **网络环境**:确保服务器可以访问外部代理源和测试 URL
4. **防火墙**:确保 7777 端口没有被防火墙阻止
5. **代理协议**:不同协议可能表现不同,建议都测试一遍
## 🐛 故障排查
### 全部请求失败
- 检查 `goproxy` 服务是否正在运行
- 检查代理池是否有可用代理(访问 WebUI 查看)
- 检查端口 7777 是否被占用
### 成功率低
- 查看 WebUI 日志,确认代理质量
- 可能需要等待池子优化(系统会自动轮换低质量代理)
- 检查网络连接是否稳定
### SOCKS5 测试失败
- 确认 curl 版本支持 SOCKS5`curl --version`
- Python 脚本需要安装 `pysocks``pip install pysocks`
- Go 脚本需要安装依赖:`go get golang.org/x/net/proxy`
### Bash 脚本时间戳错误macOS
如果看到 `value too great for base` 错误:
- 脚本已自动使用 Python3 获取毫秒时间戳macOS 兼容)
- 确保系统已安装 Python3macOS 自带)
- 或安装 GNU coreutils`brew install coreutils`
---
**Happy Testing! 🚀**

122
test/test_proxy.go Normal file
View File

@@ -0,0 +1,122 @@
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"os"
"os/signal"
"strings"
"syscall"
"time"
)
const (
proxyHost = "127.0.0.1"
testURL = "http://ip-api.com/json/?fields=countryCode,query"
delaySeconds = 1
)
var proxyPort = "7777"
type IPResponse struct {
Query string `json:"query"`
CountryCode string `json:"countryCode"`
}
var (
totalCount = 0
successCount = 0
)
// 国家代码转 emoji 旗帜
func countryToEmoji(countryCode string) string {
if countryCode == "" {
return "🌐"
}
countryCode = strings.ToUpper(countryCode)
if len(countryCode) != 2 {
return "🌐"
}
// 将国家代码转换为 emoji
// A=127462, 所以 'US' -> 🇺🇸
first := rune(countryCode[0]) - 'A' + 127462
second := rune(countryCode[1]) - 'A' + 127462
return string([]rune{first, second})
}
func printStats() {
fmt.Println()
fmt.Println("---")
lossCount := totalCount - successCount
lossRate := 0.0
if totalCount > 0 {
lossRate = float64(lossCount) / float64(totalCount) * 100
}
fmt.Printf("%d requests transmitted, %d received, %d failed, %.1f%% packet loss\n",
totalCount, successCount, lossCount, lossRate)
os.Exit(0)
}
func testHTTPProxyContinuous() {
fmt.Printf("PROXY %s:%s (%s): continuous mode\n", proxyHost, proxyPort, testURL)
fmt.Println()
proxyURL, _ := url.Parse(fmt.Sprintf("http://%s:%s", proxyHost, proxyPort))
client := &http.Client{
Transport: &http.Transport{
Proxy: http.ProxyURL(proxyURL),
},
Timeout: 15 * time.Second,
}
// 捕获 Ctrl+C 信号
sigChan := make(chan os.Signal, 1)
signal.Notify(sigChan, os.Interrupt, syscall.SIGTERM)
go func() {
<-sigChan
printStats()
}()
for {
totalCount++
start := time.Now()
resp, err := client.Get(testURL)
elapsed := time.Since(start).Milliseconds()
if err != nil {
fmt.Printf("proxy #%d: request failed (%v)\n", totalCount, err)
time.Sleep(delaySeconds * time.Second)
continue
}
body, _ := io.ReadAll(resp.Body)
resp.Body.Close()
var ipResp IPResponse
if err := json.Unmarshal(body, &ipResp); err == nil {
flag := countryToEmoji(ipResp.CountryCode)
fmt.Printf("proxy from %s %s: seq=%d time=%dms\n", flag, ipResp.Query, totalCount, elapsed)
successCount++
} else {
fmt.Printf("proxy #%d: parse error\n", totalCount)
}
time.Sleep(delaySeconds * time.Second)
}
}
func main() {
// 支持指定端口号
if len(os.Args) > 1 {
proxyPort = os.Args[1]
}
testHTTPProxyContinuous()
}

97
test/test_proxy.py Normal file
View File

@@ -0,0 +1,97 @@
#!/usr/bin/env python3
"""
GoProxy 持续测试脚本 - 类似 ping 命令的简洁输出
按 Ctrl+C 停止测试
"""
import requests
import time
import sys
import signal
from requests.exceptions import RequestException
# 配置
PROXY_HOST = "127.0.0.1"
PROXY_PORT = int(sys.argv[1]) if len(sys.argv) > 1 and sys.argv[1].isdigit() else 7777
TEST_URL = "http://ip-api.com/json/?fields=countryCode,query"
DELAY_SECONDS = 1
# 统计变量
total_count = 0
success_count = 0
def country_to_emoji(country_code):
"""将国家代码转换为 emoji 旗帜"""
if not country_code or country_code == "null":
return "🌐"
# 将国家代码转换为区域指示符号
# A=127462, 所以 'US' -> 🇺🇸
try:
first = ord(country_code[0].upper()) - ord('A') + 127462
second = ord(country_code[1].upper()) - ord('A') + 127462
return chr(first) + chr(second)
except:
return "🌐"
def signal_handler(sig, frame):
"""处理 Ctrl+C 信号"""
print()
print("---")
loss_count = total_count - success_count
loss_rate = 0.0
if total_count > 0:
loss_rate = loss_count / total_count * 100
print(f"{total_count} requests transmitted, {success_count} received, {loss_count} failed, {loss_rate:.1f}% packet loss")
sys.exit(0)
def test_http_proxy_continuous():
"""持续测试 HTTP 代理"""
global total_count, success_count
proxy_url = f"http://{PROXY_HOST}:{PROXY_PORT}"
proxies = {
"http": proxy_url,
"https": proxy_url,
}
print(f"PROXY {PROXY_HOST}:{PROXY_PORT} ({TEST_URL}): continuous mode")
print()
# 注册信号处理
signal.signal(signal.SIGINT, signal_handler)
while True:
total_count += 1
try:
start_time = time.time()
response = requests.get(
TEST_URL,
proxies=proxies,
timeout=15,
)
elapsed = int((time.time() - start_time) * 1000)
if response.status_code == 200:
data = response.json()
exit_ip = data.get("query", "Unknown")
country_code = data.get("countryCode", "")
flag = country_to_emoji(country_code)
print(f"proxy from {flag} {exit_ip}: seq={total_count} time={elapsed}ms")
success_count += 1
else:
print(f"proxy #{total_count}: request failed (HTTP {response.status_code})")
except RequestException as e:
error_msg = str(e).split(':')[0]
print(f"proxy #{total_count}: {error_msg}")
time.sleep(DELAY_SECONDS)
if __name__ == "__main__":
test_http_proxy_continuous()

95
test/test_proxy.sh Executable file
View File

@@ -0,0 +1,95 @@
#!/bin/bash
# GoProxy 持续测试脚本 - 类似 ping 命令的简洁输出
# 按 Ctrl+C 停止测试
# 用法: ./test_proxy.sh [端口号默认7777]
PROXY_HOST="127.0.0.1"
PROXY_PORT="${1:-7777}"
TEST_URL="http://ip-api.com/json/?fields=countryCode,query"
DELAY=1
# 统计变量
total=0
success=0
fail=0
# 获取毫秒时间戳(兼容 macOS 和 Linux
get_ms_time() {
python3 -c 'import time; print(int(time.time() * 1000))'
}
# 国家代码转 emoji 旗帜
country_to_emoji() {
local country_code="$1"
if [ -z "$country_code" ] || [ "$country_code" = "null" ]; then
echo "🌐"
return
fi
# 将国家代码转换为 emoji使用 Unicode 区域指示符)
# 每个字母转换为对应的区域指示符号字符
local first="${country_code:0:1}"
local second="${country_code:1:1}"
# A=127462, 所以 A->🇦 就是 127462B->🇧 就是 127463
# 使用 printf 和 Unicode 编码
python3 -c "print(chr(127462 + ord('$first') - ord('A')) + chr(127462 + ord('$second') - ord('A')))"
}
# 捕获 Ctrl+C 信号
trap ctrl_c INT
function ctrl_c() {
echo ""
echo "---"
loss_rate=$(awk "BEGIN {printf \"%.1f\", ($total - $success)/$total*100}")
echo "$total requests transmitted, $success received, $((total - success)) failed, ${loss_rate}% packet loss"
exit 0
}
# 测试 HTTP 代理
test_http_proxy() {
echo "PROXY $PROXY_HOST:$PROXY_PORT ($TEST_URL): continuous mode"
echo ""
while true; do
total=$((total + 1))
# 使用 HTTP 代理发送请求
start_time=$(get_ms_time)
response=$(curl -x "http://${PROXY_HOST}:${PROXY_PORT}" \
-s \
-w "\n%{http_code}" \
--connect-timeout 10 \
--max-time 15 \
"${TEST_URL}" 2>&1)
end_time=$(get_ms_time)
elapsed=$((end_time - start_time))
# 分离响应体和状态码
http_code=$(echo "$response" | tail -n 1)
body=$(echo "$response" | sed '$d')
if [ "$http_code" = "200" ]; then
exit_ip=$(echo "$body" | grep -o '"query":"[^"]*"' | cut -d'"' -f4)
country_code=$(echo "$body" | grep -o '"countryCode":"[^"]*"' | cut -d'"' -f4)
if [ -n "$exit_ip" ]; then
flag=$(country_to_emoji "$country_code")
echo "proxy from $flag $exit_ip: seq=$total time=${elapsed}ms"
success=$((success + 1))
else
echo "proxy #$total: parse error"
fail=$((fail + 1))
fi
else
echo "proxy #$total: request failed (HTTP $http_code)"
fail=$((fail + 1))
fi
sleep $DELAY
done
}
# 主函数
test_http_proxy

View File

@@ -1,6 +1,7 @@
package validator
import (
"encoding/json"
"fmt"
"io"
"log"
@@ -10,8 +11,8 @@ import (
"time"
"golang.org/x/net/proxy"
"proxy-pool/config"
"proxy-pool/storage"
"goproxy/config"
"goproxy/storage"
)
type Validator struct {
@@ -43,9 +44,41 @@ func New(concurrency, timeoutSec int, validateURL string) *Validator {
}
type Result struct {
Proxy storage.Proxy
Valid bool
Latency time.Duration
Proxy storage.Proxy
Valid bool
Latency time.Duration
ExitIP string
ExitLocation string
}
// getExitIPInfo 通过代理获取出口 IP 和地理位置
func getExitIPInfo(client *http.Client) (string, string) {
// 使用 ip-api.com 返回 JSON 格式的 IP 信息
resp, err := client.Get("http://ip-api.com/json/?fields=status,country,countryCode,city,query")
if err != nil {
return "", ""
}
defer resp.Body.Close()
var result struct {
Status string `json:"status"`
Query string `json:"query"` // IP 地址
Country string `json:"country"`
CountryCode string `json:"countryCode"`
City string `json:"city"`
}
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil || result.Status != "success" {
return "", ""
}
// 返回格式IP, "国家代码 城市"
location := result.CountryCode
if result.City != "" {
location = fmt.Sprintf("%s %s", result.CountryCode, result.City)
}
return result.Query, location
}
// ValidateAll 并发验证所有代理,返回验证结果
@@ -70,8 +103,8 @@ func (v *Validator) ValidateStream(proxies []storage.Proxy) <-chan Result {
go func(px storage.Proxy) {
defer wg.Done()
defer func() { <-sem }()
valid, latency := v.ValidateOne(px)
ch <- Result{Proxy: px, Valid: valid, Latency: latency}
valid, latency, exitIP, exitLocation := v.ValidateOne(px)
ch <- Result{Proxy: px, Valid: valid, Latency: latency, ExitIP: exitIP, ExitLocation: exitLocation}
}(p)
}
wg.Wait()
@@ -81,8 +114,8 @@ func (v *Validator) ValidateStream(proxies []storage.Proxy) <-chan Result {
return ch
}
// ValidateOne 验证单个代理是否可用,返回是否有效延迟
func (v *Validator) ValidateOne(p storage.Proxy) (bool, time.Duration) {
// ValidateOne 验证单个代理是否可用,返回是否有效延迟、出口IP和地理位置
func (v *Validator) ValidateOne(p storage.Proxy) (bool, time.Duration, string, string) {
var client *http.Client
var err error
@@ -93,33 +126,50 @@ func (v *Validator) ValidateOne(p storage.Proxy) (bool, time.Duration) {
client, err = newSOCKS5Client(p.Address, v.timeout)
default:
log.Printf("unknown protocol %s for %s", p.Protocol, p.Address)
return false, 0
return false, 0, "", ""
}
if err != nil {
return false, 0
return false, 0, "", ""
}
start := time.Now()
resp, err := client.Get(v.validateURL)
latency := time.Since(start)
if err != nil {
return false, 0
return false, 0, "", ""
}
defer resp.Body.Close()
io.Copy(io.Discard, resp.Body)
// 验证状态码200 或 204 都接受)
if resp.StatusCode != http.StatusOK && resp.StatusCode != http.StatusNoContent {
return false, latency
return false, latency, "", ""
}
// 响应时间过滤
if v.maxResponseMs > 0 && latency > time.Duration(v.maxResponseMs)*time.Millisecond {
return false, latency
return false, latency, "", ""
}
return true, latency
// 获取出口 IP 和地理位置(仅在验证通过时)
exitIP, exitLocation := getExitIPInfo(client)
// 必须能获取到出口信息
if exitIP == "" || exitLocation == "" {
return false, latency, exitIP, exitLocation
}
// 过滤中国大陆出口香港的countryCode是HK不是CN
if len(exitLocation) >= 2 {
countryCode := exitLocation[:2]
if countryCode == "CN" {
// 中国大陆出口,直接拒绝
return false, latency, exitIP, exitLocation
}
}
return true, latency, exitIP, exitLocation
}
func newHTTPClient(address string, timeout time.Duration) (*http.Client, error) {

File diff suppressed because it is too large Load Diff

View File

@@ -5,31 +5,45 @@ const loginHTML = `<!DOCTYPE html>
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>ProxyGo - 登录</title>
<title>GoProxy — 身份验证</title>
<link href="https://fonts.googleapis.com/css2?family=JetBrains+Mono:wght@400;600;700&family=Share+Tech+Mono&display=swap" rel="stylesheet">
<style>
*{box-sizing:border-box;margin:0;padding:0}
body{background:#0f172a;color:#e2e8f0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',sans-serif;display:flex;align-items:center;justify-content:center;min-height:100vh}
.card{background:#1e293b;border:1px solid #334155;border-radius:12px;padding:40px;width:360px;box-shadow:0 20px 60px rgba(0,0,0,0.5)}
h1{font-size:24px;font-weight:700;margin-bottom:8px;color:#f1f5f9}
.sub{color:#94a3b8;font-size:14px;margin-bottom:32px}
label{display:block;font-size:13px;color:#94a3b8;margin-bottom:6px}
input[type=password]{width:100%;padding:10px 14px;background:#0f172a;border:1px solid #334155;border-radius:8px;color:#f1f5f9;font-size:15px;outline:none;transition:border 0.2s}
input[type=password]:focus{border-color:#6366f1}
button{width:100%;margin-top:20px;padding:11px;background:#6366f1;color:#fff;border:none;border-radius:8px;font-size:15px;font-weight:600;cursor:pointer;transition:background 0.2s}
button:hover{background:#4f46e5}
.logo{font-size:32px;margin-bottom:16px}
body{background:#0a0a0a;color:#00ff41;font-family:JetBrains Mono,monospace;display:flex;align-items:center;justify-content:center;min-height:100vh;position:relative}
body::before{content:'';position:fixed;top:0;left:0;width:100%;height:100%;background:repeating-linear-gradient(0deg,rgba(0,255,65,0.03) 0px,transparent 1px,transparent 2px,rgba(0,255,65,0.03) 3px);pointer-events:none;z-index:9999}
body::after{content:'';position:fixed;top:0;left:0;width:100%;height:100%;background:radial-gradient(ellipse at center,rgba(0,255,65,0.08) 0%,transparent 70%);pointer-events:none;z-index:9998}
.card{border:1px solid #00ff41;padding:64px;width:440px;background:#111;box-shadow:0 0 40px rgba(0,255,65,0.3);position:relative;z-index:1}
h1{font-size:32px;font-weight:700;margin-bottom:8px;letter-spacing:0.15em;text-transform:uppercase;color:#00ff41;text-shadow:0 0 15px #00ff41}
.sub{color:#00cc33;font-size:12px;margin-bottom:48px;font-family:JetBrains Mono,monospace;letter-spacing:0.08em;text-transform:uppercase}
label{display:block;font-size:10px;text-transform:uppercase;letter-spacing:0.1em;color:#00cc33;margin-bottom:8px;font-weight:600}
input[type=password]{width:100%;padding:16px;background:#0d0d0d;border:1px solid #1a3a1a;color:#00ff41;font-size:16px;font-family:JetBrains Mono,monospace;outline:none;transition:all 0.2s}
input[type=password]:focus{border-color:#00ff41;background:#111;box-shadow:0 0 10px rgba(0,255,65,0.3)}
button{width:100%;margin-top:24px;padding:16px;background:#00ff41;color:#000;border:1px solid #00ff41;font-size:12px;font-weight:700;cursor:pointer;transition:all 0.2s;text-transform:uppercase;letter-spacing:0.1em;font-family:JetBrains Mono,monospace;box-shadow:0 0 15px rgba(0,255,65,0.5)}
button:hover{box-shadow:0 0 25px rgba(0,255,65,0.8);transform:translateY(-1px)}
.logo{font-size:64px;margin-bottom:24px;line-height:1;font-weight:700;letter-spacing:0.1em;color:#00ff41;text-shadow:0 0 20px #00ff41}
.tip{color:#888;font-size:10px;margin-top:24px;line-height:1.6;letter-spacing:0.05em;text-align:center}
.tip a{color:#00ff41;text-decoration:none;border-bottom:1px solid transparent;transition:all 0.2s}
.tip a:hover{border-bottom-color:#00ff41;text-shadow:0 0 8px #00ff41}
.github{position:absolute;top:20px;right:20px;color:#00ff41;opacity:0.6;transition:all 0.3s}
.github:hover{opacity:1;transform:scale(1.1);filter:drop-shadow(0 0 8px #00ff41)}
</style>
</head>
<body>
<a href="https://github.com/isboyjc/ProxyGo" target="_blank" class="github" title="GitHub">
<svg width="32" height="32" viewBox="0 0 16 16" fill="currentColor">
<path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"/>
</svg>
</a>
<div class="card">
<div class="logo"></div>
<h1>ProxyGo</h1>
<p class="sub">代理池管理系统</p>
<div class="logo">[GP]</div>
<h1>GoProxy</h1>
<p class="sub">// Intelligent Proxy Pool</p>
<form method="POST" action="/login">
<label>管理密码</label>
<input type="password" name="password" placeholder="请输入密码" autofocus>
<button type="submit">登录</button>
<label>> Password</label>
<input type="password" name="password" placeholder="****************" autofocus>
<button type="submit">[ AUTHENTICATE ]</button>
</form>
<p class="tip">访客模式可<a href="/">查看数据</a>,管理员登录后可完全控制</p>
</div>
</body>
</html>`
@@ -39,33 +53,47 @@ const loginHTMLWithError = `<!DOCTYPE html>
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>ProxyGo - 登录</title>
<title>GoProxy — 身份验证</title>
<link href="https://fonts.googleapis.com/css2?family=JetBrains+Mono:wght@400;600;700&family=Share+Tech+Mono&display=swap" rel="stylesheet">
<style>
*{box-sizing:border-box;margin:0;padding:0}
body{background:#0f172a;color:#e2e8f0;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',sans-serif;display:flex;align-items:center;justify-content:center;min-height:100vh}
.card{background:#1e293b;border:1px solid #334155;border-radius:12px;padding:40px;width:360px;box-shadow:0 20px 60px rgba(0,0,0,0.5)}
h1{font-size:24px;font-weight:700;margin-bottom:8px;color:#f1f5f9}
.sub{color:#94a3b8;font-size:14px;margin-bottom:32px}
label{display:block;font-size:13px;color:#94a3b8;margin-bottom:6px}
input[type=password]{width:100%;padding:10px 14px;background:#0f172a;border:1px solid #334155;border-radius:8px;color:#f1f5f9;font-size:15px;outline:none;transition:border 0.2s}
input[type=password]:focus{border-color:#6366f1}
button{width:100%;margin-top:20px;padding:11px;background:#6366f1;color:#fff;border:none;border-radius:8px;font-size:15px;font-weight:600;cursor:pointer;transition:background 0.2s}
button:hover{background:#4f46e5}
.logo{font-size:32px;margin-bottom:16px}
.error{background:#450a0a;border:1px solid #7f1d1d;color:#fca5a5;padding:10px 14px;border-radius:8px;font-size:13px;margin-bottom:16px}
body{background:#0a0a0a;color:#00ff41;font-family:JetBrains Mono,monospace;display:flex;align-items:center;justify-content:center;min-height:100vh;position:relative}
body::before{content:'';position:fixed;top:0;left:0;width:100%;height:100%;background:repeating-linear-gradient(0deg,rgba(0,255,65,0.03) 0px,transparent 1px,transparent 2px,rgba(0,255,65,0.03) 3px);pointer-events:none;z-index:9999}
body::after{content:'';position:fixed;top:0;left:0;width:100%;height:100%;background:radial-gradient(ellipse at center,rgba(0,255,65,0.08) 0%,transparent 70%);pointer-events:none;z-index:9998}
.card{border:1px solid #00ff41;padding:64px;width:440px;background:#111;box-shadow:0 0 40px rgba(0,255,65,0.3);position:relative;z-index:1}
h1{font-size:32px;font-weight:700;margin-bottom:8px;letter-spacing:0.15em;text-transform:uppercase;color:#00ff41;text-shadow:0 0 15px #00ff41}
.sub{color:#00cc33;font-size:12px;margin-bottom:48px;font-family:JetBrains Mono,monospace;letter-spacing:0.08em;text-transform:uppercase}
label{display:block;font-size:10px;text-transform:uppercase;letter-spacing:0.1em;color:#00cc33;margin-bottom:8px;font-weight:600}
input[type=password]{width:100%;padding:16px;background:#0d0d0d;border:1px solid #1a3a1a;color:#00ff41;font-size:16px;font-family:JetBrains Mono,monospace;outline:none;transition:all 0.2s}
input[type=password]:focus{border-color:#00ff41;background:#111;box-shadow:0 0 10px rgba(0,255,65,0.3)}
button{width:100%;margin-top:24px;padding:16px;background:#00ff41;color:#000;border:1px solid #00ff41;font-size:12px;font-weight:700;cursor:pointer;transition:all 0.2s;text-transform:uppercase;letter-spacing:0.1em;font-family:JetBrains Mono,monospace;box-shadow:0 0 15px rgba(0,255,65,0.5)}
button:hover{box-shadow:0 0 25px rgba(0,255,65,0.8);transform:translateY(-1px)}
.logo{font-size:64px;margin-bottom:24px;line-height:1;font-weight:700;letter-spacing:0.1em;color:#00ff41;text-shadow:0 0 20px #00ff41}
.error{background:#ff0033;color:#fff;padding:16px;font-size:11px;margin-bottom:24px;font-family:JetBrains Mono,monospace;font-weight:600;border:1px solid #ff0033;box-shadow:0 0 15px rgba(255,0,51,0.5);text-transform:uppercase;letter-spacing:0.05em}
.tip{color:#888;font-size:10px;margin-top:24px;line-height:1.6;letter-spacing:0.05em;text-align:center}
.tip a{color:#00ff41;text-decoration:none;border-bottom:1px solid transparent;transition:all 0.2s}
.tip a:hover{border-bottom-color:#00ff41;text-shadow:0 0 8px #00ff41}
.github{position:absolute;top:20px;right:20px;color:#00ff41;opacity:0.6;transition:all 0.3s}
.github:hover{opacity:1;transform:scale(1.1);filter:drop-shadow(0 0 8px #00ff41)}
</style>
</head>
<body>
<a href="https://github.com/isboyjc/ProxyGo" target="_blank" class="github" title="GitHub">
<svg width="32" height="32" viewBox="0 0 16 16" fill="currentColor">
<path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"/>
</svg>
</a>
<div class="card">
<div class="logo"></div>
<h1>ProxyGo</h1>
<p class="sub">代理池管理系统</p>
<div class="error">密码错误,请重试</div>
<div class="logo">[GP]</div>
<h1>GoProxy</h1>
<p class="sub">// Intelligent Proxy Pool</p>
<div class="error">[!] ACCESS DENIED - INVALID PASSWORD</div>
<form method="POST" action="/login">
<label>管理密码</label>
<input type="password" name="password" placeholder="请输入密码" autofocus>
<button type="submit">登录</button>
<label>> Password</label>
<input type="password" name="password" placeholder="****************" autofocus>
<button type="submit">[ AUTHENTICATE ]</button>
</form>
<p class="tip">访客模式可<a href="/">查看数据</a>,管理员登录后可完全控制</p>
</div>
</body>
</html>`

View File

@@ -9,9 +9,11 @@ import (
"sync"
"time"
"proxy-pool/config"
"proxy-pool/logger"
"proxy-pool/storage"
"goproxy/config"
"goproxy/logger"
"goproxy/pool"
"goproxy/storage"
"goproxy/validator"
)
// 简单内存 session
@@ -44,12 +46,19 @@ type FetchTrigger func()
type Server struct {
storage *storage.Storage
cfg *config.Config
poolMgr *pool.Manager
fetchTrigger FetchTrigger
configChanged chan<- struct{}
}
func New(s *storage.Storage, cfg *config.Config, ft FetchTrigger, cc chan<- struct{}) *Server {
return &Server{storage: s, cfg: cfg, fetchTrigger: ft, configChanged: cc}
func New(s *storage.Storage, cfg *config.Config, pm *pool.Manager, ft FetchTrigger, cc chan<- struct{}) *Server {
return &Server{
storage: s,
cfg: cfg,
poolMgr: pm,
fetchTrigger: ft,
configChanged: cc,
}
}
func (s *Server) Start() {
@@ -57,12 +66,22 @@ func (s *Server) Start() {
mux.HandleFunc("/", s.handleIndex)
mux.HandleFunc("/login", s.handleLogin)
mux.HandleFunc("/logout", s.handleLogout)
mux.HandleFunc("/api/stats", s.authMiddleware(s.apiStats))
mux.HandleFunc("/api/proxies", s.authMiddleware(s.apiProxies))
// 只读 API访客可访问
mux.HandleFunc("/api/stats", s.readOnlyMiddleware(s.apiStats))
mux.HandleFunc("/api/proxies", s.readOnlyMiddleware(s.apiProxies))
mux.HandleFunc("/api/logs", s.readOnlyMiddleware(s.apiLogs))
mux.HandleFunc("/api/pool/status", s.readOnlyMiddleware(s.apiPoolStatus))
mux.HandleFunc("/api/pool/quality", s.readOnlyMiddleware(s.apiQualityDistribution))
mux.HandleFunc("/api/config", s.readOnlyMiddleware(s.apiConfig))
mux.HandleFunc("/api/auth/check", s.apiAuthCheck) // 检查登录状态
// 管理员 API需要登录
mux.HandleFunc("/api/proxy/delete", s.authMiddleware(s.apiDeleteProxy))
mux.HandleFunc("/api/proxy/refresh", s.authMiddleware(s.apiRefreshProxy))
mux.HandleFunc("/api/fetch", s.authMiddleware(s.apiFetch))
mux.HandleFunc("/api/logs", s.authMiddleware(s.apiLogs))
mux.HandleFunc("/api/config", s.authMiddleware(s.apiConfig))
mux.HandleFunc("/api/refresh-latency", s.authMiddleware(s.apiRefreshLatency))
mux.HandleFunc("/api/config/save", s.authMiddleware(s.apiConfigSave))
log.Printf("WebUI listening on %s", s.cfg.WebUIPort)
go func() {
@@ -72,6 +91,7 @@ func (s *Server) Start() {
}()
}
// authMiddleware 管理员权限中间件(必须登录)
func (s *Server) authMiddleware(next http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
if !validSession(r) {
@@ -86,11 +106,16 @@ func (s *Server) authMiddleware(next http.HandlerFunc) http.HandlerFunc {
}
}
func (s *Server) handleIndex(w http.ResponseWriter, r *http.Request) {
if !validSession(r) {
http.Redirect(w, r, "/login", http.StatusFound)
return
// readOnlyMiddleware 只读中间件(访客可访问,但会标记是否为管理员)
func (s *Server) readOnlyMiddleware(next http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
// 访客和管理员都可以访问,通过 validSession 判断权限
next(w, r)
}
}
func (s *Server) handleIndex(w http.ResponseWriter, r *http.Request) {
// 允许访客访问(只读模式),管理员登录后有完整权限
w.Header().Set("Content-Type", "text/html; charset=utf-8")
fmt.Fprint(w, dashboardHTML)
}
@@ -129,6 +154,20 @@ func (s *Server) handleLogout(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, "/login", http.StatusFound)
}
// apiAuthCheck 检查当前用户是否为管理员
func (s *Server) apiAuthCheck(w http.ResponseWriter, r *http.Request) {
isAdmin := validSession(r)
jsonOK(w, map[string]interface{}{
"isAdmin": isAdmin,
"mode": func() string {
if isAdmin {
return "admin"
}
return "guest"
}(),
})
}
func (s *Server) apiStats(w http.ResponseWriter, r *http.Request) {
total, _ := s.storage.Count()
httpCount, _ := s.storage.CountByProtocol("http")
@@ -173,6 +212,60 @@ func (s *Server) apiDeleteProxy(w http.ResponseWriter, r *http.Request) {
jsonOK(w, map[string]string{"status": "deleted"})
}
func (s *Server) apiRefreshProxy(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
jsonError(w, "method not allowed", http.StatusMethodNotAllowed)
return
}
var req struct {
Address string `json:"address"`
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil || req.Address == "" {
jsonError(w, "invalid request", http.StatusBadRequest)
return
}
// 从数据库获取代理信息
proxies, err := s.storage.GetAll()
if err != nil {
jsonError(w, "failed to get proxy", http.StatusInternalServerError)
return
}
var targetProxy *storage.Proxy
for i := range proxies {
if proxies[i].Address == req.Address {
targetProxy = &proxies[i]
break
}
}
if targetProxy == nil {
jsonError(w, "proxy not found", http.StatusNotFound)
return
}
// 异步验证并更新
go func() {
cfg := config.Get()
v := validator.New(1, cfg.ValidateTimeout, cfg.ValidateURL)
log.Printf("[webui] refreshing proxy: %s", req.Address)
valid, latency, exitIP, exitLocation := v.ValidateOne(*targetProxy)
if valid {
latencyMs := int(latency.Milliseconds())
s.storage.UpdateExitInfo(req.Address, exitIP, exitLocation, latencyMs)
log.Printf("[webui] proxy refreshed: %s latency=%dms grade=%s", req.Address, latencyMs, storage.CalculateQualityGrade(latencyMs))
} else {
s.storage.Delete(req.Address)
log.Printf("[webui] proxy validation failed, removed: %s", req.Address)
}
}()
jsonOK(w, map[string]string{"status": "refresh started"})
}
func (s *Server) apiFetch(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
jsonError(w, "method not allowed", http.StatusMethodNotAllowed)
@@ -182,54 +275,173 @@ func (s *Server) apiFetch(w http.ResponseWriter, r *http.Request) {
jsonOK(w, map[string]string{"status": "fetch started"})
}
func (s *Server) apiLogs(w http.ResponseWriter, r *http.Request) {
lines := logger.GetLines(200)
jsonOK(w, map[string]interface{}{"lines": lines})
}
func (s *Server) apiConfig(w http.ResponseWriter, r *http.Request) {
cfg := config.Get()
if r.Method == http.MethodGet {
jsonOK(w, map[string]interface{}{
"fetch_interval": cfg.FetchInterval,
"check_interval": cfg.CheckInterval,
"validate_concurrency": cfg.ValidateConcurrency,
"validate_timeout": cfg.ValidateTimeout,
})
return
}
func (s *Server) apiRefreshLatency(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
jsonError(w, "method not allowed", http.StatusMethodNotAllowed)
return
}
var req struct {
FetchInterval int `json:"fetch_interval"`
CheckInterval int `json:"check_interval"`
ValidateConcurrency int `json:"validate_concurrency"`
ValidateTimeout int `json:"validate_timeout"`
go func() {
log.Println("[webui] refreshing latency for all proxies...")
proxies, err := s.storage.GetAll()
if err != nil {
log.Printf("[webui] get proxies error: %v", err)
return
}
if len(proxies) == 0 {
log.Println("[webui] no proxies to refresh")
return
}
cfg := config.Get()
validate := validator.New(cfg.ValidateConcurrency, cfg.ValidateTimeout, cfg.ValidateURL)
log.Printf("[webui] refreshing latency for %d proxies...", len(proxies))
updated := 0
for r := range validate.ValidateStream(proxies) {
if r.Valid {
latencyMs := int(r.Latency.Milliseconds())
s.storage.UpdateExitInfo(r.Proxy.Address, r.ExitIP, r.ExitLocation, latencyMs)
updated++
} else {
s.storage.Delete(r.Proxy.Address)
}
}
log.Printf("[webui] latency refresh done: updated=%d", updated)
}()
jsonOK(w, map[string]string{"status": "refresh started"})
}
func (s *Server) apiLogs(w http.ResponseWriter, r *http.Request) {
lines := logger.GetLines(100)
jsonOK(w, map[string]interface{}{"lines": lines})
}
// apiConfig 获取配置
func (s *Server) apiConfig(w http.ResponseWriter, r *http.Request) {
cfg := config.Get()
httpSlots, socks5Slots := cfg.CalculateSlots()
jsonOK(w, map[string]interface{}{
// 池子配置
"pool_max_size": cfg.PoolMaxSize,
"pool_http_ratio": cfg.PoolHTTPRatio,
"pool_min_per_protocol": cfg.PoolMinPerProtocol,
"pool_http_slots": httpSlots,
"pool_socks5_slots": socks5Slots,
// 延迟配置
"max_latency_ms": cfg.MaxLatencyMs,
"max_latency_emergency": cfg.MaxLatencyEmergency,
"max_latency_healthy": cfg.MaxLatencyHealthy,
// 验证配置
"validate_concurrency": cfg.ValidateConcurrency,
"validate_timeout": cfg.ValidateTimeout,
// 健康检查配置
"health_check_interval": cfg.HealthCheckInterval,
"health_check_batch_size": cfg.HealthCheckBatchSize,
// 优化配置
"optimize_interval": cfg.OptimizeInterval,
"replace_threshold": cfg.ReplaceThreshold,
})
}
// apiConfigSave 保存配置
func (s *Server) apiConfigSave(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
jsonError(w, "method not allowed", http.StatusMethodNotAllowed)
return
}
var req struct {
PoolMaxSize int `json:"pool_max_size"`
PoolHTTPRatio float64 `json:"pool_http_ratio"`
PoolMinPerProtocol int `json:"pool_min_per_protocol"`
MaxLatencyMs int `json:"max_latency_ms"`
MaxLatencyEmergency int `json:"max_latency_emergency"`
MaxLatencyHealthy int `json:"max_latency_healthy"`
ValidateConcurrency int `json:"validate_concurrency"`
ValidateTimeout int `json:"validate_timeout"`
HealthCheckInterval int `json:"health_check_interval"`
HealthCheckBatchSize int `json:"health_check_batch_size"`
OptimizeInterval int `json:"optimize_interval"`
ReplaceThreshold float64 `json:"replace_threshold"`
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
jsonError(w, "invalid request", http.StatusBadRequest)
return
}
if req.FetchInterval <= 0 || req.CheckInterval <= 0 || req.ValidateConcurrency <= 0 || req.ValidateTimeout <= 0 {
jsonError(w, "all values must be positive", http.StatusBadRequest)
// 验证配置有效性
if req.PoolMaxSize <= 0 || req.PoolHTTPRatio <= 0 || req.PoolHTTPRatio > 1 {
jsonError(w, "invalid pool config", http.StatusBadRequest)
return
}
if err := config.Save(req.FetchInterval, req.CheckInterval, req.ValidateConcurrency, req.ValidateTimeout); err != nil {
// 记录旧配置
oldCfg := config.Get()
oldSize := oldCfg.PoolMaxSize
oldRatio := oldCfg.PoolHTTPRatio
// 更新配置
newCfg := *oldCfg
newCfg.PoolMaxSize = req.PoolMaxSize
newCfg.PoolHTTPRatio = req.PoolHTTPRatio
newCfg.PoolMinPerProtocol = req.PoolMinPerProtocol
newCfg.MaxLatencyMs = req.MaxLatencyMs
newCfg.MaxLatencyEmergency = req.MaxLatencyEmergency
newCfg.MaxLatencyHealthy = req.MaxLatencyHealthy
newCfg.ValidateConcurrency = req.ValidateConcurrency
newCfg.ValidateTimeout = req.ValidateTimeout
newCfg.HealthCheckInterval = req.HealthCheckInterval
newCfg.HealthCheckBatchSize = req.HealthCheckBatchSize
newCfg.OptimizeInterval = req.OptimizeInterval
newCfg.ReplaceThreshold = req.ReplaceThreshold
if err := config.Save(&newCfg); err != nil {
jsonError(w, "save config error: "+err.Error(), http.StatusInternalServerError)
return
}
// 通知定时器重置
// 通知配置变更
select {
case s.configChanged <- struct{}{}:
default:
}
log.Printf("[config] updated: fetch=%dm check=%dm concurrency=%d timeout=%ds",
req.FetchInterval, req.CheckInterval, req.ValidateConcurrency, req.ValidateTimeout)
// 如果池子大小或比例变更,调整池子
if oldSize != req.PoolMaxSize || oldRatio != req.PoolHTTPRatio {
go s.poolMgr.AdjustForConfigChange(oldSize, oldRatio)
}
log.Printf("[config] 配置已更新: 池子=%d HTTP=%.0f%% 延迟=%dms",
req.PoolMaxSize, req.PoolHTTPRatio*100, req.MaxLatencyMs)
jsonOK(w, map[string]string{"status": "saved"})
}
// apiPoolStatus 获取池子状态
func (s *Server) apiPoolStatus(w http.ResponseWriter, r *http.Request) {
status, err := s.poolMgr.GetStatus()
if err != nil {
jsonError(w, err.Error(), http.StatusInternalServerError)
return
}
jsonOK(w, status)
}
// apiQualityDistribution 获取质量分布
func (s *Server) apiQualityDistribution(w http.ResponseWriter, r *http.Request) {
dist, err := s.storage.GetQualityDistribution()
if err != nil {
jsonError(w, err.Error(), http.StatusInternalServerError)
return
}
jsonOK(w, dist)
}
func jsonOK(w http.ResponseWriter, data interface{}) {
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(data)