mirror of
https://github.com/isboyjc/GoProxy.git
synced 2026-05-06 20:02:54 +08:00
feat: ✨ implement geo-filtering with whitelist and blacklist support
- Added support for geo-filtering in the proxy pool, allowing configuration of allowed and blocked countries via environment variables. - Updated `.env.example` and `docker-compose.yml` to include `ALLOWED_COUNTRIES` for whitelist functionality. - Enhanced `CLAUDE.md`, `GEO_FILTER.md`, and `README.md` to document the new geo-filtering features and usage instructions. - Modified proxy validation logic to prioritize whitelist over blacklist during admission checks. - Improved WebUI to allow dynamic configuration of geo-filter settings.
This commit is contained in:
@@ -14,6 +14,11 @@ SOCKS5_STABLE_PORT=7780 # SOCKS5 最低延迟代理端口
|
||||
# 默认屏蔽中国大陆(CN),香港(HK)、澳门(MO)、台湾(TW)不受影响
|
||||
BLOCKED_COUNTRIES=CN
|
||||
|
||||
# 允许的国家代码白名单(逗号分隔,如 US,JP,KR,SG)
|
||||
# 非空时优先于黑名单(BLOCKED_COUNTRIES 被忽略)
|
||||
# 留空 = 使用黑名单模式
|
||||
ALLOWED_COUNTRIES=
|
||||
|
||||
# 代理服务认证配置
|
||||
# ⚠️ 代理端口默认对外开放,强烈建议启用认证!
|
||||
# 使用方式:curl -x http://username:password@host:port https://example.com
|
||||
|
||||
2
.gitignore
vendored
2
.gitignore
vendored
@@ -12,6 +12,8 @@ proxy-pool
|
||||
*.test
|
||||
vendor/
|
||||
|
||||
CUSTOM_PROXY_DESIGN.md
|
||||
|
||||
# Test binaries
|
||||
test/test_proxy
|
||||
test/test_proxy.exe
|
||||
|
||||
@@ -70,7 +70,8 @@ main.go (orchestrator)
|
||||
|
||||
- **Pool state machine**: healthy → warning → critical → emergency. State determines fetch mode (optimize/refill/emergency) and latency thresholds.
|
||||
- **Slot-based capacity**: Pool has fixed size split between HTTP/SOCKS5 by configurable ratio (default 3:7). Each protocol has guaranteed minimum slots.
|
||||
- **Smart admission**: New proxies enter if slots available, or replace worst existing proxy if significantly faster (30%+ by default via `ReplaceThreshold`). HTTP proxies must also pass an HTTPS CONNECT tunnel test (random real HTTPS site visit with retry) before admission.
|
||||
- **Smart admission**: New proxies enter if slots available, or replace worst existing proxy if significantly faster (30%+ by default via `ReplaceThreshold`). HTTP proxies must also pass an HTTPS CONNECT tunnel test (random real HTTPS site visit with retry) before admission. Geo-filter (whitelist/blacklist) is applied during validation before admission.
|
||||
- **Geo-filter**: Whitelist (`AllowedCountries`) takes priority — if non-empty, only those countries pass. Otherwise blacklist (`BlockedCountries`) rejects listed countries. Configurable via env vars, `config.json`, or WebUI at runtime. On startup, existing proxies violating the filter are cleaned from the database.
|
||||
- **Protocol-parallel validation**: `smartFetchAndFill` splits candidates by protocol and validates SOCKS5/HTTP concurrently. SOCKS5 fills faster (no HTTPS check overhead); HTTP validation runs in parallel without blocking SOCKS5 admission.
|
||||
- **Circuit breaker on sources**: `SourceManager` tracks consecutive failures per source URL. 3 fails → degraded, 5 → disabled for 30min.
|
||||
- **Auto-retry on proxy failure**: Both HTTP and SOCKS5 servers retry with different upstream proxies on failure (up to `MaxRetry` times), deleting failed proxies immediately.
|
||||
@@ -95,9 +96,10 @@ main.go (orchestrator)
|
||||
|
||||
### Configuration
|
||||
|
||||
- Environment variables: `WEBUI_PASSWORD`, `PROXY_AUTH_ENABLED`, `PROXY_AUTH_USERNAME`, `PROXY_AUTH_PASSWORD`, `BLOCKED_COUNTRIES`, `DATA_DIR`
|
||||
- Persistent config: `config.json` (or `$DATA_DIR/config.json`) — pool capacity, latency thresholds, intervals. Editable via WebUI.
|
||||
- Environment variables: `WEBUI_PASSWORD`, `PROXY_AUTH_ENABLED`, `PROXY_AUTH_USERNAME`, `PROXY_AUTH_PASSWORD`, `BLOCKED_COUNTRIES`, `ALLOWED_COUNTRIES`, `DATA_DIR`
|
||||
- Persistent config: `config.json` (or `$DATA_DIR/config.json`) — pool capacity, latency thresholds, intervals, geo-filter (blocked/allowed countries). Editable via WebUI.
|
||||
- Config is loaded once at startup via `config.Load()`, updated in-memory via `config.Save()`. Thread-safe via `sync.RWMutex`.
|
||||
- Geo-filter: `ALLOWED_COUNTRIES` (whitelist) takes priority over `BLOCKED_COUNTRIES` (blacklist). When whitelist is non-empty, only listed countries are admitted; blacklist is ignored. Both are comma-separated country codes (e.g. `US,JP,KR`). Configurable at runtime via WebUI; `config.json` values override env vars after first save.
|
||||
|
||||
### Storage
|
||||
|
||||
|
||||
120
GEO_FILTER.md
120
GEO_FILTER.md
@@ -1,14 +1,25 @@
|
||||
# 地理过滤配置指南
|
||||
|
||||
GoProxy 支持通过国家代码过滤代理的出口位置,让你可以灵活控制代理池的地理分布。
|
||||
GoProxy 支持通过国家代码过滤代理的出口位置,让你可以灵活控制代理池的地理分布。支持黑名单(屏蔽指定国家)和白名单(仅允许指定国家)两种模式。
|
||||
|
||||
## 🌍 配置方式
|
||||
|
||||
### 过滤模式
|
||||
|
||||
GoProxy 提供两种互斥的过滤模式:
|
||||
|
||||
| 模式 | 环境变量 | 说明 |
|
||||
|------|---------|------|
|
||||
| 黑名单 | `BLOCKED_COUNTRIES` | 屏蔽指定国家,其余放行(默认 `CN`) |
|
||||
| 白名单 | `ALLOWED_COUNTRIES` | 仅允许指定国家,其余拒绝 |
|
||||
|
||||
> **优先级**:白名单非空时生效,黑名单被忽略。白名单为空时黑名单生效。
|
||||
|
||||
### 环境变量配置
|
||||
|
||||
通过 `BLOCKED_COUNTRIES` 环境变量设置需要屏蔽的国家代码:
|
||||
|
||||
```bash
|
||||
# === 黑名单模式(默认) ===
|
||||
|
||||
# 默认:屏蔽中国大陆(CN)
|
||||
BLOCKED_COUNTRIES=CN
|
||||
|
||||
@@ -17,15 +28,32 @@ BLOCKED_COUNTRIES=CN,RU,KP,IR
|
||||
|
||||
# 不屏蔽任何国家(留空)
|
||||
BLOCKED_COUNTRIES=
|
||||
|
||||
# === 白名单模式 ===
|
||||
|
||||
# 仅允许美国、日本、韩国、新加坡的代理入池
|
||||
ALLOWED_COUNTRIES=US,JP,KR,SG
|
||||
|
||||
# 仅允许欧美代理
|
||||
ALLOWED_COUNTRIES=US,CA,GB,DE,FR,NL,SE
|
||||
```
|
||||
|
||||
### WebUI 动态配置
|
||||
|
||||
管理员登录 WebUI 后,在配置面板的「地理过滤」区域可以动态修改黑名单和白名单,保存后立即生效,无需重启。
|
||||
|
||||
> **配置优先级**:WebUI 保存(config.json)> 环境变量 > 默认值。首次启动时环境变量生效,一旦通过 WebUI 保存过,后续以 config.json 为准。
|
||||
|
||||
### Docker Compose 配置
|
||||
|
||||
编辑 `.env` 文件:
|
||||
|
||||
```bash
|
||||
# 屏蔽中国大陆和俄罗斯
|
||||
# 黑名单模式:屏蔽中国大陆和俄罗斯
|
||||
BLOCKED_COUNTRIES=CN,RU
|
||||
|
||||
# 或白名单模式:仅允许美日韩新
|
||||
ALLOWED_COUNTRIES=US,JP,KR,SG
|
||||
```
|
||||
|
||||
启动服务:
|
||||
@@ -36,36 +64,71 @@ docker compose up -d
|
||||
### Docker Run 配置
|
||||
|
||||
```bash
|
||||
# 黑名单模式
|
||||
docker run -d --name proxygo \
|
||||
-p 127.0.0.1:7776:7776 -p 127.0.0.1:7777:7777 -p 7778:7778 \
|
||||
-e BLOCKED_COUNTRIES=CN,RU \
|
||||
-e WEBUI_PASSWORD=your_password \
|
||||
-v "$(pwd)/data:/app/data" \
|
||||
ghcr.io/isboyjc/goproxy:latest
|
||||
|
||||
# 白名单模式
|
||||
docker run -d --name proxygo \
|
||||
-p 127.0.0.1:7776:7776 -p 127.0.0.1:7777:7777 -p 7778:7778 \
|
||||
-e ALLOWED_COUNTRIES=US,JP,KR,SG \
|
||||
-e WEBUI_PASSWORD=your_password \
|
||||
-v "$(pwd)/data:/app/data" \
|
||||
ghcr.io/isboyjc/goproxy:latest
|
||||
```
|
||||
|
||||
### 本地运行配置
|
||||
|
||||
```bash
|
||||
# 黑名单模式
|
||||
export BLOCKED_COUNTRIES=CN,RU,KP
|
||||
go run .
|
||||
|
||||
# 白名单模式
|
||||
export ALLOWED_COUNTRIES=US,JP,KR,SG
|
||||
go run .
|
||||
```
|
||||
|
||||
## 🗺️ 工作机制
|
||||
|
||||
### 过滤逻辑
|
||||
|
||||
```
|
||||
代理验证时:
|
||||
if 白名单非空:
|
||||
出口国家在白名单中 → 放行
|
||||
出口国家不在白名单中 → 拒绝
|
||||
else if 黑名单非空:
|
||||
出口国家在黑名单中 → 拒绝
|
||||
出口国家不在黑名单中 → 放行
|
||||
else:
|
||||
全部放行
|
||||
```
|
||||
|
||||
### 双重过滤
|
||||
|
||||
地理过滤在两个阶段生效:
|
||||
|
||||
**1. 启动清理阶段**
|
||||
- 程序启动时自动扫描数据库
|
||||
- 删除所有屏蔽国家出口的代理
|
||||
- 日志输出:`🧹 已清理 X 个屏蔽国家出口代理 (屏蔽: [CN RU])`
|
||||
- 白名单模式:删除所有不在白名单中的代理
|
||||
- 黑名单模式:删除所有屏蔽国家出口的代理
|
||||
- 日志输出示例:
|
||||
- `🧹 已清理 X 个非白名单国家出口代理 (允许: [US JP KR])`
|
||||
- `🧹 已清理 X 个屏蔽国家出口代理 (屏蔽: [CN RU])`
|
||||
|
||||
**2. 验证阶段**
|
||||
- 新抓取的代理在验证时检查出口位置
|
||||
- 如果出口国家在屏蔽列表中,直接拒绝入池
|
||||
- 不会占用池子容量
|
||||
- 根据当前过滤模式决定是否允许入池
|
||||
- 不符合条件的代理不会占用池子容量
|
||||
|
||||
**3. 运行时更新**
|
||||
- 通过 WebUI 修改过滤配置后立即生效
|
||||
- 已在池中的代理会在下一轮健康检查时自然淘汰
|
||||
|
||||
### 国家代码识别
|
||||
|
||||
@@ -80,7 +143,7 @@ go run .
|
||||
RU Moscow → 国家代码 RU(俄罗斯)
|
||||
```
|
||||
|
||||
匹配规则:`exit_location LIKE 'CC %'`(国家代码 + 空格 + 城市)
|
||||
匹配规则:提取 `exit_location` 前两个字符作为国家代码进行匹配
|
||||
|
||||
## 📋 常用国家代码
|
||||
|
||||
@@ -135,14 +198,28 @@ BLOCKED_COUNTRIES=CN,RU,KP,IR,SY
|
||||
- 地缘政治考虑
|
||||
- 防止特定地区的代理质量问题
|
||||
|
||||
### 场景 3:仅使用欧美代理
|
||||
### 场景 3:仅使用欧美代理(白名单模式)
|
||||
|
||||
```bash
|
||||
# 屏蔽亚洲、非洲、中东等地区(示例,需根据实际需求调整)
|
||||
BLOCKED_COUNTRIES=CN,IN,TH,VN,ID,PH,BD,PK,IR,IQ,SA,EG,NG,ZA
|
||||
ALLOWED_COUNTRIES=US,CA,GB,DE,FR,NL,SE
|
||||
```
|
||||
|
||||
### 场景 4:不做地理限制
|
||||
**适用**:
|
||||
- 需要精确控制代理来源国家
|
||||
- 只需要特定地区的 IP
|
||||
- 比黑名单排除大量国家更简洁
|
||||
|
||||
### 场景 4:仅使用亚太代理(白名单模式)
|
||||
|
||||
```bash
|
||||
ALLOWED_COUNTRIES=JP,KR,SG,HK,TW
|
||||
```
|
||||
|
||||
**适用**:
|
||||
- 需要亚太地区低延迟代理
|
||||
- 针对亚太区域的业务场景
|
||||
|
||||
### 场景 5:不做地理限制
|
||||
|
||||
```bash
|
||||
BLOCKED_COUNTRIES=
|
||||
@@ -189,9 +266,11 @@ sqlite3 data/proxy.db "
|
||||
|
||||
1. **大小写不敏感**:国家代码会自动转为大写(`cn` → `CN`)
|
||||
2. **空格自动处理**:前后空格会自动去除
|
||||
3. **重启生效**:修改配置后需要重启服务
|
||||
4. **已有代理清理**:启动时会清理数据库中的屏蔽国家代理
|
||||
5. **香港独立识别**:
|
||||
3. **白名单优先**:白名单非空时黑名单被忽略
|
||||
4. **运行时可调**:通过 WebUI 修改后立即生效,无需重启
|
||||
5. **已有代理处理**:配置变更后,已入池代理在下一轮健康检查时自然淘汰
|
||||
6. **持久化**:通过 WebUI 保存的配置写入 config.json,重启后优先于环境变量
|
||||
7. **香港独立识别**:
|
||||
- 中国大陆代码:`CN`
|
||||
- 香港代码:`HK`(独立的国家代码)
|
||||
- 设置 `BLOCKED_COUNTRIES=CN` 不会影响香港代理
|
||||
@@ -236,7 +315,8 @@ go run .
|
||||
## 💡 最佳实践
|
||||
|
||||
1. **默认配置**:保持默认 `BLOCKED_COUNTRIES=CN`,适合大多数场景
|
||||
2. **生产环境**:根据业务合规要求设置屏蔽国家
|
||||
3. **测试环境**:可以设置为空(`BLOCKED_COUNTRIES=`)以获取更多代理
|
||||
4. **定期调整**:根据代理质量和可用性调整屏蔽列表
|
||||
5. **配合筛选**:利用 WebUI 的国家筛选器查看各国代理分布,辅助决策
|
||||
2. **精确控制**:需要特定国家代理时,使用白名单模式(`ALLOWED_COUNTRIES`)比排除大量国家更简洁
|
||||
3. **生产环境**:根据业务合规要求设置过滤规则
|
||||
4. **测试环境**:可以两个都留空以获取更多代理
|
||||
5. **动态调整**:通过 WebUI 实时调整过滤规则,观察效果后再决定最终配置
|
||||
6. **配合筛选**:利用 WebUI 的国家筛选器查看各国代理分布,辅助决策
|
||||
|
||||
35
README.md
35
README.md
@@ -174,7 +174,7 @@ go build -o proxygo .
|
||||
程序启动后会:
|
||||
1. 加载配置(环境变量 + `config.json`)
|
||||
2. 初始化数据库和限流器
|
||||
3. 清理不符合条件的代理(屏蔽国家出口、无地理信息)
|
||||
3. 清理不符合条件的代理(不符合地理过滤规则、无地理信息)
|
||||
4. 启动 WebUI(`:7778`)
|
||||
5. 立即执行智能填充(按需抓取 + 严格验证)
|
||||
6. 启动后台协程:
|
||||
@@ -476,6 +476,7 @@ docker compose up -d
|
||||
| 变量 | 默认值 | 说明 |
|
||||
|------|--------|------|
|
||||
| `BLOCKED_COUNTRIES` | `CN` | 屏蔽的国家代码(逗号分隔,如 `CN,RU`,留空=不屏蔽) |
|
||||
| `ALLOWED_COUNTRIES` | 空 | 允许的国家代码白名单(非空时优先于黑名单,如 `US,JP,KR`) |
|
||||
| `PROXY_AUTH_ENABLED` | `false` | 是否启用代理认证(对外开放时强烈建议启用) |
|
||||
| `PROXY_AUTH_USERNAME` | `proxy` | 代理认证用户名 |
|
||||
| `PROXY_AUTH_PASSWORD` | 空 | 代理认证密码 |
|
||||
@@ -516,7 +517,14 @@ WEBUI_PASSWORD=admin_pass
|
||||
EOF
|
||||
docker compose up -d
|
||||
|
||||
# 场景 4:不屏蔽任何国家
|
||||
# 场景 4:白名单模式(仅允许指定国家)
|
||||
cat > .env << EOF
|
||||
ALLOWED_COUNTRIES=US,JP,KR,SG
|
||||
WEBUI_PASSWORD=admin_pass
|
||||
EOF
|
||||
docker compose up -d
|
||||
|
||||
# 场景 5:不做地理限制
|
||||
cat > .env << EOF
|
||||
BLOCKED_COUNTRIES=
|
||||
WEBUI_PASSWORD=admin_pass
|
||||
@@ -725,6 +733,7 @@ proxies = {'http': 'socks5://myuser:secure_pass_123@server-ip:7779', 'https': 's
|
||||
| `PROXY_AUTH_USERNAME` | `proxy` | 代理认证用户名 |
|
||||
| `PROXY_AUTH_PASSWORD` | 空 | 代理认证密码(原始密码,自动哈希) |
|
||||
| `BLOCKED_COUNTRIES` | `CN` | 屏蔽的国家代码(逗号分隔,如 `CN,RU,KP`,留空=不屏蔽) |
|
||||
| `ALLOWED_COUNTRIES` | 空 | 允许的国家代码白名单(非空时优先于黑名单,如 `US,JP,KR`) |
|
||||
|
||||
## 🎨 WebUI 使用指南
|
||||
|
||||
@@ -1043,22 +1052,24 @@ A:
|
||||
|
||||
### Q: 如何配置地理过滤?
|
||||
A:
|
||||
通过 `BLOCKED_COUNTRIES` 环境变量配置需要屏蔽的国家:
|
||||
支持黑名单和白名单两种模式,白名单优先:
|
||||
|
||||
```bash
|
||||
# 默认屏蔽中国大陆(CN)
|
||||
BLOCKED_COUNTRIES=CN
|
||||
# === 黑名单模式(默认) ===
|
||||
BLOCKED_COUNTRIES=CN # 屏蔽中国大陆
|
||||
BLOCKED_COUNTRIES=CN,RU,KP # 屏蔽多个国家
|
||||
BLOCKED_COUNTRIES= # 不屏蔽任何国家
|
||||
|
||||
# 屏蔽多个国家(逗号分隔)
|
||||
BLOCKED_COUNTRIES=CN,RU,KP
|
||||
|
||||
# 不屏蔽任何国家(留空)
|
||||
BLOCKED_COUNTRIES=
|
||||
# === 白名单模式(优先于黑名单) ===
|
||||
ALLOWED_COUNTRIES=US,JP,KR,SG # 仅允许这些国家入池
|
||||
```
|
||||
|
||||
也可以通过 WebUI 配置面板的「地理过滤」区域动态修改,保存后立即生效。
|
||||
|
||||
**工作机制**:
|
||||
- **验证阶段**:检测到屏蔽国家出口直接拒绝入池
|
||||
- **启动清理**:自动删除数据库中屏蔽国家的代理
|
||||
- **白名单优先**:`ALLOWED_COUNTRIES` 非空时,仅允许白名单国家入池,黑名单被忽略
|
||||
- **验证阶段**:新代理验证时检查出口国家,不符合条件的直接拒绝
|
||||
- **启动清理**:自动删除数据库中不符合过滤规则的代理
|
||||
- **精确匹配**:使用 ISO 3166-1 alpha-2 国家代码(CN、HK、US 等)
|
||||
|
||||
**常用国家代码**:`CN`=中国大陆 | `HK`=香港 | `RU`=俄罗斯 | `US`=美国 | `JP`=日本 | `SG`=新加坡
|
||||
|
||||
@@ -48,6 +48,7 @@ type Config struct {
|
||||
|
||||
// 地理过滤配置
|
||||
BlockedCountries []string // 屏蔽的国家代码列表(如 ["CN", "RU"],默认 ["CN"])
|
||||
AllowedCountries []string // 允许的国家代码列表(白名单,非空时优先于黑名单)
|
||||
|
||||
// SQLite 数据库路径
|
||||
DBPath string
|
||||
@@ -139,6 +140,19 @@ func DefaultConfig() *Config {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 读取白名单配置(白名单非空时优先于黑名单)
|
||||
var allowedCountries []string
|
||||
if allowedEnv := os.Getenv("ALLOWED_COUNTRIES"); allowedEnv != "" {
|
||||
countries := strings.Split(allowedEnv, ",")
|
||||
allowedCountries = make([]string, 0, len(countries))
|
||||
for _, c := range countries {
|
||||
c = strings.TrimSpace(strings.ToUpper(c))
|
||||
if c != "" {
|
||||
allowedCountries = append(allowedCountries, c)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return &Config{
|
||||
// 基础服务配置
|
||||
@@ -158,6 +172,7 @@ func DefaultConfig() *Config {
|
||||
|
||||
// 地理过滤配置
|
||||
BlockedCountries: blockedCountries,
|
||||
AllowedCountries: allowedCountries,
|
||||
|
||||
// 池子容量配置
|
||||
PoolMaxSize: 100, // 总容量
|
||||
@@ -264,6 +279,14 @@ func Load() *Config {
|
||||
if saved.CheckInterval > 0 {
|
||||
cfg.CheckInterval = saved.CheckInterval
|
||||
}
|
||||
|
||||
// 地理过滤配置(config.json 优先于环境变量)
|
||||
if saved.BlockedCountries != nil {
|
||||
cfg.BlockedCountries = saved.BlockedCountries
|
||||
}
|
||||
if saved.AllowedCountries != nil {
|
||||
cfg.AllowedCountries = saved.AllowedCountries
|
||||
}
|
||||
}
|
||||
}
|
||||
cfgMu.Lock()
|
||||
@@ -303,6 +326,10 @@ type savedConfig struct {
|
||||
OptimizeInterval int `json:"optimize_interval"`
|
||||
ReplaceThreshold float64 `json:"replace_threshold"`
|
||||
|
||||
// 地理过滤配置
|
||||
BlockedCountries []string `json:"blocked_countries,omitempty"`
|
||||
AllowedCountries []string `json:"allowed_countries,omitempty"`
|
||||
|
||||
// 兼容旧配置
|
||||
FetchInterval int `json:"fetch_interval,omitempty"`
|
||||
CheckInterval int `json:"check_interval,omitempty"`
|
||||
@@ -327,6 +354,8 @@ func Save(cfg *Config) error {
|
||||
HealthCheckBatchSize: cfg.HealthCheckBatchSize,
|
||||
OptimizeInterval: cfg.OptimizeInterval,
|
||||
ReplaceThreshold: cfg.ReplaceThreshold,
|
||||
BlockedCountries: cfg.BlockedCountries,
|
||||
AllowedCountries: cfg.AllowedCountries,
|
||||
FetchInterval: cfg.FetchInterval,
|
||||
CheckInterval: cfg.CheckInterval,
|
||||
}, "", " ")
|
||||
|
||||
@@ -25,6 +25,7 @@ services:
|
||||
- PROXY_AUTH_USERNAME=${PROXY_AUTH_USERNAME:-proxy}
|
||||
- PROXY_AUTH_PASSWORD=${PROXY_AUTH_PASSWORD}
|
||||
- BLOCKED_COUNTRIES=${BLOCKED_COUNTRIES:-CN}
|
||||
- ALLOWED_COUNTRIES=${ALLOWED_COUNTRIES}
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-qO-", "http://localhost:7778/"]
|
||||
interval: 30s
|
||||
|
||||
@@ -20,21 +20,25 @@ type Source struct {
|
||||
|
||||
// 快速更新源(5-30分钟更新)- 用于紧急和补充模式
|
||||
var fastUpdateSources = []Source{
|
||||
// proxifly - 每5分钟更新
|
||||
{"https://cdn.jsdelivr.net/gh/proxifly/free-proxy-list@main/proxies/http/data.txt", "http"},
|
||||
{"https://cdn.jsdelivr.net/gh/proxifly/free-proxy-list@main/proxies/socks4/data.txt", "socks5"},
|
||||
{"https://cdn.jsdelivr.net/gh/proxifly/free-proxy-list@main/proxies/socks5/data.txt", "socks5"},
|
||||
// ProxyScraper - 每30分钟更新
|
||||
{"https://raw.githubusercontent.com/ProxyScraper/ProxyScraper/main/http.txt", "http"},
|
||||
{"https://raw.githubusercontent.com/ProxyScraper/ProxyScraper/main/socks4.txt", "socks5"},
|
||||
{"https://raw.githubusercontent.com/ProxyScraper/ProxyScraper/main/socks5.txt", "socks5"},
|
||||
// monosans - 每小时更新
|
||||
{"https://raw.githubusercontent.com/monosans/proxy-list/main/proxies/http.txt", "http"},
|
||||
// prxchk - 频繁更新
|
||||
{"https://raw.githubusercontent.com/prxchk/proxy-list/main/http.txt", "http"},
|
||||
{"https://raw.githubusercontent.com/prxchk/proxy-list/main/socks5.txt", "socks5"},
|
||||
{"https://raw.githubusercontent.com/prxchk/proxy-list/main/socks4.txt", "socks5"},
|
||||
// sunny9577 - 自动抓取更新
|
||||
{"https://cdn.jsdelivr.net/gh/sunny9577/proxy-scraper/generated/http_proxies.txt", "http"},
|
||||
{"https://cdn.jsdelivr.net/gh/sunny9577/proxy-scraper/generated/socks5_proxies.txt", "socks5"},
|
||||
{"https://cdn.jsdelivr.net/gh/sunny9577/proxy-scraper/generated/socks4_proxies.txt", "socks5"},
|
||||
}
|
||||
|
||||
// 慢速更新源(每天更新)- 用于优化轮换模式
|
||||
var slowUpdateSources = []Source{
|
||||
// TheSpeedX - 每天更新
|
||||
// TheSpeedX - 每天更新,量大
|
||||
{"https://raw.githubusercontent.com/TheSpeedX/SOCKS-List/master/http.txt", "http"},
|
||||
{"https://raw.githubusercontent.com/TheSpeedX/SOCKS-List/master/socks4.txt", "socks5"},
|
||||
{"https://raw.githubusercontent.com/TheSpeedX/SOCKS-List/master/socks5.txt", "socks5"},
|
||||
@@ -44,6 +48,25 @@ var slowUpdateSources = []Source{
|
||||
// databay-labs - 备用源
|
||||
{"https://cdn.jsdelivr.net/gh/databay-labs/free-proxy-list/http.txt", "http"},
|
||||
{"https://cdn.jsdelivr.net/gh/databay-labs/free-proxy-list/socks5.txt", "socks5"},
|
||||
// Anonym0usWork1221 - 量大质量尚可
|
||||
{"https://cdn.jsdelivr.net/gh/Anonym0usWork1221/Free-Proxies/proxy_files/http_proxies.txt", "http"},
|
||||
{"https://cdn.jsdelivr.net/gh/Anonym0usWork1221/Free-Proxies/proxy_files/socks5_proxies.txt", "socks5"},
|
||||
{"https://cdn.jsdelivr.net/gh/Anonym0usWork1221/Free-Proxies/proxy_files/socks4_proxies.txt", "socks5"},
|
||||
// ALIILAPRO
|
||||
{"https://cdn.jsdelivr.net/gh/ALIILAPRO/Proxy/http.txt", "http"},
|
||||
{"https://cdn.jsdelivr.net/gh/ALIILAPRO/Proxy/socks4.txt", "socks5"},
|
||||
// vakhov/fresh-proxy-list
|
||||
{"https://cdn.jsdelivr.net/gh/vakhov/fresh-proxy-list/http.txt", "http"},
|
||||
{"https://cdn.jsdelivr.net/gh/vakhov/fresh-proxy-list/socks5.txt", "socks5"},
|
||||
{"https://cdn.jsdelivr.net/gh/vakhov/fresh-proxy-list/socks4.txt", "socks5"},
|
||||
// Zaeem20
|
||||
{"https://cdn.jsdelivr.net/gh/Zaeem20/FREE_PROXIES_LIST/http.txt", "http"},
|
||||
{"https://cdn.jsdelivr.net/gh/Zaeem20/FREE_PROXIES_LIST/socks4.txt", "socks5"},
|
||||
// hookzof - socks5 专项
|
||||
{"https://cdn.jsdelivr.net/gh/hookzof/socks5_list/proxy.txt", "socks5"},
|
||||
// proxy4parsing
|
||||
{"https://cdn.jsdelivr.net/gh/proxy4parsing/proxy-list/http.txt", "http"},
|
||||
{"https://cdn.jsdelivr.net/gh/proxy4parsing/proxy-list/socks5.txt", "socks5"},
|
||||
}
|
||||
|
||||
// 所有源
|
||||
|
||||
9
main.go
9
main.go
@@ -59,7 +59,14 @@ func main() {
|
||||
|
||||
// 清理无效代理
|
||||
totalDeleted := 0
|
||||
if len(cfg.BlockedCountries) > 0 {
|
||||
if len(cfg.AllowedCountries) > 0 {
|
||||
// 白名单模式:清理不在白名单中的代理
|
||||
if deleted, err := store.DeleteNotAllowedCountries(cfg.AllowedCountries); err == nil && deleted > 0 {
|
||||
log.Printf("[main] 🧹 已清理 %d 个非白名单国家出口代理 (允许: %v)", deleted, cfg.AllowedCountries)
|
||||
totalDeleted += int(deleted)
|
||||
}
|
||||
} else if len(cfg.BlockedCountries) > 0 {
|
||||
// 黑名单模式:清理屏蔽国家的代理
|
||||
if deleted, err := store.DeleteBlockedCountries(cfg.BlockedCountries); err == nil && deleted > 0 {
|
||||
log.Printf("[main] 🧹 已清理 %d 个屏蔽国家出口代理 (屏蔽: %v)", deleted, cfg.BlockedCountries)
|
||||
totalDeleted += int(deleted)
|
||||
|
||||
@@ -110,6 +110,11 @@ func (m *Manager) NeedsFetch(status *PoolStatus) (bool, string, string) {
|
||||
httpPct := float64(status.HTTP) / float64(status.HTTPSlots)
|
||||
socks5Pct := float64(status.SOCKS5) / float64(status.SOCKS5Slots)
|
||||
|
||||
// 如果两个协议都缺(都<50%),同时补充两个协议
|
||||
if httpPct < 0.5 && socks5Pct < 0.5 {
|
||||
return true, "refill", ""
|
||||
}
|
||||
// 只有一个协议缺时,优先补充更缺的
|
||||
if httpPct < 0.5 {
|
||||
return true, "refill", "http"
|
||||
}
|
||||
|
||||
@@ -5,6 +5,7 @@ import (
|
||||
"fmt"
|
||||
"log"
|
||||
"math/rand"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
_ "github.com/mattn/go-sqlite3"
|
||||
@@ -621,12 +622,12 @@ func (s *Storage) DeleteBlockedCountries(countryCodes []string) (int64, error) {
|
||||
if len(countryCodes) == 0 {
|
||||
return 0, nil
|
||||
}
|
||||
|
||||
|
||||
var totalDeleted int64
|
||||
for _, code := range countryCodes {
|
||||
// exit_location 格式:如 "CN Beijing" 或 "HK Hong Kong"
|
||||
// 使用 LIKE 'CODE %' 来匹配国家代码(后面有空格表示有城市信息)
|
||||
res, err := s.db.Exec(`DELETE FROM proxies WHERE exit_location LIKE ?`, code+" %")
|
||||
// exit_location 格式:如 "CN Beijing" 或 "CN"(仅国家代码)
|
||||
// 同时匹配 "CODE" 和 "CODE ..." 两种情况
|
||||
res, err := s.db.Exec(`DELETE FROM proxies WHERE exit_location = ? OR exit_location LIKE ?`, code, code+" %")
|
||||
if err != nil {
|
||||
return totalDeleted, err
|
||||
}
|
||||
@@ -636,6 +637,29 @@ func (s *Storage) DeleteBlockedCountries(countryCodes []string) (int64, error) {
|
||||
return totalDeleted, nil
|
||||
}
|
||||
|
||||
// DeleteNotAllowedCountries 删除不在白名单中的代理
|
||||
func (s *Storage) DeleteNotAllowedCountries(allowedCodes []string) (int64, error) {
|
||||
if len(allowedCodes) == 0 {
|
||||
return 0, nil
|
||||
}
|
||||
|
||||
// 构建 WHERE 条件:exit_location 不以任何白名单国家代码开头
|
||||
// 即:NOT (exit_location = 'US' OR exit_location LIKE 'US %' OR ...)
|
||||
conditions := make([]string, 0, len(allowedCodes)*2)
|
||||
args := make([]interface{}, 0, len(allowedCodes)*2)
|
||||
for _, code := range allowedCodes {
|
||||
conditions = append(conditions, "exit_location = ?", "exit_location LIKE ?")
|
||||
args = append(args, code, code+" %")
|
||||
}
|
||||
|
||||
query := `DELETE FROM proxies WHERE exit_location != '' AND NOT (` + strings.Join(conditions, " OR ") + `)`
|
||||
res, err := s.db.Exec(query, args...)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return res.RowsAffected()
|
||||
}
|
||||
|
||||
// DeleteWithoutExitInfo 删除没有出口信息的代理
|
||||
func (s *Storage) DeleteWithoutExitInfo() (int64, error) {
|
||||
res, err := s.db.Exec(`DELETE FROM proxies WHERE exit_ip = '' OR exit_location = ''`)
|
||||
|
||||
@@ -208,13 +208,28 @@ func (v *Validator) ValidateOne(p storage.Proxy) (bool, time.Duration, string, s
|
||||
return false, latency, exitIP, exitLocation
|
||||
}
|
||||
|
||||
// 过滤屏蔽国家出口(根据配置)
|
||||
if v.cfg != nil && len(v.cfg.BlockedCountries) > 0 && len(exitLocation) >= 2 {
|
||||
// 地理过滤:白名单优先,否则走黑名单
|
||||
if v.cfg != nil && len(exitLocation) >= 2 {
|
||||
countryCode := exitLocation[:2]
|
||||
for _, blocked := range v.cfg.BlockedCountries {
|
||||
if countryCode == blocked {
|
||||
if len(v.cfg.AllowedCountries) > 0 {
|
||||
// 白名单模式:不在白名单中则拒绝
|
||||
allowed := false
|
||||
for _, a := range v.cfg.AllowedCountries {
|
||||
if countryCode == a {
|
||||
allowed = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !allowed {
|
||||
return false, latency, exitIP, exitLocation
|
||||
}
|
||||
} else if len(v.cfg.BlockedCountries) > 0 {
|
||||
// 黑名单模式
|
||||
for _, blocked := range v.cfg.BlockedCountries {
|
||||
if countryCode == blocked {
|
||||
return false, latency, exitIP, exitLocation
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -379,6 +379,22 @@ tr:hover{background:var(--gray-2);box-shadow:inset 0 0 20px rgba(0,255,65,0.05)}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="form-section">
|
||||
<div class="form-section-title" data-i18n="config.section_geo_filter">地理过滤</div>
|
||||
<div class="form-grid">
|
||||
<div class="form-group">
|
||||
<label data-i18n="config.allowed_countries">允许国家(白名单)</label>
|
||||
<input type="text" id="cfg-allowed-countries" placeholder="US,JP,KR,SG">
|
||||
<div class="form-help" data-i18n="config.allowed_countries_help">非空时仅允许这些国家入池,忽略黑名单</div>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label data-i18n="config.blocked_countries">屏蔽国家(黑名单)</label>
|
||||
<input type="text" id="cfg-blocked-countries" placeholder="CN,RU,KP">
|
||||
<div class="form-help" data-i18n="config.blocked_countries_help">白名单为空时生效</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="modal-actions">
|
||||
<button class="btn btn-secondary" onclick="closeSettings()" data-i18n="config.cancel">取消</button>
|
||||
<button class="btn" onclick="saveConfig()" data-i18n="config.save">保存配置</button>
|
||||
@@ -453,6 +469,11 @@ const i18n = {
|
||||
'config.optimize_interval': '优化间隔(分钟)',
|
||||
'config.replace_threshold': '替换阈值',
|
||||
'config.replace_threshold_help': '新代理需快30%',
|
||||
'config.section_geo_filter': '地理过滤',
|
||||
'config.allowed_countries': '允许国家(白名单)',
|
||||
'config.allowed_countries_help': '非空时仅允许这些国家入池,忽略黑名单',
|
||||
'config.blocked_countries': '屏蔽国家(黑名单)',
|
||||
'config.blocked_countries_help': '白名单为空时生效',
|
||||
'config.cancel': '取消',
|
||||
'config.save': '保存配置',
|
||||
'msg.fetch_confirm': '确定开始抓取代理吗?',
|
||||
@@ -527,6 +548,11 @@ const i18n = {
|
||||
'config.optimize_interval': 'Optimize Interval (min)',
|
||||
'config.replace_threshold': 'Replace Threshold',
|
||||
'config.replace_threshold_help': 'New proxy must be 30% faster',
|
||||
'config.section_geo_filter': 'Geo Filter',
|
||||
'config.allowed_countries': 'Allowed Countries (Whitelist)',
|
||||
'config.allowed_countries_help': 'When set, only these countries are allowed; blacklist is ignored',
|
||||
'config.blocked_countries': 'Blocked Countries (Blacklist)',
|
||||
'config.blocked_countries_help': 'Effective only when whitelist is empty',
|
||||
'config.cancel': 'Cancel',
|
||||
'config.save': 'Save Configuration',
|
||||
'msg.fetch_confirm': 'Start proxy fetch?',
|
||||
@@ -877,6 +903,8 @@ async function openSettings() {
|
||||
document.getElementById('cfg-health-batch').value = cfg.health_check_batch_size;
|
||||
document.getElementById('cfg-optimize-interval').value = cfg.optimize_interval;
|
||||
document.getElementById('cfg-replace-threshold').value = cfg.replace_threshold;
|
||||
document.getElementById('cfg-blocked-countries').value = (cfg.blocked_countries || []).join(',');
|
||||
document.getElementById('cfg-allowed-countries').value = (cfg.allowed_countries || []).join(',');
|
||||
|
||||
document.getElementById('settings-modal').classList.add('show');
|
||||
}
|
||||
@@ -899,6 +927,8 @@ async function saveConfig() {
|
||||
health_check_batch_size: parseInt(document.getElementById('cfg-health-batch').value),
|
||||
optimize_interval: parseInt(document.getElementById('cfg-optimize-interval').value),
|
||||
replace_threshold: parseFloat(document.getElementById('cfg-replace-threshold').value),
|
||||
blocked_countries: document.getElementById('cfg-blocked-countries').value.split(',').map(s => s.trim().toUpperCase()).filter(s => s),
|
||||
allowed_countries: document.getElementById('cfg-allowed-countries').value.split(',').map(s => s.trim().toUpperCase()).filter(s => s),
|
||||
};
|
||||
|
||||
const result = await api('/api/config/save', {
|
||||
|
||||
@@ -336,23 +336,27 @@ func (s *Server) apiConfig(w http.ResponseWriter, r *http.Request) {
|
||||
"pool_min_per_protocol": cfg.PoolMinPerProtocol,
|
||||
"pool_http_slots": httpSlots,
|
||||
"pool_socks5_slots": socks5Slots,
|
||||
|
||||
|
||||
// 延迟配置
|
||||
"max_latency_ms": cfg.MaxLatencyMs,
|
||||
"max_latency_emergency": cfg.MaxLatencyEmergency,
|
||||
"max_latency_healthy": cfg.MaxLatencyHealthy,
|
||||
|
||||
|
||||
// 验证配置
|
||||
"validate_concurrency": cfg.ValidateConcurrency,
|
||||
"validate_timeout": cfg.ValidateTimeout,
|
||||
|
||||
|
||||
// 健康检查配置
|
||||
"health_check_interval": cfg.HealthCheckInterval,
|
||||
"health_check_batch_size": cfg.HealthCheckBatchSize,
|
||||
|
||||
|
||||
// 优化配置
|
||||
"optimize_interval": cfg.OptimizeInterval,
|
||||
"replace_threshold": cfg.ReplaceThreshold,
|
||||
|
||||
// 地理过滤配置
|
||||
"blocked_countries": cfg.BlockedCountries,
|
||||
"allowed_countries": cfg.AllowedCountries,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -364,18 +368,20 @@ func (s *Server) apiConfigSave(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
|
||||
var req struct {
|
||||
PoolMaxSize int `json:"pool_max_size"`
|
||||
PoolHTTPRatio float64 `json:"pool_http_ratio"`
|
||||
PoolMinPerProtocol int `json:"pool_min_per_protocol"`
|
||||
MaxLatencyMs int `json:"max_latency_ms"`
|
||||
MaxLatencyEmergency int `json:"max_latency_emergency"`
|
||||
MaxLatencyHealthy int `json:"max_latency_healthy"`
|
||||
ValidateConcurrency int `json:"validate_concurrency"`
|
||||
ValidateTimeout int `json:"validate_timeout"`
|
||||
HealthCheckInterval int `json:"health_check_interval"`
|
||||
HealthCheckBatchSize int `json:"health_check_batch_size"`
|
||||
OptimizeInterval int `json:"optimize_interval"`
|
||||
ReplaceThreshold float64 `json:"replace_threshold"`
|
||||
PoolMaxSize int `json:"pool_max_size"`
|
||||
PoolHTTPRatio float64 `json:"pool_http_ratio"`
|
||||
PoolMinPerProtocol int `json:"pool_min_per_protocol"`
|
||||
MaxLatencyMs int `json:"max_latency_ms"`
|
||||
MaxLatencyEmergency int `json:"max_latency_emergency"`
|
||||
MaxLatencyHealthy int `json:"max_latency_healthy"`
|
||||
ValidateConcurrency int `json:"validate_concurrency"`
|
||||
ValidateTimeout int `json:"validate_timeout"`
|
||||
HealthCheckInterval int `json:"health_check_interval"`
|
||||
HealthCheckBatchSize int `json:"health_check_batch_size"`
|
||||
OptimizeInterval int `json:"optimize_interval"`
|
||||
ReplaceThreshold float64 `json:"replace_threshold"`
|
||||
BlockedCountries []string `json:"blocked_countries"`
|
||||
AllowedCountries []string `json:"allowed_countries"`
|
||||
}
|
||||
|
||||
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
||||
@@ -408,6 +414,8 @@ func (s *Server) apiConfigSave(w http.ResponseWriter, r *http.Request) {
|
||||
newCfg.HealthCheckBatchSize = req.HealthCheckBatchSize
|
||||
newCfg.OptimizeInterval = req.OptimizeInterval
|
||||
newCfg.ReplaceThreshold = req.ReplaceThreshold
|
||||
newCfg.BlockedCountries = req.BlockedCountries
|
||||
newCfg.AllowedCountries = req.AllowedCountries
|
||||
|
||||
if err := config.Save(&newCfg); err != nil {
|
||||
jsonError(w, "save config error: "+err.Error(), http.StatusInternalServerError)
|
||||
|
||||
Reference in New Issue
Block a user