fix: improve IP classification and source fallback

This commit is contained in:
Aimili
2026-08-27 03:16:44 +08:00
parent 635ebd4db0
commit f190d746ef
8 changed files with 341 additions and 83 deletions
+2 -2
View File
@@ -7,9 +7,9 @@ on:
workflow_dispatch:
inputs:
release_tag:
description: Existing formal tag to publish, for example v2.1.1
description: Existing formal tag to publish, for example v2.1.2
required: true
default: v2.1.1
default: v2.1.2
type: string
concurrency:
+2 -1
View File
@@ -2,7 +2,8 @@ name: Update VPNGate mirror
on:
schedule:
- cron: "*/15 * * * *"
# Offset from quarter-hour peaks because GitHub may delay busy schedules.
- cron: "7,22,37,52 * * * *"
workflow_dispatch:
push:
branches: [main]
+23 -43
View File
@@ -1,63 +1,43 @@
# AimiliVPN V2.1.1 正式版
# AimiliVPN V2.1.2 正式版
V2.1.1 是一次以稳定性、浏览器兼容性和 VPS 实际使用体验为重点的维护更新。本次版本已在 Ubuntu 22.04 VPS 上通过安装、节点获取、代理出口、节点切换、断开与恢复测试
V2.1.2 重点修复 IP 类型误判和部分 VPS 获取节点列表等待过久的问题,并增强 GitHub Pages、VPS 最近缓存和程序内置快照的回退体验
## Bug 修复
- 修复 Edge 和部分 Chromium 浏览器打开管理页面后高占用、卡顿甚至浏览器失去响应的问题:移除高成本的 `backdrop-filter`、固定背景和远程字体依赖,并为“减少动态效果”系统设置提供兼容处理
- 修复前端每 10 秒轮询时,即使节点和状态完全没有变化,仍会重新排序、生成表格 HTML 和刷新页面的问题;相同快照现在会直接跳过渲染
- 修复节点列表已经按每页 50 条分页,但分页栏始终隐藏,导致第 51 条之后的节点无法查看的问题
- 修复多个轮询请求可能重叠、页面切到后台后仍持续刷新,以及日志窗口重复拉取和渲染过多内容造成的卡顿
- 修复连接、断开、单节点检测、收藏和设置操作可能被连续点击重复提交的问题,并为网络请求增加合理的超时与失败恢复
- 修复弹窗关闭、键盘焦点、Esc、Tab 循环和移动端滚动在不同浏览器中的不一致行为
- 修复登录和管理密码被意外删除首尾空格的问题;前端和 `ml password` 现在保持一致
- 修复 SOCKS5 客户端未提供“无需认证”方法时,服务端仍错误接受连接的问题。
- 修复 `ml status` 一直显示旧版 `v2.0`,以及首次安装提示 5-30 秒但实际最多等待 90 秒的问题。
- 修复 `ml update` 重启服务后可能读取上一次连接留下的节点 ID、过早提示“已就绪”的竞态;现在会清理旧状态,并确认 OpenVPN 进程和 `tun0` 均已建立后才报告成功。
- 修复正式版 GitHub Actions 发布脚本的 Shell 块问题。
- 修复`proxy=true` 直接等同于机房 IP 的分类错误。住宅宽带用户运行 VPNGate 后可能被风险库标记为代理,但其网络归属仍然是住宅;现在代理属性与住宅/移动/机房类型分开保存
- 修复 Sony、Korea Telecom、JCOM、SK Broadband、KDDI、Cable TV 等消费宽带节点容易被误标为机房 IP 的问题
- 保留对真实机房网络的识别:`hosting=true` 仍直接判为机房;SoftEther、hosting、cloud、server、data center、VPS 等明确数据中心供应商特征也仍判为机房
- 修复升级后旧版错误 IP 分类缓存继续生效最多 7 天的问题。分类缓存加入版本号,V2.1.2 会自动重新检测旧缓存,不需要用户手动删除运行数据
- 修复只给少量连通性检测成功节点补充 IP 类型、节点表中大部分节点长期显示未知的问题。后台任务现在会批量补全整个节点列表,同时只合并运营商和分类字段,不覆盖连接、延迟或检测状态
- 修复 VPNGate 官方接口持续缓慢传输时可能突破原有 socket 超时、拖慢备用源切换的问题。每个网络节点源现在增加 6 秒总时限
- 修复官方 HTTPS 已超过总时限后仍继续等待同一主机 HTTP 的重复慢请求;超时后会直接尝试 GitHub Pages HTTPS。证书或 TLS 不兼容等非超时错误仍保留 HTTP 回退,兼容旧系统和不同 VPS 环境
## 性能与体验优化
## 节点源与镜像优化
- 节点接口继续移除不需要发送到浏览器的 OpenVPN 配置正文,约百个节点的响应保持在较小体积
- 节点请求自动合并;页面不可见时暂停普通轮询;节点表最多渲染 50 行,日志窗口最多渲染最近 300 行,服务端最多读取最近 500 条日志
- 国家多选、状态筛选、IP 类型筛选、收藏面板和分页均在前端即时响应,不需要重新请求节点数据
- 管理弹窗补齐标准对话框语义、可见焦点、焦点恢复和屏幕阅读器状态提示
- 登录、节点更新、代理检测、节点切换、断开、设置保存和日志接口统一加入超时控制,网络异常时不会无限等待。
- 安装器生成的账号、密码和安全路径改用 Python `secrets` 加密安全随机源。
## 文档与功能更新
- Web 管理端版本标识和更新检测升级为完整的 `V2.1.1 正式版`,仍只跟踪 GitHub `main` 稳定分支和正式 Release。
- README 重新整理快速安装、代理连接、VPS 推荐、项目推广和法律使用边界,删除重复的版本流水账与虚拟币赞助内容。
- 保留简体中文主文档,并提供 English、日本語、한국어说明。
- 正式版继续提供 HTTP 和 SOCKS5 两种代理协议,兼容不同 VPS 和客户端环境。
- 节点顺序保持为:VPNGate 官方 HTTPS、官方 HTTP、GitHub Pages HTTPS、GitHub Pages HTTP、VPS 最近有效缓存、程序内置初始快照
- GitHub Pages 定时同步从整刻 15 分钟调整为每小时第 7、22、37、52 分钟,降低 GitHub Actions 高峰期调度延迟概率
- GitHub Pages 与官方 HTTPS 获取到的快照继续执行相同的 CSV 字段、大小、Base64 和 OpenVPN 危险指令校验
- HTTP 节点源继续只作为兼容回退,不覆盖最后一次通过 HTTPS 获得的可信本地快照
## 验证结果
- Ubuntu 22.04 x86_64 VPS 一键安装成功,服务保持运行且无异常重启
- HTTP 与 SOCKS5 代理出口验证通过。
- 节点测试、切换、断开和恢复原节点的完整流程通过
- 本地 35 项单元测试、Python 编译、前端 JavaScript 语法、`install.sh` 语法和 Docker Compose 配置检查通过
- 99 个节点分页为 50/49,国家多选、空状态、收藏、设置、网关、日志和 Web 更新检测均通过浏览器交互测试。
- 11.5 秒空闲轮询观测期间,相同节点快照产生 0 次 DOM 变更。
- 390x844 手机视口和 1440x900 桌面视口没有页面级横向溢出或控件重叠。
- 40 项单元测试通过,覆盖住宅/代理分离、真实机房识别、旧缓存迁移、后台全量富化、连接状态保护和慢速官方源回退
- Python 编译、前端 JavaScript 语法、`install.sh` 语法和 Docker Compose 配置检查通过。
- 测试 VPS 上 VPNGate 官方 HTTPS/HTTP、GitHub Pages HTTPS/HTTP、VPS 最近缓存和内置快照均能下载、解析并生成候选节点
- 发布流水线对 Python 3.9、3.11、3.13 运行完整测试,并分别构建验证 `linux/amd64``linux/386``linux/arm64``linux/arm/v7`
## 下载与兼容架构
## 下载与更新
- GitHub Release 提供 `aimilivpn-v2.1.1-linux-source.tar.gz` 通用 Linux Python 源码包`sha256sums.txt`
- GHCR 发布 `2.1.1``2.1``latest` 三组镜像标签。
- Docker 镜像在发布前分别对 `linux/amd64``linux/386``linux/arm64``linux/arm/v7` 执行构建和导入冒烟测试。
- Docker 需要 Linux、`/dev/net/tun`、host 网络以及 `NET_ADMIN``NET_RAW` 能力。
- GitHub Release 提供 `aimilivpn-v2.1.2-linux-source.tar.gz``sha256sums.txt`
- GHCR 发布 `2.1.2``2.1``latest` 三组镜像标签。
## 更新方式
Python 源码安装:
Python 源码安装更新:
```bash
ml update
```
Docker Compose
Docker Compose 更新
```bash
docker compose pull
+1 -1
View File
@@ -1 +1 @@
2.1.1
2.1.2
+2 -2
View File
@@ -204,9 +204,9 @@ def get_app_version():
try:
with open(os.path.join(INSTALL_DIR, "VERSION"), "r", encoding="utf-8") as f:
version = f.read().strip().lstrip("vV")
return version or "2.1.1"
return version or "2.1.2"
except Exception:
return "2.1.1"
return "2.1.2"
def load_ui_cfg():
import json
+144 -6
View File
@@ -85,6 +85,8 @@ class ManagerLogicTests(unittest.TestCase):
mock.patch.object(manager, "API_CACHE_FILE", root / "api_snapshot.csv"),
mock.patch.object(manager, "API_CACHE_META_FILE", root / "api_snapshot.meta.json"),
mock.patch.object(manager, "BUNDLED_SNAPSHOT_FILE", root / "bundled_snapshot.csv"),
mock.patch.object(manager.vpn_utils, "DATA_DIR", root),
mock.patch.object(manager.vpn_utils, "IP_CACHE_FILE", root / "ip_cache.json"),
]
for patcher in self.path_patches:
patcher.start()
@@ -156,6 +158,142 @@ class ManagerLogicTests(unittest.TestCase):
self.assertEqual(5, sum(node.get("probe_status") == "available" for node in stored))
self.assertEqual(7, sum(node.get("probe_status") == "not_checked" for node in stored))
def test_ip_classification_separates_proxy_use_from_network_type(self) -> None:
residential, residential_reason = manager.vpn_utils.classify_ip_type(
{
"isp": "Sony Network Communications Inc.",
"org": "Sony Network Communications Inc.",
"proxy": True,
"hosting": False,
"mobile": False,
}
)
softether, softether_reason = manager.vpn_utils.classify_ip_type(
{
"isp": "SoftEther",
"org": "SoftEther Corporation",
"proxy": True,
"hosting": False,
"mobile": False,
}
)
hosting, hosting_reason = manager.vpn_utils.classify_ip_type(
{"proxy": True, "hosting": True, "mobile": False}
)
mobile, mobile_reason = manager.vpn_utils.classify_ip_type(
{"proxy": False, "hosting": False, "mobile": True}
)
self.assertEqual(("residential", "consumer_or_unclassified_network"), (residential, residential_reason))
self.assertEqual(("hosting", "proxy_provider_datacenter"), (softether, softether_reason))
self.assertEqual(("hosting", "hosting_flag"), (hosting, hosting_reason))
self.assertEqual(("mobile", "mobile_flag"), (mobile, mobile_reason))
def test_ip_enrichment_reclassifies_legacy_cache_and_keeps_proxy_quality(self) -> None:
ip = "118.240.250.95"
manager.vpn_utils.IP_CACHE_FILE.write_text(
json.dumps(
{
ip: {
"ip_type": "hosting",
"quality": "proxy",
"cached_at": 9999999999,
"classification_version": 1,
}
}
),
encoding="utf-8",
)
api_result = [
{
"status": "success",
"query": ip,
"country": "Japan",
"regionName": "Tokyo",
"city": "Tokyo",
"isp": "Sony Network Communications Inc.",
"org": "Sony Network Communications Inc.",
"as": "AS2527 Sony Network Communications Inc.",
"asname": "Sony Network Communications Inc.",
"proxy": True,
"hosting": False,
"mobile": False,
}
]
response = mock.MagicMock()
response.read.return_value = json.dumps(api_result).encode("utf-8")
response.__enter__.return_value = response
node = {"id": "sony", "ip": ip}
with mock.patch.object(manager.vpn_utils.urllib.request, "urlopen", return_value=response) as urlopen_mock:
manager.vpn_utils.enrich_ip_info([node])
self.assertEqual("residential", node["ip_type"])
self.assertEqual("proxy", node["quality"])
self.assertTrue(node["is_proxy"])
self.assertFalse(node["is_hosting"])
urlopen_mock.assert_called_once()
cache = json.loads(manager.vpn_utils.IP_CACHE_FILE.read_text(encoding="utf-8"))
self.assertEqual(manager.vpn_utils.IP_CLASSIFICATION_VERSION, cache[ip]["classification_version"])
def test_background_ip_enrichment_merges_metadata_without_replacing_status(self) -> None:
nodes = self.write_nodes(2)
nodes[0]["probe_status"] = "available"
manager.write_json(manager.NODES_FILE, nodes)
def fake_enrich(items):
for item in items:
item["ip_type"] = "residential"
item["quality"] = "proxy"
item["owner"] = "Consumer ISP"
item["is_proxy"] = True
with mock.patch.object(manager.vpn_utils, "enrich_ip_info", side_effect=fake_enrich):
changed = manager.enrich_stored_nodes()
stored = manager.read_nodes()
self.assertGreater(changed, 0)
self.assertEqual("available", next(node for node in stored if node["id"] == "node-0")["probe_status"])
self.assertTrue(all(node["ip_type"] == "residential" for node in stored))
def test_source_deadline_skips_same_host_http_and_uses_github_https(self) -> None:
csv_text = valid_snapshot()
def fake_fetch(url, verify_ssl=True, deadline_seconds=None):
if url == manager.API_HTTPS_URL:
raise manager.SourceDeadlineExceeded("slow official source")
if url == manager.MIRROR_HTTPS_URL:
return csv_text
raise AssertionError(f"unexpected source: {url}")
with (
mock.patch.object(manager, "fetch_api_text_with_deadline", side_effect=fake_fetch) as fetch_mock,
mock.patch.object(manager, "load_blacklist", return_value={}),
mock.patch.object(manager, "log_to_json"),
):
nodes = manager.fetch_candidates()
self.assertEqual(1, len(nodes))
self.assertEqual(
[manager.API_HTTPS_URL, manager.MIRROR_HTTPS_URL],
[call.args[0] for call in fetch_mock.call_args_list],
)
def test_source_deadline_limits_total_fetch_time(self) -> None:
def slow_fetch(url, verify_ssl=True):
threading.Event().wait(0.1)
return valid_snapshot()
with mock.patch.object(manager, "fetch_api_text", side_effect=slow_fetch):
started = manager.time.monotonic()
with self.assertRaises(manager.SourceDeadlineExceeded):
manager.fetch_api_text_with_deadline(
manager.API_HTTPS_URL,
deadline_seconds=0.01,
)
self.assertLess(manager.time.monotonic() - started, 0.08)
def test_node_probe_stops_after_systemic_openvpn_failure(self) -> None:
nodes = self.write_nodes(12)
@@ -461,8 +599,8 @@ class ManagerLogicTests(unittest.TestCase):
self.assertEqual(519, entries[-1]["index"])
def test_web_update_controls_only_expose_stable_main_channel(self) -> None:
self.assertEqual("2.1.1", manager.APP_VERSION)
self.assertEqual("V2.1.1 正式版", manager.APP_VERSION_LABEL)
self.assertEqual("2.1.2", manager.APP_VERSION)
self.assertEqual("V2.1.2 正式版", manager.APP_VERSION_LABEL)
self.assertIn("检测更新", manager.INDEX_HTML)
self.assertIn("/api/check_update", manager.INDEX_HTML)
self.assertIn("/tree/main", manager.INDEX_HTML)
@@ -496,7 +634,7 @@ class ManagerLogicTests(unittest.TestCase):
def test_release_workflow_uses_full_patch_version(self) -> None:
workflow_text = (manager.ROOT_DIR / ".github" / "workflows" / "release.yml").read_text(encoding="utf-8")
self.assertIn("default: v2.1.1", workflow_text)
self.assertIn("default: v2.1.2", workflow_text)
self.assertIn("AimiliVPN V$(tr -d '\\r\\n' < VERSION) 正式版", workflow_text)
self.assertNotIn("cut -d. -f1,2 VERSION", workflow_text)
@@ -523,8 +661,8 @@ class ManagerLogicTests(unittest.TestCase):
def test_latest_release_check_reports_current_formal_version(self) -> None:
release = {
"tag_name": "v2.1.1",
"name": "AimiliVPN V2.1.1 正式版",
"tag_name": "v2.1.2",
"name": "AimiliVPN V2.1.2 正式版",
"draft": False,
"prerelease": False,
}
@@ -532,7 +670,7 @@ class ManagerLogicTests(unittest.TestCase):
result = manager.check_latest_release()
self.assertFalse(result["update_available"])
self.assertEqual("V2.1.1 正式版", result["current_version_label"])
self.assertEqual("V2.1.2 正式版", result["current_version_label"])
def test_latest_release_check_reports_source_update_command(self) -> None:
release = {"tag_name": "v2.2.0", "draft": False, "prerelease": False}
+62 -23
View File
@@ -15,11 +15,18 @@ from typing import Any
ROOT_DIR = Path(__file__).resolve().parent
DATA_DIR = Path(os.environ["VPNGATE_DATA_DIR"]).resolve() if os.environ.get("VPNGATE_DATA_DIR") else ROOT_DIR / "vpngate_data"
IP_CACHE_FILE = DATA_DIR / "ip_cache.json"
IP_CLASSIFICATION_VERSION = 2
IP_CACHE_TTL_SECONDS = 7 * 24 * 3600
ip_cache_lock = threading.RLock()
physical_interface_lock = threading.Lock()
physical_interface_cache: tuple[str | None, float] = (None, 0.0)
DATACENTER_PROVIDER_PATTERN = re.compile(
r"(?:\b(?:cloud|colo|colocation|data[ -]?center|hosting|servers?|vps)\b|softether)",
re.IGNORECASE,
)
COUNTRY_TRANSLATIONS = {
"Japan": "日本",
"Korea Republic of": "韩国",
@@ -389,6 +396,39 @@ def save_ip_cache(cache: dict[str, dict[str, Any]]) -> None:
except Exception:
pass
def classify_ip_type(item: dict[str, Any]) -> tuple[str, str]:
"""Classify network ownership without confusing VPN use with hosting."""
if item.get("mobile"):
return "mobile", "mobile_flag"
if item.get("hosting"):
return "hosting", "hosting_flag"
provider_text = " ".join(
str(item.get(key) or "")
for key in ("isp", "org", "as", "asname")
)
if item.get("proxy") and DATACENTER_PROVIDER_PATTERN.search(provider_text):
return "hosting", "proxy_provider_datacenter"
# A residential volunteer running VPNGate is commonly marked as a proxy.
# Proxy use is retained in quality/is_proxy and must not change ownership.
return "residential", "consumer_or_unclassified_network"
def apply_ip_cache_entry(node: dict[str, Any], entry: dict[str, Any]) -> None:
for key in (
"owner",
"asn",
"as_name",
"location",
"ip_type",
"quality",
"is_proxy",
"is_hosting",
"is_mobile",
"ip_type_reason",
):
node[key] = entry.get(key, "")
def enrich_ip_info(nodes: list[dict[str, Any]]) -> None:
# 1. Read cache thread-safely
with ip_cache_lock:
@@ -401,14 +441,14 @@ def enrich_ip_info(nodes: list[dict[str, Any]]) -> None:
ip = node.get("ip") or node.get("remote_host")
if not ip:
continue
if ip in cache and now - cache[ip].get("cached_at", 0) < 7 * 24 * 3600:
cache_entry = cache.get(ip) if isinstance(cache.get(ip), dict) else None
if (
cache_entry
and cache_entry.get("classification_version") == IP_CLASSIFICATION_VERSION
and now - cache_entry.get("cached_at", 0) < IP_CACHE_TTL_SECONDS
):
cached = cache[ip]
node["owner"] = cached.get("owner", "")
node["asn"] = cached.get("asn", "")
node["as_name"] = cached.get("as_name", "")
node["location"] = cached.get("location", "")
node["ip_type"] = cached.get("ip_type", "")
node["quality"] = cached.get("quality", "")
apply_ip_cache_entry(node, cached)
else:
if ip not in ips_to_query:
ips_to_query.append(ip)
@@ -425,7 +465,10 @@ def enrich_ip_info(nodes: list[dict[str, Any]]) -> None:
request = urllib.request.Request(
"http://ip-api.com/batch?lang=zh-CN&fields=status,message,query,country,regionName,city,isp,org,as,asname,proxy,hosting,mobile",
data=payload,
headers={"Content-Type": "application/json", "User-Agent": "vpngate-manager/2.2"},
headers={
"Content-Type": "application/json",
"User-Agent": f"AimiliVPN-IP-Classifier/{IP_CLASSIFICATION_VERSION}",
},
method="POST",
)
try:
@@ -442,19 +485,15 @@ def enrich_ip_info(nodes: list[dict[str, Any]]) -> None:
if not query_ip:
continue
ip_type = "residential"
if item.get("mobile"):
ip_type = "mobile"
elif item.get("hosting") or item.get("proxy"):
ip_type = "hosting"
ip_type, ip_type_reason = classify_ip_type(item)
quality = "normal"
if item.get("proxy"):
quality = "proxy"
if item.get("mobile"):
quality = "mobile"
elif item.get("hosting"):
quality = "datacenter"
elif item.get("mobile"):
quality = "mobile"
elif item.get("proxy"):
quality = "proxy"
loc = " ".join(part for part in [item.get("country"), item.get("regionName"), item.get("city")] if part)
@@ -465,6 +504,11 @@ def enrich_ip_info(nodes: list[dict[str, Any]]) -> None:
"location": loc,
"ip_type": ip_type,
"quality": quality,
"is_proxy": bool(item.get("proxy")),
"is_hosting": bool(item.get("hosting")),
"is_mobile": bool(item.get("mobile")),
"ip_type_reason": ip_type_reason,
"classification_version": IP_CLASSIFICATION_VERSION,
"cached_at": now,
}
except Exception as e:
@@ -484,12 +528,7 @@ def enrich_ip_info(nodes: list[dict[str, Any]]) -> None:
ip = node.get("ip") or node.get("remote_host")
if ip in new_entries:
cached = new_entries[ip]
node["owner"] = cached.get("owner", "")
node["asn"] = cached.get("asn", "")
node["as_name"] = cached.get("as_name", "")
node["location"] = cached.get("location", "")
node["ip_type"] = cached.get("ip_type", "")
node["quality"] = cached.get("quality", "")
apply_ip_cache_entry(node, cached)
def diagnose_api_failure(api_url: str = "https://www.vpngate.net/api/iphone/") -> tuple[int, str]:
+105 -5
View File
@@ -105,6 +105,7 @@ CHECK_INTERVAL_SECONDS = env_int("CHECK_INTERVAL_SECONDS", 1260, 1)
TARGET_VALID_NODES = env_int("TARGET_VALID_NODES", 3, 1)
MAX_SCAN_ROWS = env_int("MAX_SCAN_ROWS", 300, 1)
API_FETCH_TIMEOUT_SECONDS = env_int("API_FETCH_TIMEOUT_SECONDS", 10, 1, 60)
API_SOURCE_DEADLINE_SECONDS = env_int("API_SOURCE_DEADLINE_SECONDS", 6, 2, 30)
OPENVPN_TEST_TIMEOUT_SECONDS = env_int("OPENVPN_TEST_TIMEOUT_SECONDS", 35, 1)
MANUAL_TEST_NODE_LIMIT = env_int("MANUAL_TEST_NODE_LIMIT", 5, 1, 20)
INITIAL_CONNECT_TEST_LIMIT = env_int("INITIAL_CONNECT_TEST_LIMIT", 10, 1, 50)
@@ -130,7 +131,7 @@ UPDATE_COMMAND = (
)
ROOT_DIR = Path(sys.executable).resolve().parent if globals().get("__compiled__") else Path(__file__).resolve().parent
DEFAULT_APP_VERSION = "2.1.1"
DEFAULT_APP_VERSION = "2.1.2"
try:
_version_text = (ROOT_DIR / "VERSION").read_text(encoding="utf-8").strip()
except OSError:
@@ -175,10 +176,27 @@ last_collector_heartbeat = 0.0
last_checker_heartbeat = 0.0
last_pinger_heartbeat = 0.0
server_start_time = time.time()
ip_enrichment_wakeup = threading.Event()
IP_ENRICHMENT_FIELDS = (
"owner",
"asn",
"as_name",
"location",
"ip_type",
"quality",
"is_proxy",
"is_hosting",
"is_mobile",
"ip_type_reason",
)
class ConnectionCancelled(RuntimeError):
pass
class SourceDeadlineExceeded(TimeoutError):
pass
def purge_expired_sessions(now: float | None = None) -> int:
current_time = time.time() if now is None else now
with lock:
@@ -776,6 +794,29 @@ def fetch_api_text(url: str | None = None, use_ssl_verify: bool = True) -> str:
with urllib.request.urlopen(request, timeout=API_FETCH_TIMEOUT_SECONDS) as response:
return read_limited(response).decode("utf-8", errors="replace")
def fetch_api_text_with_deadline(
url: str,
use_ssl_verify: bool = True,
deadline_seconds: int | None = None,
) -> str:
deadline = deadline_seconds or API_SOURCE_DEADLINE_SECONDS
result_queue: queue.Queue[tuple[bool, Any]] = queue.Queue(maxsize=1)
def worker() -> None:
try:
result_queue.put((True, fetch_api_text(url, use_ssl_verify)))
except BaseException as exc:
result_queue.put((False, exc))
threading.Thread(target=worker, daemon=True).start()
try:
ok, value = result_queue.get(timeout=deadline)
except queue.Empty as exc:
raise SourceDeadlineExceeded(f"节点源超过 {deadline} 秒总时限") from exc
if ok:
return str(value)
raise value
def parse_release_version(value: Any) -> tuple[int, int, int]:
match = re.search(r"(?i)(?:^|[^a-z0-9])v?(\d+)(?:\.(\d+))?(?:\.(\d+))?", str(value or "").strip())
if not match:
@@ -992,15 +1033,22 @@ def fetch_candidates() -> list[dict[str, Any]]:
load_ui_config().get("discovery_countries")
)
last_err: Exception | None = None
deadline_hosts: set[str] = set()
log_to_json("INFO", "Main", "开始按官方、GitHub Pages、本地缓存顺序拉取节点列表...")
for source_name, url in api_network_sources():
source_host = (urllib.parse.urlsplit(url).hostname or "").lower()
if url.startswith("http://") and source_host in deadline_hosts:
msg = f"跳过同主机慢速 HTTP 节点源 {source_name}: {url}"
print(f"[fetch_candidates] {msg}", flush=True)
log_to_json("WARNING", "Main", msg)
continue
try:
msg = f"尝试节点源 {source_name}: {url}"
print(f"[fetch_candidates] {msg}", flush=True)
log_to_json("INFO", "Main", msg)
api_text = fetch_api_text(url, True)
api_text = fetch_api_text_with_deadline(url, True)
rows = parse_vpngate_rows(api_text)
candidates = rows_to_candidates(rows, blacklist)
if not candidates:
@@ -1038,6 +1086,8 @@ def fetch_candidates() -> list[dict[str, Any]]:
return filtered_candidates
except Exception as e:
last_err = e
if isinstance(e, SourceDeadlineExceeded) and url.startswith("https://"):
deadline_hosts.add(source_host)
print(f"[fetch_candidates] 节点源 {source_name} 失败: {e}", flush=True)
log_to_json("WARNING", "Main", f"节点源 {source_name} 失败: {e}")
@@ -1528,6 +1578,50 @@ def sort_all_nodes(nodes: list[dict[str, Any]]) -> list[dict[str, Any]]:
)
return available_nodes + untested_nodes + unavailable_nodes
def enrich_stored_nodes() -> int:
"""Enrich every listed IP, then merge only metadata into the latest state."""
with lock:
snapshot = read_nodes()
if not snapshot:
return 0
vpn_utils.enrich_ip_info(snapshot)
enriched_by_id = {
str(node.get("id") or ""): node
for node in snapshot
if node.get("id") and node.get("ip_type")
}
if not enriched_by_id:
return 0
changed = 0
with lock:
current_nodes = read_nodes()
for current in current_nodes:
enriched = enriched_by_id.get(str(current.get("id") or ""))
if not enriched:
continue
for field in IP_ENRICHMENT_FIELDS:
new_value = enriched.get(field, "")
if current.get(field, "") != new_value:
current[field] = new_value
changed += 1
if changed:
write_json(NODES_FILE, sort_all_nodes(current_nodes))
return changed
def ip_enrichment_loop() -> None:
while True:
nodes_exist = bool(read_nodes())
if nodes_exist:
try:
enrich_stored_nodes()
except Exception as exc:
print(f"[IP 类型] 后台批量识别失败: {exc}", flush=True)
log_to_json("WARNING", "Main", f"后台批量识别 IP 类型失败: {exc}")
ip_enrichment_wakeup.wait(300 if nodes_exist else 5)
ip_enrichment_wakeup.clear()
def apply_routing_filters(
nodes: list[dict[str, Any]],
ui_cfg: dict[str, Any],
@@ -2314,6 +2408,10 @@ def maintain_valid_nodes(force: bool = False) -> str:
"location",
"ip_type",
"quality",
"is_proxy",
"is_hosting",
"is_mobile",
"ip_type_reason",
]:
if previous.get(key) not in (None, ""):
cand[key] = previous.get(key)
@@ -2332,6 +2430,7 @@ def maintain_valid_nodes(force: bool = False) -> str:
pass
write_json(NODES_FILE, merged)
ip_enrichment_wakeup.set()
initial_tested_ids: set[str] = set()
fast_results: list[dict[str, Any]] = []
@@ -3985,12 +4084,12 @@ INDEX_HTML = r"""<!doctype html>
<div class="dropdown">
<button id="github_btn" class="btn-primary" type="button" aria-expanded="false" aria-controls="github_dropdown" style="background: rgba(255, 255, 255, 0.08); border: 1px solid var(--border-color); color: var(--text-primary);">
<svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" viewBox="0 0 16 16" style="vertical-align: middle; margin-right: 4px;"><path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.012 8.012 0 0 0 16 8c0-4.42-3.58-8-8-8z"/></svg>
<span id="github_version_label">V2.1.1 正式版</span>
<span id="github_version_label">V2.1.2 正式版</span>
<svg xmlns="http://www.w3.org/2000/svg" style="width:12px; height:12px; margin-left: 2px;" fill="none" viewBox="0 0 24 24" stroke="currentColor" stroke-width="3"><path stroke-linecap="round" stroke-linejoin="round" d="M19 9l-7 7-7-7" /></svg>
</button>
<div id="github_dropdown" class="dropdown-content github-dropdown">
<div class="version-current">
<div id="current_version_label" class="version-current-label">V2.1.1 正式版</div>
<div id="current_version_label" class="version-current-label">V2.1.2 正式版</div>
<div id="deployment_mode_label" class="version-current-meta">Python 源码部署 · 更新通道main</div>
</div>
<button id="check_update_btn" type="button" onclick="checkForUpdate(event)">
@@ -4815,7 +4914,7 @@ function stableSortNodes() {
}
function render(){
const versionLabel = state.app_version_label || "V2.1.1 正式版";
const versionLabel = state.app_version_label || "V2.1.2 正式版";
if ($("github_version_label")) $("github_version_label").textContent = versionLabel;
if ($("current_version_label")) $("current_version_label").textContent = versionLabel;
if ($("deployment_mode_label")) {
@@ -7139,6 +7238,7 @@ def main() -> None:
print("[警告] 代理网关启动超时,继续执行脚本...", flush=True)
threading.Thread(target=collector_loop, daemon=True).start()
threading.Thread(target=ip_enrichment_loop, daemon=True).start()
threading.Thread(target=background_proxy_checker, daemon=True).start()
threading.Thread(target=active_node_pinger, daemon=True).start()