mirror of
https://github.com/baoweise-bot/aimili-vpngate.git
synced 2026-09-04 23:26:47 +08:00
fix: improve IP classification and source fallback
This commit is contained in:
@@ -7,9 +7,9 @@ on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
release_tag:
|
||||
description: Existing formal tag to publish, for example v2.1.1
|
||||
description: Existing formal tag to publish, for example v2.1.2
|
||||
required: true
|
||||
default: v2.1.1
|
||||
default: v2.1.2
|
||||
type: string
|
||||
|
||||
concurrency:
|
||||
|
||||
@@ -2,7 +2,8 @@ name: Update VPNGate mirror
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: "*/15 * * * *"
|
||||
# Offset from quarter-hour peaks because GitHub may delay busy schedules.
|
||||
- cron: "7,22,37,52 * * * *"
|
||||
workflow_dispatch:
|
||||
push:
|
||||
branches: [main]
|
||||
|
||||
+23
-43
@@ -1,63 +1,43 @@
|
||||
# AimiliVPN V2.1.1 正式版
|
||||
# AimiliVPN V2.1.2 正式版
|
||||
|
||||
V2.1.1 是一次以稳定性、浏览器兼容性和 VPS 实际使用体验为重点的维护更新。本次版本已在 Ubuntu 22.04 VPS 上通过安装、节点获取、代理出口、节点切换、断开与恢复测试。
|
||||
V2.1.2 重点修复 IP 类型误判和部分 VPS 获取节点列表等待过久的问题,并增强 GitHub Pages、VPS 最近缓存和程序内置快照的回退体验。
|
||||
|
||||
## Bug 修复
|
||||
|
||||
- 修复 Edge 和部分 Chromium 浏览器打开管理页面后高占用、卡顿甚至浏览器失去响应的问题:移除高成本的 `backdrop-filter`、固定背景和远程字体依赖,并为“减少动态效果”系统设置提供兼容处理。
|
||||
- 修复前端每 10 秒轮询时,即使节点和状态完全没有变化,仍会重新排序、生成表格 HTML 和刷新页面的问题;相同快照现在会直接跳过渲染。
|
||||
- 修复节点列表已经按每页 50 条分页,但分页栏始终隐藏,导致第 51 条之后的节点无法查看的问题。
|
||||
- 修复多个轮询请求可能重叠、页面切到后台后仍持续刷新,以及日志窗口重复拉取和渲染过多内容造成的卡顿。
|
||||
- 修复连接、断开、单节点检测、收藏和设置操作可能被连续点击重复提交的问题,并为网络请求增加合理的超时与失败恢复。
|
||||
- 修复弹窗关闭、键盘焦点、Esc、Tab 循环和移动端滚动在不同浏览器中的不一致行为。
|
||||
- 修复登录和管理密码被意外删除首尾空格的问题;前端和 `ml password` 现在保持一致。
|
||||
- 修复 SOCKS5 客户端未提供“无需认证”方法时,服务端仍错误接受连接的问题。
|
||||
- 修复 `ml status` 一直显示旧版 `v2.0`,以及首次安装提示 5-30 秒但实际最多等待 90 秒的问题。
|
||||
- 修复 `ml update` 重启服务后可能读取上一次连接留下的节点 ID、过早提示“已就绪”的竞态;现在会清理旧状态,并确认 OpenVPN 进程和 `tun0` 均已建立后才报告成功。
|
||||
- 修复正式版 GitHub Actions 发布脚本的 Shell 块问题。
|
||||
- 修复把 `proxy=true` 直接等同于机房 IP 的分类错误。住宅宽带用户运行 VPNGate 后可能被风险库标记为代理,但其网络归属仍然是住宅;现在代理属性与住宅/移动/机房类型分开保存。
|
||||
- 修复 Sony、Korea Telecom、JCOM、SK Broadband、KDDI、Cable TV 等消费宽带节点容易被误标为机房 IP 的问题。
|
||||
- 保留对真实机房网络的识别:`hosting=true` 仍直接判为机房;SoftEther、hosting、cloud、server、data center、VPS 等明确数据中心供应商特征也仍判为机房。
|
||||
- 修复升级后旧版错误 IP 分类缓存继续生效最多 7 天的问题。分类缓存加入版本号,V2.1.2 会自动重新检测旧缓存,不需要用户手动删除运行数据。
|
||||
- 修复只给少量连通性检测成功节点补充 IP 类型、节点表中大部分节点长期显示未知的问题。后台任务现在会批量补全整个节点列表,同时只合并运营商和分类字段,不覆盖连接、延迟或检测状态。
|
||||
- 修复 VPNGate 官方接口持续缓慢传输时可能突破原有 socket 超时、拖慢备用源切换的问题。每个网络节点源现在增加 6 秒总时限。
|
||||
- 修复官方 HTTPS 已超过总时限后仍继续等待同一主机 HTTP 的重复慢请求;超时后会直接尝试 GitHub Pages HTTPS。证书或 TLS 不兼容等非超时错误仍保留 HTTP 回退,兼容旧系统和不同 VPS 环境。
|
||||
|
||||
## 性能与体验优化
|
||||
## 节点源与镜像优化
|
||||
|
||||
- 节点接口继续移除不需要发送到浏览器的 OpenVPN 配置正文,约百个节点的响应保持在较小体积。
|
||||
- 节点请求自动合并;页面不可见时暂停普通轮询;节点表最多渲染 50 行,日志窗口最多渲染最近 300 行,服务端最多读取最近 500 条日志。
|
||||
- 国家多选、状态筛选、IP 类型筛选、收藏面板和分页均在前端即时响应,不需要重新请求节点数据。
|
||||
- 管理弹窗补齐标准对话框语义、可见焦点、焦点恢复和屏幕阅读器状态提示。
|
||||
- 登录、节点更新、代理检测、节点切换、断开、设置保存和日志接口统一加入超时控制,网络异常时不会无限等待。
|
||||
- 安装器生成的账号、密码和安全路径改用 Python `secrets` 加密安全随机源。
|
||||
|
||||
## 文档与功能更新
|
||||
|
||||
- Web 管理端版本标识和更新检测升级为完整的 `V2.1.1 正式版`,仍只跟踪 GitHub `main` 稳定分支和正式 Release。
|
||||
- README 重新整理快速安装、代理连接、VPS 推荐、项目推广和法律使用边界,删除重复的版本流水账与虚拟币赞助内容。
|
||||
- 保留简体中文主文档,并提供 English、日本語、한국어说明。
|
||||
- 正式版继续提供 HTTP 和 SOCKS5 两种代理协议,兼容不同 VPS 和客户端环境。
|
||||
- 节点顺序保持为:VPNGate 官方 HTTPS、官方 HTTP、GitHub Pages HTTPS、GitHub Pages HTTP、VPS 最近有效缓存、程序内置初始快照。
|
||||
- GitHub Pages 定时同步从整刻 15 分钟调整为每小时第 7、22、37、52 分钟,降低 GitHub Actions 高峰期调度延迟概率。
|
||||
- GitHub Pages 与官方 HTTPS 获取到的快照继续执行相同的 CSV 字段、大小、Base64 和 OpenVPN 危险指令校验。
|
||||
- HTTP 节点源继续只作为兼容回退,不覆盖最后一次通过 HTTPS 获得的可信本地快照。
|
||||
|
||||
## 验证结果
|
||||
|
||||
- Ubuntu 22.04 x86_64 VPS 一键安装成功,服务保持运行且无异常重启。
|
||||
- HTTP 与 SOCKS5 代理出口验证通过。
|
||||
- 节点测试、切换、断开和恢复原节点的完整流程通过。
|
||||
- 本地 35 项单元测试、Python 编译、前端 JavaScript 语法、`install.sh` 语法和 Docker Compose 配置检查通过。
|
||||
- 99 个节点分页为 50/49,国家多选、空状态、收藏、设置、网关、日志和 Web 更新检测均通过浏览器交互测试。
|
||||
- 11.5 秒空闲轮询观测期间,相同节点快照产生 0 次 DOM 变更。
|
||||
- 390x844 手机视口和 1440x900 桌面视口没有页面级横向溢出或控件重叠。
|
||||
- 40 项单元测试通过,覆盖住宅/代理分离、真实机房识别、旧缓存迁移、后台全量富化、连接状态保护和慢速官方源回退。
|
||||
- Python 编译、前端 JavaScript 语法、`install.sh` 语法和 Docker Compose 配置检查通过。
|
||||
- 测试 VPS 上 VPNGate 官方 HTTPS/HTTP、GitHub Pages HTTPS/HTTP、VPS 最近缓存和内置快照均能下载、解析并生成候选节点。
|
||||
- 发布流水线对 Python 3.9、3.11、3.13 运行完整测试,并分别构建验证 `linux/amd64`、`linux/386`、`linux/arm64`、`linux/arm/v7`。
|
||||
|
||||
## 下载与兼容架构
|
||||
## 下载与更新
|
||||
|
||||
- GitHub Release 提供 `aimilivpn-v2.1.1-linux-source.tar.gz` 通用 Linux Python 源码包和 `sha256sums.txt`。
|
||||
- GHCR 发布 `2.1.1`、`2.1`、`latest` 三组镜像标签。
|
||||
- Docker 镜像在发布前分别对 `linux/amd64`、`linux/386`、`linux/arm64`、`linux/arm/v7` 执行构建和导入冒烟测试。
|
||||
- Docker 需要 Linux、`/dev/net/tun`、host 网络以及 `NET_ADMIN`、`NET_RAW` 能力。
|
||||
- GitHub Release 提供 `aimilivpn-v2.1.2-linux-source.tar.gz` 和 `sha256sums.txt`。
|
||||
- GHCR 发布 `2.1.2`、`2.1`、`latest` 三组镜像标签。
|
||||
|
||||
## 更新方式
|
||||
|
||||
Python 源码安装:
|
||||
Python 源码安装更新:
|
||||
|
||||
```bash
|
||||
ml update
|
||||
```
|
||||
|
||||
Docker Compose:
|
||||
Docker Compose 更新:
|
||||
|
||||
```bash
|
||||
docker compose pull
|
||||
|
||||
+2
-2
@@ -204,9 +204,9 @@ def get_app_version():
|
||||
try:
|
||||
with open(os.path.join(INSTALL_DIR, "VERSION"), "r", encoding="utf-8") as f:
|
||||
version = f.read().strip().lstrip("vV")
|
||||
return version or "2.1.1"
|
||||
return version or "2.1.2"
|
||||
except Exception:
|
||||
return "2.1.1"
|
||||
return "2.1.2"
|
||||
|
||||
def load_ui_cfg():
|
||||
import json
|
||||
|
||||
+144
-6
@@ -85,6 +85,8 @@ class ManagerLogicTests(unittest.TestCase):
|
||||
mock.patch.object(manager, "API_CACHE_FILE", root / "api_snapshot.csv"),
|
||||
mock.patch.object(manager, "API_CACHE_META_FILE", root / "api_snapshot.meta.json"),
|
||||
mock.patch.object(manager, "BUNDLED_SNAPSHOT_FILE", root / "bundled_snapshot.csv"),
|
||||
mock.patch.object(manager.vpn_utils, "DATA_DIR", root),
|
||||
mock.patch.object(manager.vpn_utils, "IP_CACHE_FILE", root / "ip_cache.json"),
|
||||
]
|
||||
for patcher in self.path_patches:
|
||||
patcher.start()
|
||||
@@ -156,6 +158,142 @@ class ManagerLogicTests(unittest.TestCase):
|
||||
self.assertEqual(5, sum(node.get("probe_status") == "available" for node in stored))
|
||||
self.assertEqual(7, sum(node.get("probe_status") == "not_checked" for node in stored))
|
||||
|
||||
def test_ip_classification_separates_proxy_use_from_network_type(self) -> None:
|
||||
residential, residential_reason = manager.vpn_utils.classify_ip_type(
|
||||
{
|
||||
"isp": "Sony Network Communications Inc.",
|
||||
"org": "Sony Network Communications Inc.",
|
||||
"proxy": True,
|
||||
"hosting": False,
|
||||
"mobile": False,
|
||||
}
|
||||
)
|
||||
softether, softether_reason = manager.vpn_utils.classify_ip_type(
|
||||
{
|
||||
"isp": "SoftEther",
|
||||
"org": "SoftEther Corporation",
|
||||
"proxy": True,
|
||||
"hosting": False,
|
||||
"mobile": False,
|
||||
}
|
||||
)
|
||||
hosting, hosting_reason = manager.vpn_utils.classify_ip_type(
|
||||
{"proxy": True, "hosting": True, "mobile": False}
|
||||
)
|
||||
mobile, mobile_reason = manager.vpn_utils.classify_ip_type(
|
||||
{"proxy": False, "hosting": False, "mobile": True}
|
||||
)
|
||||
|
||||
self.assertEqual(("residential", "consumer_or_unclassified_network"), (residential, residential_reason))
|
||||
self.assertEqual(("hosting", "proxy_provider_datacenter"), (softether, softether_reason))
|
||||
self.assertEqual(("hosting", "hosting_flag"), (hosting, hosting_reason))
|
||||
self.assertEqual(("mobile", "mobile_flag"), (mobile, mobile_reason))
|
||||
|
||||
def test_ip_enrichment_reclassifies_legacy_cache_and_keeps_proxy_quality(self) -> None:
|
||||
ip = "118.240.250.95"
|
||||
manager.vpn_utils.IP_CACHE_FILE.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
ip: {
|
||||
"ip_type": "hosting",
|
||||
"quality": "proxy",
|
||||
"cached_at": 9999999999,
|
||||
"classification_version": 1,
|
||||
}
|
||||
}
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
api_result = [
|
||||
{
|
||||
"status": "success",
|
||||
"query": ip,
|
||||
"country": "Japan",
|
||||
"regionName": "Tokyo",
|
||||
"city": "Tokyo",
|
||||
"isp": "Sony Network Communications Inc.",
|
||||
"org": "Sony Network Communications Inc.",
|
||||
"as": "AS2527 Sony Network Communications Inc.",
|
||||
"asname": "Sony Network Communications Inc.",
|
||||
"proxy": True,
|
||||
"hosting": False,
|
||||
"mobile": False,
|
||||
}
|
||||
]
|
||||
response = mock.MagicMock()
|
||||
response.read.return_value = json.dumps(api_result).encode("utf-8")
|
||||
response.__enter__.return_value = response
|
||||
node = {"id": "sony", "ip": ip}
|
||||
|
||||
with mock.patch.object(manager.vpn_utils.urllib.request, "urlopen", return_value=response) as urlopen_mock:
|
||||
manager.vpn_utils.enrich_ip_info([node])
|
||||
|
||||
self.assertEqual("residential", node["ip_type"])
|
||||
self.assertEqual("proxy", node["quality"])
|
||||
self.assertTrue(node["is_proxy"])
|
||||
self.assertFalse(node["is_hosting"])
|
||||
urlopen_mock.assert_called_once()
|
||||
cache = json.loads(manager.vpn_utils.IP_CACHE_FILE.read_text(encoding="utf-8"))
|
||||
self.assertEqual(manager.vpn_utils.IP_CLASSIFICATION_VERSION, cache[ip]["classification_version"])
|
||||
|
||||
def test_background_ip_enrichment_merges_metadata_without_replacing_status(self) -> None:
|
||||
nodes = self.write_nodes(2)
|
||||
nodes[0]["probe_status"] = "available"
|
||||
manager.write_json(manager.NODES_FILE, nodes)
|
||||
|
||||
def fake_enrich(items):
|
||||
for item in items:
|
||||
item["ip_type"] = "residential"
|
||||
item["quality"] = "proxy"
|
||||
item["owner"] = "Consumer ISP"
|
||||
item["is_proxy"] = True
|
||||
|
||||
with mock.patch.object(manager.vpn_utils, "enrich_ip_info", side_effect=fake_enrich):
|
||||
changed = manager.enrich_stored_nodes()
|
||||
|
||||
stored = manager.read_nodes()
|
||||
self.assertGreater(changed, 0)
|
||||
self.assertEqual("available", next(node for node in stored if node["id"] == "node-0")["probe_status"])
|
||||
self.assertTrue(all(node["ip_type"] == "residential" for node in stored))
|
||||
|
||||
def test_source_deadline_skips_same_host_http_and_uses_github_https(self) -> None:
|
||||
csv_text = valid_snapshot()
|
||||
|
||||
def fake_fetch(url, verify_ssl=True, deadline_seconds=None):
|
||||
if url == manager.API_HTTPS_URL:
|
||||
raise manager.SourceDeadlineExceeded("slow official source")
|
||||
if url == manager.MIRROR_HTTPS_URL:
|
||||
return csv_text
|
||||
raise AssertionError(f"unexpected source: {url}")
|
||||
|
||||
with (
|
||||
mock.patch.object(manager, "fetch_api_text_with_deadline", side_effect=fake_fetch) as fetch_mock,
|
||||
mock.patch.object(manager, "load_blacklist", return_value={}),
|
||||
mock.patch.object(manager, "log_to_json"),
|
||||
):
|
||||
nodes = manager.fetch_candidates()
|
||||
|
||||
self.assertEqual(1, len(nodes))
|
||||
self.assertEqual(
|
||||
[manager.API_HTTPS_URL, manager.MIRROR_HTTPS_URL],
|
||||
[call.args[0] for call in fetch_mock.call_args_list],
|
||||
)
|
||||
|
||||
def test_source_deadline_limits_total_fetch_time(self) -> None:
|
||||
def slow_fetch(url, verify_ssl=True):
|
||||
threading.Event().wait(0.1)
|
||||
return valid_snapshot()
|
||||
|
||||
with mock.patch.object(manager, "fetch_api_text", side_effect=slow_fetch):
|
||||
started = manager.time.monotonic()
|
||||
with self.assertRaises(manager.SourceDeadlineExceeded):
|
||||
manager.fetch_api_text_with_deadline(
|
||||
manager.API_HTTPS_URL,
|
||||
deadline_seconds=0.01,
|
||||
)
|
||||
|
||||
self.assertLess(manager.time.monotonic() - started, 0.08)
|
||||
|
||||
def test_node_probe_stops_after_systemic_openvpn_failure(self) -> None:
|
||||
nodes = self.write_nodes(12)
|
||||
|
||||
@@ -461,8 +599,8 @@ class ManagerLogicTests(unittest.TestCase):
|
||||
self.assertEqual(519, entries[-1]["index"])
|
||||
|
||||
def test_web_update_controls_only_expose_stable_main_channel(self) -> None:
|
||||
self.assertEqual("2.1.1", manager.APP_VERSION)
|
||||
self.assertEqual("V2.1.1 正式版", manager.APP_VERSION_LABEL)
|
||||
self.assertEqual("2.1.2", manager.APP_VERSION)
|
||||
self.assertEqual("V2.1.2 正式版", manager.APP_VERSION_LABEL)
|
||||
self.assertIn("检测更新", manager.INDEX_HTML)
|
||||
self.assertIn("/api/check_update", manager.INDEX_HTML)
|
||||
self.assertIn("/tree/main", manager.INDEX_HTML)
|
||||
@@ -496,7 +634,7 @@ class ManagerLogicTests(unittest.TestCase):
|
||||
def test_release_workflow_uses_full_patch_version(self) -> None:
|
||||
workflow_text = (manager.ROOT_DIR / ".github" / "workflows" / "release.yml").read_text(encoding="utf-8")
|
||||
|
||||
self.assertIn("default: v2.1.1", workflow_text)
|
||||
self.assertIn("default: v2.1.2", workflow_text)
|
||||
self.assertIn("AimiliVPN V$(tr -d '\\r\\n' < VERSION) 正式版", workflow_text)
|
||||
self.assertNotIn("cut -d. -f1,2 VERSION", workflow_text)
|
||||
|
||||
@@ -523,8 +661,8 @@ class ManagerLogicTests(unittest.TestCase):
|
||||
|
||||
def test_latest_release_check_reports_current_formal_version(self) -> None:
|
||||
release = {
|
||||
"tag_name": "v2.1.1",
|
||||
"name": "AimiliVPN V2.1.1 正式版",
|
||||
"tag_name": "v2.1.2",
|
||||
"name": "AimiliVPN V2.1.2 正式版",
|
||||
"draft": False,
|
||||
"prerelease": False,
|
||||
}
|
||||
@@ -532,7 +670,7 @@ class ManagerLogicTests(unittest.TestCase):
|
||||
result = manager.check_latest_release()
|
||||
|
||||
self.assertFalse(result["update_available"])
|
||||
self.assertEqual("V2.1.1 正式版", result["current_version_label"])
|
||||
self.assertEqual("V2.1.2 正式版", result["current_version_label"])
|
||||
|
||||
def test_latest_release_check_reports_source_update_command(self) -> None:
|
||||
release = {"tag_name": "v2.2.0", "draft": False, "prerelease": False}
|
||||
|
||||
+62
-23
@@ -15,11 +15,18 @@ from typing import Any
|
||||
ROOT_DIR = Path(__file__).resolve().parent
|
||||
DATA_DIR = Path(os.environ["VPNGATE_DATA_DIR"]).resolve() if os.environ.get("VPNGATE_DATA_DIR") else ROOT_DIR / "vpngate_data"
|
||||
IP_CACHE_FILE = DATA_DIR / "ip_cache.json"
|
||||
IP_CLASSIFICATION_VERSION = 2
|
||||
IP_CACHE_TTL_SECONDS = 7 * 24 * 3600
|
||||
|
||||
ip_cache_lock = threading.RLock()
|
||||
physical_interface_lock = threading.Lock()
|
||||
physical_interface_cache: tuple[str | None, float] = (None, 0.0)
|
||||
|
||||
DATACENTER_PROVIDER_PATTERN = re.compile(
|
||||
r"(?:\b(?:cloud|colo|colocation|data[ -]?center|hosting|servers?|vps)\b|softether)",
|
||||
re.IGNORECASE,
|
||||
)
|
||||
|
||||
COUNTRY_TRANSLATIONS = {
|
||||
"Japan": "日本",
|
||||
"Korea Republic of": "韩国",
|
||||
@@ -389,6 +396,39 @@ def save_ip_cache(cache: dict[str, dict[str, Any]]) -> None:
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def classify_ip_type(item: dict[str, Any]) -> tuple[str, str]:
|
||||
"""Classify network ownership without confusing VPN use with hosting."""
|
||||
if item.get("mobile"):
|
||||
return "mobile", "mobile_flag"
|
||||
if item.get("hosting"):
|
||||
return "hosting", "hosting_flag"
|
||||
|
||||
provider_text = " ".join(
|
||||
str(item.get(key) or "")
|
||||
for key in ("isp", "org", "as", "asname")
|
||||
)
|
||||
if item.get("proxy") and DATACENTER_PROVIDER_PATTERN.search(provider_text):
|
||||
return "hosting", "proxy_provider_datacenter"
|
||||
|
||||
# A residential volunteer running VPNGate is commonly marked as a proxy.
|
||||
# Proxy use is retained in quality/is_proxy and must not change ownership.
|
||||
return "residential", "consumer_or_unclassified_network"
|
||||
|
||||
def apply_ip_cache_entry(node: dict[str, Any], entry: dict[str, Any]) -> None:
|
||||
for key in (
|
||||
"owner",
|
||||
"asn",
|
||||
"as_name",
|
||||
"location",
|
||||
"ip_type",
|
||||
"quality",
|
||||
"is_proxy",
|
||||
"is_hosting",
|
||||
"is_mobile",
|
||||
"ip_type_reason",
|
||||
):
|
||||
node[key] = entry.get(key, "")
|
||||
|
||||
def enrich_ip_info(nodes: list[dict[str, Any]]) -> None:
|
||||
# 1. Read cache thread-safely
|
||||
with ip_cache_lock:
|
||||
@@ -401,14 +441,14 @@ def enrich_ip_info(nodes: list[dict[str, Any]]) -> None:
|
||||
ip = node.get("ip") or node.get("remote_host")
|
||||
if not ip:
|
||||
continue
|
||||
if ip in cache and now - cache[ip].get("cached_at", 0) < 7 * 24 * 3600:
|
||||
cache_entry = cache.get(ip) if isinstance(cache.get(ip), dict) else None
|
||||
if (
|
||||
cache_entry
|
||||
and cache_entry.get("classification_version") == IP_CLASSIFICATION_VERSION
|
||||
and now - cache_entry.get("cached_at", 0) < IP_CACHE_TTL_SECONDS
|
||||
):
|
||||
cached = cache[ip]
|
||||
node["owner"] = cached.get("owner", "")
|
||||
node["asn"] = cached.get("asn", "")
|
||||
node["as_name"] = cached.get("as_name", "")
|
||||
node["location"] = cached.get("location", "")
|
||||
node["ip_type"] = cached.get("ip_type", "")
|
||||
node["quality"] = cached.get("quality", "")
|
||||
apply_ip_cache_entry(node, cached)
|
||||
else:
|
||||
if ip not in ips_to_query:
|
||||
ips_to_query.append(ip)
|
||||
@@ -425,7 +465,10 @@ def enrich_ip_info(nodes: list[dict[str, Any]]) -> None:
|
||||
request = urllib.request.Request(
|
||||
"http://ip-api.com/batch?lang=zh-CN&fields=status,message,query,country,regionName,city,isp,org,as,asname,proxy,hosting,mobile",
|
||||
data=payload,
|
||||
headers={"Content-Type": "application/json", "User-Agent": "vpngate-manager/2.2"},
|
||||
headers={
|
||||
"Content-Type": "application/json",
|
||||
"User-Agent": f"AimiliVPN-IP-Classifier/{IP_CLASSIFICATION_VERSION}",
|
||||
},
|
||||
method="POST",
|
||||
)
|
||||
try:
|
||||
@@ -442,19 +485,15 @@ def enrich_ip_info(nodes: list[dict[str, Any]]) -> None:
|
||||
if not query_ip:
|
||||
continue
|
||||
|
||||
ip_type = "residential"
|
||||
if item.get("mobile"):
|
||||
ip_type = "mobile"
|
||||
elif item.get("hosting") or item.get("proxy"):
|
||||
ip_type = "hosting"
|
||||
ip_type, ip_type_reason = classify_ip_type(item)
|
||||
|
||||
quality = "normal"
|
||||
if item.get("proxy"):
|
||||
quality = "proxy"
|
||||
if item.get("mobile"):
|
||||
quality = "mobile"
|
||||
elif item.get("hosting"):
|
||||
quality = "datacenter"
|
||||
elif item.get("mobile"):
|
||||
quality = "mobile"
|
||||
elif item.get("proxy"):
|
||||
quality = "proxy"
|
||||
|
||||
loc = " ".join(part for part in [item.get("country"), item.get("regionName"), item.get("city")] if part)
|
||||
|
||||
@@ -465,6 +504,11 @@ def enrich_ip_info(nodes: list[dict[str, Any]]) -> None:
|
||||
"location": loc,
|
||||
"ip_type": ip_type,
|
||||
"quality": quality,
|
||||
"is_proxy": bool(item.get("proxy")),
|
||||
"is_hosting": bool(item.get("hosting")),
|
||||
"is_mobile": bool(item.get("mobile")),
|
||||
"ip_type_reason": ip_type_reason,
|
||||
"classification_version": IP_CLASSIFICATION_VERSION,
|
||||
"cached_at": now,
|
||||
}
|
||||
except Exception as e:
|
||||
@@ -484,12 +528,7 @@ def enrich_ip_info(nodes: list[dict[str, Any]]) -> None:
|
||||
ip = node.get("ip") or node.get("remote_host")
|
||||
if ip in new_entries:
|
||||
cached = new_entries[ip]
|
||||
node["owner"] = cached.get("owner", "")
|
||||
node["asn"] = cached.get("asn", "")
|
||||
node["as_name"] = cached.get("as_name", "")
|
||||
node["location"] = cached.get("location", "")
|
||||
node["ip_type"] = cached.get("ip_type", "")
|
||||
node["quality"] = cached.get("quality", "")
|
||||
apply_ip_cache_entry(node, cached)
|
||||
|
||||
|
||||
def diagnose_api_failure(api_url: str = "https://www.vpngate.net/api/iphone/") -> tuple[int, str]:
|
||||
|
||||
+105
-5
@@ -105,6 +105,7 @@ CHECK_INTERVAL_SECONDS = env_int("CHECK_INTERVAL_SECONDS", 1260, 1)
|
||||
TARGET_VALID_NODES = env_int("TARGET_VALID_NODES", 3, 1)
|
||||
MAX_SCAN_ROWS = env_int("MAX_SCAN_ROWS", 300, 1)
|
||||
API_FETCH_TIMEOUT_SECONDS = env_int("API_FETCH_TIMEOUT_SECONDS", 10, 1, 60)
|
||||
API_SOURCE_DEADLINE_SECONDS = env_int("API_SOURCE_DEADLINE_SECONDS", 6, 2, 30)
|
||||
OPENVPN_TEST_TIMEOUT_SECONDS = env_int("OPENVPN_TEST_TIMEOUT_SECONDS", 35, 1)
|
||||
MANUAL_TEST_NODE_LIMIT = env_int("MANUAL_TEST_NODE_LIMIT", 5, 1, 20)
|
||||
INITIAL_CONNECT_TEST_LIMIT = env_int("INITIAL_CONNECT_TEST_LIMIT", 10, 1, 50)
|
||||
@@ -130,7 +131,7 @@ UPDATE_COMMAND = (
|
||||
)
|
||||
|
||||
ROOT_DIR = Path(sys.executable).resolve().parent if globals().get("__compiled__") else Path(__file__).resolve().parent
|
||||
DEFAULT_APP_VERSION = "2.1.1"
|
||||
DEFAULT_APP_VERSION = "2.1.2"
|
||||
try:
|
||||
_version_text = (ROOT_DIR / "VERSION").read_text(encoding="utf-8").strip()
|
||||
except OSError:
|
||||
@@ -175,10 +176,27 @@ last_collector_heartbeat = 0.0
|
||||
last_checker_heartbeat = 0.0
|
||||
last_pinger_heartbeat = 0.0
|
||||
server_start_time = time.time()
|
||||
ip_enrichment_wakeup = threading.Event()
|
||||
|
||||
IP_ENRICHMENT_FIELDS = (
|
||||
"owner",
|
||||
"asn",
|
||||
"as_name",
|
||||
"location",
|
||||
"ip_type",
|
||||
"quality",
|
||||
"is_proxy",
|
||||
"is_hosting",
|
||||
"is_mobile",
|
||||
"ip_type_reason",
|
||||
)
|
||||
|
||||
class ConnectionCancelled(RuntimeError):
|
||||
pass
|
||||
|
||||
class SourceDeadlineExceeded(TimeoutError):
|
||||
pass
|
||||
|
||||
def purge_expired_sessions(now: float | None = None) -> int:
|
||||
current_time = time.time() if now is None else now
|
||||
with lock:
|
||||
@@ -776,6 +794,29 @@ def fetch_api_text(url: str | None = None, use_ssl_verify: bool = True) -> str:
|
||||
with urllib.request.urlopen(request, timeout=API_FETCH_TIMEOUT_SECONDS) as response:
|
||||
return read_limited(response).decode("utf-8", errors="replace")
|
||||
|
||||
def fetch_api_text_with_deadline(
|
||||
url: str,
|
||||
use_ssl_verify: bool = True,
|
||||
deadline_seconds: int | None = None,
|
||||
) -> str:
|
||||
deadline = deadline_seconds or API_SOURCE_DEADLINE_SECONDS
|
||||
result_queue: queue.Queue[tuple[bool, Any]] = queue.Queue(maxsize=1)
|
||||
|
||||
def worker() -> None:
|
||||
try:
|
||||
result_queue.put((True, fetch_api_text(url, use_ssl_verify)))
|
||||
except BaseException as exc:
|
||||
result_queue.put((False, exc))
|
||||
|
||||
threading.Thread(target=worker, daemon=True).start()
|
||||
try:
|
||||
ok, value = result_queue.get(timeout=deadline)
|
||||
except queue.Empty as exc:
|
||||
raise SourceDeadlineExceeded(f"节点源超过 {deadline} 秒总时限") from exc
|
||||
if ok:
|
||||
return str(value)
|
||||
raise value
|
||||
|
||||
def parse_release_version(value: Any) -> tuple[int, int, int]:
|
||||
match = re.search(r"(?i)(?:^|[^a-z0-9])v?(\d+)(?:\.(\d+))?(?:\.(\d+))?", str(value or "").strip())
|
||||
if not match:
|
||||
@@ -992,15 +1033,22 @@ def fetch_candidates() -> list[dict[str, Any]]:
|
||||
load_ui_config().get("discovery_countries")
|
||||
)
|
||||
last_err: Exception | None = None
|
||||
deadline_hosts: set[str] = set()
|
||||
|
||||
log_to_json("INFO", "Main", "开始按官方、GitHub Pages、本地缓存顺序拉取节点列表...")
|
||||
|
||||
for source_name, url in api_network_sources():
|
||||
source_host = (urllib.parse.urlsplit(url).hostname or "").lower()
|
||||
if url.startswith("http://") and source_host in deadline_hosts:
|
||||
msg = f"跳过同主机慢速 HTTP 节点源 {source_name}: {url}"
|
||||
print(f"[fetch_candidates] {msg}", flush=True)
|
||||
log_to_json("WARNING", "Main", msg)
|
||||
continue
|
||||
try:
|
||||
msg = f"尝试节点源 {source_name}: {url}"
|
||||
print(f"[fetch_candidates] {msg}", flush=True)
|
||||
log_to_json("INFO", "Main", msg)
|
||||
api_text = fetch_api_text(url, True)
|
||||
api_text = fetch_api_text_with_deadline(url, True)
|
||||
rows = parse_vpngate_rows(api_text)
|
||||
candidates = rows_to_candidates(rows, blacklist)
|
||||
if not candidates:
|
||||
@@ -1038,6 +1086,8 @@ def fetch_candidates() -> list[dict[str, Any]]:
|
||||
return filtered_candidates
|
||||
except Exception as e:
|
||||
last_err = e
|
||||
if isinstance(e, SourceDeadlineExceeded) and url.startswith("https://"):
|
||||
deadline_hosts.add(source_host)
|
||||
print(f"[fetch_candidates] 节点源 {source_name} 失败: {e}", flush=True)
|
||||
log_to_json("WARNING", "Main", f"节点源 {source_name} 失败: {e}")
|
||||
|
||||
@@ -1528,6 +1578,50 @@ def sort_all_nodes(nodes: list[dict[str, Any]]) -> list[dict[str, Any]]:
|
||||
)
|
||||
return available_nodes + untested_nodes + unavailable_nodes
|
||||
|
||||
def enrich_stored_nodes() -> int:
|
||||
"""Enrich every listed IP, then merge only metadata into the latest state."""
|
||||
with lock:
|
||||
snapshot = read_nodes()
|
||||
if not snapshot:
|
||||
return 0
|
||||
|
||||
vpn_utils.enrich_ip_info(snapshot)
|
||||
enriched_by_id = {
|
||||
str(node.get("id") or ""): node
|
||||
for node in snapshot
|
||||
if node.get("id") and node.get("ip_type")
|
||||
}
|
||||
if not enriched_by_id:
|
||||
return 0
|
||||
|
||||
changed = 0
|
||||
with lock:
|
||||
current_nodes = read_nodes()
|
||||
for current in current_nodes:
|
||||
enriched = enriched_by_id.get(str(current.get("id") or ""))
|
||||
if not enriched:
|
||||
continue
|
||||
for field in IP_ENRICHMENT_FIELDS:
|
||||
new_value = enriched.get(field, "")
|
||||
if current.get(field, "") != new_value:
|
||||
current[field] = new_value
|
||||
changed += 1
|
||||
if changed:
|
||||
write_json(NODES_FILE, sort_all_nodes(current_nodes))
|
||||
return changed
|
||||
|
||||
def ip_enrichment_loop() -> None:
|
||||
while True:
|
||||
nodes_exist = bool(read_nodes())
|
||||
if nodes_exist:
|
||||
try:
|
||||
enrich_stored_nodes()
|
||||
except Exception as exc:
|
||||
print(f"[IP 类型] 后台批量识别失败: {exc}", flush=True)
|
||||
log_to_json("WARNING", "Main", f"后台批量识别 IP 类型失败: {exc}")
|
||||
ip_enrichment_wakeup.wait(300 if nodes_exist else 5)
|
||||
ip_enrichment_wakeup.clear()
|
||||
|
||||
def apply_routing_filters(
|
||||
nodes: list[dict[str, Any]],
|
||||
ui_cfg: dict[str, Any],
|
||||
@@ -2314,6 +2408,10 @@ def maintain_valid_nodes(force: bool = False) -> str:
|
||||
"location",
|
||||
"ip_type",
|
||||
"quality",
|
||||
"is_proxy",
|
||||
"is_hosting",
|
||||
"is_mobile",
|
||||
"ip_type_reason",
|
||||
]:
|
||||
if previous.get(key) not in (None, ""):
|
||||
cand[key] = previous.get(key)
|
||||
@@ -2332,6 +2430,7 @@ def maintain_valid_nodes(force: bool = False) -> str:
|
||||
pass
|
||||
|
||||
write_json(NODES_FILE, merged)
|
||||
ip_enrichment_wakeup.set()
|
||||
|
||||
initial_tested_ids: set[str] = set()
|
||||
fast_results: list[dict[str, Any]] = []
|
||||
@@ -3985,12 +4084,12 @@ INDEX_HTML = r"""<!doctype html>
|
||||
<div class="dropdown">
|
||||
<button id="github_btn" class="btn-primary" type="button" aria-expanded="false" aria-controls="github_dropdown" style="background: rgba(255, 255, 255, 0.08); border: 1px solid var(--border-color); color: var(--text-primary);">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" viewBox="0 0 16 16" style="vertical-align: middle; margin-right: 4px;"><path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.012 8.012 0 0 0 16 8c0-4.42-3.58-8-8-8z"/></svg>
|
||||
<span id="github_version_label">V2.1.1 正式版</span>
|
||||
<span id="github_version_label">V2.1.2 正式版</span>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" style="width:12px; height:12px; margin-left: 2px;" fill="none" viewBox="0 0 24 24" stroke="currentColor" stroke-width="3"><path stroke-linecap="round" stroke-linejoin="round" d="M19 9l-7 7-7-7" /></svg>
|
||||
</button>
|
||||
<div id="github_dropdown" class="dropdown-content github-dropdown">
|
||||
<div class="version-current">
|
||||
<div id="current_version_label" class="version-current-label">V2.1.1 正式版</div>
|
||||
<div id="current_version_label" class="version-current-label">V2.1.2 正式版</div>
|
||||
<div id="deployment_mode_label" class="version-current-meta">Python 源码部署 · 更新通道:main</div>
|
||||
</div>
|
||||
<button id="check_update_btn" type="button" onclick="checkForUpdate(event)">
|
||||
@@ -4815,7 +4914,7 @@ function stableSortNodes() {
|
||||
}
|
||||
|
||||
function render(){
|
||||
const versionLabel = state.app_version_label || "V2.1.1 正式版";
|
||||
const versionLabel = state.app_version_label || "V2.1.2 正式版";
|
||||
if ($("github_version_label")) $("github_version_label").textContent = versionLabel;
|
||||
if ($("current_version_label")) $("current_version_label").textContent = versionLabel;
|
||||
if ($("deployment_mode_label")) {
|
||||
@@ -7139,6 +7238,7 @@ def main() -> None:
|
||||
print("[警告] 代理网关启动超时,继续执行脚本...", flush=True)
|
||||
|
||||
threading.Thread(target=collector_loop, daemon=True).start()
|
||||
threading.Thread(target=ip_enrichment_loop, daemon=True).start()
|
||||
threading.Thread(target=background_proxy_checker, daemon=True).start()
|
||||
threading.Thread(target=active_node_pinger, daemon=True).start()
|
||||
|
||||
|
||||
Reference in New Issue
Block a user