Compare commits
10 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
be75e5b65c | ||
|
|
c7ece6620c | ||
|
|
7e9da4b82a | ||
|
|
c657c92b27 | ||
|
|
d43163703e | ||
|
|
1150450718 | ||
|
|
df6483afa8 | ||
|
|
29de2eadf8 | ||
|
|
318689f163 | ||
|
|
6b9563b858 |
44
.github/workflows/daily_keywords.yml
vendored
Normal file
44
.github/workflows/daily_keywords.yml
vendored
Normal file
@@ -0,0 +1,44 @@
|
||||
name: Daily Trends Factory
|
||||
|
||||
on:
|
||||
schedule:
|
||||
# 每天 UTC 18:00 运行 (北京时间凌晨 02:00)
|
||||
- cron: '0 18 * * *'
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
|
||||
jobs:
|
||||
update-trends:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout Code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: '3.10'
|
||||
|
||||
- name: Execute Trends Engine
|
||||
run: python scripts/fetch_trends.py
|
||||
|
||||
- name: Commit and Push
|
||||
run: |
|
||||
git config --global user.name "github-actions[bot]"
|
||||
git config --global user.email "github-actions[bot]@users.noreply.github.com"
|
||||
|
||||
git add data/keywords/
|
||||
|
||||
# 防御机制:如果没有新数据,就静默退出,不产生空提交
|
||||
if git diff --staged --quiet; then
|
||||
echo "No changes, skipping."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# 策略:放弃危险的 amend 强制覆盖,采用带日期的标准安全提交
|
||||
git commit -m "chore(data): 🤖 自动机兵:刷新全战区热点词库 [$(date +'%Y-%m-%d')]"
|
||||
git push origin main
|
||||
10
README.md
10
README.md
@@ -18,9 +18,13 @@
|
||||
|
||||
- 🏭 **自动化指纹兵工厂 (Automated UA Factory)**:依托 GitHub Actions CI/CD 流水线,每月 1 日无人值守全自动生成 4000+ 带绝对物理分区的真实终端设备数据。配合边缘节点的守护进程静默拉取,实现千万级指纹资产的“自动驾驶”级演进。
|
||||
|
||||
- 📡 **OTA 动态活体词库 (Dynamic Trends)**:v3.3.0 跨时代跃升。彻底废弃静态搜索词,引入 GitHub Actions 云端流水线。每天自动抓取美、日、德、英等全战区当日 Google 热搜榜单,并通过边缘节点每日静默同步,让您的 IP 搜索行为永远贴合当地当天的真实网络脉搏。
|
||||
|
||||
- 🔀 **智能错峰调度 (Thundering Herd Mitigation)**:v3.3.0 架构升级。首创节点部署时间戳锚定逻辑。边缘节点按需智能分频(每日拉取几百行轻量词库,每月按 30 天周期错峰拉取千万级指纹库),完美化解“惊群效应”,彻底抹平统一升级时的数据并发特征,隐匿于无形。
|
||||
|
||||
- 🖧 **底层路由死锁 (Hard-Bind Routing)**:v3.2.1 热修复升级。底层探测引擎强力接管 curl 核心参数 (--interface),强制将发出的每一滴伪装流量死死绑定在您设定的物理网卡或隧道 IP 上,彻底杜绝双栈或多网卡环境下的流量溢出漏洞。
|
||||
|
||||
- 🎯 **多级容灾与高精度探针 (High-Precision Probe)**:v3.2.2 底层重构。重写战报模块与底层协议自适应逻辑,植入多级 ISP 容灾探针链路,并按“底层数据共识原则”智能清洗冗余 AS 号。确保在纯 V6、隧道或弱网环境下,数据获取依然 100% 精准畅通。
|
||||
- 🎯 **多级容灾与高精度探针 (High-Precision Probe)**:v3.2.2 底层重构。重写战报模块与底层协议自适应逻辑,植入多级 ISP 容灾探针链路,并按“底层数据共识原则”智能清洗冗余 AS 号。确保在纯 V6、隧道或弱网环境下,数据获取依然 100% 精准畅通。
|
||||
|
||||
- 🔄 **平滑热更新装甲 (Smooth Upgrade Engine)**:v3.2.2 体验进化。全系植入状态机嗅探逻辑。无论是 Master 司令部还是 Agent 边缘节点,再次执行部署脚本时将自动识别并继承历史配置、SQLite 数据库与锚定 IP,一键回车即可瞬间完成无损换代,告别繁琐的重复配置。
|
||||
|
||||
@@ -58,7 +62,7 @@
|
||||
|
||||
## 🚀 极速部署 (Quick Start)
|
||||
|
||||
v3.2.x 提供了两种接入模式,请根据您的战术需求选择:
|
||||
v3.3.x 提供了两种接入模式,请根据您的战术需求选择:
|
||||
|
||||
### 🔹 模式 A:官方公共模式 (最简、推荐)
|
||||
**适合不想折腾、只想快速养护 IP 的新兵。**
|
||||
@@ -86,7 +90,7 @@ bash <(curl -sL https://raw.githubusercontent.com/hotyue/IP-Sentinel/main/core/i
|
||||
```
|
||||
3. **激活节点**:同上,将暗号转发给您自己的机器人即可。
|
||||
|
||||
### ⚠️ 平滑升级指引 (Upgrade to v3.2.2)
|
||||
### ⚠️ 平滑升级指引 (Upgrade to v3.3.0)
|
||||
|
||||
得益于 **v3.2.2 全新引入的平滑热更新引擎 (Smooth Upgrade Engine)**,系统升级现已变得极其优雅与安全。
|
||||
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
#!/bin/bash
|
||||
|
||||
# ==========================================================
|
||||
# 脚本名称: install.sh (IP-Sentinel 分布式边缘节点部署脚本 v3.2.2 - Global Nexus)
|
||||
# 核心功能: 区域选择、模块按需开启、官方机器人一键配置、平滑热更新
|
||||
# 脚本名称: install.sh (IP-Sentinel 分布式边缘节点部署脚本 v3.3.0 - OTA 活体引擎)
|
||||
# 核心功能: 区域选择、模块按需开启、官方机器人一键配置、平滑热更新、分频错峰调度
|
||||
# ==========================================================
|
||||
|
||||
# 你的 GitHub 仓库 Raw 数据直链前缀
|
||||
@@ -404,8 +404,11 @@ crontab -l 2>/dev/null | grep -v "ip_sentinel" > /tmp/cron_backup
|
||||
|
||||
# 核心养护模块: 每 30 分钟触发一次
|
||||
echo "*/30 * * * * ${INSTALL_DIR}/core/runner.sh >/dev/null 2>&1" >> /tmp/cron_backup
|
||||
# 养料更新模块: 每周日凌晨 3 点静默去云端更新热数据
|
||||
echo "0 3 * * 0 ${INSTALL_DIR}/core/updater.sh >/dev/null 2>&1" >> /tmp/cron_backup
|
||||
# 养料更新模块: (v3.3.0升级) 每天凌晨 3 点触发,由中枢自动进行分频调度
|
||||
echo "0 3 * * * ${INSTALL_DIR}/core/updater.sh >/dev/null 2>&1" >> /tmp/cron_backup
|
||||
|
||||
# [v3.3.0 新增] 初始化 UA 指纹库更新时间戳,确立 30 天滚动周期的计算锚点
|
||||
echo $(date +%s) > "${INSTALL_DIR}/core/.ua_last_update"
|
||||
|
||||
# 如果配置了联控,启动 Webhook 与战报任务
|
||||
if [[ -n "$TG_TOKEN" ]] && [[ -n "$CHAT_ID" ]]; then
|
||||
@@ -440,7 +443,7 @@ if [[ -n "$TG_TOKEN" ]] && [[ -n "$CHAT_ID" ]]; then
|
||||
-d "text=✨ *IP-Sentinel 引擎热更新完成!*
|
||||
📍 节点:\`${NODE_NAME}\`
|
||||
🌐 IP:\`${BIND_IP}\`
|
||||
🚀 状态:v3.2.2 协议自适应与高精度探针已就绪" >/dev/null 2>&1
|
||||
🚀 状态:v3.3.0 OTA 动态活体养护引擎已部署" >/dev/null 2>&1
|
||||
echo -e "\033[32m✅ 升级成功通知已推送到您的 Telegram!\033[0m"
|
||||
else
|
||||
echo -e "\n📡 正在向指挥部发送注册暗号..."
|
||||
@@ -486,7 +489,12 @@ if [[ -n "$TG_TOKEN" ]]; then
|
||||
elif command -v firewall-cmd >/dev/null 2>&1 && systemctl is-active firewalld | grep -qw active; then
|
||||
FW_MSG="firewall-cmd --zone=public --add-port=$AGENT_PORT/tcp --permanent && firewall-cmd --reload"
|
||||
elif command -v iptables >/dev/null 2>&1; then
|
||||
FW_MSG="iptables -I INPUT -p tcp --dport $AGENT_PORT -j ACCEPT"
|
||||
# 智能双栈雷达:根据绑定的 IP 属性,动态下发对应的防火墙放行指令
|
||||
if [[ "$BIND_IP" == *":"* ]]; then
|
||||
FW_MSG="ip6tables -I INPUT -p tcp --dport $AGENT_PORT -j ACCEPT"
|
||||
else
|
||||
FW_MSG="iptables -I INPUT -p tcp --dport $AGENT_PORT -j ACCEPT"
|
||||
fi
|
||||
fi
|
||||
|
||||
echo -e "\033[33m⚠️ 警告:请务必确保本机及云服务商安全组放行了 TCP $AGENT_PORT 端口!\033[0m"
|
||||
|
||||
@@ -1,16 +1,16 @@
|
||||
#!/bin/bash
|
||||
|
||||
# ==========================================================
|
||||
# 脚本名称: updater.sh (IP-Sentinel V3.1.4 养料注入与维护中枢)
|
||||
# 核心功能: 静默更新热数据(指纹/词库/LBS规则)、清理瘦身日志
|
||||
# 脚本名称: updater.sh (IP-Sentinel V3.3.0 养料注入与分频调度中枢)
|
||||
# 核心功能: 静默更新热搜词/LBS、指纹库错峰调度、强制出站死锁
|
||||
# ==========================================================
|
||||
|
||||
INSTALL_DIR="/opt/ip_sentinel"
|
||||
CONFIG_FILE="${INSTALL_DIR}/config.conf"
|
||||
# 你的 GitHub 仓库 Raw 数据直链前缀 (统一标准)
|
||||
UA_TIME_FILE="${INSTALL_DIR}/core/.ua_last_update"
|
||||
|
||||
# GitHub 仓库 Raw 数据直链前缀
|
||||
REPO_RAW_URL="https://raw.githubusercontent.com/hotyue/IP-Sentinel/main"
|
||||
# 临时改为私库地址用于测试
|
||||
# REPO_RAW_URL="https://git.94211762.xyz/hotyue/IP-Sentinel/raw/branch/main"
|
||||
|
||||
# 1. 加载本地冷数据配置
|
||||
if [ ! -f "$CONFIG_FILE" ]; then
|
||||
@@ -26,49 +26,94 @@ log() {
|
||||
|
||||
log "Updater" "INFO " "========== 触发后台静默 OTA 热数据更新 =========="
|
||||
|
||||
# 3. 容灾机制拉取 UA 指纹池 (强制遵循锚点协议)
|
||||
TMP_UA="/tmp/ip_sentinel_ua.txt"
|
||||
curl -${IP_PREF:-4} -sL "${REPO_RAW_URL}/data/user_agents.txt" -o "$TMP_UA"
|
||||
if [ -s "$TMP_UA" ]; then
|
||||
mv "$TMP_UA" "${INSTALL_DIR}/data/user_agents.txt"
|
||||
log "Updater" "INFO " "✅ 设备指纹池 (User-Agents) 更新成功"
|
||||
else
|
||||
log "Updater" "WARN " "❌ UA 池拉取失败,保留本地旧数据防崩溃"
|
||||
rm -f "$TMP_UA"
|
||||
# ==========================================================
|
||||
# 🛡️ 终极护城河:构建强锚定出站的 curl 请求引擎
|
||||
# ==========================================================
|
||||
# 基础参数:跟随 install.sh 锁定的协议偏好 (4 或 6)
|
||||
CURL_CMD="curl -${IP_PREF:-4} -sL"
|
||||
|
||||
# 【防坑核心】如果用户配置了死锁锚点,必须强制绑定网卡,杜绝流量溢出!
|
||||
if [ -n "$BIND_IP" ]; then
|
||||
# curl 的 --interface 参数不支持带方括号的 IPv6 地址,必须强行脱壳
|
||||
RAW_BIND_IP=$(echo "$BIND_IP" | tr -d '[]')
|
||||
CURL_CMD="$CURL_CMD --interface $RAW_BIND_IP"
|
||||
fi
|
||||
|
||||
# 4. 容灾机制拉取当地最新搜索词库
|
||||
# ==========================================================
|
||||
# 3. 容灾机制拉取 UA 指纹池 (V3.3.0 引入 30 天错峰防惊群逻辑)
|
||||
# ==========================================================
|
||||
NOW=$(date +%s)
|
||||
LAST_UPDATE=0
|
||||
|
||||
# 读取上一次更新的时间戳
|
||||
if [ -f "$UA_TIME_FILE" ]; then
|
||||
# tr -d 清除可能存在的换行或回车符,防止算术崩溃
|
||||
LAST_UPDATE=$(cat "$UA_TIME_FILE" | tr -d '\r\n')
|
||||
fi
|
||||
|
||||
# 校验数据合法性,防崩溃
|
||||
if ! [[ "$LAST_UPDATE" =~ ^[0-9]+$ ]]; then
|
||||
LAST_UPDATE=0
|
||||
fi
|
||||
|
||||
DIFF=$((NOW - LAST_UPDATE))
|
||||
|
||||
# 距离上次拉取超过 30 天 (2592000 秒),才执行下载
|
||||
if [ "$DIFF" -ge 2592000 ] || [ "$LAST_UPDATE" -eq 0 ]; then
|
||||
TMP_UA="/tmp/ip_sentinel_ua.txt"
|
||||
# 使用重装升级后的 CURL_CMD
|
||||
$CURL_CMD "${REPO_RAW_URL}/data/user_agents.txt" -o "$TMP_UA"
|
||||
|
||||
if [ -s "$TMP_UA" ]; then
|
||||
mv "$TMP_UA" "${INSTALL_DIR}/data/user_agents.txt"
|
||||
echo "$NOW" > "$UA_TIME_FILE"
|
||||
log "Updater" "INFO " "✅ 设备指纹池 (User-Agents) 30天错峰滚动更新成功"
|
||||
else
|
||||
log "Updater" "WARN " "❌ UA 池拉取失败,保留本地旧数据防崩溃"
|
||||
rm -f "$TMP_UA"
|
||||
fi
|
||||
else
|
||||
DAYS_LEFT=$(((2592000 - DIFF) / 86400))
|
||||
log "Updater" "INFO " "⏳ 设备指纹池处于 30 天静默期 (剩余约 ${DAYS_LEFT} 天),跳过拉取"
|
||||
fi
|
||||
|
||||
# ==========================================================
|
||||
# 4. 容灾机制拉取当地最新搜索词库 (每日高频拉取,保证活体新鲜度)
|
||||
# ==========================================================
|
||||
TMP_KW="/tmp/ip_sentinel_kw.txt"
|
||||
curl -${IP_PREF:-4} -sL "${REPO_RAW_URL}/data/keywords/kw_${REGION_CODE}.txt" -o "$TMP_KW"
|
||||
$CURL_CMD "${REPO_RAW_URL}/data/keywords/kw_${REGION_CODE}.txt" -o "$TMP_KW"
|
||||
|
||||
if [ -s "$TMP_KW" ]; then
|
||||
mv "$TMP_KW" "${INSTALL_DIR}/data/keywords/kw_${REGION_CODE}.txt"
|
||||
log "Updater" "INFO " "✅ 区域搜索词库 (kw_${REGION_CODE}) 更新成功"
|
||||
log "Updater" "INFO " "✅ 区域搜索词库 (kw_${REGION_CODE}) 每日同步成功"
|
||||
else
|
||||
log "Updater" "WARN " "❌ 搜索词库拉取失败,保留本地旧数据防崩溃"
|
||||
rm -f "$TMP_KW"
|
||||
fi
|
||||
|
||||
# 5. 【V3.1.4 核心升级】自适应拉取本地 LBS 专属 JSON 规则库
|
||||
# 利用 find 穿透寻找本地唯一的 json
|
||||
# ==========================================================
|
||||
# 5. 自适应拉取本地 LBS 专属 JSON 规则库 (每日同步)
|
||||
# ==========================================================
|
||||
REGION_JSON_FILE=$(find "${INSTALL_DIR}/data/regions" -name "*.json" 2>/dev/null | head -n 1)
|
||||
|
||||
if [ -n "$REGION_JSON_FILE" ] && [ -f "$REGION_JSON_FILE" ]; then
|
||||
# 提取本地文件的相对路径 (例如: data/regions/US/CA/San_Jose.json)
|
||||
REL_PATH=${REGION_JSON_FILE#*${INSTALL_DIR}/}
|
||||
TMP_JSON="/tmp/ip_sentinel_region.json"
|
||||
|
||||
# 按照相同的相对路径去云端拉取更新
|
||||
curl -${IP_PREF:-4} -sL "${REPO_RAW_URL}/${REL_PATH}" -o "$TMP_JSON"
|
||||
$CURL_CMD "${REPO_RAW_URL}/${REL_PATH}" -o "$TMP_JSON"
|
||||
|
||||
if [ -s "$TMP_JSON" ]; then
|
||||
mv "$TMP_JSON" "$REGION_JSON_FILE"
|
||||
log "Updater" "INFO " "✅ 核心战区规则库 ($REL_PATH) 更新成功"
|
||||
log "Updater" "INFO " "✅ 核心战区规则库 ($REL_PATH) 每日同步成功"
|
||||
else
|
||||
log "Updater" "WARN " "❌ 战区规则库拉取失败,保留本地旧数据"
|
||||
rm -f "$TMP_JSON"
|
||||
fi
|
||||
fi
|
||||
|
||||
# ==========================================================
|
||||
# 6. 日志防满瘦身机制 (保留最近 2000 行)
|
||||
# ==========================================================
|
||||
if [ -f "$LOG_FILE" ]; then
|
||||
tail -n 2000 "$LOG_FILE" > "${LOG_FILE}.tmp"
|
||||
mv "${LOG_FILE}.tmp" "$LOG_FILE"
|
||||
|
||||
@@ -3,5 +3,4 @@
|
||||
最新のニュース 速報
|
||||
ゴールド 相場 チャート
|
||||
近くの静かなカフェ
|
||||
地震 速報
|
||||
円安 影響 生活
|
||||
@@ -1,7 +1,7 @@
|
||||
#!/bin/bash
|
||||
|
||||
# ==========================================================
|
||||
# 脚本名称: install_master.sh (IP-Sentinel 控制中枢部署脚本 v3.2.2)
|
||||
# 脚本名称: install_master.sh (IP-Sentinel 控制中枢部署脚本 v3.3.0)
|
||||
# 核心功能: 部署/卸载调度中枢、SQLite 资产管理、平滑热更新引擎
|
||||
# ==========================================================
|
||||
|
||||
@@ -15,7 +15,7 @@ DB_FILE="${MASTER_DIR}/sentinel.db"
|
||||
|
||||
echo "========================================================"
|
||||
# [修改] 将欢迎语改为更通用的文案,因为现在不仅能部署,还能卸载
|
||||
echo " 🧠 欢迎使用 IP-Sentinel Master (控制中枢) v3.2.2"
|
||||
echo " 🧠 欢迎使用 IP-Sentinel Master (控制中枢) v3.3.0"
|
||||
echo "========================================================"
|
||||
|
||||
# [新增] 交互式操作菜单:支持选择部署或调用卸载程序
|
||||
|
||||
81
scripts/fetch_trends.py
Normal file
81
scripts/fetch_trends.py
Normal file
@@ -0,0 +1,81 @@
|
||||
import urllib.request
|
||||
import xml.etree.ElementTree as ET
|
||||
import os
|
||||
import json
|
||||
import re
|
||||
|
||||
# ================== [路径防弹装甲] ==================
|
||||
# 无论在哪里执行该脚本,都能精准反推项目根目录
|
||||
SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
|
||||
PROJECT_ROOT = os.path.dirname(SCRIPT_DIR)
|
||||
|
||||
MAP_JSON_PATH = os.path.join(PROJECT_ROOT, "data", "map.json")
|
||||
DATA_DIR = os.path.join(PROJECT_ROOT, "data", "keywords")
|
||||
# ====================================================
|
||||
|
||||
# 特殊战区代码映射 (Google Trends RSS 要求)
|
||||
GEO_FIX = {'UK': 'GB'}
|
||||
|
||||
HEADERS = {
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36'
|
||||
}
|
||||
|
||||
def get_active_regions():
|
||||
"""动态提取 map.json 中的战区 (修正:精准穿透 JSON 数组结构)"""
|
||||
try:
|
||||
with open(MAP_JSON_PATH, 'r', encoding='utf-8') as f:
|
||||
data = json.load(f)
|
||||
# 穿透到 countries 列表,提取所有的国家代码 (id)
|
||||
regions = []
|
||||
for country in data.get('countries', []):
|
||||
if 'id' in country:
|
||||
regions.append(country['id'])
|
||||
return regions
|
||||
except Exception as e:
|
||||
print(f"❌ [读取地图失败]: {e}")
|
||||
return []
|
||||
|
||||
def fetch_trends(region_code):
|
||||
"""从 Google Trends 抓取当日热搜"""
|
||||
geo = GEO_FIX.get(region_code, region_code)
|
||||
url = f"https://trends.google.com/trending/rss?geo={geo}"
|
||||
try:
|
||||
req = urllib.request.Request(url, headers=HEADERS)
|
||||
with urllib.request.urlopen(req, timeout=10) as response:
|
||||
xml_data = response.read()
|
||||
root = ET.fromstring(xml_data)
|
||||
return [re.sub(r'[\n\r\t]', ' ', item.find('title').text).strip()
|
||||
for item in root.findall('./channel/item')
|
||||
if item.find('title') is not None]
|
||||
except Exception as e:
|
||||
print(f"⚠️ {region_code} 抓取异常: {e}")
|
||||
return []
|
||||
|
||||
def update_file(region, new_words):
|
||||
"""滑动窗口更新,保留 200 条最热记录"""
|
||||
os.makedirs(DATA_DIR, exist_ok=True)
|
||||
file_path = os.path.join(DATA_DIR, f"kw_{region}.txt")
|
||||
old_words = []
|
||||
if os.path.exists(file_path):
|
||||
with open(file_path, 'r', encoding='utf-8') as f:
|
||||
old_words = [l.strip() for l in f if l.strip()]
|
||||
|
||||
# 新词排在最前面,去重
|
||||
combined = new_words + [w for w in old_words if w not in new_words]
|
||||
final_list = combined[:200]
|
||||
|
||||
with open(file_path, 'w', encoding='utf-8') as f:
|
||||
f.write('\n'.join(final_list) + '\n')
|
||||
print(f"✅ [同步完成] {region}: 注入 {len(new_words)} 条新热点")
|
||||
|
||||
if __name__ == '__main__':
|
||||
regions = get_active_regions()
|
||||
if not regions:
|
||||
print("🛑 未发现活跃战区,请检查 map.json")
|
||||
exit(1)
|
||||
|
||||
for r in regions:
|
||||
print(f"📡 正在拉取 {r} 战区情报...")
|
||||
words = fetch_trends(r)
|
||||
if words:
|
||||
update_file(r, words)
|
||||
Reference in New Issue
Block a user