lilong.129 f6e7e970f8 feat: 实现 AIQuery 功能并支持 OutputSchema ...

- 新增 AIQuery 方法到 StepMobile，支持使用自然语言从屏幕中提取信息
- 实现 AIQuery 在 driver_ext_ai.go 中的完整功能，包括屏幕截图和 LLM 查询
- 添加 OutputSchema 支持，允许用户定义自定义输出格式进行结构化查询
- 新增 ToolAIQuery MCP 工具，完整集成到 MCP 服务器中
- 在 ActionOptions 中添加 OutputSchema 字段和 WithOutputSchema 选项函数
- 添加 ACTION_Query 的配置支持和字段映射
- 完善测试覆盖：
  * 添加 TestAIQuery 单元测试，包含多种 OutputSchema 使用场景
  * 添加 TestToolAIQuery MCP 工具测试
  * 定义 GameInfo、UIElementInfo 等结构体用于测试
- 更新文档：
  * 在 docs/uixt/ai.md 中添加完整的 AIQuery 使用指南
  * 包含基本用法、OutputSchema 示例、最佳实践等
- 支持复杂的嵌套结构体和数组类型的 OutputSchema
- 与现有 AIAction、AIAssert 功能保持一致的 API 设计

2025-06-13 10:27:08 +08:00

..

ai

fix: get resultsPath

2025-06-12 14:51:15 +08:00

demo

…

option

feat: 实现 AIQuery 功能并支持 OutputSchema

2025-06-13 10:27:08 +08:00

types

refactor: move tool request types to option

2025-05-24 23:51:58 +08:00

android_device.go

…

android_driver_adb.go

fix: optimize report command to avoid creating timestamp directories

2025-06-10 12:06:08 +08:00

android_driver_uia2.go

refactor: merge DoAction to mcp server tools

2025-05-25 23:53:07 +08:00

android_key.go

…

android_layout.go

…

android_test.go

refactor: move Call function

2025-06-09 20:52:32 +08:00

browser_device.go

change: move code

2025-05-26 16:08:27 +08:00

browser_driver.go

change: fix logs

2025-06-05 20:26:18 +08:00

cache_test.go

fix: handle GetOrCreateXTDriver when serial is empty

2025-05-26 21:25:25 +08:00

cache.go

feat: implement multi-model service configuration support

2025-06-06 22:17:59 +08:00

device.go

…

driver_ext_ai_test.go

feat: optimize test report UI and add LLM usage tracking

2025-06-09 17:04:55 +08:00

driver_ext_ai.go

feat: 实现 AIQuery 功能并支持 OutputSchema

2025-06-13 10:27:08 +08:00

driver_ext_popups.go

…

driver_ext_screenshot.go

fix: optimize report command to avoid creating timestamp directories

2025-06-10 12:06:08 +08:00

driver_ext_swipe.go

change: fix logs

2025-06-05 20:26:18 +08:00

driver_ext_tap.go

…

driver_ext_test.go

fix(uixt): fix uixt__input not working and add comprehensive unit tests

2025-06-07 15:03:29 +08:00

driver_handler.go

fix: optimize report command to avoid creating timestamp directories

2025-06-10 12:06:08 +08:00

driver_session_test.go

…

driver_session.go

refactor: simplify AI action execution and improve sub-action handling

2025-06-08 19:16:37 +08:00

driver_utils_test.go

…

driver_utils.go

fix: optimize report command to avoid creating timestamp directories

2025-06-10 12:06:08 +08:00

driver.go

refactor: uixt MCP Server

2025-05-24 23:22:00 +08:00

evalite

…

harmony_device.go

…

harmony_driver_hdc.go

refactor: merge DoAction to mcp server tools

2025-05-25 23:53:07 +08:00

harmony_test.go

…

ios_device.go

…

ios_driver_wda.go

fix: optimize report command to avoid creating timestamp directories

2025-06-10 12:06:08 +08:00

ios_test.go

…

mcp_server_test.go

feat: 实现 AIQuery 功能并支持 OutputSchema

2025-06-13 10:27:08 +08:00

mcp_server.go

feat: 实现 AIQuery 功能并支持 OutputSchema

2025-06-13 10:27:08 +08:00

mcp_tools_ai.go

feat: 实现 AIQuery 功能并支持 OutputSchema

2025-06-13 10:27:08 +08:00

mcp_tools_app.go

feat: add mcp tool ToolGetForegroundApp

2025-06-08 19:25:09 +08:00

mcp_tools_button.go

feat: optimize MCP tools response format with automatic schema generation

2025-06-05 23:17:06 +08:00

mcp_tools_device.go

feat: add mcp tool ToolScreenRecord

2025-06-09 17:18:26 +08:00

mcp_tools_input.go

feat: optimize MCP tools response format with automatic schema generation

2025-06-05 23:17:06 +08:00

mcp_tools_screen.go

feat: optimize MCP tools response format with automatic schema generation

2025-06-05 23:17:06 +08:00

mcp_tools_swipe.go

change: remove unnecessary logs

2025-06-10 13:19:36 +08:00

mcp_tools_touch.go

feat: optimize MCP tools response format with automatic schema generation

2025-06-05 23:17:06 +08:00

mcp_tools_utility.go

feat: optimize MCP tools response format with automatic schema generation

2025-06-05 23:17:06 +08:00

mcp_tools_web.go

feat: optimize MCP tools response format with automatic schema generation

2025-06-05 23:17:06 +08:00

sdk.go

feat: optimize ILLMService interface to support different models for each component

2025-06-11 12:18:31 +08:00