Commit Graph

35 Commits

Author SHA1 Message Date
lilong.129
58ce7efb8b change: upgrade eino sdk 2025-07-03 21:47:45 +08:00
lilong.129
1cdf14b85a change: update error logs 2025-06-30 16:57:41 +08:00
lilong.129
81a92ae155 docs: update AI module README with latest features
- Add comprehensive documentation for the new Query functionality
- Update interface method names from Call to Plan for consistency
- Add OpenAI GPT-4O model support documentation
- Include detailed usage examples for basic and custom schema queries
- Add configuration examples for multiple model services
- Document new features like ResetHistory, Usage statistics, and automatic type conversion
- Expand advanced features section with custom output format examples
- Update all code examples to reflect the latest API changes

The documentation now reflects the current state of the AI module with all three core capabilities:
- Planning (renamed from Call)
- Assertion
- Query (new feature)

All examples and configurations are updated to match the latest implementation.
2025-06-10 20:52:44 +08:00
lilong.129
39acadb0a7 feat: add MCP tools registration to LLM service
- Add RegisterTools method to ILLMService interface
- Create shared MCP to eino tool converter
- Auto-register built-in uixt tools in XTDriver initialization
- Refactor MCPHost to use shared converter
- Add comprehensive test coverage for tool conversion

This enables doubao-1.5-thinking-vision-pro model to access
MCP tools through function calling mechanism.
2025-06-09 22:19:43 +08:00
lilong.129
6e1bd5bbe2 feat: optimize MCP tools response format with automatic schema generation
- Remove all manual ReturnSchema() methods from tools
- Implement automatic schema generation using reflection
- Unify response format to flat structure with action/success/message fields
- Simplify tool implementation by removing MCPResponse embedding
- Update documentation to reflect new architecture
- Achieve ~70% code reduction while maintaining type safety
2025-06-05 23:17:06 +08:00
lilong.129
d883aa6a21 change: rename VLM name 2025-06-05 18:09:25 +08:00
lilong.129
0add3231ff refactor: merge ActionSummary and Thought fields to eliminate duplication
- Remove redundant ActionSummary field from PlanningResult struct
- Update parsers to use unified Thought field instead of duplicate fields
- Modify chat interface to display Thought instead of ActionSummary
- Update planner logging to use thought instead of summary
- Adjust prompt templates to use thought field consistently
- Switch test LLM service from UI-TARS to DoubaoVL
- Add default parameter handling for sleep tool
2025-06-05 14:19:09 +08:00
lilong.129
37028c4263 feat(mcphost): optimize shutdown logging to avoid false error messages
- Add identification for normal shutdown pipe errors in startStdioLog
- Optimize stdio log error handling logic to distinguish between normal shutdown and actual errors
- Add proper handling for SIGTERM (exit status 143) in isSignalError function
- Add debug logging for MCP config loading process
- Ensure clean shutdown without confusing error messages
2025-06-03 13:21:38 +08:00
lilong.129
9089bd9324 feat: 重构 MCP 工具导出逻辑并完善返回值类型系统 2025-05-31 00:28:24 +08:00
lilong.129
d3011d467e feat: enhance signal handling and graceful shutdown for MCP integration 2025-05-29 00:59:17 +08:00
lilong.129
4ea08b0198 feat: integrate MCP tools with UI actions and improve environment variable inheritance 2025-05-28 22:58:59 +08:00
lilong.129
866cc0e4d2 feat: implement MCP hooks integration with anti_risk option 2025-05-27 19:46:08 +08:00
lilong.129
7045a9d452 change: check call tool result error 2025-05-26 15:30:51 +08:00
lilong.129
f20fdd51bc feat: Validate model type and model name compatibility 2025-05-26 09:40:28 +08:00
lilong.129
0a68701faa refactor: move uixt mcp server 2025-05-24 10:46:50 +08:00
lilong.129
19ddcb40cc change: update ui-tars prompt 2025-05-23 22:05:21 +08:00
lilong.129
c377664518 refactor: add LLMServiceTypeDoubaoVL 2025-05-22 15:34:11 +08:00
lilong.129
269fe2de23 fix: tap_xy, swipe handler 2025-05-21 22:51:51 +08:00
lilong.129
bb592548b4 fix: chat with screenshot 2025-05-21 22:35:16 +08:00
lilong.129
d58bbaeb5f fix: uixt tool take_screenshot 2025-05-21 18:05:12 +08:00
lilong.129
60e608f101 feat: add uixt tool swipe 2025-05-21 17:47:53 +08:00
lilong.129
7724cf0062 feat: add uixt tool press_button 2025-05-21 17:25:17 +08:00
lilong.129
5c68760cca feat: add uixt tool get_screen_size 2025-05-21 16:51:39 +08:00
lilong.129
0d6a37ecef feat: add uixt tool terminate_app 2025-05-21 16:42:53 +08:00
lilong.129
03553a4962 feat: add uixt tool launch_app 2025-05-21 16:37:42 +08:00
lilong.129
495443a2c4 feat: add uixt tool list_packages 2025-05-21 16:24:54 +08:00
lilong.129
044eb07a35 feat: add uixt tool select_device 2025-05-20 23:11:22 +08:00
lilong.129
1fa87819ae feat: add uixt tool list_available_devices 2025-05-20 22:58:16 +08:00
lilong.129
0c20fe7b02 feat: add argument --with-uixt to start built-in uixt MCP server 2025-05-20 22:36:46 +08:00
lilong.129
037e69315e change: remove unused code 2025-05-20 18:03:54 +08:00
lilong.129
83434cca1e feat: load uixt mcp server in mcphost 2025-05-20 17:07:33 +08:00
lilong.129
5066c64368 refactor: move server models to uixt/types 2025-05-20 14:21:18 +08:00
lilong.129
3f1ee03529 refactor: mcphost planner 2025-05-18 21:55:01 +08:00
lilong.129
e35d644acf docs: update docs 2025-05-17 12:20:45 +08:00
lilong.129
e94dacb5b2 refactor: move mcphost package to top level 2025-05-17 11:55:26 +08:00