Commit Graph

4473 Commits

Author SHA1 Message Date
lilong.129
229fd4678c fix: use localhost instead of 127.0.0.1 2025-05-27 20:16:10 +08:00
lilong.129
866cc0e4d2 feat: implement MCP hooks integration with anti_risk option 2025-05-27 19:46:08 +08:00
lilong.129
f4cc74b3ca docs: update dev instruct 2025-05-27 15:34:41 +08:00
lilong.129
6c60383f70 docs: add architecture 2025-05-27 15:28:03 +08:00
lilong.129
3936c0f487 change: remove unused code 2025-05-27 13:42:51 +08:00
lilong.129
404865ba6b refactor: complete ActionOptions unification and pointer type optimization 2025-05-27 13:34:12 +08:00
lilong.129
7fb966b7ba refactor: improve ActionMethod type safety and eliminate type conversions 2025-05-27 11:49:30 +08:00
lilong.129
466fe39cb9 docs: add comprehensive migration summary for ActionOptions and Request integration
- Document the complete integration process of ActionOptions and Request structures
- Include detailed statistics: 40 tools migrated with 100% test pass rate
- Provide technical implementation details and usage examples
- Record backward compatibility guarantees and migration helpers
- Summarize code quality improvements and performance optimizations
- Outline future development plans and goals

This documentation serves as a complete record of the unification initiative
and provides guidance for future development and maintenance.
2025-05-26 23:13:19 +08:00
lilong.129
a47d65bc4e feat: migrate all remaining MCP tools to use UnifiedActionRequest
- Migrated 39 remaining MCP tools from individual Request structures to UnifiedActionRequest
- All tools now use unifiedReq.GetMCPOptions(ACTION_*) instead of option.NewMCPOptions(Request{})
- Completed the unification of parameter definitions across all 40 MCP tools
- Eliminated duplication between ActionOptions and Request structures
- All tests pass, confirming successful migration

Tools migrated:
- Basic operations: TapAbsXY, TapByOCR, TapByCV, DoubleTapXY
- Device management: ListPackages, ScreenShot, GetScreenSize, PressButton
- App management: LaunchApp, TerminateApp, AppInstall, AppUninstall, AppClear
- Swipe operations: SwipeDirection, SwipeCoordinate, SwipeToTapApp, SwipeToTapText, SwipeToTapTexts
- Input/Navigation: Input, Home, Back, Drag
- Web operations: WebLoginNoneUI, SecondaryClick, HoverBySelector, TapBySelector, SecondaryClickBySelector, WebCloseTab
- System utilities: SetIme, GetSource, ClosePopups
- Sleep operations: SleepMS, SleepRandom
- AI/Task management: AIAction, Finished

This completes the ActionOptions and Request structures integration initiative.
2025-05-26 23:10:58 +08:00
lilong.129
8181e4244a refactor ToolSwipe to delegate to existing tools instead of duplicating logic
- Modified ToolSwipe.ConvertActionToCallToolRequest to delegate to ToolSwipeDirection and ToolSwipeCoordinate
- Removed duplicate parameter handling logic in favor of reusing existing implementations
- Fixed linter error by removing unused variable
- Maintained backward compatibility while reducing code duplication
- All tests pass, confirming the refactoring is successful
2025-05-26 22:42:50 +08:00
lilong.129
6ae4c300c1 add generic swipe tool with auto-detection of direction vs coordinate params
- Added ACTION_Swipe to option/action.go for generic swipe functionality
- Implemented ToolSwipe in mcp_server.go that automatically detects parameter type:
  - String params (up/down/left/right) use direction-based swipe logic
  - Array params [fromX, fromY, toX, toY] use coordinate-based swipe logic
- Added comprehensive test coverage for ToolSwipe in mcp_server_test.go
- Updated tool registration to include the new generic swipe tool
- All tests pass, confirming backward compatibility with existing tools
2025-05-26 22:39:23 +08:00
lilong.129
8895e9e970 merge mcp_tools_test.go into mcp_server_test.go
- Merged all individual MCP tool test functions from mcp_tools_test.go into mcp_server_test.go
- Added require import for additional test assertions
- Removed duplicate TestMCPServer4XTDriver function
- Deleted the original mcp_tools_test.go file
- All 39 MCP tools now have comprehensive unit tests in a single file
- Tests cover tool name, description, options, and request conversion functionality
2025-05-26 22:32:26 +08:00
lilong.129
77f5683f9a fix: remove unnecessary IgnoreNotFoundError and MaxRetryTimes from coordinate-based tap tools
- Removed IgnoreNotFoundError and MaxRetryTimes parameters from TapRequest, TapAbsXYRequest, and DoubleTapXYRequest structures
- Updated corresponding tool implementations to remove references to these non-existent fields
- These parameters are not applicable to coordinate-based operations as they don't involve element searching
- Only OCR/CV-based operations need these error handling parameters

This ensures that only relevant tools have the ignore_NotFoundError functionality,
making the API more consistent and avoiding confusion.
2025-05-26 22:10:08 +08:00
lilong.129
df65f9a828 fix: MCP server ignore_NotFoundError option not working
- Fixed TapByOCR and TapByCV tools to properly handle ignore_NotFoundError option
- Added option parameters to all MCP tool request structures
- Fixed ConvertActionToCallToolRequest methods to extract action options
- Added extractActionOptionsToArguments helper function for consistent option handling
- Extended fix to all MCP tools: SwipeToTapApp, SwipeToTapText, SwipeToTapTexts, TapXY, TapAbsXY
- Added comprehensive tests for option parameter handling
- Updated test expectations to match actual registered tools

This ensures that when ignore_NotFoundError is set to true, OCR/CV operations
will return nil instead of throwing errors when target elements are not found,
allowing tests to continue execution as expected.
2025-05-26 22:02:01 +08:00
lilong.129
9a5e0849de fix: handle GetOrCreateXTDriver when serial is empty 2025-05-26 21:25:25 +08:00
lilong.129
2569670c7f feat: implement unified XTDriver cache 2025-05-26 19:39:46 +08:00
lilong.129
1bd2b1ba5e change: move code 2025-05-26 16:08:27 +08:00
lilong.129
7045a9d452 change: check call tool result error 2025-05-26 15:30:51 +08:00
lilong.129
5eafcc8a2b merge master 2025-05-26 14:53:36 +08:00
lilong.129
f20fdd51bc feat: Validate model type and model name compatibility 2025-05-26 09:40:28 +08:00
lilong.129
4e74247cab fix: miss tool call ID 2025-05-26 09:28:46 +08:00
lilong.129
36c5044402 feat: add mcp tool finished 2025-05-26 09:05:48 +08:00
lilong.129
e60c362257 change: rename function 2025-05-26 08:49:06 +08:00
lilong.129
778344c826 change: remove call function tool 2025-05-26 00:43:01 +08:00
lilong.129
a888022cbc refactor: split driver cache 2025-05-26 00:35:56 +08:00
lilong.129
2e17d9df16 refactor: merge DoAction to mcp server tools 2025-05-25 23:53:07 +08:00
lilong.129
7986c4899f refactor: move DoAction to MCP tools call 2025-05-25 08:10:57 +08:00
lilong.129
4ff2692f02 refactor: move action options 2025-05-25 00:15:18 +08:00
lilong.129
97dad38b7b refactor: move tool request types to option 2025-05-24 23:51:58 +08:00
lilong.129
f65d8aebbd refactor: move model struct to types 2025-05-24 23:32:10 +08:00
lilong.129
02611d3d5a refactor: uixt MCP Server 2025-05-24 23:22:00 +08:00
lilong.129
0a68701faa refactor: move uixt mcp server 2025-05-24 10:46:50 +08:00
lilong.129
014140ccc7 change: append tool call message for planner 2025-05-24 10:28:55 +08:00
lilong.129
b639b4473f test: update unittests 2025-05-24 01:00:30 +08:00
lilong.129
81c854f963 refactor: merge ai parser 2025-05-24 00:25:44 +08:00
lilong.129
19ddcb40cc change: update ui-tars prompt 2025-05-23 22:05:21 +08:00
lilong.129
009bfa4ecb refactor: replace ui-tars parser with https://github.com/bytedance/UI-TARS/blob/main/codes/ui_tars/action_parser.py 2025-05-22 22:52:47 +08:00
lilong.129
3b77ade24f refactor: json asserter 2025-05-22 18:22:12 +08:00
lilong.129
c377664518 refactor: add LLMServiceTypeDoubaoVL 2025-05-22 15:34:11 +08:00
lilong.129
269fe2de23 fix: tap_xy, swipe handler 2025-05-21 22:51:51 +08:00
lilong.129
bb592548b4 fix: chat with screenshot 2025-05-21 22:35:16 +08:00
lilong.129
d58bbaeb5f fix: uixt tool take_screenshot 2025-05-21 18:05:12 +08:00
lilong.129
60e608f101 feat: add uixt tool swipe 2025-05-21 17:47:53 +08:00
lilong.129
7724cf0062 feat: add uixt tool press_button 2025-05-21 17:25:17 +08:00
lilong.129
5c68760cca feat: add uixt tool get_screen_size 2025-05-21 16:51:39 +08:00
lilong.129
0d6a37ecef feat: add uixt tool terminate_app 2025-05-21 16:42:53 +08:00
lilong.129
03553a4962 feat: add uixt tool launch_app 2025-05-21 16:37:42 +08:00
lilong.129
495443a2c4 feat: add uixt tool list_packages 2025-05-21 16:24:54 +08:00
lilong.129
044eb07a35 feat: add uixt tool select_device 2025-05-20 23:11:22 +08:00
lilong.129
1fa87819ae feat: add uixt tool list_available_devices 2025-05-20 22:58:16 +08:00