Commit Graph

4566 Commits

Author SHA1 Message Date
lilong.129
e3432a192d refactor: improve validation process by adding variable substitution and enhancing error handling 2025-06-18 23:14:43 +08:00
lilong.129
e40db65287 feat: enhance report generation with new AI query and validation display features 2025-06-18 22:35:19 +08:00
lilong.129
a3f2ff37bc refactor: replace hardcoded log messages with constants for better maintainability 2025-06-18 17:17:29 +08:00
lilong.129
1f3366453e feat: implement structured response parsing with enhanced error recovery and UTF-8 sanitization 2025-06-18 16:59:35 +08:00
lilong.129
6965cf9fe9 refactor: enhance screenshot functionality with session saving and optional CV processing 2025-06-18 16:13:45 +08:00
lilong.129
780870ffd9 feat: enhance log processing with improved time handling and sorting 2025-06-18 13:52:21 +08:00
lilong.129
a890981e2d fix: update StartTime to use UnixMilli for better precision across step functions 2025-06-18 13:51:44 +08:00
lilong.129
3d2707fa36 feat: add home and back action mappings to planner prompts 2025-06-18 12:01:53 +08:00
lilong.129
64ed72057f fix: update step failed in ParseStep 2025-06-18 11:31:28 +08:00
lilong.129
d20504f41f fix: add default context for UIXTConfig 2025-06-17 19:46:45 +08:00
李隆
d748acc682 Merge 'merge-wings' into 'master'
fix: create results dir

See merge request: !100
2025-06-17 11:41:12 +00:00
lilong.129
f21745c29d fix: create results dir 2025-06-17 14:09:49 +08:00
lilong.129
a78ba90d33 refactor: config results path 2025-06-15 23:31:36 +08:00
李隆
c62913ba1a Merge branch 'mcp-plugin' into 'master'
AI features & MCP Tools

See merge request iesqa/httprunner!88
2025-06-15 12:19:25 +00:00
lilong.129
4050fc3ffc change: update example llk 2025-06-15 00:47:20 +08:00
lilong.129
69b4b92904 feat: NewUIXTRunner 2025-06-15 00:42:03 +08:00
lilong.129
1145f424b1 feat: implement two-level auto popup handler configuration
- Add AutoPopupHandler field to both TConfig and StepConfig
- Support testcase-level global configuration via TConfig.EnableAutoPopupHandler()
- Support step-level specific configuration via StepMobile.EnableAutoPopupHandler()
- Priority: testcase config > step config > default disabled
- Simplify Loops field type from *types.IntOrString to int in StepConfig
- Update documentation to reflect new structure
2025-06-14 12:11:04 +08:00
lilong.129
b271e655b1 feat: add MCP plugin support and optimize AI service configuration
- Add UIXT runner with MCP plugin support
   - Refactor AI service options handling
   - Optimize configuration parsing for LLM and CV services
   - Update dependencies to latest versions
2025-06-13 20:24:57 +08:00
lilong.129
409cd693f0 refactor: GetScreenshotBase64WithSize 2025-06-13 12:01:21 +08:00
lilong.129
f6e7e970f8 feat: 实现 AIQuery 功能并支持 OutputSchema
- 新增 AIQuery 方法到 StepMobile,支持使用自然语言从屏幕中提取信息
- 实现 AIQuery 在 driver_ext_ai.go 中的完整功能,包括屏幕截图和 LLM 查询
- 添加 OutputSchema 支持,允许用户定义自定义输出格式进行结构化查询
- 新增 ToolAIQuery MCP 工具,完整集成到 MCP 服务器中
- 在 ActionOptions 中添加 OutputSchema 字段和 WithOutputSchema 选项函数
- 添加 ACTION_Query 的配置支持和字段映射
- 完善测试覆盖:
  * 添加 TestAIQuery 单元测试,包含多种 OutputSchema 使用场景
  * 添加 TestToolAIQuery MCP 工具测试
  * 定义 GameInfo、UIElementInfo 等结构体用于测试
- 更新文档:
  * 在 docs/uixt/ai.md 中添加完整的 AIQuery 使用指南
  * 包含基本用法、OutputSchema 示例、最佳实践等
- 支持复杂的嵌套结构体和数组类型的 OutputSchema
- 与现有 AIAction、AIAssert 功能保持一致的 API 设计
2025-06-13 10:27:08 +08:00
lilong.129
fb0418fa95 feat: add LianLianKan (连连看) game bot implementation
- Add complete LianLianKan game bot with AI-powered interface analysis
- Implement static analysis solver with 0-2 turn connection algorithms
- Support cross-platform game automation (Android, iOS, HarmonyOS, Browser)
- Include comprehensive test suite with real game data
- Add command line tool and documentation
- Integrate with HttpRunner @/uixt module and Doubao AI models
2025-06-12 17:51:23 +08:00
lilong.129
72df285fed fix: get resultsPath 2025-06-12 14:51:15 +08:00
lilong.129
51ee639cac docs: update docs 2025-06-11 14:57:08 +08:00
lilong.129
fbc888655f feat: optimize ILLMService interface to support different models for each component
- Add LLMServiceConfig to support mixed model configuration
- Enable Planner, Asserter, Querier to use different optimal models
- Provide recommended configurations for various use cases
- Maintain backward compatibility with existing API
- Update documentation to reflect current state without iteration history
- Merge test files and add comprehensive configuration tests
- Resolve circular dependency by moving config to option package
2025-06-11 12:18:31 +08:00
lilong.129
50414ec74d fix(ai): 修复 OpenAI 结构化输出解析问题并重构代码结构
- 修复 OpenAI structured output 的 properties 包装层解析问题
- 重构 parseCustomSchemaResult 函数,提高代码可维护性:
  - 拆分为多个职责单一的小函数
  - 消除重复的字段提取逻辑
  - 采用清晰的策略模式处理不同解析场景
- 增强测试用例,添加具体的数值和结构验证
- 保持完全向后兼容,所有现有测试通过

Fixes: TestQueryFunctionality/ComprehensiveAnalysis 测试失败问题
2025-06-11 11:15:02 +08:00
lilong.129
caf75b087b fix: remove unneccessary tests 2025-06-10 22:52:52 +08:00
lilong.129
514d321188 refactor: remove toggle buttons and expand all actions by default in HTML report 2025-06-10 21:24:21 +08:00
lilong.129
81a92ae155 docs: update AI module README with latest features
- Add comprehensive documentation for the new Query functionality
- Update interface method names from Call to Plan for consistency
- Add OpenAI GPT-4O model support documentation
- Include detailed usage examples for basic and custom schema queries
- Add configuration examples for multiple model services
- Document new features like ResetHistory, Usage statistics, and automatic type conversion
- Expand advanced features section with custom output format examples
- Update all code examples to reflect the latest API changes

The documentation now reflects the current state of the AI module with all three core capabilities:
- Planning (renamed from Call)
- Assertion
- Query (new feature)

All examples and configurations are updated to match the latest implementation.
2025-06-10 20:52:44 +08:00
lilong.129
c513e56d30 feat: add Query method to ILLMService interface
- Add Query method to ILLMService interface for unified AI service access
- Update combinedLLMService to include querier functionality
- Add comprehensive tests for ILLMService Query method
- Support both basic query and custom schema query through unified interface
- Add environment variable checks for test reliability

This allows users to access all AI capabilities (planning, assertion, and query) 
through a single ILLMService interface, providing better API consistency and ease of use.
2025-06-10 20:45:49 +08:00
lilong.129
7c45acd061 feat: add AI Querier module with custom output schema support and refactor common model calling logic
- Add new AI Querier module for structured information extraction from screenshots
- Support custom output schema for structured data response
- Implement automatic type conversion and data validation
- Add comprehensive test suite with various data structure examples
- Refactor callModelWithLogging to utils.go as shared function for planner, asserter, and querier
- Eliminate code duplication across AI modules (30+ lines of repeated code)
- Improve maintainability with unified logging and timing logic
- Add environment variable checks in test setup to handle missing API keys gracefully

Key features:
- Custom output schema support with JSON Schema generation
- Automatic data type conversion with reflection
- Fallback mechanisms for robust parsing
- Comprehensive documentation and usage examples
- Backward compatibility with existing functionality
2025-06-10 20:41:35 +08:00
lilong.129
fa9a53d2ae change: add test 2025-06-10 18:16:36 +08:00
lilong.129
304abe653a feat: optimize HTML report layout and clean up redundant code
- Redesign planning section with three-column layout
- Improve screenshot display with adaptive sizing
- Enhance actions details presentation
- Add compact request toggle functionality
- Remove unused CSS styles and redundant code
- Improve responsive design for mobile devices
2025-06-10 18:13:19 +08:00
lilong.129
9c906934fd fix: resolve Chinese character encoding issue in HTML report downloads
- Add decodeBase64UTF8 function to properly handle UTF-8 encoded Base64 content
- Replace atob() with TextDecoder for correct Chinese character decoding
- Explicitly specify UTF-8 charset when creating download Blob
- Fix garbled Chinese text when downloading summary.json from HTML report
2025-06-10 17:07:08 +08:00
lilong.129
98bd41ff33 fix: add direction parameter support for scroll operations in UI-TARS parser
- Handle direction parameter in convertProcessedArgs for scroll actions
- Ensure scroll operations map to swipe with both coordinates and direction
- Add comprehensive test coverage for scroll action parsing
- Fix issue where scroll direction was missing from tool call arguments
2025-06-10 16:40:10 +08:00
lilong.129
c322d7c36c fix: improve JSON extraction to handle UTF-8 Chinese characters properly
- Replace byte-based brace counting with UTF-8 aware rune iteration
- Add proper string state tracking to handle escaped quotes
- Add comprehensive test cases for Chinese character handling
- Fix parsing errors when JSON contains Chinese text like 2048经典
2025-06-10 16:09:50 +08:00
lilong.129
88ae8faee1 feat: enhance VLM response parsing and DOUBAO model support
- Fix JSON extraction logic by prioritizing brace counting method
- Add support for DOUBAO string array coordinate format
- Introduce IS_UI_TARS helper function for model type checking
- Add comprehensive tests for JSON parsing and coordinate handling
- Improve error handling with retry delays for LLM service failures
2025-06-10 15:56:13 +08:00
lilong.129
4959c2e47e feat: extractJSONFromContent 2025-06-10 14:08:44 +08:00
lilong.129
7dc0f869be fix: extracts JSON content from various formats in the response 2025-06-10 14:02:41 +08:00
lilong.129
90401eeb78 change: remove unnecessary logs 2025-06-10 13:19:36 +08:00
lilong.129
f5f6d177ab fix: optimize report command to avoid creating timestamp directories
- Implement lazy loading for directory creation in config.go
- Add logFile parameter to InitLogger for better control
- Use dynamic directory existence check instead of flags
- Report command now uses console-only logging to prevent directory creation
- Support both JSON and colorized console output formats
- Maintain backward compatibility for all other commands

Changes:
- config.go: Convert directory paths to getter methods with lazy creation
- logger.go: Add logFile parameter and improve logging control
- cmd/root.go: Detect report command and disable file logging
- uixt/*: Update all references to use new getter methods

Fixes the issue where 'hrp report results/' would create unwanted timestamp directories
2025-06-10 12:06:08 +08:00
lilong.129
6588d95154 fix: 修复 summary.json 中文乱码问题
- 改进 Dump2JSON 函数的文件写入方式,确保 UTF-8 编码正确处理
- 添加文件同步操作防止数据不完整
- 新增 UTF-8 编码测试验证修复效果
- 同步改进 HTML 报告生成的文件写入方式
2025-06-10 11:03:10 +08:00
lilong.129
12cebef3b9 change: set llm timeout to 120s 2025-06-09 22:42:19 +08:00
lilong.129
39acadb0a7 feat: add MCP tools registration to LLM service
- Add RegisterTools method to ILLMService interface
- Create shared MCP to eino tool converter
- Auto-register built-in uixt tools in XTDriver initialization
- Refactor MCPHost to use shared converter
- Add comprehensive test coverage for tool conversion

This enables doubao-1.5-thinking-vision-pro model to access
MCP tools through function calling mechanism.
2025-06-09 22:19:43 +08:00
lilong.129
dd52faef57 refactor: move Call function 2025-06-09 20:52:32 +08:00
lilong.129
f1544d4a5c feat: implement separate log levels for console and file output
- Console logger respects user-specified log level
- File logger always uses DEBUG level to capture all logs
- Add custom leveledMultiWriter for different output levels
- Remove global log level setting for more granular control
2025-06-09 19:16:39 +08:00
lilong.129
533c1f4bff feat: add mcp tool ToolScreenRecord 2025-06-09 17:18:26 +08:00
lilong.129
96da4515a1 feat: optimize test report UI and add LLM usage tracking 2025-06-09 17:04:55 +08:00
lilong.129
e85802cdda feat: add download for summary.json and hrp.log in report.html 2025-06-09 00:29:27 +08:00
lilong.129
a91a10ac13 docs: update cmd docs 2025-06-09 00:06:23 +08:00
lilong.129
cf360c8c46 feat: compress image data for html report 2025-06-08 23:48:23 +08:00