mirror of
https://github.com/httprunner/httprunner.git
synced 2026-05-12 02:21:29 +08:00
Merge pull request #910 from httprunner/v3
## 3.0.6 (2020-05-29) **Added** - feat: make referenced testcase as pytest class **Fixed** - fix: ensure converted python file in utf-8 encoding - fix: duplicate running referenced testcase - fix: ensure compatibility issues between testcase format v2 and v3 - fix: ensure compatibility with deprecated cli args in v2, include --failfast/--report-file/--save-tests - fix: UnicodeDecodeError when request body in protobuf **Changed** - change: make `allure-pytest`, `requests-toolbelt`, `filetype` as optional dependencies - change: move all unittests to tests folder - change: save testcase log in PWD/logs/ directory
This commit is contained in:
debugtalk
2
.github/workflows/unittest.yml
vendored
2
.github/workflows/unittest.yml
vendored
@@ -31,7 +31,7 @@ jobs:
|
||||
poetry run hmake
|
||||
poetry run hrun
|
||||
poetry run har2case
|
||||
poetry run coverage run --source=httprunner -m pytest httprunner
|
||||
poetry run coverage run --source=httprunner -m pytest tests
|
||||
poetry run coverage xml
|
||||
poetry run coverage report -m
|
||||
- name: Codecov
|
||||
|
||||
@@ -1,5 +1,25 @@
|
||||
# Release History
|
||||
|
||||
## 3.0.6 (2020-05-29)
|
||||
|
||||
**Added**
|
||||
|
||||
- feat: make referenced testcase as pytest class
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix: ensure converted python file in utf-8 encoding
|
||||
- fix: duplicate running referenced testcase
|
||||
- fix: ensure compatibility issues between testcase format v2 and v3
|
||||
- fix: ensure compatibility with deprecated cli args in v2, include --failfast/--report-file/--save-tests
|
||||
- fix: UnicodeDecodeError when request body in protobuf
|
||||
|
||||
**Changed**
|
||||
|
||||
- change: make `allure-pytest`, `requests-toolbelt`, `filetype` as optional dependencies
|
||||
- change: move all unittests to tests folder
|
||||
- change: save testcase log in PWD/logs/ directory
|
||||
|
||||
## 3.0.5 (2020-05-22)
|
||||
|
||||
**Added**
|
||||
@@ -83,476 +103,3 @@
|
||||
- generate reports/logs folder in current working directory
|
||||
- remove cli `--validate`
|
||||
- remove cli `--pretty`
|
||||
|
||||
## 2.5.7 (2020-02-21)
|
||||
|
||||
**Changed**
|
||||
|
||||
- feat: validate with python script, display print message
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix: validate script missing indents in html report
|
||||
- fix: validate with python script, display lineno error
|
||||
|
||||
## 2.5.6 (2020-02-19)
|
||||
|
||||
**Added**
|
||||
|
||||
- feat: save variables and export data to JSON files (named xx.io.json) when specified `--save-tests`
|
||||
|
||||
**Changed**
|
||||
|
||||
- change: alter HttpRunner default log_level to WARNING
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix: abort test when failed to parse all cases
|
||||
- fix: log error when parse failed
|
||||
|
||||
## 2.5.5 (2020-01-06)
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix: HTTP method missed "CONNECT", "TRACE"
|
||||
|
||||
**Changed**
|
||||
|
||||
- change: remove method validation from runner.Runner
|
||||
|
||||
## 2.5.4 (2020-01-03)
|
||||
|
||||
**Added**
|
||||
|
||||
- doc: add examples in json schema
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix #835: UnicodeDecodeError when loading json schema files
|
||||
- fix: RecursionError caused by checking root dir incorrectly on Windows
|
||||
|
||||
## 2.5.3 (2020-01-03)
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix json schema: variables maybe in string type, e.g. '${prepare_variables()}'
|
||||
- fix json schema: post json maybe in string type, e.g. '${prepare_post_data()}', '$post_data'
|
||||
|
||||
## 2.5.2 (2020-01-02)
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix #826: Windows does not support file name include ":"
|
||||
- fix #819: maximum recursion error in locusts
|
||||
- fix #818: request missed url in setup_hooks
|
||||
- fix #808: project_working_directory is not initialized when running passed in data structure
|
||||
|
||||
## 2.5.1 (2020-01-02)
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix: RefResolutionError on Windows platform
|
||||
|
||||
## 2.5.0 (2020-01-01)
|
||||
|
||||
**Added**
|
||||
|
||||
- feat: add json schema validation for api
|
||||
- feat: add json schema validation for testcase v1 & v2
|
||||
- feat: add json schema validation for testsuite v1 & v2
|
||||
|
||||
**Changed**
|
||||
|
||||
- refactor: use loader.load_cases to validate test files
|
||||
- refactor: use is_test_path to check if path is valid json/yaml file or a existed directory
|
||||
- refactor: use is_test_content to check if data_structure is apis/testcases/testsuites
|
||||
|
||||
## 2.4.9 (2019-12-29)
|
||||
|
||||
**Added**
|
||||
|
||||
- test: add unittest for cli
|
||||
|
||||
**Changed**
|
||||
|
||||
- change: html report name defaults to be in UTC ISO 8601 format
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix: display validators in report when validate raised exception
|
||||
- fix: eval validator python script before validating
|
||||
- fix: do not strip string content when preparing lazy data
|
||||
- fix: catch ApiNotFound exception when loading testcases
|
||||
- fix: print exception string with exception stage
|
||||
|
||||
## 2.4.8 (2019-12-25)
|
||||
|
||||
**Added**
|
||||
|
||||
- feat: store parse failed api/testcase/testsuite file path in `logs/xxx.parse_failed.json`
|
||||
- feat: add exception SummaryEmpty
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix: display request & response details in report when extraction failed
|
||||
- fix: include CHANGELOG in package
|
||||
|
||||
**Changed**
|
||||
|
||||
- change: use sys.exit(code) in hrun main
|
||||
|
||||
## 2.4.7 (2019-12-24)
|
||||
|
||||
**Added**
|
||||
|
||||
- feat: report user id to sentry
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix #797: locusts command error
|
||||
|
||||
## 2.4.6 (2019-12-23)
|
||||
|
||||
**Added**
|
||||
|
||||
- feat: report tests start event and running exception to sentry
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix: ensure initializing sentry_sdk on startup
|
||||
|
||||
**Fixed**
|
||||
|
||||
## 2.4.5 (2019-12-20)
|
||||
|
||||
**Added**
|
||||
|
||||
- feat: integrate sentry sdk
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix: catch UnicodeDecodeError when json loads request body
|
||||
- fix: display indented json for request json body
|
||||
|
||||
**Changed**
|
||||
|
||||
- change: detect request/response bytes encoding, instead of assuming utf-8
|
||||
- refactor: make report as submodule
|
||||
|
||||
## 2.4.4 (2019-12-17)
|
||||
|
||||
**Added**
|
||||
|
||||
- feat: add keyword `body` to reference response body
|
||||
|
||||
**Changed**
|
||||
|
||||
- refactor: dumps request/response headers, display indented json in html report
|
||||
- refactor: dumps request/response body if it is in json format, display indented json in html report
|
||||
- change: unify response field(content/json/text) to `body` in html report
|
||||
|
||||
## 2.4.3 (2019-12-16)
|
||||
|
||||
**Added**
|
||||
|
||||
- feat: load api content on demand
|
||||
|
||||
**Changed**
|
||||
|
||||
- refactor: use poetry>=1.0.0
|
||||
- test: migrate from travis CI to github actions
|
||||
- test: migrate from coveralls to codecov
|
||||
- test: run matrix tests on linux/macos/~~windows~~ and Python 2.7/3.5/3.6/3.7/3.8
|
||||
|
||||
## 2.4.2 (2019-12-13)
|
||||
|
||||
**Changed**
|
||||
|
||||
- refactor: replace with open file handler, avoid reading files into memory
|
||||
- refactor: rename plugin to extension, httprunner/plugins -> httprunner/ext
|
||||
- docs: update installation doc for developers
|
||||
|
||||
## 2.4.1 (2019-12-12)
|
||||
|
||||
**Added**
|
||||
|
||||
- feat: add `upload` keyword for upload test, see [doc](https://docs.httprunner.org/prepare/upload-case/)
|
||||
- test: pip install package
|
||||
- test: hrun command
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix: typo testfile_paths
|
||||
- fix: check if locustio installed
|
||||
- fix: dump json file name is empty when running relative testfile
|
||||
|
||||
## 2.4.0 (2019-12-11)
|
||||
|
||||
**Added**
|
||||
|
||||
- feat: validate with python script, ref #773
|
||||
- feat: rearrange html report, failed testcases will be displayed on top.
|
||||
|
||||
**Changed**
|
||||
|
||||
- refactor: make loader as submodule, split to check/locate/load/buildup
|
||||
- refactor: make built_in as submodule, split to comparators and functions
|
||||
- refactor: adjust code for context and validator
|
||||
- docs: update cli argument help
|
||||
- adjust format code, remove unused import
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix: keep setup/teardown hooks original order when merge & override.
|
||||
- fix: length comparator exceptions when running in CSV data-driven mode.
|
||||
|
||||
## 2.3.3 (2019-12-04)
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix #768: dump json file path error when folder name contains dot, such as `a.b.c`
|
||||
|
||||
**Changed**
|
||||
|
||||
- change: rename builtin function, sleep_N_secs => sleep
|
||||
|
||||
## 2.3.2 (2019-11-01)
|
||||
|
||||
**Added**
|
||||
|
||||
- docs: add docs content to repo, visit at `https://docs.httprunner.org`
|
||||
- docs: update developer interface docs
|
||||
|
||||
**Changed**
|
||||
|
||||
- rename `render_html_report` to `gen_html_report`
|
||||
- make gen_html_report separate with HttpRunner().run_tests()
|
||||
- `--report-file`: specify report file path, this has higher priority than specifying report dir.
|
||||
- remove `summary` property from HttpRunner
|
||||
|
||||
## 2.3.1 (2019-10-28)
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix locusts entry configuration
|
||||
|
||||
**Changed**
|
||||
|
||||
- update PyPi classifiers
|
||||
|
||||
## 2.3.0 (2019-10-27)
|
||||
|
||||
**Added**
|
||||
|
||||
- feat: implement plugin system prototype, make locusts as plugin
|
||||
- test: add Python 3.8 to Travis-CI
|
||||
- feat: add `__main__.py`, `python -m httprunner` can be used to hrun tests
|
||||
|
||||
**Changed**
|
||||
|
||||
- update dependency versions in pyproject.toml
|
||||
- rename folder, httprunner/templates => httprunner/static
|
||||
- log httprunner version before running tests
|
||||
- remove unused import & code
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix #707: duration stat error in multiple testsuites
|
||||
|
||||
## 2.2.6 (2019-09-18)
|
||||
|
||||
**Added**
|
||||
|
||||
- feat: config variables support parsing from function
|
||||
- feat: support [jsonpath](https://goessner.net/articles/JsonPath/) to parse json response [#679](https://github.com/httprunner/httprunner/pull/679)
|
||||
- feat: generate html report with specified report file [#704](https://github.com/httprunner/httprunner/pull/704)
|
||||
|
||||
**Changed**
|
||||
|
||||
- remove unused import
|
||||
- adjust code format
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix: dev-rules link 404
|
||||
|
||||
## 2.2.5 (2019-07-28)
|
||||
|
||||
**Added**
|
||||
|
||||
- log HttpRunner version when initializing
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix #658: sys.exit 1 if any testcase failed
|
||||
- fix ModuleNotFoundError in debugging mode if httprunner uninstalled
|
||||
|
||||
## 2.2.4 (2019-07-18)
|
||||
|
||||
**Changed**
|
||||
|
||||
- replace pipenv & setup.py with poetry
|
||||
- drop support for Python 3.4 as it was EOL on 2019-03-16
|
||||
- relocate debugging scripts, move from main-debug.py to httprunner.cli
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix #574: delete unnecessary code
|
||||
- fix #551: raise if times is not digit
|
||||
- fix #572: tests_def_mapping["testcases"] typo error
|
||||
|
||||
## 2.2.3 (2019-06-30)
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix yaml FullLoader AttributeError when PyYAML version < 5.1
|
||||
|
||||
## 2.2.2 (2019-06-26)
|
||||
|
||||
**Changed**
|
||||
|
||||
- `extract` is used to replace `output` when passing former teststep's (as a testcase) export value to next teststep
|
||||
- `export` is used to replace `output` in testcase config
|
||||
|
||||
## 2.2.1 (2019-06-25)
|
||||
|
||||
**Added**
|
||||
|
||||
- add demo api/testcase/testsuite to new created scaffold project
|
||||
- update default `.gitignore` of new created scaffold project
|
||||
- add demo content to `debugtalk.py`/`.env` of new created scaffold project
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix extend with testcase reference in format version 2
|
||||
- fix ImportError when locustio is not installed
|
||||
- fix YAMLLoadWarning by specify yaml loader
|
||||
|
||||
## 2.2.0 (2019-06-24)
|
||||
|
||||
**Added**
|
||||
|
||||
- support testcase/testsuite in format version 2
|
||||
|
||||
**Fixed**
|
||||
|
||||
- add wheel in dev packages
|
||||
- fix exception when teststep name reference former extracted variable
|
||||
|
||||
## 2.1.3 (2019-04-24)
|
||||
|
||||
**Fixed**
|
||||
|
||||
- replace eval mechanism with builtins to prevent security vulnerabilities
|
||||
- ImportError for builtins in Python2.7
|
||||
|
||||
## 2.1.2 (2019-04-17)
|
||||
|
||||
**Added**
|
||||
|
||||
- support new variable notation ${var}
|
||||
- use \$\$ to escape \$ notation
|
||||
- add Python 3.7 for travis CI
|
||||
|
||||
**Fixed**
|
||||
|
||||
- match duplicate variable/function in single raw string
|
||||
- escape '{' and '}' notation in raw string
|
||||
- print_info: TypeError when value is None
|
||||
- display api name when running api as testcase
|
||||
|
||||
## 2.1.1 (2019-04-11)
|
||||
|
||||
**Changed**
|
||||
|
||||
refactor upload files mechanism with [requests-toolbelt](https://toolbelt.readthedocs.io/en/latest/user.html#multipart-form-data-encoder):
|
||||
|
||||
- simplify usage syntax, detect mimetype with [filetype](https://github.com/h2non/filetype.py).
|
||||
- support upload multiple fields.
|
||||
|
||||
## 2.1.0 (2019-04-10)
|
||||
|
||||
**Added**
|
||||
|
||||
- implement json dump Python objects when save tests
|
||||
- implement lazy parser
|
||||
- remove project_mapping from parse_tests result
|
||||
|
||||
**Fixed**
|
||||
|
||||
- reference output variables
|
||||
- pass output variables between testcases
|
||||
|
||||
## 2.0.6 (2019-03-18)
|
||||
|
||||
**Added**
|
||||
|
||||
- create .gitignore file when initializing new project
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix CSV relative path detection
|
||||
- fix current validators displaying the former one when they are empty
|
||||
|
||||
## 2.0.5 (2019-03-04)
|
||||
|
||||
**Added**
|
||||
|
||||
- implement method to get variables and output
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix xss in response json
|
||||
|
||||
## 2.0.4 (2019-02-28)
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix verify priority with nested testcase
|
||||
- fix function in config variables called multiple times
|
||||
- dump loaded tests when running tests_mapping directly
|
||||
|
||||
## 2.0.3 (2019-02-24)
|
||||
|
||||
**Fixed**
|
||||
|
||||
- fix verify priority: teststep > config
|
||||
- fix Chinese charactor in log_file encoding error in Windows
|
||||
- fix dump file with Chinese charactor in Python 3
|
||||
|
||||
## 2.0.2 (2019-01-21)
|
||||
|
||||
**Fixed**
|
||||
|
||||
- each teststeps in one testcase share the same session
|
||||
- fix duplicate API definition output
|
||||
|
||||
**Changed**
|
||||
|
||||
- display result from hook functions in DEBUG level log
|
||||
- change log level of "Variables & Output" to INFO
|
||||
- print Invalid testcase path or testcases
|
||||
- print testcase output in INFO level log
|
||||
|
||||
## 2.0.1 (2019-01-18)
|
||||
|
||||
**Fixed**
|
||||
|
||||
- override current teststep variables with former testcase output variables
|
||||
- Fixed compatibility when testcase name is empty
|
||||
- skip undefined variable when parsing string content
|
||||
|
||||
**Changed**
|
||||
|
||||
- add back request method in report
|
||||
|
||||
## 2.0.0 (2019-01-01)
|
||||
|
||||
**Changed**
|
||||
|
||||
- Massive Refactor and Simplification
|
||||
- Redesign testcase structure
|
||||
- Module pipline
|
||||
- Start Semantic Versioning
|
||||
- Switch to Apache 2.0 license
|
||||
- Change logo
|
||||
|
||||
14
docs/FAQ.md
14
docs/FAQ.md
@@ -1,14 +0,0 @@
|
||||
# 常见问题
|
||||
|
||||
## HTTPS SSLError
|
||||
|
||||
请求 HTTPS 接口时，若本地开启了代理软件（Charles/Fiddler），由于 HTTPS 证书的原因，会导致 SSLError 的报错。
|
||||
|
||||
解决的方式是，在 config 中增加 `verify: False`，原理见 requests 的 [`SSL Cert Verification`](https://requests.kennethreitz.org/en/master/user/advanced/#ssl-cert-verification) 部分。
|
||||
|
||||
```yaml
|
||||
config:
|
||||
name: XXX
|
||||
base_url: XXX
|
||||
verify: False
|
||||
```
|
||||
@@ -1,145 +0,0 @@
|
||||
## 运行环境
|
||||
|
||||
HttpRunner 是一个基于 Python 开发的测试框架，可以运行在 macOS、Linux、Windows 系统平台上。
|
||||
|
||||
**Python 版本**：HttpRunner 支持 Python 3.5 及以上的所有版本，并使用 Travis-CI 进行了[持续集成测试][travis-ci]，测试覆盖的版本包括 2.7/3.5/3.6/3.7/3.8。虽然 HttpRunner 暂时保留了对 Python 2.7 的兼容支持，但强烈建议使用 Python 3.6 及以上版本。
|
||||
|
||||
**操作系统**：推荐使用 macOS/Linux。
|
||||
|
||||
## 安装方式
|
||||
|
||||
HttpRunner 的稳定版本托管在 PyPI 上，可以使用 `pip` 进行安装。
|
||||
|
||||
```bash
|
||||
$ pip install httprunner
|
||||
```
|
||||
|
||||
如果你需要使用最新的开发版本，那么可以采用项目的 GitHub 仓库地址进行安装：
|
||||
|
||||
```bash
|
||||
$ pip install git+https://github.com/HttpRunner/HttpRunner.git@master
|
||||
```
|
||||
|
||||
## 版本升级
|
||||
|
||||
假如你之前已经安装过了 HttpRunner，现在需要升级到最新版本，那么你可以使用`-U`参数。该参数对以上三种安装方式均生效。
|
||||
|
||||
```bash
|
||||
$ pip install -U HttpRunner
|
||||
$ pip install -U git+https://github.com/HttpRunner/HttpRunner.git@master
|
||||
```
|
||||
|
||||
## 安装校验
|
||||
|
||||
在 HttpRunner 安装成功后，系统中会新增如下 5 个命令：
|
||||
|
||||
- `httprunner`: 核心命令
|
||||
- `ate`: 曾经用过的命令（当时框架名称为 ApiTestEngine），功能与 httprunner 完全相同
|
||||
- `hrun`: httprunner 的缩写，功能与 httprunner 完全相同
|
||||
- `locusts`: 基于 [Locust][Locust] 实现[性能测试](run-tests/load-test.md)
|
||||
- [`har2case`][har2case]: 辅助工具，可将标准通用的 HAR 格式（HTTP Archive）转换为`YAML/JSON`格式的测试用例
|
||||
|
||||
httprunner、hrun、ate 三个命令完全等价，功能特性完全相同，个人推荐使用`hrun`命令。
|
||||
|
||||
运行如下命令，若正常显示版本号，则说明 HttpRunner 安装成功。
|
||||
|
||||
```text
|
||||
$ hrun -V
|
||||
2.4.1
|
||||
|
||||
$ har2case -V
|
||||
0.3.1
|
||||
```
|
||||
|
||||
## 开发者模式
|
||||
|
||||
默认情况下，安装 HttpRunner 的时候只会安装运行 HttpRunner 的必要依赖库。
|
||||
|
||||
如果你不仅仅是使用 HttpRunner，还需要对 HttpRunner 进行开发调试（debug），那么就需要进行如下操作。
|
||||
|
||||
HttpRunner 使用 [poetry][poetry] 对依赖包进行管理，若你还没有安装 poetry，需要先执行如下命令进行安装：
|
||||
|
||||
```bash
|
||||
$ curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python
|
||||
```
|
||||
|
||||
获取 HttpRunner 源码：
|
||||
|
||||
```bash
|
||||
$ git clone https://github.com/HttpRunner/HttpRunner.git
|
||||
```
|
||||
|
||||
进入仓库目录，安装所有依赖：
|
||||
|
||||
```bash
|
||||
$ poetry install
|
||||
```
|
||||
|
||||
运行单元测试，若测试全部通过，则说明环境正常。
|
||||
|
||||
```bash
|
||||
$ poetry run python -m unittest discover
|
||||
```
|
||||
|
||||
查看 HttpRunner 的依赖情况：
|
||||
|
||||
```bash
|
||||
$ poetry show --tree
|
||||
colorama 0.4.1 Cross-platform colored terminal text.
|
||||
colorlog 4.0.2 Log formatting with colors!
|
||||
└── colorama *
|
||||
coverage 4.5.4 Code coverage measurement for Python
|
||||
coveralls 1.8.2 Show coverage stats online via coveralls.io
|
||||
├── coverage >=3.6,<5.0
|
||||
├── docopt >=0.6.1
|
||||
├── requests >=1.0.0
|
||||
│ ├── certifi >=2017.4.17
|
||||
│ ├── chardet >=3.0.2,<3.1.0
|
||||
│ ├── idna >=2.5,<2.9
|
||||
│ └── urllib3 >=1.21.1,<1.25.0 || >1.25.0,<1.25.1 || >1.25.1,<1.26
|
||||
└── urllib3 *
|
||||
filetype 1.0.5 Infer file type and MIME type of any file/buffer. No external dependencies.
|
||||
flask 0.12.4 A microframework based on Werkzeug, Jinja2 and good intentions
|
||||
├── click >=2.0
|
||||
├── itsdangerous >=0.21
|
||||
├── jinja2 >=2.4
|
||||
│ └── markupsafe >=0.23
|
||||
└── werkzeug >=0.7
|
||||
future 0.18.1 Clean single-source support for Python 3 and 2
|
||||
har2case 0.3.1 Convert HAR(HTTP Archive) to YAML/JSON testcases for HttpRunner.
|
||||
└── pyyaml *
|
||||
jinja2 2.10.3 A very fast and expressive template engine.
|
||||
└── markupsafe >=0.23
|
||||
jsonpath 0.82 An XPath for JSON
|
||||
pyyaml 5.1.2 YAML parser and emitter for Python
|
||||
requests 2.22.0 Python HTTP for Humans.
|
||||
├── certifi >=2017.4.17
|
||||
├── chardet >=3.0.2,<3.1.0
|
||||
├── idna >=2.5,<2.9
|
||||
└── urllib3 >=1.21.1,<1.25.0 || >1.25.0,<1.25.1 || >1.25.1,<1.26
|
||||
requests-toolbelt 0.9.1 A utility belt for advanced users of python-requests
|
||||
└── requests >=2.0.1,<3.0.0
|
||||
├── certifi >=2017.4.17
|
||||
├── chardet >=3.0.2,<3.1.0
|
||||
├── idna >=2.5,<2.9
|
||||
└── urllib3 >=1.21.1,<1.25.0 || >1.25.0,<1.25.1 || >1.25.1,<1.26
|
||||
```
|
||||
|
||||
调试运行方式：
|
||||
|
||||
```bash
|
||||
# 调试运行 hrun
|
||||
$ poetry run python -m httprunner -h
|
||||
|
||||
# 调试运行 locusts
|
||||
$ pipenv run python -m httprunner.ext.locusts -h
|
||||
```
|
||||
|
||||
## Docker
|
||||
|
||||
TODO
|
||||
|
||||
[travis-ci]: https://travis-ci.org/HttpRunner/HttpRunner
|
||||
[Locust]: http://locust.io/
|
||||
[har2case]: https://github.com/HttpRunner/har2case
|
||||
[poetry]: https://github.com/sdispater/poetry
|
||||
@@ -1,41 +0,0 @@
|
||||
# HttpRunner V2.x 中文使用文档
|
||||
|
||||
## 在线阅读
|
||||
|
||||
本文档托管在`GitHub Pages`上，访问地址：
|
||||
|
||||
https://docs.httprunner.org
|
||||
|
||||
## 本地预览
|
||||
|
||||
### 安装依赖
|
||||
|
||||
本项目文档采用[`mkdocs`][mkdocs]生成，如需在本地预览查看，则需安装该工具。
|
||||
|
||||
```bash
|
||||
$ pip install mkdocs
|
||||
```
|
||||
|
||||
`mkdocs`支持主题配置，本项目选择了第三方的[`mkdocs-material`][mkdocs-material]。
|
||||
|
||||
```bash
|
||||
$ pip install mkdocs-material
|
||||
```
|
||||
|
||||
### 启动本地server
|
||||
|
||||
在项目根目录中运行如下命令：
|
||||
|
||||
```bash
|
||||
$ mkdocs serve
|
||||
INFO - Building documentation...
|
||||
INFO - Cleaning site directory
|
||||
[I 180211 22:48:35 server:283] Serving on http://127.0.0.1:8000
|
||||
[I 180211 22:48:35 handlers:60] Start watching changes
|
||||
[I 180211 22:48:35 handlers:62] Start detecting changes
|
||||
```
|
||||
|
||||
然后在浏览器中访问`http://127.0.0.1:8000`即可。
|
||||
|
||||
[mkdocs]: http://www.mkdocs.org/
|
||||
[mkdocs-material]: https://squidfunk.github.io/mkdocs-material/
|
||||
@@ -1,67 +0,0 @@
|
||||
## 测试用例（testcase）
|
||||
|
||||
从 2.0 版本开始，HttpRunner 开始对测试用例的定义进行进一步的明确，参考 [wiki][wiki_testcase] 上的描述。
|
||||
|
||||
> A test case is a specification of the inputs, execution conditions, testing procedure, and expected results that define a single test to be executed to achieve a particular software testing objective, such as to exercise a particular program path or to verify compliance with a specific requirement.
|
||||
|
||||
概括下来，一条测试用例（testcase）应该是为了测试某个特定的功能逻辑而精心设计的，并且至少包含如下几点：
|
||||
|
||||
- 明确的测试目的（achieve a particular software testing objective）
|
||||
- 明确的输入（inputs）
|
||||
- 明确的运行环境（execution conditions）
|
||||
- 明确的测试步骤描述（testing procedure）
|
||||
- 明确的预期结果（expected results）
|
||||
|
||||
对应地，HttpRunner 的测试用例描述方式进行如下设计：
|
||||
|
||||
- 测试用例应该是完整且独立的，每条测试用例应该是都可以独立运行的；在 HttpRunner 中，每个 `YAML/JSON` 文件对应一条测试用例。
|
||||
- 测试用例包含 `测试脚本` 和 `测试数据` 两部分：
|
||||
- `测试用例 = 测试脚本 + 测试数据`
|
||||
- `测试脚本` 重点是描述测试的 `业务功能逻辑`，包括预置条件、测试步骤、预期结果等，并且可以结合辅助函数（debugtalk.py）实现复杂的运算逻辑；可以将 `测试脚本` 理解为编程语言中的 `类（class）`；
|
||||
- `测试数据` 重点是对应测试的 `业务数据逻辑`，可以理解为类的实例化数据；
|
||||
- `测试数据` 和 `测试脚本` 分离后，就可以比较方便地实现数据驱动测试，通过对测试脚本传入一组数据，实现同一业务功能在不同数据逻辑下的测试验证。
|
||||
|
||||
|
||||
## 测试步骤（teststep）
|
||||
|
||||
测试用例是测试步骤的 `有序` 集合，而对于接口测试来说，每一个测试步骤应该就对应一个 API 的请求描述。
|
||||
|
||||
## 测试用例集（testsuite）
|
||||
|
||||
`测试用例集` 是 `测试用例` 的 `无序` 集合，集合中的测试用例应该都是相互独立，不存在先后依赖关系的。
|
||||
|
||||
如果确实存在先后依赖关系怎么办，例如登录功能和下单功能。正确的做法应该是，在下单测试用例的前置步骤中执行登录操作。
|
||||
|
||||
```yaml
|
||||
- config:
|
||||
name: order product
|
||||
|
||||
- test:
|
||||
name: login
|
||||
testcase: testcases/login.yml
|
||||
|
||||
- test:
|
||||
name: add to cart
|
||||
api: api/add_cart.yml
|
||||
|
||||
- test:
|
||||
name: make order
|
||||
api: api/make_order.yml
|
||||
```
|
||||
|
||||
## 测试场景
|
||||
|
||||
`测试场景` 和 `测试用例集` 是同一概念，都是 `测试用例` 的 `无序` 集合。
|
||||
|
||||
|
||||
- 接口
|
||||
- 测试用例集
|
||||
- 参数
|
||||
- 变量
|
||||
- 测试脚本（YAML/JSON）
|
||||
- debugtalk.py
|
||||
- 环境变量
|
||||
|
||||
## 项目根目录
|
||||
|
||||
[wiki_testcase]: https://en.wikipedia.org/wiki/Test_case
|
||||
@@ -1,4 +0,0 @@
|
||||
username,password,phone
|
||||
test1,111111,18600000001
|
||||
test2,222222,18600000002
|
||||
test3,333333,18600000003
|
||||
|
@@ -1,223 +0,0 @@
|
||||
import hashlib
|
||||
import hmac
|
||||
import json
|
||||
import random
|
||||
import string
|
||||
from functools import wraps
|
||||
|
||||
from flask import Flask, make_response, request
|
||||
|
||||
SECRET_KEY = "DebugTalk"
|
||||
|
||||
app = Flask(__name__)
|
||||
|
||||
""" storage all users' data
|
||||
data structure:
|
||||
users_dict = {
|
||||
'uid1': {
|
||||
'name': 'name1',
|
||||
'password': 'pwd1'
|
||||
},
|
||||
'uid2': {
|
||||
'name': 'name2',
|
||||
'password': 'pwd2'
|
||||
}
|
||||
}
|
||||
"""
|
||||
users_dict = {}
|
||||
|
||||
""" storage all token data
|
||||
data structure:
|
||||
token_dict = {
|
||||
'device_sn1': 'token1',
|
||||
'device_sn2': 'token1'
|
||||
}
|
||||
"""
|
||||
token_dict = {}
|
||||
|
||||
|
||||
def gen_random_string(str_len):
|
||||
""" generate random string with specified length
|
||||
"""
|
||||
return ''.join(
|
||||
random.choice(string.ascii_letters + string.digits) for _ in range(str_len))
|
||||
|
||||
def get_sign(*args):
|
||||
content = ''.join(args).encode('ascii')
|
||||
sign_key = SECRET_KEY.encode('ascii')
|
||||
sign = hmac.new(sign_key, content, hashlib.sha1).hexdigest()
|
||||
return sign
|
||||
|
||||
def gen_md5(*args):
|
||||
return hashlib.md5("".join(args).encode('utf-8')).hexdigest()
|
||||
|
||||
|
||||
def validate_request(func):
|
||||
|
||||
@wraps(func)
|
||||
def wrapper(*args, **kwargs):
|
||||
device_sn = request.headers.get('device_sn', "")
|
||||
token = request.headers.get('token', "")
|
||||
|
||||
if not device_sn or not token:
|
||||
result = {
|
||||
'success': False,
|
||||
'msg': "device_sn or token is null."
|
||||
}
|
||||
response = make_response(json.dumps(result), 401)
|
||||
response.headers["Content-Type"] = "application/json"
|
||||
return response
|
||||
|
||||
if token_dict.get(device_sn) != token:
|
||||
result = {
|
||||
'success': False,
|
||||
'msg': "Authorization failed!"
|
||||
}
|
||||
response = make_response(json.dumps(result), 403)
|
||||
response.headers["Content-Type"] = "application/json"
|
||||
return response
|
||||
|
||||
return func(*args, **kwargs)
|
||||
|
||||
return wrapper
|
||||
|
||||
|
||||
@app.route('/')
|
||||
def index():
|
||||
return "Hello World!"
|
||||
|
||||
@app.route('/api/get-token', methods=['POST'])
|
||||
def get_token():
|
||||
device_sn = request.headers.get('device_sn', "")
|
||||
os_platform = request.headers.get('os_platform', "")
|
||||
app_version = request.headers.get('app_version', "")
|
||||
data = request.get_json()
|
||||
sign = data.get('sign', "")
|
||||
|
||||
expected_sign = get_sign(device_sn, os_platform, app_version)
|
||||
|
||||
if expected_sign != sign:
|
||||
result = {
|
||||
'success': False,
|
||||
'msg': "Authorization failed!"
|
||||
}
|
||||
response = make_response(json.dumps(result), 403)
|
||||
else:
|
||||
token = gen_random_string(16)
|
||||
token_dict[device_sn] = token
|
||||
|
||||
result = {
|
||||
'success': True,
|
||||
'token': token
|
||||
}
|
||||
response = make_response(json.dumps(result))
|
||||
|
||||
response.headers["Content-Type"] = "application/json"
|
||||
return response
|
||||
|
||||
@app.route('/api/users')
|
||||
@validate_request
|
||||
def get_users():
|
||||
users_list = [user for uid, user in users_dict.items()]
|
||||
users = {
|
||||
'success': True,
|
||||
'count': len(users_list),
|
||||
'items': users_list
|
||||
}
|
||||
response = make_response(json.dumps(users))
|
||||
response.headers["Content-Type"] = "application/json"
|
||||
return response
|
||||
|
||||
@app.route('/api/reset-all')
|
||||
@validate_request
|
||||
def clear_users():
|
||||
users_dict.clear()
|
||||
result = {
|
||||
'success': True
|
||||
}
|
||||
response = make_response(json.dumps(result))
|
||||
response.headers["Content-Type"] = "application/json"
|
||||
return response
|
||||
|
||||
@app.route('/api/users/<int:uid>', methods=['POST'])
|
||||
@validate_request
|
||||
def create_user(uid):
|
||||
user = request.get_json()
|
||||
if uid not in users_dict:
|
||||
result = {
|
||||
'success': True,
|
||||
'msg': "user created successfully."
|
||||
}
|
||||
status_code = 201
|
||||
users_dict[uid] = user
|
||||
else:
|
||||
result = {
|
||||
'success': False,
|
||||
'msg': "user already existed."
|
||||
}
|
||||
status_code = 500
|
||||
|
||||
response = make_response(json.dumps(result), status_code)
|
||||
response.headers["Content-Type"] = "application/json"
|
||||
return response
|
||||
|
||||
@app.route('/api/users/<int:uid>')
|
||||
@validate_request
|
||||
def get_user(uid):
|
||||
user = users_dict.get(uid, {})
|
||||
if user:
|
||||
result = {
|
||||
'success': True,
|
||||
'data': user
|
||||
}
|
||||
status_code = 200
|
||||
else:
|
||||
result = {
|
||||
'success': False,
|
||||
'data': user
|
||||
}
|
||||
status_code = 404
|
||||
|
||||
response = make_response(json.dumps(result), status_code)
|
||||
response.headers["Content-Type"] = "application/json"
|
||||
return response
|
||||
|
||||
@app.route('/api/users/<int:uid>', methods=['PUT'])
|
||||
@validate_request
|
||||
def update_user(uid):
|
||||
user = users_dict.get(uid, {})
|
||||
if user:
|
||||
user = request.get_json()
|
||||
success = True
|
||||
status_code = 200
|
||||
users_dict[uid] = user
|
||||
else:
|
||||
success = False
|
||||
status_code = 404
|
||||
|
||||
result = {
|
||||
'success': success,
|
||||
'data': user
|
||||
}
|
||||
response = make_response(json.dumps(result), status_code)
|
||||
response.headers["Content-Type"] = "application/json"
|
||||
return response
|
||||
|
||||
@app.route('/api/users/<int:uid>', methods=['DELETE'])
|
||||
@validate_request
|
||||
def delete_user(uid):
|
||||
user = users_dict.pop(uid, {})
|
||||
if user:
|
||||
success = True
|
||||
status_code = 200
|
||||
else:
|
||||
success = False
|
||||
status_code = 404
|
||||
|
||||
result = {
|
||||
'success': success,
|
||||
'data': user
|
||||
}
|
||||
response = make_response(json.dumps(result), status_code)
|
||||
response.headers["Content-Type"] = "application/json"
|
||||
return response
|
||||
@@ -1,3 +0,0 @@
|
||||
app_version
|
||||
2.8.5
|
||||
2.8.6
|
||||
|
@@ -1,48 +0,0 @@
|
||||
import hashlib
|
||||
import hmac
|
||||
import random
|
||||
import string
|
||||
import time
|
||||
|
||||
SECRET_KEY = "DebugTalk"
|
||||
|
||||
def gen_random_string(str_len):
|
||||
random_char_list = []
|
||||
for _ in range(str_len):
|
||||
random_char = random.choice(string.ascii_letters + string.digits)
|
||||
random_char_list.append(random_char)
|
||||
|
||||
random_string = ''.join(random_char_list)
|
||||
return random_string
|
||||
|
||||
def get_sign(*args):
|
||||
content = ''.join(args).encode('ascii')
|
||||
sign_key = SECRET_KEY.encode('ascii')
|
||||
sign = hmac.new(sign_key, content, hashlib.sha1).hexdigest()
|
||||
return sign
|
||||
|
||||
def gen_user_id():
|
||||
return int(time.time() * 1000)
|
||||
|
||||
def get_user_id():
|
||||
return [
|
||||
{"user_id": 1001},
|
||||
{"user_id": 1002},
|
||||
{"user_id": 1003},
|
||||
{"user_id": 1004}
|
||||
]
|
||||
|
||||
def get_account(num):
|
||||
accounts = []
|
||||
for index in range(1, num+1):
|
||||
accounts.append(
|
||||
{"username": "user%s" % index, "password": str(index) * 6},
|
||||
)
|
||||
|
||||
return accounts
|
||||
|
||||
def get_os_platform():
|
||||
return [
|
||||
{"os_platform": "ios"},
|
||||
{"os_platform": "android"}
|
||||
]
|
||||
@@ -1,10 +0,0 @@
|
||||
config:
|
||||
name: get token with parameters
|
||||
|
||||
testcases:
|
||||
get token with $user_agent, $app_version, $os_platform:
|
||||
testcase: demo-testcase-get-token.yml
|
||||
parameters:
|
||||
user_agent: ["iOS/10.1", "iOS/10.2", "iOS/10.3"]
|
||||
app_version: ${P(app_version.csv)}
|
||||
os_platform: ${get_os_platform()}
|
||||
@@ -1,58 +0,0 @@
|
||||
[
|
||||
{
|
||||
"config": {
|
||||
"name": "testcase description",
|
||||
"variables": {}
|
||||
}
|
||||
},
|
||||
{
|
||||
"test": {
|
||||
"name": "/api/get-token",
|
||||
"request": {
|
||||
"url": "http://127.0.0.1:5000/api/get-token",
|
||||
"method": "POST",
|
||||
"headers": {
|
||||
"User-Agent": "python-requests/2.18.4",
|
||||
"device_sn": "FwgRiO7CNA50DSU",
|
||||
"os_platform": "ios",
|
||||
"app_version": "2.8.6",
|
||||
"Content-Type": "application/json"
|
||||
},
|
||||
"json": {
|
||||
"sign": "9c0c7e51c91ae963c833a4ccbab8d683c4a90c98"
|
||||
}
|
||||
},
|
||||
"validate": [
|
||||
{"eq": ["status_code", 200]},
|
||||
{"eq": ["headers.Content-Type", "application/json"]},
|
||||
{"eq": ["content.success", true]},
|
||||
{"eq": ["content.token", "baNLX1zhFYP11Seb"]}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"test": {
|
||||
"name": "/api/users/1000",
|
||||
"request": {
|
||||
"url": "http://127.0.0.1:5000/api/users/1000",
|
||||
"method": "POST",
|
||||
"headers": {
|
||||
"User-Agent": "python-requests/2.18.4",
|
||||
"device_sn": "FwgRiO7CNA50DSU",
|
||||
"token": "baNLX1zhFYP11Seb",
|
||||
"Content-Type": "application/json"
|
||||
},
|
||||
"json": {
|
||||
"name": "user1",
|
||||
"password": "123456"
|
||||
}
|
||||
},
|
||||
"validate": [
|
||||
{"eq": ["status_code", 201]},
|
||||
{"eq": ["headers.Content-Type", "application/json"]},
|
||||
{"eq": ["content.success", true]},
|
||||
{"eq": ["content.msg", "user created successfully."]}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
@@ -1,41 +0,0 @@
|
||||
- config:
|
||||
name: testcase description
|
||||
variables: {}
|
||||
|
||||
- test:
|
||||
name: /api/get-token
|
||||
request:
|
||||
headers:
|
||||
Content-Type: application/json
|
||||
User-Agent: python-requests/2.18.4
|
||||
app_version: 2.8.6
|
||||
device_sn: FwgRiO7CNA50DSU
|
||||
os_platform: ios
|
||||
json:
|
||||
sign: 9c0c7e51c91ae963c833a4ccbab8d683c4a90c98
|
||||
method: POST
|
||||
url: http://127.0.0.1:5000/api/get-token
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
- eq: [headers.Content-Type, application/json]
|
||||
- eq: [content.success, true]
|
||||
- eq: [content.token, baNLX1zhFYP11Seb]
|
||||
|
||||
- test:
|
||||
name: /api/users/1000
|
||||
request:
|
||||
headers:
|
||||
Content-Type: application/json
|
||||
User-Agent: python-requests/2.18.4
|
||||
device_sn: FwgRiO7CNA50DSU
|
||||
token: baNLX1zhFYP11Seb
|
||||
json:
|
||||
name: user1
|
||||
password: '123456'
|
||||
method: POST
|
||||
url: http://127.0.0.1:5000/api/users/1000
|
||||
validate:
|
||||
- eq: [status_code, 201]
|
||||
- eq: [headers.Content-Type, application/json]
|
||||
- eq: [content.success, true]
|
||||
- eq: [content.msg, user created successfully.]
|
||||
@@ -1,57 +0,0 @@
|
||||
[
|
||||
{
|
||||
"config": {
|
||||
"name": "testcase description",
|
||||
"variables": {}
|
||||
}
|
||||
},
|
||||
{
|
||||
"test": {
|
||||
"name": "/api/get-token",
|
||||
"request": {
|
||||
"url": "http://127.0.0.1:5000/api/get-token",
|
||||
"method": "POST",
|
||||
"headers": {
|
||||
"User-Agent": "python-requests/2.18.4",
|
||||
"device_sn": "FwgRiO7CNA50DSU",
|
||||
"os_platform": "ios",
|
||||
"app_version": "2.8.6",
|
||||
"Content-Type": "application/json"
|
||||
},
|
||||
"json": {
|
||||
"sign": "9c0c7e51c91ae963c833a4ccbab8d683c4a90c98"
|
||||
}
|
||||
},
|
||||
"validate": [
|
||||
{"eq": ["status_code", 200]},
|
||||
{"eq": ["headers.Content-Type", "application/json"]},
|
||||
{"eq": ["content.success", true]}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"test": {
|
||||
"name": "/api/users/1000",
|
||||
"request": {
|
||||
"url": "http://127.0.0.1:5000/api/users/1000",
|
||||
"method": "POST",
|
||||
"headers": {
|
||||
"User-Agent": "python-requests/2.18.4",
|
||||
"device_sn": "FwgRiO7CNA50DSU",
|
||||
"token": "baNLX1zhFYP11Seb",
|
||||
"Content-Type": "application/json"
|
||||
},
|
||||
"json": {
|
||||
"name": "user1",
|
||||
"password": "123456"
|
||||
}
|
||||
},
|
||||
"validate": [
|
||||
{"eq": ["status_code", 201]},
|
||||
{"eq": ["headers.Content-Type", "application/json"]},
|
||||
{"eq": ["content.success", true]},
|
||||
{"eq": ["content.msg", "user created successfully."]}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
@@ -1,40 +0,0 @@
|
||||
- config:
|
||||
name: testcase description
|
||||
variables: {}
|
||||
|
||||
- test:
|
||||
name: /api/get-token
|
||||
request:
|
||||
headers:
|
||||
Content-Type: application/json
|
||||
User-Agent: python-requests/2.18.4
|
||||
app_version: 2.8.6
|
||||
device_sn: FwgRiO7CNA50DSU
|
||||
os_platform: ios
|
||||
json:
|
||||
sign: 9c0c7e51c91ae963c833a4ccbab8d683c4a90c98
|
||||
method: POST
|
||||
url: http://127.0.0.1:5000/api/get-token
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
- eq: [headers.Content-Type, application/json]
|
||||
- eq: [content.success, true]
|
||||
|
||||
- test:
|
||||
name: /api/users/1000
|
||||
request:
|
||||
headers:
|
||||
Content-Type: application/json
|
||||
User-Agent: python-requests/2.18.4
|
||||
device_sn: FwgRiO7CNA50DSU
|
||||
token: baNLX1zhFYP11Seb
|
||||
json:
|
||||
name: user1
|
||||
password: '123456'
|
||||
method: POST
|
||||
url: http://127.0.0.1:5000/api/users/1000
|
||||
validate:
|
||||
- eq: [status_code, 201]
|
||||
- eq: [headers.Content-Type, application/json]
|
||||
- eq: [content.success, true]
|
||||
- eq: [content.msg, user created successfully.]
|
||||
@@ -1,60 +0,0 @@
|
||||
[
|
||||
{
|
||||
"config": {
|
||||
"name": "testcase description",
|
||||
"variables": {}
|
||||
}
|
||||
},
|
||||
{
|
||||
"test": {
|
||||
"name": "/api/get-token",
|
||||
"request": {
|
||||
"url": "http://127.0.0.1:5000/api/get-token",
|
||||
"method": "POST",
|
||||
"headers": {
|
||||
"User-Agent": "python-requests/2.18.4",
|
||||
"device_sn": "FwgRiO7CNA50DSU",
|
||||
"os_platform": "ios",
|
||||
"app_version": "2.8.6",
|
||||
"Content-Type": "application/json"
|
||||
},
|
||||
"json": {
|
||||
"sign": "9c0c7e51c91ae963c833a4ccbab8d683c4a90c98"
|
||||
}
|
||||
},
|
||||
"extract": [
|
||||
{"token": "content.token"}
|
||||
],
|
||||
"validate": [
|
||||
{"eq": ["status_code", 200]},
|
||||
{"eq": ["headers.Content-Type", "application/json"]},
|
||||
{"eq": ["content.success", true]}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"test": {
|
||||
"name": "/api/users/1000",
|
||||
"request": {
|
||||
"url": "http://127.0.0.1:5000/api/users/1000",
|
||||
"method": "POST",
|
||||
"headers": {
|
||||
"User-Agent": "python-requests/2.18.4",
|
||||
"device_sn": "FwgRiO7CNA50DSU",
|
||||
"token": "$token",
|
||||
"Content-Type": "application/json"
|
||||
},
|
||||
"json": {
|
||||
"name": "user1",
|
||||
"password": "123456"
|
||||
}
|
||||
},
|
||||
"validate": [
|
||||
{"eq": ["status_code", 201]},
|
||||
{"eq": ["headers.Content-Type", "application/json"]},
|
||||
{"eq": ["content.success", true]},
|
||||
{"eq": ["content.msg", "user created successfully."]}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
@@ -1,42 +0,0 @@
|
||||
- config:
|
||||
name: testcase description
|
||||
variables: {}
|
||||
|
||||
- test:
|
||||
name: /api/get-token
|
||||
request:
|
||||
headers:
|
||||
Content-Type: application/json
|
||||
User-Agent: python-requests/2.18.4
|
||||
app_version: 2.8.6
|
||||
device_sn: FwgRiO7CNA50DSU
|
||||
os_platform: ios
|
||||
json:
|
||||
sign: 9c0c7e51c91ae963c833a4ccbab8d683c4a90c98
|
||||
method: POST
|
||||
url: http://127.0.0.1:5000/api/get-token
|
||||
extract:
|
||||
token: content.token
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
- eq: [headers.Content-Type, application/json]
|
||||
- eq: [content.success, true]
|
||||
|
||||
- test:
|
||||
name: /api/users/1000
|
||||
request:
|
||||
headers:
|
||||
Content-Type: application/json
|
||||
User-Agent: python-requests/2.18.4
|
||||
device_sn: FwgRiO7CNA50DSU
|
||||
token: $token
|
||||
json:
|
||||
name: user1
|
||||
password: '123456'
|
||||
method: POST
|
||||
url: http://127.0.0.1:5000/api/users/1000
|
||||
validate:
|
||||
- eq: [status_code, 201]
|
||||
- eq: [headers.Content-Type, application/json]
|
||||
- eq: [content.success, true]
|
||||
- eq: [content.msg, user created successfully.]
|
||||
@@ -1,61 +0,0 @@
|
||||
[
|
||||
{
|
||||
"config": {
|
||||
"name": "testcase description",
|
||||
"base_url": "http://127.0.0.1:5000",
|
||||
"variables": {}
|
||||
}
|
||||
},
|
||||
{
|
||||
"test": {
|
||||
"name": "/api/get-token",
|
||||
"request": {
|
||||
"url": "/api/get-token",
|
||||
"method": "POST",
|
||||
"headers": {
|
||||
"User-Agent": "python-requests/2.18.4",
|
||||
"device_sn": "FwgRiO7CNA50DSU",
|
||||
"os_platform": "ios",
|
||||
"app_version": "2.8.6",
|
||||
"Content-Type": "application/json"
|
||||
},
|
||||
"json": {
|
||||
"sign": "9c0c7e51c91ae963c833a4ccbab8d683c4a90c98"
|
||||
}
|
||||
},
|
||||
"extract": [
|
||||
{"token": "content.token"}
|
||||
],
|
||||
"validate": [
|
||||
{"eq": ["status_code", 200]},
|
||||
{"eq": ["headers.Content-Type", "application/json"]},
|
||||
{"eq": ["content.success", true]}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"test": {
|
||||
"name": "/api/users/1000",
|
||||
"request": {
|
||||
"url": "/api/users/1000",
|
||||
"method": "POST",
|
||||
"headers": {
|
||||
"User-Agent": "python-requests/2.18.4",
|
||||
"device_sn": "FwgRiO7CNA50DSU",
|
||||
"token": "$token",
|
||||
"Content-Type": "application/json"
|
||||
},
|
||||
"json": {
|
||||
"name": "user1",
|
||||
"password": "123456"
|
||||
}
|
||||
},
|
||||
"validate": [
|
||||
{"eq": ["status_code", 201]},
|
||||
{"eq": ["headers.Content-Type", "application/json"]},
|
||||
{"eq": ["content.success", true]},
|
||||
{"eq": ["content.msg", "user created successfully."]}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
@@ -1,43 +0,0 @@
|
||||
- config:
|
||||
name: testcase description
|
||||
base_url: http://127.0.0.1:5000
|
||||
variables: {}
|
||||
|
||||
- test:
|
||||
name: /api/get-token
|
||||
request:
|
||||
headers:
|
||||
Content-Type: application/json
|
||||
User-Agent: python-requests/2.18.4
|
||||
app_version: 2.8.6
|
||||
device_sn: FwgRiO7CNA50DSU
|
||||
os_platform: ios
|
||||
json:
|
||||
sign: 9c0c7e51c91ae963c833a4ccbab8d683c4a90c98
|
||||
method: POST
|
||||
url: /api/get-token
|
||||
extract:
|
||||
token: content.token
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
- eq: [headers.Content-Type, application/json]
|
||||
- eq: [content.success, true]
|
||||
|
||||
- test:
|
||||
name: /api/users/1000
|
||||
request:
|
||||
headers:
|
||||
Content-Type: application/json
|
||||
User-Agent: python-requests/2.18.4
|
||||
device_sn: FwgRiO7CNA50DSU
|
||||
token: $token
|
||||
json:
|
||||
name: user1
|
||||
password: '123456'
|
||||
method: POST
|
||||
url: /api/users/1000
|
||||
validate:
|
||||
- eq: [status_code, 201]
|
||||
- eq: [headers.Content-Type, application/json]
|
||||
- eq: [content.success, true]
|
||||
- eq: [content.msg, user created successfully.]
|
||||
@@ -1,70 +0,0 @@
|
||||
[
|
||||
{
|
||||
"config": {
|
||||
"name": "testcase description",
|
||||
"base_url": "http://127.0.0.1:5000",
|
||||
"variables": {}
|
||||
}
|
||||
},
|
||||
{
|
||||
"test": {
|
||||
"name": "/api/get-token",
|
||||
"variables": {
|
||||
"device_sn": "FwgRiO7CNA50DSU",
|
||||
"os_platform": "ios",
|
||||
"app_version": "2.8.6"
|
||||
},
|
||||
"request": {
|
||||
"url": "/api/get-token",
|
||||
"method": "POST",
|
||||
"headers": {
|
||||
"User-Agent": "python-requests/2.18.4",
|
||||
"device_sn": "$device_sn",
|
||||
"os_platform": "$os_platform",
|
||||
"app_version": "$app_version",
|
||||
"Content-Type": "application/json"
|
||||
},
|
||||
"json": {
|
||||
"sign": "9c0c7e51c91ae963c833a4ccbab8d683c4a90c98"
|
||||
}
|
||||
},
|
||||
"extract": [
|
||||
{"token": "content.token"}
|
||||
],
|
||||
"validate": [
|
||||
{"eq": ["status_code", 200]},
|
||||
{"eq": ["headers.Content-Type", "application/json"]},
|
||||
{"eq": ["content.success", true]}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"test": {
|
||||
"name": "/api/users/$user_id",
|
||||
"variables": {
|
||||
"device_sn": "FwgRiO7CNA50DSU",
|
||||
"user_id": "1000"
|
||||
},
|
||||
"request": {
|
||||
"url": "/api/users/$user_id",
|
||||
"method": "POST",
|
||||
"headers": {
|
||||
"User-Agent": "python-requests/2.18.4",
|
||||
"device_sn": "$device_sn",
|
||||
"token": "$token",
|
||||
"Content-Type": "application/json"
|
||||
},
|
||||
"json": {
|
||||
"name": "user1",
|
||||
"password": "123456"
|
||||
}
|
||||
},
|
||||
"validate": [
|
||||
{"eq": ["status_code", 201]},
|
||||
{"eq": ["headers.Content-Type", "application/json"]},
|
||||
{"eq": ["content.success", true]},
|
||||
{"eq": ["content.msg", "user created successfully."]}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
@@ -1,50 +0,0 @@
|
||||
- config:
|
||||
name: testcase description
|
||||
base_url: http://127.0.0.1:5000
|
||||
variables: {}
|
||||
|
||||
- test:
|
||||
name: /api/get-token
|
||||
variables:
|
||||
app_version: 2.8.6
|
||||
device_sn: FwgRiO7CNA50DSU
|
||||
os_platform: ios
|
||||
request:
|
||||
headers:
|
||||
Content-Type: application/json
|
||||
User-Agent: python-requests/2.18.4
|
||||
app_version: $app_version
|
||||
device_sn: $device_sn
|
||||
os_platform: $os_platform
|
||||
json:
|
||||
sign: 9c0c7e51c91ae963c833a4ccbab8d683c4a90c98
|
||||
method: POST
|
||||
url: /api/get-token
|
||||
extract:
|
||||
token: content.token
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
- eq: [headers.Content-Type, application/json]
|
||||
- eq: [content.success, true]
|
||||
|
||||
- test:
|
||||
name: /api/users/$user_id
|
||||
variables:
|
||||
device_sn: FwgRiO7CNA50DSU
|
||||
user_id: 1000
|
||||
request:
|
||||
headers:
|
||||
Content-Type: application/json
|
||||
User-Agent: python-requests/2.18.4
|
||||
device_sn: $device_sn
|
||||
token: $token
|
||||
json:
|
||||
name: user1
|
||||
password: '123456'
|
||||
method: POST
|
||||
url: /api/users/$user_id
|
||||
validate:
|
||||
- eq: [status_code, 201]
|
||||
- eq: [headers.Content-Type, application/json]
|
||||
- eq: [content.success, true]
|
||||
- eq: [content.msg, user created successfully.]
|
||||
@@ -1,70 +0,0 @@
|
||||
[
|
||||
{
|
||||
"config": {
|
||||
"name": "testcase description",
|
||||
"base_url": "http://127.0.0.1:5000",
|
||||
"variables": {
|
||||
"device_sn": "FwgRiO7CNA50DSU"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"test": {
|
||||
"name": "/api/get-token",
|
||||
"variables": {
|
||||
"os_platform": "ios",
|
||||
"app_version": "2.8.6"
|
||||
},
|
||||
"request": {
|
||||
"url": "/api/get-token",
|
||||
"method": "POST",
|
||||
"headers": {
|
||||
"User-Agent": "python-requests/2.18.4",
|
||||
"device_sn": "$device_sn",
|
||||
"os_platform": "$os_platform",
|
||||
"app_version": "$app_version",
|
||||
"Content-Type": "application/json"
|
||||
},
|
||||
"json": {
|
||||
"sign": "9c0c7e51c91ae963c833a4ccbab8d683c4a90c98"
|
||||
}
|
||||
},
|
||||
"extract": [
|
||||
{"token": "content.token"}
|
||||
],
|
||||
"validate": [
|
||||
{"eq": ["status_code", 200]},
|
||||
{"eq": ["headers.Content-Type", "application/json"]},
|
||||
{"eq": ["content.success", true]}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"test": {
|
||||
"name": "/api/users/$user_id",
|
||||
"variables": {
|
||||
"user_id": "1000"
|
||||
},
|
||||
"request": {
|
||||
"url": "/api/users/$user_id",
|
||||
"method": "POST",
|
||||
"headers": {
|
||||
"User-Agent": "python-requests/2.18.4",
|
||||
"device_sn": "$device_sn",
|
||||
"token": "$token",
|
||||
"Content-Type": "application/json"
|
||||
},
|
||||
"json": {
|
||||
"name": "user1",
|
||||
"password": "123456"
|
||||
}
|
||||
},
|
||||
"validate": [
|
||||
{"eq": ["status_code", 201]},
|
||||
{"eq": ["headers.Content-Type", "application/json"]},
|
||||
{"eq": ["content.success", true]},
|
||||
{"eq": ["content.msg", "user created successfully."]}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
@@ -1,49 +0,0 @@
|
||||
- config:
|
||||
name: testcase description
|
||||
base_url: http://127.0.0.1:5000
|
||||
variables:
|
||||
device_sn: FwgRiO7CNA50DSU
|
||||
|
||||
- test:
|
||||
name: /api/get-token
|
||||
variables:
|
||||
app_version: 2.8.6
|
||||
os_platform: ios
|
||||
request:
|
||||
headers:
|
||||
Content-Type: application/json
|
||||
User-Agent: python-requests/2.18.4
|
||||
app_version: $app_version
|
||||
device_sn: $device_sn
|
||||
os_platform: $os_platform
|
||||
json:
|
||||
sign: 9c0c7e51c91ae963c833a4ccbab8d683c4a90c98
|
||||
method: POST
|
||||
url: /api/get-token
|
||||
extract:
|
||||
token: content.token
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
- eq: [headers.Content-Type, application/json]
|
||||
- eq: [content.success, true]
|
||||
|
||||
- test:
|
||||
name: /api/users/$user_id
|
||||
variables:
|
||||
user_id: 1000
|
||||
request:
|
||||
headers:
|
||||
Content-Type: application/json
|
||||
User-Agent: python-requests/2.18.4
|
||||
device_sn: $device_sn
|
||||
token: $token
|
||||
json:
|
||||
name: user1
|
||||
password: '123456'
|
||||
method: POST
|
||||
url: /api/users/$user_id
|
||||
validate:
|
||||
- eq: [status_code, 201]
|
||||
- eq: [headers.Content-Type, application/json]
|
||||
- eq: [content.success, true]
|
||||
- eq: [content.msg, user created successfully.]
|
||||
@@ -1,70 +0,0 @@
|
||||
[
|
||||
{
|
||||
"config": {
|
||||
"name": "testcase description",
|
||||
"base_url": "http://127.0.0.1:5000",
|
||||
"variables": {
|
||||
"device_sn": "${gen_random_string(15)}"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"test": {
|
||||
"name": "/api/get-token",
|
||||
"variables": {
|
||||
"os_platform": "ios",
|
||||
"app_version": "2.8.6"
|
||||
},
|
||||
"request": {
|
||||
"url": "/api/get-token",
|
||||
"method": "POST",
|
||||
"headers": {
|
||||
"User-Agent": "python-requests/2.18.4",
|
||||
"device_sn": "$device_sn",
|
||||
"os_platform": "$os_platform",
|
||||
"app_version": "$app_version",
|
||||
"Content-Type": "application/json"
|
||||
},
|
||||
"json": {
|
||||
"sign": "${get_sign($device_sn, $os_platform, $app_version)}"
|
||||
}
|
||||
},
|
||||
"extract": [
|
||||
{"token": "content.token"}
|
||||
],
|
||||
"validate": [
|
||||
{"eq": ["status_code", 200]},
|
||||
{"eq": ["headers.Content-Type", "application/json"]},
|
||||
{"eq": ["content.success", true]}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"test": {
|
||||
"name": "/api/users/$user_id",
|
||||
"variables": {
|
||||
"user_id": "${gen_user_id()}"
|
||||
},
|
||||
"request": {
|
||||
"url": "/api/users/$user_id",
|
||||
"method": "POST",
|
||||
"headers": {
|
||||
"User-Agent": "python-requests/2.18.4",
|
||||
"device_sn": "$device_sn",
|
||||
"token": "$token",
|
||||
"Content-Type": "application/json"
|
||||
},
|
||||
"json": {
|
||||
"name": "user1",
|
||||
"password": "123456"
|
||||
}
|
||||
},
|
||||
"validate": [
|
||||
{"eq": ["status_code", 201]},
|
||||
{"eq": ["headers.Content-Type", "application/json"]},
|
||||
{"eq": ["content.success", true]},
|
||||
{"eq": ["content.msg", "user created successfully."]}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
@@ -1,49 +0,0 @@
|
||||
- config:
|
||||
name: testcase description
|
||||
base_url: http://127.0.0.1:5000
|
||||
variables:
|
||||
device_sn: ${gen_random_string(15)}
|
||||
|
||||
- test:
|
||||
name: /api/get-token
|
||||
variables:
|
||||
app_version: 2.8.6
|
||||
os_platform: ios
|
||||
request:
|
||||
headers:
|
||||
Content-Type: application/json
|
||||
User-Agent: python-requests/2.18.4
|
||||
app_version: $app_version
|
||||
device_sn: $device_sn
|
||||
os_platform: $os_platform
|
||||
json:
|
||||
sign: ${get_sign($device_sn, $os_platform, $app_version)}
|
||||
method: POST
|
||||
url: /api/get-token
|
||||
extract:
|
||||
token: content.token
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
- eq: [headers.Content-Type, application/json]
|
||||
- eq: [content.success, true]
|
||||
|
||||
- test:
|
||||
name: /api/users/$user_id
|
||||
variables:
|
||||
user_id: ${gen_user_id()}
|
||||
request:
|
||||
headers:
|
||||
Content-Type: application/json
|
||||
User-Agent: python-requests/2.18.4
|
||||
device_sn: $device_sn
|
||||
token: $token
|
||||
json:
|
||||
name: user1
|
||||
password: '123456'
|
||||
method: POST
|
||||
url: /api/users/$user_id
|
||||
validate:
|
||||
- eq: [status_code, 201]
|
||||
- eq: [headers.Content-Type, application/json]
|
||||
- eq: [content.success, true]
|
||||
- eq: [content.msg, user created successfully.]
|
||||
@@ -1,13 +0,0 @@
|
||||
{
|
||||
"config": {
|
||||
"name": "create users with parameters"
|
||||
},
|
||||
"testcases": {
|
||||
"create user $user_id": {
|
||||
"testcase": "demo-quickstart-6.yml",
|
||||
"parameters": {
|
||||
"user_id": [1001, 1002, 1003, 1004]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,8 +0,0 @@
|
||||
config:
|
||||
name: testcase description
|
||||
|
||||
testcases:
|
||||
create user $user_id:
|
||||
testcase: demo-quickstart-6.yml
|
||||
parameters:
|
||||
user_id: [1001, 1002, 1003, 1004]
|
||||
@@ -1,221 +0,0 @@
|
||||
{
|
||||
"log": {
|
||||
"version": "1.2",
|
||||
"creator": {
|
||||
"name": "Charles Proxy",
|
||||
"version": "4.2.1"
|
||||
},
|
||||
"entries": [
|
||||
{
|
||||
"startedDateTime": "2018-02-19T17:30:00.904+08:00",
|
||||
"time": 3,
|
||||
"request": {
|
||||
"method": "POST",
|
||||
"url": "http://127.0.0.1:5000/api/get-token",
|
||||
"httpVersion": "HTTP/1.1",
|
||||
"cookies": [],
|
||||
"headers": [
|
||||
{
|
||||
"name": "Host",
|
||||
"value": "127.0.0.1:5000"
|
||||
},
|
||||
{
|
||||
"name": "User-Agent",
|
||||
"value": "python-requests/2.18.4"
|
||||
},
|
||||
{
|
||||
"name": "Accept-Encoding",
|
||||
"value": "gzip, deflate"
|
||||
},
|
||||
{
|
||||
"name": "Accept",
|
||||
"value": "*/*"
|
||||
},
|
||||
{
|
||||
"name": "Connection",
|
||||
"value": "keep-alive"
|
||||
},
|
||||
{
|
||||
"name": "device_sn",
|
||||
"value": "FwgRiO7CNA50DSU"
|
||||
},
|
||||
{
|
||||
"name": "os_platform",
|
||||
"value": "ios"
|
||||
},
|
||||
{
|
||||
"name": "app_version",
|
||||
"value": "2.8.6"
|
||||
},
|
||||
{
|
||||
"name": "Content-Length",
|
||||
"value": "52"
|
||||
},
|
||||
{
|
||||
"name": "Content-Type",
|
||||
"value": "application/json"
|
||||
}
|
||||
],
|
||||
"queryString": [],
|
||||
"postData": {
|
||||
"mimeType": "application/json",
|
||||
"text": "{\"sign\": \"9c0c7e51c91ae963c833a4ccbab8d683c4a90c98\"}"
|
||||
},
|
||||
"headersSize": 299,
|
||||
"bodySize": 52
|
||||
},
|
||||
"response": {
|
||||
"_charlesStatus": "COMPLETE",
|
||||
"status": 200,
|
||||
"statusText": "OK",
|
||||
"httpVersion": "HTTP/1.0",
|
||||
"cookies": [],
|
||||
"headers": [
|
||||
{
|
||||
"name": "Content-Type",
|
||||
"value": "application/json"
|
||||
},
|
||||
{
|
||||
"name": "Content-Length",
|
||||
"value": "46"
|
||||
},
|
||||
{
|
||||
"name": "Server",
|
||||
"value": "Werkzeug/0.14.1 Python/3.6.4"
|
||||
},
|
||||
{
|
||||
"name": "Date",
|
||||
"value": "Mon, 19 Feb 2018 09:30:00 GMT"
|
||||
},
|
||||
{
|
||||
"name": "Proxy-Connection",
|
||||
"value": "Close"
|
||||
}
|
||||
],
|
||||
"content": {
|
||||
"size": 46,
|
||||
"mimeType": "application/json",
|
||||
"text": "eyJzdWNjZXNzIjogdHJ1ZSwgInRva2VuIjogImJhTkxYMXpoRllQMTFTZWIifQ==",
|
||||
"encoding": "base64"
|
||||
},
|
||||
"redirectURL": null,
|
||||
"headersSize": 175,
|
||||
"bodySize": 46
|
||||
},
|
||||
"serverIPAddress": "127.0.0.1",
|
||||
"cache": {},
|
||||
"timings": {
|
||||
"dns": 1,
|
||||
"connect": 0,
|
||||
"ssl": -1,
|
||||
"send": 0,
|
||||
"wait": 1,
|
||||
"receive": 1
|
||||
}
|
||||
},
|
||||
{
|
||||
"startedDateTime": "2018-02-19T17:30:00.911+08:00",
|
||||
"time": 3,
|
||||
"request": {
|
||||
"method": "POST",
|
||||
"url": "http://127.0.0.1:5000/api/users/1000",
|
||||
"httpVersion": "HTTP/1.1",
|
||||
"cookies": [],
|
||||
"headers": [
|
||||
{
|
||||
"name": "Host",
|
||||
"value": "127.0.0.1:5000"
|
||||
},
|
||||
{
|
||||
"name": "User-Agent",
|
||||
"value": "python-requests/2.18.4"
|
||||
},
|
||||
{
|
||||
"name": "Accept-Encoding",
|
||||
"value": "gzip, deflate"
|
||||
},
|
||||
{
|
||||
"name": "Accept",
|
||||
"value": "*/*"
|
||||
},
|
||||
{
|
||||
"name": "Connection",
|
||||
"value": "keep-alive"
|
||||
},
|
||||
{
|
||||
"name": "device_sn",
|
||||
"value": "FwgRiO7CNA50DSU"
|
||||
},
|
||||
{
|
||||
"name": "token",
|
||||
"value": "baNLX1zhFYP11Seb"
|
||||
},
|
||||
{
|
||||
"name": "Content-Length",
|
||||
"value": "39"
|
||||
},
|
||||
{
|
||||
"name": "Content-Type",
|
||||
"value": "application/json"
|
||||
}
|
||||
],
|
||||
"queryString": [],
|
||||
"postData": {
|
||||
"mimeType": "application/json",
|
||||
"text": "{\"name\": \"user1\", \"password\": \"123456\"}"
|
||||
},
|
||||
"headersSize": 265,
|
||||
"bodySize": 39
|
||||
},
|
||||
"response": {
|
||||
"_charlesStatus": "COMPLETE",
|
||||
"status": 201,
|
||||
"statusText": "CREATED",
|
||||
"httpVersion": "HTTP/1.0",
|
||||
"cookies": [],
|
||||
"headers": [
|
||||
{
|
||||
"name": "Content-Type",
|
||||
"value": "application/json"
|
||||
},
|
||||
{
|
||||
"name": "Content-Length",
|
||||
"value": "54"
|
||||
},
|
||||
{
|
||||
"name": "Server",
|
||||
"value": "Werkzeug/0.14.1 Python/3.6.4"
|
||||
},
|
||||
{
|
||||
"name": "Date",
|
||||
"value": "Mon, 19 Feb 2018 09:30:00 GMT"
|
||||
},
|
||||
{
|
||||
"name": "Proxy-Connection",
|
||||
"value": "Close"
|
||||
}
|
||||
],
|
||||
"content": {
|
||||
"size": 54,
|
||||
"mimeType": "application/json",
|
||||
"text": "eyJzdWNjZXNzIjogdHJ1ZSwgIm1zZyI6ICJ1c2VyIGNyZWF0ZWQgc3VjY2Vzc2Z1bGx5LiJ9",
|
||||
"encoding": "base64"
|
||||
},
|
||||
"redirectURL": null,
|
||||
"headersSize": 77,
|
||||
"bodySize": 54
|
||||
},
|
||||
"serverIPAddress": "127.0.0.1",
|
||||
"cache": {},
|
||||
"timings": {
|
||||
"dns": 0,
|
||||
"connect": 0,
|
||||
"ssl": -1,
|
||||
"send": 0,
|
||||
"wait": 3,
|
||||
"receive": 0
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -1,98 +0,0 @@
|
||||
[
|
||||
{
|
||||
"config": {
|
||||
"name": "testcase description",
|
||||
"variables": {}
|
||||
}
|
||||
},
|
||||
{
|
||||
"test": {
|
||||
"name": "/api/get-token",
|
||||
"request": {
|
||||
"url": "http://127.0.0.1:5000/api/get-token",
|
||||
"method": "POST",
|
||||
"headers": {
|
||||
"User-Agent": "python-requests/2.18.4",
|
||||
"device_sn": "FwgRiO7CNA50DSU",
|
||||
"os_platform": "ios",
|
||||
"app_version": "2.8.6",
|
||||
"Content-Type": "application/json"
|
||||
},
|
||||
"json": {
|
||||
"sign": "9c0c7e51c91ae963c833a4ccbab8d683c4a90c98"
|
||||
}
|
||||
},
|
||||
"validate": [
|
||||
{
|
||||
"eq": [
|
||||
"status_code",
|
||||
200
|
||||
]
|
||||
},
|
||||
{
|
||||
"eq": [
|
||||
"headers.Content-Type",
|
||||
"application/json"
|
||||
]
|
||||
},
|
||||
{
|
||||
"eq": [
|
||||
"content.success",
|
||||
true
|
||||
]
|
||||
},
|
||||
{
|
||||
"eq": [
|
||||
"content.token",
|
||||
"baNLX1zhFYP11Seb"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"test": {
|
||||
"name": "/api/users/1000",
|
||||
"request": {
|
||||
"url": "http://127.0.0.1:5000/api/users/1000",
|
||||
"method": "POST",
|
||||
"headers": {
|
||||
"User-Agent": "python-requests/2.18.4",
|
||||
"device_sn": "FwgRiO7CNA50DSU",
|
||||
"token": "baNLX1zhFYP11Seb",
|
||||
"Content-Type": "application/json"
|
||||
},
|
||||
"json": {
|
||||
"name": "user1",
|
||||
"password": "123456"
|
||||
}
|
||||
},
|
||||
"validate": [
|
||||
{
|
||||
"eq": [
|
||||
"status_code",
|
||||
201
|
||||
]
|
||||
},
|
||||
{
|
||||
"eq": [
|
||||
"headers.Content-Type",
|
||||
"application/json"
|
||||
]
|
||||
},
|
||||
{
|
||||
"eq": [
|
||||
"content.success",
|
||||
true
|
||||
]
|
||||
},
|
||||
{
|
||||
"eq": [
|
||||
"content.msg",
|
||||
"user created successfully."
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
@@ -1,55 +0,0 @@
|
||||
- config:
|
||||
name: testcase description
|
||||
variables: {}
|
||||
- test:
|
||||
name: /api/get-token
|
||||
request:
|
||||
headers:
|
||||
Content-Type: application/json
|
||||
User-Agent: python-requests/2.18.4
|
||||
app_version: 2.8.6
|
||||
device_sn: FwgRiO7CNA50DSU
|
||||
os_platform: ios
|
||||
json:
|
||||
sign: 9c0c7e51c91ae963c833a4ccbab8d683c4a90c98
|
||||
method: POST
|
||||
url: http://127.0.0.1:5000/api/get-token
|
||||
validate:
|
||||
- eq:
|
||||
- status_code
|
||||
- 200
|
||||
- eq:
|
||||
- headers.Content-Type
|
||||
- application/json
|
||||
- eq:
|
||||
- content.success
|
||||
- true
|
||||
- eq:
|
||||
- content.token
|
||||
- baNLX1zhFYP11Seb
|
||||
- test:
|
||||
name: /api/users/1000
|
||||
request:
|
||||
headers:
|
||||
Content-Type: application/json
|
||||
User-Agent: python-requests/2.18.4
|
||||
device_sn: FwgRiO7CNA50DSU
|
||||
token: baNLX1zhFYP11Seb
|
||||
json:
|
||||
name: user1
|
||||
password: '123456'
|
||||
method: POST
|
||||
url: http://127.0.0.1:5000/api/users/1000
|
||||
validate:
|
||||
- eq:
|
||||
- status_code
|
||||
- 201
|
||||
- eq:
|
||||
- headers.Content-Type
|
||||
- application/json
|
||||
- eq:
|
||||
- content.success
|
||||
- true
|
||||
- eq:
|
||||
- content.msg
|
||||
- user created successfully.
|
||||
@@ -1,27 +0,0 @@
|
||||
- config:
|
||||
name: get token
|
||||
base_url: http://127.0.0.1:5000
|
||||
variables:
|
||||
device_sn: ${gen_random_string(15)}
|
||||
os_platform: 'ios'
|
||||
app_version: '2.8.6'
|
||||
|
||||
- test:
|
||||
name: get token with $device_sn, $os_platform, $app_version
|
||||
request:
|
||||
headers:
|
||||
Content-Type: application/json
|
||||
User-Agent: python-requests/2.18.4
|
||||
app_version: $app_version
|
||||
device_sn: $device_sn
|
||||
os_platform: $os_platform
|
||||
json:
|
||||
sign: ${get_sign($device_sn, $os_platform, $app_version)}
|
||||
method: POST
|
||||
url: /api/get-token
|
||||
extract:
|
||||
token: content.token
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
- eq: [headers.Content-Type, application/json]
|
||||
- eq: [content.success, true]
|
||||
@@ -1,34 +0,0 @@
|
||||
- config:
|
||||
name: "user management testcase."
|
||||
parameters:
|
||||
- user_agent: ["iOS/10.1", "iOS/10.2", "iOS/10.3"]
|
||||
- app_version: ${P(app_version.csv)}
|
||||
- os_platform: ${get_os_platform()}
|
||||
variables:
|
||||
- user_agent: 'iOS/10.3'
|
||||
- device_sn: ${gen_random_string(15)}
|
||||
- os_platform: 'ios'
|
||||
- app_version: '2.8.6'
|
||||
request:
|
||||
base_url: http://127.0.0.1:5000
|
||||
headers:
|
||||
Content-Type: application/json
|
||||
device_sn: $device_sn
|
||||
|
||||
- test:
|
||||
name: get token with $user_agent, $os_platform, $app_version
|
||||
request:
|
||||
url: /api/get-token
|
||||
method: POST
|
||||
headers:
|
||||
app_version: $app_version
|
||||
os_platform: $os_platform
|
||||
user_agent: $user_agent
|
||||
json:
|
||||
sign: ${get_sign($user_agent, $device_sn, $os_platform, $app_version)}
|
||||
extract:
|
||||
- token: content.token
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
- eq: [headers.Content-Type, application/json]
|
||||
- eq: [content.success, true]
|
||||
File diff suppressed because one or more lines are too long
@@ -1,66 +0,0 @@
|
||||
- config:
|
||||
name: testcase description
|
||||
variables: {}
|
||||
verify: False
|
||||
|
||||
- test:
|
||||
name: /account/sign_in
|
||||
request:
|
||||
headers:
|
||||
If-None-Match: W/"bc9ae267fdcbd89bf1dfaea10dea2b0e"
|
||||
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36
|
||||
(KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36
|
||||
method: GET
|
||||
url: https://testerhome.com/account/sign_in
|
||||
extract:
|
||||
X_CSRF_Token: <meta name="csrf-token" content="(.*)" />
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
- eq: [headers.Content-Type, text/html; charset=utf-8]
|
||||
|
||||
- test:
|
||||
name: /assets/big_logo-cd32144f74c18746f3dce33e1040e7dfe4c07c8e611e37f3868b1c16b5095da3.png
|
||||
request:
|
||||
headers:
|
||||
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36
|
||||
(KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36
|
||||
method: GET
|
||||
url: https://testerhome.com/assets/big_logo-cd32144f74c18746f3dce33e1040e7dfe4c07c8e611e37f3868b1c16b5095da3.png
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
- eq: [headers.Content-Type, image/png]
|
||||
|
||||
- test:
|
||||
name: /account/sign_in
|
||||
request:
|
||||
data:
|
||||
commit: Sign In
|
||||
user[login]: debugtalk
|
||||
user[password]: XXXXXXXX
|
||||
user[remember_me]: '1'
|
||||
utf8: ✓
|
||||
headers:
|
||||
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
|
||||
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36
|
||||
(KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36
|
||||
X-CSRF-Token: $X_CSRF_Token
|
||||
X-Requested-With: XMLHttpRequest
|
||||
method: POST
|
||||
url: https://testerhome.com/account/sign_in
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
- eq: [headers.Content-Type, text/javascript; charset=utf-8]
|
||||
|
||||
- test:
|
||||
name: /
|
||||
request:
|
||||
headers:
|
||||
If-None-Match: W/"bad62c68dac27b01151516aad5c7f0be"
|
||||
Turbolinks-Referrer: https://testerhome.com/account/sign_in
|
||||
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36
|
||||
(KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36
|
||||
method: GET
|
||||
url: https://testerhome.com/
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
- eq: [headers.Content-Type, text/html; charset=utf-8]
|
||||
@@ -1,5 +0,0 @@
|
||||
user_id
|
||||
1001
|
||||
1002
|
||||
1003
|
||||
1004
|
||||
|
@@ -1,2 +0,0 @@
|
||||
|
||||
|
||||
@@ -1,339 +0,0 @@
|
||||
# 开发扩展
|
||||
|
||||
HttpRunner 除了作为命令行工具使用外，还可以作为软件包集成到你自己的项目中。
|
||||
|
||||
简单来说，HttpRunner 提供了运行 YAML/JSON 格式测试用例的能力，并能返回详细的测试结果信息。
|
||||
|
||||
## HttpRunner class
|
||||
|
||||
### TL;DR
|
||||
|
||||
HttpRunner 以 `类（class）` 的形式对外提供调用支持，类名为 `HttpRunner`。使用方式如下：
|
||||
|
||||
```python
|
||||
from httprunner.api import HttpRunner
|
||||
|
||||
runner = HttpRunner(
|
||||
failfast=True,
|
||||
save_tests=True,
|
||||
log_level="INFO",
|
||||
log_file="test.log"
|
||||
)
|
||||
summary = runner.run(path_or_tests)
|
||||
```
|
||||
|
||||
### 初始化参数说明
|
||||
|
||||
通常情况下，初始化 `HttpRunner` 时的参数有如下几个：
|
||||
|
||||
- `failfast`（可选）: 设置为 True 时，测试在首次遇到错误或失败时会停止运行；默认值为 False
|
||||
- `save_tests`（可选）: 设置为 True 时，会将运行过程中的状态（loaded/parsed/summary）保存为 JSON 文件，存储于 logs 目录下；默认为 False
|
||||
- `log_level`（可选）: 设置日志级别，默认为 "INFO"
|
||||
- `log_file`（可选）: 设置日志文件路径，指定后将同时输出日志文件；默认不输出日志文件
|
||||
|
||||
### 调用方法说明
|
||||
|
||||
在 `HttpRunner` 中，对外提供了一个 `run` 方法，用于运行测试用例。
|
||||
|
||||
run 方法有三个参数：
|
||||
|
||||
- `path_or_tests`（必传）: 指定要运行的测试用例；支持传入两类参数
|
||||
- str: YAML/JSON 格式测试用例文件路径
|
||||
- dict: 标准的测试用例结构体
|
||||
- `dot_env_path`（可选）: 指定加载环境变量文件（.env）的路径，默认值为当前工作目录（PWD）中的 `.env` 文件
|
||||
- `mapping`（可选）: 变量映射，可用于对传入测试用例中的变量进行覆盖替换。
|
||||
|
||||
#### 传入测试用例文件路径
|
||||
|
||||
指定测试用例文件路径支持三种形式：
|
||||
|
||||
- YAML/JSON 文件路径，支持绝对路径和相对路径
|
||||
- 包含 YAML/JSON 文件的文件夹，支持绝对路径和相对路径
|
||||
- 文件路径和文件夹路径的混合情况（list/set）
|
||||
|
||||
```python
|
||||
# 文件路径
|
||||
runner.run("docs/data/demo-quickstart-2.yml")
|
||||
|
||||
# 文件夹路径
|
||||
runner.run("docs/data/")
|
||||
|
||||
# 混合情况
|
||||
runner.run(["docs/data/", "files/demo-quickstart-2.yml"])
|
||||
```
|
||||
|
||||
如需指定加载环境变量文件（.env）的路径，或者需要对测试用例中的变量进行覆盖替换，则可使用 `dot_env_path` 和 `mapping` 参数。
|
||||
|
||||
```python
|
||||
# dot_env_path
|
||||
runner.run("docs/data/demo-quickstart-2.yml", dot_env_path="/path/to/.env")
|
||||
|
||||
# mapping
|
||||
override_mapping = {
|
||||
"device_sn": "XXX"
|
||||
}
|
||||
runner.run("docs/data/demo-quickstart-2.yml", mapping=override_mapping)
|
||||
```
|
||||
|
||||
#### 传入标准的测试用例结构体
|
||||
|
||||
除了传入测试用例文件路径，还可以直接传入标准的测试用例结构体。
|
||||
|
||||
以 [demo-quickstart-2.yml](/data/demo-quickstart-2.yml) 为例，对应的数据结构体如下所示：
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"config": {
|
||||
"name": "testcase description",
|
||||
"request": {
|
||||
"base_url": "",
|
||||
"headers": {
|
||||
"User-Agent": "python-requests/2.18.4"
|
||||
}
|
||||
},
|
||||
"variables": [],
|
||||
"output": ["token"],
|
||||
"path": "/abs-path/to/demo-quickstart-2.yml",
|
||||
"refs": {
|
||||
"env": {},
|
||||
"debugtalk": {
|
||||
"variables": {
|
||||
"SECRET_KEY": "DebugTalk"
|
||||
},
|
||||
"functions": {
|
||||
"gen_random_string": <function gen_random_string at 0x108596268>,
|
||||
"get_sign": <function get_sign at 0x1085962f0>,
|
||||
"get_user_id": <function get_user_id at 0x108596378>,
|
||||
"get_account": <function get_account at 0x108596400>,
|
||||
"get_os_platform": <function get_os_platform at 0x108596488>
|
||||
}
|
||||
},
|
||||
"def-api": {},
|
||||
"def-testcase": {}
|
||||
}
|
||||
},
|
||||
"teststeps": [
|
||||
{
|
||||
"name": "/api/get-token",
|
||||
"request": {
|
||||
"url": "http://127.0.0.1:5000/api/get-token",
|
||||
"method": "POST",
|
||||
"headers": {"Content-Type": "application/json", "app_version": "2.8.6", "device_sn": "FwgRiO7CNA50DSU", "os_platform": "ios", "user_agent": "iOS/10.3"},
|
||||
"json": {"sign": "9c0c7e51c91ae963c833a4ccbab8d683c4a90c98"}
|
||||
},
|
||||
"extract": [
|
||||
{"token": "content.token"}
|
||||
],
|
||||
"validate": [
|
||||
{"eq": ["status_code", 200]},
|
||||
{"eq": ["headers.Content-Type", "application/json"]},
|
||||
{"eq": ["content.success", true]}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "/api/users/1000",
|
||||
"request": {"url": "http://127.0.0.1:5000/api/users/1000", "method": "POST", "headers": {"Content-Type": "application/json", "device_sn": "FwgRiO7CNA50DSU", "token": "$token"},
|
||||
"json": {"name": "user1", "password": "123456"}},
|
||||
"validate": [
|
||||
{"eq": ["status_code", 201]},
|
||||
{"eq": ["headers.Content-Type", "application/json"]},
|
||||
{"eq": ["content.success", true]},
|
||||
{"eq": ["content.msg", "user created successfully."]}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{...} # another testcase
|
||||
]
|
||||
```
|
||||
|
||||
传入测试用例结构体时，支持传入单个结构体（dict），或者多个结构体（list of dict）。
|
||||
|
||||
```python
|
||||
# 运行单个结构体
|
||||
runner.run(testcase)
|
||||
|
||||
# 运行多个结构体
|
||||
runner.run([testcase1, testcase2])
|
||||
```
|
||||
|
||||
### 加载 `debugtalk.py` && `.env`
|
||||
|
||||
通过传入测试用例文件路径运行测试用例时，HttpRunner 会自动以指定测试用例文件路径为起点，向上搜索 `debugtalk.py` 文件，并将 `debugtalk.py` 文件所在的文件目录作为当前工作目录（PWD）。
|
||||
|
||||
同时，HttpRunner 会在当前工作目录（PWD）下搜索 `.env` 文件，以及 `api` 和 `testcases` 文件夹，并自动进行加载。
|
||||
|
||||
最终加载得到的存储结构如下所示：
|
||||
|
||||
```json
|
||||
{
|
||||
"env": {},
|
||||
"debugtalk": {
|
||||
"variables": {
|
||||
"SECRET_KEY": "DebugTalk"
|
||||
},
|
||||
"functions": {
|
||||
"gen_random_string": <function gen_random_string at 0x108596268>,
|
||||
"get_sign": <function get_sign at 0x1085962f0>,
|
||||
"get_user_id": <function get_user_id at 0x108596378>,
|
||||
"get_account": <function get_account at 0x108596400>,
|
||||
"get_os_platform": <function get_os_platform at 0x108596488>
|
||||
}
|
||||
},
|
||||
"def-api": {},
|
||||
"def-testcase": {}
|
||||
}
|
||||
```
|
||||
|
||||
其中，`env` 对应的是 `.env` 文件中的环境变量，`debugtalk` 对应的是 `debugtalk.py` 文件中定义的变量和函数，`def-api` 对应的是 `api` 文件夹下定义的接口描述，`def-testcase` 对应的是 `testcases` 文件夹下定义的测试用例。
|
||||
|
||||
通过传入标准的测试用例结构体执行测试时，传入的数据应包含所有信息，包括 `debugtalk.py`、`.env`、依赖的 api 和 测试用例等；因此也无需再使用 `dot_env_path` 和 `mapping` 参数，所有信息都要通过 `refs` 传入。
|
||||
|
||||
## 返回详细测试结果数据
|
||||
|
||||
运行完成后，通过 `run()` 方法的返回结果可获取详尽的运行结果数据。
|
||||
|
||||
```python
|
||||
summary = runner.run(path_or_tests)
|
||||
```
|
||||
|
||||
其数据结构为：
|
||||
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"stat": {
|
||||
"testsRun": 2,
|
||||
"failures": 0,
|
||||
"errors": 0,
|
||||
"skipped": 0,
|
||||
"expectedFailures": 0,
|
||||
"unexpectedSuccesses": 0,
|
||||
"successes": 2
|
||||
},
|
||||
"time": {
|
||||
"start_at": 1538449655.944404,
|
||||
"duration": 0.03181314468383789
|
||||
},
|
||||
"platform": {
|
||||
"httprunner_version": "1.5.14",
|
||||
"python_version": "CPython 3.6.5+",
|
||||
"platform": "Darwin-17.6.0-x86_64-i386-64bit"
|
||||
},
|
||||
"details": [
|
||||
{
|
||||
"success": true,
|
||||
"name": "testcase description",
|
||||
"base_url": "",
|
||||
"stat": {"testsRun": 2, "failures": 0, "errors": 0, "skipped": 0, "expectedFailures": 0, "unexpectedSuccesses": 0, "successes": 2},
|
||||
"time": {"start_at": 1538449655.944404, "duration": 0.03181314468383789},
|
||||
"records": [
|
||||
{
|
||||
"name": "/api/get-token",
|
||||
"status": "success",
|
||||
"attachment": "",
|
||||
"meta_data": {
|
||||
"request": {
|
||||
"url": "http://127.0.0.1:5000/api/get-token",
|
||||
"method": "POST",
|
||||
"headers": {"User-Agent": "python-requests/2.18.4", "Accept-Encoding": "gzip, deflate", "Accept": "*/*", "Connection": "keep-alive", "Content-Type": "application/json", "app_version": "2.8.6", "device_sn": "FwgRiO7CNA50DSU", "os_platform": "ios", "user_agent": "iOS/10.3", "Content-Length": "52"},
|
||||
"start_timestamp": 1538449655.944801,
|
||||
"json": {"sign": "9c0c7e51c91ae963c833a4ccbab8d683c4a90c98"},
|
||||
"body": b'{"sign": "9c0c7e51c91ae963c833a4ccbab8d683c4a90c98"}'
|
||||
},
|
||||
"response": {
|
||||
"status_code": 200,
|
||||
"headers": {"Content-Type": "application/json", "Content-Length": "46", "Server": "Werkzeug/0.14.1 Python/3.6.5+", "Date": "Tue, 02 Oct 2018 03:07:35 GMT"},
|
||||
"content_size": 46,
|
||||
"response_time_ms": 12.87,
|
||||
"elapsed_ms": 6.955,
|
||||
"encoding": null,
|
||||
"content": b'{"success": true, "token": "CcQ7dBjZZbjIXRkG"}',
|
||||
"content_type": "application/json",
|
||||
"ok": true,
|
||||
"url": "http://127.0.0.1:5000/api/get-token",
|
||||
"reason": "OK",
|
||||
"cookies": {},
|
||||
"text": '{"success": true, "token": "CcQ7dBjZZbjIXRkG"}',
|
||||
"json": {"success": true, "token": "CcQ7dBjZZbjIXRkG"}
|
||||
},
|
||||
"validators": [
|
||||
{"check": "status_code", "expect": 200, "comparator": "eq", "check_value": 200, "check_result": "pass"},
|
||||
{"check": "headers.Content-Type", "expect": "application/json", "comparator": "eq", "check_value": "application/json", "check_result": "pass"},
|
||||
{"check": "content.success", "expect": true, "comparator": "eq", "check_value": true, "check_result": "pass"}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "/api/users/1000",
|
||||
"status": "success",
|
||||
"attachment": "",
|
||||
"meta_data": {
|
||||
"request": {
|
||||
"url": "http://127.0.0.1:5000/api/users/1000",
|
||||
"method": "POST",
|
||||
"headers": {"User-Agent": "python-requests/2.18.4", "Accept-Encoding": "gzip, deflate", "Accept": "*/*", "Connection": "keep-alive", "Content-Type": "application/json", "device_sn": "FwgRiO7CNA50DSU", "token": "CcQ7dBjZZbjIXRkG", "Content-Length": "39"},
|
||||
"start_timestamp": 1538449655.958944,
|
||||
"json": {"name": "user1", "password": "123456"},
|
||||
"body": b'{"name": "user1", "password": "123456"}'
|
||||
},
|
||||
"response": {
|
||||
"status_code": 201,
|
||||
"headers": {"Content-Type": "application/json", "Content-Length": "54", "Server": "Werkzeug/0.14.1 Python/3.6.5+", "Date": "Tue, 02 Oct 2018 03:07:35 GMT"},
|
||||
"content_size": 54,
|
||||
"response_time_ms": 3.34,
|
||||
"elapsed_ms": 2.16,
|
||||
"encoding": null,
|
||||
"content": b'{"success": true, "msg": "user created successfully."}',
|
||||
"content_type": "application/json",
|
||||
"ok": true,
|
||||
"url": "http://127.0.0.1:5000/api/users/1000",
|
||||
"reason": "CREATED",
|
||||
"cookies": {},
|
||||
"text": '{"success": true, "msg": "user created successfully."}',
|
||||
"json": {"success": true, "msg": "user created successfully."}
|
||||
},
|
||||
"validators": [
|
||||
{"check": "status_code", "expect": 201, "comparator": "eq", "check_value": 201, "check_result": "pass"},
|
||||
{"check": "headers.Content-Type", "expect": "application/json", "comparator": "eq", "check_value": "application/json", "check_result": "pass"},
|
||||
{"check": "content.success", "expect": true, "comparator": "eq", "check_value": true, "check_result": "pass"},
|
||||
{"check": "content.msg", "expect": "user created successfully.", "comparator": "eq", "check_value": "user created successfully.", "check_result": "pass"}
|
||||
]
|
||||
}
|
||||
}
|
||||
],
|
||||
"in_out": {
|
||||
"in": {"SECRET_KEY": "DebugTalk"},
|
||||
"out": {"token": "CcQ7dBjZZbjIXRkG"}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## 生成 HTML 测试报告
|
||||
|
||||
如需生成 HTML 测试报告，可调用 `report.gen_html_report` 方法。
|
||||
|
||||
```python
|
||||
from httprunner import report
|
||||
|
||||
report_path = report.gen_html_report(
|
||||
summary,
|
||||
report_template="/path/to/custom_report_template",
|
||||
report_dir="/path/to/reports_dir",
|
||||
report_file="/path/to/report_file_path"
|
||||
)
|
||||
```
|
||||
|
||||
`gen_html_report()` 的参数有四个：
|
||||
|
||||
- summary（必传）: 测试运行结果汇总数据
|
||||
- report_template（可选）: 指定自定义的 HTML 报告模板，模板必须采用 Jinja2 的格式
|
||||
- report_dir（可选）: 指定生成报告的文件夹路径
|
||||
- report_file（可选）: 指定生成报告的文件路径，该参数的优先级高于 report_dir
|
||||
|
||||
关于测试报告的详细内容，请查看[测试报告](/run-tests/report/)部分。
|
||||
|
||||
[TextTestRunner]: https://docs.python.org/3.6/library/unittest.html#unittest.TextTestRunner
|
||||
@@ -1,30 +0,0 @@
|
||||
## 案例介绍
|
||||
|
||||
- 被测案例：[klook](https://www.klook.com/)
|
||||
- 案例作者：[readyou](https://github.com/readyou)
|
||||
|
||||
我们团队选择了 HttpRunner 作为接口测试框架，并整理了一份案例，供大家参考。
|
||||
|
||||
## 注意事项
|
||||
|
||||
1. 本例子中有些地方用到了`localhost:8085`作为base_url，这些接口是不能访问的，仅作为示例学习怎样组织测试用例。
|
||||
2. `https://maps.googleapis.com`是可以用的，自己申请一个key，替换掉文件中的`your_google_map_key`即可。
|
||||
|
||||
## 相关文件说明
|
||||
|
||||
模块 | 文件 | 用途 | 备注
|
||||
---|----|------|------
|
||||
google map 接口测试 | api/find_place_api.yml | google map根据名称搜索地址的api | 比较全面地使用了api可以使用的关键字：name, base_url, request, variables, validate, extract
|
||||
google map 接口测试 | testcases/find_place_testcase.yml | google map根据名称搜索地址的testcase | 使用了testcase标准的写法：testcase由teststep组成，teststep中引用api(just_request_testcase.yml中演示了直接使用request而不是引用api的方式)。teststep中还使用了variables。
|
||||
google map 接口测试 | testcases/place_detail_testcase.yml | google map获取地址详情的testcase | config中使用variables
|
||||
google map 接口测试 | testsuites/place_detail_testsuite.yml | google map接口测试的testsuite包含上面两个testcase | 使用了多种方式来做数据驱动测试
|
||||
| |
|
||||
klook地理位置搜索接口测试 | api/search_area_by_name_api.yml | 根据名字查询区域（支持多语言）——api |
|
||||
klook地理位置搜索接口测试 | api/search_area_by_name_testcase.yml | 根据名字查询区域（支持多语言）——testcase |
|
||||
klook地理位置搜索接口测试 | api/get_area_groups_api.yml | 查询地理位置下面的组——api |
|
||||
klook地理位置搜索接口测试 | api/get_area_groups_testcase.yml | 查询地理位置下面的组——testcase |
|
||||
klook地理位置搜索接口测试 | api/area_manage_testsuite.yml | 区域管理——testsuite |
|
||||
| |
|
||||
baidu首页demo | testcases/just_request_testcase.yml | 提取百度首页title的demo | 演示了直接使用request而不是引用api的方式，使用了teardown_hooks的使用
|
||||
|
||||
完整的案例访问[地址](https://github.com/httprunner/httprunner/tree/master/docs/examples/demo-klook)。
|
||||
@@ -1,19 +0,0 @@
|
||||
name: find place from text
|
||||
base_url: https://maps.googleapis.com
|
||||
request:
|
||||
method: GET
|
||||
url: /maps/api/place/findplacefromtext/json
|
||||
params:
|
||||
key: $key
|
||||
inputtype: textquery
|
||||
input: $input
|
||||
fields: 'formatted_address,geometry,name,permanently_closed,place_id,plus_code,types'
|
||||
language: zh_CN
|
||||
variables:
|
||||
input: 宝安
|
||||
key: your_google_map_key
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
- eq: [content.status, OK]
|
||||
extract:
|
||||
- place_id: content.candidates.0.place_id
|
||||
@@ -1,11 +0,0 @@
|
||||
name: get_groups
|
||||
base_url: http://localhost:8085
|
||||
request:
|
||||
url: /v1/transferairportadminsrv/area/getGroups
|
||||
method: POST
|
||||
headers:
|
||||
Content-Type: application/x-www-form-urlencoded
|
||||
Accept-language: zh_CN
|
||||
cookie: $admin_cookie
|
||||
data:
|
||||
parentAreaId: 7037 # 广东省
|
||||
@@ -1,17 +0,0 @@
|
||||
name: get place detail
|
||||
request:
|
||||
method: GET
|
||||
url: https://maps.googleapis.com/maps/api/place/details/json
|
||||
params:
|
||||
key: $key
|
||||
placeid: $place_id
|
||||
fields: 'address_component,formatted_address,geometry'
|
||||
language: zh-CN
|
||||
variables:
|
||||
key: your_google_map_key
|
||||
place_id: ChIJzyoujG6SAzQRRD3Jr26PFfM
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
- eq: [content.status, OK]
|
||||
extract:
|
||||
- place_name: content.result.formatted_address
|
||||
@@ -1,11 +0,0 @@
|
||||
name: search_area_by_name
|
||||
base_url: http://localhost:8085
|
||||
request:
|
||||
url: /v1/transferairportadminsrv/area/search_area_by_name
|
||||
method: GET
|
||||
headers:
|
||||
Content-Type: application/x-www-form-urlencoded
|
||||
Accept-language: zh_CN
|
||||
cookie: $admin_cookie
|
||||
params:
|
||||
areaName: $in
|
||||
@@ -1,3 +0,0 @@
|
||||
input,formatted_address,address_components_len
|
||||
娄底,中国湖南省娄底市,3
|
||||
新化,中国湖南省娄底市新化县,4
|
||||
|
@@ -1,11 +0,0 @@
|
||||
from utils.validators.validators_of_area import *
|
||||
from utils.validators.validators_of_common import *
|
||||
from utils.setup_hooks import *
|
||||
from utils.teardown_hooks import *
|
||||
|
||||
__all__ = [
|
||||
klook_len_eq,
|
||||
check_search_area_result,
|
||||
exists_default_group,
|
||||
teardown_hook_set_encoding
|
||||
]
|
||||
@@ -1,19 +0,0 @@
|
||||
name: find place from text testcase
|
||||
config:
|
||||
base_url: https://maps.googleapis.com
|
||||
# config 这里的variables优先级更高
|
||||
# 如果本testcase被其他testcase引用，variables无法覆盖这里配置的值
|
||||
# variables:
|
||||
# input: 福田
|
||||
# formatted_address: 中国广东省深圳市福田区
|
||||
teststeps:
|
||||
- name: find place
|
||||
api: api/find_place_api.yml
|
||||
# 如果config上面配置了，这里的同名变量会被覆盖
|
||||
variables:
|
||||
input: 深圳
|
||||
formatted_address: 中国广东省深圳市
|
||||
validate:
|
||||
- eq: [content.candidates.0.formatted_address, $formatted_address]
|
||||
extract:
|
||||
- formatted_address: formatted_address"\s?:\s?"(.*)"
|
||||
@@ -1,13 +0,0 @@
|
||||
config:
|
||||
base_url: http://127.0.0.1:8085
|
||||
variables:
|
||||
admin_cookie: 'cookies'
|
||||
|
||||
teststeps:
|
||||
- name: get group by parent area id
|
||||
api: api/get_area_groups_api.yml
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
- eq: [headers.Content-Type, application/json]
|
||||
- eq: [content.success, true]
|
||||
- exists_default_group: [content.result, '']
|
||||
@@ -1,9 +0,0 @@
|
||||
teststeps:
|
||||
- name: extract title
|
||||
request:
|
||||
method: GET
|
||||
url: http://www.baidu.com
|
||||
extract:
|
||||
- title: <title>(.*)</title>
|
||||
teardown_hooks:
|
||||
- ${teardown_hook_set_encoding($response, utf-8)}
|
||||
@@ -1,19 +0,0 @@
|
||||
name: get place detail
|
||||
config:
|
||||
base_url: https://maps.googleapis.com
|
||||
variables:
|
||||
input: 娄底
|
||||
formatted_address: 中国湖南省娄底市
|
||||
address_components_len: 3
|
||||
teststeps:
|
||||
# 文档里面说这里可以使用testcase，测试了一下，这种方式还有问题，使用testcase时，api里面的变量无法正常覆盖
|
||||
# 所以：teststep中最好不要使用testcase引用
|
||||
- name: find place
|
||||
api: api/find_place_api.yml
|
||||
validate:
|
||||
- eq: [content.candidates.0.formatted_address, $formatted_address]
|
||||
- name: get place detail
|
||||
api: api/place_detail_api.yml
|
||||
validate:
|
||||
- eq: [content.result.formatted_address, $formatted_address]
|
||||
- klook_len_eq: [content.result.address_components, $address_components_len]
|
||||
@@ -1,14 +0,0 @@
|
||||
config:
|
||||
base_url: http://localhost:8085
|
||||
variables:
|
||||
admin_cookie: 'cookies'
|
||||
|
||||
teststeps:
|
||||
- name: search area by name
|
||||
api: api/search_area_by_name_api.yml
|
||||
variables:
|
||||
in: 北京
|
||||
out: 中国北京市
|
||||
validate:
|
||||
- eq: [content.success, true]
|
||||
- check_search_area_result: [content.result, $out]
|
||||
@@ -1,16 +0,0 @@
|
||||
config:
|
||||
name: area manage testsuite
|
||||
variables:
|
||||
admin_cookie: 'cookies'
|
||||
base_url: http://127.0.0.1:8085
|
||||
verify: False
|
||||
|
||||
testcases:
|
||||
- name: search area by name
|
||||
testcase: testcases/search_area_by_name_api_testcase.yml
|
||||
parameters:
|
||||
in-out:
|
||||
- ['北京', '中国, 北京市']
|
||||
- ['广东', '中国, 广东省']
|
||||
- ['shenzhen', '中国, 广东省, 深圳市']
|
||||
- ['서울', '韩国, 首尔']
|
||||
@@ -1,22 +0,0 @@
|
||||
name: get place detail testsuites
|
||||
config:
|
||||
base_url: https://maps.googleapis.com
|
||||
testcases:
|
||||
- name: find place from text
|
||||
testcase: testcases/find_place_testcase.yml
|
||||
parameters:
|
||||
input-formatted_address:
|
||||
- [娄底, 中国湖南省娄底市]
|
||||
- [新化, 中国湖南省娄底市新化县]
|
||||
- ['北京', '中国北京市']
|
||||
- ['广东', '中国广东省']
|
||||
- ['shenzhen', '中国广东省深圳市']
|
||||
- ['서울', '韩国汉城']
|
||||
|
||||
- name: get place detail
|
||||
testcase: testcases/place_detail_testcase.yml
|
||||
parameters:
|
||||
input-formatted_address-address_components_len: ${P(data/place_detail.csv)}
|
||||
# input-formatted_address-address_components_len:
|
||||
# - [娄底, 中国湖南省娄底市, 3]
|
||||
# - [新化, 中国湖南省娄底市新化县, 4]
|
||||
@@ -1,6 +0,0 @@
|
||||
def teardown_hook_set_encoding(response, encoding):
|
||||
"""
|
||||
Set encoding of response.
|
||||
"""
|
||||
response.resp_obj.encoding = encoding
|
||||
return response
|
||||
@@ -1,18 +0,0 @@
|
||||
def check_search_area_result(content, expect_name):
|
||||
print(content, expect_name)
|
||||
found = False
|
||||
for item in content:
|
||||
if item['fullName'] == expect_name:
|
||||
found = True
|
||||
break
|
||||
assert found
|
||||
|
||||
|
||||
def exists_default_group(content, expect):
|
||||
found = False
|
||||
for item in content:
|
||||
if item['defaultGroup']:
|
||||
print('defaultGroup found, id={}, parentAreaId={}'.format(item['id'], item['parentAreaId']))
|
||||
found = True
|
||||
break
|
||||
assert found
|
||||
@@ -1,3 +0,0 @@
|
||||
def klook_len_eq(check_value, expect_value):
|
||||
# 校验列表、字典、字符串等长度是否相等
|
||||
assert len(check_value) == int(expect_value)
|
||||
@@ -1,246 +0,0 @@
|
||||
|
||||
## 案例介绍
|
||||
|
||||
通过接口自动化实现 TesterHome 的登录退出功能。
|
||||
|
||||
|
||||
|
||||
功能描述：
|
||||
|
||||
- 进入[登录页面](https://testerhome.com/account/sign_in)
|
||||
- 输入账号和密码
|
||||
- 点击【Sign In】进行登录
|
||||
|
||||
## 准备工作
|
||||
|
||||
### 抓包生成 HAR 文件
|
||||
|
||||
在浏览器中人工进行登录操作，同时使用抓包工具进行抓包。抓包时建议使用过滤器（Filter），常用的做法是采用被测系统的 host，将无关请求过滤掉。
|
||||
|
||||
|
||||
|
||||
选择需要生成测试用例的请求，导出为 HTTP Archive (.har) 格式的文件。
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### 转换生成测试用例
|
||||
|
||||
成功安装 HttpRunner 后，系统中会新增 `har2case` 命令，使用该命令可将 HAR 数据包转换为 HttpRunner 支持的 `YAML/JSON` 测试用例文件。
|
||||
|
||||
```bash
|
||||
$ har2case docs/data/testerhome-login.har -2y
|
||||
INFO:root:Start to generate testcase.
|
||||
INFO:root:dump testcase to YAML format.
|
||||
INFO:root:Generate YAML testcase successfully: docs/data/testerhome-login.yml
|
||||
```
|
||||
|
||||
生成的测试用例内容如下：
|
||||
|
||||
<details>
|
||||
<summary>点击查看</summary>
|
||||
|
||||
```yaml
|
||||
- config:
|
||||
name: testcase description
|
||||
variables: {}
|
||||
|
||||
- test:
|
||||
name: /account/sign_in
|
||||
request:
|
||||
headers:
|
||||
If-None-Match: W/"bc9ae267fdcbd89bf1dfaea10dea2b0e"
|
||||
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36
|
||||
(KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36
|
||||
method: GET
|
||||
url: https://testerhome.com/account/sign_in
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
- eq: [headers.Content-Type, text/html; charset=utf-8]
|
||||
|
||||
- test:
|
||||
name: /assets/big_logo-cd32144f74c18746f3dce33e1040e7dfe4c07c8e611e37f3868b1c16b5095da3.png
|
||||
request:
|
||||
headers:
|
||||
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36
|
||||
(KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36
|
||||
method: GET
|
||||
url: https://testerhome.com/assets/big_logo-cd32144f74c18746f3dce33e1040e7dfe4c07c8e611e37f3868b1c16b5095da3.png
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
- eq: [headers.Content-Type, image/png]
|
||||
|
||||
- test:
|
||||
name: /account/sign_in
|
||||
request:
|
||||
data:
|
||||
commit: Sign In
|
||||
user[login]: debugtalk
|
||||
user[password]: XXXXXXXX
|
||||
user[remember_me]: '1'
|
||||
utf8: ✓
|
||||
headers:
|
||||
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
|
||||
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36
|
||||
(KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36
|
||||
X-CSRF-Token: 0zAKFDDPnNI2+Vwq/iwDPR9vo7KWobfNLAye4EaGBTlsSxMzTNf39lLF9z35f5mcROM7JgOP+azBCuDe84G+XA==
|
||||
X-Requested-With: XMLHttpRequest
|
||||
method: POST
|
||||
url: https://testerhome.com/account/sign_in
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
- eq: [headers.Content-Type, text/javascript; charset=utf-8]
|
||||
|
||||
- test:
|
||||
name: /
|
||||
request:
|
||||
headers:
|
||||
If-None-Match: W/"bad62c68dac27b01151516aad5c7f0be"
|
||||
Turbolinks-Referrer: https://testerhome.com/account/sign_in
|
||||
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36
|
||||
(KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36
|
||||
method: GET
|
||||
url: https://testerhome.com/
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
- eq: [headers.Content-Type, text/html; charset=utf-8]
|
||||
```
|
||||
</details>
|
||||
|
||||
### 首次运行测试用例
|
||||
|
||||
成功安装 HttpRunner 后，系统中会新增 `hrun` 命令，该命令是 HttpRunner 的核心命令，用于运行 HttpRunner 支持的 `YAML/JSON` 测试用例文件。
|
||||
|
||||
生成测试用例后，我们可以先尝试运行一次，大多数情况，如果被测场景中不存在关联的情况，是可以直接运行成功的。
|
||||
|
||||
```bash
|
||||
$ hrun docs/data/testerhome-login.yml --failfast --log-level info
|
||||
INFO Start to run testcase: testcase description
|
||||
/account/sign_in
|
||||
INFO GET https://testerhome.com/account/sign_in
|
||||
INFO status_code: 200, response_time(ms): 189.66 ms, response_length: 12584 bytes
|
||||
|
||||
.
|
||||
/assets/big_logo-cd32144f74c18746f3dce33e1040e7dfe4c07c8e611e37f3868b1c16b5095da3.png
|
||||
INFO GET https://testerhome.com/assets/big_logo-cd32144f74c18746f3dce33e1040e7dfe4c07c8e611e37f3868b1c16b5095da3.png
|
||||
INFO status_code: 200, response_time(ms): 83.98 ms, response_length: 15229 bytes
|
||||
|
||||
.
|
||||
/account/sign_in
|
||||
INFO POST https://testerhome.com/account/sign_in
|
||||
INFO status_code: 200, response_time(ms): 172.8 ms, response_length: 89 bytes
|
||||
|
||||
.
|
||||
/
|
||||
INFO GET https://testerhome.com/
|
||||
INFO status_code: 200, response_time(ms): 257.41 ms, response_length: 52463 bytes
|
||||
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 4 tests in 0.722s
|
||||
|
||||
OK
|
||||
INFO Start to render Html report ...
|
||||
INFO Generated Html report: /Users/debugtalk/MyProjects/HttpRunner-dev/httprunner-docs-v2x/reports/1555662601.html
|
||||
```
|
||||
|
||||
比较幸运，脚本在没有做任何修改的情况下运行成功了。
|
||||
|
||||
|
||||
## 调试 & 优化测试用例
|
||||
|
||||
虽然脚本运行成功了，但是为了更好地管理和维护脚本，需要对脚本进行优化调整。
|
||||
|
||||
### 关联处理
|
||||
|
||||
查看录制生成的脚本，可以看到在发起登录请求时包含了 `X-CSRF-Token`，如果熟悉网络信息安全的基础知识，就会联想到该字段是动态变化的，每次都是先从服务器端返回至客户端，客户端在后续发起请求时需要携带该字段。
|
||||
|
||||
```yaml
|
||||
- test:
|
||||
name: /account/sign_in
|
||||
request:
|
||||
data:
|
||||
commit: Sign In
|
||||
user[login]: debugtalk
|
||||
user[password]: XXXXXXXX
|
||||
user[remember_me]: '1'
|
||||
utf8: ✓
|
||||
headers:
|
||||
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
|
||||
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36
|
||||
(KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36
|
||||
X-CSRF-Token: 0zAKFDDPnNI2+Vwq/iwDPR9vo7KWobfNLAye4EaGBTlsSxMzTNf39lLF9z35f5mcROM7JgOP+azBCuDe84G+XA==
|
||||
X-Requested-With: XMLHttpRequest
|
||||
method: POST
|
||||
url: https://testerhome.com/account/sign_in
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
- eq: [headers.Content-Type, text/html; charset=utf-8]
|
||||
```
|
||||
|
||||
虽然当前直接运行录制生成的脚本也是成功的，但很有可能在过了一段时间后，`X-CSRF-Token` 失效，脚本也就无法再成功运行了。因此在测试脚本中，该字段不能写死为抓包时获取的值，而是要每次动态地从前面的接口响应中获取。
|
||||
|
||||
那要怎么确定该字段是在之前的哪个接口中返回的呢？
|
||||
|
||||
操作方式也很简单，可以在抓包工具中对该字段进行搜索，特别地，搜索范围限定为响应内容（Response Header、Response Body）。
|
||||
|
||||
即可搜索得到该字段是在哪个接口中从服务器端返回值客户端的。
|
||||
|
||||
|
||||
|
||||
有时候可能搜索会得到多个结果，那么在确定是使用哪个接口响应的时候，遵循两个原则即可：
|
||||
|
||||
- 响应一定是出现在当前接口之前
|
||||
- 如果在当前接口之前存在多个接口均有此返回，那么取最靠近当前接口的即可
|
||||
|
||||
通过前面的搜索可知，`X-CSRF-Token` 的值是在第一个接口中响应返回的。
|
||||
|
||||
|
||||
|
||||
确定出具体的接口后，那么就可以在测试脚本中从该接口使用 `extract` 提取对应的字段，然后在后续接口中引用提取出的字段。
|
||||
|
||||
在当前案例中，第一个接口的响应内容为 HTML 页面，要提取字段可以使用正则匹配的方式。具体的做法就是指定目标字段的左右边界，目标字段使用 `(.*)` 匹配获取。
|
||||
|
||||
```yaml
|
||||
- test:
|
||||
name: /account/sign_in
|
||||
request:
|
||||
headers:
|
||||
If-None-Match: W/"bc9ae267fdcbd89bf1dfaea10dea2b0e"
|
||||
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36
|
||||
(KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36
|
||||
method: GET
|
||||
url: https://testerhome.com/account/sign_in
|
||||
extract:
|
||||
X_CSRF_Token: <meta name="csrf-token" content="(.*)" />
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
- eq: [headers.Content-Type, text/html; charset=utf-8]
|
||||
```
|
||||
|
||||
然后，在后续使用到该字段的接口中，引用提取出的字段即可。
|
||||
|
||||
```yaml
|
||||
- test:
|
||||
name: /account/sign_in
|
||||
request:
|
||||
data:
|
||||
commit: Sign In
|
||||
user[login]: debugtalk
|
||||
user[password]: XXXXXXXX
|
||||
user[remember_me]: '1'
|
||||
utf8: ✓
|
||||
headers:
|
||||
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
|
||||
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36
|
||||
(KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36
|
||||
X-CSRF-Token: $X_CSRF_Token
|
||||
X-Requested-With: XMLHttpRequest
|
||||
method: POST
|
||||
url: https://testerhome.com/account/sign_in
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
- eq: [headers.Content-Type, text/html; charset=utf-8]
|
||||
```
|
||||
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 46 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 514 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 111 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 184 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 202 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 158 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 112 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 106 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 289 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 64 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 27 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 361 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 409 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 269 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 183 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 636 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 596 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 224 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 545 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 189 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 735 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 109 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 194 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 80 KiB |
@@ -1,36 +1,49 @@
|
||||
|
||||
HttpRunner 是一款面向 HTTP(S) 协议的通用测试框架，只需编写维护一份 `YAML/JSON` 脚本，即可实现自动化测试、性能测试、线上监控、持续集成等多种测试需求。
|
||||
# HttpRunner
|
||||
|
||||
此文档适用于全新发布的 `HttpRunner 2.x` 版本，`1.x` 版本的使用文档请查看[历史链接][httprunner1]。
|
||||
[](https://pepy.tech/project/httprunner)
|
||||
[](https://github.com/httprunner/httprunner/actions)
|
||||
[](https://github.com/httprunner/httprunner/actions)
|
||||
[](https://codecov.io/gh/httprunner/httprunner)
|
||||
[](https://pypi.python.org/pypi/httprunner)
|
||||
[](https://pypi.python.org/pypi/httprunner)
|
||||
[](https://testerhome.com/github_statistics)
|
||||
|
||||
## 设计理念
|
||||
*HttpRunner* is a simple & elegant, yet powerful HTTP(S) testing framework. Enjoy! ✨ 🚀 ✨
|
||||
|
||||
- 充分复用优秀的开源项目，不追求重复造轮子，而是将强大的轮子组装成战车
|
||||
- 遵循 `约定大于配置` 的准则，在框架功能中融入自动化测试最佳工程实践
|
||||
- 追求投入产出比，一份投入即可实现多种测试需求
|
||||
> This docs site is corresponding to the latest version `3.x`, for `2.x` you can reference [`archive link`](https://v2.httprunner.org/).
|
||||
|
||||
## 核心特性
|
||||
## Design Philosophy
|
||||
|
||||
- 继承 [Requests][Requests] 的全部特性，轻松实现 HTTP(S) 的各种测试需求
|
||||
- 采用 `YAML/JSON` 的形式描述测试场景，保障测试用例描述的统一性和可维护性
|
||||
- 借助辅助函数（debugtalk.py），在测试脚本中轻松实现复杂的动态计算逻辑
|
||||
- 支持完善的测试用例分层机制，充分实现测试用例的复用
|
||||
- 测试前后支持完善的 hook 机制
|
||||
- 响应结果支持丰富的校验机制
|
||||
- 基于 HAR 实现接口录制和用例生成功能（[har2case][har2case]）
|
||||
- 结合 [Locust][Locust] 框架，无需额外的工作即可实现分布式性能测试
|
||||
- 执行方式采用 CLI 调用，可与 Jenkins 等持续集成工具完美结合
|
||||
- 测试结果统计报告简洁清晰，附带详尽统计信息和日志记录
|
||||
- 极强的可扩展性，轻松实现二次开发和 Web 平台化
|
||||
- Convention over configuration
|
||||
- ROI matters
|
||||
- Embrace open source, leverage [`requests`][requests], [`pytest`][pytest], [`pydantic`][pydantic], [`allure`][allure] and [`locust`][locust].
|
||||
|
||||
## 更多信息
|
||||
## Key Features
|
||||
|
||||
- Inherit all powerful features of [`requests`][requests], just have fun to handle HTTP(S) in human way.
|
||||
- Define testcase in YAML or JSON format, run with [`pytest`][pytest] in concise and elegant manner.
|
||||
- Record and generate testcases with [`HAR`][HAR] support.
|
||||
- Supports `variables`/`extract`/`validate`/`hooks` mechanisms to create extremely complex test scenarios.
|
||||
- With `debugtalk.py` plugin, any function can be used in any part of your testcase.
|
||||
- With [`jmespath`][jmespath], extract and validate json response has never been easier.
|
||||
- With [`pytest`][pytest], hundreds of plugins are readily available.
|
||||
- With [`allure`][allure], test report can be pretty nice and powerful.
|
||||
- With reuse of [`locust`][locust], you can run performance test without extra work.
|
||||
- CLI command supported, perfect combination with `CI/CD`.
|
||||
|
||||
## Subscribe
|
||||
|
||||
关注 HttpRunner 的微信公众号，第一时间获得最新资讯。
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
[httprunner1]: https://v1.httprunner.org/
|
||||
[Requests]: http://docs.python-requests.org/en/master/
|
||||
[Locust]: http://locust.io/
|
||||
[har2case]: https://github.com/HttpRunner/har2case
|
||||
[requests]: http://docs.python-requests.org/en/master/
|
||||
[pytest]: https://docs.pytest.org/
|
||||
[pydantic]: https://pydantic-docs.helpmanual.io/
|
||||
[locust]: http://locust.io/
|
||||
[jmespath]: https://jmespath.org/
|
||||
[allure]: https://docs.qameta.io/allure/
|
||||
[HAR]: http://httparchive.org/
|
||||
|
||||
64
docs/installation.md
Normal file
64
docs/installation.md
Normal file
@@ -0,0 +1,64 @@
|
||||
|
||||
`HttpRunner` is developed with Python, it supports Python `3.6+` and most operating systems. Combination of Python `3.6/3.7/3.8` and `macOS/Linux/Windows` are tested continuously on [GitHub-Actions][github-actions].
|
||||
|
||||
## Installation
|
||||
|
||||
`HttpRunner` is available on [`PyPI`][PyPI] and can be installed through `pip`.
|
||||
|
||||
```bash
|
||||
$ pip install httprunner
|
||||
```
|
||||
|
||||
If you want to keep up with the latest version, you can install with github repository url.
|
||||
|
||||
```bash
|
||||
$ pip install git+https://github.com/httprunner/httprunner.git@master
|
||||
```
|
||||
|
||||
If you have installed `HttpRunner` before and want to upgrade to the latest version, you can use the `-U` option.
|
||||
|
||||
```bash
|
||||
$ pip install -U httprunner
|
||||
$ pip install -U git+https://github.com/httprunner/httprunner.git@master
|
||||
```
|
||||
|
||||
## Check Installation
|
||||
|
||||
When HttpRunner is installed, 4 commands will be added in your system.
|
||||
|
||||
- `httprunner`: main command, used for all functions
|
||||
- `hrun`: alias for `httprunner run`, used to run YAML/JSON testcases
|
||||
- `hmake`: alias for `httprunner make`, used to convert YAML/JSON testcases to pytest files
|
||||
- `har2case`: alias for `httprunner har2case`, used to convert HAR to YAML/JSON testcases
|
||||
|
||||
To see `HttpRunner` version:
|
||||
|
||||
```text
|
||||
$ httprunner -V
|
||||
3.0.6
|
||||
```
|
||||
|
||||
To see available options, run:
|
||||
|
||||
```text
|
||||
$ httprunner -h
|
||||
usage: httprunner [-h] [-V] {run,startproject,har2case,make} ...
|
||||
|
||||
One-stop solution for HTTP(S) testing.
|
||||
|
||||
positional arguments:
|
||||
{run,startproject,har2case,make}
|
||||
sub-command help
|
||||
run Make HttpRunner testcases and run with pytest.
|
||||
startproject Create a new project with template structure.
|
||||
har2case Convert HAR(HTTP Archive) to YAML/JSON testcases for
|
||||
HttpRunner.
|
||||
make Convert YAML/JSON testcases to pytest cases.
|
||||
|
||||
optional arguments:
|
||||
-h, --help show this help message and exit
|
||||
-V, --version show version
|
||||
```
|
||||
|
||||
[PyPI]: https://pypi.python.org/pypi
|
||||
[github-actions]: https://github.com/httprunner/httprunner/actions
|
||||
@@ -1,11 +0,0 @@
|
||||
|
||||
(function(i,s,o,g,r,a,m){i["SlardarMonitorObject"]=r;i[r]=i[r]||function(){(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date;a=s.createElement(o),m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)})(window,document,"script","https://i.snssdk.com/slardar/sdk.js?bid=httprunner","Slardar");
|
||||
window.Slardar('config', {
|
||||
sampleRate: 1,
|
||||
bid: 'httprunner',
|
||||
ignoreAjax: [],
|
||||
ignoreStatic: [],
|
||||
hookFetch: true,
|
||||
enableSizeStats: true
|
||||
});
|
||||
window.Slardar('send', 'pageview');
|
||||
@@ -1,112 +0,0 @@
|
||||
|
||||
## 环境变量的作用
|
||||
|
||||
在自动化测试中，有时需要借助环境变量实现某些特定的目的，常见的场景包括：
|
||||
|
||||
- 切换测试环境
|
||||
- 切换测试配置
|
||||
- 存储敏感数据（从[信息安全](/prepare/security/)的角度出发）
|
||||
|
||||
## 设置环境变量
|
||||
|
||||
### 在终端中预设环境变量
|
||||
|
||||
使用环境变量之前，需要先在系统中设置环境变量名称和值，传统的方式为使用 export 命令（Windows系统中使用 set 命令）：
|
||||
|
||||
```bash
|
||||
$ export UserName=admin
|
||||
$ echo $UserName
|
||||
admin
|
||||
$ export Password=123456
|
||||
$ echo $Password
|
||||
123456
|
||||
```
|
||||
|
||||
然后，在程序中就可以对系统中的环境变量进行读取。
|
||||
|
||||
```bash
|
||||
$ python
|
||||
>>> import os
|
||||
>>> os.environ["UserName"]
|
||||
'admin'
|
||||
```
|
||||
|
||||
### 通过 .env 文件设置环境变量
|
||||
|
||||
除了这种方式，HttpRunner 还借鉴了 pipenv [加载 `.env` 的方式][pipenv_load_env]。
|
||||
|
||||
默认情况下，在自动化测试项目的根目录中，创建 `.env` 文件，并将敏感数据信息放置到其中，存储采用 `name=value` 的格式：
|
||||
|
||||
```bash
|
||||
$ cat .env
|
||||
UserName=admin
|
||||
Password=123456
|
||||
PROJECT_KEY=ABCDEFGH
|
||||
```
|
||||
|
||||
同时，`.env` 文件不应该添加到代码仓库中，建议将 `.env` 加入到 `.gitignore` 中。
|
||||
|
||||
HttpRunner 运行时，会自动将 `.env` 文件中的内容加载到运行时（RunTime）的环境变量中，然后在运行时中就可以对环境变量进行读取了。
|
||||
|
||||
若需加载不位于自动化项目根目录中的 `.env`，或者其它名称的 `.env` 文件（例如 `production.env`），可以采用 `--dot-env-path` 参数指定文件路径：
|
||||
|
||||
```bash
|
||||
$ hrun /path/to/testcase.yml --dot-env-path /path/to/.env --log-level debug
|
||||
INFO Loading environment variables from /path/to/.env
|
||||
DEBUG Loaded variable: UserName
|
||||
DEBUG Loaded variable: Password
|
||||
DEBUG Loaded variable: PROJECT_KEY
|
||||
...
|
||||
```
|
||||
|
||||
## 引用环境变量
|
||||
|
||||
在 HttpRunner 中内置了函数 `environ`（简称 `ENV`），可用于在 YAML/JSON 脚本中直接引用环境变量。
|
||||
|
||||
```yaml
|
||||
- test:
|
||||
name: login
|
||||
request:
|
||||
url: http://host/api/login
|
||||
method: POST
|
||||
headers:
|
||||
Content-Type: application/json
|
||||
json:
|
||||
username: ${ENV(UserName)}
|
||||
password: ${ENV(Password)}
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
```
|
||||
|
||||
若还需对读取的环境变量做进一步处理，则可以在 `debugtalk.py` 通过 Python 内置的函数 `os.environ` 对环境变量进行引用，然后再实现处理逻辑。
|
||||
|
||||
例如，若发起请求的密码需要先与密钥进行拼接并生成 MD5，那么就可以在 `debugtalk.py` 文件中实现如下函数：
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
def get_encrypt_password():
|
||||
raw_passwd = os.environ["Password"]
|
||||
PROJECT_KEY = os.environ["PROJECT_KEY"])
|
||||
password = (raw_passwd + PROJECT_KEY).encode('ascii')
|
||||
return hmac.new(password, hashlib.sha1).hexdigest()
|
||||
```
|
||||
|
||||
然后，在 YAML/JSON 格式的测试用例中，就可以通过`${func()}`的方式引用环境变量的值了。
|
||||
|
||||
```yaml
|
||||
- test:
|
||||
name: login
|
||||
request:
|
||||
url: http://host/api/login
|
||||
method: POST
|
||||
headers:
|
||||
Content-Type: application/json
|
||||
json:
|
||||
username: ${ENV(UserName)}
|
||||
password: ${get_encrypt_password()}
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
```
|
||||
|
||||
[pipenv_load_env]: https://docs.pipenv.org/advanced/#automatic-loading-of-env
|
||||
@@ -1,581 +0,0 @@
|
||||
## 介绍
|
||||
|
||||
在自动化测试中，经常会遇到如下场景：
|
||||
|
||||
- 测试搜索功能，只有一个搜索输入框，但有 10 种不同类型的搜索关键字；
|
||||
- 测试账号登录功能，需要输入用户名和密码，按照等价类划分后有 20 种组合情况。
|
||||
|
||||
这里只是随意找了两个典型的例子，相信大家都有遇到过很多类似的场景。总结下来，就是在我们的自动化测试脚本中存在参数，并且我们需要采用不同的参数去运行。
|
||||
|
||||
经过概括，参数基本上分为两种类型：
|
||||
|
||||
- 单个独立参数：例如前面的第一种场景，我们只需要变换搜索关键字这一个参数
|
||||
- 多个具有关联性的参数：例如前面的第二种场景，我们需要变换用户名和密码两个参数，并且这两个参数需要关联组合
|
||||
|
||||
然后，对于参数而言，我们可能具有一个参数列表，在脚本运行时需要按照不同的规则去取值，例如顺序取值、随机取值、循环取值等等。
|
||||
|
||||
这就是典型的参数化和数据驱动。
|
||||
|
||||
如需了解 HttpRunner 参数化数据驱动机制的实现原理和技术细节，可前往阅读[《HttpRunner 实现参数化数据驱动机制》](http://debugtalk.com/post/httprunner-data-driven/)。
|
||||
|
||||
## 测试用例集（testsuite）准备
|
||||
|
||||
从 2.0.0 版本开始，HttpRunner 不再支持在测试用例文件中进行参数化配置；参数化的功能需要在 testsuite 中实现。变更的目的是让测试用例（testcase）的概念更纯粹，关于测试用例和测试用例集的概念定义，详见[《测试用例组织》](/prepare/parameters/)。
|
||||
|
||||
参数化机制需要在测试用例集（testsuite）中实现。如需实现数据驱动机制，需要创建一个 testsuite，在 testsuite 中引用测试用例，并定义参数化配置。
|
||||
|
||||
测试用例集（testsuite）的格式如下所示：
|
||||
|
||||
```yaml
|
||||
config:
|
||||
name: testsuite description
|
||||
|
||||
testcases:
|
||||
testcase1_name:
|
||||
testcase: /path/to/testcase1
|
||||
|
||||
testcase2_name:
|
||||
testcase: /path/to/testcase2
|
||||
```
|
||||
|
||||
需要注意的是，testsuite 和 testcase 的格式存在较大区别，详见[《测试用例组织》](/prepare/testcase-structure/)。
|
||||
|
||||
|
||||
## 参数配置概述
|
||||
|
||||
如需对某测试用例（testcase）实现参数化数据驱动，需要使用 `parameters` 关键字，定义参数名称并指定数据源取值方式。
|
||||
|
||||
参数名称的定义分为两种情况：
|
||||
|
||||
- 独立参数单独进行定义；
|
||||
- 多个参数具有关联性的参数需要将其定义在一起，采用短横线（`-`）进行连接。
|
||||
|
||||
数据源指定支持三种方式：
|
||||
|
||||
- 在 YAML/JSON 中直接指定参数列表：该种方式最为简单易用，适合参数列表比较小的情况
|
||||
- 通过内置的 parameterize（可简写为P）函数引用 CSV 文件：该种方式需要准备 CSV 数据文件，适合数据量比较大的情况
|
||||
- 调用 debugtalk.py 中自定义的函数生成参数列表：该种方式最为灵活，可通过自定义 Python 函数实现任意场景的数据驱动机制，当需要动态生成参数列表时也需要选择该种方式
|
||||
|
||||
三种方式可根据实际项目需求进行灵活选择，同时支持多种方式的组合使用。假如测试用例中定义了多个参数，那么测试用例在运行时会对参数进行笛卡尔积组合，覆盖所有参数组合情况。
|
||||
|
||||
使用方式概览如下：
|
||||
|
||||
```yaml
|
||||
config:
|
||||
name: "demo"
|
||||
|
||||
testcases:
|
||||
testcase1_name:
|
||||
testcase: /path/to/testcase1
|
||||
parameters:
|
||||
user_agent: ["iOS/10.1", "iOS/10.2", "iOS/10.3"]
|
||||
user_id: ${P(user_id.csv)}
|
||||
username-password: ${get_account(10)}
|
||||
```
|
||||
|
||||
|
||||
## 参数配置详解
|
||||
|
||||
将参数名称定义和数据源指定方式进行组合，共有 6 种形式。现分别针对每一类情况进行详细说明。
|
||||
|
||||
### 独立参数 & 直接指定参数列表
|
||||
|
||||
对于参数列表比较小的情况，最简单的方式是直接在 YAML/JSON 中指定参数列表内容。
|
||||
|
||||
例如，对于独立参数 `user_id`，参数列表为 `[1001, 1002, 1003, 1004]`，那么就可以按照如下方式进行配置：
|
||||
|
||||
```yaml
|
||||
config:
|
||||
name: testcase description
|
||||
|
||||
testcases:
|
||||
create user:
|
||||
testcase: demo-quickstart-6.yml
|
||||
parameters:
|
||||
user_id: [1001, 1002, 1003, 1004]
|
||||
```
|
||||
|
||||
进行该配置后，测试用例在运行时就会对 user_id 实现数据驱动，即分别使用 `[1001, 1002, 1003, 1004]` 四个值运行测试用例。
|
||||
|
||||
<details>
|
||||
<summary>点击查看运行日志</summary>
|
||||
|
||||
```text
|
||||
$ hrun docs/data/demo-quickstart-7.json
|
||||
INFO Start to run testcase: create user 1001
|
||||
/api/get-token
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
INFO status_code: 200, response_time(ms): 8.95 ms, response_length: 46 bytes
|
||||
|
||||
.
|
||||
/api/users/1001
|
||||
INFO POST http://127.0.0.1:5000/api/users/1001
|
||||
INFO status_code: 201, response_time(ms): 3.02 ms, response_length: 54 bytes
|
||||
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 2 tests in 0.021s
|
||||
|
||||
OK
|
||||
INFO Start to run testcase: create user 1002
|
||||
/api/get-token
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
INFO status_code: 200, response_time(ms): 2.78 ms, response_length: 46 bytes
|
||||
|
||||
.
|
||||
/api/users/1002
|
||||
INFO POST http://127.0.0.1:5000/api/users/1002
|
||||
INFO status_code: 201, response_time(ms): 2.84 ms, response_length: 54 bytes
|
||||
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 2 tests in 0.007s
|
||||
|
||||
OK
|
||||
INFO Start to run testcase: create user 1003
|
||||
/api/get-token
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
INFO status_code: 200, response_time(ms): 2.92 ms, response_length: 46 bytes
|
||||
|
||||
.
|
||||
/api/users/1003
|
||||
INFO POST http://127.0.0.1:5000/api/users/1003
|
||||
INFO status_code: 201, response_time(ms): 5.56 ms, response_length: 54 bytes
|
||||
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 2 tests in 0.011s
|
||||
|
||||
OK
|
||||
INFO Start to run testcase: create user 1004
|
||||
/api/get-token
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
INFO status_code: 200, response_time(ms): 5.25 ms, response_length: 46 bytes
|
||||
|
||||
.
|
||||
/api/users/1004
|
||||
INFO POST http://127.0.0.1:5000/api/users/1004
|
||||
INFO status_code: 201, response_time(ms): 7.02 ms, response_length: 54 bytes
|
||||
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 2 tests in 0.016s
|
||||
|
||||
OK
|
||||
INFO Start to render Html report ...
|
||||
INFO Generated Html report: /Users/debugtalk/MyProjects/HttpRunner-dev/httprunner-docs-v2x/reports/1548518757.html
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
可以看出，测试用例总共运行了 4 次，并且每次运行时都是采用的不同 user_id。
|
||||
|
||||
### 关联参数 & 直接指定参数列表
|
||||
|
||||
对于具有关联性的多个参数，例如 username 和 password，那么就可以按照如下方式进行配置：
|
||||
|
||||
```yaml
|
||||
config:
|
||||
name: "demo"
|
||||
|
||||
testcases:
|
||||
testcase1_name:
|
||||
testcase: /path/to/testcase1
|
||||
parameters:
|
||||
username-password:
|
||||
- ["user1", "111111"]
|
||||
- ["user2", "222222"]
|
||||
- ["user3", "333333"]
|
||||
```
|
||||
|
||||
进行该配置后，测试用例在运行时就会对 username 和 password 实现数据驱动，即分别使用 `{"username": "user1", "password": "111111"}`、`{"username": "user2", "password": "222222"}`、`{"username": "user3", "password": "333333"}` 运行 3 次测试，并且保证参数值总是成对使用。
|
||||
|
||||
### 独立参数 & 引用 CSV 文件
|
||||
|
||||
对于已有参数列表，并且数据量比较大的情况，比较适合的方式是将参数列表值存储在 CSV 数据文件中。
|
||||
|
||||
对于 CSV 数据文件，需要遵循如下几项约定的规则：
|
||||
|
||||
- CSV 文件中的第一行必须为参数名称，从第二行开始为参数值，每个（组）值占一行；
|
||||
- 若同一个 CSV 文件中具有多个参数，则参数名称和数值的间隔符需实用英文逗号；
|
||||
- 在 YAML/JSON 文件引用 CSV 文件时，文件路径为基于项目根目录（debugtalk.py 所在路径）的相对路径。
|
||||
|
||||
例如，user_id 的参数取值范围为 1001～2000，那么我们就可以创建 user_id.csv，并且在文件中按照如下形式进行描述。
|
||||
|
||||
```csv
|
||||
user_id
|
||||
1001
|
||||
1002
|
||||
...
|
||||
1999
|
||||
2000
|
||||
```
|
||||
|
||||
然后在 YAML/JSON 测试用例文件中，就可以通过内置的 `parameterize`（可简写为 `P`）函数引用 CSV 文件。
|
||||
|
||||
假设项目的根目录下有 data 文件夹，user_id.csv 位于其中，那么 user_id.csv 的引用描述如下：
|
||||
|
||||
```yaml
|
||||
config:
|
||||
name: "demo"
|
||||
|
||||
testcases:
|
||||
testcase1_name:
|
||||
testcase: /path/to/testcase1
|
||||
parameters:
|
||||
user_id: ${P(data/user_id.csv)}
|
||||
```
|
||||
|
||||
即 `P` 函数的参数（CSV 文件路径）是相对于项目根目录的相对路径。当然，这里也可以使用 CSV 文件在系统中的绝对路径，不过这样的话在项目路径变动时就会出现问题，因此推荐使用相对路径的形式。
|
||||
|
||||
### 关联参数 & 引用 CSV 文件
|
||||
|
||||
对于具有关联性的多个参数，例如 username 和 password，那么就可以创建 [account.csv](/data/account.csv)，并在文件中按照如下形式进行描述。
|
||||
|
||||
```csv
|
||||
username,password
|
||||
test1,111111
|
||||
test2,222222
|
||||
test3,333333
|
||||
```
|
||||
|
||||
然后在 YAML/JSON 测试用例文件中，就可以通过内置的 `parameterize`（可简写为 `P`）函数引用 CSV 文件。
|
||||
|
||||
假设项目的根目录下有 data 文件夹，account.csv 位于其中，那么 account.csv 的引用描述如下：
|
||||
|
||||
```yaml
|
||||
config:
|
||||
name: "demo"
|
||||
|
||||
testcases:
|
||||
testcase1_name:
|
||||
testcase: /path/to/testcase1
|
||||
parameters:
|
||||
username-password: ${P(data/account.csv)}
|
||||
```
|
||||
|
||||
需要说明的是，在 parameters 中指定的参数名称必须与 CSV 文件中第一行的参数名称一致，顺序可以不一致，参数个数也可以不一致。
|
||||
|
||||
例如，在 [account.csv](/data/account.csv) 文件中可以包含多个参数，username、password、phone、age：
|
||||
|
||||
```csv
|
||||
username,password,phone,age
|
||||
test1,111111,18600000001,21
|
||||
test2,222222,18600000002,22
|
||||
test3,333333,18600000003,23
|
||||
```
|
||||
|
||||
而在 YAML/JSON 测试用例文件中指定参数时，可以只使用部分参数，并且参数顺序无需与 CSV 文件中参数名称的顺序一致。
|
||||
|
||||
```yaml
|
||||
config:
|
||||
name: "demo"
|
||||
|
||||
testcases:
|
||||
testcase1_name:
|
||||
testcase: /path/to/testcase1
|
||||
parameters:
|
||||
phone-username: ${P(account.csv)}
|
||||
```
|
||||
|
||||
### 独立参数 & 引用自定义函数
|
||||
|
||||
对于没有现成参数列表，或者需要更灵活的方式动态生成参数的情况，可以通过在 debugtalk.py 中自定义函数生成参数列表，并在 YAML/JSON 引用自定义函数的方式。
|
||||
|
||||
例如，若需对 user_id 进行参数化数据驱动，参数取值范围为 1001～1004，那么就可以在 debugtalk.py 中定义一个函数，返回参数列表。
|
||||
|
||||
```python
|
||||
def get_user_id():
|
||||
return [
|
||||
{"user_id": 1001},
|
||||
{"user_id": 1002},
|
||||
{"user_id": 1003},
|
||||
{"user_id": 1004}
|
||||
]
|
||||
```
|
||||
|
||||
然后，在 YAML/JSON 的 parameters 中就可以通过调用自定义函数的形式来指定数据源。
|
||||
|
||||
```yaml
|
||||
config:
|
||||
name: "demo"
|
||||
|
||||
testcases:
|
||||
testcase1_name:
|
||||
testcase: /path/to/testcase1
|
||||
parameters:
|
||||
user_id: ${get_user_id()}
|
||||
```
|
||||
|
||||
另外，通过函数的传参机制，还可以实现更灵活的参数生成功能，在调用函数时指定需要生成的参数个数。
|
||||
|
||||
### 关联参数 & 引用自定义函数
|
||||
|
||||
对于具有关联性的多个参数，实现方式也类似。
|
||||
|
||||
例如，在 debugtalk.py 中定义函数 get_account，生成指定数量的账号密码参数列表。
|
||||
|
||||
```python
|
||||
def get_account(num):
|
||||
accounts = []
|
||||
for index in range(1, num+1):
|
||||
accounts.append(
|
||||
{"username": "user%s" % index, "password": str(index) * 6},
|
||||
)
|
||||
|
||||
return accounts
|
||||
```
|
||||
|
||||
那么在 YAML/JSON 的 parameters 中就可以调用自定义函数生成指定数量的参数列表。
|
||||
|
||||
```yaml
|
||||
config:
|
||||
name: "demo"
|
||||
|
||||
testcases:
|
||||
testcase1_name:
|
||||
testcase: /path/to/testcase1
|
||||
parameters:
|
||||
username-password: ${get_account(10)}
|
||||
```
|
||||
|
||||
> 需要注意的是，在自定义函数中，生成的参数列表必须为 `list of dict` 的数据结构，该设计主要是为了与 CSV 文件的处理机制保持一致。
|
||||
|
||||
|
||||
## 参数化运行
|
||||
|
||||
完成以上参数定义和数据源准备工作之后，参数化运行与普通测试用例的运行完全一致。
|
||||
|
||||
采用 hrun 命令运行自动化测试：
|
||||
|
||||
```bash
|
||||
$ hrun tests/data/demo_parameters.yml
|
||||
```
|
||||
|
||||
采用 locusts 命令运行性能测试：
|
||||
|
||||
```bash
|
||||
$ locusts -f tests/data/demo_parameters.yml
|
||||
```
|
||||
|
||||
区别在于，自动化测试时遍历一遍后会终止执行，性能测试时每个并发用户都会循环遍历所有参数。
|
||||
|
||||
## 案例演示
|
||||
|
||||
假设我们有一个获取 token 的[测试用例](/data/demo-testcase-get-token.yml)。
|
||||
|
||||
<details>
|
||||
<summary>点击查看 YAML 测试用例</summary>
|
||||
```yaml
|
||||
- config:
|
||||
name: get token
|
||||
base_url: http://127.0.0.1:5000
|
||||
variables:
|
||||
device_sn: ${gen_random_string(15)}
|
||||
os_platform: 'ios'
|
||||
app_version: '2.8.6'
|
||||
|
||||
- test:
|
||||
name: get token with $device_sn, $os_platform, $app_version
|
||||
request:
|
||||
headers:
|
||||
Content-Type: application/json
|
||||
User-Agent: python-requests/2.18.4
|
||||
app_version: $app_version
|
||||
device_sn: $device_sn
|
||||
os_platform: $os_platform
|
||||
json:
|
||||
sign: ${get_sign($device_sn, $os_platform, $app_version)}
|
||||
method: POST
|
||||
url: /api/get-token
|
||||
extract:
|
||||
token: content.token
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
- eq: [headers.Content-Type, application/json]
|
||||
- eq: [content.success, true]
|
||||
```
|
||||
</details>
|
||||
|
||||
如果我们需要使用 device_sn、app_version 和 os_platform 这三个参数来进行参数化数据驱动，那么就可以创建一个 [testsuite](/data/demo-parameters-get-token.yml)，并且进行参数化配置。
|
||||
|
||||
```yaml
|
||||
config:
|
||||
name: get token with parameters
|
||||
|
||||
testcases:
|
||||
get token with $user_agent, $app_version, $os_platform:
|
||||
testcase: demo-testcase-get-token.yml
|
||||
parameters:
|
||||
user_agent: ["iOS/10.1", "iOS/10.2", "iOS/10.3"]
|
||||
app_version: ${P(app_version.csv)}
|
||||
os_platform: ${get_os_platform()}
|
||||
```
|
||||
|
||||
其中，`user_agent` 使用了直接指定参数列表的形式。
|
||||
|
||||
[app_version](/data/app_version.csv) 通过 CSV 文件进行参数配置，对应的文件内容为：
|
||||
|
||||
```csv
|
||||
app_version
|
||||
2.8.5
|
||||
2.8.6
|
||||
```
|
||||
|
||||
os_platform 使用自定义函数的形式生成参数列表，对应的函数内容为：
|
||||
|
||||
```python
|
||||
def get_os_platform():
|
||||
return [
|
||||
{"os_platform": "ios"},
|
||||
{"os_platform": "android"}
|
||||
]
|
||||
```
|
||||
|
||||
那么，经过笛卡尔积组合，应该总共有 `3*2*2=12` 种参数组合情况。
|
||||
|
||||
|
||||
<details>
|
||||
<summary>点击查看完整运行日志</summary>
|
||||
```text
|
||||
$ hrun docs/data/demo-parameters-get-token.yml
|
||||
INFO Start to run testcase: get token with iOS/10.1, 2.8.5, ios
|
||||
get token with PBJda7SXM2ReWlu, ios, 2.8.5
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
INFO status_code: 200, response_time(ms): 10.66 ms, response_length: 46 bytes
|
||||
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 1 test in 0.026s
|
||||
|
||||
OK
|
||||
INFO Start to run testcase: get token with iOS/10.1, 2.8.5, android
|
||||
get token with PBJda7SXM2ReWlu, android, 2.8.5
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
INFO status_code: 200, response_time(ms): 3.03 ms, response_length: 46 bytes
|
||||
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 1 test in 0.004s
|
||||
|
||||
OK
|
||||
INFO Start to run testcase: get token with iOS/10.1, 2.8.6, ios
|
||||
get token with PBJda7SXM2ReWlu, ios, 2.8.6
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
INFO status_code: 200, response_time(ms): 10.76 ms, response_length: 46 bytes
|
||||
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 1 test in 0.012s
|
||||
|
||||
OK
|
||||
INFO Start to run testcase: get token with iOS/10.1, 2.8.6, android
|
||||
get token with PBJda7SXM2ReWlu, android, 2.8.6
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
INFO status_code: 200, response_time(ms): 4.49 ms, response_length: 46 bytes
|
||||
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 1 test in 0.006s
|
||||
|
||||
OK
|
||||
INFO Start to run testcase: get token with iOS/10.2, 2.8.5, ios
|
||||
get token with PBJda7SXM2ReWlu, ios, 2.8.5
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
INFO status_code: 200, response_time(ms): 4.39 ms, response_length: 46 bytes
|
||||
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 1 test in 0.006s
|
||||
|
||||
OK
|
||||
INFO Start to run testcase: get token with iOS/10.2, 2.8.5, android
|
||||
get token with PBJda7SXM2ReWlu, android, 2.8.5
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
INFO status_code: 200, response_time(ms): 4.04 ms, response_length: 46 bytes
|
||||
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 1 test in 0.005s
|
||||
|
||||
OK
|
||||
INFO Start to run testcase: get token with iOS/10.2, 2.8.6, ios
|
||||
get token with PBJda7SXM2ReWlu, ios, 2.8.6
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
INFO status_code: 200, response_time(ms): 3.44 ms, response_length: 46 bytes
|
||||
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 1 test in 0.004s
|
||||
|
||||
OK
|
||||
INFO Start to run testcase: get token with iOS/10.2, 2.8.6, android
|
||||
get token with PBJda7SXM2ReWlu, android, 2.8.6
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
INFO status_code: 200, response_time(ms): 4.03 ms, response_length: 46 bytes
|
||||
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 1 test in 0.005s
|
||||
|
||||
OK
|
||||
INFO Start to run testcase: get token with iOS/10.3, 2.8.5, ios
|
||||
get token with PBJda7SXM2ReWlu, ios, 2.8.5
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
INFO status_code: 200, response_time(ms): 5.14 ms, response_length: 46 bytes
|
||||
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 1 test in 0.008s
|
||||
|
||||
OK
|
||||
INFO Start to run testcase: get token with iOS/10.3, 2.8.5, android
|
||||
get token with PBJda7SXM2ReWlu, android, 2.8.5
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
INFO status_code: 200, response_time(ms): 7.62 ms, response_length: 46 bytes
|
||||
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 1 test in 0.010s
|
||||
|
||||
OK
|
||||
INFO Start to run testcase: get token with iOS/10.3, 2.8.6, ios
|
||||
get token with PBJda7SXM2ReWlu, ios, 2.8.6
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
INFO status_code: 200, response_time(ms): 4.88 ms, response_length: 46 bytes
|
||||
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 1 test in 0.006s
|
||||
|
||||
OK
|
||||
INFO Start to run testcase: get token with iOS/10.3, 2.8.6, android
|
||||
get token with PBJda7SXM2ReWlu, android, 2.8.6
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
INFO status_code: 200, response_time(ms): 5.41 ms, response_length: 46 bytes
|
||||
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 1 test in 0.008s
|
||||
|
||||
OK
|
||||
INFO Start to render Html report ...
|
||||
INFO Generated Html report: /Users/debugtalk/MyProjects/HttpRunner-dev/httprunner-docs-v2x/reports/1551950193.html
|
||||
```
|
||||
</details>
|
||||
@@ -1,30 +0,0 @@
|
||||
|
||||
## 文件类型说明
|
||||
|
||||
在 HttpRunner 自动化测试项目中，主要存在如下几类文件：
|
||||
|
||||
- `YAML/JSON`（必须）：测试用例文件，存储接口测试相关信息
|
||||
- `debugtalk.py`（可选）：存储项目中逻辑运算辅助函数
|
||||
- 该文件存在时，将作为项目根目录定位标记，其所在目录即被视为项目工程根目录
|
||||
- 该文件不存在时，运行测试的所在路径（`CWD`）将被视为项目工程根目录
|
||||
- 测试用例文件中的相对路径（例如`.csv`）均需基于项目工程根目录
|
||||
- 运行测试后，测试报告文件夹（`reports`）会生成在项目工程根目录
|
||||
- `.env`（可选）：存储项目环境变量，通常用于存储项目敏感信息
|
||||
- `.csv`（可选）：项目数据文件，用于进行数据驱动
|
||||
- `reports`：默认生成测试报告的存储文件夹
|
||||
|
||||
## 项目文件结构
|
||||
|
||||
对于接口数比较少，或者测试场景比较简单的项目，组织测试用例时无需分层。在此种情况下，项目文件的目录结构没有任何要求，在项目中只需要一堆 `YAML/JSON` 文件即可，每一个文件单独对应一条测试用例；根据需要，项目中可能还会有 `debugtalk.py`、`.env`等文件。
|
||||
|
||||
推荐的项目文件目录结构示例如下：
|
||||
|
||||
```bash
|
||||
$ tree demo -a
|
||||
demo
|
||||
├── .env
|
||||
├── debugtalk.py
|
||||
├── reports
|
||||
├── testcase1.yml
|
||||
└── testcase2.json
|
||||
```
|
||||
@@ -1,38 +0,0 @@
|
||||
|
||||
为了简化测试用例的编写工作，HttpRunner 实现了测试用例生成的功能，对应的转换工具为一个独立的项目：[har2case][har2case]。
|
||||
|
||||
简单来说，就是当前主流的抓包工具和浏览器都支持将抓取得到的数据包导出为标准通用的 HAR 格式（HTTP Archive），然后 HttpRunner 实现了将 HAR 格式的数据包转换为`YAML/JSON`格式的测试用例文件的功能。
|
||||
|
||||
## 获取 HAR 数据包
|
||||
|
||||
在转换生成测试用例之前，需要先将抓取得到的数据包导出为 HAR 格式的文件。在`Charles Proxy`中的操作方式为，选中需要转换的接口（可多选或全选），点击右键，在悬浮的菜单目录中点击【Export...】，格式选择`HTTP Archive(.har)`后保存即可；假设我们保存的文件名称为 demo.har。
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## 转换生成测试用例
|
||||
|
||||
然后，在命令行终端中运行 har2case 命令，即可将 demo.har 转换为 HttpRunner 的测试用例文件。
|
||||
|
||||
使用 `har2case` 转换脚本时默认转换为 JSON 格式。
|
||||
|
||||
```bash
|
||||
$ har2case docs/data/demo-quickstart.har
|
||||
INFO:root:Start to generate testcase.
|
||||
INFO:root:dump testcase to JSON format.
|
||||
INFO:root:Generate JSON testcase successfully: docs/data/demo-quickstart.json
|
||||
```
|
||||
|
||||
加上 `-2y`/`--to-yml` 参数后转换为 YAML 格式。
|
||||
|
||||
```bash
|
||||
$ har2case docs/data/demo-quickstart.har -2y
|
||||
INFO:root:Start to generate testcase.
|
||||
INFO:root:dump testcase to YAML format.
|
||||
INFO:root:Generate YAML testcase successfully: docs/data/demo-quickstart.yml
|
||||
```
|
||||
|
||||
两种格式完全等价，YAML 格式更简洁，JSON 格式支持的工具更丰富，大家可根据个人喜好进行选择。
|
||||
|
||||
[har2case]: https://github.com/HttpRunner/har2case
|
||||
@@ -1,145 +0,0 @@
|
||||
## 概述
|
||||
|
||||
HttpRunner 从 `1.4.5` 版本开始实现了全新的 hook 机制，可以在请求前和请求后调用钩子函数。
|
||||
|
||||
## 调用 hook 函数
|
||||
|
||||
hook 机制分为两个层级：
|
||||
|
||||
- 测试用例层面（testcase）
|
||||
- 测试步骤层面（teststep）
|
||||
|
||||
### 测试用例层面（testcase）
|
||||
|
||||
在 YAML/JSON 测试用例的 `config` 中新增关键字 `setup_hooks` 和 `teardown_hooks`。
|
||||
|
||||
- setup_hooks: 在整个用例开始执行前触发 hook 函数，主要用于准备工作。
|
||||
- teardown_hooks: 在整个用例结束执行后触发 hook 函数，主要用于测试后的清理工作。
|
||||
|
||||
```yaml
|
||||
- config:
|
||||
name: basic test with httpbin
|
||||
request:
|
||||
base_url: http://127.0.0.1:3458/
|
||||
setup_hooks:
|
||||
- ${hook_print(setup)}
|
||||
teardown_hooks:
|
||||
- ${hook_print(teardown)}
|
||||
```
|
||||
|
||||
### 测试步骤层面（teststep）
|
||||
|
||||
在 YAML/JSON 测试步骤的 `test` 中新增关键字 `setup_hooks` 和 `teardown_hooks`。
|
||||
|
||||
- setup_hooks: 在 HTTP 请求发送前执行 hook 函数，主要用于准备工作；也可以实现对请求的 request 内容进行预处理。
|
||||
- teardown_hooks: 在 HTTP 请求发送后执行 hook 函数，主要用于测试后的清理工作；也可以实现对响应的 response 进行修改，例如进行加解密等处理。
|
||||
|
||||
```json
|
||||
"test": {
|
||||
"name": "get token with $user_agent, $os_platform, $app_version",
|
||||
"request": {
|
||||
"url": "/api/get-token",
|
||||
"method": "POST",
|
||||
"headers": {
|
||||
"app_version": "$app_version",
|
||||
"os_platform": "$os_platform",
|
||||
"user_agent": "$user_agent"
|
||||
},
|
||||
"json": {
|
||||
"sign": "${get_sign($user_agent, $device_sn, $os_platform, $app_version)}"
|
||||
}
|
||||
},
|
||||
"validate": [
|
||||
{"eq": ["status_code", 200]}
|
||||
],
|
||||
"setup_hooks": [
|
||||
"${setup_hook_prepare_kwargs($request)}",
|
||||
"${setup_hook_httpntlmauth($request)}"
|
||||
],
|
||||
"teardown_hooks": [
|
||||
"${teardown_hook_sleep_N_secs($response, 2)}"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## 编写 hook 函数
|
||||
|
||||
hook 函数的定义放置在项目的 `debugtalk.py` 中，在 YAML/JSON 中调用 hook 函数仍然是采用 `${func($a, $b)}` 的形式。
|
||||
|
||||
对于测试用例层面的 hook 函数，与 YAML/JSON 中自定义的函数完全相同，可通过自定义参数传参的形式来实现灵活应用。
|
||||
|
||||
```python
|
||||
def hook_print(msg):
|
||||
print(msg)
|
||||
```
|
||||
|
||||
对于单个测试用例层面的 hook 函数，除了可传入自定义参数外，还可以传入与当前测试用例相关的信息，包括请求的 `$request` 和响应的 `$response`，用于实现更复杂场景的灵活应用。
|
||||
|
||||
### setup_hooks
|
||||
|
||||
在测试步骤层面的 setup_hooks 函数中，除了可传入自定义参数外，还可以传入 `$request`，该参数对应着当前测试步骤 request 的全部内容。因为 request 是可变参数类型（dict），因此该函数参数为引用传递，当我们需要对请求参数进行预处理时尤其有用。
|
||||
|
||||
e.g.
|
||||
|
||||
```python
|
||||
def setup_hook_prepare_kwargs(request):
|
||||
if request["method"] == "POST":
|
||||
content_type = request.get("headers", {}).get("content-type")
|
||||
if content_type and "data" in request:
|
||||
# if request content-type is application/json, request data should be dumped
|
||||
if content_type.startswith("application/json") and isinstance(request["data"], (dict, list)):
|
||||
request["data"] = json.dumps(request["data"])
|
||||
|
||||
if isinstance(request["data"], str):
|
||||
request["data"] = request["data"].encode('utf-8')
|
||||
|
||||
def setup_hook_httpntlmauth(request):
|
||||
if "httpntlmauth" in request:
|
||||
from requests_ntlm import HttpNtlmAuth
|
||||
auth_account = request.pop("httpntlmauth")
|
||||
request["auth"] = HttpNtlmAuth(
|
||||
auth_account["username"], auth_account["password"])
|
||||
```
|
||||
|
||||
通过上述的 `setup_hook_prepare_kwargs` 函数，可以实现根据请求方法和请求的 Content-Type 来对请求的 data 进行加工处理；通过 `setup_hook_httpntlmauth` 函数，可以实现 HttpNtlmAuth 权限授权。
|
||||
|
||||
### teardown_hooks
|
||||
|
||||
在测试步骤层面的 teardown_hooks 函数中，除了可传入自定义参数外，还可以传入 `$response`，该参数对应着当前请求的响应实例（requests.Response）。
|
||||
|
||||
e.g.
|
||||
|
||||
```python
|
||||
def teardown_hook_sleep_N_secs(response, n_secs):
|
||||
""" sleep n seconds after request
|
||||
"""
|
||||
if response.status_code == 200:
|
||||
time.sleep(0.1)
|
||||
else:
|
||||
time.sleep(n_secs)
|
||||
```
|
||||
|
||||
通过上述的 `teardown_hook_sleep_N_secs` 函数，可以根据接口响应的状态码来进行不同时间的延迟等待。
|
||||
|
||||
另外，在 teardown_hooks 函数中还可以对 response 进行修改。当我们需要先对响应内容进行处理（例如加解密、参数运算），再进行参数提取（extract）和校验（validate）时尤其有用。
|
||||
|
||||
例如在下面的测试步骤中，在执行测试后，通过 teardown_hooks 函数将响应结果的状态码和 headers 进行了修改，然后再进行了校验。
|
||||
|
||||
```yaml
|
||||
- test:
|
||||
name: alter response
|
||||
request:
|
||||
url: /headers
|
||||
method: GET
|
||||
teardown_hooks:
|
||||
- ${alter_response($response)}
|
||||
validate:
|
||||
- eq: ["status_code", 500]
|
||||
- eq: ["headers.content-type", "html/text"]
|
||||
```
|
||||
|
||||
```python
|
||||
def alter_response(response):
|
||||
response.status_code = 500
|
||||
response.headers["Content-Type"] = "html/text"
|
||||
```
|
||||
@@ -1,19 +0,0 @@
|
||||
|
||||
## 背景
|
||||
|
||||
很多时候项目代码在运行时需要使用到账号、密码、key等敏感数据信息，但是从信息安全的角度考虑，我们是不能将这些敏感数据提交到代码仓库的，主要原因有两个：
|
||||
|
||||
- 加强权限管控：参与项目的开发人员可能会有很多，大家都有读取代码仓库的权限，但是像 key 这类极度敏感的信息不应该所有人都有权限获取；
|
||||
- 减少代码泄漏的危害性：假如代码出现泄漏，敏感数据信息不应该也同时泄漏。
|
||||
|
||||
## 解决方案
|
||||
|
||||
那代码部署到服务器或 Jenkins 执行机后，运行时要使用到这些敏感数据信息，该怎么操作呢？
|
||||
|
||||
推荐的操作方式为：
|
||||
|
||||
- 对服务器进行权限管控，只有运维人员（或者核心开发人员）才有登录服务器的权限；
|
||||
- 运维人员（或者核心开发人员）：在运行的机器上将敏感数据设置到系统的环境变量中；
|
||||
- 普通开发人员：只需要知道敏感信息的变量名称，在代码中通过读取环境变量的方式获取敏感数据。
|
||||
|
||||
存储敏感数据（设置环境变量）和使用敏感数据（引用环境变量）的具体方法，可参考[环境变量](/prepare/dot-env/)使用说明文档。
|
||||
@@ -1,272 +0,0 @@
|
||||
|
||||
|
||||
## 测试用例分层模型
|
||||
|
||||
在自动化测试领域，自动化测试用例的可维护性是极其重要的因素，直接关系到自动化测试能否持续有效地在项目中开展。
|
||||
|
||||
概括来说，测试用例分层机制的核心是将接口定义、测试步骤、测试用例、测试场景进行分离，单独进行描述和维护，从而尽可能地减少自动化测试用例的维护成本。
|
||||
|
||||
逻辑关系图如下所示：
|
||||
|
||||
|
||||
|
||||
同时，强调如下几点核心概念：
|
||||
|
||||
- 测试用例（testcase）应该是完整且独立的，每条测试用例应该是都可以独立运行的
|
||||
- 测试用例是测试步骤（teststep）的 `有序` 集合，每一个测试步骤对应一个 API 的请求描述
|
||||
- 测试用例集（testsuite）是测试用例的 `无序` 集合，集合中的测试用例应该都是相互独立，不存在先后依赖关系的；如果确实存在先后依赖关系，那就需要在测试用例中完成依赖的处理
|
||||
|
||||
如果对于上述第三点感觉难以理解，不妨看下上图中的示例：
|
||||
|
||||
- testcase1 依赖于 testcase2，那么就可以在测试步骤（teststep12）中对 testcase2 进行引用，然后 testcase1 就是完整且可独立运行的；
|
||||
- 在 testsuite 中，testcase1 与 testcase2 相互独立，运行顺序就不再有先后依赖关系了。
|
||||
|
||||
## 分层描述详解
|
||||
|
||||
理解了测试用例分层模型，接下来我们再来看下在分层模型下，接口、测试用例、测试用例集的描述形式。
|
||||
|
||||
### 接口定义（API）
|
||||
|
||||
为了更好地对接口描述进行管理，推荐使用独立的文件对接口描述进行存储，即每个文件对应一个接口描述。
|
||||
|
||||
接口定义描述的主要内容包括：**name**、variables、**request**、base_url、validate 等，形式如下：
|
||||
|
||||
```yaml
|
||||
name: get headers
|
||||
base_url: http://httpbin.org
|
||||
variables:
|
||||
expected_status_code: 200
|
||||
request:
|
||||
url: /headers
|
||||
method: GET
|
||||
validate:
|
||||
- eq: ["status_code", $expected_status_code]
|
||||
- eq: [content.headers.Host, "httpbin.org"]
|
||||
```
|
||||
|
||||
其中，name 和 request 部分是必须的，request 中的描述形式与 [requests.request](http://docs.python-requests.org/en/master/api/) 完全相同。
|
||||
|
||||
另外，API 描述需要尽量保持完整，做到可以单独运行。如果在接口描述中存在变量引用的情况，可在 variables 中对参数进行定义。通过这种方式，可以很好地实现单个接口的调试。
|
||||
|
||||
```bash
|
||||
$ hrun api/get_headers.yml
|
||||
INFO Start to run testcase: get headers
|
||||
headers
|
||||
INFO GET http://httpbin.org/headers
|
||||
INFO status_code: 200, response_time(ms): 477.32 ms, response_length: 157 bytes
|
||||
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 1 test in 0.478s
|
||||
|
||||
OK
|
||||
```
|
||||
|
||||
### 测试用例（testcase）
|
||||
|
||||
#### 引用接口定义
|
||||
|
||||
有了接口的定义描述后，我们编写测试场景时就可以直接引用接口定义了。
|
||||
|
||||
在测试步骤（teststep）中，可通过 `api` 字段引用接口定义，引用方式为对应 API 文件的路径，绝对路径或相对路径均可。推荐使用相对路径，路径基准为项目根目录，即 `debugtalk.py` 所在的目录路径。
|
||||
|
||||
```yaml
|
||||
- config:
|
||||
name: "setup and reset all."
|
||||
variables:
|
||||
user_agent: 'iOS/10.3'
|
||||
device_sn: "TESTCASE_SETUP_XXX"
|
||||
os_platform: 'ios'
|
||||
app_version: '2.8.6'
|
||||
base_url: "http://127.0.0.1:5000"
|
||||
verify: False
|
||||
output:
|
||||
- session_token
|
||||
|
||||
- test:
|
||||
name: get token (setup)
|
||||
api: api/get_token.yml
|
||||
variables:
|
||||
user_agent: 'iOS/10.3'
|
||||
device_sn: $device_sn
|
||||
os_platform: 'ios'
|
||||
app_version: '2.8.6'
|
||||
extract:
|
||||
- session_token: content.token
|
||||
validate:
|
||||
- eq: ["status_code", 200]
|
||||
- len_eq: ["content.token", 16]
|
||||
|
||||
- test:
|
||||
name: reset all users
|
||||
api: api/reset_all.yml
|
||||
variables:
|
||||
token: $session_token
|
||||
```
|
||||
|
||||
若需要控制或改变接口定义中的参数值，可在测试步骤中指定 variables 参数，覆盖 API 中的 variables 实现。
|
||||
|
||||
同样地，在测试步骤中定义 validate 后，也会与 API 中的 validate 合并覆盖。因此推荐的做法是，在 API 定义中的 validate 只描述最基本的校验项，例如 status_code，对于与业务逻辑相关的更多校验项，在测试步骤的 validate 中进行描述。
|
||||
|
||||
#### 引用测试用例
|
||||
|
||||
在测试用例的测试步骤中，除了可以引用接口定义，还可以引用其它测试用例。通过这种方式，可以在避免重复描述的同时，解决测试用例的依赖关系，从而保证每个测试用例都是独立可运行的。
|
||||
|
||||
在测试步骤（teststep）中，可通过 `testcase` 字段引用其它测试用例，引用方式为对应测试用例文件的路径，绝对路径或相对路径均可。推荐使用相对路径，路径基准为项目根目录，即 `debugtalk.py` 所在的目录路径。
|
||||
|
||||
例如，在上面的测试用例（"setup and reset all."）中，实现了对获取 token 功能的测试；同时，在很多其它功能中都会依赖于获取 token 的功能，如果将该功能的测试步骤脚本拷贝到其它功能的测试用例中，那么就会存在大量重复，当需要对该部分进行修改时就需要修改所有地方，显然不便于维护。
|
||||
|
||||
比较好的做法是，在其它功能的测试用例（如创建用户）中，引用获取 token 功能的测试用例（testcases/setup.yml）作为一个测试步骤，从而创建用户（"create user and check result."）这个测试用例也变得独立可运行了。
|
||||
|
||||
```yaml
|
||||
- config:
|
||||
name: "create user and check result."
|
||||
id: create_user
|
||||
base_url: "http://127.0.0.1:5000"
|
||||
variables:
|
||||
uid: 9001
|
||||
device_sn: "TESTCASE_CREATE_XXX"
|
||||
output:
|
||||
- session_token
|
||||
|
||||
- test:
|
||||
name: setup and reset all (override) for $device_sn.
|
||||
testcase: testcases/setup.yml
|
||||
output:
|
||||
- session_token
|
||||
|
||||
- test:
|
||||
name: create user and check result.
|
||||
variables:
|
||||
token: $session_token
|
||||
testcase: testcases/deps/check_and_create.yml
|
||||
```
|
||||
|
||||
### 测试用例集（testsuite）
|
||||
|
||||
当测试用例数量比较多以后，为了方便管理和实现批量运行，通常需要使用测试用例集来对测试用例进行组织。
|
||||
|
||||
在前文的测试用例分层模型中也强调了，测试用例集（testsuite）是测试用例的 `无序` 集合，集合中的测试用例应该都是相互独立，不存在先后依赖关系的；如果确实存在先后依赖关系，那就需要在测试用例中完成依赖的处理。
|
||||
|
||||
因为是 `无序` 集合，因此测试用例集的描述形式会与测试用例有些不同，在每个测试用例集文件中，第一层级存在两类字段：
|
||||
|
||||
- config: 测试用例集的总体配置参数
|
||||
- testcases: 值为字典结构（无序），key 为测试用例的名称，value 为测试用例的内容；在引用测试用例时也可以指定 variables，实现对引用测试用例中 variables 的覆盖。
|
||||
|
||||
#### 非参数化场景
|
||||
|
||||
```yaml
|
||||
config:
|
||||
name: create users with uid
|
||||
variables:
|
||||
device_sn: ${gen_random_string(15)}
|
||||
var_a: ${gen_random_string(5)}
|
||||
var_b: $var_a
|
||||
base_url: "http://127.0.0.1:5000"
|
||||
|
||||
testcases:
|
||||
create user 1000 and check result.:
|
||||
testcase: testcases/create_user.yml
|
||||
variables:
|
||||
uid: 1000
|
||||
var_c: ${gen_random_string(5)}
|
||||
var_d: $var_c
|
||||
|
||||
create user 1001 and check result.:
|
||||
testcase: testcases/create_user.yml
|
||||
variables:
|
||||
uid: 1001
|
||||
var_c: ${gen_random_string(5)}
|
||||
var_d: $var_c
|
||||
```
|
||||
|
||||
#### 参数化场景（parameters）
|
||||
|
||||
对于参数化场景，可通过 parameters 实现，描述形式如下所示。
|
||||
|
||||
```yaml
|
||||
config:
|
||||
name: create users with parameters
|
||||
variables:
|
||||
device_sn: ${gen_random_string(15)}
|
||||
base_url: "http://127.0.0.1:5000"
|
||||
|
||||
testcases:
|
||||
create user $uid and check result for $device_sn.:
|
||||
testcase: testcases/create_user.yml
|
||||
variables:
|
||||
uid: 1000
|
||||
device_sn: TESTSUITE_XXX
|
||||
parameters:
|
||||
uid: [101, 102, 103]
|
||||
device_sn: [TESTSUITE_X1, TESTSUITE_X2]
|
||||
```
|
||||
|
||||
参数化后，parameters 中的变量将采用笛卡尔积组合形成参数列表，依次覆盖 variables 中的参数，驱动测试用例的运行。
|
||||
|
||||
## 文件目录结构管理 && 脚手架工具
|
||||
|
||||
在对测试用例文件进行组织管理时，对于文件的存储位置均没有要求和限制，在引用时只需要指定对应的文件路径即可。但从约定大于配置的角度，最好是按照推荐的文件夹名称进行存储管理，并可通过子目录实现项目模块分类管理。
|
||||
|
||||
推荐的方式汇总如下：
|
||||
|
||||
- `debugtalk.py` 放置在项目根目录下，假设为 `PRJ_ROOT_DIR`
|
||||
- `.env` 放置在项目根目录下，路径为 `PRJ_ROOT_DIR/.env`
|
||||
- 接口定义（API）放置在 `PRJ_ROOT_DIR/api/` 目录下
|
||||
- 测试用例（testcase）放置在 `PRJ_ROOT_DIR/testcases/` 目录下
|
||||
- 测试用例集（testsuite）文件必须放置在 `PRJ_ROOT_DIR/testsuites/` 目录下
|
||||
- data 文件夹：存储参数化文件，或者项目依赖的文件，路径为 `PRJ_ROOT_DIR/data/`
|
||||
- reports 文件夹：存储 HTML 测试报告，生成路径为 `PRJ_ROOT_DIR/reports/`
|
||||
|
||||
目录结构如下所示：
|
||||
|
||||
```bash
|
||||
$ tree tests
|
||||
tests
|
||||
├── .env
|
||||
├── data
|
||||
│ ├── app_version.csv
|
||||
│ └── account.csv
|
||||
├── api
|
||||
│ ├── create_user.yml
|
||||
│ ├── get_headers.yml
|
||||
│ ├── get_token.yml
|
||||
│ ├── get_user.yml
|
||||
│ └── reset_all.yml
|
||||
├── debugtalk.py
|
||||
├── testcases
|
||||
│ ├── create_user.yml
|
||||
│ ├── deps
|
||||
│ │ └── check_and_create.yml
|
||||
│ └── setup.yml
|
||||
└── testsuites
|
||||
├── create_users.yml
|
||||
└── create_users_with_parameters.yml
|
||||
```
|
||||
|
||||
**项目脚手架**
|
||||
|
||||
同时，在 `HttpRunner` 中实现了一个脚手架工具，可以快速创建项目的目录结构。该想法来源于 `Django` 的 `django-admin.py startproject project_name`。
|
||||
|
||||
使用方式也与 `Django` 类似，只需要通过 `--startproject` 指定新项目的名称即可。
|
||||
|
||||
```bash
|
||||
$ hrun --startproject demo
|
||||
Start to create new project: demo
|
||||
CWD: /Users/debugtalk/MyProjects/examples
|
||||
|
||||
created folder: demo
|
||||
created folder: demo/api
|
||||
created folder: demo/testcases
|
||||
created folder: demo/testsuites
|
||||
created folder: demo/reports
|
||||
created file: demo/debugtalk.py
|
||||
created file: demo/.env
|
||||
```
|
||||
|
||||
|
||||
## 相关参考
|
||||
|
||||
- [《HttpRunner 的测试用例分层机制（已过期）》](/post/HttpRunner-testcase-layer)
|
||||
- 测试用例分层详细示例：[HttpRunner/tests](https://github.com/HttpRunner/HttpRunner/tree/master/tests)
|
||||
@@ -1,178 +0,0 @@
|
||||
|
||||
## YAML & JSON
|
||||
|
||||
HttpRunner 的测试用例支持两种文件格式：YAML 和 JSON。
|
||||
|
||||
JSON 和 YAML 格式的测试用例完全等价，包含的信息内容也完全相同。
|
||||
|
||||
- 对于新手来说，推荐使用 JSON 格式，虽然描述形式上稍显累赘，但是不容易出错（大多编辑器都具有 JSON 格式的检测功能）；同时，HttpRunner 也内置了 JSON 格式正确性检测和样式美化功能，详情可查看[《Validate & Prettify》](/testcase//validate-pretty.md)。
|
||||
- 对于熟悉 YAML 格式的人来说，编写维护 YAML 格式的测试用例会更简洁，但前提是要保证 YAML 格式没有语法错误。
|
||||
|
||||
对于两种格式的展示差异，可以对比查看 [demo-quickstart-6.json](/data/demo-quickstart-6.json) 和 [demo-quickstart-6.yml](/data/demo-quickstart-6.yml) 获取初步的印象。
|
||||
|
||||
后面为了更清晰的描述，统一采用 JSON 格式作为示例。
|
||||
|
||||
## 测试用例结构
|
||||
|
||||
|
||||
|
||||
在 HttpRunner 中，测试用例组织主要基于三个概念：
|
||||
|
||||
- 测试用例集（testsuite）：对应一个文件夹，包含单个或多个测试用例（`YAML/JSON`）文件
|
||||
- 测试用例（testcase）：对应一个 `YAML/JSON` 文件，包含单个或多个测试步骤
|
||||
- 测试步骤（teststep）：对应 `YAML/JSON` 文件中的一个 `test`，描述单次接口测试的全部内容，包括发起接口请求、解析响应结果、校验结果等
|
||||
|
||||
对于单个 `YAML/JSON` 文件来说，数据存储结构为 `list of dict` 的形式，其中可能包含一个全局配置项（config）和若干个测试步骤（test）。
|
||||
|
||||
具体地：
|
||||
|
||||
- config：作为整个测试用例的全局配置项
|
||||
- test：对应单个测试步骤（teststep），测试用例存在顺序关系，运行时将从前往后依次运行各个测试步骤
|
||||
|
||||
|
||||
对应的 JSON 格式如下所示：
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"config": {...}
|
||||
},
|
||||
{
|
||||
"test": {...}
|
||||
},
|
||||
{
|
||||
"test": {...}
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
## 变量空间（context）作用域
|
||||
|
||||
在测试用例内部，HttpRunner 划分了两层变量空间作用域（context）。
|
||||
|
||||
- config：作为整个测试用例的全局配置项，作用域为整个测试用例；
|
||||
- test：测试步骤的变量空间（context）会继承或覆盖 config 中定义的内容；
|
||||
- 若某变量在 config 中定义了，在某 test 中没有定义，则该 test 会继承该变量
|
||||
- 若某变量在 config 和某 test 中都定义了，则该 test 中使用自己定义的变量值
|
||||
- 各个测试步骤（test）的变量空间相互独立，互不影响；
|
||||
- 如需在多个测试步骤（test）中传递参数值，则需要使用 extract 关键字，并且只能从前往后传递
|
||||
|
||||
|
||||
|
||||
|
||||
## config
|
||||
|
||||
```json
|
||||
"config": {
|
||||
"name": "testcase description",
|
||||
"parameters": [
|
||||
{"user_agent": ["iOS/10.1", "iOS/10.2", "iOS/10.3"]},
|
||||
{"app_version": "${P(app_version.csv)}"},
|
||||
{"os_platform": "${get_os_platform()}"}
|
||||
],
|
||||
"variables": [
|
||||
{"user_agent": "iOS/10.3"},
|
||||
{"device_sn": "${gen_random_string(15)}"},
|
||||
{"os_platform": "ios"}
|
||||
],
|
||||
"request": {
|
||||
"base_url": "http://127.0.0.1:5000",
|
||||
"headers": {
|
||||
"Content-Type": "application/json",
|
||||
"device_sn": "$device_sn"
|
||||
}
|
||||
},
|
||||
"output": [
|
||||
"token"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Key | required? | format | descrption
|
||||
--- | --------- | ------ | ----------
|
||||
name | Yes | string | 测试用例的名称，在测试报告中将作为标题
|
||||
variables | No | list of dict | 定义的全局变量，作用域为整个用例
|
||||
parameters | No | list of dict | 全局参数，用于实现数据化驱动，作用域为整个用例
|
||||
request | No | dict | request 的公共参数，作用域为整个用例；常用参数包括 base_url 和 headers
|
||||
|
||||
**request**
|
||||
|
||||
Key | required? | format | descrption
|
||||
---------|----------|---------|---------
|
||||
base_url | No | string | 测试用例请求 URL 的公共 host，指定该参数后，test 中的 url 可以只描述 path 部分
|
||||
headers | No | dict | request 中 headers 的公共参数，作用域为整个用例
|
||||
output | No | list | 整个用例输出的参数列表，可输出的参数包括公共的 variable 和 extract 的参数; 在 log-level 为 debug 模式下，会在 terminal 中打印出参数内容
|
||||
|
||||
## test
|
||||
|
||||
```json
|
||||
"test": {
|
||||
"name": "get token with $user_agent, $os_platform, $app_version",
|
||||
"request": {
|
||||
"url": "/api/get-token",
|
||||
"method": "POST",
|
||||
"headers": {
|
||||
"app_version": "$app_version",
|
||||
"os_platform": "$os_platform",
|
||||
"user_agent": "$user_agent"
|
||||
},
|
||||
"json": {
|
||||
"sign": "${get_sign($user_agent, $device_sn, $os_platform, $app_version)}"
|
||||
},
|
||||
"extract": [
|
||||
{"token": "content.token"}
|
||||
],
|
||||
"validate": [
|
||||
{"eq": ["status_code", 200]},
|
||||
{"eq": ["headers.Content-Type", "application/json"]},
|
||||
{"eq": ["content.success", true]}
|
||||
],
|
||||
"setup_hooks": [],
|
||||
"teardown_hooks": []
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Key | required? | format | descrption
|
||||
--- | --------- | ------ | ----------
|
||||
name | Yes | string | 测试步骤的名称，在测试报告中将作为测试步骤的名称
|
||||
request | Yes | dict | HTTP 请求的详细内容；可用参数详见 [python-requests][1] 官方文档
|
||||
variables | No | list of dict | 测试步骤中定义的变量，作用域为当前测试步骤
|
||||
extract | No | list | 从当前 HTTP 请求的响应结果中提取参数，并保存到参数变量中（例如`token`），后续测试用例可通过`$token`的形式进行引用
|
||||
validate | No | list | 测试用例中定义的结果校验项，作用域为当前测试用例，用于实现对当前测试用例运行结果的校验
|
||||
setup_hooks | No | list | 在 HTTP 请求发送前执行 hook 函数，主要用于准备工作
|
||||
teardown_hooks | No | list | 在 HTTP 请求发送后执行 hook 函数，主要用户测试后的清理工作
|
||||
|
||||
**extract**
|
||||
|
||||
支持多种提取方式：
|
||||
|
||||
- 响应结果为 JSON 结构，可采用`.`运算符的方式，例如`headers.Content-Type`、`content.success`；
|
||||
- 响应结果为 text/html 结构，可采用正则表达式的方式，例如`blog-motto\">(.*)</h2>`
|
||||
- 详情可阅读[《ApiTestEngine，不再局限于API的测试》][2]
|
||||
|
||||
**validate**
|
||||
|
||||
支持两种格式：
|
||||
|
||||
- `{"comparator_name": [check_item, expect_value]}`
|
||||
- `{"check": check_item, "comparator": comparator_name, "expect": expect_value}`
|
||||
|
||||
**hooks**
|
||||
|
||||
setup_hooks 函数放置于 debugtalk.py 中，并且必须包含三个参数：
|
||||
|
||||
- method: 请求方法，e.g. GET, POST, PUT
|
||||
- url: 请求 URL
|
||||
- kwargs: request 的参数字典
|
||||
|
||||
teardown_hooks 函数放置于 debugtalk.py 中，并且必须包含一个参数：
|
||||
|
||||
- resp_obj: requests.Response 实例
|
||||
|
||||
关于 `setup_hooks` 和 `teardown_hooks` 的更多内容，请参考[《hook 机制》][3]。
|
||||
|
||||
|
||||
[1]: http://docs.python-requests.org/en/master/api/#main-interface
|
||||
[2]: http://debugtalk.com/post/apitestengine-not-only-about-json-api/
|
||||
[3]: ../../prepare/request-hook/
|
||||
@@ -1,51 +0,0 @@
|
||||
|
||||
对于上传文件类型的测试场景，HttpRunner 集成 [requests_toolbelt][1] 实现了上传功能。
|
||||
|
||||
在使用之前，确保已安装如下依赖库：
|
||||
|
||||
- [requests_toolbelt](https://github.com/requests/toolbelt)
|
||||
- [filetype](https://github.com/h2non/filetype.py)
|
||||
|
||||
使用内置 `upload` 关键字，可轻松实现上传功能（适用版本：2.4.1+）。
|
||||
|
||||
```yaml
|
||||
- test:
|
||||
name: upload file
|
||||
request:
|
||||
url: http://httpbin.org/upload
|
||||
method: POST
|
||||
headers:
|
||||
Cookie: session=AAA-BBB-CCC
|
||||
upload:
|
||||
file: "data/file_to_upload"
|
||||
field1: "value1"
|
||||
field2: "value2"
|
||||
validate:
|
||||
- eq: ["status_code", 200]
|
||||
```
|
||||
|
||||
同时，你也可以继续使用之前描述形式（适用版本：2.0+）。
|
||||
|
||||
```yaml
|
||||
- test:
|
||||
name: upload file
|
||||
variables:
|
||||
file: "data/file_to_upload"
|
||||
field1: "value1"
|
||||
field2: "value2"
|
||||
m_encoder: ${multipart_encoder(file=$file, field1=$field1, field2=$field2)}
|
||||
request:
|
||||
url: http://httpbin.org/upload
|
||||
method: POST
|
||||
headers:
|
||||
Content-Type: ${multipart_content_type($m_encoder)}
|
||||
Cookie: session=AAA-BBB-CCC
|
||||
data: $m_encoder
|
||||
validate:
|
||||
- eq: ["status_code", 200]
|
||||
```
|
||||
|
||||
参考案例：[httprunner/tests/httpbin/upload.yml][2]
|
||||
|
||||
[1]: https://toolbelt.readthedocs.io/en/latest/uploading-data.html
|
||||
[2]: https://github.com/httprunner/httprunner/blob/master/tests/httpbin/upload.yml
|
||||
@@ -1,624 +0,0 @@
|
||||
本文将通过一个简单的示例来展示 HttpRunner 的核心功能使用方法。
|
||||
|
||||
## 案例介绍
|
||||
|
||||
该案例作为被测服务，主要有两类接口：
|
||||
|
||||
- 权限校验，获取 token
|
||||
- 支持 CRUD 操作的 RESTful APIs，所有接口的请求头域中都必须包含有效的 token
|
||||
|
||||
案例的实现形式为 flask 应用服务（[api_server.py](data/api_server.py)），启动方式如下：
|
||||
|
||||
```text
|
||||
$ export FLASK_APP=docs/data/api_server.py
|
||||
$ export FLASK_ENV=development
|
||||
$ flask run
|
||||
* Serving Flask app "docs/data/api_server.py" (lazy loading)
|
||||
* Environment: development
|
||||
* Debug mode: on
|
||||
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
|
||||
* Restarting with stat
|
||||
* Debugger is active!
|
||||
* Debugger PIN: 989-476-348
|
||||
```
|
||||
|
||||
服务启动成功后，我们就可以开始对其进行测试了。
|
||||
|
||||
## 测试准备
|
||||
|
||||
### 抓包分析
|
||||
|
||||
在开始测试之前，我们需要先了解接口的请求和响应细节，而最佳的方式就是采用 `Charles Proxy` 或者 `Fiddler` 这类网络抓包工具进行抓包分析。
|
||||
|
||||
例如，在本案例中，我们先进行权限校验，然后成功创建一个用户，对应的网络抓包内容如下图所示：
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
通过抓包，我们可以看到具体的接口信息，包括请求的URL、Method、headers、参数和响应内容等内容，基于这些信息，我们就可以开始编写测试用例了。
|
||||
|
||||
### 生成测试用例
|
||||
|
||||
为了简化测试用例的编写工作，HttpRunner 实现了测试用例生成的功能。
|
||||
|
||||
首先，需要将抓取得到的数据包导出为 HAR 格式的文件，假设导出的文件名称为 [demo-quickstart.har](data/demo-quickstart.har)。
|
||||
|
||||
然后，在命令行终端中运行如下命令，即可将 demo-quickstart.har 转换为 HttpRunner 的测试用例文件。
|
||||
|
||||
```bash
|
||||
$ har2case docs/data/demo-quickstart.har -2y
|
||||
INFO:root:Start to generate testcase.
|
||||
INFO:root:dump testcase to YAML format.
|
||||
INFO:root:Generate YAML testcase successfully: docs/data/demo-quickstart.yml
|
||||
```
|
||||
|
||||
使用 `har2case` 转换脚本时默认转换为 JSON 格式，加上 `-2y` 参数后转换为 YAML 格式。两种格式完全等价，YAML 格式更简洁，JSON 格式支持的工具更丰富，大家可根据个人喜好进行选择。关于 [har2case][har2case] 的详细使用说明，请查看[《录制生成测试用例》](/prepare/record/)。
|
||||
|
||||
经过转换，在源 demo-quickstart.har 文件的同级目录下生成了相同文件名称的 YAML 格式测试用例文件 [demo-quickstart.yml](data/demo-quickstart.yml)，其内容如下：
|
||||
|
||||
```yaml
|
||||
- config:
|
||||
name: testcase description
|
||||
variables: {}
|
||||
|
||||
- test:
|
||||
name: /api/get-token
|
||||
request:
|
||||
headers:
|
||||
Content-Type: application/json
|
||||
User-Agent: python-requests/2.18.4
|
||||
app_version: 2.8.6
|
||||
device_sn: FwgRiO7CNA50DSU
|
||||
os_platform: ios
|
||||
json:
|
||||
sign: 9c0c7e51c91ae963c833a4ccbab8d683c4a90c98
|
||||
method: POST
|
||||
url: http://127.0.0.1:5000/api/get-token
|
||||
validate:
|
||||
- eq: [status_code, 200]
|
||||
- eq: [headers.Content-Type, application/json]
|
||||
- eq: [content.success, true]
|
||||
- eq: [content.token, baNLX1zhFYP11Seb]
|
||||
|
||||
- test:
|
||||
name: /api/users/1000
|
||||
request:
|
||||
headers:
|
||||
Content-Type: application/json
|
||||
User-Agent: python-requests/2.18.4
|
||||
device_sn: FwgRiO7CNA50DSU
|
||||
token: baNLX1zhFYP11Seb
|
||||
json:
|
||||
name: user1
|
||||
password: '123456'
|
||||
method: POST
|
||||
url: http://127.0.0.1:5000/api/users/1000
|
||||
validate:
|
||||
- eq: [status_code, 201]
|
||||
- eq: [headers.Content-Type, application/json]
|
||||
- eq: [content.success, true]
|
||||
- eq: [content.msg, user created successfully.]
|
||||
```
|
||||
|
||||
现在我们只需要知道如下几点：
|
||||
|
||||
- 每个 YAML/JSON 文件对应一个测试用例（testcase）
|
||||
- 每个测试用例为一个`list of dict`结构，其中可能包含全局配置项（config）和若干个测试步骤（test）
|
||||
- `config` 为全局配置项，作用域为整个测试用例
|
||||
- `test` 对应单个测试步骤，作用域仅限于本身
|
||||
|
||||
如上便是 HttpRunner 测试用例的基本结构。
|
||||
|
||||
关于测试用例的更多内容，请查看[《测试用例结构描述》](/prepare/testcase-structure/)。
|
||||
|
||||
### 首次运行测试用例
|
||||
|
||||
测试用例就绪后，我们可以开始调试运行了。
|
||||
|
||||
为了演示测试用例文件的迭代优化过程，我们先将 demo-quickstart.json 重命名为 [demo-quickstart-0.json](data/demo-quickstart-0.json)（对应的 YAML 格式：[demo-quickstart-0.yml](data/demo-quickstart-0.yml)）。
|
||||
|
||||
运行测试用例的命令为`hrun`，后面直接指定测试用例文件的路径即可。
|
||||
|
||||
```text
|
||||
$ hrun docs/data/demo-quickstart-0.yml
|
||||
INFO Start to run testcase: testcase description
|
||||
/api/get-token
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
INFO status_code: 200, response_time(ms): 9.26 ms, response_length: 46 bytes
|
||||
|
||||
ERROR validate: content.token equals baNLX1zhFYP11Seb(str) ==> fail
|
||||
tXGuSQgOCVXcltkz(str) equals baNLX1zhFYP11Seb(str)
|
||||
ERROR ******************************** DETAILED REQUEST & RESPONSE ********************************
|
||||
====== request details ======
|
||||
url: http://127.0.0.1:5000/api/get-token
|
||||
method: POST
|
||||
headers: {'Content-Type': 'application/json', 'User-Agent': 'python-requests/2.18.4', 'app_version': '2.8.6', 'device_sn': 'FwgRiO7CNA50DSU', 'os_platform': 'ios'}
|
||||
json: {'sign': '9c0c7e51c91ae963c833a4ccbab8d683c4a90c98'}
|
||||
verify: True
|
||||
|
||||
====== response details ======
|
||||
status_code: 200
|
||||
headers: {'Content-Type': 'application/json', 'Content-Length': '46', 'Server': 'Werkzeug/0.14.1 Python/3.7.0', 'Date': 'Sat, 26 Jan 2019 14:43:55 GMT'}
|
||||
body: '{"success": true, "token": "tXGuSQgOCVXcltkz"}'
|
||||
|
||||
F
|
||||
/api/users/1000
|
||||
INFO POST http://127.0.0.1:5000/api/users/1000
|
||||
ERROR 403 Client Error: FORBIDDEN for url: http://127.0.0.1:5000/api/users/1000
|
||||
ERROR validate: status_code equals 201(int) ==> fail
|
||||
403(int) equals 201(int)
|
||||
ERROR validate: content.success equals True(bool) ==> fail
|
||||
False(bool) equals True(bool)
|
||||
ERROR validate: content.msg equals user created successfully.(str) ==> fail
|
||||
Authorization failed!(str) equals user created successfully.(str)
|
||||
ERROR ******************************** DETAILED REQUEST & RESPONSE ********************************
|
||||
====== request details ======
|
||||
url: http://127.0.0.1:5000/api/users/1000
|
||||
method: POST
|
||||
headers: {'Content-Type': 'application/json', 'User-Agent': 'python-requests/2.18.4', 'device_sn': 'FwgRiO7CNA50DSU', 'token': 'baNLX1zhFYP11Seb'}
|
||||
json: {'name': 'user1', 'password': '123456'}
|
||||
verify: True
|
||||
|
||||
====== response details ======
|
||||
status_code: 403
|
||||
headers: {'Content-Type': 'application/json', 'Content-Length': '50', 'Server': 'Werkzeug/0.14.1 Python/3.7.0', 'Date': 'Sat, 26 Jan 2019 14:43:55 GMT'}
|
||||
body: '{"success": false, "msg": "Authorization failed!"}'
|
||||
|
||||
F
|
||||
|
||||
======================================================================
|
||||
FAIL: test_0000_000 (httprunner.api.TestSequense)
|
||||
/api/get-token
|
||||
----------------------------------------------------------------------
|
||||
Traceback (most recent call last):
|
||||
File "/Users/debugtalk/.pyenv/versions/3.6-dev/lib/python3.6/site-packages/httprunner/api.py", line 54, in test
|
||||
test_runner.run_test(test_dict)
|
||||
httprunner.exceptions.ValidationFailure: validate: content.token equals baNLX1zhFYP11Seb(str) ==> fail
|
||||
tXGuSQgOCVXcltkz(str) equals baNLX1zhFYP11Seb(str)
|
||||
|
||||
During handling of the above exception, another exception occurred:
|
||||
|
||||
Traceback (most recent call last):
|
||||
File "/Users/debugtalk/.pyenv/versions/3.6-dev/lib/python3.6/site-packages/httprunner/api.py", line 56, in test
|
||||
self.fail(str(ex))
|
||||
AssertionError: validate: content.token equals baNLX1zhFYP11Seb(str) ==> fail
|
||||
tXGuSQgOCVXcltkz(str) equals baNLX1zhFYP11Seb(str)
|
||||
|
||||
======================================================================
|
||||
FAIL: test_0001_000 (httprunner.api.TestSequense)
|
||||
/api/users/1000
|
||||
----------------------------------------------------------------------
|
||||
Traceback (most recent call last):
|
||||
File "/Users/debugtalk/.pyenv/versions/3.6-dev/lib/python3.6/site-packages/httprunner/api.py", line 54, in test
|
||||
test_runner.run_test(test_dict)
|
||||
httprunner.exceptions.ValidationFailure: validate: status_code equals 201(int) ==> fail
|
||||
403(int) equals 201(int)
|
||||
validate: content.success equals True(bool) ==> fail
|
||||
False(bool) equals True(bool)
|
||||
validate: content.msg equals user created successfully.(str) ==> fail
|
||||
Authorization failed!(str) equals user created successfully.(str)
|
||||
|
||||
During handling of the above exception, another exception occurred:
|
||||
|
||||
Traceback (most recent call last):
|
||||
File "/Users/debugtalk/.pyenv/versions/3.6-dev/lib/python3.6/site-packages/httprunner/api.py", line 56, in test
|
||||
self.fail(str(ex))
|
||||
AssertionError: validate: status_code equals 201(int) ==> fail
|
||||
403(int) equals 201(int)
|
||||
validate: content.success equals True(bool) ==> fail
|
||||
False(bool) equals True(bool)
|
||||
validate: content.msg equals user created successfully.(str) ==> fail
|
||||
Authorization failed!(str) equals user created successfully.(str)
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 2 tests in 0.026s
|
||||
|
||||
FAILED (failures=2)
|
||||
INFO Start to render Html report ...
|
||||
INFO Generated Html report: /Users/debugtalk/MyProjects/HttpRunner-dev/httprunner-docs-v2x/reports/1548513835.html
|
||||
```
|
||||
|
||||
非常不幸，两个接口的测试用例均运行失败了。
|
||||
|
||||
## 优化测试用例
|
||||
|
||||
从两个测试步骤的报错信息和堆栈信息（Traceback）可以看出，第一个步骤失败的原因是获取的 token 与预期值不一致，第二个步骤失败的原因是请求权限校验失败（403）。
|
||||
|
||||
接下来我们将逐步进行进行优化。
|
||||
|
||||
### 调整校验器
|
||||
|
||||
默认情况下，[har2case][har2case] 生成用例时，若 HTTP 请求的响应内容为 JSON 格式，则会将第一层级中的所有`key-value`转换为 validator。
|
||||
|
||||
例如上面的第一个测试步骤，生成的 validator 为：
|
||||
|
||||
```json
|
||||
"validate": [
|
||||
{"eq": ["status_code", 200]},
|
||||
{"eq": ["headers.Content-Type", "application/json"]},
|
||||
{"eq": ["content.success", true]},
|
||||
{"eq": ["content.token", "baNLX1zhFYP11Seb"]}
|
||||
]
|
||||
```
|
||||
|
||||
运行测试用例时，就会对上面的各个项进行校验。
|
||||
|
||||
问题在于，请求`/api/get-token`接口时，每次生成的 token 都会是不同的，因此将生成的 token 作为校验项的话，校验自然就无法通过了。
|
||||
|
||||
正确的做法是，在测试步骤的 validate 中应该去掉这类动态变化的值。
|
||||
|
||||
去除该项后，将用例另存为 [demo-quickstart-1.json](data/demo-quickstart-1.json)（对应的 YAML 格式：[demo-quickstart-1.yml](data/demo-quickstart-1.yml)）。
|
||||
|
||||
再次运行测试用例，运行结果如下：
|
||||
|
||||
```text
|
||||
$ hrun docs/data/demo-quickstart-1.yml
|
||||
INFO Start to run testcase: testcase description
|
||||
/api/get-token
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
INFO status_code: 200, response_time(ms): 6.61 ms, response_length: 46 bytes
|
||||
|
||||
.
|
||||
/api/users/1000
|
||||
INFO POST http://127.0.0.1:5000/api/users/1000
|
||||
ERROR 403 Client Error: FORBIDDEN for url: http://127.0.0.1:5000/api/users/1000
|
||||
ERROR validate: status_code equals 201(int) ==> fail
|
||||
403(int) equals 201(int)
|
||||
ERROR validate: content.success equals True(bool) ==> fail
|
||||
False(bool) equals True(bool)
|
||||
ERROR validate: content.msg equals user created successfully.(str) ==> fail
|
||||
Authorization failed!(str) equals user created successfully.(str)
|
||||
ERROR ******************************** DETAILED REQUEST & RESPONSE ********************************
|
||||
====== request details ======
|
||||
url: http://127.0.0.1:5000/api/users/1000
|
||||
method: POST
|
||||
headers: {'Content-Type': 'application/json', 'User-Agent': 'python-requests/2.18.4', 'device_sn': 'FwgRiO7CNA50DSU', 'token': 'baNLX1zhFYP11Seb'}
|
||||
json: {'name': 'user1', 'password': '123456'}
|
||||
verify: True
|
||||
|
||||
====== response details ======
|
||||
status_code: 403
|
||||
headers: {'Content-Type': 'application/json', 'Content-Length': '50', 'Server': 'Werkzeug/0.14.1 Python/3.7.0', 'Date': 'Sat, 26 Jan 2019 14:45:34 GMT'}
|
||||
body: '{"success": false, "msg": "Authorization failed!"}'
|
||||
|
||||
F
|
||||
|
||||
======================================================================
|
||||
FAIL: test_0001_000 (httprunner.api.TestSequense)
|
||||
/api/users/1000
|
||||
----------------------------------------------------------------------
|
||||
Traceback (most recent call last):
|
||||
File "/Users/debugtalk/.pyenv/versions/3.6-dev/lib/python3.6/site-packages/httprunner/api.py", line 54, in test
|
||||
test_runner.run_test(test_dict)
|
||||
httprunner.exceptions.ValidationFailure: validate: status_code equals 201(int) ==> fail
|
||||
403(int) equals 201(int)
|
||||
validate: content.success equals True(bool) ==> fail
|
||||
False(bool) equals True(bool)
|
||||
validate: content.msg equals user created successfully.(str) ==> fail
|
||||
Authorization failed!(str) equals user created successfully.(str)
|
||||
|
||||
During handling of the above exception, another exception occurred:
|
||||
|
||||
Traceback (most recent call last):
|
||||
File "/Users/debugtalk/.pyenv/versions/3.6-dev/lib/python3.6/site-packages/httprunner/api.py", line 56, in test
|
||||
self.fail(str(ex))
|
||||
AssertionError: validate: status_code equals 201(int) ==> fail
|
||||
403(int) equals 201(int)
|
||||
validate: content.success equals True(bool) ==> fail
|
||||
False(bool) equals True(bool)
|
||||
validate: content.msg equals user created successfully.(str) ==> fail
|
||||
Authorization failed!(str) equals user created successfully.(str)
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 2 tests in 0.018s
|
||||
|
||||
FAILED (failures=1)
|
||||
INFO Start to render Html report ...
|
||||
INFO Generated Html report: /Users/debugtalk/MyProjects/HttpRunner-dev/httprunner-docs-v2x/reports/1548513934.html
|
||||
```
|
||||
|
||||
经过修改，第一个测试步骤已经运行成功了，第二个步骤仍然运行失败（403），还是因为权限校验的原因。
|
||||
|
||||
### 参数关联
|
||||
|
||||
我们继续查看 [demo-quickstart-1.json](data/demo-quickstart-1.json)，会发现第二个测试步骤的请求 headers 中的 token 仍然是硬编码的，即抓包时获取到的值。在我们再次运行测试用例时，这个 token 已经失效了，所以会出现 403 权限校验失败的问题。
|
||||
|
||||
正确的做法是，我们应该在每次运行测试用例的时候，先动态获取到第一个测试步骤中的 token，然后在后续测试步骤的请求中使用前面获取到的 token。
|
||||
|
||||
在 HttpRunner 中，支持参数提取（`extract`）和参数引用的功能（`$var`）。
|
||||
|
||||
在测试步骤（test）中，若需要从响应结果中提取参数，则可使用 `extract` 关键字。extract 的列表中可指定一个或多个需要提取的参数。
|
||||
|
||||
在提取参数时，当 HTTP 的请求响应结果为 JSON 格式，则可以采用`.`运算符的方式，逐级往下获取到参数值；响应结果的整体内容引用方式为 content 或者 body。
|
||||
|
||||
例如，第一个接口`/api/get-token`的响应结果为：
|
||||
|
||||
```json
|
||||
{"success": true, "token": "ZQkYhbaQ6q8UFFNE"}
|
||||
```
|
||||
|
||||
那么要获取到 token 参数，就可以使用 content.token 的方式；具体的写法如下：
|
||||
|
||||
```json
|
||||
"extract": [
|
||||
{"token": "content.token"}
|
||||
]
|
||||
```
|
||||
|
||||
其中，token 作为提取后的参数名称，可以在后续使用 `$token` 进行引用。
|
||||
|
||||
```json
|
||||
"headers": {
|
||||
"device_sn": "FwgRiO7CNA50DSU",
|
||||
"token": "$token",
|
||||
"Content-Type": "application/json"
|
||||
}
|
||||
```
|
||||
|
||||
修改后的测试用例另存为 [demo-quickstart-2.json](data/demo-quickstart-2.json)（对应的 YAML 格式：[demo-quickstart-2.yml](data/demo-quickstart-2.yml)）。
|
||||
|
||||
再次运行测试用例，运行结果如下：
|
||||
|
||||
```text
|
||||
$ hrun docs/data/demo-quickstart-2.yml
|
||||
INFO Start to run testcase: testcase description
|
||||
/api/get-token
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
INFO status_code: 200, response_time(ms): 8.32 ms, response_length: 46 bytes
|
||||
|
||||
.
|
||||
/api/users/1000
|
||||
INFO POST http://127.0.0.1:5000/api/users/1000
|
||||
INFO status_code: 201, response_time(ms): 3.02 ms, response_length: 54 bytes
|
||||
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 2 tests in 0.019s
|
||||
|
||||
OK
|
||||
INFO Start to render Html report ...
|
||||
INFO Generated Html report: /Users/debugtalk/MyProjects/HttpRunner-dev/httprunner-docs-v2x/reports/1548514191.html
|
||||
```
|
||||
|
||||
经过修改，第二个测试步骤也运行成功了。
|
||||
|
||||
### base_url
|
||||
|
||||
虽然测试步骤运行都成功了，但是仍然有继续优化的地方。
|
||||
|
||||
继续查看 [demo-quickstart-2.json](data/demo-quickstart-2.json)，我们会发现在每个测试步骤的 URL 中，都采用的是完整的描述（host+path），但大多数情况下同一个用例中的 host 都是相同的，区别仅在于 path 部分。
|
||||
|
||||
因此，我们可以将各个测试步骤（test） URL 的 `base_url` 抽取出来，放到全局配置模块（config）中，在测试步骤中的 URL 只保留 PATH 部分。
|
||||
|
||||
```yaml
|
||||
- config:
|
||||
name: testcase description
|
||||
base_url: http://127.0.0.1:5000
|
||||
|
||||
- test:
|
||||
name: get token
|
||||
request:
|
||||
url: /api/get-token
|
||||
```
|
||||
|
||||
调整后的测试用例另存为 [demo-quickstart-3.json](data/demo-quickstart-3.json)（对应的 YAML 格式：[demo-quickstart-3.yml](data/demo-quickstart-3.yml)）。
|
||||
|
||||
重启 flask 应用服务后再次运行测试用例，所有的测试步骤仍然运行成功。
|
||||
|
||||
### 变量的申明和引用
|
||||
|
||||
继续查看 [demo-quickstart-3.json](data/demo-quickstart-3.json)，我们会发现测试用例中存在较多硬编码的参数，例如 app_version、device_sn、os_platform、user_id 等。
|
||||
|
||||
大多数情况下，我们可以不用修改这些硬编码的参数，测试用例也能正常运行。但是为了更好地维护测试用例，例如同一个参数值在测试步骤中出现多次，那么比较好的做法是，将这些参数定义为变量，然后在需要参数的地方进行引用。
|
||||
|
||||
在 HttpRunner 中，支持变量申明（`variables`）和引用（`$var`）的机制。在 config 和 test 中均可以通过 `variables` 关键字定义变量，然后在测试步骤中可以通过 `$ + 变量名称` 的方式引用变量。区别在于，在 config 中定义的变量为全局的，整个测试用例（testcase）的所有地方均可以引用；在 test 中定义的变量作用域仅局限于当前测试步骤（teststep）。
|
||||
|
||||
对上述各个测试步骤中硬编码的参数进行变量申明和引用调整后，新的测试用例另存为 [demo-quickstart-4.json](data/demo-quickstart-4.json)（对应的 YAML 格式：[demo-quickstart-4.yml](data/demo-quickstart-4.yml)）。
|
||||
|
||||
重启 flask 应用服务后再次运行测试用例，所有的测试步骤仍然运行成功。
|
||||
|
||||
### 抽取公共变量
|
||||
|
||||
查看 [demo-quickstart-4.json](data/demo-quickstart-4.json) 可以看出，两个测试步骤中都定义了 device_sn。针对这类公共的参数，我们可以将其统一定义在 config 的 variables 中，在测试步骤中就不用再重复定义。
|
||||
|
||||
```yaml
|
||||
- config:
|
||||
name: testcase description
|
||||
base_url: http://127.0.0.1:5000
|
||||
variables:
|
||||
device_sn: FwgRiO7CNA50DSU
|
||||
```
|
||||
|
||||
调整后的测试用例见 [demo-quickstart-5.json](data/demo-quickstart-5.json)（对应的 YAML 格式：[demo-quickstart-5.yml](data/demo-quickstart-5.yml)）。
|
||||
|
||||
### 实现动态运算逻辑
|
||||
|
||||
在 [demo-quickstart-5.yml](data/demo-quickstart-5.yml) 中，参数 device_sn 代表的是设备的 SN 编码，虽然采用硬编码的方式暂时不影响测试用例的运行，但这与真实的用户场景不大相符。
|
||||
|
||||
假设 device_sn 的格式为 15 长度的字符串，那么我们就可以在每次运行测试用例的时候，针对 device_sn 生成一个 15 位长度的随机字符串。与此同时，sign 字段是根据 headers 中的各个字段拼接后生成得到的 MD5 值，因此在 device_sn 变动后，sign 也应该重新进行计算，否则就会再次出现签名校验失败的问题。
|
||||
|
||||
然而，HttpRunner 的测试用例都是采用 YAML/JSON 格式进行描述的，在文本格式中如何执行代码运算呢？
|
||||
|
||||
HttpRunner 的实现方式为，支持热加载的插件机制（`debugtalk.py`），可以在 YAML/JSON 中调用 Python 函数。
|
||||
|
||||
具体地做法，我们可以在测试用例文件的同级或其父级目录中创建一个 debugtalk.py 文件，然后在其中定义相关的函数和变量。
|
||||
|
||||
例如，针对 device_sn 的随机字符串生成功能，我们可以定义一个 gen_random_string 函数；针对 sign 的签名算法，我们可以定义一个 get_sign 函数。
|
||||
|
||||
```python
|
||||
import hashlib
|
||||
import hmac
|
||||
import random
|
||||
import string
|
||||
|
||||
SECRET_KEY = "DebugTalk"
|
||||
|
||||
def gen_random_string(str_len):
|
||||
random_char_list = []
|
||||
for _ in range(str_len):
|
||||
random_char = random.choice(string.ascii_letters + string.digits)
|
||||
random_char_list.append(random_char)
|
||||
|
||||
random_string = ''.join(random_char_list)
|
||||
return random_string
|
||||
|
||||
def get_sign(*args):
|
||||
content = ''.join(args).encode('ascii')
|
||||
sign_key = SECRET_KEY.encode('ascii')
|
||||
sign = hmac.new(sign_key, content, hashlib.sha1).hexdigest()
|
||||
return sign
|
||||
```
|
||||
|
||||
然后，我们在 YAML/JSON 测试用例文件中，就可以对定义的函数进行调用，对定义的变量进行引用了。引用变量的方式仍然与前面讲的一样，采用`$ + 变量名称`的方式；调用函数的方式为`${func($var)}`。
|
||||
|
||||
例如，生成 15 位长度的随机字符串并赋值给 device_sn 的代码为：
|
||||
|
||||
```json
|
||||
"variables": [
|
||||
{"device_sn": "${gen_random_string(15)}"}
|
||||
]
|
||||
```
|
||||
|
||||
使用 $user_agent、$device_sn、$os_platform、$app_version 根据签名算法生成 sign 值的代码为：
|
||||
|
||||
```json
|
||||
"json": {
|
||||
"sign": "${get_sign($user_agent, $device_sn, $os_platform, $app_version)}"
|
||||
}
|
||||
```
|
||||
|
||||
对测试用例进行上述调整后，另存为 [demo-quickstart-6.json](data/demo-quickstart-6.json)（对应的 YAML 格式：[demo-quickstart-6.yml](data/demo-quickstart-6.yml)）。
|
||||
|
||||
重启 flask 应用服务后再次运行测试用例，所有的测试步骤仍然运行成功。
|
||||
|
||||
### 参数化数据驱动
|
||||
|
||||
> 请确保你使用的 HttpRunner 版本号不低于 2.0.0
|
||||
|
||||
在 [demo-quickstart-6.yml](data/demo-quickstart-6.yml) 中，user_id 仍然是写死的值，假如我们需要创建 user_id 为 1001～1004 的用户，那我们只能不断地去修改 user_id，然后运行测试用例，重复操作 4 次？或者我们在测试用例文件中将创建用户的 test 复制 4 份，然后在每一份里面分别使用不同的 user_id ？
|
||||
|
||||
很显然，不管是采用上述哪种方式，都会很繁琐，并且也无法应对灵活多变的测试需求。
|
||||
|
||||
针对这类需求，HttpRunner 支持参数化数据驱动的功能。
|
||||
|
||||
在 HttpRunner 中，若要采用数据驱动的方式来运行测试用例，需要创建一个文件，对测试用例进行引用，并使用 `parameters` 关键字定义参数并指定数据源取值方式。
|
||||
|
||||
例如，我们需要在创建用户的接口中对 user_id 进行参数化，参数化列表为 1001～1004，并且取值方式为顺序取值，那么最简单的描述方式就是直接指定参数列表。具体的编写方式为，新建一个测试场景文件 [demo-quickstart-7.yml](data/demo-quickstart-7.yml)（对应的 JSON 格式：[demo-quickstart-7.json](data/demo-quickstart-7.json)），内容如下所示：
|
||||
|
||||
```yaml
|
||||
config:
|
||||
name: testcase description
|
||||
|
||||
testcases:
|
||||
create user:
|
||||
testcase: demo-quickstart-6.yml
|
||||
parameters:
|
||||
user_id: [1001, 1002, 1003, 1004]
|
||||
```
|
||||
|
||||
仅需如上配置，针对 user_id 的参数化数据驱动就完成了。
|
||||
|
||||
重启 flask 应用服务后再次运行测试用例，测试用例运行情况如下所示：
|
||||
|
||||
<details>
|
||||
<summary>点击查看运行日志</summary>
|
||||
|
||||
```text
|
||||
$ hrun docs/data/demo-quickstart-7.json
|
||||
INFO Start to run testcase: create user 1001
|
||||
/api/get-token
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
INFO status_code: 200, response_time(ms): 8.95 ms, response_length: 46 bytes
|
||||
|
||||
.
|
||||
/api/users/1001
|
||||
INFO POST http://127.0.0.1:5000/api/users/1001
|
||||
INFO status_code: 201, response_time(ms): 3.02 ms, response_length: 54 bytes
|
||||
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 2 tests in 0.021s
|
||||
|
||||
OK
|
||||
INFO Start to run testcase: create user 1002
|
||||
/api/get-token
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
INFO status_code: 200, response_time(ms): 2.78 ms, response_length: 46 bytes
|
||||
|
||||
.
|
||||
/api/users/1002
|
||||
INFO POST http://127.0.0.1:5000/api/users/1002
|
||||
INFO status_code: 201, response_time(ms): 2.84 ms, response_length: 54 bytes
|
||||
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 2 tests in 0.007s
|
||||
|
||||
OK
|
||||
INFO Start to run testcase: create user 1003
|
||||
/api/get-token
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
INFO status_code: 200, response_time(ms): 2.92 ms, response_length: 46 bytes
|
||||
|
||||
.
|
||||
/api/users/1003
|
||||
INFO POST http://127.0.0.1:5000/api/users/1003
|
||||
INFO status_code: 201, response_time(ms): 5.56 ms, response_length: 54 bytes
|
||||
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 2 tests in 0.011s
|
||||
|
||||
OK
|
||||
INFO Start to run testcase: create user 1004
|
||||
/api/get-token
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
INFO status_code: 200, response_time(ms): 5.25 ms, response_length: 46 bytes
|
||||
|
||||
.
|
||||
/api/users/1004
|
||||
INFO POST http://127.0.0.1:5000/api/users/1004
|
||||
INFO status_code: 201, response_time(ms): 7.02 ms, response_length: 54 bytes
|
||||
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 2 tests in 0.016s
|
||||
|
||||
OK
|
||||
INFO Start to render Html report ...
|
||||
INFO Generated Html report: /Users/debugtalk/MyProjects/HttpRunner-dev/httprunner-docs-v2x/reports/1548518757.html
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
可以看出，测试用例总共运行了 4 次，并且每次运行时都是采用的不同 user_id。
|
||||
|
||||
关于参数化数据驱动，这里只描述了最简单的场景和使用方式，如需了解更多，请进一步阅读[《数据驱动使用手册》](/prepare/parameters/)。
|
||||
|
||||
[har2case]: https://github.com/HttpRunner/har2case
|
||||
|
||||
## 查看测试报告
|
||||
|
||||
在每次使用 hrun 命令运行测试用例后，均会生成一份 HTML 格式的测试报告。报告文件位于 reports 目录下，文件名称为测试用例的开始运行时间。
|
||||
|
||||
例如，在运行完 [demo-quickstart-1.json](data/demo-quickstart-1.json) 后，将生成如下形式的测试报告：
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
关于测试报告的详细内容，请查看[《测试报告》](/run-tests/report/)部分。
|
||||
|
||||
## 总结
|
||||
|
||||
到此为止，HttpRunner 的核心功能就介绍完了，掌握本文中的功能特性，足以帮助你应对日常项目工作中至少 80% 的自动化测试需求。
|
||||
|
||||
当然，HttpRunner 不止于此，如需挖掘 HttpRunner 的更多特性，实现更复杂场景的自动化测试需求，可继续阅读后续文档。
|
||||
@@ -1,17 +0,0 @@
|
||||
|
||||
## 开发笔记
|
||||
|
||||
- [DebugTalk](http://debugtalk.com/tags/HttpRunner/)
|
||||
|
||||
## 演讲
|
||||
|
||||
- [MTSC 2018][MTSC2018]: [《大疆互联网的一站式自动化测试解决方案（基于HttpRunner）》][dji-httprunner]
|
||||
- [PyCon China 2018][PyConChina2018]: [《借助 Python 开源生态打造企业级自动化测试框架（HttpRunner）》][PyCon-HttpRunner]
|
||||
- [MTSC 2019][MTSC2019]: [《HttpRunner 2.0 技术架构与接口测试应用》][httprunner-2.0]
|
||||
|
||||
[MTSC2018]: https://www.bagevent.com/event/1193113
|
||||
[dji-httprunner]: https://github.com/debugtalk/speech/blob/master/DJI-HttpRunner.pdf
|
||||
[PyConChina2018]: http://cn.pycon.org/2018/city_beijing.html
|
||||
[PyCon-HttpRunner]: https://github.com/debugtalk/speech/blob/master/PyCon-HttpRunner.pdf
|
||||
[MTSC2019]: https://testerhome.com/mtsc/2019
|
||||
[httprunner-2.0]: https://github.com/debugtalk/speech/blob/master/MTSC2019-HttpRunner-2.0.pdf
|
||||
@@ -1,218 +0,0 @@
|
||||
|
||||
HttpRunner 在命令行中启动测试时，通过指定参数，可实现丰富的测试特性控制。
|
||||
|
||||
```text
|
||||
$ hrun -h
|
||||
usage: hrun [-h] [-V] [--log-level LOG_LEVEL] [--log-file LOG_FILE]
|
||||
[--dot-env-path DOT_ENV_PATH] [--report-template REPORT_TEMPLATE]
|
||||
[--report-dir REPORT_DIR] [--failfast] [--save-tests]
|
||||
[--startproject STARTPROJECT]
|
||||
[--validate [VALIDATE [VALIDATE ...]]]
|
||||
[--prettify [PRETTIFY [PRETTIFY ...]]]
|
||||
[testcase_paths [testcase_paths ...]]
|
||||
|
||||
One-stop solution for HTTP(S) testing.
|
||||
|
||||
positional arguments:
|
||||
testcase_paths testcase file path
|
||||
|
||||
optional arguments:
|
||||
-h, --help show this help message and exit
|
||||
-V, --version show version
|
||||
--log-level LOG_LEVEL
|
||||
Specify logging level, default is INFO.
|
||||
--log-file LOG_FILE Write logs to specified file path.
|
||||
--dot-env-path DOT_ENV_PATH
|
||||
Specify .env file path, which is useful for keeping
|
||||
sensitive data.
|
||||
--report-template REPORT_TEMPLATE
|
||||
specify report template path.
|
||||
--report-dir REPORT_DIR
|
||||
specify report save directory.
|
||||
--failfast Stop the test run on the first error or failure.
|
||||
--save-tests Save loaded tests and parsed tests to JSON file.
|
||||
--startproject STARTPROJECT
|
||||
Specify new project name.
|
||||
--validate [VALIDATE [VALIDATE ...]]
|
||||
Validate JSON testcase format.
|
||||
--prettify [PRETTIFY [PRETTIFY ...]]
|
||||
Prettify JSON testcase format.
|
||||
```
|
||||
|
||||
## 指定测试用例路径
|
||||
|
||||
使用 HttpRunner 指定测试用例路径时，支持多种方式。
|
||||
|
||||
使用 hrun 命令外加单个测试用例文件的路径，运行单个测试用例，并生成一个测试报告文件：
|
||||
|
||||
```text
|
||||
$ hrun filepath/testcase.yml
|
||||
```
|
||||
|
||||
将多个测试用例文件放置到文件夹中，指定文件夹路径可将文件夹下所有测试用例作为测试用例集进行运行，并生成一个测试报告文件：
|
||||
|
||||
```text
|
||||
$ hrun testcases_folder_path
|
||||
```
|
||||
|
||||
## failfast
|
||||
|
||||
默认情况下，HttpRunner 会运行指定用例集中的所有测试用例，并统计测试结果。
|
||||
|
||||
> 对于某些依赖于执行顺序的测试用例，例如需要先登录成功才能执行后续接口请求的场景，当前面的测试用例执行失败后，后续的测试用例也都必将失败，因此没有继续执行的必要了。
|
||||
|
||||
若希望测试用例在运行过程中，遇到失败时不再继续运行后续用例，则可通过在命令中添加`--failfast`实现。
|
||||
|
||||
```text
|
||||
$ hrun filepath/testcase.yml --failfast
|
||||
```
|
||||
|
||||
## 日志级别
|
||||
|
||||
默认情况下，HttpRunner 运行时的日志级别为`INFO`，只会包含最基本的信息，包括用例名称、请求的URL和Method、响应结果的状态码、耗时和内容大小。
|
||||
|
||||
```text
|
||||
$ hrun docs/data/demo-quickstart-6.json
|
||||
INFO Start to run testcase: testcase description
|
||||
/api/get-token
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
INFO status_code: 200, response_time(ms): 9.08 ms, response_length: 46 bytes
|
||||
|
||||
.
|
||||
/api/users/1548560655759
|
||||
INFO POST http://127.0.0.1:5000/api/users/1548560655759
|
||||
INFO status_code: 201, response_time(ms): 2.89 ms, response_length: 54 bytes
|
||||
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 2 tests in 0.019s
|
||||
|
||||
OK
|
||||
INFO Start to render Html report ...
|
||||
INFO Generated Html report: /Users/debugtalk/MyProjects/HttpRunner-dev/httprunner-docs-v2x/reports/1548560655.html
|
||||
```
|
||||
|
||||
若需要查看到更详尽的信息，例如请求的参数和响应的详细内容，可以将日志级别设置为`DEBUG`，即在命令中添加`--log-level debug`。
|
||||
|
||||
```
|
||||
$ hrun docs/data/demo-quickstart-6.json --log-level debug
|
||||
INFO Start to run testcase: testcase description
|
||||
/api/get-token
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
DEBUG request kwargs(raw): {'headers': {'User-Agent': 'python-requests/2.18.4', 'device_sn': 'W5ACRDytKRQJPhC', 'os_platform': 'ios', 'app_version': '2.8.6', 'Content-Type': 'application/json'}, 'json': {'sign': '2e7c3b5d560a1c8a859edcb9c8b0d3f8349abeff'}, 'verify': True}
|
||||
DEBUG processed request:
|
||||
> POST http://127.0.0.1:5000/api/get-token
|
||||
> kwargs: {'headers': {'User-Agent': 'python-requests/2.18.4', 'device_sn': 'W5ACRDytKRQJPhC', 'os_platform': 'ios', 'app_version': '2.8.6', 'Content-Type': 'application/json'}, 'json': {'sign': '2e7c3b5d560a1c8a859edcb9c8b0d3f8349abeff'}, 'verify': True, 'timeout': 120}
|
||||
DEBUG
|
||||
================== request details ==================
|
||||
url : 'http://127.0.0.1:5000/api/get-token'
|
||||
method : 'POST'
|
||||
headers : {'User-Agent': 'python-requests/2.18.4', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'device_sn': 'W5ACRDytKRQJPhC', 'os_platform': 'ios', 'app_version': '2.8.6', 'Content-Type': 'application/json', 'Content-Length': '52'}
|
||||
body : b'{"sign": "2e7c3b5d560a1c8a859edcb9c8b0d3f8349abeff"}'
|
||||
|
||||
DEBUG
|
||||
================== response details ==================
|
||||
ok : True
|
||||
url : 'http://127.0.0.1:5000/api/get-token'
|
||||
status_code : 200
|
||||
reason : 'OK'
|
||||
cookies : {}
|
||||
encoding : None
|
||||
headers : {'Content-Type': 'application/json', 'Content-Length': '46', 'Server': 'Werkzeug/0.14.1 Python/3.6.5+', 'Date': 'Sun, 27 Jan 2019 03:45:16 GMT'}
|
||||
content_type : 'application/json'
|
||||
json : {'success': True, 'token': 'o6uakmubLrCbpRRS'}
|
||||
|
||||
INFO status_code: 200, response_time(ms): 9.28 ms, response_length: 46 bytes
|
||||
|
||||
DEBUG start to extract from response object.
|
||||
DEBUG extract: content.token => o6uakmubLrCbpRRS
|
||||
DEBUG start to validate.
|
||||
DEBUG extract: status_code => 200
|
||||
DEBUG validate: status_code equals 200(int) ==> pass
|
||||
DEBUG extract: headers.Content-Type => application/json
|
||||
DEBUG validate: headers.Content-Type equals application/json(str) ==> pass
|
||||
DEBUG extract: content.success => True
|
||||
DEBUG validate: content.success equals True(bool) ==> pass
|
||||
.
|
||||
/api/users/1548560716736
|
||||
INFO POST http://127.0.0.1:5000/api/users/1548560716736
|
||||
DEBUG request kwargs(raw): {'headers': {'User-Agent': 'python-requests/2.18.4', 'device_sn': 'W5ACRDytKRQJPhC', 'token': 'o6uakmubLrCbpRRS', 'Content-Type': 'application/json'}, 'json': {'name': 'user1', 'password': '123456'}, 'verify': True}
|
||||
DEBUG processed request:
|
||||
> POST http://127.0.0.1:5000/api/users/1548560716736
|
||||
> kwargs: {'headers': {'User-Agent': 'python-requests/2.18.4', 'device_sn': 'W5ACRDytKRQJPhC', 'token': 'o6uakmubLrCbpRRS', 'Content-Type': 'application/json'}, 'json': {'name': 'user1', 'password': '123456'}, 'verify': True, 'timeout': 120}
|
||||
DEBUG
|
||||
================== request details ==================
|
||||
url : 'http://127.0.0.1:5000/api/users/1548560716736'
|
||||
method : 'POST'
|
||||
headers : {'User-Agent': 'python-requests/2.18.4', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'device_sn': 'W5ACRDytKRQJPhC', 'token': 'o6uakmubLrCbpRRS', 'Content-Type': 'application/json', 'Content-Length': '39'}
|
||||
body : b'{"name": "user1", "password": "123456"}'
|
||||
|
||||
DEBUG
|
||||
================== response details ==================
|
||||
ok : True
|
||||
url : 'http://127.0.0.1:5000/api/users/1548560716736'
|
||||
status_code : 201
|
||||
reason : 'CREATED'
|
||||
cookies : {}
|
||||
encoding : None
|
||||
headers : {'Content-Type': 'application/json', 'Content-Length': '54', 'Server': 'Werkzeug/0.14.1 Python/3.6.5+', 'Date': 'Sun, 27 Jan 2019 03:45:16 GMT'}
|
||||
content_type : 'application/json'
|
||||
json : {'success': True, 'msg': 'user created successfully.'}
|
||||
|
||||
INFO status_code: 201, response_time(ms): 2.77 ms, response_length: 54 bytes
|
||||
|
||||
DEBUG start to validate.
|
||||
DEBUG extract: status_code => 201
|
||||
DEBUG validate: status_code equals 201(int) ==> pass
|
||||
DEBUG extract: headers.Content-Type => application/json
|
||||
DEBUG validate: headers.Content-Type equals application/json(str) ==> pass
|
||||
DEBUG extract: content.success => True
|
||||
DEBUG validate: content.success equals True(bool) ==> pass
|
||||
DEBUG extract: content.msg => user created successfully.
|
||||
DEBUG validate: content.msg equals user created successfully.(str) ==> pass
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 2 tests in 0.022s
|
||||
|
||||
OK
|
||||
DEBUG No html report template specified, use default.
|
||||
INFO Start to render Html report ...
|
||||
INFO Generated Html report: /Users/debugtalk/MyProjects/HttpRunner-dev/httprunner-docs-v2x/reports/1548560716.html
|
||||
```
|
||||
|
||||
## 保存详细过程数据
|
||||
|
||||
为了方便定位问题，运行测试时可指定 `--save-tests` 参数，即可将运行过程的中间数据保存为日志文件。
|
||||
|
||||
```text
|
||||
$ hrun docs/data/demo-quickstart-6.json --save-tests
|
||||
dump file: /Users/debugtalk/MyProjects/HttpRunner-dev/httprunner-docs-v2x/docs/data/logs/demo-quickstart-6.loaded.json
|
||||
dump file: /Users/debugtalk/MyProjects/HttpRunner-dev/httprunner-docs-v2x/docs/data/logs/demo-quickstart-6.parsed.json
|
||||
INFO Start to run testcase: testcase description
|
||||
/api/get-token
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
INFO status_code: 200, response_time(ms): 11.42 ms, response_length: 46 bytes
|
||||
|
||||
.
|
||||
/api/users/1548560768589
|
||||
INFO POST http://127.0.0.1:5000/api/users/1548560768589
|
||||
INFO status_code: 201, response_time(ms): 2.8 ms, response_length: 54 bytes
|
||||
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 2 tests in 0.028s
|
||||
|
||||
OK
|
||||
dump file: /Users/debugtalk/MyProjects/HttpRunner-dev/httprunner-docs-v2x/docs/data/logs/demo-quickstart-6.summary.json
|
||||
INFO Start to render Html report ...
|
||||
INFO Generated Html report: /Users/debugtalk/MyProjects/HttpRunner-dev/httprunner-docs-v2x/reports/1548560768.html
|
||||
```
|
||||
|
||||
日志文件将保存在项目根目录的 `logs` 文件夹中，生成的文件有如下三个（XXX为测试用例名称）：
|
||||
|
||||
- `XXX.loaded.json`：测试用例加载后的数据结构内容，加载包括测试用例文件（YAML/JSON）、debugtalk.py、.env 等所有项目文件，例如 [`demo-quickstart-6.loaded.json`](/data/logs/demo-quickstart-6.loaded.json)
|
||||
- `XXX.parsed.json`：测试用例解析后的数据结构内容，解析内容包括测试用例引用（API/testcase）、变量计算和替换、base_url 拼接等，例如 [`demo-quickstart-6.parsed.json`](/data/logs/demo-quickstart-6.parsed.json)
|
||||
- `XXX.summary.json`：测试报告生成前的数据结构内容，例如 [`demo-quickstart-6.summary.json`](/data/logs/demo-quickstart-6.summary.json)
|
||||
@@ -1,157 +0,0 @@
|
||||
HttpRunner 通过复用 [`Locust`][Locust]，可以在无需对 YAML/JSON 进行任何修改的情况下，直接运行性能测试。
|
||||
|
||||
## 原理图
|
||||
|
||||
|
||||
|
||||
## 安装依赖包
|
||||
|
||||
安装完成 HttpRunner 后，系统中会新增`locusts`命令，但不会同时安装 Locust。
|
||||
|
||||
在系统中未安装 Locust 的情况下，运行`locusts`命令时会出现如下提示。
|
||||
|
||||
```bash
|
||||
$ locusts -V
|
||||
WARNING Locust is not installed, install first and try again.
|
||||
install command: pip install locustio
|
||||
```
|
||||
|
||||
Locust 的安装方式如下：
|
||||
|
||||
```text
|
||||
$ pip install locustio
|
||||
```
|
||||
|
||||
安装完成后，执行 `locusts -V` 可查看到 Locust 的版本号。
|
||||
|
||||
```text
|
||||
$ locusts -V
|
||||
[2017-08-26 23:45:42,246] bogon/INFO/stdout: Locust 0.8a2
|
||||
[2017-08-26 23:45:42,246] bogon/INFO/stdout:
|
||||
```
|
||||
|
||||
执行 `locusts -h`，可查看到使用帮助文档。
|
||||
|
||||
```text
|
||||
$ locusts -h
|
||||
Usage: locust [options] [LocustClass [LocustClass2 ... ]]
|
||||
|
||||
Options:
|
||||
-h, --help show this help message and exit
|
||||
-H HOST, --host=HOST Host to load test in the following format:
|
||||
http://10.21.32.33
|
||||
--web-host=WEB_HOST Host to bind the web interface to. Defaults to '' (all
|
||||
interfaces)
|
||||
-P PORT, --port=PORT, --web-port=PORT
|
||||
Port on which to run web host
|
||||
-f LOCUSTFILE, --locustfile=LOCUSTFILE
|
||||
Python module file to import, e.g. '../other.py'.
|
||||
Default: locustfile
|
||||
--csv=CSVFILEBASE, --csv-base-name=CSVFILEBASE
|
||||
Store current request stats to files in CSV format.
|
||||
--master Set locust to run in distributed mode with this
|
||||
process as master
|
||||
--slave Set locust to run in distributed mode with this
|
||||
process as slave
|
||||
--master-host=MASTER_HOST
|
||||
Host or IP address of locust master for distributed
|
||||
load testing. Only used when running with --slave.
|
||||
Defaults to 127.0.0.1.
|
||||
--master-port=MASTER_PORT
|
||||
The port to connect to that is used by the locust
|
||||
master for distributed load testing. Only used when
|
||||
running with --slave. Defaults to 5557. Note that
|
||||
slaves will also connect to the master node on this
|
||||
port + 1.
|
||||
--master-bind-host=MASTER_BIND_HOST
|
||||
Interfaces (hostname, ip) that locust master should
|
||||
bind to. Only used when running with --master.
|
||||
Defaults to * (all available interfaces).
|
||||
--master-bind-port=MASTER_BIND_PORT
|
||||
Port that locust master should bind to. Only used when
|
||||
running with --master. Defaults to 5557. Note that
|
||||
Locust will also use this port + 1, so by default the
|
||||
master node will bind to 5557 and 5558.
|
||||
--expect-slaves=EXPECT_SLAVES
|
||||
How many slaves master should expect to connect before
|
||||
starting the test (only when --no-web used).
|
||||
--no-web Disable the web interface, and instead start running
|
||||
the test immediately. Requires -c and -r to be
|
||||
specified.
|
||||
-c NUM_CLIENTS, --clients=NUM_CLIENTS
|
||||
Number of concurrent clients. Only used together with
|
||||
--no-web
|
||||
-r HATCH_RATE, --hatch-rate=HATCH_RATE
|
||||
The rate per second in which clients are spawned. Only
|
||||
used together with --no-web
|
||||
-n NUM_REQUESTS, --num-request=NUM_REQUESTS
|
||||
Number of requests to perform. Only used together with
|
||||
--no-web
|
||||
-L LOGLEVEL, --loglevel=LOGLEVEL
|
||||
Choose between DEBUG/INFO/WARNING/ERROR/CRITICAL.
|
||||
Default is INFO.
|
||||
--logfile=LOGFILE Path to log file. If not set, log will go to
|
||||
stdout/stderr
|
||||
--print-stats Print stats in the console
|
||||
--only-summary Only print the summary stats
|
||||
--no-reset-stats Do not reset statistics once hatching has been
|
||||
completed
|
||||
-l, --list Show list of possible locust classes and exit
|
||||
--show-task-ratio print table of the locust classes' task execution
|
||||
ratio
|
||||
--show-task-ratio-json
|
||||
print json data of the locust classes' task execution
|
||||
ratio
|
||||
-V, --version show program's version number and exit
|
||||
```
|
||||
|
||||
可以看出，`loucsts` 命令与 `locust` 命令的用法基本相同。
|
||||
|
||||
相比于 `locust` 命令，`loucsts`命令主要存在如下两项差异。
|
||||
|
||||
## 运行性能测试
|
||||
|
||||
在 `-f` 参数后面，`loucsts` 命令不仅可以指定 Locust 支持的 Python 文件，同时可以直接指定 YAML/JSON 格式的测试用例文件。在具体实现上，当 `-f` 指定 YAML/JSON 格式的测试用例文件时，会先将其转换为 Python 格式的 locustfile，然后再将 locustfile.py 传给 locust 命令。
|
||||
|
||||
```bash
|
||||
$ locusts -f examples/first-testcase.yml
|
||||
[2017-08-18 17:20:43,915] Leos-MacBook-Air.local/INFO/locust.main: Starting web monitor at *:8089
|
||||
[2017-08-18 17:20:43,918] Leos-MacBook-Air.local/INFO/locust.main: Starting Locust 0.8a2
|
||||
```
|
||||
|
||||
执行上述命令后，即完成了 Locust 服务的启动，后续就可以在 Locust 的 Web 管理界面中进行操作了，使用方式与 Locust 完全相同。
|
||||
|
||||
## 多进程运行模式
|
||||
|
||||
默认情况下，在 Locust 中如需使用 master-slave 模式启动多个进程（使用多核处理器的能力），只能先启动 master，然后再逐一启动若干个 slave。
|
||||
|
||||
```text
|
||||
$ locust -f locustfile.py --master
|
||||
$ locust -f locustfile.py --slave &
|
||||
$ locust -f locustfile.py --slave &
|
||||
$ locust -f locustfile.py --slave &
|
||||
$ locust -f locustfile.py --slave &
|
||||
```
|
||||
|
||||
在 HttpRunner 中，新增实现 `--processes` 参数，可以一次性启动 1 个 master 和多个 salve。若在 `--processes` 参数后没有指定具体的数值，则启动的 slave 个数与机器的 CPU 核数相同。
|
||||
|
||||
```bash
|
||||
$ locusts -f examples/first-testcase.yml --processes 4
|
||||
[2017-08-26 23:51:47,071] bogon/INFO/locust.main: Starting web monitor at *:8089
|
||||
[2017-08-26 23:51:47,075] bogon/INFO/locust.main: Starting Locust 0.8a2
|
||||
[2017-08-26 23:51:47,078] bogon/INFO/locust.main: Starting Locust 0.8a2
|
||||
[2017-08-26 23:51:47,080] bogon/INFO/locust.main: Starting Locust 0.8a2
|
||||
[2017-08-26 23:51:47,083] bogon/INFO/locust.main: Starting Locust 0.8a2
|
||||
[2017-08-26 23:51:47,084] bogon/INFO/locust.runners: Client 'bogon_656e0af8e968a8533d379dd252422ad3' reported as ready. Currently 1 clients ready to swarm.
|
||||
[2017-08-26 23:51:47,085] bogon/INFO/locust.runners: Client 'bogon_09f73850252ee4ec739ed77d3c4c6dba' reported as ready. Currently 2 clients ready to swarm.
|
||||
[2017-08-26 23:51:47,084] bogon/INFO/locust.main: Starting Locust 0.8a2
|
||||
[2017-08-26 23:51:47,085] bogon/INFO/locust.runners: Client 'bogon_869f7ed671b1a9952b56610f01e2006f' reported as ready. Currently 3 clients ready to swarm.
|
||||
[2017-08-26 23:51:47,085] bogon/INFO/locust.runners: Client 'bogon_80a804cda36b80fac17b57fd2d5e7cdb' reported as ready. Currently 4 clients ready to swarm.
|
||||
```
|
||||
|
||||
|
||||
|
||||
Enjoy!
|
||||
|
||||
|
||||
[Locust]: http://locust.io/
|
||||
@@ -1,113 +0,0 @@
|
||||
|
||||
使用 HttpRunner 执行完自动化测试后，会在当前路径的 reports 目录下生成一份 HTML 格式的测试报告。
|
||||
|
||||
## 默认情况
|
||||
|
||||
默认情况下，生成的测试报告文件会位于项目根目录的 reports 文件夹中，文件名称为测试开始的时间戳。
|
||||
|
||||
```bash
|
||||
$ hrun docs/data/demo-quickstart-6.yml
|
||||
INFO Start to run testcase: testcase description
|
||||
/api/get-token
|
||||
INFO POST http://127.0.0.1:5000/api/get-token
|
||||
INFO status_code: 200, response_time(ms): 10.05 ms, response_length: 46 bytes
|
||||
|
||||
.
|
||||
/api/users/1548561170497
|
||||
INFO POST http://127.0.0.1:5000/api/users/1548561170497
|
||||
INFO status_code: 201, response_time(ms): 2.88 ms, response_length: 54 bytes
|
||||
|
||||
.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 2 tests in 0.034s
|
||||
|
||||
OK
|
||||
INFO Start to render Html report ...
|
||||
INFO Generated Html report: /Users/debugtalk/MyProjects/HttpRunner-dev/httprunner-docs-v2x/reports/1548561170.html
|
||||
```
|
||||
|
||||
点击查看[测试报告](/data/reports/1548561170.html)。
|
||||
|
||||
## 默认报告样式
|
||||
|
||||
在 HttpRunner 中自带了一个 Jinja2 格式的报告模版，默认情况下，生成的报告样式均基于该模版（[httprunner/templates/default_report_template.html][default_report]）。
|
||||
|
||||
测试报告形式如下：
|
||||
|
||||
在 Summary 中，会罗列本次测试的整体信息，包括测试开始时间、总运行时长、运行的Python版本和系统环境、运行结果统计数据。
|
||||
|
||||
|
||||
|
||||
在 Details 中，会详细展示每一条测试用例的运行结果。
|
||||
|
||||
点击测试用例对应的 log 按钮，会在弹出框中展示该用例执行的详细数据，包括请求的 headers 和 body、响应的 headers 和 body、校验结果、响应、响应耗时（elapsed）等信息。
|
||||
|
||||
|
||||
|
||||
|
||||
若测试用例运行不成功（failed/error/skipped），则在该测试用例的 detail 中会出现 traceback 按钮，点击该按钮后，会在弹出框中展示失败的堆栈日志，或者 skipped 的原因。
|
||||
|
||||
|
||||
|
||||
点击查看[测试报告](/data/reports/1548561464.html)。
|
||||
|
||||
## 自定义
|
||||
|
||||
除了默认的报告样式，HttpRunner 还支持使用自定义的报告模板。
|
||||
|
||||
### 编写自定义模板（Jinja2格式）
|
||||
|
||||
自定义模板需要采用 [Jinja2][Jinja2] 的格式，其中可以使用的数据可参考[数据结构示例][summary_data]。
|
||||
|
||||
例如，我们需要在自定义模板中展示测试结果的统计数据，就可以采用如下方式进行描述：
|
||||
|
||||
```html
|
||||
<tr>
|
||||
<th>TOTAL</th>
|
||||
<th>SUCCESS</th>
|
||||
<th>FAILED</th>
|
||||
<th>ERROR</th>
|
||||
<th>SKIPPED</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>{{stat.testsRun}}</td>
|
||||
<td>{{stat.successes}}</td>
|
||||
<td>{{stat.failures}}</td>
|
||||
<td>{{stat.errors}}</td>
|
||||
<td>{{stat.skipped}}</td>
|
||||
</tr>
|
||||
```
|
||||
|
||||
在自定义报告模板时，可以参考 HttpRunner 的[默认报告模板][default_report]，要实现更复杂的模版功能，可参考 [Jinja2][Jinja2] 的使用文档。
|
||||
|
||||
### 使用自定义模板
|
||||
|
||||
使用自定义模版时，需要通过 `--report-template` 指定报告模板的路径，然后测试运行完成后，就会采用自定义的模板生成测试报告。
|
||||
|
||||
```bash
|
||||
$ hrun docs/data/demo-quickstart-2.yml --report-template /path/to/custom_report_template
|
||||
...
|
||||
同上，省略
|
||||
|
||||
INFO render with html report template: /path/to/custom_report_template
|
||||
INFO Start to render Html report ...
|
||||
INFO Generated Html report: reports/1532078874.html
|
||||
```
|
||||
|
||||
[Jinja2]: http://jinja.pocoo.org/docs/latest
|
||||
[default_report]: https://github.com/HttpRunner/HttpRunner/blob/master/httprunner/templates/report_template.html
|
||||
[summary_data]: /development/#_6
|
||||
|
||||
### 指定报告生成路径
|
||||
|
||||
默认情况下，生成的测试报告文件会位于项目根目录的 reports 文件夹中。如需指定生成报告的路径，可以使用 `--report-dir` 参数。
|
||||
|
||||
```bash
|
||||
$ hrun docs/data/demo-quickstart-2.yml --dirreport-name /other/path/
|
||||
...
|
||||
同上，省略
|
||||
|
||||
INFO Start to render Html report ...
|
||||
INFO Generated Html report: /other/path/1532078874.html
|
||||
```
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
## 金牌赞助商（Gold Sponsor）
|
||||
|
||||
[<img src="./assets/hogwarts.png" alt="霍格沃兹测试学院" width="400">](https://testing-studio.com)
|
||||
[<img src="/assets/hogwarts.png" alt="霍格沃兹测试学院" width="400">](https://testing-studio.com)
|
||||
|
||||
> [霍格沃兹测试学院](https://testing-studio.com) 是由测吧（北京）科技有限公司与知名软件测试社区 [TesterHome](https://testerhome.com/) 合作的高端教育品牌。由 BAT 一线**测试大咖执教**，提供**实战驱动**的接口自动化测试、移动自动化测试、性能测试、持续集成与 DevOps 等技术培训，以及测试开发优秀人才内推服务。[点击学习!](https://ke.qq.com/course/254956?flowToken=1014690)
|
||||
|
||||
@@ -12,7 +12,7 @@
|
||||
|
||||
### 开源服务赞助商（Open Source Sponsor）
|
||||
|
||||
[<img src="./assets/sentry-logo-black.svg" alt="Sentry" width="150">](https://sentry.io/_/open-source/)
|
||||
[<img src="/assets/sentry-logo-black.svg" alt="Sentry" width="150">](https://sentry.io/_/open-source/)
|
||||
|
||||
HttpRunner is in Sentry Sponsored plan.
|
||||
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
# NOTICE: Generated By HttpRunner. DO'NOT EDIT!
|
||||
# NOTICE: Generated By HttpRunner. DO NOT EDIT!
|
||||
# FROM: examples/httpbin/basic.yml
|
||||
|
||||
from httprunner import HttpRunner, TConfig, TStep
|
||||
|
||||
|
||||
@@ -9,6 +10,7 @@ class TestCaseBasic(HttpRunner):
|
||||
"name": "basic test with httpbin",
|
||||
"base_url": "https://httpbin.org/",
|
||||
"path": "examples/httpbin/basic_test.py",
|
||||
"variables": {},
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
# NOTICE: Generated By HttpRunner. DO'NOT EDIT!
|
||||
# NOTICE: Generated By HttpRunner. DO NOT EDIT!
|
||||
# FROM: examples/httpbin/hooks.yml
|
||||
|
||||
from httprunner import HttpRunner, TConfig, TStep
|
||||
|
||||
|
||||
@@ -11,6 +12,7 @@ class TestCaseHooks(HttpRunner):
|
||||
"setup_hooks": ["${hook_print(setup)}"],
|
||||
"teardown_hooks": ["${hook_print(teardown)}"],
|
||||
"path": "examples/httpbin/hooks_test.py",
|
||||
"variables": {},
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
# NOTICE: Generated By HttpRunner. DO'NOT EDIT!
|
||||
# NOTICE: Generated By HttpRunner. DO NOT EDIT!
|
||||
# FROM: examples/httpbin/load_image.yml
|
||||
|
||||
from httprunner import HttpRunner, TConfig, TStep
|
||||
|
||||
|
||||
@@ -9,6 +10,7 @@ class TestCaseLoadImage(HttpRunner):
|
||||
"name": "load images",
|
||||
"base_url": "${get_httpbin_server()}",
|
||||
"path": "examples/httpbin/load_image_test.py",
|
||||
"variables": {},
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user