GithubBackup/httprunner
1
0
Fork 0
mirror of https://github.com/httprunner/httprunner.git synced 2026-05-13 08:59:44 +08:00
Code Issues Packages Projects Releases Wiki Activity

docs: move docs to httprunner/docs repo

Browse Source
This commit is contained in:
debugtalk
2022-03-23 17:01:46 +08:00
parent 566cc9df82
commit 1b997c3817
20 changed files with 0 additions and 1615 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
docs/CNAME
View File

@@ -1 +0,0 @@
docs.httprunner.org

68
docs/README.md
View File

@@ -1,68 +0,0 @@
# HttpRunner
[![downloads](https://pepy.tech/badge/httprunner)](https://pepy.tech/project/httprunner)
[![unittest](https://github.com/httprunner/httprunner/workflows/unittest/badge.svg)](https://github.com/httprunner/httprunner/actions)
[![integration-test](https://github.com/httprunner/httprunner/workflows/integration_test/badge.svg)](https://github.com/httprunner/httprunner/actions)
[![codecov](https://codecov.io/gh/httprunner/httprunner/branch/master/graph/badge.svg)](https://codecov.io/gh/httprunner/httprunner)
[![pypi version](https://img.shields.io/pypi/v/httprunner.svg)](https://pypi.python.org/pypi/httprunner)
[![pyversions](https://img.shields.io/pypi/pyversions/httprunner.svg)](https://pypi.python.org/pypi/httprunner)
[![TesterHome](https://img.shields.io/badge/TTF-TesterHome-2955C5.svg)](https://testerhome.com/github_statistics)
*HttpRunner* is a simple & elegant, yet powerful HTTP(S) testing framework. Enjoy! ✨ 🚀 ✨
> 欢迎参加 HttpRunner [用户调研问卷][survey]，你的反馈将帮助 HttpRunner 更好地成长！
## Design Philosophy
- Convention over configuration
- ROI matters
- Embrace open source, leverage [`requests`][requests], [`pytest`][pytest], [`pydantic`][pydantic], [`allure`][allure] and [`locust`][locust].
## Key Features
- [x] Inherit all powerful features of [`requests`][requests], just have fun to handle HTTP(S) in human way.
- [x] Define testcase in YAML or JSON format, run with [`pytest`][pytest] in concise and elegant manner.
- [x] Record and generate testcases with [`HAR`][HAR] support.
- [x] Supports `variables`/`extract`/`validate`/`hooks` mechanisms to create extremely complex test scenarios.
- [x] With `debugtalk.py` plugin, any function can be used in any part of your testcase.
- [x] With [`jmespath`][jmespath], extract and validate json response has never been easier.
- [x] With [`pytest`][pytest], hundreds of plugins are readily available.
- [x] With [`allure`][allure], test report can be pretty nice and powerful.
- [x] With reuse of [`locust`][locust], you can run performance test without extra work.
- [x] CLI command supported, perfect combination with `CI/CD`.
## Sponsors
Thank you to all our sponsors! ✨🍰✨ ([become a sponsor](sponsors.md))
### 金牌赞助商（Gold Sponsor）
[<img src="assets/hogwarts.jpeg" alt="霍格沃兹测试开发学社" width="400">](https://ceshiren.com/)
> [霍格沃兹测试开发学社](http://qrcode.testing-studio.com/f?from=httprunner&url=https://ceshiren.com)是业界领先的测试开发技术高端教育品牌，隶属于[测吧（北京）科技有限公司](http://qrcode.testing-studio.com/f?from=httprunner&url=https://www.testing-studio.com) 。学院课程由一线大厂测试经理与资深测试开发专家参与研发，实战驱动。课程涵盖 web/app 自动化测试、接口测试、性能测试、安全测试、持续集成/持续交付/DevOps，测试左移&右移、精准测试、测试平台开发、测试管理等内容，帮助测试工程师实现测试开发技术转型。通过优秀的学社制度（奖学金、内推返学费、行业竞赛等多种方式）来实现学员、学社及用人企业的三方共赢。
> [进入测试开发技术能力测评!](http://qrcode.testing-studio.com/f?from=httprunner&url=https://ceshiren.com/t/topic/14940)
### 开源服务赞助商（Open Source Sponsor）
[<img src="assets/sentry-logo-black.svg" alt="Sentry" width="150">](https://sentry.io/_/open-source/)
HttpRunner is in Sentry Sponsored plan.
## Subscribe
关注 HttpRunner 的微信公众号，第一时间获得最新资讯。
<img src="assets/qrcode.jpg" alt="HttpRunner" width="200">
如果你期望加入 HttpRunner 核心用户群，请填写[用户调研问卷][survey]并留下你的联系方式，作者将拉你进群。
[requests]: http://docs.python-requests.org/en/master/
[pytest]: https://docs.pytest.org/
[pydantic]: https://pydantic-docs.helpmanual.io/
[locust]: http://locust.io/
[jmespath]: https://jmespath.org/
[allure]: https://docs.qameta.io/allure/
[HAR]: http://httparchive.org/
[survey]: https://wj.qq.com/s2/9699514/0d19/

BIN
docs/images/charles-capture-example-postman-echo.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 244 KiB

BIN
docs/images/charles-export-session.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 178 KiB

BIN
docs/images/charles-save-har.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 329 KiB

BIN
docs/images/httprunner-chain-call-config.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 206 KiB

BIN
docs/images/httprunner-chain-call-step-validate.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 308 KiB

BIN
docs/images/httprunner-formats.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 122 KiB

BIN
docs/images/httprunner-step-request-validate.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 369 KiB

BIN
docs/images/httprunner-testcase.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 119 KiB

68
docs/installation.md
View File

@@ -1,68 +0,0 @@
`HttpRunner` is developed with Python, it supports Python `3.6+` and most operating systems. Combination of Python `3.6/3.7/3.8` and `macOS/Linux/Windows` are tested continuously on [GitHub-Actions][github-actions].
## Installation
`HttpRunner` is available on [`PyPI`][PyPI] and can be installed through `pip`.
```bash
$ pip3 install httprunner
```
If you want to keep up with the latest version, you can install with github repository url.
```bash
$ pip3 install git+https://github.com/httprunner/httprunner.git@master
```
If you have installed `HttpRunner` before and want to upgrade to the latest version, you can use the `-U` option.
```bash
$ pip3 install -U httprunner
$ pip3 install -U git+https://github.com/httprunner/httprunner.git@master
```
## Check Installation
When HttpRunner is installed, 5 commands will be added in your system.
- `httprunner`: main command, used for all functions
- `hrun`: alias for `httprunner run`, used to run YAML/JSON/pytest testcases
- `hmake`: alias for `httprunner make`, used to convert YAML/JSON testcases to pytest files
- `har2case`: alias for `httprunner har2case`, used to convert HAR to YAML/JSON testcases
- `locusts`: used to run load test with [locust][locust]
To see `HttpRunner` version:
```text
$ httprunner -V # hrun -V
3.1.0
```
To see available options, run:
```text
$ httprunner -h
usage: httprunner [-h] [-V] {run,startproject,har2case,make} ...
One-stop solution for HTTP(S) testing.
positional arguments:
{run,startproject,har2case,make}
sub-command help
run Make HttpRunner testcases and run with pytest.
startproject Create a new project with template structure.
har2case Convert HAR(HTTP Archive) to YAML/JSON testcases for
HttpRunner.
make Convert YAML/JSON testcases to pytest cases.
optional arguments:
-h, --help show this help message and exit
-V, --version show version
```
> Notice: `locusts` is an individual command, for the reason to monkey patch ssl at beginning to avoid RecursionError when running locust.
[PyPI]: https://pypi.python.org/pypi
[github-actions]: https://github.com/httprunner/httprunner/actions
[locust]: http://locust.io/

3
docs/quickstart.md
View File

@@ -1,3 +0,0 @@
# Quick Start
First of all, remember HttpRunner is a simple yet powerful HTTP(S) testing framework. This document will help you to learn HttpRunner in 10 minutes.

26
docs/sponsors.md
View File

@@ -1,26 +0,0 @@
# 赞助商
感谢各位对 HttpRunner 的赞助支持！
## 金牌赞助商（Gold Sponsor）
[<img src="https://raw.githubusercontent.com/httprunner/httprunner/master/docs/assets/hogwarts.jpeg" alt="霍格沃兹测试开发学社" width="400">](https://ceshiren.com/)
> [霍格沃兹测试开发学社](http://qrcode.testing-studio.com/f?from=httprunner&url=https://ceshiren.com)是业界领先的测试开发技术高端教育品牌，隶属于[测吧（北京）科技有限公司](http://qrcode.testing-studio.com/f?from=httprunner&url=https://www.testing-studio.com) 。学院课程由一线大厂测试经理与资深测试开发专家参与研发，实战驱动。课程涵盖 web/app 自动化测试、接口测试、性能测试、安全测试、持续集成/持续交付/DevOps，测试左移&右移、精准测试、测试平台开发、测试管理等内容，帮助测试工程师实现测试开发技术转型。通过优秀的学社制度（奖学金、内推返学费、行业竞赛等多种方式）来实现学员、学社及用人企业的三方共赢。
> [进入测试开发技术能力测评!](http://qrcode.testing-studio.com/f?from=httprunner&url=https://ceshiren.com/t/topic/14940)
### 开源服务赞助商（Open Source Sponsor）
[<img src="https://docs.httprunner.org/assets/sentry-logo-black.svg" alt="Sentry" width="150">](https://sentry.io/_/open-source/)
HttpRunner is in Sentry Sponsored plan.
## 成为赞助商
如果你所在的公司或个人也想对 HttpRunner 进行赞助，可参考如下方案，具体可联系[项目作者](mailto:debugtalk@gmail.com)。
| 等级 | 金牌赞助商<br/>（Gold Sponsor） | 银牌赞助商<br/>（Silver Sponsor）| 个人赞赏 |
|:---:|:---:|:---:|:---:|
| 金额 | ￥20000/年 | ￥8000/年 | 任意 |
| 权益 | 公司 logo（大）和链接展示在 README.md<br/>200 字的宣传文案 | 公司 logo（中）和链接展示在 README.md<br/>80 字的宣传文案| 个人 ID 和链接展示在 sponsors.md |

79
docs/user/concepts.md
View File

@@ -1,79 +0,0 @@
## debugtalk.py
Based on the philosophy of `Convention over configuration`, each project should and could only have one `debugtalk.py` file. This file has multiple functions.
- As the root path anchor of the project, the relative paths in testcase, such as referencing testcases or CSV files, are all based on this root path
- Store custom python functions, the functions called in the testcase are all defined in this file
## variables priority
There are several different types of `variables`, and the priory can be confusing. The best way to avoid confusion is to use different variable names. However, if you must use the same variable names, you should understand the priority strategy.
### testcase
```yaml
config:
name: xxx
variables: # config variables
varA: "configA"
varB: "configB"
varC: "configC"
parameters: # parameter variables
varA: ["paramA1"]
varB: ["paramB1"]
teststeps:
-
name: step 1
variables: # step variables
varA: "step1A"
request:
url: /$varA/$varB/$varC # varA="step1A", varB="paramB1", varC="configC"
method: GET
extract: # extracted variables
varA: body.data.A # suppose varA="extractVarA"
varB: body.data.B # suppose varB="extractVarB"
-
name: step 2
varialbes:
varA: "step2A"
request:
url: /$varA/$varB/$varC # varA="step2A", varB="extractVarB", varC="configC"
method: GET
```
In a testcase, variables priority are in the following order:
- step variables > extracted variables, e.g. step 2, varA="step2A"
- parameter variables > config variables, e.g. step 1, varB="paramB1"
- extracted variables > parameter variables > config variables, e.g. step 2, varB="extractVarB"
- config variables are in the lowest priority, e.g. step 1/2, varC="configC"
### testsuite
```yaml
config:
name: xxx
variables: # testsuite config variables
varA: "configA"
varB: "configB"
varC: "configC"
testcases:
-
name: case 1
variables: # testcase variables
varA: "case1A"
testcase: /path/to/testcase1
export: ["varA", "varB"] # export variables
-
name: case 2
varialbes: # testcase variables
varA: "case2A"
testcase: /path/to/testcase2
```
In a testsuite, variables priority are in the following order:
- testcase variables > export variables > testsuite config variables > referenced testcase config variables

327
docs/user/gen_tests.md
View File

@@ -1,327 +0,0 @@
# Record & Generate testcase
## capture HTTP request and response
Before we write testcases, we should know the details of the API. It is a good choice to use a web debugging proxy tool like `Charles Proxy` to capture the HTTP traffic.
For example, the image below illustrates post form data to [`postman-echo.com`][postman-echo].
![](/images/charles-capture-example-postman-echo.png)
## export sessions to HAR file
Then we can select captured request & response and export sessions to HTTP archive (.har) file.
![](/images/charles-export-session.png)
![](/images/charles-save-har.png)
## generate testcase with har2case
When you get HAR file, you can use builtin command `har2case` to convert it to HttpRunner testcase.
### help
```text
$ har2case -h
usage: har2case har2case [-h] [-2y] [-2j] [--filter FILTER]
[--exclude EXCLUDE]
[har_source_file]
positional arguments:
har_source_file Specify HAR source file
optional arguments:
-h, --help show this help message and exit
-2y, --to-yml, --to-yaml
Convert to YAML format, if not specified, convert to
pytest format by default.
-2j, --to-json Convert to JSON format, if not specified, convert to
pytest format by default.
--filter FILTER Specify filter keyword, only url include filter string
will be converted.
--exclude EXCLUDE Specify exclude keyword, url that includes exclude
string will be ignored, multiple keywords can be
joined with '|'
```
### generate testcase (pytest)
Since HttpRunner `3.0.7`, `har2case` will convert HAR file to pytest by default, and it is extremely recommended to write and maintain testcases in pytest format instead of former `YAML/JSON` format.
```text
$ har2case har/postman-echo-post-form.har
2020-06-15 15:08:01.187 | INFO | httprunner.ext.har2case.core:gen_testcase:332 - Start to generate testcase from har/postman-echo-post-form.har
2020-06-15 15:08:01.187 | INFO | httprunner.ext.har2case.core:_make_testcase:323 - Extract info from HAR file and prepare for testcase.
2020-06-15 15:08:01.191 | INFO | httprunner.loader:load_dot_env_file:130 - Loading environment variables from /Users/debugtalk/Desktop/demo/.env
2020-06-15 15:08:01.191 | DEBUG | httprunner.utils:set_os_environ:32 - Set OS environment variable: USERNAME
2020-06-15 15:08:01.191 | DEBUG | httprunner.utils:set_os_environ:32 - Set OS environment variable: PASSWORD
2020-06-15 15:08:01.193 | INFO | httprunner.make:make_testcase:310 - start to make testcase: /Users/debugtalk/Desktop/demo/har/postman-echo-post-form.har
2020-06-15 15:08:01.193 | INFO | httprunner.make:make_testcase:383 - generated testcase: /Users/debugtalk/Desktop/demo/har/postman_echo_post_form_test.py
2020-06-15 15:08:01.194 | INFO | httprunner.make:format_pytest_with_black:147 - format pytest cases with black ...
reformatted /Users/debugtalk/Desktop/demo/har/postman_echo_post_form_test.py
All done! ✨ 🍰 ✨
1 file reformatted.
2020-06-15 15:08:01.469 | INFO | httprunner.ext.har2case.core:gen_testcase:353 - generated testcase: /Users/debugtalk/Desktop/demo/har/postman_echo_post_form_test.py
```
The generated pytest file is a standard Python file shown as below.
```python
# NOTE: Generated By HttpRunner v3.0.12
# FROM: har/postman-echo-post-form.har
from httprunner import HttpRunner, Config, Step, RunRequest, RunTestCase
class TestCasePostmanEchoPostForm(HttpRunner):
config = Config("testcase description").verify(False)
teststeps = [
Step(
RunRequest("/get")
.get("https://postman-echo.com/get")
.with_params(**{"foo1": "bar1", "foo2": "bar2"})
.with_headers(
**{
"User-Agent": "PostmanRuntime/7.24.1",
"Accept": "*/*",
"Cache-Control": "no-cache",
"Postman-Token": "6606343b-10e5-4165-a89f-6c301b762ce0",
"Host": "postman-echo.com",
"Accept-Encoding": "gzip, deflate, br",
"Connection": "keep-alive",
"Cookie": "sails.sid=s%3AQG_EVeNRw8k1xxZ6v_SG401VTpmJDSRu.fTAGx3JnZUT7S0c2%2FrD9cxUhQemIsm78nifYZYHpPCU",
}
)
.with_cookies(
**{
"sails.sid": "s%3AQG_EVeNRw8k1xxZ6v_SG401VTpmJDSRu.fTAGx3JnZUT7S0c2%2FrD9cxUhQemIsm78nifYZYHpPCU"
}
)
.validate()
.assert_equal("status_code", 200)
.assert_equal('headers."Content-Type"', "application/json; charset=utf-8")
.assert_equal(
"body.url", "https://postman-echo.com/get?foo1=bar1&foo2=bar2"
)
),
Step(
RunRequest("/post")
.post("https://postman-echo.com/post")
.with_headers(
**{
"User-Agent": "PostmanRuntime/7.24.1",
"Accept": "*/*",
"Cache-Control": "no-cache",
"Postman-Token": "3e408e9d-25ca-4b31-b04b-7f4898a8cd49",
"Host": "postman-echo.com",
"Accept-Encoding": "gzip, deflate, br",
"Connection": "keep-alive",
"Content-Type": "application/x-www-form-urlencoded",
"Content-Length": "19",
"Cookie": "sails.sid=s%3AQG_EVeNRw8k1xxZ6v_SG401VTpmJDSRu.fTAGx3JnZUT7S0c2%2FrD9cxUhQemIsm78nifYZYHpPCU",
}
)
.with_cookies(
**{
"sails.sid": "s%3AQG_EVeNRw8k1xxZ6v_SG401VTpmJDSRu.fTAGx3JnZUT7S0c2%2FrD9cxUhQemIsm78nifYZYHpPCU"
}
)
.with_data({"foo1": "bar1", "foo2": "bar2"})
.validate()
.assert_equal("status_code", 200)
.assert_equal('headers."Content-Type"', "application/json; charset=utf-8")
.assert_equal("body.data", "")
.assert_equal("body.url", "https://postman-echo.com/post")
),
]
if __name__ == "__main__":
TestCasePostmanEchoPostForm().test_start()
```
And it can be run with `hrun` command or the native `pytest` command. In fact, `hrun` is only a wrapper of `pytest`, thus the effect is basically the same.
```text
$ hrun har/postman_echo_post_form_test.py
2020-06-15 15:23:03.502 | INFO | httprunner.loader:load_dot_env_file:130 - Loading environment variables from /Users/debugtalk/Desktop/demo/.env
2020-06-15 15:23:03.502 | DEBUG | httprunner.utils:set_os_environ:32 - Set OS environment variable: USERNAME
2020-06-15 15:23:03.502 | DEBUG | httprunner.utils:set_os_environ:32 - Set OS environment variable: PASSWORD
2020-06-15 15:23:03.503 | INFO | httprunner.make:format_pytest_with_black:147 - format pytest cases with black ...
All done! ✨ 🍰 ✨
1 file left unchanged.
2020-06-15 15:23:03.662 | INFO | httprunner.cli:main_run:56 - start to run tests with pytest. HttpRunner version: 3.0.12
====================================================================== test session starts ======================================================================
platform darwin -- Python 3.7.5, pytest-5.4.2, py-1.8.1, pluggy-0.13.1
rootdir: /Users/debugtalk/Desktop/demo
plugins: metadata-1.9.0, allure-pytest-2.8.16, html-2.1.1
collected 1 item
har/postman_echo_post_form_test.py . [100%]
======================================================================= 1 passed in 2.60s =======================================================================
```
```text
$ pytest har/postman_echo_post_form_test.py
====================================================================== test session starts ======================================================================
platform darwin -- Python 3.7.5, pytest-5.4.2, py-1.8.1, pluggy-0.13.1
rootdir: /Users/debugtalk/Desktop/demo
plugins: metadata-1.9.0, allure-pytest-2.8.16, html-2.1.1
collected 1 item
har/postman_echo_post_form_test.py . [100%]
================================================================= 1 passed, 1 warning in 4.11s ==================================================================
```
### generate testcase (YAML/JSON)
Of course, you can also generate former `YAML/JSON` testcase format. Just add `-2y/--to-yml` or `-2j/--to-json` argument to `har2case`.
```text
$ har2case har/postman-echo-post-form.har -2j
2020-06-15 15:32:02.955 | INFO | httprunner.ext.har2case.core:gen_testcase:332 - Start to generate testcase from har/postman-echo-post-form.har
2020-06-15 15:32:02.955 | INFO | httprunner.ext.har2case.core:_make_testcase:323 - Extract info from HAR file and prepare for testcase.
2020-06-15 15:32:02.958 | INFO | httprunner.ext.har2case.utils:dump_json:122 - dump testcase to JSON format.
2020-06-15 15:32:02.959 | INFO | httprunner.ext.har2case.utils:dump_json:131 - Generate JSON testcase successfully: har/postman-echo-post-form.json
2020-06-15 15:32:02.959 | INFO | httprunner.ext.har2case.core:gen_testcase:353 - generated testcase: har/postman-echo-post-form.json
```
```json
{
"config": {
"name": "testcase description",
"variables": {},
"verify": false
},
"teststeps": [
{
"name": "/get",
"request": {
"url": "https://postman-echo.com/get",
"params": {
"foo1": "bar1",
"foo2": "bar2"
},
"method": "GET",
"cookies": {
"sails.sid": "s%3AQG_EVeNRw8k1xxZ6v_SG401VTpmJDSRu.fTAGx3JnZUT7S0c2%2FrD9cxUhQemIsm78nifYZYHpPCU"
},
"headers": {
"User-Agent": "PostmanRuntime/7.24.1",
"Accept": "*/*",
"Cache-Control": "no-cache",
"Postman-Token": "6606343b-10e5-4165-a89f-6c301b762ce0",
"Host": "postman-echo.com",
"Accept-Encoding": "gzip, deflate, br",
"Connection": "keep-alive",
"Cookie": "sails.sid=s%3AQG_EVeNRw8k1xxZ6v_SG401VTpmJDSRu.fTAGx3JnZUT7S0c2%2FrD9cxUhQemIsm78nifYZYHpPCU"
}
},
"validate": [
{
"eq": [
"status_code",
200
]
},
{
"eq": [
"headers.Content-Type",
"application/json; charset=utf-8"
]
},
{
"eq": [
"body.url",
"https://postman-echo.com/get?foo1=bar1&foo2=bar2"
]
}
]
},
{
"name": "/post",
"request": {
"url": "https://postman-echo.com/post",
"method": "POST",
"cookies": {
"sails.sid": "s%3AQG_EVeNRw8k1xxZ6v_SG401VTpmJDSRu.fTAGx3JnZUT7S0c2%2FrD9cxUhQemIsm78nifYZYHpPCU"
},
"headers": {
"User-Agent": "PostmanRuntime/7.24.1",
"Accept": "*/*",
"Cache-Control": "no-cache",
"Postman-Token": "3e408e9d-25ca-4b31-b04b-7f4898a8cd49",
"Host": "postman-echo.com",
"Accept-Encoding": "gzip, deflate, br",
"Connection": "keep-alive",
"Content-Type": "application/x-www-form-urlencoded",
"Content-Length": "19",
"Cookie": "sails.sid=s%3AQG_EVeNRw8k1xxZ6v_SG401VTpmJDSRu.fTAGx3JnZUT7S0c2%2FrD9cxUhQemIsm78nifYZYHpPCU"
},
"data": {
"foo1": "bar1",
"foo2": "bar2"
}
},
"validate": [
{
"eq": [
"status_code",
200
]
},
{
"eq": [
"headers.Content-Type",
"application/json; charset=utf-8"
]
},
{
"eq": [
"body.data",
""
]
},
{
"eq": [
"body.url",
"https://postman-echo.com/post"
]
}
]
}
]
}
```
The `YAML/JSON` testcase has the same info with `pytest` testcase, and you can run `YAML/JSON` testcase with `hrun` command.
```text
$ hrun har/postman-echo-post-form.json
2020-06-15 15:37:15.621 | INFO | httprunner.loader:load_dot_env_file:130 - Loading environment variables from /Users/debugtalk/Desktop/demo/.env
2020-06-15 15:37:15.622 | DEBUG | httprunner.utils:set_os_environ:32 - Set OS environment variable: USERNAME
2020-06-15 15:37:15.622 | DEBUG | httprunner.utils:set_os_environ:32 - Set OS environment variable: PASSWORD
2020-06-15 15:37:15.623 | INFO | httprunner.make:make_testcase:310 - start to make testcase: /Users/debugtalk/Desktop/demo/har/postman-echo-post-form.json
2020-06-15 15:37:15.625 | INFO | httprunner.make:make_testcase:383 - generated testcase: /Users/debugtalk/Desktop/demo/har/postman_echo_post_form_test.py
2020-06-15 15:37:15.625 | INFO | httprunner.make:format_pytest_with_black:147 - format pytest cases with black ...
reformatted /Users/debugtalk/Desktop/demo/har/postman_echo_post_form_test.py
All done! ✨ 🍰 ✨
1 file reformatted, 1 file left unchanged.
2020-06-15 15:37:15.962 | INFO | httprunner.cli:main_run:56 - start to run tests with pytest. HttpRunner version: 3.0.12
====================================================================== test session starts ======================================================================
platform darwin -- Python 3.7.5, pytest-5.4.2, py-1.8.1, pluggy-0.13.1
rootdir: /Users/debugtalk/Desktop/demo
plugins: metadata-1.9.0, allure-pytest-2.8.16, html-2.1.1
collected 1 item
har/postman_echo_post_form_test.py . [100%]
======================================================================= 1 passed in 2.03s =======================================================================
```
[postman-echo]: https://docs.postman-echo.com/?version=latest

168
docs/user/run_loadtest.md
View File

@@ -1,168 +0,0 @@
# Run load test
> Integrated since HttpRunner 3.1.0
With reuse of [`Locust`][Locust], you can run load test without extra work.
```text
$ locusts -V
locust 1.0.3
```
For full usage, you can run `locusts -h` to see help, and you will find that it is the same with `locust -h`.
The only difference is the `-f/--locustfile` argument. When you specify `-f` with a YAML/JSON testcase file, it will convert to pytest first and then run load test with HttpRunner's builtin locustfile(httprunner/ext/locust/locustfile.py).
```text
$ locusts -f examples/postman_echo/request_methods/request_with_variables.yml
2020-06-18 18:14:29.877 | INFO | httprunner.make:make_testcase:317 - start to make testcase: /Users/debugtalk/MyProjects/HttpRunner-dev/HttpRunner/examples/postman_echo/request_methods/request_with_variables.yml
2020-06-18 18:14:29.878 | INFO | httprunner.make:make_testcase:390 - generated testcase: /Users/debugtalk/MyProjects/HttpRunner-dev/HttpRunner/examples/postman_echo/request_methods/request_with_variables_test.py
2020-06-18 18:14:29.878 | INFO | httprunner.make:format_pytest_with_black:154 - format pytest cases with black ...
reformatted /Users/debugtalk/MyProjects/HttpRunner-dev/HttpRunner/examples/postman_echo/request_methods/request_with_variables_test.py
All done! ✨ 🍰 ✨
1 file reformatted, 1 file left unchanged.
[2020-06-18 18:14:30,241] LeodeMacBook-Pro.local/INFO/locust.main: Starting web interface at http://:8089
[2020-06-18 18:14:30,249] LeodeMacBook-Pro.local/INFO/locust.main: Starting Locust 1.0.3
```
In this case, you can reuse all features of [`Locust`][Locust].
```text
$ locusts -h
Usage: locust [OPTIONS] [UserClass ...]
Common options:
-h, --help show this help message and exit
-f LOCUSTFILE, --locustfile LOCUSTFILE
Python module file to import, e.g. '../other.py'.
Default: locustfile
--config CONFIG Config file path
-H HOST, --host HOST Host to load test in the following format:
http://10.21.32.33
-u NUM_USERS, --users NUM_USERS
Number of concurrent Locust users. Only used together
with --headless
-r HATCH_RATE, --hatch-rate HATCH_RATE
The rate per second in which users are spawned. Only
used together with --headless
-t RUN_TIME, --run-time RUN_TIME
Stop after the specified amount of time, e.g. (300s,
20m, 3h, 1h30m, etc.). Only used together with
--headless
-l, --list Show list of possible User classes and exit
Web UI options:
--web-host WEB_HOST Host to bind the web interface to. Defaults to '*'
(all interfaces)
--web-port WEB_PORT, -P WEB_PORT
Port on which to run web host
--headless Disable the web interface, and instead start the load
test immediately. Requires -u and -t to be specified.
--web-auth WEB_AUTH Turn on Basic Auth for the web interface. Should be
supplied in the following format: username:password
--tls-cert TLS_CERT Optional path to TLS certificate to use to serve over
HTTPS
--tls-key TLS_KEY Optional path to TLS private key to use to serve over
HTTPS
Master options:
Options for running a Locust Master node when running Locust distributed. A Master node need Worker nodes that connect to it before it can run load tests.
--master Set locust to run in distributed mode with this
process as master
--master-bind-host MASTER_BIND_HOST
Interfaces (hostname, ip) that locust master should
bind to. Only used when running with --master.
Defaults to * (all available interfaces).
--master-bind-port MASTER_BIND_PORT
Port that locust master should bind to. Only used when
running with --master. Defaults to 5557.
--expect-workers EXPECT_WORKERS
How many workers master should expect to connect
before starting the test (only when --headless used).
Worker options:
Options for running a Locust Worker node when running Locust distributed.
Only the LOCUSTFILE (-f option) need to be specified when starting a Worker, since other options such as -u, -r, -t are specified on the Master node.
--worker Set locust to run in distributed mode with this
process as worker
--master-host MASTER_HOST
Host or IP address of locust master for distributed
load testing. Only used when running with --worker.
Defaults to 127.0.0.1.
--master-port MASTER_PORT
The port to connect to that is used by the locust
master for distributed load testing. Only used when
running with --worker. Defaults to 5557.
Tag options:
Locust tasks can be tagged using the @tag decorator. These options let specify which tasks to include or exclude during a test.
-T [TAG [TAG ...]], --tags [TAG [TAG ...]]
List of tags to include in the test, so only tasks
with any matching tags will be executed
-E [TAG [TAG ...]], --exclude-tags [TAG [TAG ...]]
List of tags to exclude from the test, so only tasks
with no matching tags will be executed
Request statistics options:
--csv CSV_PREFIX Store current request stats to files in CSV format.
Setting this option will generate three files:
[CSV_PREFIX]_stats.csv, [CSV_PREFIX]_stats_history.csv
and [CSV_PREFIX]_failures.csv
--csv-full-history Store each stats entry in CSV format to
_stats_history.csv file
--print-stats Print stats in the console
--only-summary Only print the summary stats
--reset-stats Reset statistics once hatching has been completed.
Should be set on both master and workers when running
in distributed mode
Logging options:
--skip-log-setup Disable Locust's logging setup. Instead, the
configuration is provided by the Locust test or Python
defaults.
--loglevel LOGLEVEL, -L LOGLEVEL
Choose between DEBUG/INFO/WARNING/ERROR/CRITICAL.
Default is INFO.
--logfile LOGFILE Path to log file. If not set, log will go to
stdout/stderr
Step load options:
--step-load Enable Step Load mode to monitor how performance
metrics varies when user load increases. Requires
--step-users and --step-time to be specified.
--step-users STEP_USERS
User count to increase by step in Step Load mode. Only
used together with --step-load
--step-time STEP_TIME
Step duration in Step Load mode, e.g. (300s, 20m, 3h,
1h30m, etc.). Only used together with --step-load
Other options:
--show-task-ratio Print table of the User classes' task execution ratio
--show-task-ratio-json
Print json data of the User classes' task execution
ratio
--version, -V Show program's version number and exit
--exit-code-on-error EXIT_CODE_ON_ERROR
Sets the process exit code to use when a test result
contain any failure or error
-s STOP_TIMEOUT, --stop-timeout STOP_TIMEOUT
Number of seconds to wait for a simulated user to
complete any executing task before exiting. Default is
to terminate immediately. This parameter only needs to
be specified for the master process when running
Locust distributed.
User classes:
UserClass Optionally specify which User classes that should be
used (available User classes can be listed with -l or
--list)
```
Enjoy!
[Locust]: http://locust.io/

450
docs/user/run_testcase.md
View File

@@ -1,450 +0,0 @@
# Run Testcase
Once testcase is ready, you can run testcase with `hrun` command.
Notice, `hrun` is an command alias of `httprunner run`, they have the same effect.
```text
hrun = httprunner run
```
## run testcases in diverse ways
`HttpRunner` can run testcases in diverse ways.
You can run single testcase by specifying testcase file path.
```text
$ hrun path/to/testcase1
```
You can also run several testcases by specifying multiple testcase file paths.
```text
$ hrun path/to/testcase1 path/to/testcase2
```
If you want to run testcases of a whole project, you can achieve this goal by specifying the project folder path.
```text
$ hrun path/to/testcase_folder/
```
## run YAML/JSON testcases
If your testcases are written in YAML/JSON format, `hrun` will firstly convert YAML/JSON testcases to pytest(python) files, and run with `pytest` command.
That is to say,
```text
hrun = make + pytest
```
In most cases, the generated pytest files are in the same folder next to origin YAML/JSON files, with the same file name except adding `_test` suffix and replace extension `.yml/yaml/.json` with `.py`.
```text
/path/to/example.yml => /path/to/example_test.py
```
However, if the testcase folder name or file name contains symbols like dot, hyphen or space, these symbols will be replaced with underscore in order to avoid syntax error in python class importing (testcase reference). Also, folder/file name starts with digit will be adding a prefix `T` because python module and class name can not be started with digit.
```text
path 1/a.b-2/3.yml => path_1/a_b_2/T3_test.py
```
## run pytest testcases
If your testcases are written in pytest format, or you want to run pytest files converted from YAML/JSON testcases, `hrun` and `pytest` commands are both okay. What you need to remember is that `hrun` only wraps `pytest`, thus all the arguments of `pytest` can be used with `hrun`.
```text
$ hrun -h
usage: hrun [options] [file_or_dir] [file_or_dir] [...]
positional arguments:
file_or_dir
general:
-k EXPRESSION only run tests which match the given substring expression. An expression is a python evaluatable expression where all names are
substring-matched against test names and their parent classes. Example: -k 'test_method or test_other' matches all test functions and
classes whose name contains 'test_method' or 'test_other', while -k 'not test_method' matches those that don't contain 'test_method' in
their names. -k 'not test_method and not test_other' will eliminate the matches. Additionally keywords are matched to classes and
functions containing extra names in their 'extra_keyword_matches' set, as well as functions which have names assigned directly to them.
The matching is case-insensitive.
-m MARKEXPR only run tests matching given mark expression. example: -m 'mark1 and not mark2'.
--markers show markers (builtin, plugin and per-project ones).
-x, --exitfirst exit instantly on first error or failed test.
--maxfail=num exit after first num failures or errors.
--strict-markers, --strict
markers not registered in the `markers` section of the configuration file raise errors.
-c file load configuration from `file` instead of trying to locate one of the implicit configuration files.
--continue-on-collection-errors
Force test execution even if collection errors occur.
--rootdir=ROOTDIR Define root directory for tests. Can be relative path: 'root_dir', './root_dir', 'root_dir/another_dir/'; absolute path:
'/home/user/root_dir'; path with variables: '$HOME/root_dir'.
--fixtures, --funcargs
show available fixtures, sorted by plugin appearance (fixtures with leading '_' are only shown with '-v')
--fixtures-per-test show fixtures per test
--import-mode={prepend,append}
prepend/append to sys.path when importing test modules, default is to prepend.
--pdb start the interactive Python debugger on errors or KeyboardInterrupt.
--pdbcls=modulename:classname
start a custom interactive Python debugger on errors. For example: --pdbcls=IPython.terminal.debugger:TerminalPdb
--trace Immediately break when running each test.
--capture=method per-test capturing method: one of fd|sys|no|tee-sys.
-s shortcut for --capture=no.
--runxfail report the results of xfail tests as if they were not marked
--lf, --last-failed rerun only the tests that failed at the last run (or all if none failed)
--ff, --failed-first run all tests but run the last failures first. This may re-order tests and thus lead to repeated fixture setup/teardown
--nf, --new-first run tests from new files first, then the rest of the tests sorted by file mtime
--cache-show=[CACHESHOW]
show cache contents, don't perform collection or tests. Optional argument: glob (default: '*').
--cache-clear remove all cache contents at start of test run.
--lfnf={all,none}, --last-failed-no-failures={all,none}
which tests to run with no previously (known) failures.
--sw, --stepwise exit on test failure and continue from last failing test next time
--stepwise-skip ignore the first failing test but stop on the next failing test
--allure-severities=SEVERITIES_SET
Comma-separated list of severity names. Tests only with these severities will be run. Possible values are: blocker, critical, normal,
minor, trivial.
--allure-epics=EPICS_SET
Comma-separated list of epic names. Run tests that have at least one of the specified feature labels.
--allure-features=FEATURES_SET
Comma-separated list of feature names. Run tests that have at least one of the specified feature labels.
--allure-stories=STORIES_SET
Comma-separated list of story names. Run tests that have at least one of the specified story labels.
--allure-link-pattern=LINK_TYPE:LINK_PATTERN
Url pattern for link type. Allows short links in test, like 'issue-1'. Text will be formatted to full url with python str.format().
reporting:
--durations=N show N slowest setup/test durations (N=0 for all).
-v, --verbose increase verbosity.
-q, --quiet decrease verbosity.
--verbosity=VERBOSE set verbosity. Default is 0.
-r chars show extra test summary info as specified by chars: (f)ailed, (E)rror, (s)kipped, (x)failed, (X)passed, (p)assed, (P)assed with output,
(a)ll except passed (p/P), or (A)ll. (w)arnings are enabled by default (see --disable-warnings), 'N' can be used to reset the list.
(default: 'fE').
--disable-warnings, --disable-pytest-warnings
disable warnings summary
-l, --showlocals show locals in tracebacks (disabled by default).
--tb=style traceback print mode (auto/long/short/line/native/no).
--show-capture={no,stdout,stderr,log,all}
Controls how captured stdout/stderr/log is shown on failed tests. Default is 'all'.
--full-trace don't cut any tracebacks (default is to cut).
--color=color color terminal output (yes/no/auto).
--pastebin=mode send failed|all info to bpaste.net pastebin service.
--junit-xml=path create junit-xml style report file at given path.
--junit-prefix=str prepend prefix to classnames in junit-xml output
--result-log=path DEPRECATED path for machine-readable result log.
--html=path create html report file at given path.
--self-contained-html
create a self-contained html file containing all necessary styles, scripts, and images - this means that the report may not render or
function where CSP restrictions are in place (see https://developer.mozilla.org/docs/Web/Security/CSP)
--css=path append given css file content to report style file.
collection:
--collect-only, --co only collect tests, don't execute them.
--pyargs try to interpret all arguments as python packages.
--ignore=path ignore path during collection (multi-allowed).
--ignore-glob=path ignore path pattern during collection (multi-allowed).
--deselect=nodeid_prefix
deselect item (via node id prefix) during collection (multi-allowed).
--confcutdir=dir only load conftest.py's relative to specified dir.
--noconftest Don't load any conftest.py files.
--keep-duplicates Keep duplicate tests.
--collect-in-virtualenv
Don't ignore tests in a local virtualenv directory
--doctest-modules run doctests in all .py modules
--doctest-report={none,cdiff,ndiff,udiff,only_first_failure}
choose another output format for diffs on doctest failure
--doctest-glob=pat doctests file matching pattern, default: test*.txt
--doctest-ignore-import-errors
ignore doctest ImportErrors
--doctest-continue-on-failure
for a given doctest, continue to run after the first failure
test session debugging and configuration:
--basetemp=dir base temporary directory for this test run.(warning: this directory is removed if it exists)
-V, --version display pytest version and information about plugins.
-h, --help show help message and configuration info
-p name early-load given plugin module name or entry point (multi-allowed). To avoid loading of plugins, use the `no:` prefix, e.g. `no:doctest`.
--trace-config trace considerations of conftest.py files.
--debug store internal tracing debug information in 'pytestdebug.log'.
-o OVERRIDE_INI, --override-ini=OVERRIDE_INI
override ini option with "option=value" style, e.g. `-o xfail_strict=True -o cache_dir=cache`.
--assert=MODE Control assertion debugging tools. 'plain' performs no assertion debugging. 'rewrite' (the default) rewrites assert statements in test
modules on import to provide assert expression information.
--setup-only only setup fixtures, do not execute tests.
--setup-show show setup of fixtures while executing tests.
--setup-plan show what fixtures and tests would be executed but don't execute anything.
pytest-warnings:
-W PYTHONWARNINGS, --pythonwarnings=PYTHONWARNINGS
set which warnings to report, see -W option of python itself.
logging:
--no-print-logs disable printing caught logs on failed tests.
--log-level=LEVEL level of messages to catch/display. Not set by default, so it depends on the root/parent log handler's effective level, where it is
"WARNING" by default.
--log-format=LOG_FORMAT
log format as used by the logging module.
--log-date-format=LOG_DATE_FORMAT
log date format as used by the logging module.
--log-cli-level=LOG_CLI_LEVEL
cli logging level.
--log-cli-format=LOG_CLI_FORMAT
log format as used by the logging module.
--log-cli-date-format=LOG_CLI_DATE_FORMAT
log date format as used by the logging module.
--log-file=LOG_FILE path to a file when logging will be written to.
--log-file-level=LOG_FILE_LEVEL
log file logging level.
--log-file-format=LOG_FILE_FORMAT
log format as used by the logging module.
--log-file-date-format=LOG_FILE_DATE_FORMAT
log date format as used by the logging module.
--log-auto-indent=LOG_AUTO_INDENT
Auto-indent multiline messages passed to the logging module. Accepts true|on, false|off or an integer.
reporting:
--alluredir=DIR Generate Allure report in the specified directory (may not exist)
--clean-alluredir Clean alluredir folder if it exists
--allure-no-capture Do not attach pytest captured logging/stdout/stderr to report
custom options:
--metadata=key value additional metadata.
--metadata-from-json=METADATA_FROM_JSON
additional metadata from a json string.
[pytest] ini-options in the first pytest.ini|tox.ini|setup.cfg file found:
markers (linelist): markers for test functions
empty_parameter_set_mark (string):
default marker for empty parametersets
norecursedirs (args): directory patterns to avoid for recursion
testpaths (args): directories to search for tests when no files or directories are given in the command line.
usefixtures (args): list of default fixtures to be used with this project
python_files (args): glob-style file patterns for Python test module discovery
python_classes (args):
prefixes or glob names for Python test class discovery
python_functions (args):
prefixes or glob names for Python test function and method discovery
disable_test_id_escaping_and_forfeit_all_rights_to_community_support (bool):
disable string escape non-ascii characters, might cause unwanted side effects(use at your own risk)
console_output_style (string):
console output: "classic", or with additional progress information ("progress" (percentage) | "count").
xfail_strict (bool): default for the strict parameter of xfail markers when not given explicitly (default: False)
enable_assertion_pass_hook (bool):
Enables the pytest_assertion_pass hook.Make sure to delete any previously generated pyc cache files.
junit_suite_name (string):
Test suite name for JUnit report
junit_logging (string):
Write captured log messages to JUnit report: one of no|log|system-out|system-err|out-err|all
junit_log_passing_tests (bool):
Capture log information for passing tests to JUnit report:
junit_duration_report (string):
Duration time to report: one of total|call
junit_family (string):
Emit XML for schema: one of legacy|xunit1|xunit2
doctest_optionflags (args):
option flags for doctests
doctest_encoding (string):
encoding used for doctest files
cache_dir (string): cache directory path.
filterwarnings (linelist):
Each line specifies a pattern for warnings.filterwarnings. Processed after -W/--pythonwarnings.
log_print (bool): default value for --no-print-logs
log_level (string): default value for --log-level
log_format (string): default value for --log-format
log_date_format (string):
default value for --log-date-format
log_cli (bool): enable log display during test run (also known as "live logging").
log_cli_level (string):
default value for --log-cli-level
log_cli_format (string):
default value for --log-cli-format
log_cli_date_format (string):
default value for --log-cli-date-format
log_file (string): default value for --log-file
log_file_level (string):
default value for --log-file-level
log_file_format (string):
default value for --log-file-format
log_file_date_format (string):
default value for --log-file-date-format
log_auto_indent (string):
default value for --log-auto-indent
faulthandler_timeout (string):
Dump the traceback of all threads if a test takes more than TIMEOUT seconds to finish. Not available on Windows.
addopts (args): extra command line options
minversion (string): minimally required pytest version
render_collapsed (bool):
Open the report with all rows collapsed. Useful for very large reports
environment variables:
PYTEST_ADDOPTS extra command line options
PYTEST_PLUGINS comma-separated plugins to load during startup
PYTEST_DISABLE_PLUGIN_AUTOLOAD set to disable plugin auto-loading
PYTEST_DEBUG set to enable debug tracing of pytest's internals
to see available markers type: pytest --markers
to see available fixtures type: pytest --fixtures
(shown according to specified file_or_dir or current dir if not specified; fixtures with leading '_' are only shown with the '-v' option
```
## execution logs
By default, `hrun` will not print details of request and response data.
```text
$ hrun examples/postman_echo/request_methods/request_with_functions.yml
2020-06-17 15:39:41.041 | INFO | httprunner.make:make_testcase:317 - start to make testcase: /Users/debugtalk/MyProjects/HttpRunner-dev/HttpRunner/examples/postman_echo/request_methods/request_with_functions.yml
2020-06-17 15:39:41.042 | INFO | httprunner.make:make_testcase:390 - generated testcase: /Users/debugtalk/MyProjects/HttpRunner-dev/HttpRunner/examples/postman_echo/request_methods/request_with_functions_test.py
2020-06-17 15:39:41.042 | INFO | httprunner.make:format_pytest_with_black:154 - format pytest cases with black ...
reformatted /Users/debugtalk/MyProjects/HttpRunner-dev/HttpRunner/examples/postman_echo/request_methods/request_with_functions_test.py
All done! ✨ 🍰 ✨
1 file reformatted, 1 file left unchanged.
2020-06-17 15:39:41.315 | INFO | httprunner.cli:main_run:56 - start to run tests with pytest. HttpRunner version: 3.0.13
====================================================================== test session starts ======================================================================
platform darwin -- Python 3.7.5, pytest-5.4.2, py-1.8.1, pluggy-0.13.1
rootdir: /Users/debugtalk/MyProjects/HttpRunner-dev/HttpRunner
plugins: metadata-1.9.0, allure-pytest-2.8.16, html-2.1.1
collected 1 item
examples/postman_echo/request_methods/request_with_functions_test.py . [100%]
======================================================================= 1 passed in 2.98s =======================================================================
```
If you want to view details of request & response data, extraction and validation, you can add an argument `-s` (shortcut for `--capture=no`).
```text
$ hrun -s examples/postman_echo/request_methods/request_with_functions.yml
2020-06-17 15:42:54.369 | INFO | httprunner.make:make_testcase:317 - start to make testcase: /Users/debugtalk/MyProjects/HttpRunner-dev/HttpRunner/examples/postman_echo/request_methods/request_with_functions.yml
2020-06-17 15:42:54.369 | INFO | httprunner.make:make_testcase:390 - generated testcase: /Users/debugtalk/MyProjects/HttpRunner-dev/HttpRunner/examples/postman_echo/request_methods/request_with_functions_test.py
2020-06-17 15:42:54.370 | INFO | httprunner.make:format_pytest_with_black:154 - format pytest cases with black ...
reformatted /Users/debugtalk/MyProjects/HttpRunner-dev/HttpRunner/examples/postman_echo/request_methods/request_with_functions_test.py
All done! ✨ 🍰 ✨
1 file reformatted, 1 file left unchanged.
2020-06-17 15:42:54.699 | INFO | httprunner.cli:main_run:56 - start to run tests with pytest. HttpRunner version: 3.0.13
====================================================================== test session starts ======================================================================
platform darwin -- Python 3.7.5, pytest-5.4.2, py-1.8.1, pluggy-0.13.1
rootdir: /Users/debugtalk/MyProjects/HttpRunner-dev/HttpRunner
plugins: metadata-1.9.0, allure-pytest-2.8.16, html-2.1.1
collected 1 item
examples/postman_echo/request_methods/request_with_functions_test.py 2020-06-17 15:42:55.017 | INFO | httprunner.runner:test_start:435 - Start to run testcase: request methods testcase with functions, TestCase ID: cc404c49-000f-485c-b4c1-ac3367a053fe
2020-06-17 15:42:55.018 | INFO | httprunner.runner:__run_step:278 - run step begin: get with params >>>>>>
2020-06-17 15:42:56.326 | DEBUG | httprunner.client:log_print:40 -
================== request details ==================
method : GET
url : https://postman-echo.com/get?foo1=bar11&foo2=bar21&sum_v=3
headers : {
"User-Agent": "HttpRunner/3.0.13",
"Accept-Encoding": "gzip, deflate",
"Accept": "*/*",
"Connection": "keep-alive",
"HRUN-Request-ID": "HRUN-cc404c49-000f-485c-b4c1-ac3367a053fe-775018",
"Content-Length": "2",
"Content-Type": "application/json"
}
cookies : {}
body : {}
2020-06-17 15:42:56.327 | DEBUG | httprunner.client:log_print:40 -
================== response details ==================
status_code : 200
headers : {
"Date": "Wed, 17 Jun 2020 07:42:56 GMT",
"Content-Type": "application/json; charset=utf-8",
"Content-Length": "477",
"Connection": "keep-alive",
"ETag": "W/\"1dd-2JtBYPcnh8D6fqLz8KFn16Oq1R0\"",
"Vary": "Accept-Encoding",
"set-cookie": "sails.sid=s%3A6J_EtUk3nkL_C2xtx-NtAXrlA5wPxEgk.gIO2yBbtvGWIIgQ%2F2mZhMkU669G3F60cvLAPWbwyoGM; Path=/; HttpOnly"
}
cookies : {
"sails.sid": "s%3A6J_EtUk3nkL_C2xtx-NtAXrlA5wPxEgk.gIO2yBbtvGWIIgQ%2F2mZhMkU669G3F60cvLAPWbwyoGM"
}
encoding : utf-8
content_type : application/json; charset=utf-8
body : {
"args": {
"foo1": "bar11",
"foo2": "bar21",
"sum_v": "3"
},
"headers": {
"x-forwarded-proto": "https",
"x-forwarded-port": "443",
"host": "postman-echo.com",
"x-amzn-trace-id": "Root=1-5ee9c980-d8e98cc72a26ef24f5819ce3",
"content-length": "2",
"user-agent": "HttpRunner/3.0.13",
"accept-encoding": "gzip, deflate",
"accept": "*/*",
"hrun-request-id": "HRUN-cc404c49-000f-485c-b4c1-ac3367a053fe-775018",
"content-type": "application/json"
},
"url": "https://postman-echo.com/get?foo1=bar11&foo2=bar21&sum_v=3"
}
2020-06-17 15:42:56.328 | INFO | httprunner.client:request:203 - status_code: 200, response_time(ms): 1307.33 ms, response_length: 477 bytes
2020-06-17 15:42:56.328 | INFO | httprunner.response:extract:152 - extract mapping: {'foo3': 'bar21'}
2020-06-17 15:42:56.328 | INFO | httprunner.response:validate:209 - assert status_code equal 200(int) ==> pass
2020-06-17 15:42:56.329 | INFO | httprunner.response:validate:209 - assert body.args.foo1 equal bar11(str) ==> pass
2020-06-17 15:42:56.329 | INFO | httprunner.response:validate:209 - assert body.args.sum_v equal 3(str) ==> pass
2020-06-17 15:42:56.329 | INFO | httprunner.response:validate:209 - assert body.args.foo2 equal bar21(str) ==> pass
2020-06-17 15:42:56.330 | INFO | httprunner.runner:__run_step:290 - run step end: get with params <<<<<<
<Omit>
2020-06-17 15:42:57.019 | INFO | httprunner.runner:test_start:444 - generate testcase log: /Users/debugtalk/MyProjects/HttpRunner-dev/HttpRunner/examples/postman_echo/logs/cc404c49-000f-485c-b4c1-ac3367a053fe.run.log
.
======================================================================= 1 passed in 2.13s =======================================================================
```
Also, an execution log file will be generated for each testcase, located in `<ProjectRootDir>/logs/TestCaseID.run.log`.
## TestCase ID & Request ID
For the sake of troubleshooting, each testcase will generate a unique ID (uuid4), and each request headers will be added a `HRUN-Request-ID` field with testcase ID automatically.
```text
HRUN-Request-ID = "HRUN-<TestCase ID>-<timestamp_six_digits>"
timestamp_six_digits = str(int(time.time() * 1000))[-6:])
```
In other words, all requests in one testcase will have the same `HRUN-Request-ID` prefix, and each request will have a unique `HRUN-Request-ID` suffix.
## Client & Server IP:PORT
Sometimes, logging remote server IP and port can be great helpful for trouble shooting, especially when there are multiple servers and we want to checkout which one returns error.
Since version `3.0.13`, HttpRunner will log client & server IP:Port in debug level.
```text
2020-06-18 11:32:38.366 | INFO | httprunner.runner:__run_step:278 - run step begin: post raw text >>>>>>
2020-06-18 11:32:38.687 | DEBUG | httprunner.client:request:187 - client IP: 10.90.205.63, Port: 62802
2020-06-18 11:32:38.687 | DEBUG | httprunner.client:request:195 - server IP: 34.233.204.163, Port: 443
```
as well as testcase summary.
```text
"address": {
"client_ip": "10.90.205.63",
"client_port": 62802,
"server_ip": "34.233.204.163",
"server_port": 443
},
```
## arguments for v2.x compatibility
Besides all the arguments of `pytest`, `hrun` also has several other arguments to keep compatibility with HttpRunner v2.x.
- `--failfast`: has no effect, this argument will be removed automatically
- `--report-file`: specify html report file path, this argument will be replaced with `--html --self-contained-html` and generate html report with `pytest-html` plugin
- `--save-tests`: if set, HttpRunner v3.x will create a pytest conftest.py file containing session fixture to aggregate each testcase's summary and dumps to summary.json

102
docs/user/scaffold.md
View File

@@ -1,102 +0,0 @@
# Scaffold
If you want to create a new project, you can use the scaffold to startup quickly.
## help
```text
$ httprunner startproject -h
usage: httprunner startproject [-h] [project_name]
positional arguments:
project_name Specify new project name.
optional arguments:
-h, --help show this help message and exit
```
## create new project
The only argument you need to specify is the project name.
```text
$ httprunner startproject demo
2020-06-15 11:53:25.498 | INFO | httprunner.scaffold:create_scaffold:37 - Create new project: demo
Project Root Dir: /Users/debugtalk/MyProjects/HttpRunner-dev/HttpRunner/demo
created folder: demo
created folder: demo/har
created folder: demo/testcases
created folder: demo/reports
created file: demo/testcases/demo_testcase_request.yml
created file: demo/testcases/demo_testcase_ref.yml
created file: demo/debugtalk.py
created file: demo/.env
created file: demo/.gitignore
$ tree demo -a
demo
├── .env
├── .gitignore
├── debugtalk.py
├── har
├── reports
└── testcases
├── demo_testcase_ref.yml
└── demo_testcase_request.yml
3 directories, 5 files
```
If you specify a project name that already exists, you will get a warning.
```text
$ httprunner startproject demo
2020-06-15 11:55:03.192 | WARNING | httprunner.scaffold:create_scaffold:32 - Project demo exists, please specify a new project name.
$ tree demo -a
demo
├── .env
├── .gitignore
├── debugtalk.py
├── har
├── reports
└── testcases
├── demo_testcase_ref.yml
└── demo_testcase_request.yml
3 directories, 5 files
```
## run scaffold project
The scaffold project has several valid testcases, so you can run tests without any edit.
```text
$ hrun demo
2020-06-15 11:57:15.883 | INFO | httprunner.loader:load_dot_env_file:130 - Loading environment variables from /Users/debugtalk/MyProjects/HttpRunner-dev/HttpRunner/demo/.env
2020-06-15 11:57:15.883 | DEBUG | httprunner.utils:set_os_environ:32 - Set OS environment variable: USERNAME
2020-06-15 11:57:15.884 | DEBUG | httprunner.utils:set_os_environ:32 - Set OS environment variable: PASSWORD
2020-06-15 11:57:15.885 | INFO | httprunner.make:make_testcase:310 - start to make testcase: /Users/debugtalk/MyProjects/HttpRunner-dev/HttpRunner/demo/testcases/demo_testcase_ref.yml
2020-06-15 11:57:15.898 | INFO | httprunner.make:make_testcase:310 - start to make testcase: /Users/debugtalk/MyProjects/HttpRunner-dev/HttpRunner/demo/testcases/demo_testcase_request.yml
2020-06-15 11:57:15.899 | INFO | httprunner.make:make_testcase:383 - generated testcase: /Users/debugtalk/MyProjects/HttpRunner-dev/HttpRunner/demo/testcases/demo_testcase_request_test.py
2020-06-15 11:57:15.900 | INFO | httprunner.make:make_testcase:383 - generated testcase: /Users/debugtalk/MyProjects/HttpRunner-dev/HttpRunner/demo/testcases/demo_testcase_ref_test.py
2020-06-15 11:57:15.911 | INFO | httprunner.make:make_testcase:310 - start to make testcase: /Users/debugtalk/MyProjects/HttpRunner-dev/HttpRunner/demo/testcases/demo_testcase_request.yml
2020-06-15 11:57:15.912 | INFO | httprunner.make:__ensure_project_meta_files:128 - copy .env to /Users/debugtalk/MyProjects/HttpRunner-dev/HttpRunner/demo/_env
2020-06-15 11:57:15.912 | INFO | httprunner.make:format_pytest_with_black:147 - format pytest cases with black ...
reformatted /Users/debugtalk/MyProjects/HttpRunner-dev/HttpRunner/demo/testcases/demo_testcase_ref_test.py
reformatted /Users/debugtalk/MyProjects/HttpRunner-dev/HttpRunner/demo/testcases/demo_testcase_request_test.py
All done! ✨ 🍰 ✨
2 files reformatted, 1 file left unchanged.
2020-06-15 11:57:16.299 | INFO | httprunner.cli:main_run:56 - start to run tests with pytest. HttpRunner version: 3.0.12
====================================================================== test session starts ======================================================================
platform darwin -- Python 3.7.5, pytest-5.4.2, py-1.8.1, pluggy-0.13.1
rootdir: /Users/debugtalk/MyProjects/HttpRunner-dev/HttpRunner
plugins: metadata-1.9.0, allure-pytest-2.8.16, html-2.1.1
collected 2 items
demo/testcases/demo_testcase_request_test.py . [ 50%]
demo/testcases/demo_testcase_ref_test.py . [100%]
======================================================================= 2 passed in 6.87s =======================================================================
```

55
docs/user/testing_report.md
View File

@@ -1,55 +0,0 @@
# Testing Report
Benefit from the integration of `pytest`, HttpRunner v3.x can make use of all the pytest plugins, including testing report plugins like `pytest-html` and `allure-pytest`.
## builtin html report
`pytest-html` plugin comes with HttpRunner installation. When you want to generate a html report for testcase execution, you can add a command argument `--html`.
```text
$ hrun /path/to/testcase --html=report.html
```
If you want to create a self-contained report, which is a single HTML file that can be more convenient when sharing results, you can add another command argument `--self-contained-html`.
```text
$ hrun /path/to/testcase --html=report.html --self-contained-html
```
You can refer to [`pytest-html`](https://pypi.org/project/pytest-html/) for more details.
## allure report
`allure-pytest` is an optional dependency for HttpRunner, thus if you want to generate allure report, you should install `allure-pytest` plugin separately.
```text
$ pip3 install "allure-pytest"
```
Or you can install HttpRunner with allure extra package.
```text
$ pip3 install "httprunner[allure]"
```
Once `allure-pytest` is ready, the following arguments can be used with `hrun/pytest` command.
- `--alluredir=DIR`: Generate Allure report in the specified directory (may not exist)
- `--clean-alluredir`: Clean alluredir folder if it exists
- `--allure-no-capture`: Do not attach pytest captured logging/stdout/stderr to report
To enable Allure listener to collect results during the test execution simply add `--alluredir` option and provide path to the folder where results should be stored. E.g.:
```text
$ hrun /path/to/testcase --alluredir=/tmp/my_allure_results
```
To see the actual report after your tests have finished, you need to use Allure commandline utility to generate report from the results.
```text
$ allure serve /tmp/my_allure_results
```
This command will show you generated report in your default browser.
You can refer to [`allure-pytest`](https://docs.qameta.io/allure/#_pytest) for more details.

268
docs/user/write_testcase.md
View File

@@ -1,268 +0,0 @@
# Write Testcase
HttpRunner v3.x supports three testcase formats, `pytest`, `YAML` and `JSON`. It is extremely recommended to write and maintain testcases in `pytest` format instead of former `YAML/JSON` format.
The format relations are illustrated as below:
![](/images/httprunner-formats.png)
## record & generate testcase
If the SUT (system under test) is ready, the most efficient way is to capture HTTP traffic first and then generate testcases with HAR file. Refer to [`Record & Generate testcase`](/user/gen_tests/) for more details.
Based on the generated pytest testcase, you can then do some adjustment as needed, thus you need to know the details of testcase format.
## testcase structure
Each testcase is a subclass of `HttpRunner`, and must have two class attributes: `config` and `teststeps`.
- config: configure testcase level settings, including `base_url`, `verify`, `variables`, `export`.
- teststeps: list of teststep (`List[Step]`), each step is corresponding to a API request or another testcase reference call. Besides, `variables`/`extract`/`validate`/`hooks` mechanisms are supported to create extremely complex test scenarios.
```python
from httprunner import HttpRunner, Config, Step, RunRequest, RunTestCase
class TestCaseRequestWithFunctions(HttpRunner):
config = (
Config("request methods testcase with functions")
.variables(
**{
"foo1": "config_bar1",
"foo2": "config_bar2",
"expect_foo1": "config_bar1",
"expect_foo2": "config_bar2",
}
)
.base_url("https://postman-echo.com")
.verify(False)
.export(*["foo3"])
)
teststeps = [
Step(
RunRequest("get with params")
.with_variables(
**{"foo1": "bar11", "foo2": "bar21", "sum_v": "${sum_two(1, 2)}"}
)
.get("/get")
.with_params(**{"foo1": "$foo1", "foo2": "$foo2", "sum_v": "$sum_v"})
.with_headers(**{"User-Agent": "HttpRunner/${get_httprunner_version()}"})
.extract()
.with_jmespath("body.args.foo2", "foo3")
.validate()
.assert_equal("status_code", 200)
.assert_equal("body.args.foo1", "bar11")
.assert_equal("body.args.sum_v", "3")
.assert_equal("body.args.foo2", "bar21")
),
Step(
RunRequest("post form data")
.with_variables(**{"foo2": "bar23"})
.post("/post")
.with_headers(
**{
"User-Agent": "HttpRunner/${get_httprunner_version()}",
"Content-Type": "application/x-www-form-urlencoded",
}
)
.with_data("foo1=$foo1&foo2=$foo2&foo3=$foo3")
.validate()
.assert_equal("status_code", 200)
.assert_equal("body.form.foo1", "$expect_foo1")
.assert_equal("body.form.foo2", "bar23")
.assert_equal("body.form.foo3", "bar21")
),
]
if __name__ == "__main__":
TestCaseRequestWithFunctions().test_start()
```
## chain call
One of the most awesome features of HttpRunner v3.x is `chain call`, with which you do not need to remember any testcase format details and you can get intelligent completion when you write testcases in IDE.
![](/images/httprunner-chain-call-config.png)
![](/images/httprunner-chain-call-step-validate.png)
## config
Each testcase should have one `config` part, in which you can configure testcase level settings.
### name (required)
Specify testcase name. This will be displayed in execution log and test report.
### base_url (optional)
Specify common schema and host part of the SUT, e.g. `https://postman-echo.com`. If `base_url` is specified, url in teststep can only set relative path part. This is especially useful if you want to switch between different SUT environments.
### variables (optional)
Specify common variables of testcase. Each teststep can reference config variable which is not set in step variables. In other words, step variables have higher priority than config variables.
### verify (optional)
Specify whether to verify the server’s TLS certificate. This is especially useful if we want to record HTTP traffic of testcase execution, because SSLError will be occurred if verify is not set or been set to True.
> SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1076)'))
### export (optional)
Specify the exported session variables of testcase. Consider each testcase as a black box, config `variables` is the input part, and config `export` is the output part. In particular, when a testcase is referenced in another testcase's step, and will be extracted some session variables to be used in subsequent teststeps, then the extracted session variables should be configured in config `export` part.
## teststeps
Each testcase should have one or multiple ordered test steps (`List[Step]`), each step is corresponding to a API request or another testcase reference call.
![](/images/httprunner-testcase.png)
> Notice: The concept of API in HttpRunner v2.x has been deprecated for simplification. You can consider API as a testcase that has only one request step.
### RunRequest(name)
`RunRequest` is used in a step to make request to API and do some extraction or validations for response.
The argument `name` of RunRequest is used to specify teststep name, which will be displayed in execution log and test report.
#### .with_variables
Specify teststep variables. The variables of each step are independent, thus if you want to share variables in multiple steps, you should define variables in config variables. Besides, the step variables will override the ones that have the same name in config variables.
#### .method(url)
Specify HTTP method and the url of SUT. These are corresponding to `method` and `url` arguments of [`requests.request`][requests.request].
If `base_url` is set in config, url can only set relative path part.
#### .with_params
Specify query string for the request url. This is corresponding to the `params` argument of [`requests.request`][requests.request].
#### .with_headers
Specify HTTP headers for the request. This is corresponding to the `headers` argument of [`requests.request`][requests.request].
#### .with_cookies
Specify HTTP request cookies. This is corresponding to the `cookies` argument of [`requests.request`][requests.request].
#### .with_data
Specify HTTP request body. This is corresponding to the `data` argument of [`requests.request`][requests.request].
#### .with_json
Specify HTTP request body in json. This is corresponding to the `json` argument of [`requests.request`][requests.request].
#### extract
##### .with_jmespath
Extract JSON response body with [jmespath][jmespath].
> with_jmespath(jmes_path: Text, var_name: Text)
- jmes_path: jmespath expression, refer to [JMESPath Tutorial][jmespath_tutorial] for more details
- var_name: the variable name that stores extracted value, it can be referenced by subsequent test steps
#### validate
##### .assert_XXX
Extract JSON response body with [jmespath][jmespath] and validate with expected value.
> assert_XXX(jmes_path: Text, expected_value: Any, message: Text = "")
- jmes_path: jmespath expression, refer to [JMESPath Tutorial][jmespath_tutorial] for more details
- expected_value: the specified expected value, variable or function reference can also be used here
- message (optional): used to indicate assertion error reason
The image below shows HttpRunner builtin validators.
![](/images/httprunner-step-request-validate.png)
### RunTestCase(name)
`RunTestCase` is used in a step to reference another testcase call.
The argument `name` of RunTestCase is used to specify teststep name, which will be displayed in execution log and test report.
#### .with_variables
Same with RunRequest's `.with_variables`.
#### .call
Specify referenced testcase class.
#### .export
Specify session variable names to export from referenced testcase. The exported variables can be referenced by subsequent test steps.
```python
import os
import sys
sys.path.insert(0, os.getcwd())
from httprunner import HttpRunner, Config, Step, RunRequest, RunTestCase
from examples.postman_echo.request_methods.request_with_functions_test import (
TestCaseRequestWithFunctions as RequestWithFunctions,
)
class TestCaseRequestWithTestcaseReference(HttpRunner):
config = (
Config("request methods testcase: reference testcase")
.variables(
**{
"foo1": "testsuite_config_bar1",
"expect_foo1": "testsuite_config_bar1",
"expect_foo2": "config_bar2",
}
)
.base_url("https://postman-echo.com")
.verify(False)
)
teststeps = [
Step(
RunTestCase("request with functions")
.with_variables(
**{"foo1": "testcase_ref_bar1", "expect_foo1": "testcase_ref_bar1"}
)
.call(RequestWithFunctions)
.export(*["foo3"])
),
Step(
RunRequest("post form data")
.with_variables(**{"foo1": "bar1"})
.post("/post")
.with_headers(
**{
"User-Agent": "HttpRunner/${get_httprunner_version()}",
"Content-Type": "application/x-www-form-urlencoded",
}
)
.with_data("foo1=$foo1&foo2=$foo3")
.validate()
.assert_equal("status_code", 200)
.assert_equal("body.form.foo1", "bar1")
.assert_equal("body.form.foo2", "bar21")
),
]
if __name__ == "__main__":
TestCaseRequestWithTestcaseReference().test_start()
```
[requests.request]: https://requests.readthedocs.io/en/master/api/#requests.request
[jmespath]: https://jmespath.org/
[jmespath_tutorial]: https://jmespath.org/tutorial.html