GithubBackup/httprunner
1
0
Fork 0
mirror of https://github.com/httprunner/httprunner.git synced 2026-07-09 22:44:05 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #2 from debugtalk/master

Browse Source 
pull debugtalk
This commit is contained in:
firefoxwang
2017-10-25 22:41:46 -05:00
committed by GitHub
parent 52fa69cc5a f5dcdaa5c5
commit 4d28e2fc86
26 changed files with 149 additions and 223 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
.travis.yml
View File

@@ -2,7 +2,6 @@ sudo: false
language: python language: python
python: python:
- 2.7 - 2.7
- 3.3
- 3.4 - 3.4
- 3.5 - 3.5
- 3.6 - 3.6

83
README.md
View File

@@ -5,7 +5,7 @@
## Design Philosophy ## Design Philosophy
Take full reuse of Python's existing powerful libraries: [`Requests`][requests], [`unittest`][unittest] and [`Locust`][Locust]. And achieve the goal of API automation test, production environment monitoring, and API performance test, with a concise and elegant manner. Take full reuse of Python's existing powerful libraries: [`Requests`][requests], [`unittest`][unittest] and [`Locust`][Locust]. And achieve the goal of API automation test, production environment monitoring, and API performance test, with a concise and elegant manner.
## Key Features ## Key Features
@@ -15,9 +15,8 @@ Take full reuse of Python's existing powerful libraries: [`Requests`][requests],
- With `debugtalk.py` plugin, module functions can be auto-discovered in recursive upward directories. - With `debugtalk.py` plugin, module functions can be auto-discovered in recursive upward directories.
- Testcases can be run in diverse ways, with single testset, multiple testsets, or entire project folder. - Testcases can be run in diverse ways, with single testset, multiple testsets, or entire project folder.
- Test report is concise and clear, with detailed log records. See [`PyUnitReport`][PyUnitReport]. - Test report is concise and clear, with detailed log records. See [`PyUnitReport`][PyUnitReport].
- Perfect combination with [Jenkins][Jenkins], running continuous integration test and production environment monitoring. Send mail notification with [`jenkins-mail-py`][jenkins-mail-py].
- With reuse of [`Locust`][Locust], you can run performance test without extra work. - With reuse of [`Locust`][Locust], you can run performance test without extra work.
- It is extensible to facilitate the implementation of web platform with [`Flask`][flask] framework. - CLI command supported, perfect combination with [Jenkins][Jenkins].
[*`Background Introduction (中文版)`*](docs/background-CN.md) | [*`Feature Descriptions (中文版)`*](docs/feature-descriptions-CN.md) [*`Background Introduction (中文版)`*](docs/background-CN.md) | [*`Feature Descriptions (中文版)`*](docs/feature-descriptions-CN.md)
@@ -39,7 +38,7 @@ To ensure the installation or upgrade is successful, you can execute command `at
```text ```text
$ ate -V $ ate -V
ApiTestEngine version: 0.7.4 ApiTestEngine version: 0.7.5
``` ```
Execute the command `ate -h` to view command help. Execute the command `ate -h` to view command help.
@@ -47,10 +46,10 @@ Execute the command `ate -h` to view command help.
```text ```text
$ ate -h $ ate -h
usage: ate [-h] [-V] [--log-level LOG_LEVEL] [--report-name REPORT_NAME] usage: ate [-h] [-V] [--log-level LOG_LEVEL] [--report-name REPORT_NAME]
[--failfast] [--failfast] [--startproject STARTPROJECT]
[testset_paths [testset_paths ...]] [testset_paths [testset_paths ...]]
Api Test Engine. ApiTestEngine.
positional arguments: positional arguments:
testset_paths testset file path testset_paths testset file path
@@ -63,65 +62,8 @@ optional arguments:
--report-name REPORT_NAME --report-name REPORT_NAME
Specify report name, default is generated time. Specify report name, default is generated time.
--failfast Stop the test run on the first error or failure. --failfast Stop the test run on the first error or failure.
``` --startproject STARTPROJECT
Specify new project name.
### use `jenkins-mail-py` plugin
If you want to use `ApiTestEngine` with Jenkins, you may need to send mail notification, and [`jenkins-mail-py`][jenkins-mail-py] will be of great help.
To install mail helper, run this command in your terminal:
```text
$ pip install -U git+https://github.com/debugtalk/jenkins-mail-py.git#egg=jenkins-mail-py
$ ate -V
jenkins-mail-py version: 0.2.5
ApiTestEngine version: 0.7.4
```
With [`jenkins-mail-py`][jenkins-mail-py] installed, you can see more optional arguments.
```text
$ ate -h
usage: ate [-h] [-V] [--log-level LOG_LEVEL] [--report-name REPORT_NAME]
[--failfast] [--mailgun-api-id MAILGUN_API_ID]
[--mailgun-api-key MAILGUN_API_KEY] [--email-sender EMAIL_SENDER]
[--email-recepients EMAIL_RECEPIENTS] [--mail-subject MAIL_SUBJECT]
[--mail-content MAIL_CONTENT] [--jenkins-job-name JENKINS_JOB_NAME]
[--jenkins-job-url JENKINS_JOB_URL]
[--jenkins-build-number JENKINS_BUILD_NUMBER]
[testset_paths [testset_paths ...]]
Api Test Engine.
positional arguments:
testset_paths testset file path
optional arguments:
-h, --help show this help message and exit
-V, --version show version
--log-level LOG_LEVEL
Specify logging level, default is INFO.
--report-name REPORT_NAME
Specify report name, default is generated time.
--failfast Stop the test run on the first error or failure.
--mailgun-api-id MAILGUN_API_ID
Specify mailgun api id.
--mailgun-api-key MAILGUN_API_KEY
Specify mailgun api key.
--email-sender EMAIL_SENDER
Specify email sender.
--email-recepients EMAIL_RECEPIENTS
Specify email recepients.
--mail-subject MAIL_SUBJECT
Specify email subject.
--mail-content MAIL_CONTENT
Specify email content.
--jenkins-job-name JENKINS_JOB_NAME
Specify jenkins job name.
--jenkins-job-url JENKINS_JOB_URL
Specify jenkins job url.
--jenkins-build-number JENKINS_BUILD_NUMBER
Specify jenkins build number.
``` ```
## Write testcases ## Write testcases
@@ -133,7 +75,7 @@ And here is testset example of typical scenario: get `token` at the beginning, a
```yaml ```yaml
- config: - config:
name: "create user testsets." name: "create user testsets."
variable_binds: variables:
- user_agent: 'iOS/10.3' - user_agent: 'iOS/10.3'
- device_sn: ${gen_random_string(15)} - device_sn: ${gen_random_string(15)}
- os_platform: 'ios' - os_platform: 'ios'
@@ -156,7 +98,7 @@ And here is testset example of typical scenario: get `token` at the beginning, a
app_version: $app_version app_version: $app_version
json: json:
sign: ${get_sign($user_agent, $device_sn, $os_platform, $app_version)} sign: ${get_sign($user_agent, $device_sn, $os_platform, $app_version)}
extract_binds: extractors:
- token: content.token - token: content.token
validators: validators:
- {"check": "status_code", "comparator": "eq", "expected": 200} - {"check": "status_code", "comparator": "eq", "expected": 200}
@@ -207,8 +149,8 @@ When you do continuous integration test or production environment monitoring wit
```text ```text
$ ate filepath/testcase.yml --report-name ${BUILD_NUMBER} \ $ ate filepath/testcase.yml --report-name ${BUILD_NUMBER} \
--mailgun-api-id samples.mailgun.org \ --mailgun-smtp-username "qa@debugtalk.com" \
--mailgun-api-key key-3ax6xnjp29jd6fds4gc373sgvjxteol0 \ --mailgun-smtp-password "12345678" \
--email-sender excited@samples.mailgun.org \ --email-sender excited@samples.mailgun.org \
--email-recepients ${MAIL_RECEPIENTS} \ --email-recepients ${MAIL_RECEPIENTS} \
--jenkins-job-name ${JOB_NAME} \ --jenkins-job-name ${JOB_NAME} \
@@ -260,7 +202,7 @@ Enjoy!
## Supported Python Versions ## Supported Python Versions
Python `2.7`, `3.3`, `3.4`, `3.5` and `3.6`. Python `2.7`, `3.4`, `3.5` and `3.6`.
`ApiTestEngine` has been tested on `macOS`, `Linux` and `Windows` platforms. `ApiTestEngine` has been tested on `macOS`, `Linux` and `Windows` platforms.
@@ -292,5 +234,4 @@ $ python main-locust -h
[flask]: http://flask.pocoo.org/ [flask]: http://flask.pocoo.org/
[PyUnitReport]: https://github.com/debugtalk/PyUnitReport [PyUnitReport]: https://github.com/debugtalk/PyUnitReport
[Jenkins]: https://jenkins.io/index.html [Jenkins]: https://jenkins.io/index.html
[jenkins-mail-py]: https://github.com/debugtalk/jenkins-mail-py.git
[quickstart]: docs/quickstart.md [quickstart]: docs/quickstart.md

2
ate/__init__.py
View File

@@ -1 +1 @@
__version__ = '0.7.4' __version__ = '0.7.5'

16
ate/cli.py
View File

@@ -35,12 +35,6 @@ def main_ate():
'--startproject', '--startproject',
help="Specify new project name.") help="Specify new project name.")
try:
from jenkins_mail_py import MailgunHelper
mailer = MailgunHelper(parser)
except ImportError:
mailer = None
args = parser.parse_args() args = parser.parse_args()
if args.version: if args.version:
@@ -63,7 +57,7 @@ def main_ate():
report name is ignored, use generated time instead.") report name is ignored, use generated time instead.")
results = {} results = {}
subject = "SUCCESS" success = True
for testset_path in set(args.testset_paths): for testset_path in set(args.testset_paths):
@@ -86,16 +80,12 @@ def main_ate():
}) })
if len(result.successes) != result.testsRun: if len(result.successes) != result.testsRun:
subject = "FAILED" success = False
for task in task_suite.tasks: for task in task_suite.tasks:
task.print_output() task.print_output()
flag_code = 0 if subject == "SUCCESS" else 1 return 0 if success is True else 1
if mailer and mailer.config_ready:
mailer.send_mail(subject, results, flag_code)
return flag_code
def main_locust(): def main_locust():
""" Performance test with locust: parse command line options and run commands. """ Performance test with locust: parse command line options and run commands.

15
ate/context.py
View File

@@ -54,8 +54,9 @@ class Context(object):
or config_dict.get('import_module_functions', []) or config_dict.get('import_module_functions', [])
self.import_module_items(module_items, level) self.import_module_items(module_items, level)
variable_binds = config_dict.get('variable_binds', OrderedDict()) variables = config_dict.get('variables') \
self.bind_variables(variable_binds, level) or config_dict.get('variable_binds', OrderedDict())
self.bind_variables(variables, level)
def import_requires(self, modules): def import_requires(self, modules):
""" import required modules dynamically """ import required modules dynamically
@@ -92,11 +93,11 @@ class Context(object):
imported_variables_dict = utils.filter_module(imported_module, "variable") imported_variables_dict = utils.filter_module(imported_module, "variable")
self.bind_variables(imported_variables_dict, level) self.bind_variables(imported_variables_dict, level)
def bind_variables(self, variable_binds, level="testcase"): def bind_variables(self, variables, level="testcase"):
""" bind variables to testset context or current testcase context. """ bind variables to testset context or current testcase context.
variables in testset context can be used in all testcases of current test suite. variables in testset context can be used in all testcases of current test suite.
@param (list or OrderDict) variable_binds, variable can be value or custom function. @param (list or OrderDict) variables, variable can be value or custom function.
if value is function, it will be called and bind result to variable. if value is function, it will be called and bind result to variable.
e.g. e.g.
OrderDict({ OrderDict({
@@ -106,10 +107,10 @@ class Context(object):
"md5": "${gen_md5($TOKEN, $json, $random)}" "md5": "${gen_md5($TOKEN, $json, $random)}"
}) })
""" """
if isinstance(variable_binds, list): if isinstance(variables, list):
variable_binds = utils.convert_to_order_dict(variable_binds) variables = utils.convert_to_order_dict(variables)
for variable_name, value in variable_binds.items(): for variable_name, value in variables.items():
variable_evale_value = self.testcase_parser.parse_content_with_bindings(value) variable_evale_value = self.testcase_parser.parse_content_with_bindings(value)
if level == "testset": if level == "testset":

8
ate/response.py
View File

@@ -56,9 +56,9 @@ class ResponseObject(object):
except AttributeError: except AttributeError:
raise exception.ParseResponseError("failed to extract bind variable in response!") raise exception.ParseResponseError("failed to extract bind variable in response!")
def extract_response(self, extract_binds): def extract_response(self, extractors):
""" extract content from requests.Response """ extract content from requests.Response
@param (list) extract_binds @param (list) extractors
[ [
{"resp_status_code": "status_code"}, {"resp_status_code": "status_code"},
{"resp_headers_content_type": "headers.content-type"}, {"resp_headers_content_type": "headers.content-type"},
@@ -68,11 +68,11 @@ class ResponseObject(object):
@return (OrderDict) variable binds ordered dict @return (OrderDict) variable binds ordered dict
""" """
extracted_variables_mapping = OrderedDict() extracted_variables_mapping = OrderedDict()
extract_binds_order_dict = utils.convert_to_order_dict(extract_binds) extract_binds_order_dict = utils.convert_to_order_dict(extractors)
for key, field in extract_binds_order_dict.items(): for key, field in extract_binds_order_dict.items():
if not isinstance(field, utils.string_type): if not isinstance(field, utils.string_type):
raise exception.ParamsError("invalid extract_binds in testcase extract_binds!") raise exception.ParamsError("invalid extractors in testcase!")
extracted_variables_mapping[key] = self.extract_field(field) extracted_variables_mapping[key] = self.extract_field(field)

32
ate/runner.py
View File

@@ -23,7 +23,7 @@ class Runner(object):
"requires": [], # optional "requires": [], # optional
"function_binds": {}, # optional "function_binds": {}, # optional
"import_module_items": [], # optional "import_module_items": [], # optional
"variable_binds": [], # optional "variables": [], # optional
"request": { "request": {
"base_url": "http://127.0.0.1:5000", "base_url": "http://127.0.0.1:5000",
"headers": { "headers": {
@@ -37,7 +37,7 @@ class Runner(object):
"requires": [], # optional "requires": [], # optional
"function_binds": {}, # optional "function_binds": {}, # optional
"import_module_items": [], # optional "import_module_items": [], # optional
"variable_binds": [], # optional "variables": [], # optional
"request": { "request": {
"url": "/api/get-token", "url": "/api/get-token",
"method": "POST", "method": "POST",
@@ -71,9 +71,9 @@ class Runner(object):
{ {
"name": "testcase description", "name": "testcase description",
"times": 3, "times": 3,
"requires": [], # optional, override "requires": [], # optional, override
"function_binds": {}, # optional, override "function_binds": {}, # optional, override
"variable_binds": [], # optional, override "variables": [], # optional, override
"request": { "request": {
"url": "http://127.0.0.1:5000/api/users/1000", "url": "http://127.0.0.1:5000/api/users/1000",
"method": "POST", "method": "POST",
@@ -84,7 +84,7 @@ class Runner(object):
}, },
"body": '{"name": "user", "password": "123456"}' "body": '{"name": "user", "password": "123456"}'
}, },
"extract_binds": [], # optional "extractors": [], # optional
"validators": [], # optional "validators": [], # optional
"setup": [], # optional "setup": [], # optional
"teardown": [] # optional "teardown": [] # optional
@@ -100,7 +100,9 @@ class Runner(object):
raise exception.ParamsError("URL or METHOD missed!") raise exception.ParamsError("URL or METHOD missed!")
run_times = int(testcase.get("times", 1)) run_times = int(testcase.get("times", 1))
extract_binds = testcase.get("extract_binds", []) extractors = testcase.get("extractors") \
or testcase.get("extractor") \
or testcase.get("extract_binds", [])
validators = testcase.get("validators", []) validators = testcase.get("validators", [])
setup_actions = testcase.get("setup", []) setup_actions = testcase.get("setup", [])
teardown_actions = testcase.get("teardown", []) teardown_actions = testcase.get("teardown", [])
@@ -115,7 +117,7 @@ class Runner(object):
resp = self.http_client_session.request(url=url, method=method, **parsed_request) resp = self.http_client_session.request(url=url, method=method, **parsed_request)
resp_obj = response.ResponseObject(resp) resp_obj = response.ResponseObject(resp)
extracted_variables_mapping = resp_obj.extract_response(extract_binds) extracted_variables_mapping = resp_obj.extract_response(extractors)
self.context.bind_variables(extracted_variables_mapping, level="testset") self.context.bind_variables(extracted_variables_mapping, level="testset")
resp_obj.validate(validators, self.context.get_testcase_variables_mapping()) resp_obj.validate(validators, self.context.get_testcase_variables_mapping())
@@ -134,22 +136,22 @@ class Runner(object):
"name": "testset description", "name": "testset description",
"requires": [], "requires": [],
"function_binds": {}, "function_binds": {},
"variable_binds": [], "variables": [],
"request": {} "request": {}
}, },
"testcases": [ "testcases": [
{ {
"name": "testcase description", "name": "testcase description",
"variable_binds": [], # optional, override "variables": [], # optional, override
"request": {}, "request": {},
"extract_binds": {}, # optional "extractors": {}, # optional
"validators": {} # optional "validators": {} # optional
}, },
testcase12 testcase12
] ]
} }
(dict) variables_mapping: (dict) variables_mapping:
passed in variables mapping, it will override variable_binds in config block passed in variables mapping, it will override variables in config block
@return (dict) test result of testset @return (dict) test result of testset
{ {
@@ -160,9 +162,9 @@ class Runner(object):
success = True success = True
config_dict = testset.get("config", {}) config_dict = testset.get("config", {})
variable_binds = config_dict.get("variable_binds", []) variables = config_dict.get("variables", [])
variables_mapping = variables_mapping or {} variables_mapping = variables_mapping or {}
config_dict["variable_binds"] = utils.override_variables_binds(variable_binds, variables_mapping) config_dict["variables"] = utils.override_variables_binds(variables, variables_mapping)
self.init_config(config_dict, level="testset") self.init_config(config_dict, level="testset")
testcases = testset.get("testcases", []) testcases = testset.get("testcases", [])
@@ -187,7 +189,7 @@ class Runner(object):
- absolute/relative folder path - absolute/relative folder path
- list/set container with file(s) and/or folder(s) - list/set container with file(s) and/or folder(s)
(dict) mapping: (dict) mapping:
passed in variables mapping, it will override variable_binds in config block passed in variables mapping, it will override variables in config block
""" """
success = True success = True
mapping = mapping or {} mapping = mapping or {}

26
ate/testcase.py
View File

@@ -329,14 +329,14 @@ def substitute_variables_with_mapping(content, mapping):
class TestcaseParser(object): class TestcaseParser(object):
def __init__(self, variables_binds={}, functions_binds={}, file_path=None): def __init__(self, variables={}, functions={}, file_path=None):
self.bind_variables(variables_binds) self.bind_variables(variables)
self.bind_functions(functions_binds) self.bind_functions(functions)
self.file_path = file_path self.file_path = file_path
def bind_variables(self, variables_binds): def bind_variables(self, variables):
""" bind variables to current testcase parser """ bind variables to current testcase parser
@param (dict) variables_binds, variables binds mapping @param (dict) variables, variables binds mapping
{ {
"authorization": "a83de0ff8d2e896dbd8efb81ba14e17d", "authorization": "a83de0ff8d2e896dbd8efb81ba14e17d",
"random": "A2dEx", "random": "A2dEx",
@@ -344,24 +344,24 @@ class TestcaseParser(object):
"uuid": 1000 "uuid": 1000
} }
""" """
self.variables_binds = variables_binds self.variables = variables
def bind_functions(self, functions_binds): def bind_functions(self, functions):
""" bind functions to current testcase parser """ bind functions to current testcase parser
@param (dict) functions_binds, functions binds mapping @param (dict) functions, functions binds mapping
{ {
"add_two_nums": lambda a, b=1: a + b "add_two_nums": lambda a, b=1: a + b
} }
""" """
self.functions_binds = functions_binds self.functions = functions
def get_bind_item(self, item_type, item_name): def get_bind_item(self, item_type, item_name):
if item_type == "function": if item_type == "function":
if item_name in self.functions_binds: if item_name in self.functions:
return self.functions_binds[item_name] return self.functions[item_name]
elif item_type == "variable": elif item_type == "variable":
if item_name in self.variables_binds: if item_name in self.variables:
return self.variables_binds[item_name] return self.variables[item_name]
else: else:
raise exception.ParamsError("bind item should only be function or variable.") raise exception.ParamsError("bind item should only be function or variable.")

16
ate/utils.py
View File

@@ -369,18 +369,18 @@ def update_ordered_dict(ordered_dict, override_mapping):
return ordered_dict return ordered_dict
def override_variables_binds(variable_binds, new_mapping): def override_variables_binds(variables, new_mapping):
""" convert variable_binds in testcase to ordered mapping, with new_mapping overrided """ convert variables in testcase to ordered mapping, with new_mapping overrided
""" """
if isinstance(variable_binds, list): if isinstance(variables, list):
variable_binds_ordered_dict = convert_to_order_dict(variable_binds) variables_ordered_dict = convert_to_order_dict(variables)
elif isinstance(variable_binds, OrderedDict): elif isinstance(variables, OrderedDict):
variable_binds_ordered_dict = variable_binds variables_ordered_dict = variables
else: else:
raise exception.ParamsError("variable_binds error!") raise exception.ParamsError("variables error!")
return update_ordered_dict( return update_ordered_dict(
variable_binds_ordered_dict, variables_ordered_dict,
new_mapping new_mapping
) )

4
docs/extraction-and-validation.md
View File

@@ -24,7 +24,7 @@ Suppose we get the following HTTP response.
} }
``` ```
In `extract_binds` and `validators`, we can do chain operation to extract data field in HTTP response. In `extractors` and `validators`, we can do chain operation to extract data field in HTTP response.
For instance, if we want to get `Content-Type` in response headers, then we can specify `headers.content-type`; if we want to get `first_name` in response content, we can specify `content.person.name.first_name`. For instance, if we want to get `Content-Type` in response headers, then we can specify `headers.content-type`; if we want to get `first_name` in response content, we can specify `content.person.name.first_name`.
@@ -46,7 +46,7 @@ content.person.cities.1
``` ```
```yaml ```yaml
extract_binds: extractors:
- content_type: headers.content-type - content_type: headers.content-type
- first_name: content.person.name.first_name - first_name: content.person.name.first_name
validators: validators:

18
docs/quickstart.md
View File

@@ -120,7 +120,7 @@ To fix this problem, we should correlate `token` field in the second API test ca
app_version: 2.8.6 app_version: 2.8.6
json: json:
sign: 19067cf712265eb5426db8d3664026c1ccea02b9 sign: 19067cf712265eb5426db8d3664026c1ccea02b9
extract_binds: extractors:
- token: content.token - token: content.token
validators: validators:
- {"check": "status_code", "comparator": "eq", "expected": 200} - {"check": "status_code", "comparator": "eq", "expected": 200}
@@ -142,7 +142,7 @@ To fix this problem, we should correlate `token` field in the second API test ca
- {"check": "content.success", "comparator": "eq", "expected": true} - {"check": "content.success", "comparator": "eq", "expected": true}
``` ```
As you see, the `token` field is no longer hardcoded, instead it is extracted from the first API request with `extract_binds` mechanism. In the meanwhile, it is assigned to `token` variable, which can be referenced by the subsequent API requests. As you see, the `token` field is no longer hardcoded, instead it is extracted from the first API request with `extractors` mechanism. In the meanwhile, it is assigned to `token` variable, which can be referenced by the subsequent API requests.
Now we save the test cases to [`quickstart-demo-rev-1.yml`][quickstart-demo-rev-1] and rerun it, and we will find that both API requests to be successful. Now we save the test cases to [`quickstart-demo-rev-1.yml`][quickstart-demo-rev-1] and rerun it, and we will find that both API requests to be successful.
@@ -154,7 +154,7 @@ In actual scenarios, each user's `device_sn` is different, so we should paramete
However, the test cases are only `YAML` documents, it is impossible to generate parameters dynamically in such text. Fortunately, we can combine `Python` scripts with `YAML/JSON` test cases in `ApiTestEngine`. However, the test cases are only `YAML` documents, it is impossible to generate parameters dynamically in such text. Fortunately, we can combine `Python` scripts with `YAML/JSON` test cases in `ApiTestEngine`.
To achieve this goal, we can utilize `debugtalk.py` plugin and `variable_binds` mechanisms. To achieve this goal, we can utilize `debugtalk.py` plugin and `variables` mechanisms.
To be specific, we can create a Python file (`examples/debugtalk.py`) and implement the related algorithm in it. The `debugtalk.py` file can not only be located beside `YAML/JSON` testset file, but also can be in any upward recursive folder. Since we want `debugtalk.py` to be importable, we should put a `__init__.py` in its folder to make it as a Python module. To be specific, we can create a Python file (`examples/debugtalk.py`) and implement the related algorithm in it. The `debugtalk.py` file can not only be located beside `YAML/JSON` testset file, but also can be in any upward recursive folder. Since we want `debugtalk.py` to be importable, we should put a `__init__.py` in its folder to make it as a Python module.
@@ -187,7 +187,7 @@ And then, we can revise our demo test case and reference the functions. Suppose
```yaml ```yaml
- test: - test:
name: get token name: get token
variable_binds: variables:
- user_agent: 'iOS/10.3' - user_agent: 'iOS/10.3'
- device_sn: ${gen_random_string(15)} - device_sn: ${gen_random_string(15)}
- os_platform: 'ios' - os_platform: 'ios'
@@ -202,7 +202,7 @@ And then, we can revise our demo test case and reference the functions. Suppose
app_version: $app_version app_version: $app_version
json: json:
sign: ${get_sign($user_agent, $device_sn, $os_platform, $app_version)} sign: ${get_sign($user_agent, $device_sn, $os_platform, $app_version)}
extract_binds: extractors:
- token: content.token - token: content.token
validators: validators:
- {"check": "status_code", "comparator": "eq", "expected": 200} - {"check": "status_code", "comparator": "eq", "expected": 200}
@@ -226,7 +226,7 @@ And then, we can revise our demo test case and reference the functions. Suppose
In this revised test case, `variable reference` and `function invoke` mechanisms are both used. In this revised test case, `variable reference` and `function invoke` mechanisms are both used.
To make fields like `device_sn` can be used more than once, we bind values to variables in `variable_binds` block. When we bind variables, we can not only bind exact value to a variable name, but also can call a function and bind the evaluated value to it. To make fields like `device_sn` can be used more than once, we bind values to variables in `variables` block. When we bind variables, we can not only bind exact value to a variable name, but also can call a function and bind the evaluated value to it.
When we want to reference a variable in the test case, we can do this with a escape character `$`. For example, `$user_agent` will not be taken as a normal string, and `ApiTestEngine` will consider it as a variable named `user_agent`, search and return its binding value. When we want to reference a variable in the test case, we can do this with a escape character `$`. For example, `$user_agent` will not be taken as a normal string, and `ApiTestEngine` will consider it as a variable named `user_agent`, search and return its binding value.
@@ -244,7 +244,7 @@ To handle this case, overall `config` block is supported in `ApiTestEngine`. If
# examples/quickstart-demo-rev-3.yml # examples/quickstart-demo-rev-3.yml
- config: - config:
name: "smoketest for CRUD users." name: "smoketest for CRUD users."
variable_binds: variables:
- device_sn: ${gen_random_string(15)} - device_sn: ${gen_random_string(15)}
request: request:
base_url: http://127.0.0.1:5000 base_url: http://127.0.0.1:5000
@@ -253,7 +253,7 @@ To handle this case, overall `config` block is supported in `ApiTestEngine`. If
- test: - test:
name: get token name: get token
variable_binds: variables:
- user_agent: 'iOS/10.3' - user_agent: 'iOS/10.3'
- os_platform: 'ios' - os_platform: 'ios'
- app_version: '2.8.6' - app_version: '2.8.6'
@@ -266,7 +266,7 @@ To handle this case, overall `config` block is supported in `ApiTestEngine`. If
app_version: $app_version app_version: $app_version
json: json:
sign: ${get_sign($user_agent, $device_sn, $os_platform, $app_version)} sign: ${get_sign($user_agent, $device_sn, $os_platform, $app_version)}
extract_binds: extractors:
- token: content.token - token: content.token
validators: validators:
- {"check": "status_code", "comparator": "eq", "expected": 200} - {"check": "status_code", "comparator": "eq", "expected": 200}

2
examples/quickstart-demo-rev-1.yml
View File

@@ -10,7 +10,7 @@
app_version: 2.8.6 app_version: 2.8.6
json: json:
sign: 19067cf712265eb5426db8d3664026c1ccea02b9 sign: 19067cf712265eb5426db8d3664026c1ccea02b9
extract_binds: extractors:
- token: content.token - token: content.token
validators: validators:
- {"check": "status_code", "comparator": "eq", "expected": 200} - {"check": "status_code", "comparator": "eq", "expected": 200}

4
examples/quickstart-demo-rev-2.yml
View File

@@ -1,6 +1,6 @@
- test: - test:
name: get token name: get token
variable_binds: variables:
- user_agent: 'iOS/10.3' - user_agent: 'iOS/10.3'
- device_sn: ${gen_random_string(15)} - device_sn: ${gen_random_string(15)}
- os_platform: 'ios' - os_platform: 'ios'
@@ -15,7 +15,7 @@
app_version: $app_version app_version: $app_version
json: json:
sign: ${get_sign($user_agent, $device_sn, $os_platform, $app_version)} sign: ${get_sign($user_agent, $device_sn, $os_platform, $app_version)}
extract_binds: extractors:
- token: content.token - token: content.token
validators: validators:
- {"check": "status_code", "comparator": "eq", "expected": 200} - {"check": "status_code", "comparator": "eq", "expected": 200}

6
examples/quickstart-demo-rev-3.yml
View File

@@ -1,6 +1,6 @@
- config: - config:
name: "smoketest for CRUD users." name: "smoketest for CRUD users."
variable_binds: variables:
- device_sn: ${gen_random_string(15)} - device_sn: ${gen_random_string(15)}
request: request:
base_url: http://127.0.0.1:5000 base_url: http://127.0.0.1:5000
@@ -9,7 +9,7 @@
- test: - test:
name: get token name: get token
variable_binds: variables:
- user_agent: 'iOS/10.3' - user_agent: 'iOS/10.3'
- os_platform: 'ios' - os_platform: 'ios'
- app_version: '2.8.6' - app_version: '2.8.6'
@@ -22,7 +22,7 @@
app_version: $app_version app_version: $app_version
json: json:
sign: ${get_sign($user_agent, $device_sn, $os_platform, $app_version)} sign: ${get_sign($user_agent, $device_sn, $os_platform, $app_version)}
extract_binds: extractors:
- token: content.token - token: content.token
validators: validators:
- {"check": "status_code", "comparator": "eq", "expected": 200} - {"check": "status_code", "comparator": "eq", "expected": 200}

1
requirements_dev.txt
View File

@@ -4,5 +4,4 @@ PyYAML
coveralls coveralls
coverage coverage
-e git+https://github.com/debugtalk/PyUnitReport.git#egg=PyUnitReport -e git+https://github.com/debugtalk/PyUnitReport.git#egg=PyUnitReport
-e git+https://github.com/debugtalk/jenkins-mail-py.git#egg=jenkins-mail-py
-e git+https://github.com/locustio/locust.git#egg=locustio -e git+https://github.com/locustio/locust.git#egg=locustio

10
setup.py
View File

@@ -30,26 +30,20 @@ setup(
"PyUnitReport" "PyUnitReport"
], ],
extras_require={ extras_require={
'mail': [ 'locustio': [
"jenkins-mail-py"
],
'locust': [
"locustio" "locustio"
] ]
}, },
dependency_links=[ dependency_links=[
"git+https://github.com/debugtalk/PyUnitReport.git#egg=PyUnitReport-0", "git+https://github.com/debugtalk/PyUnitReport.git#egg=PyUnitReport-0",
"git+https://github.com/debugtalk/jenkins-mail-py.git#egg=jenkins-mail-py-0",
"git+https://github.com/locustio/locust.git#egg=locust-0" "git+https://github.com/locustio/locust.git#egg=locust-0"
], ],
classifiers=[ classifiers=[
"Development Status :: 3 - Alpha", "Development Status :: 3 - Alpha",
'Programming Language :: Python :: 2.7', 'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3.3',
'Programming Language :: Python :: 3.4', 'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5', 'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6', 'Programming Language :: Python :: 3.6'
'Programming Language :: Python :: 3.7-dev'
], ],
entry_points={ entry_points={
'console_scripts': [ 'console_scripts': [

8
tests/data/demo_binds.yml
View File

@@ -1,5 +1,5 @@
bind_variables: bind_variables:
variable_binds: variables:
- TOKEN: "debugtalk" - TOKEN: "debugtalk"
- token: $TOKEN - token: $TOKEN
@@ -7,7 +7,7 @@ bind_lambda_functions:
function_binds: function_binds:
add_one: "lambda x: x + 1" add_one: "lambda x: x + 1"
add_two_nums: "lambda x, y: x + y" add_two_nums: "lambda x, y: x + y"
variable_binds: variables:
- add1: ${add_one(2)} - add1: ${add_one(2)}
- sum2nums: ${add_two_nums(2, 3)} - sum2nums: ${add_two_nums(2, 3)}
@@ -19,7 +19,7 @@ bind_lambda_functions_with_import:
function_binds: function_binds:
gen_random_string: "lambda str_len: ''.join(random.choice(string.ascii_letters + string.digits) for _ in range(str_len))" gen_random_string: "lambda str_len: ''.join(random.choice(string.ascii_letters + string.digits) for _ in range(str_len))"
gen_md5: "lambda *str_args: hashlib.md5(''.join(str_args).encode('utf-8')).hexdigest()" gen_md5: "lambda *str_args: hashlib.md5(''.join(str_args).encode('utf-8')).hexdigest()"
variable_binds: variables:
- TOKEN: debugtalk - TOKEN: debugtalk
- random: ${gen_random_string(5)} - random: ${gen_random_string(5)}
- data: "{'name': 'user', 'password': '123456'}" - data: "{'name': 'user', 'password': '123456'}"
@@ -29,7 +29,7 @@ bind_module_functions:
function_binds: function_binds:
import_module_items: import_module_items:
- tests.data.debugtalk - tests.data.debugtalk
variable_binds: variables:
- TOKEN: debugtalk - TOKEN: debugtalk
- random: ${gen_random_string(5)} - random: ${gen_random_string(5)}
- data: "{'name': 'user', 'password': '123456'}" - data: "{'name': 'user', 'password': '123456'}"

2
tests/data/demo_testset_hardcode.json
View File

@@ -16,7 +16,7 @@
"sign": "f1219719911caae89ccc301679857ebfda115ca2" "sign": "f1219719911caae89ccc301679857ebfda115ca2"
} }
}, },
"extract_binds": [ "extractors": [
{ {
"token": "content.token" "token": "content.token"
} }

2
tests/data/demo_testset_hardcode.yml
View File

@@ -11,7 +11,7 @@
app_version: '2.8.6' app_version: '2.8.6'
json: json:
sign: f1219719911caae89ccc301679857ebfda115ca2 sign: f1219719911caae89ccc301679857ebfda115ca2
extract_binds: extractors:
- token: content.token - token: content.token
validators: validators:
- {"check": "status_code", "comparator": "eq", "expected": 200} - {"check": "status_code", "comparator": "eq", "expected": 200}

16
tests/data/demo_testset_layer.yml
View File

@@ -1,6 +1,6 @@
- config: - config:
name: "create user testsets." name: "user management testset."
variable_binds: variables:
- user_agent: 'iOS/10.3' - user_agent: 'iOS/10.3'
- device_sn: ${gen_random_string(15)} - device_sn: ${gen_random_string(15)}
- os_platform: 'ios' - os_platform: 'ios'
@@ -16,7 +16,7 @@
- test: - test:
name: get token name: get token
api: get_token($user_agent, $device_sn, $os_platform, $app_version) api: get_token($user_agent, $device_sn, $os_platform, $app_version)
extract_binds: extractors:
- token: content.token - token: content.token
- test: - test:
@@ -35,7 +35,7 @@
- test: - test:
name: create user which does not exist name: create user which does not exist
variable_binds: variables:
- user_name: "user1" - user_name: "user1"
- user_password: "123456" - user_password: "123456"
api: create_user(1000, $user_name, $user_password, $token) api: create_user(1000, $user_name, $user_password, $token)
@@ -53,7 +53,7 @@
- test: - test:
name: create user which exists name: create user which exists
variable_binds: variables:
- user_name: "user1" - user_name: "user1"
- user_password: "123456" - user_password: "123456"
api: create_user(1000, $user_name, $user_password, $token) api: create_user(1000, $user_name, $user_password, $token)
@@ -63,7 +63,7 @@
- test: - test:
name: update user which exists name: update user which exists
variable_binds: variables:
- user_name: "user1" - user_name: "user1"
- user_password: "654321" - user_password: "654321"
api: update_user(1000, $user_name, $user_password, $token) api: update_user(1000, $user_name, $user_password, $token)
@@ -72,7 +72,7 @@
- {"check": "content.success", "expected": true} - {"check": "content.success", "expected": true}
- test: - test:
name: get user that has been created name: get user that has been updated
api: get_user(1000, $token) api: get_user(1000, $token)
validators: validators:
- {"check": "status_code", "expected": 200} - {"check": "status_code", "expected": 200}
@@ -102,7 +102,7 @@
- test: - test:
name: create user which has been deleted name: create user which has been deleted
variable_binds: variables:
- user_name: "user1" - user_name: "user1"
- user_password: "123456" - user_password: "123456"
api: create_user(1000, $user_name, $user_password, $token) api: create_user(1000, $user_name, $user_password, $token)

6
tests/data/demo_testset_template_import_functions.yml
View File

@@ -2,7 +2,7 @@
name: "create user testsets." name: "create user testsets."
import_module_items: import_module_items:
- tests.data.debugtalk - tests.data.debugtalk
variable_binds: variables:
- user_agent: 'iOS/10.3' - user_agent: 'iOS/10.3'
- device_sn: ${gen_random_string(15)} - device_sn: ${gen_random_string(15)}
- os_platform: 'ios' - os_platform: 'ios'
@@ -25,7 +25,7 @@
app_version: $app_version app_version: $app_version
json: json:
sign: ${get_sign($user_agent, $device_sn, $os_platform, $app_version)} sign: ${get_sign($user_agent, $device_sn, $os_platform, $app_version)}
extract_binds: extractors:
- token: content.token - token: content.token
validators: validators:
- {"check": "status_code", "comparator": "eq", "expected": 200} - {"check": "status_code", "comparator": "eq", "expected": 200}
@@ -33,7 +33,7 @@
- test: - test:
name: create user which does not exist name: create user which does not exist
variable_binds: variables:
- user_name: "user1" - user_name: "user1"
- user_password: "123456" - user_password: "123456"
request: request:

6
tests/data/demo_testset_template_lambda_functions.yml
View File

@@ -12,7 +12,7 @@
'DebugTalk'.encode('ascii'), 'DebugTalk'.encode('ascii'),
''.join(args).encode('ascii'), ''.join(args).encode('ascii'),
hashlib.sha1).hexdigest()" hashlib.sha1).hexdigest()"
variable_binds: variables:
- user_agent: 'iOS/10.3' - user_agent: 'iOS/10.3'
- device_sn: ${gen_random_string_lambda(15)} - device_sn: ${gen_random_string_lambda(15)}
- os_platform: 'ios' - os_platform: 'ios'
@@ -35,7 +35,7 @@
app_version: $app_version app_version: $app_version
json: json:
sign: ${get_sign_lambda($user_agent, $device_sn, $os_platform, $app_version)} sign: ${get_sign_lambda($user_agent, $device_sn, $os_platform, $app_version)}
extract_binds: extractors:
- token: content.token - token: content.token
validators: validators:
- {"check": "status_code", "comparator": "eq", "expected": 200} - {"check": "status_code", "comparator": "eq", "expected": 200}
@@ -43,7 +43,7 @@
- test: - test:
name: create user which does not exist name: create user which does not exist
variable_binds: variables:
- user_name: "user1" - user_name: "user1"
- user_password: "123456" - user_password: "123456"
request: request:

8
tests/data/demo_testset_variables.yml
View File

@@ -1,6 +1,6 @@
- config: - config:
name: "create user testsets." name: "create user testsets."
variable_binds: variables:
- device_sn: 'HZfFBh6tU59EdXJ' - device_sn: 'HZfFBh6tU59EdXJ'
request: request:
base_url: $BASE_URL base_url: $BASE_URL
@@ -10,7 +10,7 @@
- test: - test:
name: get token name: get token
variable_binds: variables:
- user_agent: 'iOS/10.3' - user_agent: 'iOS/10.3'
- os_platform: 'ios' - os_platform: 'ios'
- app_version: '2.8.6' - app_version: '2.8.6'
@@ -26,7 +26,7 @@
app_version: $app_version app_version: $app_version
json: json:
sign: $sign sign: $sign
extract_binds: extractors:
- token: content.token - token: content.token
validators: validators:
- {"check": "status_code", "comparator": "eq", "expected": 200} - {"check": "status_code", "comparator": "eq", "expected": 200}
@@ -34,7 +34,7 @@
- test: - test:
name: create user which does not exist name: create user which does not exist
variable_binds: variables:
- user_name: "user1" - user_name: "user1"
- user_password: "123456" - user_password: "123456"
request: request:

38
tests/test_context.py
View File

@@ -18,11 +18,11 @@ class VariableBindsUnittest(unittest.TestCase):
self.assertIn("get_timestamp", self.context.testset_functions_config) self.assertIn("get_timestamp", self.context.testset_functions_config)
self.assertIn("gen_random_string", self.context.testset_functions_config) self.assertIn("gen_random_string", self.context.testset_functions_config)
variable_binds = [ variables = [
{"random": "${gen_random_string(5)}"}, {"random": "${gen_random_string(5)}"},
{"timestamp10": "${get_timestamp(10)}"} {"timestamp10": "${get_timestamp(10)}"}
] ]
self.context.bind_variables(variable_binds) self.context.bind_variables(variables)
context_variables = self.context.get_testcase_variables_mapping() context_variables = self.context.get_testcase_variables_mapping()
self.assertEqual(len(context_variables["random"]), 5) self.assertEqual(len(context_variables["random"]), 5)
@@ -31,7 +31,7 @@ class VariableBindsUnittest(unittest.TestCase):
def test_context_bind_testset_variables(self): def test_context_bind_testset_variables(self):
# testcase in JSON format # testcase in JSON format
testcase1 = { testcase1 = {
"variable_binds": [ "variables": [
{"GLOBAL_TOKEN": "debugtalk"}, {"GLOBAL_TOKEN": "debugtalk"},
{"token": "$GLOBAL_TOKEN"} {"token": "$GLOBAL_TOKEN"}
] ]
@@ -40,8 +40,8 @@ class VariableBindsUnittest(unittest.TestCase):
testcase2 = self.testcases["bind_variables"] testcase2 = self.testcases["bind_variables"]
for testcase in [testcase1, testcase2]: for testcase in [testcase1, testcase2]:
variable_binds = testcase['variable_binds'] variables = testcase['variables']
self.context.bind_variables(variable_binds, level="testset") self.context.bind_variables(variables, level="testset")
testset_variables = self.context.testset_shared_variables_mapping testset_variables = self.context.testset_shared_variables_mapping
testcase_variables = self.context.get_testcase_variables_mapping() testcase_variables = self.context.get_testcase_variables_mapping()
@@ -54,7 +54,7 @@ class VariableBindsUnittest(unittest.TestCase):
def test_context_bind_testcase_variables(self): def test_context_bind_testcase_variables(self):
testcase1 = { testcase1 = {
"variable_binds": [ "variables": [
{"GLOBAL_TOKEN": "debugtalk"}, {"GLOBAL_TOKEN": "debugtalk"},
{"token": "$GLOBAL_TOKEN"} {"token": "$GLOBAL_TOKEN"}
] ]
@@ -62,8 +62,8 @@ class VariableBindsUnittest(unittest.TestCase):
testcase2 = self.testcases["bind_variables"] testcase2 = self.testcases["bind_variables"]
for testcase in [testcase1, testcase2]: for testcase in [testcase1, testcase2]:
variable_binds = testcase['variable_binds'] variables = testcase['variables']
self.context.bind_variables(variable_binds) self.context.bind_variables(variables)
testset_variables = self.context.testset_shared_variables_mapping testset_variables = self.context.testset_shared_variables_mapping
testcase_variables = self.context.get_testcase_variables_mapping() testcase_variables = self.context.get_testcase_variables_mapping()
@@ -80,7 +80,7 @@ class VariableBindsUnittest(unittest.TestCase):
"add_one": lambda x: x + 1, "add_one": lambda x: x + 1,
"add_two_nums": lambda x, y: x + y "add_two_nums": lambda x, y: x + y
}, },
"variable_binds": [ "variables": [
{"add1": "${add_one(2)}"}, {"add1": "${add_one(2)}"},
{"sum2nums": "${add_two_nums(2,3)}"} {"sum2nums": "${add_two_nums(2,3)}"}
] ]
@@ -91,8 +91,8 @@ class VariableBindsUnittest(unittest.TestCase):
function_binds = testcase.get('function_binds', {}) function_binds = testcase.get('function_binds', {})
self.context.bind_functions(function_binds) self.context.bind_functions(function_binds)
variable_binds = testcase['variable_binds'] variables = testcase['variables']
self.context.bind_variables(variable_binds) self.context.bind_variables(variables)
context_variables = self.context.get_testcase_variables_mapping() context_variables = self.context.get_testcase_variables_mapping()
self.assertIn("add1", context_variables) self.assertIn("add1", context_variables)
@@ -107,7 +107,7 @@ class VariableBindsUnittest(unittest.TestCase):
"gen_random_string": "lambda str_len: ''.join(random.choice(string.ascii_letters + string.digits) for _ in range(str_len))", "gen_random_string": "lambda str_len: ''.join(random.choice(string.ascii_letters + string.digits) for _ in range(str_len))",
"gen_md5": "lambda *str_args: hashlib.md5(''.join(str_args).encode('utf-8')).hexdigest()" "gen_md5": "lambda *str_args: hashlib.md5(''.join(str_args).encode('utf-8')).hexdigest()"
}, },
"variable_binds": [ "variables": [
{"TOKEN": "debugtalk"}, {"TOKEN": "debugtalk"},
{"random": "${gen_random_string(5)}"}, {"random": "${gen_random_string(5)}"},
{"data": '{"name": "user", "password": "123456"}'}, {"data": '{"name": "user", "password": "123456"}'},
@@ -123,8 +123,8 @@ class VariableBindsUnittest(unittest.TestCase):
function_binds = testcase.get('function_binds', {}) function_binds = testcase.get('function_binds', {})
self.context.bind_functions(function_binds) self.context.bind_functions(function_binds)
variable_binds = testcase['variable_binds'] variables = testcase['variables']
self.context.bind_variables(variable_binds) self.context.bind_variables(variables)
context_variables = self.context.get_testcase_variables_mapping() context_variables = self.context.get_testcase_variables_mapping()
self.assertIn("TOKEN", context_variables) self.assertIn("TOKEN", context_variables)
@@ -144,7 +144,7 @@ class VariableBindsUnittest(unittest.TestCase):
def test_import_module_items(self): def test_import_module_items(self):
testcase1 = { testcase1 = {
"import_module_items": ["tests.data.debugtalk"], "import_module_items": ["tests.data.debugtalk"],
"variable_binds": [ "variables": [
{"TOKEN": "debugtalk"}, {"TOKEN": "debugtalk"},
{"random": "${gen_random_string(5)}"}, {"random": "${gen_random_string(5)}"},
{"data": '{"name": "user", "password": "123456"}'}, {"data": '{"name": "user", "password": "123456"}'},
@@ -157,8 +157,8 @@ class VariableBindsUnittest(unittest.TestCase):
module_items = testcase.get('import_module_items', []) module_items = testcase.get('import_module_items', [])
self.context.import_module_items(module_items) self.context.import_module_items(module_items)
variable_binds = testcase['variable_binds'] variables = testcase['variables']
self.context.bind_variables(variable_binds) self.context.bind_variables(variables)
context_variables = self.context.get_testcase_variables_mapping() context_variables = self.context.get_testcase_variables_mapping()
self.assertIn("TOKEN", context_variables) self.assertIn("TOKEN", context_variables)
@@ -182,7 +182,7 @@ class VariableBindsUnittest(unittest.TestCase):
test_runner = runner.Runner() test_runner = runner.Runner()
testcase = { testcase = {
"import_module_items": ["tests.data.debugtalk"], "import_module_items": ["tests.data.debugtalk"],
"variable_binds": [ "variables": [
{"TOKEN": "debugtalk"}, {"TOKEN": "debugtalk"},
{"random": "${gen_random_string(5)}"}, {"random": "${gen_random_string(5)}"},
{"data": '{"name": "user", "password": "123456"}'}, {"data": '{"name": "user", "password": "123456"}'},
@@ -206,7 +206,7 @@ class VariableBindsUnittest(unittest.TestCase):
self.assertIn("random", parsed_request["headers"]) self.assertIn("random", parsed_request["headers"])
self.assertEqual(len(parsed_request["headers"]["random"]), 5) self.assertEqual(len(parsed_request["headers"]["random"]), 5)
self.assertIn("data", parsed_request) self.assertIn("data", parsed_request)
self.assertEqual(parsed_request["data"], testcase["variable_binds"][2]["data"]) self.assertEqual(parsed_request["data"], testcase["variables"][2]["data"])
self.assertEqual(parsed_request["headers"]["SECRET_KEY"], "DebugTalk") self.assertEqual(parsed_request["headers"]["SECRET_KEY"], "DebugTalk")
def test_exec_content_functions(self): def test_exec_content_functions(self):

2
tests/test_runner.py
View File

@@ -53,7 +53,7 @@ class TestRunner(ApiServerUnittest):
"sign": "f1219719911caae89ccc301679857ebfda115ca2" "sign": "f1219719911caae89ccc301679857ebfda115ca2"
} }
}, },
"extract_binds": [ "extractors": [
{"token": "content.token"} {"token": "content.token"}
], ],
"validators": [ "validators": [

40
tests/test_testcase.py
View File

@@ -55,7 +55,7 @@ class TestcaseParserUnittest(unittest.TestCase):
) )
def test_eval_content_variables(self): def test_eval_content_variables(self):
variable_binds = { variables = {
"var_1": "abc", "var_1": "abc",
"var_2": "def", "var_2": "def",
"var_3": 123, "var_3": 123,
@@ -63,7 +63,7 @@ class TestcaseParserUnittest(unittest.TestCase):
"var_5": True, "var_5": True,
"var_6": None "var_6": None
} }
testcase_parser = testcase.TestcaseParser(variables_binds=variable_binds) testcase_parser = testcase.TestcaseParser(variables=variables)
self.assertEqual( self.assertEqual(
testcase_parser.eval_content_variables("$var_1"), testcase_parser.eval_content_variables("$var_1"),
"abc" "abc"
@@ -157,11 +157,11 @@ class TestcaseParserUnittest(unittest.TestCase):
) )
def test_parse_content_with_bindings_variables(self): def test_parse_content_with_bindings_variables(self):
variables_binds = { variables = {
"str_1": "str_value1", "str_1": "str_value1",
"str_2": "str_value2" "str_2": "str_value2"
} }
testcase_parser = testcase.TestcaseParser(variables_binds=variables_binds) testcase_parser = testcase.TestcaseParser(variables=variables)
self.assertEqual( self.assertEqual(
testcase_parser.parse_content_with_bindings("$str_1"), testcase_parser.parse_content_with_bindings("$str_1"),
"str_value1" "str_value1"
@@ -184,11 +184,11 @@ class TestcaseParserUnittest(unittest.TestCase):
) )
def test_parse_content_with_bindings_multiple_identical_variables(self): def test_parse_content_with_bindings_multiple_identical_variables(self):
variables_binds = { variables = {
"userid": 100, "userid": 100,
"data": 1498 "data": 1498
} }
testcase_parser = testcase.TestcaseParser(variables_binds=variables_binds) testcase_parser = testcase.TestcaseParser(variables=variables)
content = "/users/$userid/training/$data?userId=$userid&data=$data" content = "/users/$userid/training/$data?userId=$userid&data=$data"
self.assertEqual( self.assertEqual(
testcase_parser.parse_content_with_bindings(content), testcase_parser.parse_content_with_bindings(content),
@@ -196,12 +196,12 @@ class TestcaseParserUnittest(unittest.TestCase):
) )
def test_parse_variables_multiple_identical_variables(self): def test_parse_variables_multiple_identical_variables(self):
variables_binds = { variables = {
"user": 100, "user": 100,
"userid": 1000, "userid": 1000,
"data": 1498 "data": 1498
} }
testcase_parser = testcase.TestcaseParser(variables_binds=variables_binds) testcase_parser = testcase.TestcaseParser(variables=variables)
content = "/users/$user/$userid/$data?userId=$userid&data=$data" content = "/users/$user/$userid/$data?userId=$userid&data=$data"
self.assertEqual( self.assertEqual(
testcase_parser.parse_content_with_bindings(content), testcase_parser.parse_content_with_bindings(content),
@@ -210,17 +210,17 @@ class TestcaseParserUnittest(unittest.TestCase):
def test_parse_content_with_bindings_functions(self): def test_parse_content_with_bindings_functions(self):
import random, string import random, string
functions_binds = { functions = {
"gen_random_string": lambda str_len: ''.join(random.choice(string.ascii_letters + string.digits) \ "gen_random_string": lambda str_len: ''.join(random.choice(string.ascii_letters + string.digits) \
for _ in range(str_len)) for _ in range(str_len))
} }
testcase_parser = testcase.TestcaseParser(functions_binds=functions_binds) testcase_parser = testcase.TestcaseParser(functions=functions)
result = testcase_parser.parse_content_with_bindings("${gen_random_string(5)}") result = testcase_parser.parse_content_with_bindings("${gen_random_string(5)}")
self.assertEqual(len(result), 5) self.assertEqual(len(result), 5)
add_two_nums = lambda a, b=1: a + b add_two_nums = lambda a, b=1: a + b
functions_binds["add_two_nums"] = add_two_nums functions["add_two_nums"] = add_two_nums
self.assertEqual( self.assertEqual(
testcase_parser.parse_content_with_bindings("${add_two_nums(1)}"), testcase_parser.parse_content_with_bindings("${add_two_nums(1)}"),
2 2
@@ -265,10 +265,10 @@ class TestcaseParserUnittest(unittest.TestCase):
) )
def test_eval_content_functions(self): def test_eval_content_functions(self):
functions_binds = { functions = {
"add_two_nums": lambda a, b=1: a + b "add_two_nums": lambda a, b=1: a + b
} }
testcase_parser = testcase.TestcaseParser(functions_binds=functions_binds) testcase_parser = testcase.TestcaseParser(functions=functions)
self.assertEqual( self.assertEqual(
testcase_parser.eval_content_functions("${add_two_nums(1, 2)}"), testcase_parser.eval_content_functions("${add_two_nums(1, 2)}"),
3 3
@@ -289,13 +289,13 @@ class TestcaseParserUnittest(unittest.TestCase):
self.assertEqual(content, "/api/900150983cd24fb0d6963f7d28e17f72") self.assertEqual(content, "/api/900150983cd24fb0d6963f7d28e17f72")
def test_parse_content_with_bindings_testcase(self): def test_parse_content_with_bindings_testcase(self):
variables_binds = { variables = {
"uid": "1000", "uid": "1000",
"random": "A2dEx", "random": "A2dEx",
"authorization": "a83de0ff8d2e896dbd8efb81ba14e17d", "authorization": "a83de0ff8d2e896dbd8efb81ba14e17d",
"data": {"name": "user", "password": "123456"} "data": {"name": "user", "password": "123456"}
} }
functions_binds = { functions = {
"add_two_nums": lambda a, b=1: a + b, "add_two_nums": lambda a, b=1: a + b,
"get_timestamp": lambda: int(time.time() * 1000) "get_timestamp": lambda: int(time.time() * 1000)
} }
@@ -310,7 +310,7 @@ class TestcaseParserUnittest(unittest.TestCase):
}, },
"body": "$data" "body": "$data"
} }
parsed_testcase = testcase.TestcaseParser(variables_binds, functions_binds)\ parsed_testcase = testcase.TestcaseParser(variables, functions)\
.parse_content_with_bindings(testcase_template) .parse_content_with_bindings(testcase_template)
self.assertEqual( self.assertEqual(
@@ -319,15 +319,15 @@ class TestcaseParserUnittest(unittest.TestCase):
) )
self.assertEqual( self.assertEqual(
parsed_testcase["headers"]["authorization"], parsed_testcase["headers"]["authorization"],
variables_binds["authorization"] variables["authorization"]
) )
self.assertEqual( self.assertEqual(
parsed_testcase["headers"]["random"], parsed_testcase["headers"]["random"],
variables_binds["random"] variables["random"]
) )
self.assertEqual( self.assertEqual(
parsed_testcase["body"], parsed_testcase["body"],
variables_binds["data"] variables["data"]
) )
self.assertEqual( self.assertEqual(
parsed_testcase["headers"]["sum"], parsed_testcase["headers"]["sum"],
@@ -417,7 +417,7 @@ class TestcaseParserUnittest(unittest.TestCase):
path = os.path.join( path = os.path.join(
os.getcwd(), 'tests/data/demo_testset_layer.yml') os.getcwd(), 'tests/data/demo_testset_layer.yml')
testsets_list = testcase.load_testcases_by_path(path) testsets_list = testcase.load_testcases_by_path(path)
self.assertIn("variable_binds", testsets_list[0]["config"]) self.assertIn("variables", testsets_list[0]["config"])
self.assertIn("request", testsets_list[0]["config"]) self.assertIn("request", testsets_list[0]["config"])
self.assertIn("request", testsets_list[0]["testcases"][0]) self.assertIn("request", testsets_list[0]["testcases"][0])
self.assertIn("url", testsets_list[0]["testcases"][0]["request"]) self.assertIn("url", testsets_list[0]["testcases"][0]["request"])