GithubBackup/httprunner
1
0
Fork 0
mirror of https://github.com/httprunner/httprunner.git synced 2026-05-12 02:21:29 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #475 from HttpRunner/2.0

Browse Source 
Release 2.0.0
This commit is contained in:
debugtalk
2019-01-01 23:24:28 +08:00
committed by GitHub
parent 63ba664cc7 2538743c6b
commit 0224d3ec66
63 changed files with 4040 additions and 2608 deletions
Download Patch File Download Diff File Expand all files Collapse all files

214
LICENSE
View File

@@ -1,21 +1,201 @@
MIT License
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
Copyright (c) 2017 Leo Lee
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
1. Definitions.
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2017 debugtalk
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

4
httprunner/__about__.py
View File

@@ -1,9 +1,9 @@
__title__ = 'HttpRunner'
__description__ = 'One-stop solution for HTTP(S) testing.'
__url__ = 'https://github.com/HttpRunner/HttpRunner'
__version__ = '1.5.15'
__version__ = '2.0.0'
__author__ = 'debugtalk'
__author_email__ = 'mail@debugtalk.com'
__license__ = 'MIT'
__license__ = 'Apache-2.0'
__copyright__ = 'Copyright 2017 debugtalk'
__cake__ = u'\u2728 \U0001f370 \u2728'

9
httprunner/__init__.py
View File

@@ -1,9 +0,0 @@
# encoding: utf-8
try:
# monkey patch at beginning to avoid RecursionError when running locust.
from gevent import monkey; monkey.patch_all()
except ImportError:
pass
from httprunner.api import HttpRunner

278
httprunner/api.py
View File

@@ -9,91 +9,83 @@ from httprunner import (exceptions, loader, logger, parser, report, runner,
class HttpRunner(object):
def __init__(self, **kwargs):
def __init__(self, failfast=False, save_tests=False, report_template=None, report_dir=None,
log_level="INFO", log_file=None):
""" initialize HttpRunner.
Args:
kwargs (dict): key-value arguments used to initialize TextTestRunner.
Commonly used arguments:
resultclass (class): HtmlTestResult or TextTestResult
failfast (bool): False/True, stop the test run on the first error or failure.
http_client_session (instance): requests.Session(), or locust.client.Session() instance.
Attributes:
project_mapping (dict): save project loaded api/testcases, environments and debugtalk.py module.
{
"debugtalk": {
"variables": {},
"functions": {}
},
"env": {},
"def-api": {},
"def-testcase": {}
}
failfast (bool): stop the test run on the first error or failure.
save_tests (bool): save loaded/parsed tests to JSON file.
report_template (str): report template file path, template should be in Jinja2 format.
report_dir (str): html report save directory.
log_level (str): logging level.
log_file (str): log file path.
"""
self.exception_stage = "initialize HttpRunner()"
self.http_client_session = kwargs.pop("http_client_session", None)
kwargs.setdefault("resultclass", report.HtmlTestResult)
kwargs = {
"failfast": failfast,
"resultclass": report.HtmlTestResult
}
self.unittest_runner = unittest.TextTestRunner(**kwargs)
self.test_loader = unittest.TestLoader()
self.summary = None
self.save_tests = save_tests
self.report_template = report_template
self.report_dir = report_dir
self._summary = None
if log_file:
logger.setup_logger(log_level, log_file)
def _add_tests(self, testcases):
def _add_tests(self, tests_mapping):
""" initialize testcase with Runner() and add to test suite.
Args:
testcases (list): parsed testcases list
tests_mapping (dict): project info and testcases list.
Returns:
tuple: unittest.TestSuite()
unittest.TestSuite()
"""
def _add_teststep(test_runner, config, teststep_dict):
""" add teststep to testcase.
def _add_test(test_runner, test_dict):
""" add test to testcase.
"""
def test(self):
try:
test_runner.run_test(teststep_dict)
test_runner.run_test(test_dict)
except exceptions.MyBaseFailure as ex:
self.fail(str(ex))
finally:
if hasattr(test_runner.http_client_session, "meta_data"):
self.meta_data = test_runner.http_client_session.meta_data
self.meta_data["validators"] = test_runner.evaluated_validators
test_runner.http_client_session.init_meta_data()
self.meta_datas = test_runner.meta_datas
try:
teststep_dict["name"] = parser.parse_data(
teststep_dict["name"],
config.get("variables", {}),
config.get("functions", {})
)
except exceptions.VariableNotFound:
pass
if "config" in test_dict:
# run nested testcase
test.__doc__ = test_dict["config"].get("name")
else:
# run api test
test.__doc__ = test_dict.get("name")
test.__doc__ = teststep_dict["name"]
return test
test_suite = unittest.TestSuite()
for testcase in testcases:
functions = tests_mapping.get("project_mapping", {}).get("functions", {})
for testcase in tests_mapping["testcases"]:
config = testcase.get("config", {})
test_runner = runner.Runner(config, self.http_client_session)
test_runner = runner.Runner(config, functions)
TestSequense = type('TestSequense', (unittest.TestCase,), {})
teststeps = testcase.get("teststeps", [])
for index, teststep_dict in enumerate(teststeps):
for times_index in range(int(teststep_dict.get("times", 1))):
tests = testcase.get("teststeps", [])
for index, test_dict in enumerate(tests):
for times_index in range(int(test_dict.get("times", 1))):
# suppose one testcase should not have more than 9999 steps,
# and one step should not run more than 999 times.
test_method_name = 'test_{:04}_{:03}'.format(index, times_index)
test_method = _add_teststep(test_runner, config, teststep_dict)
test_method = _add_test(test_runner, test_dict)
setattr(TestSequense, test_method_name, test_method)
loaded_testcase = self.test_loader.loadTestsFromTestCase(TestSequense)
setattr(loaded_testcase, "config", config)
setattr(loaded_testcase, "teststeps", testcase.get("teststeps", []))
setattr(loaded_testcase, "teststeps", tests)
setattr(loaded_testcase, "runner", test_runner)
test_suite.addTest(loaded_testcase)
@@ -127,9 +119,16 @@ class HttpRunner(object):
tests_results (list): list of (testcase, result)
"""
self.summary = {
summary = {
"success": True,
"stat": {},
"stat": {
"testcases": {
"total": len(tests_results),
"success": 0,
"fail": 0
},
"teststeps": {}
},
"time": {},
"platform": report.get_platform(),
"details": []
@@ -139,81 +138,67 @@ class HttpRunner(object):
testcase, result = tests_result
testcase_summary = report.get_summary(result)
self.summary["success"] &= testcase_summary["success"]
if testcase_summary["success"]:
summary["stat"]["testcases"]["success"] += 1
else:
summary["stat"]["testcases"]["fail"] += 1
summary["success"] &= testcase_summary["success"]
testcase_summary["name"] = testcase.config.get("name")
testcase_summary["base_url"] = testcase.config.get("request", {}).get("base_url", "")
in_out = utils.get_testcase_io(testcase)
utils.print_io(in_out)
testcase_summary["in_out"] = in_out
report.aggregate_stat(self.summary["stat"], testcase_summary["stat"])
report.aggregate_stat(self.summary["time"], testcase_summary["time"])
report.aggregate_stat(summary["stat"]["teststeps"], testcase_summary["stat"])
report.aggregate_stat(summary["time"], testcase_summary["time"])
self.summary["details"].append(testcase_summary)
summary["details"].append(testcase_summary)
def _run_tests(self, testcases, mapping=None):
""" start to run test with variables mapping.
Args:
testcases (list): list of testcase_dict, each testcase is corresponding to a YAML/JSON file
[
{ # testcase data structure
"config": {
"name": "desc1",
"path": "testcase1_path",
"variables": [], # optional
"request": {} # optional
"refs": {
"debugtalk": {
"variables": {},
"functions": {}
},
"env": {},
"def-api": {},
"def-testcase": {}
}
},
"teststeps": [
# teststep data structure
{
'name': 'test step desc2',
'variables': [], # optional
'extract': [], # optional
'validate': [],
'request': {},
'function_meta': {}
},
teststep2 # another teststep dict
]
},
testcase_dict_2 # another testcase dict
]
mapping (dict): if mapping is specified, it will override variables in config block.
Returns:
instance: HttpRunner() instance
return summary
def run_tests(self, tests_mapping):
""" run testcase/testsuite data
"""
# parse tests
self.exception_stage = "parse tests"
parsed_testcases_list = parser.parse_tests(testcases, mapping)
parsed_tests_mapping = parser.parse_tests(tests_mapping)
if self.save_tests:
utils.dump_tests(parsed_tests_mapping, "parsed")
# add tests to test suite
self.exception_stage = "add tests to test suite"
test_suite = self._add_tests(parsed_testcases_list)
test_suite = self._add_tests(parsed_tests_mapping)
# run test suite
self.exception_stage = "run test suite"
results = self._run_suite(test_suite)
# aggregate results
self.exception_stage = "aggregate results"
self._aggregate(results)
self._summary = self._aggregate(results)
return self
# generate html report
self.exception_stage = "generate html report"
report.stringify_summary(self._summary)
def run(self, path_or_testcases, dot_env_path=None, mapping=None):
""" main interface, run testcases with variables mapping.
if self.save_tests:
utils.dump_summary(self._summary, tests_mapping["project_mapping"])
report_path = report.render_html_report(
self._summary,
self.report_template,
self.report_dir
)
return report_path
def run_path(self, path, dot_env_path=None, mapping=None):
""" run testcase/testsuite file or folder.
Args:
path_or_testcases (str/list/dict): testcase file/foler path, or valid testcases.
path (str): testcase/testsuite file/foler path.
dot_env_path (str): specified .env file path.
mapping (dict): if mapping is specified, it will override variables in config block.
@@ -221,37 +206,70 @@ class HttpRunner(object):
instance: HttpRunner() instance
"""
# load tests
self.exception_stage = "load tests"
tests_mapping = loader.load_tests(path, dot_env_path)
tests_mapping["project_mapping"]["test_path"] = path
if validator.is_testcases(path_or_testcases):
if isinstance(path_or_testcases, dict):
testcases = [path_or_testcases]
else:
testcases = path_or_testcases
elif validator.is_testcase_path(path_or_testcases):
testcases = loader.load_tests(path_or_testcases, dot_env_path)
if mapping:
tests_mapping["project_mapping"]["variables"] = mapping
if self.save_tests:
utils.dump_tests(tests_mapping, "loaded")
return self.run_tests(tests_mapping)
def run(self, path_or_tests, dot_env_path=None, mapping=None):
""" main interface.
Args:
path_or_tests:
str: testcase/testsuite file/foler path
dict: valid testcase/testsuite data
"""
if validator.is_testcase_path(path_or_tests):
return self.run_path(path_or_tests, dot_env_path, mapping)
elif validator.is_testcases(path_or_tests):
return self.run_tests(path_or_tests)
else:
raise exceptions.ParamsError("invalid testcase path or testcases.")
return self._run_tests(testcases, mapping)
def gen_html_report(self, html_report_name=None, html_report_template=None):
""" generate html report and return report path.
Args:
html_report_name (str): output html report file name
html_report_template (str): report template file path, template should be in Jinja2 format
Returns:
str: generated html report path
@property
def summary(self):
""" get test reuslt summary.
"""
if not self.summary:
raise exceptions.MyBaseError("run method should be called before gen_html_report.")
return self._summary
self.exception_stage = "generate report"
return report.render_html_report(
self.summary,
html_report_name,
html_report_template
)
def prepare_locust_tests(path):
""" prepare locust testcases
Args:
path (str): testcase file path.
Returns:
dict: locust tests data
{
"functions": {},
"tests": []
}
"""
tests_mapping = loader.load_tests(path)
parsed_tests_mapping = parser.parse_tests(tests_mapping)
functions = parsed_tests_mapping.get("project_mapping", {}).get("functions", {})
tests = []
for testcase in parsed_tests_mapping["testcases"]:
testcase_weight = testcase.get("config", {}).pop("weight", 1)
for _ in range(testcase_weight):
tests.append(testcase)
return {
"functions": functions,
"tests": tests
}

11
httprunner/built_in.py
View File

@@ -132,17 +132,6 @@ def endswith(check_value, expect_value):
""" built-in hooks
"""
def setup_hook_prepare_kwargs(request):
if request["method"] == "POST":
content_type = request.get("headers", {}).get("content-type")
if content_type and "data" in request:
# if request content-type is application/json, request data should be dumped
if content_type.startswith("application/json") and isinstance(request["data"], (dict, list)):
request["data"] = json.dumps(request["data"])
if isinstance(request["data"], str):
request["data"] = request["data"].encode('utf-8')
def sleep_N_secs(n_secs):
""" sleep n seconds
"""

76
httprunner/cli.py
View File

@@ -1,22 +1,16 @@
# encoding: utf-8
import argparse
import multiprocessing
import os
import sys
import unittest
from httprunner import logger
from httprunner.__about__ import __description__, __version__
from httprunner.api import HttpRunner
from httprunner.compat import is_py2
from httprunner.utils import (create_scaffold, get_python2_retire_msg,
prettify_json_file, validate_json_file)
def main_hrun():
""" API test: parse command line options and run commands.
"""
import argparse
from httprunner import logger
from httprunner.__about__ import __description__, __version__
from httprunner.api import HttpRunner
from httprunner.compat import is_py2
from httprunner.utils import (create_scaffold, get_python2_retire_msg,
prettify_json_file, validate_json_file)
parser = argparse.ArgumentParser(description=__description__)
parser.add_argument(
'-V', '--version', dest='version', action='store_true',
@@ -24,15 +18,6 @@ def main_hrun():
parser.add_argument(
'testcase_paths', nargs='*',
help="testcase file path")
parser.add_argument(
'--no-html-report', action='store_true', default=False,
help="do not generate html report.")
parser.add_argument(
'--html-report-name',
help="specify html report name, only effective when generating html report.")
parser.add_argument(
'--html-report-template',
help="specify html report template path.")
parser.add_argument(
'--log-level', default='INFO',
help="Specify logging level, default is INFO.")
@@ -42,9 +27,18 @@ def main_hrun():
parser.add_argument(
'--dot-env-path',
help="Specify .env file path, which is useful for keeping sensitive data.")
parser.add_argument(
'--report-template',
help="specify report template path.")
parser.add_argument(
'--report-dir',
help="specify report save directory.")
parser.add_argument(
'--failfast', action='store_true', default=False,
help="Stop the test run on the first error or failure.")
parser.add_argument(
'--save-tests', action='store_true', default=False,
help="Save loaded tests and parsed tests to JSON file.")
parser.add_argument(
'--startproject',
help="Specify new project name.")
@@ -77,30 +71,32 @@ def main_hrun():
create_scaffold(project_name)
exit(0)
runner = HttpRunner(
failfast=args.failfast,
save_tests=args.save_tests,
report_template=args.report_template,
report_dir=args.report_dir
)
try:
runner = HttpRunner(
failfast=args.failfast
)
runner.run(
args.testcase_paths,
dot_env_path=args.dot_env_path
)
for path in args.testcase_paths:
runner.run(path, dot_env_path=args.dot_env_path)
except Exception:
logger.log_error("!!!!!!!!!! exception stage: {} !!!!!!!!!!".format(runner.exception_stage))
raise
if not args.no_html_report:
runner.gen_html_report(
html_report_name=args.html_report_name,
html_report_template=args.html_report_template
)
return 0
summary = runner.summary
return 0 if summary["success"] else 1
def main_locust():
""" Performance test with locust: parse command line options and run commands.
"""
# monkey patch ssl at beginning to avoid RecursionError when running locust.
from gevent import monkey; monkey.patch_ssl()
import multiprocessing
import sys
from httprunner import logger
try:
from httprunner import locusts
except ImportError:
@@ -114,7 +110,7 @@ def main_locust():
sys.argv.extend(["-h"])
if sys.argv[1] in ["-h", "--help", "-V", "--version"]:
locusts.main()
locusts.start_locust_main()
sys.exit(0)
# set logging level
@@ -129,7 +125,7 @@ def main_locust():
loglevel = sys.argv[loglevel_index]
else:
# default
loglevel = "INFO"
loglevel = "WARNING"
logger.setup_logger(loglevel)
@@ -180,4 +176,4 @@ def main_locust():
sys.argv.pop(processes_index)
locusts.run_locusts_with_processes(sys.argv, processes_count)
else:
locusts.main()
locusts.start_locust_main()

183
httprunner/client.py
View File

@@ -1,20 +1,17 @@
# encoding: utf-8
import re
import time
import requests
import urllib3
from httprunner import logger
from httprunner.exceptions import ParamsError
from httprunner.utils import build_url, lower_dict_keys, omit_long_data
from requests import Request, Response
from requests.exceptions import (InvalidSchema, InvalidURL, MissingSchema,
RequestException)
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
absolute_http_url_regexp = re.compile(r"^https?://", re.I)
class ApiResponse(Response):
@@ -42,37 +39,96 @@ class HttpSession(requests.Session):
self.base_url = base_url if base_url else ""
self.init_meta_data()
def _build_url(self, path):
""" prepend url with hostname unless it's already an absolute URL """
if absolute_http_url_regexp.match(path):
return path
elif self.base_url:
return "{}/{}".format(self.base_url.rstrip("/"), path.lstrip("/"))
else:
raise ParamsError("base url missed!")
def init_meta_data(self):
""" initialize meta_data, it will store detail data of request and response
"""
self.meta_data = {
"request": {
"url": "N/A",
"method": "N/A",
"headers": {},
"start_timestamp": None
},
"response": {
"status_code": "N/A",
"headers": {},
"name": "",
"data": [
{
"request": {
"url": "N/A",
"method": "N/A",
"headers": {}
},
"response": {
"status_code": "N/A",
"headers": {},
"encoding": None,
"content_type": ""
}
}
],
"stat": {
"content_size": "N/A",
"response_time_ms": "N/A",
"elapsed_ms": "N/A",
"encoding": None,
"content": None,
"content_type": ""
}
}
def get_req_resp_record(self, resp_obj):
""" get request and response info from Response() object.
"""
def log_print(req_resp_dict, r_type):
msg = "\n================== {} details ==================\n".format(r_type)
for key, value in req_resp_dict[r_type].items():
msg += "{:<16} : {}\n".format(key, repr(value))
logger.log_debug(msg)
req_resp_dict = {
"request": {},
"response": {}
}
# record actual request info
req_resp_dict["request"]["url"] = resp_obj.request.url
req_resp_dict["request"]["headers"] = dict(resp_obj.request.headers)
request_body = resp_obj.request.body
if request_body:
request_content_type = lower_dict_keys(
req_resp_dict["request"]["headers"]
).get("content-type")
if request_content_type and "multipart/form-data" in request_content_type:
# upload file type
req_resp_dict["request"]["body"] = "upload file stream (OMITTED)"
else:
req_resp_dict["request"]["body"] = request_body
# log request details in debug mode
log_print(req_resp_dict, "request")
# record response info
req_resp_dict["response"]["ok"] = resp_obj.ok
req_resp_dict["response"]["url"] = resp_obj.url
req_resp_dict["response"]["status_code"] = resp_obj.status_code
req_resp_dict["response"]["reason"] = resp_obj.reason
req_resp_dict["response"]["cookies"] = resp_obj.cookies or {}
req_resp_dict["response"]["encoding"] = resp_obj.encoding
resp_headers = dict(resp_obj.headers)
req_resp_dict["response"]["headers"] = resp_headers
lower_resp_headers = lower_dict_keys(resp_headers)
content_type = lower_resp_headers.get("content-type", "")
req_resp_dict["response"]["content_type"] = content_type
if "image" in content_type:
# response is image type, record bytes content only
req_resp_dict["response"]["content"] = resp_obj.content
else:
try:
# try to record json data
req_resp_dict["response"]["json"] = resp_obj.json()
except ValueError:
# only record at most 512 text charactors
resp_text = resp_obj.text
req_resp_dict["response"]["text"] = omit_long_data(resp_text)
# log response details in debug mode
log_print(req_resp_dict, "response")
return req_resp_dict
def request(self, method, url, name=None, **kwargs):
"""
Constructs and sends a :py:class:`requests.Request`.
@@ -112,63 +168,42 @@ class HttpSession(requests.Session):
:param cert: (optional)
if String, path to ssl client cert file (.pem). If Tuple, ('cert', 'key') pair.
"""
def log_print(request_response):
msg = "\n================== {} details ==================\n".format(request_response)
for key, value in self.meta_data[request_response].items():
msg += "{:<16} : {}\n".format(key, repr(value))
logger.log_debug(msg)
# record test name
self.meta_data["name"] = name
# record original request info
self.meta_data["request"]["method"] = method
self.meta_data["request"]["url"] = url
self.meta_data["request"].update(kwargs)
self.meta_data["request"]["start_timestamp"] = time.time()
self.meta_data["data"][0]["request"]["method"] = method
self.meta_data["data"][0]["request"]["url"] = url
kwargs.setdefault("timeout", 120)
self.meta_data["data"][0]["request"].update(kwargs)
# prepend url with hostname unless it's already an absolute URL
url = self._build_url(url)
url = build_url(self.base_url, url)
kwargs.setdefault("timeout", 120)
start_timestamp = time.time()
response = self._send_request_safe_mode(method, url, **kwargs)
# record the consumed time
self.meta_data["response"]["response_time_ms"] = \
round((time.time() - self.meta_data["request"]["start_timestamp"]) * 1000, 2)
self.meta_data["response"]["elapsed_ms"] = response.elapsed.microseconds / 1000.0
# record actual request info
self.meta_data["request"]["url"] = (response.history and response.history[0] or response).request.url
self.meta_data["request"]["headers"] = dict(response.request.headers)
self.meta_data["request"]["body"] = response.request.body
# log request details in debug mode
log_print("request")
# record response info
self.meta_data["response"]["ok"] = response.ok
self.meta_data["response"]["url"] = response.url
self.meta_data["response"]["status_code"] = response.status_code
self.meta_data["response"]["reason"] = response.reason
self.meta_data["response"]["headers"] = dict(response.headers)
self.meta_data["response"]["cookies"] = response.cookies or {}
self.meta_data["response"]["encoding"] = response.encoding
self.meta_data["response"]["content"] = response.content
self.meta_data["response"]["text"] = response.text
self.meta_data["response"]["content_type"] = response.headers.get("Content-Type", "")
try:
self.meta_data["response"]["json"] = response.json()
except ValueError:
self.meta_data["response"]["json"] = None
response_time_ms = round((time.time() - start_timestamp) * 1000, 2)
# get the length of the content, but if the argument stream is set to True, we take
# the size from the content-length header, in order to not trigger fetching of the body
if kwargs.get("stream", False):
self.meta_data["response"]["content_size"] = int(self.meta_data["response"]["headers"].get("content-length") or 0)
content_size = int(dict(response.headers).get("content-length") or 0)
else:
self.meta_data["response"]["content_size"] = len(response.content or "")
content_size = len(response.content or "")
# log response details in debug mode
log_print("response")
# record the consumed time
self.meta_data["stat"] = {
"response_time_ms": response_time_ms,
"elapsed_ms": response.elapsed.microseconds / 1000.0,
"content_size": content_size
}
# record request and response histories, include 30X redirection
response_list = response.history + [response]
self.meta_data["data"] = [
self.get_req_resp_record(resp_obj)
for resp_obj in response_list
]
try:
response.raise_for_status()
@@ -176,10 +211,10 @@ class HttpSession(requests.Session):
logger.log_error(u"{exception}".format(exception=str(e)))
else:
logger.log_info(
"""status_code: {}, response_time(ms): {} ms, response_length: {} bytes""".format(
self.meta_data["response"]["status_code"],
self.meta_data["response"]["response_time_ms"],
self.meta_data["response"]["content_size"]
"""status_code: {}, response_time(ms): {} ms, response_length: {} bytes\n""".format(
response.status_code,
response_time_ms,
content_size
)
)

169
httprunner/context.py
View File

@@ -1,80 +1,63 @@
# encoding: utf-8
import copy
from httprunner import exceptions, logger, parser, utils
from httprunner.compat import OrderedDict
class Context(object):
""" Manages context functions and variables.
context has two levels, testcase and teststep.
class SessionContext(object):
""" HttpRunner session, store runtime variables.
Examples:
>>> functions={...}
>>> variables = {"SECRET_KEY": "DebugTalk"}
>>> context = SessionContext(functions, variables)
Equivalent to:
>>> context = SessionContext(functions)
>>> context.update_session_variables(variables)
"""
def __init__(self, variables=None, functions=None):
""" init Context with testcase variables and functions.
"""
# testcase level context
## TESTCASE_SHARED_VARIABLES_MAPPING and TESTCASE_SHARED_FUNCTIONS_MAPPING are unchangeable.
if isinstance(variables, list):
self.TESTCASE_SHARED_VARIABLES_MAPPING = utils.convert_mappinglist_to_orderdict(variables)
else:
# dict
self.TESTCASE_SHARED_VARIABLES_MAPPING = variables or OrderedDict()
def __init__(self, functions, variables=None):
self.session_variables_mapping = utils.ensure_mapping_format(variables or {})
self.FUNCTIONS_MAPPING = functions
self.init_test_variables()
self.validation_results = []
self.TESTCASE_SHARED_FUNCTIONS_MAPPING = functions or OrderedDict()
# testcase level request, will not change
self.TESTCASE_SHARED_REQUEST_MAPPING = {}
self.evaluated_validators = []
self.init_context_variables(level="testcase")
def init_context_variables(self, level="testcase"):
""" initialize testcase/teststep context
def init_test_variables(self, variables_mapping=None):
""" init test variables, called when each test(api) starts.
variables_mapping will be evaluated first.
Args:
level (enum): "testcase" or "teststep"
"""
if level == "testcase":
# testcase level runtime context, will be updated with extracted variables in each teststep.
self.testcase_runtime_variables_mapping = copy.deepcopy(self.TESTCASE_SHARED_VARIABLES_MAPPING)
# teststep level context, will be altered in each teststep.
# teststep config shall inherit from testcase configs,
# but can not change testcase configs, that's why we use copy.deepcopy here.
self.teststep_variables_mapping = copy.deepcopy(self.testcase_runtime_variables_mapping)
def update_context_variables(self, variables, level):
""" update context variables, with level specified.
Args:
variables (list/OrderedDict): testcase config block or teststep block
[
{"TOKEN": "debugtalk"},
{"random": "${gen_random_string(5)}"},
{"json": {'name': 'user', 'password': '123456'}},
{"md5": "${gen_md5($TOKEN, $json, $random)}"}
]
OrderDict({
"TOKEN": "debugtalk",
variables_mapping (dict)
{
"random": "${gen_random_string(5)}",
"json": {'name': 'user', 'password': '123456'},
"md5": "${gen_md5($TOKEN, $json, $random)}"
})
level (enum): "testcase" or "teststep"
"authorization": "${gen_md5($TOKEN, $data, $random)}",
"data": '{"name": "user", "password": "123456"}',
"TOKEN": "debugtalk",
}
"""
if isinstance(variables, list):
variables = utils.convert_mappinglist_to_orderdict(variables)
variables_mapping = variables_mapping or {}
variables_mapping = utils.ensure_mapping_format(variables_mapping)
for variable_name, variable_value in variables.items():
variable_eval_value = self.eval_content(variable_value)
self.test_variables_mapping = {}
# priority: extracted variable > teststep variable
self.test_variables_mapping.update(variables_mapping)
self.test_variables_mapping.update(self.session_variables_mapping)
if level == "testcase":
self.testcase_runtime_variables_mapping[variable_name] = variable_eval_value
for variable_name, variable_value in variables_mapping.items():
variable_value = self.eval_content(variable_value)
self.update_test_variables(variable_name, variable_value)
self.update_teststep_variables_mapping(variable_name, variable_eval_value)
def update_test_variables(self, variable_name, variable_value):
""" update test variables, these variables are only valid in the current test.
"""
self.test_variables_mapping[variable_name] = variable_value
def update_session_variables(self, variables_mapping):
""" update session with extracted variables mapping.
these variables are valid in the whole running session.
"""
variables_mapping = utils.ensure_mapping_format(variables_mapping)
self.session_variables_mapping.update(variables_mapping)
self.test_variables_mapping.update(self.session_variables_mapping)
def eval_content(self, content):
""" evaluate content recursively, take effect on each variable and function in content.
@@ -82,51 +65,10 @@ class Context(object):
"""
return parser.parse_data(
content,
self.teststep_variables_mapping,
self.TESTCASE_SHARED_FUNCTIONS_MAPPING
self.test_variables_mapping,
self.FUNCTIONS_MAPPING
)
def update_testcase_runtime_variables_mapping(self, variables):
""" update testcase_runtime_variables_mapping with extracted vairables in teststep.
Args:
variables (OrderDict): extracted variables in teststep
"""
for variable_name, variable_value in variables.items():
self.testcase_runtime_variables_mapping[variable_name] = variable_value
self.update_teststep_variables_mapping(variable_name, variable_value)
def update_teststep_variables_mapping(self, variable_name, variable_value):
""" bind and update testcase variables mapping
"""
self.teststep_variables_mapping[variable_name] = variable_value
def get_parsed_request(self, request_dict, level="teststep"):
""" get parsed request with variables and functions.
Args:
request_dict (dict): request config mapping
level (enum): "testcase" or "teststep"
Returns:
dict: parsed request dict
"""
if level == "testcase":
# testcase config request dict has been parsed in parse_tests
self.TESTCASE_SHARED_REQUEST_MAPPING = copy.deepcopy(request_dict)
return self.TESTCASE_SHARED_REQUEST_MAPPING
else:
# teststep
return self.eval_content(
utils.deep_update_dict(
copy.deepcopy(self.TESTCASE_SHARED_REQUEST_MAPPING),
request_dict
)
)
def __eval_check_item(self, validator, resp_obj):
""" evaluate check item in validator.
@@ -188,7 +130,7 @@ class Context(object):
"""
# TODO: move comparator uniform to init_test_suites
comparator = utils.get_uniform_comparator(validator_dict["comparator"])
validate_func = parser.get_mapping_function(comparator, self.TESTCASE_SHARED_FUNCTIONS_MAPPING)
validate_func = parser.get_mapping_function(comparator, self.FUNCTIONS_MAPPING)
check_item = validator_dict["check"]
check_value = validator_dict["check_value"]
@@ -226,11 +168,12 @@ class Context(object):
def validate(self, validators, resp_obj):
""" make validations
"""
evaluated_validators = []
if not validators:
return evaluated_validators
return
logger.log_info("start to validate.")
logger.log_debug("start to validate.")
self.validation_results = []
validate_pass = True
failures = []
@@ -247,10 +190,8 @@ class Context(object):
validate_pass = False
failures.append(str(ex))
evaluated_validators.append(evaluated_validator)
self.validation_results.append(evaluated_validator)
if not validate_pass:
failures_string = "\n".join([failure for failure in failures])
raise exceptions.ValidationFailure(failures_string)
return evaluated_validators

3
httprunner/exceptions.py
View File

@@ -47,6 +47,9 @@ class FunctionNotFound(NotFoundError):
class VariableNotFound(NotFoundError):
pass
class EnvNotFound(NotFoundError):
pass
class ApiNotFound(NotFoundError):
pass

871
httprunner/loader.py
View File

File diff suppressed because it is too large Load Diff

14
httprunner/locusts.py
View File

@@ -7,7 +7,6 @@ import sys
from httprunner.logger import color_print
from httprunner import loader
from locust.main import main
def parse_locustfile(file_path):
@@ -31,6 +30,7 @@ def parse_locustfile(file_path):
return locustfile_path
def gen_locustfile(testcase_file_path):
""" generate locustfile from template.
"""
@@ -49,17 +49,25 @@ def gen_locustfile(testcase_file_path):
return locustfile_path
def start_locust_main():
from locust.main import main
main()
def start_master(sys_argv):
sys_argv.append("--master")
sys.argv = sys_argv
main()
start_locust_main()
def start_slave(sys_argv):
if "--slave" not in sys_argv:
sys_argv.extend(["--slave"])
sys.argv = sys_argv
main()
start_locust_main()
def run_locusts_with_processes(sys_argv, processes_count):
processes = []

784
httprunner/parser.py
View File

@@ -149,20 +149,27 @@ def parse_function(content):
def parse_validator(validator):
""" parse validator, validator maybe in two format
@param (dict) validator
format1: this is kept for compatiblity with the previous versions.
{"check": "status_code", "comparator": "eq", "expect": 201}
{"check": "$resp_body_success", "comparator": "eq", "expect": True}
format2: recommended new version
{'eq': ['status_code', 201]}
{'eq': ['$resp_body_success', True]}
@return (dict) validator info
{
"check": "status_code",
"expect": 201,
"comparator": "eq"
}
""" parse validator
Args:
validator (dict): validator maybe in two formats:
format1: this is kept for compatiblity with the previous versions.
{"check": "status_code", "comparator": "eq", "expect": 201}
{"check": "$resp_body_success", "comparator": "eq", "expect": True}
format2: recommended new version
{'eq': ['status_code', 201]}
{'eq': ['$resp_body_success', True]}
Returns
dict: validator info
{
"check": "status_code",
"expect": 201,
"comparator": "eq"
}
"""
if not isinstance(validator, dict):
raise exceptions.ParamsError("invalid validator: {}".format(validator))
@@ -255,7 +262,7 @@ def substitute_variables(content, variables_mapping):
return content
def parse_parameters(parameters, variables_mapping, functions_mapping):
def parse_parameters(parameters, variables_mapping=None, functions_mapping=None):
""" parse parameters and generate cartesian product.
Args:
@@ -265,7 +272,7 @@ def parse_parameters(parameters, variables_mapping, functions_mapping):
(2) call built-in parameterize function, "${parameterize(account.csv)}"
(3) call custom function in debugtalk.py, "${gen_app_version()}"
variables_mapping (dict): variables mapping loaded from debugtalk.py
variables_mapping (dict): variables mapping loaded from testcase config
functions_mapping (dict): functions mapping loaded from debugtalk.py
Returns:
@@ -280,9 +287,12 @@ def parse_parameters(parameters, variables_mapping, functions_mapping):
>>> parse_parameters(parameters)
"""
variables_mapping = variables_mapping or {}
functions_mapping = functions_mapping or {}
parsed_parameters_list = []
for parameter in parameters:
parameter_name, parameter_content = list(parameter.items())[0]
parameters = utils.ensure_mapping_format(parameters)
for parameter_name, parameter_content in parameters.items():
parameter_name_list = parameter_name.split("-")
if isinstance(parameter_content, list):
@@ -305,16 +315,33 @@ def parse_parameters(parameters, variables_mapping, functions_mapping):
else:
# (2) & (3)
parsed_parameter_content = parse_data(parameter_content, variables_mapping, functions_mapping)
# e.g. [{'app_version': '2.8.5'}, {'app_version': '2.8.6'}]
# e.g. [{"username": "user1", "password": "111111"}, {"username": "user2", "password": "222222"}]
if not isinstance(parsed_parameter_content, list):
raise exceptions.ParamsError("parameters syntax error!")
parameter_content_list = [
# get subset by parameter name
{key: parameter_item[key] for key in parameter_name_list}
for parameter_item in parsed_parameter_content
]
parameter_content_list = []
for parameter_item in parsed_parameter_content:
if isinstance(parameter_item, dict):
# get subset by parameter name
# {"app_version": "${gen_app_version()}"}
# gen_app_version() => [{'app_version': '2.8.5'}, {'app_version': '2.8.6'}]
# {"username-password": "${get_account()}"}
# get_account() => [
# {"username": "user1", "password": "111111"},
# {"username": "user2", "password": "222222"}
# ]
parameter_dict = {key: parameter_item[key] for key in parameter_name_list}
elif isinstance(parameter_item, (list, tuple)):
# {"username-password": "${get_account()}"}
# get_account() => [("user1", "111111"), ("user2", "222222")]
parameter_dict = dict(zip(parameter_name_list, parameter_item))
elif len(parameter_name_list) == 1:
# {"user_agent": "${get_user_agent()}"}
# get_user_agent() => ["iOS/10.1", "iOS/10.2"]
parameter_dict = {
parameter_name_list[0]: parameter_item
}
parameter_content_list.append(parameter_dict)
parsed_parameters_list.append(parameter_content_list)
@@ -325,34 +352,6 @@ def parse_parameters(parameters, variables_mapping, functions_mapping):
## parse content with variables and functions mapping
###############################################################################
def get_builtin_item(item_type, item_name):
"""
Args:
item_type (enum): "variables" or "functions"
item_name (str): variable name or function name
Returns:
variable or function with the name of item_name
"""
# override built_in module with debugtalk.py module
from httprunner import loader
built_in_module = loader.load_builtin_module()
if item_type == "variables":
try:
return built_in_module["variables"][item_name]
except KeyError:
raise exceptions.VariableNotFound("{} is not found.".format(item_name))
else:
# item_type == "functions":
try:
return built_in_module["functions"][item_name]
except KeyError:
raise exceptions.FunctionNotFound("{} is not found.".format(item_name))
def get_mapping_variable(variable_name, variables_mapping):
""" get variable from variables_mapping.
@@ -367,10 +366,10 @@ def get_mapping_variable(variable_name, variables_mapping):
exceptions.VariableNotFound: variable is not found.
"""
if variable_name in variables_mapping:
try:
return variables_mapping[variable_name]
else:
return get_builtin_item("variables", variable_name)
except KeyError:
raise exceptions.VariableNotFound("{} is not found.".format(variable_name))
def get_mapping_function(function_name, functions_mapping):
@@ -392,12 +391,15 @@ def get_mapping_function(function_name, functions_mapping):
return functions_mapping[function_name]
try:
return get_builtin_item("functions", function_name)
except exceptions.FunctionNotFound:
# check if HttpRunner builtin functions
from httprunner import loader
built_in_functions = loader.load_builtin_functions()
return built_in_functions[function_name]
except KeyError:
pass
try:
# check if builtin functions
# check if Python builtin functions
item_func = eval(function_name)
if callable(item_func):
# is builtin function
@@ -436,8 +438,14 @@ def parse_string_functions(content, variables_mapping, functions_mapping):
kwargs = parse_data(kwargs, variables_mapping, functions_mapping)
if func_name in ["parameterize", "P"]:
if len(args) != 1 or kwargs:
raise exceptions.ParamsError("P() should only pass in one argument!")
from httprunner import loader
eval_value = loader.load_csv_file(*args, **kwargs)
eval_value = loader.load_csv_file(args[0])
elif func_name in ["environ", "ENV"]:
if len(args) != 1 or kwargs:
raise exceptions.ParamsError("ENV() should only pass in one argument!")
eval_value = utils.get_os_environ(args[0])
else:
func = get_mapping_function(func_name, functions_mapping)
eval_value = func(*args, **kwargs)
@@ -456,7 +464,7 @@ def parse_string_functions(content, variables_mapping, functions_mapping):
return content
def parse_string_variables(content, variables_mapping):
def parse_string_variables(content, variables_mapping, functions_mapping):
""" parse string content with variables mapping.
Args:
@@ -469,7 +477,7 @@ def parse_string_variables(content, variables_mapping):
Examples:
>>> content = "/api/users/$uid"
>>> variables_mapping = {"$uid": 1000}
>>> parse_string_variables(content, variables_mapping)
>>> parse_string_variables(content, variables_mapping, {})
"/api/users/1000"
"""
@@ -477,30 +485,52 @@ def parse_string_variables(content, variables_mapping):
for variable_name in variables_list:
variable_value = get_mapping_variable(variable_name, variables_mapping)
if variable_name == "request" and isinstance(variable_value, dict) \
and "url" in variable_value and "method" in variable_value:
# call setup_hooks action with $request
for key, value in variable_value.items():
variable_value[key] = parse_data(
value,
variables_mapping,
functions_mapping
)
parsed_variable_value = variable_value
elif "${}".format(variable_name) == variable_value:
# variable_name = "token"
# variables_mapping = {"token": "$token"}
parsed_variable_value = variable_value
else:
parsed_variable_value = parse_data(
variable_value,
variables_mapping,
functions_mapping
)
# TODO: replace variable label from $var to {{var}}
if "${}".format(variable_name) == content:
# content is a variable
content = variable_value
content = parsed_variable_value
else:
# content contains one or several variables
if not isinstance(variable_value, str):
variable_value = builtin_str(variable_value)
if not isinstance(parsed_variable_value, str):
parsed_variable_value = builtin_str(parsed_variable_value)
content = content.replace(
"${}".format(variable_name),
variable_value, 1
parsed_variable_value, 1
)
return content
def parse_data(content, variables_mapping=None, functions_mapping=None):
def parse_data(content, variables_mapping=None, functions_mapping=None, raise_if_variable_not_found=True):
""" parse content with variables mapping
Args:
content (str/dict/list/numeric/bool/type): content to be parsed
variables_mapping (dict): variables mapping.
functions_mapping (dict): functions mapping.
raise_if_variable_not_found (bool): if set False, exception will not raise when VariableNotFound occurred.
Returns:
parsed content.
@@ -528,144 +558,556 @@ def parse_data(content, variables_mapping=None, functions_mapping=None):
if isinstance(content, (list, set, tuple)):
return [
parse_data(item, variables_mapping, functions_mapping)
parse_data(
item,
variables_mapping,
functions_mapping,
raise_if_variable_not_found
)
for item in content
]
if isinstance(content, dict):
parsed_content = {}
for key, value in content.items():
parsed_key = parse_data(key, variables_mapping, functions_mapping)
parsed_value = parse_data(value, variables_mapping, functions_mapping)
parsed_key = parse_data(
key,
variables_mapping,
functions_mapping,
raise_if_variable_not_found
)
parsed_value = parse_data(
value,
variables_mapping,
functions_mapping,
raise_if_variable_not_found
)
parsed_content[parsed_key] = parsed_value
return parsed_content
if isinstance(content, basestring):
# content is in string format here
variables_mapping = variables_mapping or {}
variables_mapping = utils.ensure_mapping_format(variables_mapping or {})
functions_mapping = functions_mapping or {}
content = content.strip()
# replace functions with evaluated value
# Notice: _eval_content_functions must be called before _eval_content_variables
content = parse_string_functions(content, variables_mapping, functions_mapping)
# replace variables with binding value
content = parse_string_variables(content, variables_mapping)
try:
# replace functions with evaluated value
# Notice: parse_string_functions must be called before parse_string_variables
content = parse_string_functions(
content,
variables_mapping,
functions_mapping
)
# replace variables with binding value
content = parse_string_variables(
content,
variables_mapping,
functions_mapping
)
except exceptions.VariableNotFound:
if raise_if_variable_not_found:
raise
return content
def parse_tests(testcases, variables_mapping=None):
""" parse testcases configs, including variables/parameters/name/request.
def _extend_with_api(test_dict, api_def_dict):
""" extend test with api definition, test will merge and override api definition.
Args:
testcases (list): testcase list, with config unparsed.
[
{ # testcase data structure
"config": {
"name": "desc1",
"path": "testcase1_path",
"variables": [], # optional
"request": {} # optional
"refs": {
"debugtalk": {
"variables": {},
"functions": {}
},
"env": {},
"def-api": {},
"def-testcase": {}
}
},
"teststeps": [
# teststep data structure
{
'name': 'test step desc2',
'variables': [], # optional
'extract': [], # optional
'validate': [],
'request': {},
'function_meta': {}
},
teststep2 # another teststep dict
]
},
testcase_dict_2 # another testcase dict
]
variables_mapping (dict): if variables_mapping is specified, it will override variables in config block.
test_dict (dict): test block
api_def_dict (dict): api definition
Returns:
list: parsed testcases list, with config variables/parameters/name/request parsed.
dict: extended test dict.
Examples:
>>> api_def_dict = {
"name": "get token 1",
"request": {...},
"validate": [{'eq': ['status_code', 200]}]
}
>>> test_dict = {
"name": "get token 2",
"extract": {"token": "content.token"},
"validate": [{'eq': ['status_code', 201]}, {'len_eq': ['content.token', 16]}]
}
>>> _extend_with_api(test_dict, api_def_dict)
{
"name": "get token 2",
"request": {...},
"extract": {"token": "content.token"},
"validate": [{'eq': ['status_code', 201]}, {'len_eq': ['content.token', 16]}]
}
"""
variables_mapping = variables_mapping or {}
parsed_testcases_list = []
# override name
api_def_name = api_def_dict.pop("name", "")
test_dict["name"] = test_dict.get("name") or api_def_name
for testcase in testcases:
testcase_config = testcase.setdefault("config", {})
project_mapping = testcase_config.pop(
"refs",
{
"debugtalk": {
"variables": {},
"functions": {}
},
"env": {},
"def-api": {},
"def-testcase": {}
}
# override variables
def_variables = api_def_dict.pop("variables", [])
test_dict["variables"] = utils.extend_variables(
def_variables,
test_dict.get("variables", {})
)
# merge & override validators TODO: relocate
def_raw_validators = api_def_dict.pop("validate", [])
ref_raw_validators = test_dict.get("validate", [])
def_validators = [
parse_validator(validator)
for validator in def_raw_validators
]
ref_validators = [
parse_validator(validator)
for validator in ref_raw_validators
]
test_dict["validate"] = utils.extend_validators(
def_validators,
ref_validators
)
# merge & override extractors
def_extrators = api_def_dict.pop("extract", {})
test_dict["extract"] = utils.extend_variables(
def_extrators,
test_dict.get("extract", {})
)
# TODO: merge & override request
test_dict["request"] = api_def_dict.pop("request", {})
# base_url & verify: priority api_def_dict > test_dict
if api_def_dict.get("base_url"):
test_dict["base_url"] = api_def_dict["base_url"]
if "verify" in api_def_dict:
test_dict["request"]["verify"] = api_def_dict["verify"]
# merge & override setup_hooks
def_setup_hooks = api_def_dict.pop("setup_hooks", [])
ref_setup_hooks = test_dict.get("setup_hooks", [])
extended_setup_hooks = list(set(def_setup_hooks + ref_setup_hooks))
if extended_setup_hooks:
test_dict["setup_hooks"] = extended_setup_hooks
# merge & override teardown_hooks
def_teardown_hooks = api_def_dict.pop("teardown_hooks", [])
ref_teardown_hooks = test_dict.get("teardown_hooks", [])
extended_teardown_hooks = list(set(def_teardown_hooks + ref_teardown_hooks))
if extended_teardown_hooks:
test_dict["teardown_hooks"] = extended_teardown_hooks
# TODO: extend with other api definition items, e.g. times
test_dict.update(api_def_dict)
return test_dict
def _extend_with_testcase(test_dict, testcase_def_dict):
""" extend test with testcase definition
test will merge and override testcase config definition.
Args:
test_dict (dict): test block
testcase_def_dict (dict): testcase definition
Returns:
dict: extended test dict.
"""
# override testcase config variables
testcase_def_dict["config"].setdefault("variables", {})
testcase_def_variables = utils.ensure_mapping_format(testcase_def_dict["config"].get("variables", {}))
testcase_def_variables.update(test_dict.pop("variables", {}))
testcase_def_dict["config"]["variables"] = testcase_def_variables
# override base_url, verify
# priority: testcase config > testsuite tests
test_base_url = test_dict.pop("base_url", "")
if not testcase_def_dict["config"].get("base_url"):
testcase_def_dict["config"]["base_url"] = test_base_url
test_verify = test_dict.pop("verify", True)
testcase_def_dict["config"].setdefault("verify", test_verify)
# override testcase config name, output, etc.
testcase_def_dict["config"].update(test_dict)
test_dict.clear()
test_dict.update(testcase_def_dict)
def __parse_config(config, project_mapping):
""" parse testcase/testsuite config, include variables and name.
"""
# get config variables
raw_config_variables = config.pop("variables", {})
raw_config_variables_mapping = utils.ensure_mapping_format(raw_config_variables)
override_variables = utils.deepcopy_dict(project_mapping.get("variables", {}))
functions = project_mapping.get("functions", {})
# override config variables with passed in variables
raw_config_variables_mapping.update(override_variables)
# parse config variables
parsed_config_variables = {}
for key, value in raw_config_variables_mapping.items():
parsed_value = parse_data(
value,
raw_config_variables_mapping,
functions,
raise_if_variable_not_found=False
)
parsed_config_variables[key] = parsed_value
if parsed_config_variables:
config["variables"] = parsed_config_variables
# parse config name
config["name"] = parse_data(
config.get("name", ""),
parsed_config_variables,
functions
)
# parse config base_url
if "base_url" in config:
config["base_url"] = parse_data(
config["base_url"],
parsed_config_variables,
functions
)
# parse config parameters
config_parameters = testcase_config.pop("parameters", [])
cartesian_product_parameters_list = parse_parameters(
config_parameters,
project_mapping["debugtalk"]["variables"],
project_mapping["debugtalk"]["functions"]
) or [{}]
for parameter_mapping in cartesian_product_parameters_list:
testcase_dict = utils.deepcopy_dict(testcase)
config = testcase_dict.get("config")
def __parse_testcase_tests(tests, config, project_mapping):
""" override tests with testcase config variables, base_url and verify.
test maybe nested testcase.
# parse config variables
raw_config_variables = config.get("variables", [])
parsed_config_variables = parse_data(
raw_config_variables,
project_mapping["debugtalk"]["variables"],
project_mapping["debugtalk"]["functions"]
variables priority:
testcase config > testcase test > testcase_def config > testcase_def test > api
base_url/verify priority:
testcase test > testcase config > testsuite test > testsuite config > api
Args:
tests (list):
config (dict):
project_mapping (dict):
"""
config_variables = config.pop("variables", {})
config_base_url = config.pop("base_url", "")
config_verify = config.pop("verify", True)
functions = project_mapping.get("functions", {})
for test_dict in tests:
# base_url & verify: priority test_dict > config
if (not test_dict.get("base_url")) and config_base_url:
test_dict["base_url"] = config_base_url
test_dict.setdefault("verify", config_verify)
# 1, testcase config => testcase tests
# override test_dict variables
test_dict["variables"] = utils.extend_variables(
test_dict.pop("variables", {}),
config_variables
)
test_dict["variables"] = parse_data(
test_dict["variables"],
test_dict["variables"],
functions,
raise_if_variable_not_found=False
)
# parse test_dict name
test_dict["name"] = parse_data(
test_dict.pop("name", ""),
test_dict["variables"],
functions,
raise_if_variable_not_found=False
)
if "testcase_def" in test_dict:
# test_dict is nested testcase
# 2, testcase test_dict => testcase_def config
testcase_def = test_dict.pop("testcase_def")
_extend_with_testcase(test_dict, testcase_def)
# 3, testcase_def config => testcase_def test_dict
_parse_testcase(test_dict, project_mapping)
else:
if "api_def" in test_dict:
# test_dict has API reference
# 2, test_dict => api
api_def_dict = test_dict.pop("api_def")
_extend_with_api(test_dict, api_def_dict)
if test_dict.get("base_url"):
# parse base_url
base_url = parse_data(
test_dict.pop("base_url"),
test_dict["variables"],
functions
)
# build path with base_url
# variable in current url maybe extracted from former api
request_url = parse_data(
test_dict["request"]["url"],
test_dict["variables"],
functions,
raise_if_variable_not_found=False
)
test_dict["request"]["url"] = utils.build_url(
base_url,
request_url
)
def _parse_testcase(testcase, project_mapping):
""" parse testcase
Args:
testcase (dict):
{
"config": {},
"teststeps": []
}
"""
testcase.setdefault("config", {})
__parse_config(testcase["config"], project_mapping)
__parse_testcase_tests(testcase["teststeps"], testcase["config"], project_mapping)
def __get_parsed_testsuite_testcases(testcases, testsuite_config, project_mapping):
""" override testscases with testsuite config variables, base_url and verify.
variables priority:
parameters > testsuite config > testcase config > testcase_def config > testcase_def tests > api
base_url priority:
testcase_def tests > testcase_def config > testcase config > testsuite config
Args:
testcases (dict):
{
"testcase1 name": {
"testcase": "testcases/create_and_check.yml",
"weight": 2,
"variables": {
"uid": 1000
},
"parameters": {
"uid": [100, 101, 102]
},
"testcase_def": {
"config": {},
"teststeps": []
}
},
"testcase2 name": {}
}
testsuite_config (dict):
{
"name": "testsuite name",
"variables": {
"device_sn": "${gen_random_string(15)}"
},
"base_url": "http://127.0.0.1:5000"
}
project_mapping (dict):
{
"env": {},
"functions": {}
}
"""
testsuite_base_url = testsuite_config.get("base_url")
testsuite_config_variables = testsuite_config.get("variables", {})
functions = project_mapping.get("functions", {})
parsed_testcase_list = []
for testcase_name, testcase in testcases.items():
parsed_testcase = testcase.pop("testcase_def")
parsed_testcase.setdefault("config", {})
parsed_testcase["path"] = testcase["testcase"]
parsed_testcase["config"]["name"] = testcase_name
if "weight" in testcase:
parsed_testcase["config"]["weight"] = testcase["weight"]
# base_url priority: testcase config > testsuite config
parsed_testcase["config"].setdefault("base_url", testsuite_base_url)
# 1, testsuite config => testcase config
# override test_dict variables
testcase_config_variables = utils.extend_variables(
testcase.pop("variables", {}),
testsuite_config_variables
)
# 2, testcase config > testcase_def config
# override testcase_def config variables
parsed_testcase_config_variables = utils.extend_variables(
parsed_testcase["config"].pop("variables", {}),
testcase_config_variables
)
# parse config variables
parsed_config_variables = {}
for key, value in parsed_testcase_config_variables.items():
try:
parsed_value = parse_data(
value,
parsed_testcase_config_variables,
functions
)
except exceptions.VariableNotFound:
pass
parsed_config_variables[key] = parsed_value
if parsed_config_variables:
parsed_testcase["config"]["variables"] = parsed_config_variables
# parse parameters
if "parameters" in testcase and testcase["parameters"]:
cartesian_product_parameters = parse_parameters(
testcase["parameters"],
parsed_config_variables,
functions
)
# priority: passed in > debugtalk.py > parameters > variables
# override variables mapping with parameters mapping
config_variables = utils.override_mapping_list(
parsed_config_variables, parameter_mapping)
# merge debugtalk.py module variables
config_variables.update(project_mapping["debugtalk"]["variables"])
# override variables mapping with passed in variables_mapping
config_variables = utils.override_mapping_list(
config_variables, variables_mapping)
for parameter_variables in cartesian_product_parameters:
# deepcopy to avoid influence between parameters
parsed_testcase_copied = utils.deepcopy_dict(parsed_testcase)
parsed_config_variables_copied = utils.deepcopy_dict(parsed_config_variables)
parsed_testcase_copied["config"]["variables"] = utils.extend_variables(
parsed_config_variables_copied,
parameter_variables
)
_parse_testcase(parsed_testcase_copied, project_mapping)
parsed_testcase_list.append(parsed_testcase_copied)
testcase_dict["config"]["variables"] = config_variables
else:
_parse_testcase(parsed_testcase, project_mapping)
parsed_testcase_list.append(parsed_testcase)
# parse config name
testcase_dict["config"]["name"] = parse_data(
testcase_dict["config"].get("name", ""),
config_variables,
project_mapping["debugtalk"]["functions"]
)
return parsed_testcase_list
# parse config request
testcase_dict["config"]["request"] = parse_data(
testcase_dict["config"].get("request", {}),
config_variables,
project_mapping["debugtalk"]["functions"]
)
# put loaded project functions to config
testcase_dict["config"]["functions"] = project_mapping["debugtalk"]["functions"]
parsed_testcases_list.append(testcase_dict)
def _parse_testsuite(testsuite, project_mapping):
testsuite.setdefault("config", {})
__parse_config(testsuite["config"], project_mapping)
parsed_testcase_list = __get_parsed_testsuite_testcases(
testsuite["testcases"],
testsuite["config"],
project_mapping
)
return parsed_testcase_list
return parsed_testcases_list
def parse_tests(tests_mapping):
""" parse tests and load to parsed testcases
tests include api, testcases and testsuites.
Args:
tests_mapping (dict): project info and testcases list.
{
"project_mapping": {
"PWD": "XXXXX",
"functions": {},
"variables": {}, # optional, priority 1
"env": {}
},
"testsuites": [
{ # testsuite data structure
"config": {},
"testcases": {
"testcase1 name": {
"variables": {
"uid": 1000
},
"parameters": {
"uid": [100, 101, 102]
},
"testcase_def": {
"config": {},
"teststeps": []
}
},
"testcase2 name": {}
}
}
],
"testcases": [
{ # testcase data structure
"config": {
"name": "desc1",
"path": "testcase1_path",
"variables": {}, # optional, priority 2
},
"teststeps": [
# test data structure
{
'name': 'test step desc1',
'variables': [], # optional, priority 3
'extract': [],
'validate': [],
'api_def': {
"variables": {} # optional, priority 4
'request': {},
}
},
test_dict_2 # another test dict
]
},
testcase_dict_2 # another testcase dict
],
"api": {
"variables": {},
"request": {}
}
}
"""
project_mapping = tests_mapping.get("project_mapping", {})
parsed_tests_mapping = {
"project_mapping": project_mapping,
"testcases": []
}
for test_type in tests_mapping:
if test_type == "testsuites":
# load testcases of testsuite
testsuites = tests_mapping["testsuites"]
for testsuite in testsuites:
parsed_testcases = _parse_testsuite(testsuite, project_mapping)
for parsed_testcase in parsed_testcases:
parsed_tests_mapping["testcases"].append(parsed_testcase)
elif test_type == "testcases":
for testcase in tests_mapping["testcases"]:
_parse_testcase(testcase, project_mapping)
parsed_tests_mapping["testcases"].append(testcase)
elif test_type == "apis":
# encapsulate api as a testcase
for api_content in tests_mapping["apis"]:
testcase = {
"teststeps": [api_content]
}
_parse_testcase(testcase, project_mapping)
parsed_tests_mapping["testcases"].append(testcase)
return parsed_tests_mapping

309
httprunner/report.py
View File

@@ -9,6 +9,7 @@ from base64 import b64encode
from collections import Iterable
from datetime import datetime
import requests
from httprunner import loader, logger
from httprunner.__about__ import __version__
from httprunner.compat import basestring, bytes, json, numeric_types
@@ -28,11 +29,25 @@ def get_platform():
def get_summary(result):
""" get summary from test result
Args:
result (instance): HtmlTestResult() instance
Returns:
dict: summary extracted from result.
{
"success": True,
"stat": {},
"time": {},
"records": []
}
"""
summary = {
"success": result.wasSuccessful(),
"stat": {
'testsRun': result.testsRun,
'total': result.testsRun,
'failures': len(result.failures),
'errors': len(result.errors),
'skipped': len(result.skipped),
@@ -40,21 +55,18 @@ def get_summary(result):
'unexpectedSuccesses': len(result.unexpectedSuccesses)
}
}
summary["stat"]["successes"] = summary["stat"]["testsRun"] \
summary["stat"]["successes"] = summary["stat"]["total"] \
- summary["stat"]["failures"] \
- summary["stat"]["errors"] \
- summary["stat"]["skipped"] \
- summary["stat"]["expectedFailures"] \
- summary["stat"]["unexpectedSuccesses"]
if getattr(result, "records", None):
summary["time"] = {
'start_at': result.start_at,
'duration': result.duration
}
summary["records"] = result.records
else:
summary["records"] = []
summary["time"] = {
'start_at': result.start_at,
'duration': result.duration
}
summary["records"] = result.records
return summary
@@ -77,49 +89,220 @@ def aggregate_stat(origin_stat, new_stat):
origin_stat[key] += new_stat[key]
def render_html_report(summary, html_report_name=None, html_report_template=None):
""" render html report with specified report name and template
if html_report_name is not specified, use current datetime
if html_report_template is not specified, use default report template
def stringify_summary(summary):
""" stringify summary, in order to dump json file and generate html report.
"""
if not html_report_template:
html_report_template = os.path.join(
for index, suite_summary in enumerate(summary["details"]):
if not suite_summary.get("name"):
suite_summary["name"] = "testcase {}".format(index)
for record in suite_summary.get("records"):
meta_datas = record['meta_datas']
__stringify_meta_datas(meta_datas)
meta_datas_expanded = []
__expand_meta_datas(meta_datas, meta_datas_expanded)
record["meta_datas_expanded"] = meta_datas_expanded
record["response_time"] = __get_total_response_time(meta_datas_expanded)
def __stringify_request(request_data):
""" stringfy HTTP request data
Args:
request_data (dict): HTTP request data in dict.
{
"url": "http://127.0.0.1:5000/api/get-token",
"method": "POST",
"headers": {
"User-Agent": "python-requests/2.20.0",
"Accept-Encoding": "gzip, deflate",
"Accept": "*/*",
"Connection": "keep-alive",
"user_agent": "iOS/10.3",
"device_sn": "TESTCASE_CREATE_XXX",
"os_platform": "ios",
"app_version": "2.8.6",
"Content-Type": "application/json",
"Content-Length": "52"
},
"json": {
"sign": "cb9d60acd09080ea66c8e63a1c78c6459ea00168"
},
"verify": false
}
"""
for key, value in request_data.items():
if isinstance(value, list):
value = json.dumps(value, indent=2, ensure_ascii=False)
elif isinstance(value, bytes):
try:
encoding = "utf-8"
value = escape(value.decode(encoding))
except UnicodeDecodeError:
pass
elif not isinstance(value, (basestring, numeric_types, Iterable)):
# class instance, e.g. MultipartEncoder()
value = repr(value)
elif isinstance(value, requests.cookies.RequestsCookieJar):
value = value.get_dict()
request_data[key] = value
def __stringify_response(response_data):
""" stringfy HTTP response data
Args:
response_data (dict):
{
"status_code": 404,
"headers": {
"Content-Type": "application/json",
"Content-Length": "30",
"Server": "Werkzeug/0.14.1 Python/3.7.0",
"Date": "Tue, 27 Nov 2018 06:19:27 GMT"
},
"encoding": "None",
"content_type": "application/json",
"ok": false,
"url": "http://127.0.0.1:5000/api/users/9001",
"reason": "NOT FOUND",
"cookies": {},
"json": {
"success": false,
"data": {}
}
}
"""
for key, value in response_data.items():
if isinstance(value, list):
value = json.dumps(value, indent=2, ensure_ascii=False)
elif isinstance(value, bytes):
try:
encoding = response_data.get("encoding")
if not encoding or encoding == "None":
encoding = "utf-8"
if key == "content" and "image" in response_data["content_type"]:
# display image
value = "data:{};base64,{}".format(
response_data["content_type"],
b64encode(value).decode(encoding)
)
else:
value = escape(value.decode(encoding))
except UnicodeDecodeError:
pass
elif not isinstance(value, (basestring, numeric_types, Iterable)):
# class instance, e.g. MultipartEncoder()
value = repr(value)
elif isinstance(value, requests.cookies.RequestsCookieJar):
value = value.get_dict()
response_data[key] = value
def __expand_meta_datas(meta_datas, meta_datas_expanded):
""" expand meta_datas to one level
Args:
meta_datas (dict/list): maybe in nested format
Returns:
list: expanded list in one level
Examples:
>>> meta_datas = [
[
dict1,
dict2
],
dict3
]
>>> meta_datas_expanded = []
>>> __expand_meta_datas(meta_datas, meta_datas_expanded)
>>> print(meta_datas_expanded)
[dict1, dict2, dict3]
"""
if isinstance(meta_datas, dict):
meta_datas_expanded.append(meta_datas)
elif isinstance(meta_datas, list):
for meta_data in meta_datas:
__expand_meta_datas(meta_data, meta_datas_expanded)
def __get_total_response_time(meta_datas_expanded):
""" caculate total response time of all meta_datas
"""
try:
response_time = 0
for meta_data in meta_datas_expanded:
response_time += meta_data["stat"]["response_time_ms"]
return "{:.2f}".format(response_time)
except TypeError:
# failure exists
return "N/A"
def __stringify_meta_datas(meta_datas):
if isinstance(meta_datas, list):
for _meta_data in meta_datas:
__stringify_meta_datas(_meta_data)
elif isinstance(meta_datas, dict):
data_list = meta_datas["data"]
for data in data_list:
__stringify_request(data["request"])
__stringify_response(data["response"])
def render_html_report(summary, report_template=None, report_dir=None):
""" render html report with specified report name and template
Args:
report_template (str): specify html report template path
report_dir (str): specify html report save directory
"""
if not report_template:
report_template = os.path.join(
os.path.abspath(os.path.dirname(__file__)),
"templates",
"report_template.html"
)
logger.log_debug("No html report template specified, use default.")
else:
logger.log_info("render with html report template: {}".format(html_report_template))
logger.log_info("render with html report template: {}".format(report_template))
logger.log_info("Start to render Html report ...")
logger.log_debug("render data: {}".format(summary))
report_dir_path = os.path.join(os.getcwd(), "reports")
report_dir = report_dir or os.path.join(os.getcwd(), "reports")
if not os.path.isdir(report_dir):
os.makedirs(report_dir)
start_at_timestamp = int(summary["time"]["start_at"])
summary["time"]["start_datetime"] = datetime.fromtimestamp(start_at_timestamp).strftime('%Y-%m-%d %H:%M:%S')
if html_report_name:
summary["html_report_name"] = html_report_name
report_dir_path = os.path.join(report_dir_path, html_report_name)
html_report_name += "-{}.html".format(start_at_timestamp)
else:
summary["html_report_name"] = ""
html_report_name = "{}.html".format(start_at_timestamp)
if not os.path.isdir(report_dir_path):
os.makedirs(report_dir_path)
report_path = os.path.join(report_dir, "{}.html".format(start_at_timestamp))
for index, suite_summary in enumerate(summary["details"]):
if not suite_summary.get("name"):
suite_summary["name"] = "test suite {}".format(index)
for record in suite_summary.get("records"):
meta_data = record['meta_data']
stringify_data(meta_data, 'request')
stringify_data(meta_data, 'response')
with io.open(html_report_template, "r", encoding='utf-8') as fp_r:
with io.open(report_template, "r", encoding='utf-8') as fp_r:
template_content = fp_r.read()
report_path = os.path.join(report_dir_path, html_report_name)
with io.open(report_path, 'w', encoding='utf-8') as fp_w:
rendered_content = Template(
template_content,
@@ -132,50 +315,9 @@ def render_html_report(summary, html_report_name=None, html_report_template=None
return report_path
def stringify_data(meta_data, request_or_response):
"""
meta_data = {
"request": {},
"response": {}
}
"""
headers = meta_data[request_or_response]["headers"]
request_or_response_dict = meta_data[request_or_response]
for key, value in request_or_response_dict.items():
if isinstance(value, list):
value = json.dumps(value, indent=2, ensure_ascii=False)
elif isinstance(value, bytes):
try:
encoding = meta_data["response"].get("encoding")
if not encoding or encoding == "None":
encoding = "utf-8"
if request_or_response == "response" and key == "content" \
and "image" in meta_data["response"]["content_type"]:
# display image
value = "data:{};base64,{}".format(
meta_data["response"]["content_type"],
b64encode(value).decode(encoding)
)
else:
value = escape(value.decode(encoding))
except UnicodeDecodeError:
pass
elif not isinstance(value, (basestring, numeric_types, Iterable)):
# class instance, e.g. MultipartEncoder()
value = repr(value)
meta_data[request_or_response][key] = value
class HtmlTestResult(unittest.TextTestResult):
"""A html result class that can generate formatted html results.
Used by TextTestRunner.
""" A html result class that can generate formatted html results.
Used by TextTestRunner.
"""
def __init__(self, stream, descriptions, verbosity):
super(HtmlTestResult, self).__init__(stream, descriptions, verbosity)
@@ -186,11 +328,8 @@ class HtmlTestResult(unittest.TextTestResult):
'name': test.shortDescription(),
'status': status,
'attachment': attachment,
"meta_data": {}
"meta_datas": test.meta_datas
}
if hasattr(test, "meta_data"):
data["meta_data"] = test.meta_data
self.records.append(data)
def startTestRun(self):

75
httprunner/response.py
View File

@@ -15,7 +15,10 @@ class ResponseObject(object):
def __init__(self, resp_obj):
""" initialize with a requests.Response object
@param (requests.Response instance) resp_obj
Args:
resp_obj (instance): requests.Response instance
"""
self.resp_obj = resp_obj
@@ -23,6 +26,8 @@ class ResponseObject(object):
try:
if key == "json":
value = self.resp_obj.json()
elif key == "cookies":
value = self.resp_obj.cookies.get_dict()
else:
value = getattr(self.resp_obj, key)
@@ -36,11 +41,22 @@ class ResponseObject(object):
def _extract_field_with_regex(self, field):
""" extract field from response content with regex.
requests.Response body could be json or html text.
@param (str) field should only be regex string that matched r".*\(.*\).*"
e.g.
self.text: "LB123abcRB789"
field: "LB[\d]*(.*)RB[\d]*"
return: abc
Args:
field (str): regex string that matched r".*\(.*\).*"
Returns:
str: matched content.
Raises:
exceptions.ExtractFailure: If no content matched with regex.
Examples:
>>> # self.text: "LB123abcRB789"
>>> filed = "LB[\d]*(.*)RB[\d]*"
>>> _extract_field_with_regex(field)
abc
"""
matched = re.search(field, self.text)
if not matched:
@@ -53,14 +69,17 @@ class ResponseObject(object):
def _extract_field_with_delimiter(self, field):
""" response content could be json or html text.
@param (str) field should be string joined by delimiter.
e.g.
"status_code"
"headers"
"cookies"
"content"
"headers.content-type"
"content.person.name.first_name"
Args:
field (str): string joined by delimiter.
e.g.
"status_code"
"headers"
"cookies"
"content"
"headers.content-type"
"content.person.name.first_name"
"""
# string.split(sep=None, maxsplit=-1) -> list of strings
# e.g. "content.person.name" => ["content", "person.name"]
@@ -82,7 +101,7 @@ class ResponseObject(object):
# cookies
elif top_query == "cookies":
cookies = self.cookies.get_dict()
cookies = self.cookies
if not sub_query:
# extract cookies
return cookies
@@ -207,21 +226,27 @@ class ResponseObject(object):
def extract_response(self, extractors):
""" extract value from requests.Response and store in OrderedDict.
@param (list) extractors
[
{"resp_status_code": "status_code"},
{"resp_headers_content_type": "headers.content-type"},
{"resp_content": "content"},
{"resp_content_person_first_name": "content.person.name.first_name"}
]
@return (OrderDict) variable binds ordered dict
Args:
extractors (list):
[
{"resp_status_code": "status_code"},
{"resp_headers_content_type": "headers.content-type"},
{"resp_content": "content"},
{"resp_content_person_first_name": "content.person.name.first_name"}
]
Returns:
OrderDict: variable binds ordered dict
"""
if not extractors:
return {}
logger.log_info("start to extract from response object.")
logger.log_debug("start to extract from response object.")
extracted_variables_mapping = OrderedDict()
extract_binds_order_dict = utils.convert_mappinglist_to_orderdict(extractors)
extract_binds_order_dict = utils.ensure_mapping_format(extractors)
for key, field in extract_binds_order_dict.items():
extracted_variables_mapping[key] = self.extract_field(field)

364
httprunner/runner.py
View File

@@ -4,138 +4,163 @@ from unittest.case import SkipTest
from httprunner import exceptions, logger, response, utils
from httprunner.client import HttpSession
from httprunner.compat import OrderedDict
from httprunner.context import Context
from httprunner.context import SessionContext
class Runner(object):
""" Running testcases.
def __init__(self, config_dict=None, http_client_session=None):
"""
"""
self.http_client_session = http_client_session
config_dict = config_dict or {}
self.evaluated_validators = []
Examples:
>>> functions={...}
>>> config = {
"name": "XXXX",
"base_url": "http://127.0.0.1",
"verify": False
}
>>> runner = Runner(config, functions)
>>> test_dict = {
"name": "test description",
"variables": [], # optional
"request": {
"url": "http://127.0.0.1:5000/api/users/1000",
"method": "GET"
}
}
>>> runner.run_test(test_dict)
"""
def __init__(self, config, functions, http_client_session=None):
""" run testcase or testsuite.
Args:
config (dict): testcase/testsuite config dict
{
"name": "ABC",
"variables": {},
"setup_hooks", [],
"teardown_hooks", []
}
http_client_session (instance): requests.Session(), or locust.client.Session() instance.
"""
base_url = config.get("base_url")
self.verify = config.get("verify", True)
self.output = config.get("output", [])
self.functions = functions
self.validation_results = []
# testcase variables
config_variables = config_dict.get("variables", {})
# testcase functions
config_functions = config_dict.get("functions", {})
# testcase setup hooks
testcase_setup_hooks = config_dict.pop("setup_hooks", [])
testcase_setup_hooks = config.get("setup_hooks", [])
# testcase teardown hooks
self.testcase_teardown_hooks = config_dict.pop("teardown_hooks", [])
self.testcase_teardown_hooks = config.get("teardown_hooks", [])
self.context = Context(config_variables, config_functions)
self.init_test(config_dict, "testcase")
self.http_client_session = http_client_session or HttpSession(base_url)
self.session_context = SessionContext(self.functions)
if testcase_setup_hooks:
self.do_hook_actions(testcase_setup_hooks)
self.do_hook_actions(testcase_setup_hooks, "setup")
def __del__(self):
if self.testcase_teardown_hooks:
self.do_hook_actions(self.testcase_teardown_hooks)
def init_test(self, test_dict, level):
""" create/update context variables binds
Args:
test_dict (dict):
level (enum): "testcase" or "teststep"
testcase:
{
"name": "testcase description",
"variables": [], # optional
"request": {
"base_url": "http://127.0.0.1:5000",
"headers": {
"User-Agent": "iOS/2.8.3"
}
}
}
teststep:
{
"name": "teststep description",
"variables": [], # optional
"request": {
"url": "/api/get-token",
"method": "POST",
"headers": {
"Content-Type": "application/json"
}
},
"json": {
"sign": "f1219719911caae89ccc301679857ebfda115ca2"
}
}
Returns:
dict: parsed request dict
self.do_hook_actions(self.testcase_teardown_hooks, "teardown")
def __clear_test_data(self):
""" clear request and response data
"""
test_dict = utils.lower_test_dict_keys(test_dict)
if not isinstance(self.http_client_session, HttpSession):
return
self.context.init_context_variables(level)
variables = test_dict.get('variables') \
or test_dict.get('variable_binds', OrderedDict())
self.context.update_context_variables(variables, level)
self.validation_results = []
self.http_client_session.init_meta_data()
request_config = test_dict.get('request', {})
parsed_request = self.context.get_parsed_request(request_config, level)
def __get_test_data(self):
""" get request/response data and validate results
"""
if not isinstance(self.http_client_session, HttpSession):
return
base_url = parsed_request.pop("base_url", None)
self.http_client_session = self.http_client_session or HttpSession(base_url)
meta_data = self.http_client_session.meta_data
meta_data["validators"] = self.validation_results
return meta_data
return parsed_request
def _handle_skip_feature(self, teststep_dict):
""" handle skip feature for teststep
def _handle_skip_feature(self, test_dict):
""" handle skip feature for test
- skip: skip current test unconditionally
- skipIf: skip current test if condition is true
- skipUnless: skip current test unless condition is true
Args:
teststep_dict (dict): teststep info
test_dict (dict): test info
Raises:
SkipTest: skip teststep
SkipTest: skip test
"""
# TODO: move skip to initialize
skip_reason = None
if "skip" in teststep_dict:
skip_reason = teststep_dict["skip"]
if "skip" in test_dict:
skip_reason = test_dict["skip"]
elif "skipIf" in teststep_dict:
skip_if_condition = teststep_dict["skipIf"]
if self.context.eval_content(skip_if_condition):
elif "skipIf" in test_dict:
skip_if_condition = test_dict["skipIf"]
if self.session_context.eval_content(skip_if_condition):
skip_reason = "{} evaluate to True".format(skip_if_condition)
elif "skipUnless" in teststep_dict:
skip_unless_condition = teststep_dict["skipUnless"]
if not self.context.eval_content(skip_unless_condition):
elif "skipUnless" in test_dict:
skip_unless_condition = test_dict["skipUnless"]
if not self.session_context.eval_content(skip_unless_condition):
skip_reason = "{} evaluate to False".format(skip_unless_condition)
if skip_reason:
raise SkipTest(skip_reason)
def do_hook_actions(self, actions):
for action in actions:
logger.log_debug("call hook: {}".format(action))
# TODO: check hook function if valid
self.context.eval_content(action)
def do_hook_actions(self, actions, hook_type):
""" call hook actions.
def run_test(self, teststep_dict):
Args:
actions (list): each action in actions list maybe in two format.
format1 (dict): assignment, the value returned by hook function will be assigned to variable.
{"var": "${func()}"}
format2 (str): only call hook functions.
${func()}
hook_type (enum): setup/teardown
"""
logger.log_debug("call {} hook actions.".format(hook_type))
for action in actions:
if isinstance(action, dict) and len(action) == 1:
# format 1
# {"var": "${func()}"}
var_name, hook_content = list(action.items())[0]
logger.log_debug("assignment with hook: {} = {}".format(var_name, hook_content))
self.session_context.update_test_variables(
var_name,
self.session_context.eval_content(hook_content)
)
else:
# format 2
logger.log_debug("call hook function: {}".format(action))
# TODO: check hook function if valid
self.session_context.eval_content(action)
def _run_test(self, test_dict):
""" run single teststep.
Args:
teststep_dict (dict): teststep info
test_dict (dict): teststep info
{
"name": "teststep description",
"skip": "skip this test unconditionally",
"times": 3,
"variables": [], # optional, override
"variables": [], # optional, override
"request": {
"url": "http://127.0.0.1:5000/api/users/1000",
"method": "POST",
@@ -144,9 +169,9 @@ class Runner(object):
"authorization": "$authorization",
"random": "$random"
},
"body": '{"name": "user", "password": "123456"}'
"json": {"name": "user", "password": "123456"}
},
"extract": [], # optional
"extract": {}, # optional
"validate": [], # optional
"setup_hooks": [], # optional
"teardown_hooks": [] # optional
@@ -158,24 +183,35 @@ class Runner(object):
exceptions.ExtractFailure
"""
# clear meta data first to ensure independence for each test
self.__clear_test_data()
# check skip
self._handle_skip_feature(teststep_dict)
self._handle_skip_feature(test_dict)
# prepare
extractors = teststep_dict.get("extract", []) or teststep_dict.get("extractors", [])
validators = teststep_dict.get("validate", []) or teststep_dict.get("validators", [])
parsed_request = self.init_test(teststep_dict, level="teststep")
self.context.update_teststep_variables_mapping("request", parsed_request)
test_dict = utils.lower_test_dict_keys(test_dict)
test_variables = test_dict.get("variables", {})
self.session_context.init_test_variables(test_variables)
# teststep name
test_name = test_dict.get("name", "")
# parse test request
raw_request = test_dict.get('request', {})
parsed_test_request = self.session_context.eval_content(raw_request)
self.session_context.update_test_variables("request", parsed_test_request)
# setup hooks
setup_hooks = teststep_dict.get("setup_hooks", [])
setup_hooks.insert(0, "${setup_hook_prepare_kwargs($request)}")
self.do_hook_actions(setup_hooks)
setup_hooks = test_dict.get("setup_hooks", [])
if setup_hooks:
self.do_hook_actions(setup_hooks, "setup")
try:
url = parsed_request.pop('url')
method = parsed_request.pop('method')
group_name = parsed_request.pop("group", None)
url = parsed_test_request.pop('url')
method = parsed_test_request.pop('method')
parsed_test_request.setdefault("verify", self.verify)
group_name = parsed_test_request.pop("group", None)
except KeyError:
raise exceptions.ParamsError("URL or METHOD missed!")
@@ -188,52 +224,144 @@ class Runner(object):
raise exceptions.ParamsError(err_msg)
logger.log_info("{method} {url}".format(method=method, url=url))
logger.log_debug("request kwargs(raw): {kwargs}".format(kwargs=parsed_request))
logger.log_debug("request kwargs(raw): {kwargs}".format(kwargs=parsed_test_request))
# request
resp = self.http_client_session.request(
method,
url,
name=group_name,
**parsed_request
name=(group_name or test_name),
**parsed_test_request
)
resp_obj = response.ResponseObject(resp)
# teardown hooks
teardown_hooks = teststep_dict.get("teardown_hooks", [])
teardown_hooks = test_dict.get("teardown_hooks", [])
if teardown_hooks:
logger.log_info("start to run teardown hooks")
self.context.update_teststep_variables_mapping("response", resp_obj)
self.do_hook_actions(teardown_hooks)
self.session_context.update_test_variables("response", resp_obj)
self.do_hook_actions(teardown_hooks, "teardown")
# extract
extractors = test_dict.get("extract", {})
extracted_variables_mapping = resp_obj.extract_response(extractors)
self.context.update_testcase_runtime_variables_mapping(extracted_variables_mapping)
self.session_context.update_session_variables(extracted_variables_mapping)
# validate
validators = test_dict.get("validate", [])
try:
self.evaluated_validators = self.context.validate(validators, resp_obj)
self.session_context.validate(validators, resp_obj)
except (exceptions.ParamsError, exceptions.ValidationFailure, exceptions.ExtractFailure):
err_msg = "{} DETAILED REQUEST & RESPONSE {}\n".format("*" * 32, "*" * 32)
# log request
err_req_msg = "request: \n"
err_req_msg += "headers: {}\n".format(parsed_request.pop("headers", {}))
for k, v in parsed_request.items():
err_req_msg += "{}: {}\n".format(k, repr(v))
logger.log_error(err_req_msg)
err_msg += "====== request details ======\n"
err_msg += "url: {}\n".format(url)
err_msg += "method: {}\n".format(method)
err_msg += "headers: {}\n".format(parsed_test_request.pop("headers", {}))
for k, v in parsed_test_request.items():
v = utils.omit_long_data(v)
err_msg += "{}: {}\n".format(k, repr(v))
err_msg += "\n"
# log response
err_resp_msg = "response: \n"
err_resp_msg += "status_code: {}\n".format(resp_obj.status_code)
err_resp_msg += "headers: {}\n".format(resp_obj.headers)
err_resp_msg += "body: {}\n".format(repr(resp_obj.text))
logger.log_error(err_resp_msg)
err_msg += "====== response details ======\n"
err_msg += "status_code: {}\n".format(resp_obj.status_code)
err_msg += "headers: {}\n".format(resp_obj.headers)
err_msg += "body: {}\n".format(repr(resp_obj.text))
logger.log_error(err_msg)
raise
finally:
self.validation_results = self.session_context.validation_results
def _run_testcase(self, testcase_dict):
""" run single testcase.
"""
self.meta_datas = []
config = testcase_dict.get("config", {})
base_url = config.get("base_url")
# each testcase should have individual session.
http_client_session = self.http_client_session.__class__(base_url)
test_runner = Runner(config, self.functions, http_client_session)
tests = testcase_dict.get("teststeps", [])
for index, test_dict in enumerate(tests):
try:
test_runner.run_test(test_dict)
except Exception:
# log exception request_type and name for locust stat
self.exception_request_type = test_runner.exception_request_type
self.exception_name = test_runner.exception_name
raise
finally:
_meta_datas = test_runner.meta_datas
self.meta_datas.append(_meta_datas)
self.session_context.update_session_variables(test_runner.extract_sessions())
def run_test(self, test_dict):
""" run single teststep of testcase.
test_dict may be in 3 types.
Args:
test_dict (dict):
# teststep
{
"name": "teststep description",
"variables": [], # optional
"request": {
"url": "http://127.0.0.1:5000/api/users/1000",
"method": "GET"
}
}
# nested testcase
{
"config": {...},
"teststeps": [
{...},
{...}
]
}
# TODO: function
{
"name": "exec function",
"function": "${func()}"
}
"""
self.meta_datas = None
if "teststeps" in test_dict:
# nested testcase
self._run_testcase(test_dict)
else:
# api
try:
self._run_test(test_dict)
except Exception:
# log exception request_type and name for locust stat
self.exception_request_type = test_dict["request"]["method"]
self.exception_name = test_dict.get("name")
raise
finally:
self.meta_datas = self.__get_test_data()
def extract_sessions(self):
"""
"""
return self.extract_output(self.output)
def extract_output(self, output_variables_list):
""" extract output variables
"""
variables_mapping = self.context.teststep_variables_mapping
variables_mapping = self.session_context.session_variables_mapping
output = {}
for variable in output_variables_list:

35
httprunner/templates/locustfile_template
View File

@@ -3,7 +3,7 @@ import random
import zmq
from httprunner.exceptions import MyBaseError, MyBaseFailure
from httprunner.loader import load_locust_tests
from httprunner.api import prepare_locust_tests
from httprunner.runner import Runner
from locust import HttpLocust, TaskSet, task
from locust.events import request_failure
@@ -15,22 +15,20 @@ logging.getLogger('locust.runners').setLevel(logging.INFO)
class WebPageTasks(TaskSet):
def on_start(self):
self.test_runner = Runner(self.locust.config, self.client)
self.testcases = load_locust_tests(self.locust.file_path)
self.test_runner = Runner(self.locust.config, self.locust.functions, self.client)
@task(weight=1)
@task
def test_any(self):
teststeps = random.choice(self.locust.tests)
for teststep in teststeps:
try:
self.test_runner.run_test(teststep)
except (MyBaseError, MyBaseFailure) as ex:
request_failure.fire(
request_type=teststep.get("request", {}).get("method"),
name=teststep.get("name"),
response_time=0,
exception=ex
)
test_dict = random.choice(self.locust.tests)
try:
self.test_runner.run_test(test_dict)
except (AssertionError, MyBaseError, MyBaseFailure) as ex:
request_failure.fire(
request_type=self.test_runner.exception_request_type,
name=self.test_runner.exception_name,
response_time=0,
exception=ex
)
class WebPageUser(HttpLocust):
@@ -39,8 +37,9 @@ class WebPageUser(HttpLocust):
max_wait = 30
file_path = "$TESTCASE_FILE"
locust_tests = load_locust_tests(file_path)
config = locust_tests["config"]
locust_tests = prepare_locust_tests(file_path)
functions = locust_tests["functions"]
tests = locust_tests["tests"]
config = {}
host = config.get('request', {}).get('base_url', '')
host = config.get('base_url', '')

114
httprunner/templates/report_template.html
View File

@@ -47,7 +47,8 @@
background-color: lightgrey;
font-size: smaller;
padding: 5px 10px;
text-align: center;
line-height: 20px;
text-align: left;
}
.details .success {
background-color: greenyellow;
@@ -75,6 +76,7 @@
a.button{
color: gray;
text-decoration: none;
display: inline-block;
}
.button:hover {
background: #2cffbd;
@@ -90,6 +92,7 @@
transition: opacity 500ms;
visibility: hidden;
opacity: 0;
line-height: 25px;
}
.overlay:target {
visibility: visible;
@@ -129,6 +132,9 @@
overflow: auto;
text-align: left;
}
.popup .separator {
color:royalblue
}
@media screen and (max-width: 700px) {
.box {
@@ -147,7 +153,6 @@
<h2>Summary</h2>
<table id="summary">
<tr>
<th>START AT</th>
<td colspan="4">{{time.start_datetime}}</td>
@@ -163,22 +168,14 @@
<td colspan="2">{{ platform.platform }}</td>
</tr>
<tr>
<th>TOTAL</th>
<th>SUCCESS</th>
<th>FAILED</th>
<th>ERROR</th>
<th>SKIPPED</th>
<!-- <th>ExpectedFailure</th>
<th>UnexpectedSuccess</th> -->
<th>STAT</th>
<th colspan="2">TESTCASES (success/fail)</th>
<th colspan="2">TESTSTEPS (success/fail/error/skip)</th>
</tr>
<tr>
<td>{{stat.testsRun}}</td>
<td>{{stat.successes}}</td>
<td>{{stat.failures}}</td>
<td>{{stat.errors}}</td>
<td>{{stat.skipped}}</td>
<!-- <td>{{stat.expectedFailures}}</td>
<td>{{stat.unexpectedSuccesses}}</td> -->
<td>total (details) =></td>
<td colspan="2">{{stat.testcases.total}} ({{stat.testcases.success}}/{{stat.testcases.fail}})</td>
<td colspan="2">{{stat.teststeps.total}} ({{stat.teststeps.successes}}/{{stat.teststeps.failures}}/{{stat.teststeps.errors}}/{{stat.teststeps.skipped}})</td>
</tr>
</table>
@@ -189,36 +186,7 @@
<h3>{{test_suite_summary.name}}</h3>
<table id="suite_{{suite_index}}" class="details">
<tr>
<th>base_url</th>
<td colspan="2">{{test_suite_summary.base_url}}</td>
<th colspan="2" class="detail">
<a class="button" href="#suite_output_{{suite_index}}">parameters & output</a>
<div id="suite_output_{{suite_index}}" class="overlay">
<div class="popup">
<h2>Parameters and Output</h2>
<a class="close" href="#suite_{{suite_index}}">&times;</a>
<div class="content">
<div style="overflow: auto">
<table>
<tr>
<th>variables</th>
<th>output</th>
</tr>
{% if in_out in test_suite_summary %}
<tr>
<td>{{test_suite_summary.in_out.in}}</td>
<td>{{test_suite_summary.in_out.out}}</td>
</tr>
{% endif %}
</table>
</div>
</div>
</div>
</div>
</td>
</tr>
<tr>
<td>TOTAL: {{test_suite_summary.stat.testsRun}}</td>
<td>TOTAL: {{test_suite_summary.stat.total}}</td>
<td>SUCCESS: {{test_suite_summary.stat.successes}}</td>
<td>FAILED: {{test_suite_summary.stat.failures}}</td>
<td>ERROR: {{test_suite_summary.stat.errors}}</td>
@@ -233,28 +201,39 @@
{% for record in test_suite_summary.records %}
{% set record_index = "{}_{}".format(suite_index, loop.index) %}
{% set record_meta_datas = record.meta_datas_expanded %}
<tr id="record_{{record_index}}">
<th class="{{record.status}}" style="width:5em;">{{record.status}}</td>
<th class="{{record.status}}" style="width:5em;">{{record.status}}</th>
<td colspan="2">{{record.name}}</td>
<td style="text-align:center;width:6em;">{{ record.meta_data.response.response_time_ms }} ms</td>
<td style="text-align:center;width:6em;">{{ record.response_time }} ms</td>
<td class="detail">
<a class="button" href="#popup_log_{{record_index}}">log</a>
<div id="popup_log_{{record_index}}" class="overlay">
{% for meta_data in record_meta_datas %}
{% set meta_data_index = "{}_{}".format(record_index, loop.index) %}
<a class="button" href="#popup_log_{{meta_data_index}}">log-{{loop.index}}</a>
<div id="popup_log_{{meta_data_index}}" class="overlay">
<div class="popup">
<h2>Request and Response data</h2>
<a class="close" href="#record_{{record_index}}">&times;</a>
<a class="close" href="#record_{{meta_data_index}}">&times;</a>
<div class="content">
<h3>Name: {{ meta_data.name }}</h3>
{% for req_resp in meta_data.data %}
{% if loop.index > 1 %}
<div class="separator">==================================== redirect to ====================================</div>
{% endif %}
<h3>Request:</h3>
<div style="overflow: auto">
<table>
{% for key, value in record.meta_data.request.items() %}
{% for key, value in req_resp.request.items() %}
<tr>
<th>{{key}}</th>
<td>
{% if key == "headers" %}
{% for header_key, header_value in record.meta_data.request.headers.items() %}
{% for header_key, header_value in req_resp.request.headers.items() %}
<div>
<strong>{{ header_key }}</strong>: {{ header_value }}
</div>
@@ -271,33 +250,33 @@
<h3>Response:</h3>
<div style="overflow: auto">
<table>
{% for key, value in record.meta_data.response.items() %}
{% if key in ["text", "json", "elapsed_ms", "response_time_ms", "content_size", "content_type"] %}
{% continue %}
{% endif %}
{% for key, value in req_resp.response.items() %}
<tr>
<th>{{key}}</th>
<td>
{% if key == "headers" %}
{% for header_key, header_value in record.meta_data.response.headers.items() %}
{% for header_key, header_value in req_resp.response.headers.items() %}
<div>
<strong>{{ header_key }}</strong>: {{ header_value }}
</div>
{% endfor %}
{% elif key == "content" %}
{% if "image" in record.meta_data.response.content_type %}
<img src="{{ record.meta_data.response.content }}" />
{% if "image" in req_resp.response.content_type %}
<img src="{{ req_resp.response.content }}" />
{% else %}
<pre>{{ record.meta_data.response.text | e }}</pre>
{{ value }}
{% endif %}
{% elif key == "text" %}
<pre>{{ req_resp.response.text | e }}</pre>
{% else %}
{{value}}
{{ value }}
{% endif %}
</td>
</tr>
{% endfor %}
</table>
</div>
{% endfor %}
<h3>Validators:</h3>
<div style="overflow: auto">
@@ -308,7 +287,7 @@
<th>expect value</th>
<th>actual value</th>
</tr>
{% for validator in record.meta_data.validators %}
{% for validator in meta_data.validators %}
<tr>
{% if validator.check_result == "pass" %}
<td class="passed">
@@ -332,15 +311,15 @@
<table>
<tr>
<th>content_size(bytes)</th>
<td>{{ record.meta_data.response.content_size }}</td>
<td>{{ meta_data.stat.content_size }}</td>
</tr>
<tr>
<th>response_time(ms)</th>
<td>{{ record.meta_data.response.response_time_ms }}</td>
<td>{{ meta_data.stat.response_time_ms }}</td>
</tr>
<tr>
<th>elapsed(ms)</th>
<td>{{ record.meta_data.response.elapsed_ms }}</td>
<td>{{ meta_data.stat.elapsed_ms }}</td>
</tr>
</table>
</div>
@@ -348,6 +327,7 @@
</div>
</div>
</div>
{% endfor %}
{% if record.attachment %}
<a class="button" href="#popup_attachment_{{record_index}}">traceback</a>
@@ -355,7 +335,7 @@
<div class="popup">
<h2>Traceback Message</h2>
<a class="close" href="#record_{{record_index}}">&times;</a>
<div class="content"><pre>{{ record.attachment }}</pre></div>
<div class="content"><pre>{{ record.attachment | e }}</pre></div>
</div>
</div>
{% endif %}

465
httprunner/utils.py
View File

@@ -1,23 +1,20 @@
# encoding: utf-8
import collections
import copy
import io
import itertools
import json
import os.path
import re
import string
from datetime import datetime
from httprunner import exceptions, logger
from httprunner.compat import OrderedDict, basestring, is_py2
from httprunner.compat import basestring, bytes, is_py2
from httprunner.exceptions import ParamsError
def remove_prefix(text, prefix):
""" remove prefix from text
"""
if text.startswith(prefix):
return text[len(prefix):]
return text
absolute_http_url_regexp = re.compile(r"^https?://", re.I)
def set_os_environ(variables_mapping):
@@ -25,13 +22,59 @@ def set_os_environ(variables_mapping):
"""
for variable in variables_mapping:
os.environ[variable] = variables_mapping[variable]
logger.log_debug("Loaded variable: {}".format(variable))
logger.log_debug("Set OS environment variable: {}".format(variable))
def unset_os_environ(variables_mapping):
""" set variables mapping to os.environ
"""
for variable in variables_mapping:
os.environ.pop(variable)
logger.log_debug("Unset OS environment variable: {}".format(variable))
def get_os_environ(variable_name):
""" get value of environment variable.
Args:
variable_name(str): variable name
Returns:
value of environment variable.
Raises:
exceptions.EnvNotFound: If environment variable not found.
"""
try:
return os.environ[variable_name]
except KeyError:
raise exceptions.EnvNotFound(variable_name)
def build_url(base_url, path):
""" prepend url with hostname unless it's already an absolute URL """
if absolute_http_url_regexp.match(path):
return path
elif base_url:
return "{}/{}".format(base_url.rstrip("/"), path.lstrip("/"))
else:
raise ParamsError("base url missed!")
def query_json(json_content, query, delimiter='.'):
""" Do an xpath-like query with json_content.
@param (dict/list/string) json_content
json_content = {
Args:
json_content (dict/list/string): content to be queried.
query (str): query string.
delimiter (str): delimiter symbol.
Returns:
str: queried result.
Examples:
>>> json_content = {
"ids": [1, 2, 3, 4],
"person": {
"name": {
@@ -42,11 +85,16 @@ def query_json(json_content, query, delimiter='.'):
"cities": ["Guangzhou", "Shenzhen"]
}
}
@param (str) query
"person.name.first_name" => "Leo"
"person.name.first_name.0" => "L"
"person.cities.0" => "Guangzhou"
@return queried result
>>>
>>> query_json(json_content, "person.name.first_name")
>>> Leo
>>>
>>> query_json(json_content, "person.name.first_name.0")
>>> L
>>>
>>> query_json(json_content, "person.cities.0")
>>> Guangzhou
"""
raise_flag = False
response_body = u"response body: {}\n".format(json_content)
@@ -104,6 +152,7 @@ def get_uniform_comparator(comparator):
else:
return comparator
def deep_update_dict(origin_dict, override_dict):
""" update origin dict with override dict recursively
e.g. origin_dict = {'a': 1, 'b': {'c': 2, 'd': 4}}
@@ -125,6 +174,31 @@ def deep_update_dict(origin_dict, override_dict):
return origin_dict
def convert_dict_to_params(src_dict):
""" convert dict to params string
Args:
src_dict (dict): source mapping data structure
Returns:
str: string params data
Examples:
>>> src_dict = {
"a": 1,
"b": 2
}
>>> convert_dict_to_params(src_dict)
>>> "a=1&b=2"
"""
return "&".join([
"{}={}".format(key, value)
for key, value in src_dict.items()
])
def lower_dict_keys(origin_dict):
""" convert keys in dict to lower case
@@ -162,6 +236,7 @@ def lower_dict_keys(origin_dict):
for key, value in origin_dict.items()
}
def lower_test_dict_keys(test_dict):
""" convert keys in test_dict to lower case, convertion will occur in two places:
1, all keys in test_dict;
@@ -176,32 +251,6 @@ def lower_test_dict_keys(test_dict):
return test_dict
def convert_mappinglist_to_orderdict(mapping_list):
""" convert mapping list to ordered dict
Args:
mapping_list (list):
[
{"a": 1},
{"b": 2}
]
Returns:
OrderedDict: converted mapping in OrderedDict
OrderDict(
{
"a": 1,
"b": 2
}
)
"""
ordered_dict = OrderedDict()
for map_dict in mapping_list:
ordered_dict.update(map_dict)
return ordered_dict
def deepcopy_dict(data):
""" deepcopy dict data, ignore file object (_io.BufferedReader)
@@ -239,75 +288,148 @@ def deepcopy_dict(data):
return copied_data
def update_ordered_dict(ordered_dict, override_mapping):
""" override ordered_dict with new mapping.
def ensure_mapping_format(variables):
""" ensure variables are in mapping format.
Args:
ordered_dict (OrderDict): original ordered dict
override_mapping (dict): new variables mapping
variables (list/dict): original variables
Returns:
OrderDict: new overrided variables mapping.
Examples:
>>> ordered_dict = OrderDict({"a": 1, "b": 2})
>>> override_mapping = {"a": 3, "c": 4}
>>> update_ordered_dict(ordered_dict, override_mapping)
OrderDict({"a": 3, "b": 2, "c": 4})
"""
new_ordered_dict = copy.copy(ordered_dict)
for var, value in override_mapping.items():
new_ordered_dict.update({var: value})
return new_ordered_dict
def override_mapping_list(variables, new_mapping):
""" override variables with new mapping.
Args:
variables (list): variables list
[
{"var_a": 1},
{"var_b": "world"}
]
new_mapping (dict): overrided variables mapping
{
"var_a": "hello"
}
Returns:
OrderedDict: overrided variables mapping.
dict: ensured variables in dict format
Examples:
>>> variables = [
{"var_a": 1},
{"var_b": "world"}
{"a": 1},
{"b": 2}
]
>>> new_mapping = {
"var_a": "hello"
>>> print(ensure_mapping_format(variables))
{
"a": 1,
"b": 2
}
>>> override_mapping_list(variables, new_mapping)
OrderedDict(
{
"var_a": "hello",
"var_b": "world"
}
)
"""
if isinstance(variables, list):
variables_ordered_dict = convert_mappinglist_to_orderdict(variables)
elif isinstance(variables, (OrderedDict, dict)):
variables_ordered_dict = variables
else:
raise exceptions.ParamsError("variables error!")
variables_dict = {}
for map_dict in variables:
variables_dict.update(map_dict)
return update_ordered_dict(
variables_ordered_dict,
new_mapping
)
return variables_dict
elif isinstance(variables, dict):
return variables
else:
raise exceptions.ParamsError("variables format error!")
def _convert_validators_to_mapping(validators):
""" convert validators list to mapping.
Args:
validators (list): validators in list
Returns:
dict: validators mapping, use (check, comparator) as key.
Examples:
>>> validators = [
{"check": "v1", "expect": 201, "comparator": "eq"},
{"check": {"b": 1}, "expect": 200, "comparator": "eq"}
]
>>> _convert_validators_to_mapping(validators)
{
("v1", "eq"): {"check": "v1", "expect": 201, "comparator": "eq"},
('{"b": 1}', "eq"): {"check": {"b": 1}, "expect": 200, "comparator": "eq"}
}
"""
validators_mapping = {}
for validator in validators:
if not isinstance(validator["check"], collections.Hashable):
check = json.dumps(validator["check"])
else:
check = validator["check"]
key = (check, validator["comparator"])
validators_mapping[key] = validator
return validators_mapping
def extend_validators(raw_validators, override_validators):
""" extend raw_validators with override_validators.
override_validators will merge and override raw_validators.
Args:
raw_validators (dict):
override_validators (dict):
Returns:
list: extended validators
Examples:
>>> raw_validators = [{'eq': ['v1', 200]}, {"check": "s2", "expect": 16, "comparator": "len_eq"}]
>>> override_validators = [{"check": "v1", "expect": 201}, {'len_eq': ['s3', 12]}]
>>> extend_validators(raw_validators, override_validators)
[
{"check": "v1", "expect": 201, "comparator": "eq"},
{"check": "s2", "expect": 16, "comparator": "len_eq"},
{"check": "s3", "expect": 12, "comparator": "len_eq"}
]
"""
if not raw_validators:
return override_validators
elif not override_validators:
return raw_validators
else:
def_validators_mapping = _convert_validators_to_mapping(raw_validators)
ref_validators_mapping = _convert_validators_to_mapping(override_validators)
def_validators_mapping.update(ref_validators_mapping)
return list(def_validators_mapping.values())
def extend_variables(raw_variables, override_variables):
""" extend raw_variables with override_variables.
override_variables will merge and override raw_variables.
Args:
raw_variables (list):
override_variables (list):
Returns:
dict: extended variables mapping
Examples:
>>> raw_variables = [{"var1": "val1"}, {"var2": "val2"}]
>>> override_variables = [{"var1": "val111"}, {"var3": "val3"}]
>>> extend_variables(raw_variables, override_variables)
{
'var1', 'val111',
'var2', 'val2',
'var3', 'val3'
}
"""
if not raw_variables:
override_variables_mapping = ensure_mapping_format(override_variables)
return override_variables_mapping
elif not override_variables:
raw_variables_mapping = ensure_mapping_format(raw_variables)
return raw_variables_mapping
else:
raw_variables_mapping = ensure_mapping_format(raw_variables)
override_variables_mapping = ensure_mapping_format(override_variables)
raw_variables_mapping.update(override_variables_mapping)
return raw_variables_mapping
def get_testcase_io(testcase):
@@ -322,13 +444,13 @@ def get_testcase_io(testcase):
dict: input(variables) and output mapping.
"""
runner = testcase.runner
variables = testcase.config.get("variables", [])
test_runner = testcase.runner
variables = testcase.config.get("variables", {})
output_list = testcase.config.get("output", [])
return {
"in": dict(variables),
"out": runner.extract_output(output_list)
"in": variables,
"out": test_runner.extract_output(output_list)
}
@@ -367,7 +489,7 @@ def print_io(in_out):
def prepare_content(var_type, in_out):
content = ""
for variable, value in in_out.items():
if isinstance(value, tuple):
if isinstance(value, (tuple, collections.deque)):
continue
elif isinstance(value, (dict, list)):
value = json.dumps(value)
@@ -426,21 +548,28 @@ def create_scaffold(project_name):
def gen_cartesian_product(*args):
""" generate cartesian product for lists
@param
(list) args
[{"a": 1}, {"a": 2}],
Args:
args (list of list): lists to be generated with cartesian product
Returns:
list: cartesian product in list
Examples:
>>> arg1 = [{"a": 1}, {"a": 2}]
>>> arg2 = [{"x": 111, "y": 112}, {"x": 121, "y": 122}]
>>> args = [arg1, arg2]
>>> gen_cartesian_product(*args)
>>> # same as below
>>> gen_cartesian_product(arg1, arg2)
[
{"x": 111, "y": 112},
{"x": 121, "y": 122}
{'a': 1, 'x': 111, 'y': 112},
{'a': 1, 'x': 121, 'y': 122},
{'a': 2, 'x': 111, 'y': 112},
{'a': 2, 'x': 121, 'y': 122}
]
@return
cartesian product in list
[
{'a': 1, 'x': 111, 'y': 112},
{'a': 1, 'x': 121, 'y': 122},
{'a': 2, 'x': 111, 'y': 112},
{'a': 2, 'x': 121, 'y': 122}
]
"""
if not args:
return []
@@ -504,6 +633,110 @@ def prettify_json_file(file_list):
print("success: {}".format(outfile))
def omit_long_data(body, omit_len=512):
""" omit too long str/bytes
"""
if not isinstance(body, basestring):
return body
body_len = len(body)
if body_len <= omit_len:
return body
omitted_body = body[0:omit_len]
appendix_str = " ... OMITTED {} CHARACTORS ...".format(body_len - omit_len)
if isinstance(body, bytes):
appendix_str = appendix_str.encode("utf-8")
return omitted_body + appendix_str
def dump_json_file(json_data, pwd_dir_path, dump_file_name):
""" dump json data to file
"""
logs_dir_path = os.path.join(pwd_dir_path, "logs")
if not os.path.isdir(logs_dir_path):
os.makedirs(logs_dir_path)
dump_file_path = os.path.join(logs_dir_path, dump_file_name)
try:
with io.open(dump_file_path, 'w', encoding='utf-8') as outfile:
if is_py2:
outfile.write(
unicode(json.dumps(
json_data,
indent=4,
separators=(',', ':'),
ensure_ascii=False
))
)
else:
json.dump(json_data, outfile, indent=4, separators=(',', ':'))
msg = "dump file: {}".format(dump_file_path)
logger.color_print(msg, "BLUE")
except TypeError:
msg = "Failed to dump json file: {}".format(dump_file_path)
logger.color_print(msg, "RED")
def _prepare_dump_info(project_mapping, tag_name):
""" prepare dump file info.
"""
test_path = project_mapping.get("test_path") or "tests_mapping"
pwd_dir_path = project_mapping.get("PWD") or os.getcwd()
file_name, file_suffix = os.path.splitext(os.path.basename(test_path.rstrip("/")))
dump_file_name = "{}.{}.json".format(file_name, tag_name)
return pwd_dir_path, dump_file_name
def dump_tests(tests_mapping, tag_name):
""" dump loaded/parsed tests data (except functions) to json file.
the dumped file is located in PWD/logs folder.
Args:
tests_mapping (dict): data to dump
tag_name (str): tag name, loaded/parsed
"""
project_mapping = tests_mapping.get("project_mapping", {})
pwd_dir_path, dump_file_name = _prepare_dump_info(project_mapping, tag_name)
tests_to_dump = {
"project_mapping": {}
}
for key in project_mapping:
if key != "functions":
tests_to_dump["project_mapping"][key] = project_mapping[key]
continue
# remove functions in order to dump
if project_mapping["functions"]:
debugtalk_py_path = os.path.join(pwd_dir_path, "debugtalk.py")
tests_to_dump["project_mapping"]["debugtalk.py"] = debugtalk_py_path
if "api" in tests_mapping:
tests_to_dump["api"] = tests_mapping["api"]
elif "testcases" in tests_mapping:
tests_to_dump["testcases"] = tests_mapping["testcases"]
elif "testsuites" in tests_mapping:
tests_to_dump["testsuites"] = tests_mapping["testsuites"]
dump_json_file(tests_to_dump, pwd_dir_path, dump_file_name)
def dump_summary(summary, project_mapping):
""" dump test result summary to json file.
"""
pwd_dir_path, dump_file_name = _prepare_dump_info(project_mapping, "summary")
dump_json_file(summary, pwd_dir_path, dump_file_name)
def get_python2_retire_msg():
retire_day = datetime(2020, 1, 1)
today = datetime.now()

56
httprunner/validator.py
View File

@@ -20,8 +20,8 @@ def is_testcase(data_structure):
"request": {} # optional
},
"teststeps": [
teststep1,
{ # teststep2
test_dict1,
{ # test_dict2
'name': 'test step desc2',
'variables': [], # optional
'extract': [], # optional
@@ -54,21 +54,50 @@ def is_testcases(data_structure):
Args:
data_structure (dict): testcase(s) should always be in the following data structure:
{
"project_mapping": {
"PWD": "XXXXX",
"functions": {},
"env": {}
},
"testcases": [
{ # testcase data structure
"config": {
"name": "desc1",
"path": "testcase1_path",
"variables": [], # optional
},
"teststeps": [
# test data structure
{
'name': 'test step desc1',
'variables': [], # optional
'extract': [], # optional
'validate': [],
'request': {}
},
test_dict_2 # another test dict
]
},
testcase_dict_2 # another testcase dict
]
}
testcase_dict
or
[
testcase_dict_1,
testcase_dict_2
]
Returns:
bool: True if data_structure is valid testcase(s), otherwise False.
"""
if not isinstance(data_structure, list):
return is_testcase(data_structure)
if not isinstance(data_structure, dict):
return False
for item in data_structure:
if "testcases" not in data_structure:
return False
testcases = data_structure["testcases"]
if not isinstance(testcases, list):
return False
for item in testcases:
if not is_testcase(item):
return False
@@ -105,10 +134,9 @@ def is_testcase_path(path):
###############################################################################
def is_function(tup):
""" Takes (name, object) tuple, returns True if it is a function.
def is_function(item):
""" Takes item object, returns True if it is a function.
"""
name, item = tup
return isinstance(item, types.FunctionType)

97
tests/api/basic.yml
View File

@@ -1,97 +0,0 @@
- api:
def: get_token($user_agent, $device_sn, $os_platform, $app_version)
request:
url: /api/get-token
method: POST
headers:
user_agent: $user_agent
device_sn: $device_sn
os_platform: $os_platform
app_version: $app_version
json:
sign: ${get_sign($user_agent, $device_sn, $os_platform, $app_version)}
validate:
- eq: ["status_code", 0]
- len_eq: ["content.token", 12]
- contains: [{"a": 1, "b": 2}, "a"]
- api:
def: create_user($uid, $user_name, $user_password, $token)
request:
url: /api/users/$uid
method: POST
headers:
token: $token
json:
name: $user_name
password: $user_password
validate:
- eq: ["status_code", 201]
- api:
def: get_user($uid, $token)
request:
url: /api/users/$uid
method: GET
headers:
token: $token
validate:
- eq: ["status_code", 200]
- api:
def: update_user($uid, $user_name, $user_password, $token)
request:
url: /api/users/$uid
method: PUT
headers:
token: $token
json:
name: $user_name
password: $user_password
validate:
- eq: ["status_code", 200]
- api:
def: delete_user($uid, $token)
request:
url: /api/users/$uid
method: DELETE
headers:
token: $token
validate:
- eq: ["status_code", 200]
- api:
def: get_users($token)
request:
url: /api/users
method: GET
headers:
token: $token
validate:
- eq: ["status_code", 200]
- api:
def: reset_all($token)
request:
url: /api/reset-all
method: GET
headers:
token: $token
validate:
- eq: ["status_code", 200]
- eq: ["content.success", true]
- api:
def: get_headers($n_secs)
request:
url: /headers
method: GET
setup_hooks:
- ${setup_hook_add_kwargs($request)}
- ${setup_hook_remove_kwargs($request)}
teardown_hooks:
- ${teardown_hook_sleep_N_secs($response, $n_secs)}
validate:
- eq: ["status_code", 200]
- contained_by: [content.headers.Host, $HTTPBIN_SERVER]

18
tests/api/create_user.yml Normal file
View File

@@ -0,0 +1,18 @@
name: create user
variables:
user_name: user0
user_password: "000000"
uid: 9000
token: XXX
request:
url: /api/users/$uid
method: POST
headers:
Content-Type: "application/json"
device_sn: $device_sn
token: $token
json:
name: $user_name
password: $user_password
validate:
- eq: ["status_code", 201]

12
tests/api/delete_user.yml Normal file
View File

@@ -0,0 +1,12 @@
variables:
uid: 9000
token: XXX
request:
url: /api/users/$uid
method: DELETE
headers:
Content-Type: "application/json"
device_sn: $device_sn
token: $token
validate:
- eq: ["status_code", 200]

16
tests/api/get_headers.yml Normal file
View File

@@ -0,0 +1,16 @@
variables:
n_secs: 1
request:
url: /headers
headers:
Content-Type: "application/json"
device_sn: $device_sn
method: GET
setup_hooks:
- ${setup_hook_add_kwargs($request)}
- ${setup_hook_remove_kwargs($request)}
teardown_hooks:
- ${teardown_hook_sleep_N_secs($response, $n_secs)}
validate:
- eq: ["status_code", 200]
- contained_by: [content.headers.Host, "${get_httpbin_server()}"]

22
tests/api/get_token.yml Normal file
View File

@@ -0,0 +1,22 @@
name: get token
variables:
user_agent: XXX
device_sn: API_XXX
os_platform: XXX
app_version: XXX
request:
url: /api/get-token
method: POST
headers:
user_agent: $user_agent
device_sn: $device_sn
os_platform: $os_platform
app_version: $app_version
Content-Type: "application/json"
device_sn: $device_sn
json:
sign: ${get_sign($user_agent, $device_sn, $os_platform, $app_version)}
validate:
- eq: ["status_code", 0]
- len_eq: ["content.token", 12]
- contains: [{"a": 1, "b": 2}, "a"]

12
tests/api/get_user.yml Normal file
View File

@@ -0,0 +1,12 @@
variables:
uid: 9000
token: XXX
request:
url: /api/users/$uid
method: GET
headers:
Content-Type: "application/json"
device_sn: $device_sn
token: $token
validate:
- eq: ["status_code", 200]

11
tests/api/get_users.yml Normal file
View File

@@ -0,0 +1,11 @@
variables:
token: XXX
request:
url: /api/users
method: GET
headers:
Content-Type: "application/json"
device_sn: $device_sn
token: $token
validate:
- eq: ["status_code", 200]

12
tests/api/reset_all.yml Normal file
View File

@@ -0,0 +1,12 @@
variables:
token: XXX
request:
url: /api/reset-all
method: GET
headers:
Content-Type: "application/json"
device_sn: $device_sn
token: $token
validate:
- eq: ["status_code", 200]
- eq: ["content.success", true]

17
tests/api/update_user.yml Normal file
View File

@@ -0,0 +1,17 @@
variables:
user_name: user0
user_password: "000000"
uid: 9000
token: XXX
request:
url: /api/users/$uid
method: PUT
headers:
Content-Type: "application/json"
device_sn: $device_sn
token: $token
json:
name: $user_name
password: $user_password
validate:
- eq: ["status_code", 200]

5
tests/api_server.py
View File

@@ -10,13 +10,14 @@ try:
from httpbin import app as httpbin_app
HTTPBIN_HOST = "127.0.0.1"
HTTPBIN_PORT = 3458
HTTPBIN_SERVER = "http://{}:{}".format(HTTPBIN_HOST, HTTPBIN_PORT)
except ImportError:
httpbin_app = None
HTTPBIN_HOST = "httpbin.org"
HTTPBIN_PORT = 80
HTTPBIN_PORT = 443
HTTPBIN_SERVER = "https://{}:{}".format(HTTPBIN_HOST, HTTPBIN_PORT)
FLASK_APP_PORT = 5000
HTTPBIN_SERVER = "http://{}:{}".format(HTTPBIN_HOST, HTTPBIN_PORT)
SECRET_KEY = "DebugTalk"
app = Flask(__name__)

10
tests/data/demo_binds.yml
View File

@@ -1,10 +0,0 @@
bind_variables:
variables:
- TOKEN: "debugtalk"
- token: $TOKEN
builtin_functions:
variables:
- length: ${len(debugtalk)}
- smallest: ${min(2, 3, 8)}
- largest: ${max(2, 3, 8)}

34
tests/data/demo_locust.yml
View File

@@ -1,34 +0,0 @@
- config:
name: basic test with httpbin
request:
base_url: https://httpbin.org/
- test:
name: index
weight: 5
request:
url: /
method: GET
validate:
- eq: ["status_code", 200]
- contains: [content, "HTTP Request &amp; Response Service"]
- test:
name: headers
weight: 3
request:
url: /headers
method: GET
validate:
- eq: ["status_code", 200]
- eq: [content.headers.Host, "httpbin.org"]
- test:
name: user-agent
weight: 2
request:
url: /user-agent
method: GET
validate:
- eq: ["status_code", 200]
- startswith: [content.user-agent, "python-requests"]

27
tests/data/demo_parameters.yml
View File

@@ -1,27 +0,0 @@
- config:
name: "user management testcase."
parameters:
- user_agent: ["iOS/10.1", "iOS/10.2", "iOS/10.3"]
- username-password:
- ["test1","111111"]
- ["test2","222222"]
variables:
- device_sn: ${gen_random_string(15)}
- os_platform: 'ios'
- app_version: 2.8.5
request:
base_url: $BASE_URL
headers:
Content-Type: application/json
device_sn: $device_sn
output:
- token
- test:
name: get token with $user_agent and $username
api: get_token($user_agent, $device_sn, $os_platform, $app_version)
extract:
- token: content.token
validate:
- "eq": ["status_code", 200]
- "len_eq": ["content.token", 16]

16
tests/data/demo_testcase.yml
View File

@@ -1,14 +1,14 @@
- config:
name: "123$var_a"
variables:
- var_a: 0
- var_c: "${sum_two(1, 2)}"
parameters:
- "var_a-var_b":
- [11, 21]
- [12, 22]
- "app_version": "${gen_app_version()}"
request: $demo_default_request
var_a: 0
var_c: "${sum_two(1, 2)}"
PROJECT_KEY: ${ENV(PROJECT_KEY)}
# parameters:
# - "var_a-var_b":
# - [11, 21]
# - [12, 22]
# - "app_version": "${gen_app_version()}"
- test:
name: testcase1-$var_a

4
tests/data/demo_testcase_cli.yml
View File

@@ -12,8 +12,8 @@
json:
sign: f1219719911caae89ccc301679857ebfda115ca2
variables:
- expect_status_code: 200
- token_len: 16
expect_status_code: 200
token_len: 16
extract:
- token: content.token
validate:

22
tests/data/demo_testcase_functions.yml
View File

@@ -1,15 +1,11 @@
- config:
name: "create user testcases."
variables:
- user_agent: 'iOS/10.3'
- device_sn: ${gen_random_string(15)}
- os_platform: 'ios'
- app_version: '2.8.6'
request:
base_url: $BASE_URL
headers:
Content-Type: application/json
device_sn: $device_sn
user_agent: 'iOS/10.3'
device_sn: ${gen_random_string(15)}
os_platform: 'ios'
app_version: '2.8.6'
base_url: ${get_base_url()}
- test:
name: get token
@@ -32,12 +28,14 @@
- test:
name: create user which does not exist
variables:
- user_name: "user1"
- user_password: "123456"
user_name: "user1"
user_password: "123456"
request:
url: /api/users/1000
method: POST
headers:
Content-Type: application/json
device_sn: $device_sn
token: $token
json:
name: $user_name
@@ -52,6 +50,8 @@
url: /api/users/1000
method: POST
headers:
Content-Type: application/json
device_sn: $device_sn
token: $token
json:
name: "user1"

8
tests/data/demo_testcase_hardcode.json
View File

@@ -20,11 +20,9 @@
{"expect_status_code": 200},
{"token_len": 16}
],
"extract": [
{
"token": "content.token"
}
],
"extract": {
"token": "content.token"
},
"validate": [
{"check": "status_code", "comparator": "eq", "expect": 200},
{"eq": ["status_code", "$expect_status_code"]},

8
tests/data/demo_testcase_hardcode.yml
View File

@@ -12,10 +12,10 @@
json:
sign: f1219719911caae89ccc301679857ebfda115ca2
variables:
- expect_status_code: 200
- token_len: 16
expect_status_code: 200
token_len: 16
extract:
- token: content.token
token: content.token
validate:
- {"check": "status_code", "comparator": "eq", "expect": 200}
- eq: ["status_code", $expect_status_code]
@@ -38,7 +38,7 @@
name: "user1"
password: "123456"
extract:
- success: content.success
success: content.success
validate:
- eq: ["status_code", 201]
- sum_status_code: ["status_code", 3]

84
tests/data/demo_testcase_layer.yml
View File

@@ -1,21 +1,17 @@
- config:
name: "user management testcase."
variables:
- user_agent: 'iOS/10.3'
- device_sn: ${gen_random_string(15)}
- os_platform: 'ios'
- app_version: '2.8.6'
request:
base_url: $BASE_URL
headers:
Content-Type: application/json
device_sn: $device_sn
user_agent: 'iOS/10.3'
device_sn: ${gen_random_string(15)}
os_platform: 'ios'
app_version: '2.8.6'
base_url: ${get_base_url()}
output:
- token
- test:
name: get token with $user_agent, $app_version
api: get_token($user_agent, $device_sn, $os_platform, $app_version)
api: api/get_token.yml
extract:
- token: content.token
validate:
@@ -25,14 +21,19 @@
- test:
name: reset all users
api: reset_all($token)
api: api/reset_all.yml
variables:
token: $token
validate:
- {"check": "status_code", "expect": 200}
- {"check": "content.success", "expect": true}
- test:
name: get user that does not exist
api: get_user(1000, $token)
api: api/get_user.yml
variables:
uid: 1000
token: $token
validate:
- {"check": "status_code", "expect": 404}
- {"check": "content.success", "expect": false}
@@ -40,16 +41,21 @@
- test:
name: create user which does not exist
variables:
- user_name: "user1"
- user_password: "123456"
api: create_user(1000, $user_name, $user_password, $token)
uid: 1000
user_name: "user1"
user_password: "123456"
token: $token
api: api/create_user.yml
validate:
- {"check": "status_code", "expect": 201}
- {"check": "content.success", "expect": true}
- test:
name: get user that has been created
api: get_user(1000, $token)
api: api/get_user.yml
variables:
uid: 1000
token: $token
validate:
- {"check": "status_code", "expect": 200}
- {"check": "content.success", "expect": true}
@@ -58,9 +64,11 @@
- test:
name: create user which exists
variables:
- user_name: "user1"
- user_password: "123456"
api: create_user(1000, $user_name, $user_password, $token)
uid: 1000
user_name: "user1"
user_password: "123456"
token: $token
api: api/create_user.yml
validate:
- {"check": "status_code", "expect": 500}
- {"check": "content.success", "expect": false}
@@ -68,16 +76,21 @@
- test:
name: update user which exists
variables:
- user_name: "user1"
- user_password: "654321"
api: update_user(1000, $user_name, $user_password, $token)
uid: 1000
user_name: "user1"
user_password: "654321"
token: $token
api: api/update_user.yml
validate:
- {"check": "status_code", "expect": 200}
- {"check": "content.success", "expect": true}
- test:
name: get user that has been updated
api: get_user(1000, $token)
api: api/get_user.yml
variables:
uid: 1000
token: $token
validate:
- {"check": "status_code", "expect": 200}
- {"check": "content.success", "expect": true}
@@ -85,21 +98,28 @@
- test:
name: get users
api: get_users($token)
api: api/get_users.yml
variables:
token: $token
validate:
- {"check": "status_code", "expect": 200}
- {"check": "content.count", "expect": 1}
- test:
name: delete user that exists
api: delete_user(1000, $token)
api: api/delete_user.yml
variables:
uid: 1000
token: $token
validate:
- {"check": "status_code", "expect": 200}
- {"check": "content.success", "expect": true}
- test:
name: get users
api: get_users($token)
api: api/get_users.yml
variables:
token: $token
validate:
- {"check": "status_code", "expect": 200}
- {"check": "content.count", "expect": 0}
@@ -107,16 +127,20 @@
- test:
name: create user which has been deleted
variables:
- user_name: "user1"
- user_password: "123456"
api: create_user(1000, $user_name, $user_password, $token)
uid: 1000
user_name: "user1"
user_password: "123456"
token: $token
api: api/create_user.yml
validate:
- {"check": "status_code", "expect": 201}
- {"check": "content.success", "expect": true}
- test:
name: get users
api: get_users($token)
api: api/get_users.yml
variables:
token: $token
validate:
- {"check": "status_code", "expect": 200}
- {"check": "content.count", "expect": 1}

24
tests/data/demo_testcase_variables.yml
View File

@@ -1,20 +1,16 @@
- config:
name: "create user testcases."
variables:
- device_sn: 'HZfFBh6tU59EdXJ'
request:
base_url: $BASE_URL
headers:
Content-Type: application/json
device_sn: $device_sn
device_sn: 'HZfFBh6tU59EdXJ'
base_url: ${get_base_url()}
- test:
name: get token
variables:
- user_agent: 'iOS/10.3'
- os_platform: 'ios'
- app_version: '2.8.6'
- sign: f1219719911caae89ccc301679857ebfda115ca2
user_agent: 'iOS/10.3'
os_platform: 'ios'
app_version: '2.8.6'
sign: f1219719911caae89ccc301679857ebfda115ca2
request:
url: /api/get-token
method: POST
@@ -35,12 +31,14 @@
- test:
name: create user which does not exist
variables:
- user_name: "user1"
- user_password: "123456"
user_name: "user1"
user_password: "123456"
request:
url: /api/users/1000
method: POST
headers:
Content-Type: application/json
device_sn: $device_sn
token: $token
json:
name: $user_name
@@ -55,6 +53,8 @@
url: /api/users/1000
method: POST
headers:
Content-Type: application/json
device_sn: $device_sn
token: $token
json:
name: "user1"

25
tests/debugtalk.py
View File

@@ -4,18 +4,23 @@ import random
import string
import time
from tests.api_server import HTTPBIN_SERVER, SECRET_KEY, gen_md5, get_sign
from tests.api_server import HTTPBIN_SERVER, gen_md5, get_sign
BASE_URL = "http://127.0.0.1:5000"
UserName = os.environ['UserName']
def get_httpbin_server():
return HTTPBIN_SERVER
demo_default_request = {
"base_url": "$BASE_URL",
"headers": {
"content-type": "application/json"
def get_base_url():
return BASE_URL
def get_default_request():
return {
"base_url": BASE_URL,
"headers": {
"content-type": "application/json"
}
}
}
def sum_two(m, n):
return m + n
@@ -40,6 +45,9 @@ def skip_test_in_production_env():
"""
return os.environ["TEST_ENV"] == "PRODUCTION"
def get_user_agent():
return ["iOS/10.1", "iOS/10.2"]
def gen_app_version():
return [
{"app_version": "2.8.5"},
@@ -52,6 +60,9 @@ def get_account():
{"username": "user2", "password": "222222"}
]
def get_account_in_tuple():
return [("user1", "111111"), ("user2", "222222")]
def gen_random_string(str_len):
random_char_list = []
for _ in range(str_len):

10
tests/httpbin/api/302_redirect.yml Normal file
View File

@@ -0,0 +1,10 @@
name: 302 redirect
request:
url: https://httpbin.org/redirect-to
params:
url: https://debugtalk.com
status_code: 302
method: GET
validate:
- eq: ["status_code", 200]

9
tests/httpbin/api/get_headers.yml Normal file
View File

@@ -0,0 +1,9 @@
name: headers
base_url: http://httpbin.org
request:
url: /headers
method: GET
validate:
- eq: ["status_code", 200]
- eq: [content.headers.Host, "httpbin.org"]

3
tests/httpbin/basic.yml
View File

@@ -1,7 +1,6 @@
- config:
name: basic test with httpbin
request:
base_url: https://httpbin.org/
base_url: https://httpbin.org/
- test:
name: index

5
tests/httpbin/hooks.yml
View File

@@ -1,7 +1,6 @@
- config:
name: basic test with httpbin
request:
base_url: $HTTPBIN_SERVER
base_url: ${get_httpbin_server()}
setup_hooks:
- ${hook_print(setup)}
teardown_hooks:
@@ -19,7 +18,7 @@
- ${teardown_hook_sleep_N_secs($response, 1)}
validate:
- eq: ["status_code", 200]
- contained_by: [content.headers.Host, $HTTPBIN_SERVER]
- contained_by: [content.headers.Host, "${get_httpbin_server()}"]
- test:
name: alter response

3
tests/httpbin/load_image.yml
View File

@@ -1,7 +1,6 @@
- config:
name: load images
request:
base_url: $HTTPBIN_SERVER
base_url: ${get_httpbin_server()}
- test:
name: get png image

13
tests/httpbin/upload.yml
View File

@@ -1,15 +1,14 @@
- config:
name: test upload file with httpbin
request:
base_url: $HTTPBIN_SERVER
base_url: ${get_httpbin_server()}
- test:
name: upload file
variable_binds:
- field_name: "file"
- file_path: "LICENSE"
- file_type: "text/html"
- multipart_encoder: ${multipart_encoder($field_name, $file_path, $file_type)}
variables:
field_name: "file"
file_path: "LICENSE"
file_type: "text/html"
multipart_encoder: ${multipart_encoder($field_name, $file_path, $file_type)}
request:
url: /post
method: POST

18
tests/locust_tests/demo_locusts.yml Normal file
View File

@@ -0,0 +1,18 @@
config:
name: create users with uid
variables:
- device_sn: ${gen_random_string(15)}
base_url: "http://127.0.0.1:5000"
testcases:
create user 1000 and check result.:
testcase: testcases/create_and_check.yml
weight: 2
variables:
uid: 1000
create user 1001 and check result.:
testcase: testcases/create_and_check.yml
weight: 3
variables:
uid: 1001

35
tests/suite/create_and_get.yml
View File

@@ -1,35 +0,0 @@
- config:
name: "create user and check result."
def: create_and_check($uid, $token)
request:
"base_url": "http://127.0.0.1:5000"
"headers":
"Content-Type": "application/json"
"device_sn": "$device_sn"
output:
- token
- test:
name: make sure user $uid does not exist
api: get_user($uid, $token)
validate:
- eq: ["status_code", 404]
- eq: ["content.success", false]
- test:
name: create user $uid
variables:
- user_name: "user1"
- user_password: "123456"
api: create_user($uid, $user_name, $user_password, $token)
validate:
- eq: ["status_code", 201]
- eq: ["content.success", true]
- test:
name: check if user $uid exists
api: get_user($uid, $token)
validate:
- eq: ["status_code", 200]
- eq: ["content.success", true]

33
tests/suite/setup.yml
View File

@@ -1,33 +0,0 @@
- config:
name: "setup and reset all."
def: setup_and_reset($device_sn)
variables:
- user_agent: 'iOS/10.3'
- device_sn: ${gen_random_string(15)}
- os_platform: 'ios'
- app_version: '2.8.6'
request:
"base_url": "http://127.0.0.1:5000"
"headers":
"Content-Type": "application/json"
"device_sn": "$device_sn"
output:
- token
- test:
name: get token
api: get_token($user_agent, $device_sn, $os_platform, $app_version)
variables:
- user_agent: 'iOS/10.3'
- device_sn: $device_sn
- os_platform: 'ios'
- app_version: '2.8.6'
extract:
- token: content.token
validate:
- eq: ["status_code", 200]
- len_eq: ["content.token", 16]
- test:
name: reset all users
api: reset_all($token)

500
tests/test_api.py
View File

@@ -1,9 +1,10 @@
import os
import shutil
import time
import unittest
from httprunner import HttpRunner, api, loader, parser
from locust import HttpLocust
from httprunner import loader, parser
from httprunner.api import HttpRunner, prepare_locust_tests
from tests.api_server import HTTPBIN_SERVER
from tests.base import ApiServerUnittest
@@ -18,7 +19,7 @@ class TestHttpRunner(ApiServerUnittest):
os.path.join(
os.getcwd(), 'tests/data/demo_testcase_hardcode.json')
]
self.testcases = [{
testcases = [{
'config': {
'name': 'testcase description',
'request': {
@@ -27,7 +28,7 @@ class TestHttpRunner(ApiServerUnittest):
},
'variables': []
},
'teststeps': [
"teststeps": [
{
'name': '/api/get-token',
'request': {
@@ -61,6 +62,10 @@ class TestHttpRunner(ApiServerUnittest):
}
]
}]
self.tests_mapping = {
"testcases": testcases
}
self.runner = HttpRunner(failfast=True)
self.reset_all()
def reset_all(self):
@@ -69,38 +74,47 @@ class TestHttpRunner(ApiServerUnittest):
return self.api_client.get(url, headers=headers)
def test_text_run_times(self):
runner = HttpRunner().run(self.testcase_cli_path)
self.assertEqual(runner.summary["stat"]["testsRun"], 10)
self.runner.run(self.testcase_cli_path)
self.assertEqual(self.runner.summary["stat"]["testcases"]["total"], 1)
self.assertEqual(self.runner.summary["stat"]["teststeps"]["total"], 10)
def test_text_skip(self):
runner = HttpRunner().run(self.testcase_cli_path)
self.assertEqual(runner.summary["stat"]["skipped"], 4)
self.runner.run(self.testcase_cli_path)
self.assertEqual(self.runner.summary["stat"]["teststeps"]["skipped"], 4)
def test_html_report(self):
runner = HttpRunner().run(self.testcase_cli_path)
report_save_dir = os.path.join(os.getcwd(), 'reports', "demo")
runner = HttpRunner(failfast=True, report_dir=report_save_dir)
runner.run(self.testcase_cli_path)
summary = runner.summary
self.assertEqual(summary["stat"]["testsRun"], 10)
self.assertEqual(summary["stat"]["skipped"], 4)
output_folder_name = "demo"
runner.gen_html_report(html_report_name=output_folder_name)
report_save_dir = os.path.join(os.getcwd(), 'reports', output_folder_name)
self.assertEqual(summary["stat"]["testcases"]["total"], 1)
self.assertEqual(summary["stat"]["teststeps"]["total"], 10)
self.assertEqual(summary["stat"]["teststeps"]["skipped"], 4)
self.assertGreater(len(os.listdir(report_save_dir)), 0)
shutil.rmtree(report_save_dir)
def test_log_file(self):
log_file_path = os.path.join(os.getcwd(), 'reports', "test_log_file.log")
runner = HttpRunner(failfast=True, log_file=log_file_path)
runner.run(self.testcase_cli_path)
self.assertTrue(os.path.isfile(log_file_path))
os.remove(log_file_path)
def test_run_testcases(self):
runner = HttpRunner().run(self.testcases)
summary = runner.summary
self.runner.run_tests(self.tests_mapping)
summary = self.runner.summary
self.assertTrue(summary["success"])
self.assertEqual(summary["stat"]["testsRun"], 2)
self.assertEqual(summary["stat"]["testcases"]["total"], 1)
self.assertEqual(summary["stat"]["teststeps"]["total"], 2)
self.assertIn("details", summary)
self.assertIn("records", summary["details"][0])
def test_run_yaml_upload(self):
runner = HttpRunner().run("tests/httpbin/upload.yml")
summary = runner.summary
self.runner.run("tests/httpbin/upload.yml")
summary = self.runner.summary
self.assertTrue(summary["success"])
self.assertEqual(summary["stat"]["testsRun"], 1)
self.assertEqual(summary["stat"]["testcases"]["total"], 1)
self.assertEqual(summary["stat"]["teststeps"]["total"], 1)
self.assertIn("details", summary)
self.assertIn("records", summary["details"][0])
@@ -133,44 +147,55 @@ class TestHttpRunner(ApiServerUnittest):
]
}
]
runner = HttpRunner().run(testcases)
summary = runner.summary
tests_mapping = {
"testcases": testcases
}
self.runner.run_tests(tests_mapping)
summary = self.runner.summary
self.assertTrue(summary["success"])
self.assertEqual(summary["stat"]["testsRun"], 1)
self.assertEqual(summary["details"][0]["records"][0]["meta_data"]["response"]["json"]["data"], "abc")
self.assertEqual(summary["stat"]["testcases"]["total"], 1)
self.assertEqual(summary["stat"]["teststeps"]["total"], 1)
self.assertEqual(
summary["details"][0]["records"][0]["meta_datas"]["data"][0]["response"]["json"]["data"],
"abc"
)
def test_html_report_repsonse_image(self):
runner = HttpRunner().run("tests/httpbin/load_image.yml")
summary = runner.summary
output_folder_name = "demo"
report = runner.gen_html_report(html_report_name=output_folder_name)
report_save_dir = os.path.join(os.getcwd(), 'reports', "demo")
runner = HttpRunner(failfast=True, report_dir=report_save_dir)
report = runner.run("tests/httpbin/load_image.yml")
self.assertTrue(os.path.isfile(report))
report_save_dir = os.path.join(os.getcwd(), 'reports', output_folder_name)
shutil.rmtree(report_save_dir)
def test_testcase_layer(self):
runner = HttpRunner(failfast=True).run("tests/testcases/smoketest.yml")
summary = runner.summary
def test_testcase_layer_with_api(self):
self.runner.run("tests/testcases/setup.yml")
summary = self.runner.summary
self.assertTrue(summary["success"])
self.assertEqual(summary["stat"]["testsRun"], 8)
self.assertEqual(summary["details"][0]["records"][0]["name"], "get token (setup)")
self.assertEqual(summary["stat"]["testcases"]["total"], 1)
self.assertEqual(summary["stat"]["teststeps"]["total"], 2)
def test_testcase_layer_with_testcase(self):
self.runner.run("tests/testsuites/create_users.yml")
summary = self.runner.summary
self.assertTrue(summary["success"])
self.assertEqual(summary["stat"]["testcases"]["total"], 2)
self.assertEqual(summary["stat"]["teststeps"]["total"], 8)
def test_run_httprunner_with_hooks(self):
testcase_file_path = os.path.join(
os.getcwd(), 'tests/httpbin/hooks.yml')
start_time = time.time()
runner = HttpRunner().run(testcase_file_path)
self.runner.run(testcase_file_path)
end_time = time.time()
summary = runner.summary
summary = self.runner.summary
self.assertTrue(summary["success"])
self.assertLess(end_time - start_time, 60)
def test_run_httprunner_with_teardown_hooks_alter_response(self):
testcases = [
{
"config": {
"name": "test teardown hooks",
"refs": loader.load_project_tests("tests")
},
"config": {"name": "test teardown hooks"},
"teststeps": [
{
"name": "test teardown hooks",
@@ -196,8 +221,13 @@ class TestHttpRunner(ApiServerUnittest):
]
}
]
runner = HttpRunner().run(testcases)
summary = runner.summary
loader.load_project_tests("tests")
tests_mapping = {
"project_mapping": loader.project_mapping,
"testcases": testcases
}
self.runner.run_tests(tests_mapping)
summary = self.runner.summary
self.assertTrue(summary["success"])
def test_run_httprunner_with_teardown_hooks_not_exist_attribute(self):
@@ -224,10 +254,15 @@ class TestHttpRunner(ApiServerUnittest):
]
}
]
runner = HttpRunner().run(testcases)
summary = runner.summary
loader.load_project_tests("tests")
tests_mapping = {
"project_mapping": loader.project_mapping,
"testcases": testcases
}
self.runner.run_tests(tests_mapping)
summary = self.runner.summary
self.assertFalse(summary["success"])
self.assertEqual(summary["stat"]["errors"], 1)
self.assertEqual(summary["stat"]["teststeps"]["errors"], 1)
def test_run_httprunner_with_teardown_hooks_error(self):
testcases = [
@@ -250,57 +285,108 @@ class TestHttpRunner(ApiServerUnittest):
]
}
]
runner = HttpRunner().run(testcases)
summary = runner.summary
loader.load_project_tests("tests")
tests_mapping = {
"project_mapping": loader.project_mapping,
"testcases": testcases
}
self.runner.run_tests(tests_mapping)
summary = self.runner.summary
self.assertFalse(summary["success"])
self.assertEqual(summary["stat"]["errors"], 1)
self.assertEqual(summary["stat"]["teststeps"]["errors"], 1)
def test_run_api(self):
path = "tests/httpbin/api/get_headers.yml"
self.runner.run(path)
summary = self.runner.summary
self.assertTrue(summary["success"])
self.assertEqual(summary["stat"]["testcases"]["total"], 1)
self.assertEqual(summary["stat"]["teststeps"]["total"], 1)
self.assertEqual(summary["stat"]["teststeps"]["successes"], 1)
def test_request_302_logs(self):
path = "tests/httpbin/api/302_redirect.yml"
self.runner.run(path)
summary = self.runner.summary
self.assertTrue(summary["success"])
self.assertEqual(summary["stat"]["testcases"]["total"], 1)
self.assertEqual(summary["stat"]["teststeps"]["total"], 1)
self.assertEqual(summary["stat"]["teststeps"]["successes"], 1)
req_resp_data = summary["details"][0]["records"][0]["meta_datas"]["data"]
self.assertEqual(len(req_resp_data), 2)
self.assertEqual(req_resp_data[0]["response"]["status_code"], 302)
self.assertEqual(req_resp_data[1]["response"]["status_code"], 200)
def test_request_with_params(self):
path = "tests/httpbin/api/302_redirect.yml"
self.runner.run(path)
summary = self.runner.summary
self.assertTrue(summary["success"])
self.assertEqual(summary["stat"]["testcases"]["total"], 1)
self.assertEqual(summary["stat"]["teststeps"]["total"], 1)
self.assertEqual(summary["stat"]["teststeps"]["successes"], 1)
req_resp_data = summary["details"][0]["records"][0]["meta_datas"]["data"]
self.assertEqual(len(req_resp_data), 2)
self.assertIn(
"url=https%3A%2F%2Fdebugtalk.com",
req_resp_data[0]["request"]["url"]
)
def test_run_api_folder(self):
api_folder = "tests/httpbin/api/"
self.runner.run(api_folder)
summary = self.runner.summary
self.assertTrue(summary["success"])
self.assertEqual(summary["stat"]["testcases"]["total"], 2)
self.assertEqual(summary["stat"]["teststeps"]["total"], 2)
self.assertEqual(summary["stat"]["teststeps"]["successes"], 2)
self.assertEqual(len(summary["details"]), 2)
self.assertEqual(summary["details"][0]["stat"]["total"], 1)
self.assertEqual(summary["details"][1]["stat"]["total"], 1)
def test_run_testcase_hardcode(self):
for testcase_file_path in self.testcase_file_path_list:
runner = HttpRunner().run(testcase_file_path)
summary = runner.summary
self.runner.run(testcase_file_path)
summary = self.runner.summary
self.assertTrue(summary["success"])
self.assertEqual(summary["stat"]["testsRun"], 3)
self.assertEqual(summary["stat"]["successes"], 3)
self.assertEqual(summary["stat"]["testcases"]["total"], 1)
self.assertEqual(summary["stat"]["teststeps"]["total"], 3)
self.assertEqual(summary["stat"]["teststeps"]["successes"], 3)
def test_run_testcases_hardcode(self):
runner = HttpRunner().run(self.testcase_file_path_list)
summary = runner.summary
self.assertTrue(summary["success"])
self.assertTrue(summary["success"])
self.assertEqual(summary["stat"]["testsRun"], 6)
self.assertEqual(summary["stat"]["successes"], 6)
def test_run_testcase_template_variables(self):
testcase_file_path = os.path.join(
os.getcwd(), 'tests/data/demo_testcase_variables.yml')
runner = HttpRunner().run(testcase_file_path)
summary = runner.summary
self.runner.run(testcase_file_path)
summary = self.runner.summary
self.assertTrue(summary["success"])
def test_run_testcase_template_import_functions(self):
testcase_file_path = os.path.join(
os.getcwd(), 'tests/data/demo_testcase_functions.yml')
runner = HttpRunner().run(testcase_file_path)
summary = runner.summary
self.runner.run(testcase_file_path)
summary = self.runner.summary
self.assertTrue(summary["success"])
def test_run_testcase_layered(self):
testcase_file_path = os.path.join(
os.getcwd(), 'tests/data/demo_testcase_layer.yml')
runner = HttpRunner().run(testcase_file_path)
summary = runner.summary
self.runner.run(testcase_file_path)
summary = self.runner.summary
self.assertTrue(summary["success"])
self.assertEqual(len(summary["details"]), 1)
def test_run_testcase_output(self):
testcase_file_path = os.path.join(
os.getcwd(), 'tests/data/demo_testcase_layer.yml')
runner = HttpRunner(failfast=True).run(testcase_file_path)
summary = runner.summary
self.runner.run(testcase_file_path)
summary = self.runner.summary
self.assertTrue(summary["success"])
self.assertIn("token", summary["details"][0]["in_out"]["out"])
self.assertIn("user_agent", summary["details"][0]["in_out"]["in"])
# TODO: add
# self.assertIn("user_agent", summary["details"][0]["in_out"]["in"])
def test_run_testcase_with_variables_mapping(self):
testcase_file_path = os.path.join(
@@ -308,70 +394,244 @@ class TestHttpRunner(ApiServerUnittest):
variables_mapping = {
"app_version": '2.9.7'
}
runner = HttpRunner(failfast=True).run(testcase_file_path, mapping=variables_mapping)
summary = runner.summary
self.runner.run(testcase_file_path, mapping=variables_mapping)
summary = self.runner.summary
self.assertTrue(summary["success"])
self.assertIn("token", summary["details"][0]["in_out"]["out"])
self.assertGreater(len(summary["details"][0]["in_out"]["in"]), 7)
# TODO: add
# self.assertGreater(len(summary["details"][0]["in_out"]["in"]), 3)
def test_run_testcase_with_parameters(self):
testcase_file_path = os.path.join(
os.getcwd(), 'tests/data/demo_parameters.yml')
runner = HttpRunner().run(testcase_file_path)
summary = runner.summary
self.assertEqual(
summary["details"][0]["in_out"]["in"]["user_agent"],
"iOS/10.1"
)
self.assertEqual(
summary["details"][2]["in_out"]["in"]["user_agent"],
"iOS/10.2"
)
self.assertEqual(
summary["details"][4]["in_out"]["in"]["user_agent"],
"iOS/10.3"
)
os.getcwd(), 'tests/testsuites/create_users_with_parameters.yml')
self.runner.run(testcase_file_path)
summary = self.runner.summary
self.assertTrue(summary["success"])
self.assertEqual(len(summary["details"]), 3 * 2)
self.assertEqual(summary["stat"]["testsRun"], 3 * 2)
self.assertIn("in", summary["details"][0]["in_out"])
self.assertIn("out", summary["details"][0]["in_out"])
def test_run_testcase_with_parameters_name(self):
testcase_file_path = os.path.join(
os.getcwd(), 'tests/data/demo_parameters.yml')
testcases = loader.load_tests(testcase_file_path)
parsed_testcases = parser.parse_tests(testcases)
self.assertEqual(summary["stat"]["testcases"]["total"], 6)
self.assertEqual(summary["stat"]["teststeps"]["total"], 3 * 2 * 4)
self.assertEqual(
summary["details"][0]["name"],
"create user 101 and check result for TESTSUITE_X1."
)
self.assertEqual(
summary["details"][5]["name"],
"create user 103 and check result for TESTSUITE_X2."
)
self.assertEqual(
summary["details"][0]["stat"]["total"],
4
)
records_name_list = [
summary["details"][i]["records"][2]["name"]
for i in range(6)
]
self.assertEqual(
set(records_name_list),
{
"create user 101 for TESTSUITE_X1",
"create user 101 for TESTSUITE_X2",
"create user 102 for TESTSUITE_X1",
"create user 102 for TESTSUITE_X2",
"create user 103 for TESTSUITE_X1",
"create user 103 for TESTSUITE_X2"
}
)
# def test_validate_response_content(self):
# # TODO: fix compatibility with Python 2.7
# testcase_file_path = os.path.join(
# os.getcwd(), 'tests/httpbin/basic.yml')
# self.runner.run(testcase_file_path)
# self.assertTrue(self.runner.summary["success"])
class TestApi(ApiServerUnittest):
def test_testcase_loader(self):
testcase_path = "tests/testcases/setup.yml"
tests_mapping = loader.load_tests(testcase_path)
project_mapping = tests_mapping["project_mapping"]
self.assertIsInstance(project_mapping, dict)
self.assertIn("PWD", project_mapping)
self.assertIn("functions", project_mapping)
self.assertIn("env", project_mapping)
testcases = tests_mapping["testcases"]
self.assertIsInstance(testcases, list)
self.assertEqual(len(testcases), 1)
testcase_config = testcases[0]["config"]
self.assertEqual(testcase_config["name"], "setup and reset all.")
self.assertIn("path", testcases[0])
testcase_tests = testcases[0]["teststeps"]
self.assertEqual(len(testcase_tests), 2)
self.assertIn("api", testcase_tests[0])
self.assertEqual(testcase_tests[0]["name"], "get token (setup)")
self.assertIsInstance(testcase_tests[0]["variables"], dict)
self.assertIn("api_def", testcase_tests[0])
self.assertEqual(testcase_tests[0]["api_def"]["request"]["url"], "/api/get-token")
def test_testcase_parser(self):
testcase_path = "tests/testcases/setup.yml"
tests_mapping = loader.load_tests(testcase_path)
parsed_tests_mapping = parser.parse_tests(tests_mapping)
parsed_testcases = parsed_tests_mapping["testcases"]
self.assertEqual(len(parsed_testcases), 1)
self.assertNotIn("variables", parsed_testcases[0]["config"])
self.assertEqual(len(parsed_testcases[0]["teststeps"]), 2)
test_dict1 = parsed_testcases[0]["teststeps"][0]
self.assertEqual(test_dict1["name"], "get token (setup)")
self.assertNotIn("api_def", test_dict1)
self.assertEqual(test_dict1["variables"]["device_sn"], "TESTCASE_SETUP_XXX")
self.assertEqual(test_dict1["request"]["url"], "http://127.0.0.1:5000/api/get-token")
def test_testcase_add_tests(self):
testcase_path = "tests/testcases/setup.yml"
tests_mapping = loader.load_tests(testcase_path)
parsed_tests_mapping = parser.parse_tests(tests_mapping)
runner = HttpRunner()
test_suite = runner._add_tests(parsed_testcases)
test_suite = runner._add_tests(parsed_tests_mapping)
self.assertEqual(len(test_suite._tests), 1)
teststeps = test_suite._tests[0].teststeps
self.assertEqual(teststeps[0]["name"], "get token (setup)")
self.assertEqual(teststeps[0]["variables"]["device_sn"], "TESTCASE_SETUP_XXX")
self.assertIn("api", teststeps[0])
def test_testcase_simple_run_suite(self):
testcase_path = "tests/testcases/setup.yml"
tests_mapping = loader.load_tests(testcase_path)
parsed_tests_mapping = parser.parse_tests(tests_mapping)
runner = HttpRunner()
test_suite = runner._add_tests(parsed_tests_mapping)
tests_results = runner._run_suite(test_suite)
self.assertEqual(len(tests_results[0][1].records), 2)
def test_testcase_complex_run_suite(self):
testcase_path = "tests/testcases/create_and_check.yml"
tests_mapping = loader.load_tests(testcase_path)
parsed_tests_mapping = parser.parse_tests(tests_mapping)
runner = HttpRunner()
test_suite = runner._add_tests(parsed_tests_mapping)
tests_results = runner._run_suite(test_suite)
self.assertEqual(len(tests_results[0][1].records), 4)
results = tests_results[0][1]
self.assertEqual(
test_suite._tests[0].teststeps[0]['name'],
'get token with iOS/10.1 and test1'
results.records[0]["name"],
"setup and reset all (override) for TESTCASE_CREATE_XXX."
)
self.assertEqual(
test_suite._tests[1].teststeps[0]['name'],
'get token with iOS/10.1 and test2'
)
self.assertEqual(
test_suite._tests[2].teststeps[0]['name'],
'get token with iOS/10.2 and test1'
)
self.assertEqual(
test_suite._tests[3].teststeps[0]['name'],
'get token with iOS/10.2 and test2'
)
self.assertEqual(
test_suite._tests[4].teststeps[0]['name'],
'get token with iOS/10.3 and test1'
)
self.assertEqual(
test_suite._tests[5].teststeps[0]['name'],
'get token with iOS/10.3 and test2'
results.records[1]["name"],
"make sure user 9001 does not exist"
)
def test_validate_response_content(self):
testcase_file_path = os.path.join(
os.getcwd(), 'tests/httpbin/basic.yml')
runner = HttpRunner().run(testcase_file_path)
self.assertTrue(runner.summary["success"])
def test_testsuite_loader(self):
testcase_path = "tests/testsuites/create_users.yml"
tests_mapping = loader.load_tests(testcase_path)
project_mapping = tests_mapping["project_mapping"]
self.assertIsInstance(project_mapping, dict)
self.assertIn("PWD", project_mapping)
self.assertIn("functions", project_mapping)
self.assertIn("env", project_mapping)
testsuites = tests_mapping["testsuites"]
self.assertIsInstance(testsuites, list)
self.assertEqual(len(testsuites), 1)
self.assertIn("path", testsuites[0])
testsuite_config = testsuites[0]["config"]
self.assertEqual(testsuite_config["name"], "create users with uid")
testcases = testsuites[0]["testcases"]
self.assertEqual(len(testcases), 2)
self.assertIn("create user 1000 and check result.", testcases)
testcase_tests = testcases["create user 1000 and check result."]
self.assertIn("testcase_def", testcase_tests)
self.assertEqual(testcase_tests["name"], "create user 1000 and check result.")
self.assertIsInstance(testcase_tests["testcase_def"], dict)
self.assertEqual(testcase_tests["testcase_def"]["config"]["name"], "create user and check result.")
self.assertEqual(len(testcase_tests["testcase_def"]["teststeps"]), 4)
self.assertEqual(
testcase_tests["testcase_def"]["teststeps"][0]["name"],
"setup and reset all (override) for $device_sn."
)
def test_testsuite_parser(self):
testcase_path = "tests/testsuites/create_users.yml"
tests_mapping = loader.load_tests(testcase_path)
parsed_tests_mapping = parser.parse_tests(tests_mapping)
parsed_testcases = parsed_tests_mapping["testcases"]
self.assertEqual(len(parsed_testcases), 2)
self.assertEqual(len(parsed_testcases[0]["teststeps"]), 4)
testcase1 = parsed_testcases[0]["teststeps"][0]
self.assertIn("setup and reset all (override)", testcase1["config"]["name"])
self.assertNotIn("testcase_def", testcase1)
self.assertEqual(len(testcase1["teststeps"]), 2)
self.assertEqual(
testcase1["teststeps"][0]["request"]["url"],
"http://127.0.0.1:5000/api/get-token"
)
self.assertEqual(len(testcase1["teststeps"][0]["variables"]["device_sn"]), 15)
def test_testsuite_add_tests(self):
testcase_path = "tests/testsuites/create_users.yml"
tests_mapping = loader.load_tests(testcase_path)
parsed_tests_mapping = parser.parse_tests(tests_mapping)
runner = HttpRunner()
test_suite = runner._add_tests(parsed_tests_mapping)
self.assertEqual(len(test_suite._tests), 2)
tests = test_suite._tests[0].teststeps
self.assertIn("setup and reset all (override)", tests[0]["config"]["name"])
def test_testsuite_run_suite(self):
testcase_path = "tests/testsuites/create_users.yml"
tests_mapping = loader.load_tests(testcase_path)
parsed_tests_mapping = parser.parse_tests(tests_mapping)
runner = HttpRunner()
test_suite = runner._add_tests(parsed_tests_mapping)
tests_results = runner._run_suite(test_suite)
self.assertEqual(len(tests_results[0][1].records), 4)
results = tests_results[0][1]
self.assertIn(
"setup and reset all (override)",
results.records[0]["name"]
)
self.assertIn(
results.records[1]["name"],
["make sure user 1000 does not exist", "make sure user 1001 does not exist"]
)
class TestLocust(unittest.TestCase):
def test_prepare_locust_tests(self):
path = os.path.join(
os.getcwd(), 'tests/locust_tests/demo_locusts.yml')
locust_tests = prepare_locust_tests(path)
self.assertIn("gen_md5", locust_tests["functions"])
self.assertEqual(len(locust_tests["tests"]), 2 + 3)
name_list = [
"create user 1000 and check result.",
"create user 1001 and check result."
]
self.assertIn(locust_tests["tests"][0]["config"]["name"], name_list)
self.assertIn(locust_tests["tests"][4]["config"]["name"], name_list)

72
tests/test_client.py
View File

@@ -1,6 +1,6 @@
from httprunner.built_in import setup_hook_prepare_kwargs
from httprunner.client import HttpSession
from httprunner.compat import bytes
from tests.api_server import HTTPBIN_SERVER
from tests.base import ApiServerUnittest
@@ -39,34 +39,48 @@ class TestHttpClient(ApiServerUnittest):
self.assertEqual(201, resp.status_code)
self.assertEqual(True, resp.json()['success'])
def test_prepare_kwargs_content_type_application_json_without_charset(self):
request = {
"url": "/path",
"method": "POST",
"headers": {
"content-type": "application/json"
},
"data": {
"a": 1,
"b": 2
}
def test_request_post_data(self):
url = "/api/users/1000"
data = {
'name': 'user1',
'password': '123456'
}
setup_hook_prepare_kwargs(request)
self.assertIsInstance(request["data"], bytes)
self.assertIn(b'"a": 1', request["data"])
self.assertIn(b'"b": 2', request["data"])
resp = self.api_client.post(url, json=data, headers=self.headers)
# b'{"name": "user1", "password": "123456"}'
self.assertIn(b'"name": "user1"', resp.request.body)
self.assertIn(b'"password": "123456"', resp.request.body)
resp = self.api_client.post(url, data=data, headers=self.headers)
# name=user1&password=123456
self.assertIn("name=user1", resp.request.body)
self.assertIn("&", resp.request.body)
self.assertIn("password=123456", resp.request.body)
def test_prepare_kwargs_content_type_application_json_charset_utf8(self):
request = {
"url": "/path",
"method": "POST",
"headers": {
"content-type": "application/json; charset=utf-8"
},
"data": {
"a": 1,
"b": 2
}
def test_request_with_cookies(self):
url = "/api/users/1000"
data = {
'name': 'user1',
'password': '123456'
}
setup_hook_prepare_kwargs(request)
self.assertIsInstance(request["data"], bytes)
cookies = {
"a": "1",
"b": "2"
}
resp = self.api_client.get(url, cookies=cookies, headers=self.headers)
self.assertEqual(resp.request._cookies["a"], "1")
self.assertEqual(resp.request._cookies["b"], "2")
def test_request_redirect(self):
url = "{}/redirect-to?url=https%3A%2F%2Fdebugtalk.com&status_code=302".format(HTTPBIN_SERVER)
headers = {"accept: text/html"}
cookies = {
"a": "1",
"b": "2"
}
resp = self.api_client.get(url, cookies=cookies, headers=self.headers)
raw_request = resp.history[0].request
self.assertEqual(raw_request._cookies["a"], "1")
self.assertEqual(raw_request._cookies["b"], "2")
redirect_request = resp.request
self.assertEqual(redirect_request.url, "https://debugtalk.com")
self.assertEqual(redirect_request._cookies["a"], "1")
self.assertEqual(redirect_request._cookies["b"], "2")

131
tests/test_context.py
View File

@@ -2,66 +2,51 @@ import os
import time
import requests
from httprunner import context, exceptions, loader, response
from httprunner import context, exceptions, loader, response, utils
from tests.base import ApiServerUnittest
class TestContext(ApiServerUnittest):
def setUp(self):
project_mapping = loader.load_project_tests(os.path.join(os.getcwd(), "tests"))
self.debugtalk_module = project_mapping["debugtalk"]
self.context = context.Context(
self.debugtalk_module["variables"],
self.debugtalk_module["functions"]
loader.load_project_tests(os.path.join(os.getcwd(), "tests"))
project_mapping = loader.project_mapping
self.context = context.SessionContext(
functions=project_mapping["functions"],
variables={"SECRET_KEY": "DebugTalk"}
)
testcase_file_path = os.path.join(os.getcwd(), 'tests/data/demo_binds.yml')
self.testcases = loader.load_file(testcase_file_path)
def test_init_context_functions(self):
context_functions = self.context.TESTCASE_SHARED_FUNCTIONS_MAPPING
context_functions = self.context.FUNCTIONS_MAPPING
self.assertIn("gen_md5", context_functions)
def test_init_context_variables(self):
def test_init_test_variables_initialize(self):
self.assertEqual(
self.context.teststep_variables_mapping["SECRET_KEY"],
"DebugTalk"
)
self.assertEqual(
self.context.testcase_runtime_variables_mapping["SECRET_KEY"],
"DebugTalk"
self.context.test_variables_mapping,
{'SECRET_KEY': 'DebugTalk'}
)
def test_update_context_testcase_level(self):
variables = [
{"TOKEN": "debugtalk"},
{"data": '{"name": "user", "password": "123456"}'}
]
self.context.update_context_variables(variables, "testcase")
self.assertEqual(
self.context.teststep_variables_mapping["TOKEN"],
"debugtalk"
)
self.assertEqual(
self.context.testcase_runtime_variables_mapping["TOKEN"],
"debugtalk"
)
def test_init_test_variables(self):
variables = {
"random": "${gen_random_string($num)}",
"authorization": "${gen_md5($TOKEN, $data, $random)}",
"data": '{"name": "$username", "password": "123456"}',
"TOKEN": "debugtalk",
"username": "user1",
"num": 6
}
self.context.init_test_variables(variables)
variables_mapping = self.context.test_variables_mapping
self.assertEqual(len(variables_mapping["random"]), 6)
self.assertEqual(len(variables_mapping["authorization"]), 32)
self.assertEqual(variables_mapping["data"], '{"name": "user1", "password": "123456"}')
def test_update_context_teststep_level(self):
variables = [
{"TOKEN": "debugtalk"},
{"data": '{"name": "user", "password": "123456"}'}
]
self.context.update_context_variables(variables, "teststep")
def test_update_seesion_variables(self):
self.context.update_session_variables({"TOKEN": "debugtalk"})
self.assertEqual(
self.context.teststep_variables_mapping["TOKEN"],
self.context.session_variables_mapping["TOKEN"],
"debugtalk"
)
self.assertNotIn(
"TOKEN",
self.context.testcase_runtime_variables_mapping
)
def test_eval_content_functions(self):
content = "${sleep_N_secs(1)}"
@@ -84,37 +69,15 @@ class TestContext(ApiServerUnittest):
# "abcDebugTalkdef"
# )
def test_update_testcase_runtime_variables_mapping(self):
variables = {"abc": 123}
self.context.update_testcase_runtime_variables_mapping(variables)
self.assertEqual(
self.context.testcase_runtime_variables_mapping["abc"],
123
)
self.assertEqual(
self.context.teststep_variables_mapping["abc"],
123
)
def test_update_teststep_variables_mapping(self):
self.context.update_teststep_variables_mapping("abc", 123)
self.assertEqual(
self.context.teststep_variables_mapping["abc"],
123
)
self.assertNotIn(
"abc",
self.context.testcase_runtime_variables_mapping
)
def test_get_parsed_request(self):
variables = [
{"TOKEN": "debugtalk"},
{"random": "${gen_random_string(5)}"},
{"data": '{"name": "user", "password": "123456"}'},
{"authorization": "${gen_md5($TOKEN, $data, $random)}"}
]
self.context.update_context_variables(variables, "teststep")
variables = {
"random": "${gen_random_string(5)}",
"data": '{"name": "user", "password": "123456"}',
"authorization": "${gen_md5($TOKEN, $data, $random)}",
"TOKEN": "debugtalk"
}
self.context.init_test_variables(variables)
request = {
"url": "http://127.0.0.1:5000/api/users/1000",
@@ -127,13 +90,16 @@ class TestContext(ApiServerUnittest):
},
"data": "$data"
}
parsed_request = self.context.get_parsed_request(request, level="teststep")
parsed_request = self.context.eval_content(request)
self.assertIn("authorization", parsed_request["headers"])
self.assertEqual(len(parsed_request["headers"]["authorization"]), 32)
self.assertIn("random", parsed_request["headers"])
self.assertEqual(len(parsed_request["headers"]["random"]), 5)
self.assertIn("data", parsed_request)
self.assertEqual(parsed_request["data"], variables[2]["data"])
self.assertEqual(
parsed_request["data"],
'{"name": "user", "password": "123456"}'
)
self.assertEqual(parsed_request["headers"]["secret_key"], "DebugTalk")
def test_do_validation(self):
@@ -157,11 +123,12 @@ class TestContext(ApiServerUnittest):
{"check": "$resp_status_code", "comparator": "eq", "expect": 201},
{"check": "$resp_body_success", "comparator": "eq", "expect": True}
]
variables = [
{"resp_status_code": 200},
{"resp_body_success": True}
]
self.context.update_context_variables(variables, "teststep")
variables = {
"resp_status_code": 200,
"resp_body_success": True
}
self.context.init_test_variables(variables)
with self.assertRaises(exceptions.ValidationFailure):
self.context.validate(validators, resp_obj)
@@ -176,7 +143,7 @@ class TestContext(ApiServerUnittest):
{"resp_status_code": 201},
{"resp_body_success": True}
]
self.context.update_context_variables(variables, "teststep")
self.context.init_test_variables(variables)
self.context.validate(validators, resp_obj)
def test_validate_exception(self):
@@ -190,7 +157,7 @@ class TestContext(ApiServerUnittest):
{"check": "$resp_status_code", "comparator": "eq", "expect": 201}
]
variables = []
self.context.update_context_variables(variables, "teststep")
self.context.init_test_variables(variables)
with self.assertRaises(exceptions.VariableNotFound):
self.context.validate(validators, resp_obj)
@@ -199,7 +166,7 @@ class TestContext(ApiServerUnittest):
variables = [
{"resp_status_code": 200}
]
self.context.update_context_variables(variables, "teststep")
self.context.init_test_variables(variables)
with self.assertRaises(exceptions.ValidationFailure):
self.context.validate(validators, resp_obj)

515
tests/test_loader.py
View File

@@ -27,7 +27,6 @@ class TestFileLoader(unittest.TestCase):
os.remove(yaml_tmp_file)
def test_load_json_file_file_format_error(self):
json_tmp_file = "tests/data/tmp.json"
# create empty file
@@ -113,19 +112,15 @@ class TestFileLoader(unittest.TestCase):
def test_load_folder_files(self):
folder = os.path.join(os.getcwd(), 'tests')
file1 = os.path.join(os.getcwd(), 'tests', 'test_utils.py')
file2 = os.path.join(os.getcwd(), 'tests', 'data', 'demo_binds.yml')
file2 = os.path.join(os.getcwd(), 'tests', 'api', 'reset_all.yml')
files = loader.load_folder_files(folder, recursive=False)
self.assertNotIn(file2, files)
self.assertEqual(files, [])
files = loader.load_folder_files(folder)
self.assertIn(file2, files)
self.assertNotIn(file1, files)
files = loader.load_folder_files(folder)
api_file = os.path.join(os.getcwd(), 'tests', 'api', 'basic.yml')
self.assertIn(api_file, files)
files = loader.load_folder_files("not_existed_foulder", recursive=False)
self.assertEqual([], files)
@@ -153,8 +148,8 @@ class TestFileLoader(unittest.TestCase):
dot_env_path = os.path.join(
os.getcwd(), "tests", "data",
)
with self.assertRaises(exceptions.FileNotFound):
loader.load_dot_env_file(dot_env_path)
env_variables_mapping = loader.load_dot_env_file(dot_env_path)
self.assertEqual(env_variables_mapping, {})
def test_locate_file(self):
with self.assertRaises(exceptions.FileNotFound):
@@ -172,310 +167,260 @@ class TestFileLoader(unittest.TestCase):
)
self.assertEqual(
loader.locate_file("tests/", "debugtalk.py"),
"tests/debugtalk.py"
os.path.join(os.getcwd(), "tests", "debugtalk.py")
)
self.assertEqual(
loader.locate_file("tests", "debugtalk.py"),
"tests/debugtalk.py"
os.path.join(os.getcwd(), "tests", "debugtalk.py")
)
self.assertEqual(
loader.locate_file("tests/base.py", "debugtalk.py"),
"tests/debugtalk.py"
os.path.join(os.getcwd(), "tests", "debugtalk.py")
)
self.assertEqual(
loader.locate_file("tests/data/demo_testcase.yml", "debugtalk.py"),
"tests/debugtalk.py"
os.path.join(os.getcwd(), "tests", "debugtalk.py")
)
def test_load_folder_content(self):
path = os.path.join(os.getcwd(), "tests", "api")
items_mapping = loader.load_folder_content(path)
file_path = os.path.join(os.getcwd(), "tests", "api", "reset_all.yml")
self.assertIn(file_path, items_mapping)
self.assertIsInstance(items_mapping[file_path], dict)
class TestModuleLoader(unittest.TestCase):
def test_filter_module_functions(self):
module_mapping = loader.load_python_module(loader)
functions_dict = module_mapping["functions"]
self.assertIn("load_python_module", functions_dict)
self.assertNotIn("is_py3", functions_dict)
module_functions = loader.load_module_functions(loader)
self.assertIn("load_module_functions", module_functions)
self.assertNotIn("is_py3", module_functions)
def test_load_debugtalk_module(self):
project_mapping = loader.load_project_tests(os.path.join(os.getcwd(), "httprunner"))
imported_module_items = project_mapping["debugtalk"]
self.assertNotIn("SECRET_KEY", imported_module_items["variables"])
self.assertNotIn("alter_response", imported_module_items["functions"])
loader.load_project_tests(os.path.join(os.getcwd(), "httprunner"))
project_mapping = loader.project_mapping
self.assertNotIn("alter_response", project_mapping["functions"])
project_mapping = loader.load_project_tests(os.path.join(os.getcwd(), "tests"))
imported_module_items = project_mapping["debugtalk"]
self.assertEqual(
imported_module_items["variables"]["SECRET_KEY"],
"DebugTalk"
)
self.assertIn("alter_response", imported_module_items["functions"])
loader.load_project_tests(os.path.join(os.getcwd(), "tests"))
project_mapping = loader.project_mapping
self.assertIn("alter_response", project_mapping["functions"])
is_status_code_200 = imported_module_items["functions"]["is_status_code_200"]
is_status_code_200 = project_mapping["functions"]["is_status_code_200"]
self.assertTrue(is_status_code_200(200))
self.assertFalse(is_status_code_200(500))
def test_get_module_item_functions(self):
from httprunner import utils
module_mapping = loader.load_python_module(utils)
get_uniform_comparator = loader.get_module_item(
module_mapping, "functions", "get_uniform_comparator")
self.assertTrue(validator.is_function(("get_uniform_comparator", get_uniform_comparator)))
self.assertEqual(get_uniform_comparator("=="), "equals")
with self.assertRaises(exceptions.FunctionNotFound):
loader.get_module_item(module_mapping, "functions", "gen_md4")
def test_get_module_item_variables(self):
dot_env_path = os.path.join(
os.getcwd(), "tests", ".env"
)
loader.load_dot_env_file(dot_env_path)
from tests import debugtalk
module_mapping = loader.load_python_module(debugtalk)
SECRET_KEY = loader.get_module_item(module_mapping, "variables", "SECRET_KEY")
self.assertTrue(validator.is_variable(("SECRET_KEY", SECRET_KEY)))
self.assertEqual(SECRET_KEY, "DebugTalk")
with self.assertRaises(exceptions.VariableNotFound):
loader.get_module_item(module_mapping, "variables", "SECRET_KEY2")
def test_locate_debugtalk_py(self):
debugtalk_path = loader.locate_debugtalk_py("tests/data/demo_testcase.yml")
def test_load_debugtalk_py(self):
loader.load_project_tests("tests/data/demo_testcase.yml")
project_working_directory = loader.project_mapping["PWD"]
debugtalk_functions = loader.project_mapping["functions"]
self.assertEqual(
debugtalk_path,
os.path.join(os.getcwd(), "tests", "debugtalk.py")
project_working_directory,
os.path.join(os.getcwd(), "tests")
)
self.assertIn("gen_md5", debugtalk_functions)
debugtalk_path = loader.locate_debugtalk_py("tests/base.py")
loader.load_project_tests("tests/base.py")
project_working_directory = loader.project_mapping["PWD"]
debugtalk_functions = loader.project_mapping["functions"]
self.assertEqual(
debugtalk_path,
os.path.join(os.getcwd(), "tests", "debugtalk.py")
project_working_directory,
os.path.join(os.getcwd(), "tests")
)
self.assertIn("gen_md5", debugtalk_functions)
debugtalk_path = loader.locate_debugtalk_py("httprunner/__init__.py")
loader.load_project_tests("httprunner/__init__.py")
project_working_directory = loader.project_mapping["PWD"]
debugtalk_functions = loader.project_mapping["functions"]
self.assertEqual(
debugtalk_path,
None
)
def test_load_tests(self):
testcase_file_path = os.path.join(
os.getcwd(), 'tests/data/demo_testcase.yml')
testcases = loader.load_tests(testcase_file_path)
self.assertIsInstance(testcases, list)
self.assertEqual(
testcases[0]["config"]["request"],
'$demo_default_request'
)
self.assertEqual(testcases[0]["config"]["name"], '123$var_a')
self.assertIn(
"sum_two",
testcases[0]["config"]["refs"]["debugtalk"]["functions"]
project_working_directory,
os.getcwd()
)
self.assertEqual(debugtalk_functions, {})
class TestSuiteLoader(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.project_mapping = loader.load_project_tests(os.path.join(os.getcwd(), "tests"))
loader.load_project_tests(os.path.join(os.getcwd(), "tests"))
cls.project_mapping = loader.project_mapping
cls.tests_def_mapping = loader.tests_def_mapping
def test_load_teststeps(self):
test_block = {
"name": "setup and reset all.",
"suite": "setup_and_reset($device_sn)",
"output": ["token", "device_sn"]
}
teststeps = loader._load_teststeps(test_block, self.project_mapping)
self.assertEqual(len(teststeps), 2)
self.assertEqual(teststeps[0]["name"], "get token")
self.assertEqual(teststeps[1]["name"], "reset all users")
def test_load_testcase(self):
raw_testcase = loader.load_file("tests/testcases/smoketest.yml")
testcase = loader._load_testcase(raw_testcase, self.project_mapping)
self.assertEqual(testcase["config"]["name"], "smoketest")
self.assertIn("device_sn", testcase["config"]["variables"][0])
self.assertEqual(len(testcase["teststeps"]), 8)
self.assertEqual(testcase["teststeps"][0]["name"], "get token")
def test_get_block_by_name(self):
ref_call = "get_user($uid, $token)"
block = loader._get_block_by_name(ref_call, "def-api", self.project_mapping)
self.assertEqual(block["request"]["url"], "/api/users/$uid")
self.assertEqual(block["function_meta"]["func_name"], "get_user")
self.assertEqual(block["function_meta"]["args"], ['$uid', '$token'])
def test_get_block_by_name_args_mismatch(self):
ref_call = "get_user($uid, $token, $var)"
with self.assertRaises(exceptions.ParamsError):
loader._get_block_by_name(ref_call, "def-api", self.project_mapping)
def test_override_block(self):
def_block = loader._get_block_by_name(
"get_token($user_agent, $device_sn, $os_platform, $app_version)",
"def-api",
self.project_mapping
)
test_block = {
"name": "override block",
def test_load_teststep_api(self):
raw_test = {
"name": "create user (override).",
"api": "api/create_user.yml",
"variables": [
{"var": 123}
],
'request': {
'url': '/api/get-token', 'method': 'POST', 'headers': {'user_agent': '$user_agent', 'device_sn': '$device_sn', 'os_platform': '$os_platform', 'app_version': '$app_version'}, 'json': {'sign': '${get_sign($user_agent, $device_sn, $os_platform, $app_version)}'}},
'validate': [
{'eq': ['status_code', 201]},
{'len_eq': ['content.token', 32]}
{"uid": "999"}
]
}
loader._extend_block(test_block, def_block)
self.assertEqual(test_block["name"], "override block")
self.assertIn({'check': 'status_code', 'expect': 201, 'comparator': 'eq'}, test_block["validate"])
self.assertIn({'check': 'content.token', 'comparator': 'len_eq', 'expect': 32}, test_block["validate"])
def test_get_test_definition_api(self):
api_def = loader._get_test_definition("get_headers", "def-api", self.project_mapping)
self.assertEqual(api_def["request"]["url"], "/headers")
self.assertEqual(len(api_def["setup_hooks"]), 2)
self.assertEqual(len(api_def["teardown_hooks"]), 1)
with self.assertRaises(exceptions.ApiNotFound):
loader._get_test_definition("get_token_XXX", "def-api", self.project_mapping)
def test_get_test_definition_suite(self):
api_def = loader._get_test_definition("create_and_check", "def-testcase", self.project_mapping)
self.assertEqual(api_def["config"]["name"], "create user and check result.")
with self.assertRaises(exceptions.TestcaseNotFound):
loader._get_test_definition("create_and_check_XXX", "def-testcase", self.project_mapping)
def test_merge_validator(self):
def_validators = [
{'eq': ['v1', 200]},
{"check": "s2", "expect": 16, "comparator": "len_eq"}
]
current_validators = [
{"check": "v1", "expect": 201},
{'len_eq': ['s3', 12]}
]
merged_validators = loader._merge_validator(def_validators, current_validators)
self.assertIn(
{"check": "v1", "expect": 201, "comparator": "eq"},
merged_validators
teststep = loader.load_teststep(raw_test)
self.assertEqual(
"create user (override).",
teststep["name"]
)
self.assertIn(
{"check": "s2", "expect": 16, "comparator": "len_eq"},
merged_validators
self.assertIn("api_def", teststep)
api_def = teststep["api_def"]
self.assertEqual(api_def["name"], "create user")
self.assertEqual(api_def["request"]["url"], "/api/users/$uid")
def test_load_teststep_testcase(self):
raw_test = {
"name": "setup and reset all (override).",
"testcase": "testcases/setup.yml",
"variables": [
{"device_sn": "$device_sn"}
],
"output": ["token", "device_sn"]
}
testcase = loader.load_teststep(raw_test)
self.assertEqual(
"setup and reset all (override).",
testcase["name"]
)
self.assertIn(
{"check": "s3", "expect": 12, "comparator": "len_eq"},
merged_validators
tests = testcase["testcase_def"]["teststeps"]
self.assertEqual(len(tests), 2)
self.assertEqual(tests[0]["name"], "get token (setup)")
self.assertEqual(tests[1]["name"], "reset all users")
def test_load_test_file_api(self):
loaded_content = loader.load_test_file("tests/api/create_user.yml")
self.assertEqual(loaded_content["type"], "api")
self.assertIn("path", loaded_content)
self.assertIn("request", loaded_content)
self.assertEqual(loaded_content["request"]["url"], "/api/users/$uid")
def test_load_test_file_testcase(self):
loaded_content = loader.load_test_file("tests/testcases/setup.yml")
self.assertEqual(loaded_content["type"], "testcase")
self.assertIn("path", loaded_content)
self.assertIn("config", loaded_content)
self.assertEqual(loaded_content["config"]["name"], "setup and reset all.")
self.assertIn("teststeps", loaded_content)
self.assertEqual(len(loaded_content["teststeps"]), 2)
def test_load_test_file_testsuite(self):
loaded_content = loader.load_test_file("tests/testsuites/create_users.yml")
self.assertEqual(loaded_content["type"], "testsuite")
testcases = loaded_content["testcases"]
self.assertEqual(len(testcases), 2)
self.assertIn('create user 1000 and check result.', testcases)
self.assertIn('testcase_def', testcases["create user 1000 and check result."])
self.assertEqual(
testcases["create user 1000 and check result."]["testcase_def"]["config"]["name"],
"create user and check result."
)
def test_merge_validator_with_dict(self):
def_validators = [
{'eq': ["a", {"v": 1}]},
{'eq': [{"b": 1}, 200]}
]
current_validators = [
{'len_eq': ['s3', 12]},
{'eq': [{"b": 1}, 201]}
]
merged_validators = loader._merge_validator(def_validators, current_validators)
self.assertEqual(len(merged_validators), 3)
self.assertIn({'check': {'b': 1}, 'expect': 201, 'comparator': 'eq'}, merged_validators)
self.assertNotIn({'check': {'b': 1}, 'expect': 200, 'comparator': 'eq'}, merged_validators)
def test_merge_extractor(self):
api_extrators = [{"var1": "val1"}, {"var2": "val2"}]
current_extractors = [{"var1": "val111"}, {"var3": "val3"}]
merged_extractors = loader._merge_extractor(api_extrators, current_extractors)
self.assertIn(
{"var1": "val111"},
merged_extractors
)
self.assertIn(
{"var2": "val2"},
merged_extractors
)
self.assertIn(
{"var3": "val3"},
merged_extractors
)
def test_load_testcases_by_path_files(self):
testcases_list = []
def test_load_tests_api_file(self):
path = os.path.join(
os.getcwd(), 'tests/api/create_user.yml')
tests_mapping = loader.load_tests(path)
project_mapping = tests_mapping["project_mapping"]
api_list = tests_mapping["apis"]
self.assertEqual(len(api_list), 1)
self.assertEqual(api_list[0]["request"]["url"], "/api/users/$uid")
def test_load_tests_testcase_file(self):
# absolute file path
path = os.path.join(
os.getcwd(), 'tests/data/demo_testcase_hardcode.json')
testcases_list = loader.load_tests(path)
tests_mapping = loader.load_tests(path)
project_mapping = tests_mapping["project_mapping"]
testcases_list = tests_mapping["testcases"]
self.assertEqual(len(testcases_list), 1)
self.assertEqual(len(testcases_list[0]["teststeps"]), 3)
self.assertEqual(
testcases_list[0]["config"]["refs"]["debugtalk"]["variables"]["SECRET_KEY"],
"DebugTalk"
)
self.assertIn("get_sign", testcases_list[0]["config"]["refs"]["debugtalk"]["functions"])
self.assertIn("get_sign", project_mapping["functions"])
# relative file path
path = 'tests/data/demo_testcase_hardcode.yml'
testcases_list = loader.load_tests(path)
tests_mapping = loader.load_tests(path)
project_mapping = tests_mapping["project_mapping"]
testcases_list = tests_mapping["testcases"]
self.assertEqual(len(testcases_list), 1)
self.assertEqual(len(testcases_list[0]["teststeps"]), 3)
self.assertEqual(
testcases_list[0]["config"]["refs"]["debugtalk"]["variables"]["SECRET_KEY"],
"DebugTalk"
self.assertIn("get_sign", project_mapping["functions"])
def test_load_tests_testcase_file_2(self):
testcase_file_path = os.path.join(
os.getcwd(), 'tests/data/demo_testcase.yml')
tests_mapping = loader.load_tests(testcase_file_path)
testcases = tests_mapping["testcases"]
self.assertIsInstance(testcases, list)
self.assertEqual(testcases[0]["config"]["name"], '123$var_a')
self.assertIn(
"sum_two",
tests_mapping["project_mapping"]["functions"]
)
self.assertEqual(
testcases[0]["config"]["variables"]["var_c"],
"${sum_two(1, 2)}"
)
self.assertEqual(
testcases[0]["config"]["variables"]["PROJECT_KEY"],
"${ENV(PROJECT_KEY)}"
)
self.assertIn("get_sign", testcases_list[0]["config"]["refs"]["debugtalk"]["functions"])
# list/set container with file(s)
path = [
os.path.join(os.getcwd(), 'tests/data/demo_testcase_hardcode.json'),
'tests/data/demo_testcase_hardcode.yml'
]
testcases_list = loader.load_tests(path)
self.assertEqual(len(testcases_list), 2)
self.assertEqual(len(testcases_list[0]["teststeps"]), 3)
self.assertEqual(len(testcases_list[1]["teststeps"]), 3)
testcases_list.extend(testcases_list)
self.assertEqual(len(testcases_list), 4)
def test_load_tests_testcase_file_with_api_ref(self):
path = os.path.join(
os.getcwd(), 'tests/data/demo_testcase_layer.yml')
tests_mapping = loader.load_tests(path)
project_mapping = tests_mapping["project_mapping"]
testcases_list = tests_mapping["testcases"]
self.assertIn('device_sn', testcases_list[0]["config"]["variables"])
self.assertIn("gen_md5", project_mapping["functions"])
self.assertIn("base_url", testcases_list[0]["config"])
test_dict0 = testcases_list[0]["teststeps"][0]
self.assertEqual(
"get token with $user_agent, $app_version",
test_dict0["name"]
)
self.assertIn("/api/get-token", test_dict0["api_def"]["request"]["url"])
self.assertIn(
{'eq': ['status_code', 200]},
test_dict0["validate"]
)
for testcase in testcases_list:
for teststep in testcase["teststeps"]:
self.assertIn('name', teststep)
self.assertIn('request', teststep)
self.assertIn('url', teststep['request'])
self.assertIn('method', teststep['request'])
def test_load_tests_testsuite_file_with_testcase_ref(self):
path = os.path.join(
os.getcwd(), 'tests/testsuites/create_users.yml')
tests_mapping = loader.load_tests(path)
project_mapping = tests_mapping["project_mapping"]
testsuites_list = tests_mapping["testsuites"]
def test_load_testcases_by_path_folder(self):
self.assertEqual(
"create users with uid",
testsuites_list[0]["config"]["name"]
)
self.assertEqual(
{'device_sn': '${gen_random_string(15)}'},
testsuites_list[0]["config"]["variables"]
)
self.assertIn(
"create user 1000 and check result.",
testsuites_list[0]["testcases"]
)
self.assertEqual(
testsuites_list[0]["testcases"]["create user 1000 and check result."]["testcase_def"]["config"]["name"],
"create user and check result."
)
def test_load_tests_folder_path(self):
# absolute folder path
path = os.path.join(os.getcwd(), 'tests/data')
testcase_list_1 = loader.load_tests(path)
tests_mapping = loader.load_tests(path)
testcase_list_1 = tests_mapping["testcases"]
self.assertGreater(len(testcase_list_1), 4)
# relative folder path
path = 'tests/data/'
testcase_list_2 = loader.load_tests(path)
tests_mapping = loader.load_tests(path)
testcase_list_2 = tests_mapping["testcases"]
self.assertEqual(len(testcase_list_1), len(testcase_list_2))
# list/set container with file(s)
path = [
os.path.join(os.getcwd(), 'tests/data'),
'tests/data/'
]
testcase_list_3 = loader.load_tests(path)
self.assertEqual(len(testcase_list_3), 2 * len(testcase_list_1))
def test_load_testcases_by_path_not_exist(self):
def test_load_tests_path_not_exist(self):
# absolute folder path
path = os.path.join(os.getcwd(), 'tests/data_not_exist')
with self.assertRaises(exceptions.FileNotFound):
@@ -486,79 +431,15 @@ class TestSuiteLoader(unittest.TestCase):
with self.assertRaises(exceptions.FileNotFound):
loader.load_tests(path)
# list/set container with file(s)
path = [
os.path.join(os.getcwd(), 'tests/data_not_exist'),
'tests/data_not_exist/'
]
with self.assertRaises(exceptions.FileNotFound):
loader.load_tests(path)
def test_load_testcases_by_path_layered(self):
path = os.path.join(
os.getcwd(), 'tests/data/demo_testcase_layer.yml')
testcases_list = loader.load_tests(path)
self.assertIn("variables", testcases_list[0]["config"])
self.assertIn("request", testcases_list[0]["config"])
self.assertIn("request", testcases_list[0]["teststeps"][0])
self.assertIn("url", testcases_list[0]["teststeps"][0]["request"])
self.assertIn("validate", testcases_list[0]["teststeps"][0])
def test_load_folder_content(self):
path = os.path.join(os.getcwd(), "tests", "api")
items_mapping = loader.load_folder_content(path)
file_path = os.path.join(os.getcwd(), "tests", "api", "basic.yml")
self.assertIn(file_path, items_mapping)
self.assertIsInstance(items_mapping[file_path], list)
def test_load_api_folder(self):
path = os.path.join(os.getcwd(), "tests", "api")
api_definition_mapping = loader.load_api_folder(path)
self.assertIn("get_token", api_definition_mapping)
self.assertIn("request", api_definition_mapping["get_token"])
self.assertIn("function_meta", api_definition_mapping["get_token"])
def test_load_testcases_folder(self):
path = os.path.join(os.getcwd(), "tests", "suite")
testcases_definition_mapping = loader.load_test_folder(path)
self.assertIn("setup_and_reset", testcases_definition_mapping)
self.assertIn("create_and_check", testcases_definition_mapping)
self.assertEqual(
testcases_definition_mapping["setup_and_reset"]["config"]["name"],
"setup and reset all."
)
self.assertEqual(
testcases_definition_mapping["setup_and_reset"]["function_meta"]["func_name"],
"setup_and_reset"
)
def test_load_testsuites_folder(self):
path = os.path.join(os.getcwd(), "tests", "testcases")
testsuites_definition_mapping = loader.load_test_folder(path)
testsute_path = os.path.join(os.getcwd(), "tests", "testcases", "smoketest.yml")
self.assertIn(
testsute_path,
testsuites_definition_mapping
)
self.assertEqual(
testsuites_definition_mapping[testsute_path]["config"]["name"],
"smoketest"
)
api_file_path = os.path.join(os.getcwd(), "tests", "api", "get_token.yml")
self.assertIn(api_file_path, api_definition_mapping)
self.assertIn("request", api_definition_mapping[api_file_path])
def test_load_project_tests(self):
project_mapping = loader.load_project_tests(os.path.join(os.getcwd(), "tests"))
self.assertEqual(project_mapping["debugtalk"]["variables"]["SECRET_KEY"], "DebugTalk")
self.assertIn("get_token", project_mapping["def-api"])
self.assertIn("setup_and_reset", project_mapping["def-testcase"])
self.assertEqual(project_mapping["env"]["PROJECT_KEY"], "ABCDEFGH")
def test_load_locust_tests(self):
path = os.path.join(
os.getcwd(), 'tests/data/demo_locust.yml')
locust_tests = loader.load_locust_tests(path)
self.assertEqual(locust_tests["config"]["refs"]["env"]["UserName"], "debugtalk")
self.assertEqual(len(locust_tests["tests"]), 10)
self.assertEqual(locust_tests["tests"][0][0]["name"], "index")
self.assertEqual(locust_tests["tests"][9][0]["name"], "user-agent")
loader.load_project_tests(os.path.join(os.getcwd(), "tests"))
api_file_path = os.path.join(os.getcwd(), "tests", "api", "get_token.yml")
self.assertIn(api_file_path, self.tests_def_mapping["api"])
self.assertEqual(self.project_mapping["env"]["PROJECT_KEY"], "ABCDEFGH")

408
tests/test_parser.py
View File

@@ -212,7 +212,7 @@ class TestParser(unittest.TestCase):
"/abc/def/abc"
)
self.assertEqual(
parser.parse_string_variables("${func($var_1, $var_2, xyz)}", variables_mapping),
parser.parse_string_variables("${func($var_1, $var_2, xyz)}", variables_mapping, {}),
"${func(abc, def, xyz)}"
)
self.assertEqual(
@@ -364,8 +364,7 @@ class TestParser(unittest.TestCase):
]
variables_mapping = {}
functions_mapping = {}
cartesian_product_parameters = parser.parse_parameters(
parameters, variables_mapping, functions_mapping)
cartesian_product_parameters = parser.parse_parameters(parameters)
self.assertEqual(
len(cartesian_product_parameters),
3 * 2
@@ -377,27 +376,34 @@ class TestParser(unittest.TestCase):
def test_parse_parameters_custom_function(self):
parameters = [
{"user_agent": "${get_user_agent()}"},
{"app_version": "${gen_app_version()}"},
{"username-password": "${get_account()}"}
{"username-password": "${get_account()}"},
{"username2-password2": "${get_account_in_tuple()}"}
]
testcase_path = os.path.join(
os.getcwd(),
"tests/data/demo_parameters.yml"
)
dot_env_path = os.path.join(
os.getcwd(), "tests", ".env"
)
loader.load_dot_env_file(dot_env_path)
from tests import debugtalk
debugtalk_module = loader.load_python_module(debugtalk)
cartesian_product_parameters = parser.parse_parameters(
parameters,
debugtalk_module["variables"],
debugtalk_module["functions"]
functions_mapping=loader.load_module_functions(debugtalk)
)
self.assertIn(
{
'user_agent': 'iOS/10.1',
'app_version': '2.8.5',
'username': 'user1',
'password': '111111',
'username2': 'user1',
'password2': '111111'
},
cartesian_product_parameters
)
self.assertEqual(
len(cartesian_product_parameters),
2 * 2
2 * 2 * 2 * 2
)
def test_parse_parameters_parameterize(self):
@@ -405,51 +411,383 @@ class TestParser(unittest.TestCase):
{"app_version": "${parameterize(tests/data/app_version.csv)}"},
{"username-password": "${parameterize(tests/data/account.csv)}"}
]
variables_mapping = {}
functions_mapping = {}
cartesian_product_parameters = parser.parse_parameters(
parameters, variables_mapping, functions_mapping)
cartesian_product_parameters = parser.parse_parameters(parameters)
self.assertEqual(
len(cartesian_product_parameters),
2 * 3
)
def test_parse_parameters_mix(self):
project_mapping = loader.load_project_tests(os.path.join(os.getcwd(), "tests"))
loader.load_project_tests(os.path.join(os.getcwd(), "tests"))
project_mapping = loader.project_mapping
parameters = [
{"user_agent": ["iOS/10.1", "iOS/10.2", "iOS/10.3"]},
{"app_version": "${gen_app_version()}"},
{"username-password": "${parameterize(tests/data/account.csv)}"}
]
variables_mapping = {}
functions_mapping = project_mapping["debugtalk"]["functions"]
testcase_path = os.path.join(
os.getcwd(),
"tests/data/demo_parameters.yml"
)
cartesian_product_parameters = parser.parse_parameters(
parameters, variables_mapping, functions_mapping)
parameters, functions_mapping=project_mapping["functions"])
self.assertEqual(
len(cartesian_product_parameters),
3 * 2 * 3
)
def test_parse_tests(self):
def test_parse_tests_testcase(self):
testcase_file_path = os.path.join(
os.getcwd(), 'tests/data/demo_testcase.yml')
testcases = loader.load_tests(testcase_file_path)
parsed_testcases = parser.parse_tests(testcases)
self.assertEqual(parsed_testcases[0]["config"]["variables"]["var_c"], 3)
self.assertEqual(len(parsed_testcases), 2 * 2)
tests_mapping = loader.load_tests(testcase_file_path)
testcases = tests_mapping["testcases"]
self.assertEqual(
parsed_testcases[0]["config"]["request"]["base_url"],
'$BASE_URL'
testcases[0]["config"]["variables"]["var_c"],
"${sum_two(1, 2)}"
)
self.assertEqual(
parsed_testcases[0]["config"]["variables"]["BASE_URL"],
'http://127.0.0.1:5000'
testcases[0]["config"]["variables"]["PROJECT_KEY"],
"${ENV(PROJECT_KEY)}"
)
parsed_tests_mapping = parser.parse_tests(tests_mapping)
parsed_testcases = parsed_tests_mapping["testcases"]
self.assertIsInstance(parsed_testcases, list)
self.assertEqual(parsed_testcases[0]["config"]["name"], '12311')
test_dict1 = parsed_testcases[0]["teststeps"][0]
self.assertEqual(test_dict1["variables"]["var_c"], 3)
self.assertEqual(test_dict1["variables"]["PROJECT_KEY"], "ABCDEFGH")
# TODO: parameters
# self.assertEqual(len(parsed_testcases), 2 * 2)
self.assertEqual(parsed_testcases[0]["config"]["name"], '1230')
def test_parse_tests_override_variables(self):
tests_mapping = {
'testcases': [
{
"config": {
'name': '',
'variables': [
{"password": "123456"},
{"creator": "user_test_001"}
]
},
"teststeps": [
{
'name': 'testcase1',
"variables": [
{"creator": "user_test_002"},
{"username": "$creator"}
],
'request': {'url': '/api1', 'method': 'GET'}
}
]
}
]
}
parsed_tests_mapping = parser.parse_tests(tests_mapping)
test_dict1_variables = parsed_tests_mapping["testcases"][0]["teststeps"][0]["variables"]
self.assertEqual(test_dict1_variables["creator"], "user_test_001")
self.assertEqual(test_dict1_variables["username"], "user_test_001")
def test_parse_tests_base_url_priority(self):
""" base_url & verify: priority test_dict > config
"""
tests_mapping = {
'testcases': [
{
"config": {
'name': '',
"base_url": "$host",
'variables': {
"host": "https://debugtalk.com"
},
"verify": False
},
"teststeps": [
{
'name': 'testcase1',
"base_url": "https://httprunner.org",
'request': {'url': '/api1', 'method': 'GET', "verify": True}
}
]
}
]
}
parsed_tests_mapping = parser.parse_tests(tests_mapping)
test_dict = parsed_tests_mapping["testcases"][0]["teststeps"][0]
self.assertEqual(test_dict["request"]["url"], "https://httprunner.org/api1")
self.assertEqual(test_dict["request"]["verify"], True)
def test_parse_tests_base_url_path_with_variable(self):
tests_mapping = {
'testcases': [
{
"config": {
'name': '',
"base_url": "$host1",
'variables': {
"host1": "https://debugtalk.com"
}
},
"teststeps": [
{
'name': 'testcase1',
"variables": {
"host2": "https://httprunner.org"
},
'request': {'url': '$host2/api1', 'method': 'GET'}
}
]
}
]
}
parsed_tests_mapping = parser.parse_tests(tests_mapping)
test_dict = parsed_tests_mapping["testcases"][0]["teststeps"][0]
self.assertEqual(test_dict["request"]["url"], "https://httprunner.org/api1")
def test_parse_tests_base_url_test_dict(self):
tests_mapping = {
'testcases': [
{
"config": {
'name': '',
"base_url": "$host1",
'variables': {
"host1": "https://debugtalk.com"
}
},
"teststeps": [
{
'name': 'testcase1',
"base_url": "$host2",
"variables": {
"host2": "https://httprunner.org"
},
'request': {'url': '/api1', 'method': 'GET'}
}
]
}
]
}
parsed_tests_mapping = parser.parse_tests(tests_mapping)
test_dict = parsed_tests_mapping["testcases"][0]["teststeps"][0]
self.assertEqual(test_dict["request"]["url"], "https://httprunner.org/api1")
def test_parse_data_with_variables(self):
variables = {
"host2": "https://httprunner.org",
"num3": "${sum_two($num2, 4)}",
"num2": "${sum_two($num1, 3)}",
"num1": "${sum_two(1, 2)}"
}
from tests.debugtalk import sum_two
functions = {
"sum_two": sum_two
}
parsed_testcase = parser.parse_data(variables, variables, functions)
self.assertEqual(parsed_testcase["num3"], 10)
self.assertEqual(parsed_testcase["num2"], 6)
self.assertEqual(parsed_testcase["num1"], 3)
def test_parse_data_with_variables_not_found(self):
variables = {
"host": "https://httprunner.org",
"num4": "${sum_two($num0, 5)}",
"num3": "${sum_two($num2, 4)}",
"num2": "${sum_two($num1, 3)}",
"num1": "${sum_two(1, 2)}"
}
from tests.debugtalk import sum_two
functions = {
"sum_two": sum_two
}
with self.assertRaises(exceptions.VariableNotFound):
parser.parse_data(variables, variables, functions)
parsed_testcase = parser.parse_data(
variables,
variables,
functions,
raise_if_variable_not_found=False
)
self.assertEqual(parsed_testcase["num3"], 10)
self.assertEqual(parsed_testcase["num2"], 6)
self.assertEqual(parsed_testcase["num1"], 3)
self.assertEqual(parsed_testcase["num4"], "${sum_two($num0, 5)}")
def test_parse_tests_variable_with_function(self):
from tests.debugtalk import sum_two
tests_mapping = {
"project_mapping": {
"functions": {
"sum_two": sum_two
}
},
'testcases': [
{
"config": {
'name': '',
"base_url": "$host1",
'variables': {
"host1": "https://debugtalk.com"
}
},
"teststeps": [
{
'name': 'testcase1',
"base_url": "$host2",
"variables": {
"host2": "https://httprunner.org",
"num3": "${sum_two($num2, 4)}",
"num2": "${sum_two($num1, 3)}",
"num1": "${sum_two(1, 2)}"
},
'request': {
'url': '/api1/?num1=$num1&num2=$num2&num3=$num3',
'method': 'GET'
}
}
]
}
]
}
parsed_tests_mapping = parser.parse_tests(tests_mapping)
test_dict = parsed_tests_mapping["testcases"][0]["teststeps"][0]
self.assertEqual(test_dict["variables"]["num3"], 10)
self.assertEqual(test_dict["variables"]["num2"], 6)
self.assertEqual(
test_dict["request"]["url"],
"https://httprunner.org/api1/?num1=3&num2=6&num3=10"
)
def test_parse_tests_variable_not_found(self):
from tests.debugtalk import sum_two
tests_mapping = {
"project_mapping": {
"functions": {
"sum_two": sum_two
}
},
'testcases': [
{
"config": {
'name': '',
"base_url": "$host1",
'variables': {
"host1": "https://debugtalk.com"
}
},
"teststeps": [
{
'name': 'testcase1',
"base_url": "$host2",
"variables": {
"host2": "https://httprunner.org",
"num4": "${sum_two($num0, 5)}",
"num3": "${sum_two($num2, 4)}",
"num2": "${sum_two($num1, 3)}",
"num1": "${sum_two(1, 2)}"
},
'request': {
'url': '/api1/?num1=$num1&num2=$num2&num3=$num3&num4=$num4',
'method': 'GET'
}
}
]
}
]
}
parsed_tests_mapping = parser.parse_tests(tests_mapping)
test_dict = parsed_tests_mapping["testcases"][0]["teststeps"][0]
self.assertEqual(test_dict["variables"]["num3"], 10)
self.assertEqual(test_dict["variables"]["num2"], 6)
self.assertEqual(test_dict["variables"]["num4"], "${sum_two($num0, 5)}")
self.assertEqual(
test_dict["request"]["url"],
"https://httprunner.org/api1/?num1=$num1&num2=$num2&num3=$num3&num4=$num4"
)
def test_parse_tests_base_url_teststep_empty(self):
""" base_url & verify: priority test_dict > config
"""
tests_mapping = {
'testcases': [
{
"config": {
'name': '',
"base_url": "$host",
'variables': {
"host": "https://debugtalk.com"
},
"verify": False
},
"teststeps": [
{
'name': 'testcase1',
"base_url": "",
'request': {'url': '/api1', 'method': 'GET', "verify": True}
}
]
}
]
}
parsed_tests_mapping = parser.parse_tests(tests_mapping)
test_dict = parsed_tests_mapping["testcases"][0]["teststeps"][0]
self.assertEqual(test_dict["request"]["url"], "https://debugtalk.com/api1")
self.assertEqual(test_dict["request"]["verify"], True)
def test_parse_environ(self):
os.environ["PROJECT_KEY"] = "ABCDEFGH"
content = {
"variables": [
{"PROJECT_KEY": "${ENV(PROJECT_KEY)}"}
]
}
result = parser.parse_data(content)
content = {
"variables": [
{"PROJECT_KEY": "${ENV(PROJECT_KEY, abc)}"}
]
}
with self.assertRaises(exceptions.ParamsError):
parser.parse_data(content)
content = {
"variables": [
{"PROJECT_KEY": "${ENV(abc=123)}"}
]
}
with self.assertRaises(exceptions.ParamsError):
parser.parse_data(content)
def test_extend_with_api(self):
loader.load_project_tests(os.path.join(os.getcwd(), "tests"))
raw_testinfo = {
"name": "get token",
"base_url": "https://debugtalk.com",
"api": "api/get_token.yml",
}
api_def_dict = loader.load_teststep(raw_testinfo)
test_block = {
"name": "override block",
"times": 3,
"variables": [
{"var": 123}
],
"base_url": "https://httprunner.org",
'request': {
'url': '/api/get-token',
'method': 'POST',
'headers': {'user_agent': '$user_agent', 'device_sn': '$device_sn', 'os_platform': '$os_platform', 'app_version': '$app_version'},
'json': {'sign': '${get_sign($user_agent, $device_sn, $os_platform, $app_version)}'}
},
'validate': [
{'eq': ['status_code', 201]},
{'len_eq': ['content.token', 32]}
]
}
extended_block = parser._extend_with_api(test_block, api_def_dict)
self.assertEqual(extended_block["base_url"], "https://debugtalk.com")
self.assertEqual(extended_block["name"], "override block")
self.assertEqual({'var': 123}, extended_block["variables"])
self.assertIn({'check': 'status_code', 'expect': 201, 'comparator': 'eq'}, extended_block["validate"])
self.assertIn({'check': 'content.token', 'comparator': 'len_eq', 'expect': 32}, extended_block["validate"])
self.assertEqual(extended_block["times"], 3)

3
tests/test_response.py
View File

@@ -8,8 +8,7 @@ from tests.base import ApiServerUnittest
class TestResponse(ApiServerUnittest):
def setUp(self):
module_mapping = loader.load_python_module(built_in)
self.functions_mapping = module_mapping["functions"]
self.functions_mapping = loader.load_module_functions(built_in)
def test_parse_response_object_json(self):
url = "http://127.0.0.1:5000/api/users"

79
tests/test_runner.py
View File

@@ -10,13 +10,16 @@ from tests.base import ApiServerUnittest
class TestRunner(ApiServerUnittest):
def setUp(self):
project_mapping = loader.load_project_tests(os.path.join(os.getcwd(), "tests"))
self.debugtalk_module = project_mapping["debugtalk"]
config_dict = {
"variables": self.debugtalk_module["variables"],
"functions": self.debugtalk_module["functions"]
loader.load_project_tests(os.path.join(os.getcwd(), "tests"))
project_mapping = loader.project_mapping
self.debugtalk_functions = project_mapping["functions"]
config = {
"name": "XXX",
"base_url": "http://127.0.0.1",
"verify": False
}
self.test_runner = runner.Runner(config_dict)
self.test_runner = runner.Runner(config, self.debugtalk_functions)
self.reset_all()
def reset_all(self):
@@ -35,11 +38,8 @@ class TestRunner(ApiServerUnittest):
for testcase_file_path in testcase_file_path_list:
testcases = loader.load_file(testcase_file_path)
config_dict = {
"variables": self.debugtalk_module["variables"],
"functions": self.debugtalk_module["functions"]
}
test_runner = runner.Runner(config_dict)
config_dict = {}
test_runner = runner.Runner(config_dict, self.debugtalk_functions)
test = testcases[0]["test"]
test_runner.run_test(test)
@@ -81,11 +81,7 @@ class TestRunner(ApiServerUnittest):
config_dict = {
"name": "basic test with httpbin",
"variables": self.debugtalk_module["variables"],
"functions": self.debugtalk_module["functions"],
"request": {
"base_url": HTTPBIN_SERVER
},
"base_url": HTTPBIN_SERVER,
"setup_hooks": [
"${sleep_N_secs(0.5)}"
"${hook_print(setup)}"
@@ -115,7 +111,7 @@ class TestRunner(ApiServerUnittest):
{"check": "status_code", "expect": 200}
]
}
test_runner = runner.Runner(config_dict)
test_runner = runner.Runner(config_dict, self.debugtalk_functions)
end_time = time.time()
# check if testcase setup hook executed
self.assertGreater(end_time - start_time, 0.5)
@@ -127,14 +123,39 @@ class TestRunner(ApiServerUnittest):
# testcase teardown hook has not been executed now
self.assertLess(end_time - start_time, 1)
def test_run_testcase_with_hooks_assignment(self):
config_dict = {
"name": "basic test with httpbin",
"base_url": HTTPBIN_SERVER
}
test = {
"name": "modify request headers",
"request": {
"url": "/anything",
"method": "POST",
"headers": {
"user_agent": "iOS/10.3",
"os_platform": "ios"
},
"data": "a=1&b=2"
},
"setup_hooks": [
{"total": "${sum_two(1, 5)}"}
],
"validate": [
{"check": "status_code", "expect": 200}
]
}
test_runner = runner.Runner(config_dict, self.debugtalk_functions)
test_runner.run_test(test)
test_variables_mapping = test_runner.session_context.test_variables_mapping
self.assertEqual(test_variables_mapping["total"], 6)
self.assertEqual(test_variables_mapping["request"]["data"], "a=1&b=2")
def test_run_testcase_with_hooks_modify_request(self):
config_dict = {
"name": "basic test with httpbin",
"variables": self.debugtalk_module["variables"],
"functions": self.debugtalk_module["functions"],
"request": {
"base_url": HTTPBIN_SERVER
}
"base_url": HTTPBIN_SERVER
}
test = {
"name": "modify request headers",
@@ -158,7 +179,7 @@ class TestRunner(ApiServerUnittest):
{"check": "content.headers.Os-Platform", "expect": "android"}
]
}
test_runner = runner.Runner(config_dict)
test_runner = runner.Runner(config_dict, self.debugtalk_functions)
test_runner.run_test(test)
def test_run_testcase_with_teardown_hooks_success(self):
@@ -183,9 +204,6 @@ class TestRunner(ApiServerUnittest):
],
"teardown_hooks": ["${teardown_hook_sleep_N_secs($response, 2)}"]
}
config_dict = {}
self.test_runner.init_test(config_dict, "testcase")
start_time = time.time()
self.test_runner.run_test(test)
end_time = time.time()
@@ -214,9 +232,6 @@ class TestRunner(ApiServerUnittest):
],
"teardown_hooks": ["${teardown_hook_sleep_N_secs($response, 2)}"]
}
config_dict = {}
self.test_runner.init_test(config_dict, "testcase")
start_time = time.time()
self.test_runner.run_test(test)
end_time = time.time()
@@ -226,8 +241,8 @@ class TestRunner(ApiServerUnittest):
def test_run_testcase_with_empty_header(self):
testcase_file_path = os.path.join(
os.getcwd(), 'tests/data/test_bugfix.yml')
testcases = loader.load_tests(testcase_file_path)
testcase = testcases[0]
tests_mapping = loader.load_tests(testcase_file_path)
testcase = tests_mapping["testcases"][0]
config_dict_headers = testcase["config"]["request"]["headers"]
test_dict_headers = testcase["teststeps"][0]["request"]["headers"]
headers = deep_update_dict(
@@ -240,8 +255,6 @@ class TestRunner(ApiServerUnittest):
testcase_file_path = os.path.join(
os.getcwd(), 'tests/data/test_bugfix.yml')
testcases = loader.load_file(testcase_file_path)
config_dict = {}
self.test_runner.init_test(config_dict, "testcase")
test = testcases[2]["test"]
self.test_runner.run_test(test)

124
tests/test_utils.py
View File

@@ -2,21 +2,12 @@ import io
import os
import shutil
from httprunner import exceptions, loader, utils
from httprunner.compat import OrderedDict
from httprunner import exceptions, loader, parser, utils
from tests.base import ApiServerUnittest
class TestUtils(ApiServerUnittest):
def test_remove_prefix(self):
full_url = "http://debugtalk.com/post/123"
prefix = "http://debugtalk.com"
self.assertEqual(
utils.remove_prefix(full_url, prefix),
"/post/123"
)
def test_set_os_environ(self):
self.assertNotIn("abc", os.environ)
variables_mapping = {
@@ -101,8 +92,7 @@ class TestUtils(ApiServerUnittest):
def current_validators(self):
from httprunner import built_in
module_mapping = loader.load_python_module(built_in)
functions_mapping = module_mapping["functions"]
functions_mapping = loader.load_module_functions(built_in)
functions_mapping["equals"](None, None)
functions_mapping["equals"](1, 1)
@@ -206,15 +196,84 @@ class TestUtils(ApiServerUnittest):
new_request_dict = utils.lower_dict_keys(request_dict)
self.assertEqual(None, request_dict)
def test_convert_to_order_dict(self):
def test_ensure_mapping_format(self):
map_list = [
{"a": 1},
{"b": 2}
]
ordered_dict = utils.convert_mappinglist_to_orderdict(map_list)
ordered_dict = utils.ensure_mapping_format(map_list)
self.assertIsInstance(ordered_dict, dict)
self.assertIn("a", ordered_dict)
def test_extend_validators(self):
def_validators = [
{'eq': ['v1', 200]},
{"check": "s2", "expect": 16, "comparator": "len_eq"}
]
current_validators = [
{"check": "v1", "expect": 201},
{'len_eq': ['s3', 12]}
]
def_validators = [
parser.parse_validator(validator)
for validator in def_validators
]
ref_validators = [
parser.parse_validator(validator)
for validator in current_validators
]
extended_validators = utils.extend_validators(def_validators, ref_validators)
self.assertIn(
{"check": "v1", "expect": 201, "comparator": "eq"},
extended_validators
)
self.assertIn(
{"check": "s2", "expect": 16, "comparator": "len_eq"},
extended_validators
)
self.assertIn(
{"check": "s3", "expect": 12, "comparator": "len_eq"},
extended_validators
)
def test_extend_validators_with_dict(self):
def_validators = [
{'eq': ["a", {"v": 1}]},
{'eq': [{"b": 1}, 200]}
]
current_validators = [
{'len_eq': ['s3', 12]},
{'eq': [{"b": 1}, 201]}
]
def_validators = [
parser.parse_validator(validator)
for validator in def_validators
]
ref_validators = [
parser.parse_validator(validator)
for validator in current_validators
]
extended_validators = utils.extend_validators(def_validators, ref_validators)
self.assertEqual(len(extended_validators), 3)
self.assertIn({'check': {'b': 1}, 'expect': 201, 'comparator': 'eq'}, extended_validators)
self.assertNotIn({'check': {'b': 1}, 'expect': 200, 'comparator': 'eq'}, extended_validators)
def test_extend_variables(self):
raw_variables = [{"var1": "val1"}, {"var2": "val2"}]
override_variables = [{"var1": "val111"}, {"var3": "val3"}]
extended_variables_mapping = utils.extend_variables(raw_variables, override_variables)
self.assertEqual(extended_variables_mapping["var1"], "val111")
self.assertEqual(extended_variables_mapping["var2"], "val2")
self.assertEqual(extended_variables_mapping["var3"], "val3")
def test_extend_variables_fix(self):
raw_variables = [{"var1": "val1"}, {"var2": "val2"}]
override_variables = {}
extended_variables_mapping = utils.extend_variables(raw_variables, override_variables)
self.assertEqual(extended_variables_mapping["var1"], "val1")
def test_deepcopy_dict(self):
data = {
'a': 1,
@@ -235,43 +294,6 @@ class TestUtils(ApiServerUnittest):
self.assertEqual(id(new_data["c"]), id(data["c"]))
# self.assertEqual(id(new_data["d"]), id(data["d"]))
def test_update_ordered_dict(self):
map_list = [
{"a": 1},
{"b": 2}
]
ordered_dict = utils.convert_mappinglist_to_orderdict(map_list)
override_mapping = {"a": 3, "c": 4}
new_dict = utils.update_ordered_dict(ordered_dict, override_mapping)
self.assertEqual(3, new_dict["a"])
self.assertEqual(4, new_dict["c"])
def test_override_variables_binds(self):
map_list = [
{"a": 1},
{"b": 2}
]
override_mapping = {"a": 3, "c": 4}
new_dict = utils.override_mapping_list(map_list, override_mapping)
self.assertEqual(3, new_dict["a"])
self.assertEqual(4, new_dict["c"])
map_list = OrderedDict(
{
"a": 1,
"b": 2
}
)
override_mapping = {"a": 3, "c": 4}
new_dict = utils.override_mapping_list(map_list, override_mapping)
self.assertEqual(3, new_dict["a"])
self.assertEqual(4, new_dict["c"])
map_list = "invalid"
override_mapping = {"a": 3, "c": 4}
with self.assertRaises(exceptions.ParamsError):
utils.override_mapping_list(map_list, override_mapping)
def test_create_scaffold(self):
project_name = "projectABC"
utils.create_scaffold(project_name)

37
tests/test_validator.py
View File

@@ -12,12 +12,34 @@ class TestValidator(unittest.TestCase):
self.assertFalse(validator.is_testcases(data_structure))
data_structure = {
"name": "desc1",
"config": {},
"api": {},
"testcases": ["testcase11", "testcase12"]
"project_mapping": {
"PWD": "XXXXX",
"functions": {},
"env": {}
},
"testcases": [
{ # testcase data structure
"config": {
"name": "desc1",
"path": "testcase1_path",
"variables": [], # optional
},
"teststeps": [
# test data structure
{
'name': 'test step desc1',
'variables': [], # optional
'extract': [], # optional
'validate': [],
'request': {}
},
# test_dict2 # another test dict
]
},
# testcase_dict_2 # another testcase dict
]
}
self.assertTrue(data_structure)
self.assertTrue(validator.is_testcases(data_structure))
data_structure = [
{
"name": "desc1",
@@ -50,6 +72,5 @@ class TestValidator(unittest.TestCase):
def test_is_function(self):
func = lambda x: x + 1
self.assertTrue(validator.is_function(("func", func)))
self.assertTrue(validator.is_function(("func", validator.is_testcase)))
self.assertTrue(validator.is_function(func))
self.assertTrue(validator.is_function(validator.is_testcase))

45
tests/testcases/create_and_check.yml Normal file
View File

@@ -0,0 +1,45 @@
- config:
name: "create user and check result."
id: create_and_check
variables:
uid: 9001
device_sn: "TESTCASE_CREATE_XXX"
- test:
name: setup and reset all (override) for $device_sn.
testcase: testcases/setup.yml
output:
- token
- test:
name: make sure user $uid does not exist
api: api/get_user.yml
variables:
uid: $uid
token: $token
validate:
- eq: ["status_code", 404]
- eq: ["content.success", false]
- test:
name: create user $uid for $device_sn
api: api/create_user.yml
variables:
user_name: "user1"
user_password: "123456"
uid: $uid
token: $token
validate:
- eq: ["status_code", 201]
- eq: ["content.success", true]
- test:
name: check if user $uid exists
api: api/get_user.yml
variables:
uid: $uid
token: $token
validate:
- eq: ["status_code", 200]
- eq: ["content.success", true]

32
tests/testcases/setup.yml Normal file
View File

@@ -0,0 +1,32 @@
- config:
name: "setup and reset all."
id: setup_and_reset
variables:
user_agent: 'iOS/10.3'
device_sn: "TESTCASE_SETUP_XXX"
os_platform: 'ios'
app_version: '2.8.6'
base_url: "http://127.0.0.1:5000"
verify: False
output:
- token
- test:
name: get token (setup)
api: api/get_token.yml
variables:
user_agent: 'iOS/10.3'
device_sn: $device_sn
os_platform: 'ios'
app_version: '2.8.6'
extract:
- token: content.token
validate:
- eq: ["status_code", 200]
- len_eq: ["content.token", 16]
- test:
name: reset all users
api: api/reset_all.yml
variables:
token: $token

24
tests/testcases/smoketest.yml
View File

@@ -1,24 +0,0 @@
- config:
name: smoketest
variables:
- device_sn: ${gen_random_string(15)}
request:
"base_url": "http://127.0.0.1:5000"
"headers":
"Content-Type": "application/json"
"device_sn": "$device_sn"
- test:
name: setup and reset all.
suite: setup_and_reset($device_sn)
output:
- token
- device_sn
- test:
name: create user 1000 and check result.
suite: create_and_check(1000, $token)
- test:
name: create user 1001 and check result.
suite: create_and_check(1001, $token)

16
tests/testsuites/create_users.yml Normal file
View File

@@ -0,0 +1,16 @@
config:
name: create users with uid
variables:
device_sn: ${gen_random_string(15)}
base_url: "http://127.0.0.1:5000"
testcases:
create user 1000 and check result.:
testcase: testcases/create_and_check.yml
variables:
uid: 1000
create user 1001 and check result.:
testcase: testcases/create_and_check.yml
variables:
uid: 1001

15
tests/testsuites/create_users_with_parameters.yml Normal file
View File

@@ -0,0 +1,15 @@
config:
name: create users with uid
variables:
device_sn: ${gen_random_string(15)}
base_url: "http://127.0.0.1:5000"
testcases:
create user $uid and check result for $device_sn.:
testcase: testcases/create_and_check.yml
variables:
uid: 1000
device_sn: TESTSUITE_XXX
parameters:
uid: [101, 102, 103]
device_sn: [TESTSUITE_X1, TESTSUITE_X2]