Merge pull request #2 from debugtalk/master

pull debugtalk
2026-05-12 11:29:48 +08:00 · 2017-10-25 22:41:46 -05:00
parent 52fa69cc5a f5dcdaa5c5
commit 4d28e2fc86
26 changed files with 149 additions and 223 deletions
--- a/.travis.yml
+++ b/.travis.yml
@@ -2,7 +2,6 @@ sudo: false
 language: python
 python:
  - 2.7
-  - 3.3
  - 3.4
  - 3.5
  - 3.6
--- a/README.md
+++ b/README.md
@@ -5,7 +5,7 @@

 ## Design Philosophy

-Take full reuse of Python's existing powerful libraries: [`Requests`][requests], [`unittest`][unittest] and [`Locust`][Locust]. And achieve the goal of API automation test, production environment monitoring, and API performance test, with a concise and  elegant manner.
+Take full reuse of Python's existing powerful libraries: [`Requests`][requests], [`unittest`][unittest] and [`Locust`][Locust]. And achieve the goal of API automation test, production environment monitoring, and API performance test, with a concise and elegant manner.

 ## Key Features

@@ -15,9 +15,8 @@ Take full reuse of Python's existing powerful libraries: [`Requests`][requests],
 - With `debugtalk.py` plugin, module functions can be auto-discovered in recursive upward directories.
 - Testcases can be run in diverse ways, with single testset, multiple testsets, or entire project folder.
 - Test report is concise and clear, with detailed log records. See [`PyUnitReport`][PyUnitReport].
- Perfect combination with [Jenkins][Jenkins], running continuous integration test and production environment monitoring. Send mail notification with [`jenkins-mail-py`][jenkins-mail-py].
 - With reuse of [`Locust`][Locust], you can run performance test without extra work.
- It is extensible to facilitate the implementation of web platform with [`Flask`][flask] framework.
+- CLI command supported, perfect combination with [Jenkins][Jenkins].

 [*`Background Introduction (中文版)`*](docs/background-CN.md) | [*`Feature Descriptions (中文版)`*](docs/feature-descriptions-CN.md)

@@ -39,7 +38,7 @@ To ensure the installation or upgrade is successful, you can execute command `at

 ```text
 $ ate -V
-ApiTestEngine version: 0.7.4
+ApiTestEngine version: 0.7.5
 ```

 Execute the command `ate -h` to view command help.
@@ -47,10 +46,10 @@ Execute the command `ate -h` to view command help.
 ```text
 $ ate -h
 usage: ate [-h] [-V] [--log-level LOG_LEVEL] [--report-name REPORT_NAME]
-           [--failfast]
+           [--failfast] [--startproject STARTPROJECT]
           [testset_paths [testset_paths ...]]

-Api Test Engine.
+ApiTestEngine.

 positional arguments:
  testset_paths         testset file path
@@ -63,65 +62,8 @@ optional arguments:
  --report-name REPORT_NAME
                        Specify report name, default is generated time.
  --failfast            Stop the test run on the first error or failure.
-```
-
-### use `jenkins-mail-py` plugin
-
-If you want to use `ApiTestEngine` with Jenkins, you may need to send mail notification, and [`jenkins-mail-py`][jenkins-mail-py] will be of great help.
-
-To install mail helper, run this command in your terminal:
-
-```text
-$ pip install -U git+https://github.com/debugtalk/jenkins-mail-py.git#egg=jenkins-mail-py
-$ ate -V
-jenkins-mail-py version: 0.2.5
-ApiTestEngine version: 0.7.4
-```
-
-With [`jenkins-mail-py`][jenkins-mail-py] installed, you can see more optional arguments.
-
-```text
-$ ate -h
-usage: ate [-h] [-V] [--log-level LOG_LEVEL] [--report-name REPORT_NAME]
-           [--failfast] [--mailgun-api-id MAILGUN_API_ID]
-           [--mailgun-api-key MAILGUN_API_KEY] [--email-sender EMAIL_SENDER]
-           [--email-recepients EMAIL_RECEPIENTS] [--mail-subject MAIL_SUBJECT]
-           [--mail-content MAIL_CONTENT] [--jenkins-job-name JENKINS_JOB_NAME]
-           [--jenkins-job-url JENKINS_JOB_URL]
-           [--jenkins-build-number JENKINS_BUILD_NUMBER]
-           [testset_paths [testset_paths ...]]
-
-Api Test Engine.
-
-positional arguments:
-  testset_paths         testset file path
-
-optional arguments:
-  -h, --help            show this help message and exit
-  -V, --version         show version
-  --log-level LOG_LEVEL
-                        Specify logging level, default is INFO.
-  --report-name REPORT_NAME
-                        Specify report name, default is generated time.
-  --failfast            Stop the test run on the first error or failure.
-  --mailgun-api-id MAILGUN_API_ID
-                        Specify mailgun api id.
-  --mailgun-api-key MAILGUN_API_KEY
-                        Specify mailgun api key.
-  --email-sender EMAIL_SENDER
-                        Specify email sender.
-  --email-recepients EMAIL_RECEPIENTS
-                        Specify email recepients.
-  --mail-subject MAIL_SUBJECT
-                        Specify email subject.
-  --mail-content MAIL_CONTENT
-                        Specify email content.
-  --jenkins-job-name JENKINS_JOB_NAME
-                        Specify jenkins job name.
-  --jenkins-job-url JENKINS_JOB_URL
-                        Specify jenkins job url.
-  --jenkins-build-number JENKINS_BUILD_NUMBER
-                        Specify jenkins build number.
+  --startproject STARTPROJECT
+                        Specify new project name.
 ```

 ## Write testcases
@@ -133,7 +75,7 @@ And here is testset example of typical scenario: get `token` at the beginning, a
 ```yaml
 - config:
    name: "create user testsets."
-    variable_binds:
+    variables:
        - user_agent: 'iOS/10.3'
        - device_sn: ${gen_random_string(15)}
        - os_platform: 'ios'
@@ -156,7 +98,7 @@ And here is testset example of typical scenario: get `token` at the beginning, a
            app_version: $app_version
        json:
            sign: ${get_sign($user_agent, $device_sn, $os_platform, $app_version)}
-    extract_binds:
+    extractors:
        - token: content.token
    validators:
        - {"check": "status_code", "comparator": "eq", "expected": 200}
@@ -207,8 +149,8 @@ When you do continuous integration test or production environment monitoring wit

 ```text
 $ ate filepath/testcase.yml --report-name ${BUILD_NUMBER} \
-    --mailgun-api-id samples.mailgun.org \
-    --mailgun-api-key key-3ax6xnjp29jd6fds4gc373sgvjxteol0 \
+    --mailgun-smtp-username "qa@debugtalk.com" \
+    --mailgun-smtp-password "12345678" \
    --email-sender excited@samples.mailgun.org \
    --email-recepients ${MAIL_RECEPIENTS} \
    --jenkins-job-name ${JOB_NAME} \
@@ -260,7 +202,7 @@ Enjoy!

 ## Supported Python Versions

-Python `2.7`, `3.3`, `3.4`, `3.5` and `3.6`.
+Python `2.7`, `3.4`, `3.5` and `3.6`.

 `ApiTestEngine` has been tested on `macOS`, `Linux` and `Windows` platforms.

@@ -292,5 +234,4 @@ $ python main-locust -h
 [flask]: http://flask.pocoo.org/
 [PyUnitReport]: https://github.com/debugtalk/PyUnitReport
 [Jenkins]: https://jenkins.io/index.html
-[jenkins-mail-py]: https://github.com/debugtalk/jenkins-mail-py.git
 [quickstart]: docs/quickstart.md
--- a/ate/init.py
+++ b/ate/init.py
@@ -1 +1 @@
-__version__ = '0.7.4'
+__version__ = '0.7.5'
--- a/ate/cli.py
+++ b/ate/cli.py
@@ -35,12 +35,6 @@ def main_ate():
        '--startproject',
        help="Specify new project name.")

-    try:
-        from jenkins_mail_py import MailgunHelper
-        mailer = MailgunHelper(parser)
-    except ImportError:
-        mailer = None
-
    args = parser.parse_args()

    if args.version:
@@ -63,7 +57,7 @@ def main_ate():
                        report name is ignored, use generated time instead.")

    results = {}
-    subject = "SUCCESS"
+    success = True

    for testset_path in set(args.testset_paths):

@@ -86,16 +80,12 @@ def main_ate():
        })

        if len(result.successes) != result.testsRun:
-            subject = "FAILED"
+            success = False

        for task in task_suite.tasks:
            task.print_output()

-    flag_code = 0 if subject == "SUCCESS" else 1
-    if mailer and mailer.config_ready:
-        mailer.send_mail(subject, results, flag_code)
-
-    return flag_code
+    return 0 if success is True else 1

 def main_locust():
    """ Performance test with locust: parse command line options and run commands.
--- a/ate/context.py
+++ b/ate/context.py
@@ -54,8 +54,9 @@ class Context(object):
            or config_dict.get('import_module_functions', [])
        self.import_module_items(module_items, level)

-        variable_binds = config_dict.get('variable_binds', OrderedDict())
-        self.bind_variables(variable_binds, level)
+        variables = config_dict.get('variables') \
+            or config_dict.get('variable_binds', OrderedDict())
+        self.bind_variables(variables, level)

    def import_requires(self, modules):
        """ import required modules dynamically
@@ -92,11 +93,11 @@ class Context(object):
            imported_variables_dict = utils.filter_module(imported_module, "variable")
            self.bind_variables(imported_variables_dict, level)

-    def bind_variables(self, variable_binds, level="testcase"):
+    def bind_variables(self, variables, level="testcase"):
        """ bind variables to testset context or current testcase context.
            variables in testset context can be used in all testcases of current test suite.

-        @param (list or OrderDict) variable_binds, variable can be value or custom function.
+        @param (list or OrderDict) variables, variable can be value or custom function.
            if value is function, it will be called and bind result to variable.
        e.g.
            OrderDict({
@@ -106,10 +107,10 @@ class Context(object):
                "md5": "${gen_md5($TOKEN, $json, $random)}"
            })
        """
-        if isinstance(variable_binds, list):
-            variable_binds = utils.convert_to_order_dict(variable_binds)
+        if isinstance(variables, list):
+            variables = utils.convert_to_order_dict(variables)

-        for variable_name, value in variable_binds.items():
+        for variable_name, value in variables.items():
            variable_evale_value = self.testcase_parser.parse_content_with_bindings(value)

            if level == "testset":
--- a/ate/response.py
+++ b/ate/response.py
@@ -56,9 +56,9 @@ class ResponseObject(object):
        except AttributeError:
            raise exception.ParseResponseError("failed to extract bind variable in response!")

-    def extract_response(self, extract_binds):
+    def extract_response(self, extractors):
        """ extract content from requests.Response
-        @param (list) extract_binds
+        @param (list) extractors
            [
                {"resp_status_code": "status_code"},
                {"resp_headers_content_type": "headers.content-type"},
@@ -68,11 +68,11 @@ class ResponseObject(object):
        @return (OrderDict) variable binds ordered dict
        """
        extracted_variables_mapping = OrderedDict()
-        extract_binds_order_dict = utils.convert_to_order_dict(extract_binds)
+        extract_binds_order_dict = utils.convert_to_order_dict(extractors)

        for key, field in extract_binds_order_dict.items():
            if not isinstance(field, utils.string_type):
-                raise exception.ParamsError("invalid extract_binds in testcase extract_binds!")
+                raise exception.ParamsError("invalid extractors in testcase!")

            extracted_variables_mapping[key] = self.extract_field(field)

--- a/ate/runner.py
+++ b/ate/runner.py
@@ -23,7 +23,7 @@ class Runner(object):
                "requires": [],         # optional
                "function_binds": {},   # optional
                "import_module_items": [],  # optional
-                "variable_binds": [],   # optional
+                "variables": [],   # optional
                "request": {
                    "base_url": "http://127.0.0.1:5000",
                    "headers": {
@@ -37,7 +37,7 @@ class Runner(object):
                "requires": [],         # optional
                "function_binds": {},   # optional
                "import_module_items": [],  # optional
-                "variable_binds": [],   # optional
+                "variables": [],   # optional
                "request": {
                    "url": "/api/get-token",
                    "method": "POST",
@@ -71,9 +71,9 @@ class Runner(object):
            {
                "name": "testcase description",
                "times": 3,
-                "requires": [],  # optional, override
-                "function_binds": {}, # optional, override
-                "variable_binds": [], # optional, override
+                "requires": [],         # optional, override
+                "function_binds": {},   # optional, override
+                "variables": [],        # optional, override
                "request": {
                    "url": "http://127.0.0.1:5000/api/users/1000",
                    "method": "POST",
@@ -84,7 +84,7 @@ class Runner(object):
                    },
                    "body": '{"name": "user", "password": "123456"}'
                },
-                "extract_binds": [], # optional
+                "extractors": [], # optional
                "validators": [],    # optional
                "setup": [],         # optional
                "teardown": []       # optional
@@ -100,7 +100,9 @@ class Runner(object):
            raise exception.ParamsError("URL or METHOD missed!")

        run_times = int(testcase.get("times", 1))
-        extract_binds = testcase.get("extract_binds", [])
+        extractors = testcase.get("extractors") \
+            or testcase.get("extractor") \
+            or testcase.get("extract_binds", [])
        validators = testcase.get("validators", [])
        setup_actions = testcase.get("setup", [])
        teardown_actions = testcase.get("teardown", [])
@@ -115,7 +117,7 @@ class Runner(object):
            resp = self.http_client_session.request(url=url, method=method, **parsed_request)
            resp_obj = response.ResponseObject(resp)

-            extracted_variables_mapping = resp_obj.extract_response(extract_binds)
+            extracted_variables_mapping = resp_obj.extract_response(extractors)
            self.context.bind_variables(extracted_variables_mapping, level="testset")

            resp_obj.validate(validators, self.context.get_testcase_variables_mapping())
@@ -134,22 +136,22 @@ class Runner(object):
                        "name": "testset description",
                        "requires": [],
                        "function_binds": {},
-                        "variable_binds": [],
+                        "variables": [],
                        "request": {}
                    },
                    "testcases": [
                        {
                            "name": "testcase description",
-                            "variable_binds": [], # optional, override
+                            "variables": [], # optional, override
                            "request": {},
-                            "extract_binds": {},  # optional
+                            "extractors": {},  # optional
                            "validators": {}      # optional
                        },
                        testcase12
                    ]
                }
            (dict) variables_mapping:
-                passed in variables mapping, it will override variable_binds in config block
+                passed in variables mapping, it will override variables in config block

        @return (dict) test result of testset
            {
@@ -160,9 +162,9 @@ class Runner(object):
        success = True
        config_dict = testset.get("config", {})

-        variable_binds = config_dict.get("variable_binds", [])
+        variables = config_dict.get("variables", [])
        variables_mapping = variables_mapping or {}
-        config_dict["variable_binds"] = utils.override_variables_binds(variable_binds, variables_mapping)
+        config_dict["variables"] = utils.override_variables_binds(variables, variables_mapping)

        self.init_config(config_dict, level="testset")
        testcases = testset.get("testcases", [])
@@ -187,7 +189,7 @@ class Runner(object):
                - absolute/relative folder path
                - list/set container with file(s) and/or folder(s)
            (dict) mapping:
-                passed in variables mapping, it will override variable_binds in config block
+                passed in variables mapping, it will override variables in config block
        """
        success = True
        mapping = mapping or {}
--- a/ate/testcase.py
+++ b/ate/testcase.py
@@ -329,14 +329,14 @@ def substitute_variables_with_mapping(content, mapping):

 class TestcaseParser(object):

-    def __init__(self, variables_binds={}, functions_binds={}, file_path=None):
-        self.bind_variables(variables_binds)
-        self.bind_functions(functions_binds)
+    def __init__(self, variables={}, functions={}, file_path=None):
+        self.bind_variables(variables)
+        self.bind_functions(functions)
        self.file_path = file_path

-    def bind_variables(self, variables_binds):
+    def bind_variables(self, variables):
        """ bind variables to current testcase parser
-        @param (dict) variables_binds, variables binds mapping
+        @param (dict) variables, variables binds mapping
            {
                "authorization": "a83de0ff8d2e896dbd8efb81ba14e17d",
                "random": "A2dEx",
@@ -344,24 +344,24 @@ class TestcaseParser(object):
                "uuid": 1000
            }
        """
-        self.variables_binds = variables_binds
+        self.variables = variables

-    def bind_functions(self, functions_binds):
+    def bind_functions(self, functions):
        """ bind functions to current testcase parser
-        @param (dict) functions_binds, functions binds mapping
+        @param (dict) functions, functions binds mapping
            {
                "add_two_nums": lambda a, b=1: a + b
            }
        """
-        self.functions_binds = functions_binds
+        self.functions = functions

    def get_bind_item(self, item_type, item_name):
        if item_type == "function":
-            if item_name in self.functions_binds:
-                return self.functions_binds[item_name]
+            if item_name in self.functions:
+                return self.functions[item_name]
        elif item_type == "variable":
-            if item_name in self.variables_binds:
-                return self.variables_binds[item_name]
+            if item_name in self.variables:
+                return self.variables[item_name]
        else:
            raise exception.ParamsError("bind item should only be function or variable.")

--- a/ate/utils.py
+++ b/ate/utils.py
@@ -369,18 +369,18 @@ def update_ordered_dict(ordered_dict, override_mapping):

    return ordered_dict

-def override_variables_binds(variable_binds, new_mapping):
-    """ convert variable_binds in testcase to ordered mapping, with new_mapping overrided
+def override_variables_binds(variables, new_mapping):
+    """ convert variables in testcase to ordered mapping, with new_mapping overrided
    """
-    if isinstance(variable_binds, list):
-        variable_binds_ordered_dict = convert_to_order_dict(variable_binds)
-    elif isinstance(variable_binds, OrderedDict):
-        variable_binds_ordered_dict = variable_binds
+    if isinstance(variables, list):
+        variables_ordered_dict = convert_to_order_dict(variables)
+    elif isinstance(variables, OrderedDict):
+        variables_ordered_dict = variables
    else:
-        raise exception.ParamsError("variable_binds error!")
+        raise exception.ParamsError("variables error!")

    return update_ordered_dict(
-        variable_binds_ordered_dict,
+        variables_ordered_dict,
        new_mapping
    )

--- a/docs/extraction-and-validation.md
+++ b/docs/extraction-and-validation.md
@@ -24,7 +24,7 @@ Suppose we get the following HTTP response.
 }
 ```

-In `extract_binds` and `validators`, we can do chain operation to extract data field in HTTP response.
+In `extractors` and `validators`, we can do chain operation to extract data field in HTTP response.

 For instance, if we want to get `Content-Type` in response headers, then we can specify `headers.content-type`; if we want to get `first_name` in response content, we can specify `content.person.name.first_name`.

@@ -46,7 +46,7 @@ content.person.cities.1
 ```

 ```yaml
-extract_binds:
+extractors:
    - content_type: headers.content-type
    - first_name: content.person.name.first_name
 validators:
--- a/docs/quickstart.md
+++ b/docs/quickstart.md
@@ -120,7 +120,7 @@ To fix this problem, we should correlate `token` field in the second API test ca
            app_version: 2.8.6
        json:
            sign: 19067cf712265eb5426db8d3664026c1ccea02b9
-    extract_binds:
+    extractors:
        - token: content.token
    validators:
        - {"check": "status_code", "comparator": "eq", "expected": 200}
@@ -142,7 +142,7 @@ To fix this problem, we should correlate `token` field in the second API test ca
        - {"check": "content.success", "comparator": "eq", "expected": true}
 ```

-As you see, the `token` field is no longer hardcoded, instead it is extracted from the first API request with `extract_binds` mechanism. In the meanwhile, it is assigned to `token` variable, which can be referenced by the subsequent API requests.
+As you see, the `token` field is no longer hardcoded, instead it is extracted from the first API request with `extractors` mechanism. In the meanwhile, it is assigned to `token` variable, which can be referenced by the subsequent API requests.

 Now we save the test cases to [`quickstart-demo-rev-1.yml`][quickstart-demo-rev-1] and rerun it, and we will find that both API requests to be successful.

@@ -154,7 +154,7 @@ In actual scenarios, each user's `device_sn` is different, so we should paramete

 However, the test cases are only `YAML` documents, it is impossible to generate parameters dynamically in such text. Fortunately, we can combine `Python` scripts with `YAML/JSON` test cases in `ApiTestEngine`.

-To achieve this goal, we can utilize `debugtalk.py` plugin and `variable_binds` mechanisms.
+To achieve this goal, we can utilize `debugtalk.py` plugin and `variables` mechanisms.

 To be specific, we can create a Python file (`examples/debugtalk.py`) and implement the related algorithm in it. The `debugtalk.py` file can not only be located beside `YAML/JSON` testset file, but also can be in any upward recursive folder. Since we want `debugtalk.py` to be importable, we should put a `__init__.py` in its folder to make it as a Python module.

@@ -187,7 +187,7 @@ And then, we can revise our demo test case and reference the functions. Suppose
 ```yaml
 - test:
    name: get token
-    variable_binds:
+    variables:
        - user_agent: 'iOS/10.3'
        - device_sn: ${gen_random_string(15)}
        - os_platform: 'ios'
@@ -202,7 +202,7 @@ And then, we can revise our demo test case and reference the functions. Suppose
            app_version: $app_version
        json:
            sign: ${get_sign($user_agent, $device_sn, $os_platform, $app_version)}
-    extract_binds:
+    extractors:
        - token: content.token
    validators:
        - {"check": "status_code", "comparator": "eq", "expected": 200}
@@ -226,7 +226,7 @@ And then, we can revise our demo test case and reference the functions. Suppose

 In this revised test case, `variable reference` and `function invoke` mechanisms are both used.

-To make fields like `device_sn` can be used more than once, we bind values to variables in `variable_binds` block. When we bind variables, we can not only bind exact value to a variable name, but also can call a function and bind the evaluated value to it.
+To make fields like `device_sn` can be used more than once, we bind values to variables in `variables` block. When we bind variables, we can not only bind exact value to a variable name, but also can call a function and bind the evaluated value to it.

 When we want to reference a variable in the test case, we can do this with a escape character `$`. For example, `$user_agent` will not be taken as a normal string, and `ApiTestEngine` will consider it as a variable named `user_agent`, search and return its binding value.

@@ -244,7 +244,7 @@ To handle this case, overall `config` block is supported in `ApiTestEngine`. If
 # examples/quickstart-demo-rev-3.yml
 - config:
    name: "smoketest for CRUD users."
-    variable_binds:
+    variables:
        - device_sn: ${gen_random_string(15)}
    request:
        base_url: http://127.0.0.1:5000
@@ -253,7 +253,7 @@ To handle this case, overall `config` block is supported in `ApiTestEngine`. If

 - test:
    name: get token
-    variable_binds:
+    variables:
        - user_agent: 'iOS/10.3'
        - os_platform: 'ios'
        - app_version: '2.8.6'
@@ -266,7 +266,7 @@ To handle this case, overall `config` block is supported in `ApiTestEngine`. If
            app_version: $app_version
        json:
            sign: ${get_sign($user_agent, $device_sn, $os_platform, $app_version)}
-    extract_binds:
+    extractors:
        - token: content.token
    validators:
        - {"check": "status_code", "comparator": "eq", "expected": 200}
--- a/examples/quickstart-demo-rev-1.yml
+++ b/examples/quickstart-demo-rev-1.yml
@@ -10,7 +10,7 @@
            app_version: 2.8.6
        json:
            sign: 19067cf712265eb5426db8d3664026c1ccea02b9
-    extract_binds:
+    extractors:
        - token: content.token
    validators:
        - {"check": "status_code", "comparator": "eq", "expected": 200}
--- a/examples/quickstart-demo-rev-2.yml
+++ b/examples/quickstart-demo-rev-2.yml
@@ -1,6 +1,6 @@
 - test:
    name: get token
-    variable_binds:
+    variables:
        - user_agent: 'iOS/10.3'
        - device_sn: ${gen_random_string(15)}
        - os_platform: 'ios'
@@ -15,7 +15,7 @@
            app_version: $app_version
        json:
            sign: ${get_sign($user_agent, $device_sn, $os_platform, $app_version)}
-    extract_binds:
+    extractors:
        - token: content.token
    validators:
        - {"check": "status_code", "comparator": "eq", "expected": 200}
--- a/examples/quickstart-demo-rev-3.yml
+++ b/examples/quickstart-demo-rev-3.yml
@@ -1,6 +1,6 @@
 - config:
    name: "smoketest for CRUD users."
-    variable_binds:
+    variables:
        - device_sn: ${gen_random_string(15)}
    request:
        base_url: http://127.0.0.1:5000
@@ -9,7 +9,7 @@

 - test:
    name: get token
-    variable_binds:
+    variables:
        - user_agent: 'iOS/10.3'
        - os_platform: 'ios'
        - app_version: '2.8.6'
@@ -22,7 +22,7 @@
            app_version: $app_version
        json:
            sign: ${get_sign($user_agent, $device_sn, $os_platform, $app_version)}
-    extract_binds:
+    extractors:
        - token: content.token
    validators:
        - {"check": "status_code", "comparator": "eq", "expected": 200}
--- a/requirements_dev.txt
+++ b/requirements_dev.txt
@@ -4,5 +4,4 @@ PyYAML
 coveralls
 coverage
 -e git+https://github.com/debugtalk/PyUnitReport.git#egg=PyUnitReport
-e git+https://github.com/debugtalk/jenkins-mail-py.git#egg=jenkins-mail-py
 -e git+https://github.com/locustio/locust.git#egg=locustio
--- a/setup.py
+++ b/setup.py
@@ -30,26 +30,20 @@ setup(
        "PyUnitReport"
    ],
    extras_require={
-        'mail': [
-            "jenkins-mail-py"
-        ],
-        'locust': [
+        'locustio': [
            "locustio"
        ]
    },
    dependency_links=[
        "git+https://github.com/debugtalk/PyUnitReport.git#egg=PyUnitReport-0",
-        "git+https://github.com/debugtalk/jenkins-mail-py.git#egg=jenkins-mail-py-0",
        "git+https://github.com/locustio/locust.git#egg=locust-0"
    ],
    classifiers=[
        "Development Status :: 3 - Alpha",
        'Programming Language :: Python :: 2.7',
-        'Programming Language :: Python :: 3.3',
        'Programming Language :: Python :: 3.4',
        'Programming Language :: Python :: 3.5',
-        'Programming Language :: Python :: 3.6',
-        'Programming Language :: Python :: 3.7-dev'
+        'Programming Language :: Python :: 3.6'
    ],
    entry_points={
        'console_scripts': [
--- a/tests/data/demo_binds.yml
+++ b/tests/data/demo_binds.yml
@@ -1,5 +1,5 @@
 bind_variables:
-    variable_binds:
+    variables:
        - TOKEN: "debugtalk"
        - token: $TOKEN

@@ -7,7 +7,7 @@ bind_lambda_functions:
    function_binds:
        add_one: "lambda x: x + 1"
        add_two_nums: "lambda x, y: x + y"
-    variable_binds:
+    variables:
        - add1: ${add_one(2)}
        - sum2nums: ${add_two_nums(2, 3)}

@@ -19,7 +19,7 @@ bind_lambda_functions_with_import:
    function_binds:
        gen_random_string: "lambda str_len: ''.join(random.choice(string.ascii_letters + string.digits) for _ in range(str_len))"
        gen_md5: "lambda *str_args: hashlib.md5(''.join(str_args).encode('utf-8')).hexdigest()"
-    variable_binds:
+    variables:
        - TOKEN: debugtalk
        - random: ${gen_random_string(5)}
        - data: "{'name': 'user', 'password': '123456'}"
@@ -29,7 +29,7 @@ bind_module_functions:
    function_binds:
        import_module_items:
            - tests.data.debugtalk
-    variable_binds:
+    variables:
        - TOKEN: debugtalk
        - random: ${gen_random_string(5)}
        - data: "{'name': 'user', 'password': '123456'}"
--- a/tests/data/demo_testset_hardcode.json
+++ b/tests/data/demo_testset_hardcode.json
@@ -16,7 +16,7 @@
                    "sign": "f1219719911caae89ccc301679857ebfda115ca2"
                }
            },
-            "extract_binds": [
+            "extractors": [
                {
                    "token": "content.token"
                }
--- a/tests/data/demo_testset_hardcode.yml
+++ b/tests/data/demo_testset_hardcode.yml
@@ -11,7 +11,7 @@
            app_version: '2.8.6'
        json:
            sign: f1219719911caae89ccc301679857ebfda115ca2
-    extract_binds:
+    extractors:
        - token: content.token
    validators:
        - {"check": "status_code", "comparator": "eq", "expected": 200}
--- a/tests/data/demo_testset_layer.yml
+++ b/tests/data/demo_testset_layer.yml
@@ -1,6 +1,6 @@
 - config:
-    name: "create user testsets."
-    variable_binds:
+    name: "user management testset."
+    variables:
        - user_agent: 'iOS/10.3'
        - device_sn: ${gen_random_string(15)}
        - os_platform: 'ios'
@@ -16,7 +16,7 @@
 - test:
    name: get token
    api: get_token($user_agent, $device_sn, $os_platform, $app_version)
-    extract_binds:
+    extractors:
        - token: content.token

 - test:
@@ -35,7 +35,7 @@

 - test:
    name: create user which does not exist
-    variable_binds:
+    variables:
        - user_name: "user1"
        - user_password: "123456"
    api: create_user(1000, $user_name, $user_password, $token)
@@ -53,7 +53,7 @@

 - test:
    name: create user which exists
-    variable_binds:
+    variables:
        - user_name: "user1"
        - user_password: "123456"
    api: create_user(1000, $user_name, $user_password, $token)
@@ -63,7 +63,7 @@

 - test:
    name: update user which exists
-    variable_binds:
+    variables:
        - user_name: "user1"
        - user_password: "654321"
    api: update_user(1000, $user_name, $user_password, $token)
@@ -72,7 +72,7 @@
        - {"check": "content.success", "expected": true}

 - test:
-    name: get user that has been created
+    name: get user that has been updated
    api: get_user(1000, $token)
    validators:
        - {"check": "status_code", "expected": 200}
@@ -102,7 +102,7 @@

 - test:
    name: create user which has been deleted
-    variable_binds:
+    variables:
        - user_name: "user1"
        - user_password: "123456"
    api: create_user(1000, $user_name, $user_password, $token)
--- a/tests/data/demo_testset_template_import_functions.yml
+++ b/tests/data/demo_testset_template_import_functions.yml
@@ -2,7 +2,7 @@
    name: "create user testsets."
    import_module_items:
        - tests.data.debugtalk
-    variable_binds:
+    variables:
        - user_agent: 'iOS/10.3'
        - device_sn: ${gen_random_string(15)}
        - os_platform: 'ios'
@@ -25,7 +25,7 @@
            app_version: $app_version
        json:
            sign: ${get_sign($user_agent, $device_sn, $os_platform, $app_version)}
-    extract_binds:
+    extractors:
        - token: content.token
    validators:
        - {"check": "status_code", "comparator": "eq", "expected": 200}
@@ -33,7 +33,7 @@

 - test:
    name: create user which does not exist
-    variable_binds:
+    variables:
        - user_name: "user1"
        - user_password: "123456"
    request:
--- a/tests/data/demo_testset_template_lambda_functions.yml
+++ b/tests/data/demo_testset_template_lambda_functions.yml
@@ -12,7 +12,7 @@
            'DebugTalk'.encode('ascii'),
            ''.join(args).encode('ascii'),
            hashlib.sha1).hexdigest()"
-    variable_binds:
+    variables:
        - user_agent: 'iOS/10.3'
        - device_sn: ${gen_random_string_lambda(15)}
        - os_platform: 'ios'
@@ -35,7 +35,7 @@
            app_version: $app_version
        json:
            sign: ${get_sign_lambda($user_agent, $device_sn, $os_platform, $app_version)}
-    extract_binds:
+    extractors:
        - token: content.token
    validators:
        - {"check": "status_code", "comparator": "eq", "expected": 200}
@@ -43,7 +43,7 @@

 - test:
    name: create user which does not exist
-    variable_binds:
+    variables:
        - user_name: "user1"
        - user_password: "123456"
    request:
--- a/tests/data/demo_testset_variables.yml
+++ b/tests/data/demo_testset_variables.yml
@@ -1,6 +1,6 @@
 - config:
    name: "create user testsets."
-    variable_binds:
+    variables:
        - device_sn: 'HZfFBh6tU59EdXJ'
    request:
        base_url: $BASE_URL
@@ -10,7 +10,7 @@

 - test:
    name: get token
-    variable_binds:
+    variables:
        - user_agent: 'iOS/10.3'
        - os_platform: 'ios'
        - app_version: '2.8.6'
@@ -26,7 +26,7 @@
            app_version: $app_version
        json:
            sign: $sign
-    extract_binds:
+    extractors:
        - token: content.token
    validators:
        - {"check": "status_code", "comparator": "eq", "expected": 200}
@@ -34,7 +34,7 @@

 - test:
    name: create user which does not exist
-    variable_binds:
+    variables:
        - user_name: "user1"
        - user_password: "123456"
    request:
--- a/tests/test_context.py
+++ b/tests/test_context.py
@@ -18,11 +18,11 @@ class VariableBindsUnittest(unittest.TestCase):
        self.assertIn("get_timestamp", self.context.testset_functions_config)
        self.assertIn("gen_random_string", self.context.testset_functions_config)

-        variable_binds = [
+        variables = [
            {"random": "${gen_random_string(5)}"},
            {"timestamp10": "${get_timestamp(10)}"}
        ]
-        self.context.bind_variables(variable_binds)
+        self.context.bind_variables(variables)
        context_variables = self.context.get_testcase_variables_mapping()

        self.assertEqual(len(context_variables["random"]), 5)
@@ -31,7 +31,7 @@ class VariableBindsUnittest(unittest.TestCase):
    def test_context_bind_testset_variables(self):
        # testcase in JSON format
        testcase1 = {
-            "variable_binds": [
+            "variables": [
                {"GLOBAL_TOKEN": "debugtalk"},
                {"token": "$GLOBAL_TOKEN"}
            ]
@@ -40,8 +40,8 @@ class VariableBindsUnittest(unittest.TestCase):
        testcase2 = self.testcases["bind_variables"]

        for testcase in [testcase1, testcase2]:
-            variable_binds = testcase['variable_binds']
-            self.context.bind_variables(variable_binds, level="testset")
+            variables = testcase['variables']
+            self.context.bind_variables(variables, level="testset")

            testset_variables = self.context.testset_shared_variables_mapping
            testcase_variables = self.context.get_testcase_variables_mapping()
@@ -54,7 +54,7 @@ class VariableBindsUnittest(unittest.TestCase):

    def test_context_bind_testcase_variables(self):
        testcase1 = {
-            "variable_binds": [
+            "variables": [
                {"GLOBAL_TOKEN": "debugtalk"},
                {"token": "$GLOBAL_TOKEN"}
            ]
@@ -62,8 +62,8 @@ class VariableBindsUnittest(unittest.TestCase):
        testcase2 = self.testcases["bind_variables"]

        for testcase in [testcase1, testcase2]:
-            variable_binds = testcase['variable_binds']
-            self.context.bind_variables(variable_binds)
+            variables = testcase['variables']
+            self.context.bind_variables(variables)

            testset_variables = self.context.testset_shared_variables_mapping
            testcase_variables = self.context.get_testcase_variables_mapping()
@@ -80,7 +80,7 @@ class VariableBindsUnittest(unittest.TestCase):
                "add_one": lambda x: x + 1,
                "add_two_nums": lambda x, y: x + y
            },
-            "variable_binds": [
+            "variables": [
                {"add1": "${add_one(2)}"},
                {"sum2nums": "${add_two_nums(2,3)}"}
            ]
@@ -91,8 +91,8 @@ class VariableBindsUnittest(unittest.TestCase):
            function_binds = testcase.get('function_binds', {})
            self.context.bind_functions(function_binds)

-            variable_binds = testcase['variable_binds']
-            self.context.bind_variables(variable_binds)
+            variables = testcase['variables']
+            self.context.bind_variables(variables)

            context_variables = self.context.get_testcase_variables_mapping()
            self.assertIn("add1", context_variables)
@@ -107,7 +107,7 @@ class VariableBindsUnittest(unittest.TestCase):
                "gen_random_string": "lambda str_len: ''.join(random.choice(string.ascii_letters + string.digits) for _ in range(str_len))",
                "gen_md5": "lambda *str_args: hashlib.md5(''.join(str_args).encode('utf-8')).hexdigest()"
            },
-            "variable_binds": [
+            "variables": [
                {"TOKEN": "debugtalk"},
                {"random": "${gen_random_string(5)}"},
                {"data": '{"name": "user", "password": "123456"}'},
@@ -123,8 +123,8 @@ class VariableBindsUnittest(unittest.TestCase):
            function_binds = testcase.get('function_binds', {})
            self.context.bind_functions(function_binds)

-            variable_binds = testcase['variable_binds']
-            self.context.bind_variables(variable_binds)
+            variables = testcase['variables']
+            self.context.bind_variables(variables)
            context_variables = self.context.get_testcase_variables_mapping()

            self.assertIn("TOKEN", context_variables)
@@ -144,7 +144,7 @@ class VariableBindsUnittest(unittest.TestCase):
    def test_import_module_items(self):
        testcase1 = {
            "import_module_items": ["tests.data.debugtalk"],
-            "variable_binds": [
+            "variables": [
                {"TOKEN": "debugtalk"},
                {"random": "${gen_random_string(5)}"},
                {"data": '{"name": "user", "password": "123456"}'},
@@ -157,8 +157,8 @@ class VariableBindsUnittest(unittest.TestCase):
            module_items = testcase.get('import_module_items', [])
            self.context.import_module_items(module_items)

-            variable_binds = testcase['variable_binds']
-            self.context.bind_variables(variable_binds)
+            variables = testcase['variables']
+            self.context.bind_variables(variables)
            context_variables = self.context.get_testcase_variables_mapping()

            self.assertIn("TOKEN", context_variables)
@@ -182,7 +182,7 @@ class VariableBindsUnittest(unittest.TestCase):
        test_runner = runner.Runner()
        testcase = {
            "import_module_items": ["tests.data.debugtalk"],
-            "variable_binds": [
+            "variables": [
                {"TOKEN": "debugtalk"},
                {"random": "${gen_random_string(5)}"},
                {"data": '{"name": "user", "password": "123456"}'},
@@ -206,7 +206,7 @@ class VariableBindsUnittest(unittest.TestCase):
        self.assertIn("random", parsed_request["headers"])
        self.assertEqual(len(parsed_request["headers"]["random"]), 5)
        self.assertIn("data", parsed_request)
-        self.assertEqual(parsed_request["data"], testcase["variable_binds"][2]["data"])
+        self.assertEqual(parsed_request["data"], testcase["variables"][2]["data"])
        self.assertEqual(parsed_request["headers"]["SECRET_KEY"], "DebugTalk")

    def test_exec_content_functions(self):
--- a/tests/test_runner.py
+++ b/tests/test_runner.py
@@ -53,7 +53,7 @@ class TestRunner(ApiServerUnittest):
                    "sign": "f1219719911caae89ccc301679857ebfda115ca2"
                }
            },
-            "extract_binds": [
+            "extractors": [
                {"token": "content.token"}
            ],
            "validators": [
--- a/tests/test_testcase.py
+++ b/tests/test_testcase.py
@@ -55,7 +55,7 @@ class TestcaseParserUnittest(unittest.TestCase):
        )

    def test_eval_content_variables(self):
-        variable_binds = {
+        variables = {
            "var_1": "abc",
            "var_2": "def",
            "var_3": 123,
@@ -63,7 +63,7 @@ class TestcaseParserUnittest(unittest.TestCase):
            "var_5": True,
            "var_6": None
        }
-        testcase_parser = testcase.TestcaseParser(variables_binds=variable_binds)
+        testcase_parser = testcase.TestcaseParser(variables=variables)
        self.assertEqual(
            testcase_parser.eval_content_variables("$var_1"),
            "abc"
@@ -157,11 +157,11 @@ class TestcaseParserUnittest(unittest.TestCase):
        )

    def test_parse_content_with_bindings_variables(self):
-        variables_binds = {
+        variables = {
            "str_1": "str_value1",
            "str_2": "str_value2"
        }
-        testcase_parser = testcase.TestcaseParser(variables_binds=variables_binds)
+        testcase_parser = testcase.TestcaseParser(variables=variables)
        self.assertEqual(
            testcase_parser.parse_content_with_bindings("$str_1"),
            "str_value1"
@@ -184,11 +184,11 @@ class TestcaseParserUnittest(unittest.TestCase):
        )

    def test_parse_content_with_bindings_multiple_identical_variables(self):
-        variables_binds = {
+        variables = {
            "userid": 100,
            "data": 1498
        }
-        testcase_parser = testcase.TestcaseParser(variables_binds=variables_binds)
+        testcase_parser = testcase.TestcaseParser(variables=variables)
        content = "/users/$userid/training/$data?userId=$userid&data=$data"
        self.assertEqual(
            testcase_parser.parse_content_with_bindings(content),
@@ -196,12 +196,12 @@ class TestcaseParserUnittest(unittest.TestCase):
        )

    def test_parse_variables_multiple_identical_variables(self):
-        variables_binds = {
+        variables = {
            "user": 100,
            "userid": 1000,
            "data": 1498
        }
-        testcase_parser = testcase.TestcaseParser(variables_binds=variables_binds)
+        testcase_parser = testcase.TestcaseParser(variables=variables)
        content = "/users/$user/$userid/$data?userId=$userid&data=$data"
        self.assertEqual(
            testcase_parser.parse_content_with_bindings(content),
@@ -210,17 +210,17 @@ class TestcaseParserUnittest(unittest.TestCase):

    def test_parse_content_with_bindings_functions(self):
        import random, string
-        functions_binds = {
+        functions = {
            "gen_random_string": lambda str_len: ''.join(random.choice(string.ascii_letters + string.digits) \
                for _ in range(str_len))
        }
-        testcase_parser = testcase.TestcaseParser(functions_binds=functions_binds)
+        testcase_parser = testcase.TestcaseParser(functions=functions)

        result = testcase_parser.parse_content_with_bindings("${gen_random_string(5)}")
        self.assertEqual(len(result), 5)

        add_two_nums = lambda a, b=1: a + b
-        functions_binds["add_two_nums"] = add_two_nums
+        functions["add_two_nums"] = add_two_nums
        self.assertEqual(
            testcase_parser.parse_content_with_bindings("${add_two_nums(1)}"),
            2
@@ -265,10 +265,10 @@ class TestcaseParserUnittest(unittest.TestCase):
        )

    def test_eval_content_functions(self):
-        functions_binds = {
+        functions = {
            "add_two_nums": lambda a, b=1: a + b
        }
-        testcase_parser = testcase.TestcaseParser(functions_binds=functions_binds)
+        testcase_parser = testcase.TestcaseParser(functions=functions)
        self.assertEqual(
            testcase_parser.eval_content_functions("${add_two_nums(1, 2)}"),
            3
@@ -289,13 +289,13 @@ class TestcaseParserUnittest(unittest.TestCase):
        self.assertEqual(content, "/api/900150983cd24fb0d6963f7d28e17f72")

    def test_parse_content_with_bindings_testcase(self):
-        variables_binds = {
+        variables = {
            "uid": "1000",
            "random": "A2dEx",
            "authorization": "a83de0ff8d2e896dbd8efb81ba14e17d",
            "data": {"name": "user", "password": "123456"}
        }
-        functions_binds = {
+        functions = {
            "add_two_nums": lambda a, b=1: a + b,
            "get_timestamp": lambda: int(time.time() * 1000)
        }
@@ -310,7 +310,7 @@ class TestcaseParserUnittest(unittest.TestCase):
            },
            "body": "$data"
        }
-        parsed_testcase = testcase.TestcaseParser(variables_binds, functions_binds)\
+        parsed_testcase = testcase.TestcaseParser(variables, functions)\
            .parse_content_with_bindings(testcase_template)

        self.assertEqual(
@@ -319,15 +319,15 @@ class TestcaseParserUnittest(unittest.TestCase):
        )
        self.assertEqual(
            parsed_testcase["headers"]["authorization"],
-            variables_binds["authorization"]
+            variables["authorization"]
        )
        self.assertEqual(
            parsed_testcase["headers"]["random"],
-            variables_binds["random"]
+            variables["random"]
        )
        self.assertEqual(
            parsed_testcase["body"],
-            variables_binds["data"]
+            variables["data"]
        )
        self.assertEqual(
            parsed_testcase["headers"]["sum"],
@@ -417,7 +417,7 @@ class TestcaseParserUnittest(unittest.TestCase):
        path = os.path.join(
            os.getcwd(), 'tests/data/demo_testset_layer.yml')
        testsets_list = testcase.load_testcases_by_path(path)
-        self.assertIn("variable_binds", testsets_list[0]["config"])
+        self.assertIn("variables", testsets_list[0]["config"])
        self.assertIn("request", testsets_list[0]["config"])
        self.assertIn("request", testsets_list[0]["testcases"][0])
        self.assertIn("url", testsets_list[0]["testcases"][0]["request"])