Categories
Uncategorized

Allure-pytest features introduced

Foreword

Allure framework is a flexible lightweight multi-language test reporting tools, which not only shows the way to the web Introduction of test results, and allows everyone to participate in the development process to maximize the extraction of useful information from the tests performed daily from dev / qa point of view, Allure report simplifies common statistical flaw: the failure of the test can be divided into bug and interrupted test, you can also configure the log, step, fixture, accessories, timing, execution history and a TMS management and bUG system integration, so the above configuration, all the developers and testers responsible for testing the information can grasp as much as possible. From the point of view manager, it provides a clear Allure “big picture,” which includes the characteristics have been covered, the location of the defect aggregate appearance execution timeline and many other things easily. allure modularity and scalability ensures that you can always fine-tune some things to make Allure is right for you, so today we are concerned that how to make reports to display more detailed information we need, and the allure of jenkins integration

Generate reports

How to write pytest Framework Program to generate test reports, and will not be explained, the specific process may refer to: pytest advanced test report of html

Note: python plugin is used allure allure-pytest

Test code

In order that we can quickly recognize all the features of the allure, accompanied by a full test code

"""
------------------------------------
@Time : 2019/8/28 19:50
@Auth : linux超
@File : conftest.py
@IDE  : PyCharm
@Motto: Real warriors,dare to face the bleak warning,dare to face the incisive error!
@QQ   : [email protected]
@GROUP: 878565760
------------------------------------
"""
import pytest
import allure



@pytest.mark.hookwrapper
def pytest_runtest_makereport(item):
    outcome = yield
    report = outcome.get_result()
    report.nodeid = report.nodeid.encode("utf-8").decode("unicode_escape")  #

Solve the garbage

@allure.step("

Open your browser

") def fixture_step(): pass @pytest.fixture def init_url(): fixture_step() yield True

conftest.py

"""
------------------------------------
@Time : 2019/9/4 21:05
@Auth : linux超
@File : test_allure_feature.py
@IDE  : PyCharm
@Motto: Real warriors,dare to face the bleak warning,dare to face the incisive error!
@QQ   : [email protected]
@GROUP: 878565760
------------------------------------
"""
import pytest
import allure
import os


def login(username=None, password=None):
    """

Login simulation

""" user = "

Super linux

" pwd = "123456" if user == username and pwd == password: return {"code": 1001, "msg": "

login successful

", "data": None} elif "" == password or password is None and username: return {"code": 1002, "msg": "

password can not be blank

", "data": None} elif "" == username or username is None and password: return {"code": 1003, "msg": "

Username can not be empty

", "data": None} else: return {"code": 1004, "msg": "

wrong user name or password

", "data": None} @allure.step("

Enter your user name

") def input_username(user): print("

Enter your user name

") return user @allure.step("

enter password

") def input_password(pwd): print("

enter password

") return pwd login_success_data = [ #

Test Data

{ "case": "

The correct user name and password correctly

", "user": "

Super linux

", "pwd": "123456", "expected": {"code": 1001, "msg": "

login successful

", "data": None} } ] login_fail_data = [ { "case": "

The correct user name, password is empty

", "user": "

Super linux

", "pwd": "", "expected": {"code": 1002, "msg": "

password can not be blank

", "data": None} }, { "case": "

User name is blank, the password is correct

", "user": "", "pwd": "

linux super-brother

", "expected": {"code": 1003, "msg": "

Username can not be empty

", "data": None} }, { "case": "

User name wrong, wrong password

", "user": "linux", "pwd": "linux", "expected": {"code": 1004, "msg": "

wrong user name or password

", "data": None} } ] username_none = [ { "case": "

The default user name parameter

", "pwd": "123456", "expected": {"code": 1003, "msg": "

Username can not be empty

", "data": None} } ] password_none = [ { "case": "

The default password parameters

", "user": "

Super linux

", "expected": {"code": 1002, "msg": "

password can not be blank

", "data": None} } ] #

Change output

ids_login_success_data = [ "

Test} {username: password} {} {} {expectations

". format(data["case"], data["user"], data["pwd"], data["expected"]) for data in login_success_data ] ids_login_fail_data = [ "

Test} {username: password} {} {} {expectations

". format(data["case"], data["user"], data["pwd"], data["expected"]) for data in login_fail_data ] ids_username_none = [ "

{} {} Test password expectation {}

". format(data["case"], data["pwd"], data["expected"]) for data in username_none ] ids_password_none = [ "

Test {} Username: {} {} expectations

". format(data["case"], data["user"], data["expected"]) for data in password_none ] @allure.feature("

Login Module

") class TestLogin(object): @allure.severity(allure.severity_level.BLOCKER) @allure.story("

Login successful test

") @allure.title("

Login success scenario - {data}

") @pytest.mark.parametrize("data", login_success_data, ids=ids_login_success_data) def test_login_success(self, data): """

Login successful test

""" user = input_username(data["user"]) pwd = input_password(data["pwd"]) result = login(user, pwd) assert result == data["expected"] @allure.severity(allure.severity_level.CRITICAL) @allure.story("

Login failed test

") @pytest.mark.parametrize("data", login_fail_data, ids=ids_login_fail_data) def test_login_fail(self, data): """

Test user name or password is incorrect

""" user = input_username(data["user"]) pwd = input_password(data["pwd"]) result = login(user, pwd) assert result == data["expected"] @allure.severity(allure.severity_level.MINOR) @allure.story("

Testing the username parameter is missing

") @pytest.mark.parametrize("data", username_none, ids=ids_username_none) def test_username_none(self, data): """

The default user name test

""" pwd = input_password(data["pwd"]) result = login(password=pwd) assert result == data["expected"] @allure.severity(allure.severity_level.MINOR) @allure.story("

Password missing test parameters

") @pytest.mark.parametrize("data", password_none, ids=ids_password_none) def test_password_none(self, data): """

The default password test

""" user = input_username(data["user"]) result = login(username=user) assert result == data["expected"] @allure.severity(allure.severity_level.MINOR) @allure.story("

Test initialization address

") @allure.testcase("https://www.cnblogs.com/linuxchao/", "

Test address

") def test_init_url(self, init_url): flag = init_url assert flag is True @allure.severity(allure.severity_level.NORMAL) @allure.story("

Test fails use cases supplemented with Example attachments

") @allure.link("https://www.cnblogs.com/linuxchao/", name="

bug link

") @allure.description("

This is a test case has failed to perform

") def test_failed(self): """

You can also add a description of the use case here, but will be covered allure.description

""" try: assert False except AssertionError as e: with open("attach.png", "rb") as f: context = f.read() allure.attach(context, "

Pictures error

", attachment_type=allure.attachment_type.PNG) raise e @allure.severity(allure.severity_level.TRIVIAL) @allure.story("

Test cases broken by

") @allure.issue("https://www.cnblogs.com/linuxchao/", "

Broken links

") def test_broken(self): """broken""" with open("broken.json", "r", encoding='utf8') as f: f.read() @allure.severity(allure.severity_level.TRIVIAL) @allure.story("

Test unconditional skip test cases

") @pytest.mark.skip(reason="

Unconditional skip

") def test_skip(self): """skip""" pass if __name__ == '__main__': pytest.main(["-vsq", "--alluredir", "./allure-results", ]) os.system(r"allure generate --clean ./allure-results -o ./allure-report")

test_allure_feature.py

"""
------------------------------------
@Time : 2019/8/28 19:45
@Auth : linux超
@File : test_allure_fixture.py
@IDE  : PyCharm
@Motto: Real warriors,dare to face the bleak warning,dare to face the incisive error!
@QQ   : [email protected]
@GROUP: 878565760
------------------------------------
"""
import pytest
import os
import allure


def function_scope_step():
    print("function_scope_step")


def class_scope_step():
    print("class_scope_step")


def module_scope_step():
    print("module_scope_step")


def session_scope_step():
    print("session_scope_step")


def step_inside_test_body():
    print("step_inside_test_body")


@pytest.fixture(params=[True, False], ids=['param_true', 'param_false'])
def function_scope_fixture_with_finalizer(request):
    if request.param:
        print('True')
    else:
        print('False')

    def function_scope_finalizer():
        function_scope_step()

    request.addfinalizer(function_scope_finalizer)


@pytest.fixture(scope='class')
def class_scope_fixture_with_finalizer(request):
    def class_finalizer_fixture():
        class_scope_step()

    request.addfinalizer(class_finalizer_fixture)


@pytest.fixture(scope='module')
def module_scope_fixture_with_finalizer(request):
    def module_finalizer_fixture():
        module_scope_step()

    request.addfinalizer(module_finalizer_fixture)


@pytest.fixture(scope='session')
def session_scope_fixture_with_finalizer(request):
    def session_finalizer_fixture():
        session_scope_step()

    request.addfinalizer(session_finalizer_fixture)


@allure.severity(allure.severity_level.BLOCKER)
@allure.feature("

fixture scene

") class TestClass(object): def test_with_scoped_finalizers(self, function_scope_fixture_with_finalizer, class_scope_fixture_with_finalizer, module_scope_fixture_with_finalizer, session_scope_fixture_with_finalizer): step_inside_test_body() if __name__ == '__main__': pytest.main(["-vsq", "--alluredir", "./allure-results", ]) os.system(r"allure generate --clean ./allure-results -o ./allure-report")

test_allure_fixture.py

[
  {
    "name": "Ignored tests",
    "matchedStatuses": ["skipped"]
  },
  {
    "name": "Infrastructure problems",
    "matchedStatuses": ["broken", "failed"],
    "messageRegex": ".*bye-bye.*"
  },
  {
    "name": "Outdated tests",
    "matchedStatuses": ["broken"],
    "traceRegex": ".*FileNotFoundException.*"
  },
  {
    "name": "Product defects",
    "matchedStatuses": ["failed"]
  },
  {
    "name": "Test defects",
    "matchedStatuses": ["broken"]
  }
]

categories.json

Browser=Chrome
Browser.Version=63.0
Stand=Production
ApiUrl=127.0.0.1/login
python.Version=3.6

environment.properties

"""
------------------------------------
@Time : 2019/9/3 14:21
@Auth : linux超
@File : run.py
@IDE  : PyCharm
@Motto: Real warriors,dare to face the bleak warning,dare to face the incisive error!
@QQ   : [email protected]
@GROUP: 878565760
------------------------------------
"""
import pytest
import os


if __name__ == '__main__':
    pytest.main(["-sq",
                 "--alluredir", "./allure-results"])
    os.system(r"allure generate --clean allure-results -o allure-report")

run.py

Directory Structure

Allure properties

Environment

Add Allure environmental information in the report, or by creating environment.properties environment.xml file and store the file to the allure-results (this directory before generating the final html reports, generate dependent files directory) directory

environment.properties

Browser=Chrome
Browser.Version=63.0
Stand=Production
ApiUrl=127.0.0.1/login
python.Version=3.6

or

environment.xml

<environment>
    <parameter>
        <key>Browserkey>
        <value>Chromevalue>
    parameter>
    <parameter>
        <key>Browser.Versionkey>
        <value>63.0value>
    parameter>
    <parameter>
        <key>Standkey>
        <value>Productionvalue>
    parameter>
        <parameter>
        <key>ApiUrlkey>
        <value>127.0.0.1/loginvalue>
    parameter>
        <parameter>
        <key>python.Versionkey>
        <value>3.6value>
    parameter>
environment>

View report execution run.py

Categories

Test Report The default statistics are two types of test results, the failure of the use cases and test failures, we can add a custom statistical type cases, the same need to create categories.json file in the directory allure-results

[
  {
    "name": "Ignored tests",
    "matchedStatuses": ["skipped"]
  },
  {
    "name": "Infrastructure problems",
    "matchedStatuses": ["broken", "failed"],
    "messageRegex": ".*bye-bye.*"
  },
  {
    "name": "Outdated tests",
    "matchedStatuses": ["broken"],
    "traceRegex": ".*FileNotFoundException.*"
  },
  {
    "name": "Product defects",
    "matchedStatuses": ["failed"]
  },
  {
    "name": "Test defects",
    "matchedStatuses": ["broken"]
  }
]

View report execution run.py

Fixtures and Finalizers

Fixtures and Finalizers is pytest test method in the beginning and end of the test calls, allure call will automatically track each fixture, and a detailed display fixture and parameters which will call, and will retain the right to call Counts

Test code

test_allure_html.py

def function_scope_step():
    print("function_scope_step")


def class_scope_step():
    print("class_scope_step")


def module_scope_step():
    print("module_scope_step")


def session_scope_step():
    print("session_scope_step")


def step_inside_test_body():
    print("step_inside_test_body")


@pytest.fixture(params=[True, False], ids=['param_true', 'param_false'])
def function_scope_fixture_with_finalizer(request):
    if request.param:
        print('True')
    else:
        print('False')

    def function_scope_finalizer():
        function_scope_step()

    request.addfinalizer(function_scope_finalizer)


@pytest.fixture(scope='class')
def class_scope_fixture_with_finalizer(request):
    def class_finalizer_fixture():
        class_scope_step()

    request.addfinalizer(class_finalizer_fixture)


@pytest.fixture(scope='module')
def module_scope_fixture_with_finalizer(request):
    def module_finalizer_fixture():
        module_scope_step()

    request.addfinalizer(module_finalizer_fixture)


@pytest.fixture(scope='session')
def session_scope_fixture_with_finalizer(request):
    def session_finalizer_fixture():
        session_scope_step()

    request.addfinalizer(session_finalizer_fixture)


class TestClass(object):

    def test_with_scoped_finalizers(self,
                                    function_scope_fixture_with_finalizer,
                                    class_scope_fixture_with_finalizer,
                                    module_scope_fixture_with_finalizer,
                                    session_scope_fixture_with_finalizer):
        step_inside_test_body()

View report execution run.py

@allure.step

pytest support some modifications function test cases require the use of @ allure.step, the test cases to display more detailed testing process allure report

Test code

test_allure_feature.py modify the following code file

@allure.step("

Enter your user name

") def input_username(): print("

Enter your user name

") @allure.step("

enter password

") def input_password(): print("

enter password

")

View report execution run.py

conftest.py

@ Allure.step modified test procedures defined in conftest.py also supports file as step fixture, and now our new conftest.py file in the project directory, write the following code

conftest.py

@allure.step("

Open your browser

") def fixture_step(): pass @pytest.fixture def init_url(): fixture_step() yield True

test_allure_feature.py file, add the following use cases

    def test_init_url(self, init_url):
        flag = init_url
        assert flag == True

View report execution run.py

allure.attach

Use allure.attach can add files, pictures, log, html code, etc. to report. We modify test_allure_feature.py in the following use cases, and in cases where the directory with pictures to add attach.png

    def test_failed(self):
        """failed"""
        try:
            assert False
        except AssertionError as e:
            with open("attach.png", "rb") as f:
                context = f.read()
                allure.attach(context, "

Pictures error

", attachment_type=allure.attachment_type.PNG) raise e

View report execution run.py

@allure.description

If you want to show the test case descriptions in the report, then you can use @ allure.description (string) or @ allure.description_html (html code) decorate your test case, test_allure_feature.py file modify the following code

    @allure.description("

This is a test case has failed to perform

") def test_failed(self): """

You can also add a description of the use case here, but will be covered allure.description

""" try: assert False except AssertionError as e: with open("attach.png", "rb") as f: context = f.read() allure.attach(context, "

Pictures error

", attachment_type=allure.attachment_type.PNG) raise e

View report execution run.py

@allure.title

Use allure.title (title) in the name of the test case can rename allure report, test_allure_feature.py following code file modification

    @allure.title("

Login success scenario - {data}

") @pytest.mark.parametrize("data", login_success_data, ids=ids_login_success_data) def test_login_success(self, data): """

Login successful test

""" user = input_username(data["user"]) pwd = input_password(data["pwd"]) result = login(user, pwd) assert result == data["expected"]

@allure.link

@allure.testcase

@allure.issue

These three features can add a link to a test case, test_allure_feature.py file modify the following code

    @allure.testcase("https://www.cnblogs.com/linuxchao/", "

Test address

") def test_init_url(self, init_url): flag = init_url assert flag == True @allure.link("https://www.cnblogs.com/linuxchao/", name="

bug link

") @allure.description("

This is a test case has failed to perform

") def test_failed(self): """

You can also add a description of the use case here, but will be covered allure.description

""" try: assert False except AssertionError as e: with open("attach.png", "rb") as f: context = f.read() allure.attach(context, "

Pictures error

", attachment_type=allure.attachment_type.PNG) raise e @allure.issue("https://www.cnblogs.com/linuxchao/", "

Broken links

") def test_broken(self): """broken""" with open("broken.json", "r", encoding='utf8') as f: f.read()

View report execution run.py

 

 

 

@allure.feature

@allure.story

feature story and is known as behavior-driven mark, since the two markers, can be more clearly grasp the function of each test case and test cases for each test scenario Adoption of the report

Test class uses the @ allure.feature modified test method test_allure_feature.py file modifications using @ allure.story

View report execution run.py

Two or more markers can be displayed not only in the test report, but may also be performed using a command or the specified test scenario module

@allure.severity

This tag is used to identify test cases or test level class, divided blocker, critical, normal, minor, trivial5 levels, the following are the test cases by level mark, and look at the test report

to sum up

These are the most characteristic feature of all the allure-pytest plug-in support in pytest, perhaps finishing is not very detailed, so if you want a detailed understanding of the effect of specific features in the report, the need to try it yourself, attach This article reference links

https://docs.qameta.io/allure/

Leave a Reply