Python pytest tutorial ( Your Ticket to a Bug-Free Code)

pytest is a robust testing framework for Python that allows you to create small, simple tests easily, yet scales to support complex functional testing for applications and libraries.

It provides you with a platform to perform Python testing by writing test cases as functions, resulting in less code and minimal amount of boilerplate code.

It also makes it easy to create test suites, or collections of test cases that are meant to test specific aspects of your Python code.

Furthermore, pytest provides tools to handle the common tasks in testing such as setup and grouping of tests.

 

 

Installing Python pytest

To start using pytest, you first need to install it. You can install pytest using pip, which is a package installer for Python that comes with the standard library.

Here is the command to install pytest:

pip install pytest

After running this command, pytest should be installed in your Python environment, and you can verify this by checking the installed version of pytest:

pytest --version

You might see output similar to:

pytest 7.3.1

This shows that pytest has been successfully installed, and the version that’s been installed is 7.3.1.

Now, you’re ready to begin using pytest for your Python testing needs.

 

Basic pytest terminology

Before we dive deeper into pytest, let’s first get to know some basic terminology:

• Test Case: This is a single unit of testing. It checks for a specific response to a particular set of inputs. In pytest, a test case is a Python function which starts with the keyword test.
• Test Suite: This is a collection of test cases, test suites, or both. It is used to aggregate tests that should be executed together.
• Test Function: It’s a single function that tests a particular unit of code.
• Unit Testing: This is a level of software testing where individual units/ components of a software are tested. The purpose is to validate that each unit of the software performs as designed. pytest provides an easy way to do unit testing in Python.
  Here is an example of a test suite, which consists of two test cases (test functions):

import pytest
def test_addition():
    assert 1 + 1 == 2
def test_subtraction():
    assert 3 - 2 == 1

When running this test suite, pytest identifies two test functions and runs them. The output might be:

test_module.py::test_addition PASSED
test_module.py::test_subtraction PASSED

This output indicates that both test functions in our test suite passed their respective tests.

 

Pytest Naming Conventions

The naming conventions in pytest are not just to keep the code consistent and clean, but also to ensure the functionality of the test discovery mechanisms.

The pytest framework makes use of these conventions to automatically identify and run the tests.

Here are some essential naming conventions used in pytest:

Test Files

The test discovery process in pytest begins with locating the test files. By convention, all test files should be named test_*.py or *_test.py. Pytest will automatically identify these as files containing tests.

Test Functions and Methods

Similarly, all test functions should start with test_, for example, test_example(). If you’re defining test methods within a class, they should also follow this naming convention.

Test Classes

When defining a class of tests, the class should begin with Test, and should not have an __init__ method. For instance, TestClassExample.

Fixture Functions

Fixture functions provide a fixed baseline so that tests execute reliably and produce consistent results. They are named as regular functions but should be descriptive of the specific state they provide. For example, def setup_database() or def login_client().

Although pytest isn’t case sensitive, Python naming conventions suggest the use of lower case words separated by underscores. This is especially important in maintaining readability in your test files.

Remember, the importance of following these conventions isn’t simply about ‘rules’. Correctly naming your tests and classes helps ensure pytest can discover and run your tests, and fixtures appropriately.

 

Running your first test

After creating your test functions, you navigate to the directory that contains your test file and run the pytest command.
For instance, consider the following test case:

import pytest
def test_addition():
    assert 1 + 1 == 2

To run this test, navigate to the directory containing the test file and run:

pytest -v

You should see output similar to:

============================= test session starts ==============================
...

test_module.py::test_addition PASSED

========================== 1 passed in 0.03 seconds ===========================

The “1 passed” message is an overall status report indicating the number of tests that passed.

The -v command-line option stands for verbose, meaning that pytest will output more detailed reports.

Command-line options

1. Specifying tests
   • -k EXPRESSION: Run tests which match the given keyword expression.
   • -m MARKEXPR: Only run tests matching given mark expression.
2. Test session
   • -x, --exitfirst: Exit instantly on first error or failed test.
3. Output
   • -s: Display print statements in output (by default, pytest captures output).
   • -v, --verbose: Increase verbosity.
   • -q, --quiet: Decrease verbosity.
4. Test execution
   • --ff, --failed-first: Run all tests but run the last failures first.
   • -n numprocesses: Shortcut for ‘–dist=each –tx=NUM*popen’, where NUM refers to the number of cores to be used. This comes from the pytest-xdist plugin for parallel execution.
5. Reporting
   • --html=path: Create an HTML report of test results. This requires the pytest-html

 

Anatomy of a pytest function

A pytest function is a Python function that begins with the word test_.

This is how pytest recognizes its tests. Here’s a simple example of a pytest function:

import pytest
def test_addition():
    assert 1 + 1 == 2

In this test function, we use the assert keyword to specify the condition we want to test. If the condition is True, the test passes; if False, the test fails and a breakdown of the failure is provided.
Running this function with pytest would output:

============================= test session starts ==============================
...

test_module.py::test_addition PASSED

========================== 1 passed in 0.03 seconds ===========================

The output provides a status report that shows our test function test_addition has passed.
The assert keyword is one of the reasons pytest can write less code compared to other testing frameworks.

 

Using assertions

In pytest, assertions are made by the assert statement, which checks if the given logical expression is true. If it is not true, the test will fail.
Here’s an example of using assertions in a pytest function:

import pytest
def test_multiplication():
    assert 2 * 3 == 6

Running this test function with pytest, the output would be:

============================= test session starts ==============================
...

test_module.py::test_multiplication PASSED

========================== 1 passed in 0.03 seconds ===========================

In this test function, the assert statement is used to check whether 2 multiplied by 3 equals 6.

As this is true, the test passes, and pytest gives us a “PASSED” output for that test.
However, if the assertion were false, pytest would provide a helpful failure explanation. For example, let’s change our test function:

import pytest
def test_multiplication():
    assert 2 * 3 == 7

Running this test function with pytest, the output would be:

============================= test session starts ==============================
...

test_module.py::test_multiplication FAILED

================================ FAILURES =================================
______________________________ test_multiplication ______________________________

    def test_multiplication():
>       assert 2 * 3 == 7
E       assert (2 * 3) == 7

test_module.py:4: AssertionError
========================== 1 failed in 0.03 seconds ===========================

In this case, pytest shows exactly where the assertion failed and why, providing a breakdown of the failure. This makes it easier to debug and fix the test.

 

Using pytest Markers

Pytest markers allow us to easily set metadata on our test functions, such as marking tests as expected to fail, or that they should be skipped under certain conditions.

Skipping tests

For example, you can use the skip marker to skip a particular test:

import pytest

@pytest.mark.skip(reason="Skip this test")
def test_addition():
    assert 1 + 1 == 2

When you run this test with pytest, the output will be:

============================= test session starts ==============================
...

test_module.py::test_addition SKIPPED

========================== 1 skipped in 0.02 seconds ===========================

Expected failures

The xfail marker indicates that a test is expected to fail:

import pytest

@pytest.mark.xfail
def test_failed_addition():
    assert 1 + 1 == 3

Running this with pytest, the output will be:

============================= test session starts ==============================
...

test_module.py::test_failed_addition XFAIL

========================== 1 xfailed in 0.02 seconds ===========================

Marking and filtering tests

You can also create your own custom markers. For example, you can mark some tests to be run only on a specific platform:

import pytest
import sys

@pytest.mark.windows
def test_windows_only_behavior():
    assert sys.platform == 'win32'

@pytest.mark.linux
def test_linux_only_behavior():
    assert sys.platform == 'linux'

You can then run only the tests that have a specific marker:

pytest -m windows

The -m flag allows you to run tests which are marked with the given marker name.

 

Handling Exceptions in pytest

pytest provides several ways to assert that exceptions are raised. The main tool for this is the pytest.raises function, which can be used to test that a certain piece of code raises a specific exception.
Let’s take an example:

import pytest
def test_division_by_zero():
    with pytest.raises(ZeroDivisionError):
        num = 1 / 0

In this test function, we are testing that dividing by zero raises a ZeroDivisionError. Running this with pytest would output:

============================= test session starts ==============================
...

test_module.py::test_division_by_zero PASSED

========================== 1 passed in 0.03 seconds ===========================

The test is successful because dividing by zero indeed raises a ZeroDivisionError.

Testing warnings

To test warnings, you can use pytest.warns. Here is an example:

import pytest
import warnings

def test_warning():
    with pytest.warns(UserWarning):
        warnings.warn("This is a warning!", UserWarning)

When you run this test with pytest, the output will be:

============================= test session starts ==============================
...

test_module.py::test_warning PASSED

========================== 1 passed in 0.03 seconds ===========================

In this test, we are checking that our code emits a UserWarning. The test is successful because our code indeed emits a UserWarning.
These are some of the ways pytest helps in testing for exceptions and warnings.

 

pytest Fixtures

pytest fixtures are functions that are run before each test function to which it is applied. Fixtures are used when we want to run some code before every test method.
Let’s start with a simple fixture:

import pytest

@pytest.fixture
def input_value():
    input = 39
    return input

def test_division(input_value):
    assert input_value % 3 == 0

Here, we are defining a fixture function input_value() that returns a value of 39. We are then using this fixture in test_division() by passing it as an argument.

pytest knows that input_value is a fixture and calls it, then it passes the return value to the test.
Running this with pytest will output:

============================= test session starts ==============================
...

test_module.py::test_division PASSED

========================== 1 passed in 0.03 seconds ===========================

Using built-in pytest fixtures

pytest also provides several built-in fixtures. For example, the tmpdir fixture provides a temporary directory unique to the test invocation:

def test_create_file(tmpdir):
    p = tmpdir.mkdir("sub").join("hello.txt")
    p.write("content")
    assert p.read() == "content"

Scope of fixtures

The scope of a fixture function can be controlled by providing a scope parameter to the @pytest.fixture decorator.

For example, if you want to create a fixture that is initialized once per module, you can use the module scope:

import pytest

@pytest.fixture(scope="module")
def input_value():
    input = 39
    return input

This fixture will only be executed once per module. Other possible values for scope are function, class, and session.

 

pytest conftest.py

conftest.py is a local plugin for shared fixture functions, hooks, plugins, and other configuration for pytest.

It gets automatically discovered by pytest, so you don’t need to import it in your test files. The name conftest stands for configuration test.

conftest.py should be located in the root directory of your project or in the directory containing your test files. pytest will find the conftest.py file at test collection time.

One common use of conftest.py is to store pytest fixture functions that can be used across multiple test files. For example, consider the following fixture function:

# conftest.py
import pytest

@pytest.fixture
def csv_data():
    return [
        {"name": "John", "age": 30, "job": "developer"},
        {"name": "Jane", "age": 25, "job": "designer"},
    ]

This csv_data fixture can now be used in any test file in the same directory as conftest.py or any subdirectories. Here’s an example of a test file using this fixture:

# test_sample.py
def test_csv_data(csv_data):
    for row in csv_data:
        assert isinstance(row["name"], str)
        assert isinstance(row["age"], int)
        assert isinstance(row["job"], str)

Running this test with pytest, you will find that the csv_data fixture from conftest.py is automatically used:

============================= test session starts ==============================
...

test_sample.py::test_csv_data PASSED

========================== 1 passed in 0.03 seconds ===========================

 

Grouping tests in classes

pytest allows you to group several tests that all make similar assertions into a class. This helps keep your tests organized and allows you to share setup or fixture code.
Here is an example of how you can use a class to group tests:

import pytest
class TestMathOperations:
    def test_addition(self):
        assert 1 + 1 == 2
    def test_subtraction(self):
        assert 3 - 2 == 1

Running this with pytest, the output would look like this:

============================= test session starts ==============================
...

test_module.py::TestMathOperations::test_addition PASSED
test_module.py::TestMathOperations::test_subtraction PASSED

========================== 2 passed in 0.04 seconds ===========================

In the output, we see that both tests have been recognized and run under the TestMathOperations test class. This way, as our test suite grows, we can keep our tests well-structured and maintainable.

 

Parameterizing tests

Parameterizing tests in pytest allows you to run a test function multiple times with different arguments. This is done using the @pytest.mark.parametrize decorator.

Parameterization helps to test the behavior of a function for various inputs and expected results without writing multiple test cases.
Here’s a simple example of parameterized tests:

import pytest

@pytest.mark.parametrize("test_input,expected", [(3, 9), (2, 4), (6, 36)])
def test_calc_square(test_input, expected):
    assert test_input**2 == expected

In this example, test_calc_square is run three times. Each time, it uses a different set of values for test_input and expected.
Running this with pytest will output:

============================= test session starts ==============================
...

test_module.py::test_calc_square[3-9] PASSED
test_module.py::test_calc_square[2-4] PASSED
test_module.py::test_calc_square[6-36] PASSED

========================== 3 passed in 0.04 seconds ===========================

Each line in the output represents one test function invocation with a different set of parameters.

This technique is powerful for reducing the amount of boilerplate code in your tests and making them more readable and maintainable.

 

pytest Mocking

Sometimes in unit testing, we want to replace parts of the system under test to isolate it from the rest of the system.

This process is called mocking, and Python’s standard library, unittest.mock, provides a powerful framework for it.
Here’s an example of how you can use mocking in pytest:

from unittest.mock import MagicMock
import pytest
def test_magic_mock():
    mock = MagicMock()
    mock.__str__.return_value = 'foobarbaz'
    assert str(mock) == 'foobarbaz'

In this example, we’re creating a MagicMock instance and specifying a return value when its __str__ method is called.

The test then asserts that this behavior works as expected. Running this test with pytest, the output would be:

============================= test session starts ==============================
...

test_module.py::test_magic_mock PASSED

========================== 1 passed in 0.03 seconds ===========================

 

pytest Configuration (via pytest.ini)

pytest can be configured to set default behaviors and variables through a configuration file named pytest.ini.

This file needs to be located in the root directory of your project, and pytest will automatically discover and use it.
Here’s an example of a pytest.ini file:

[pytest]
minversion = 7.0
addopts = -ra -q
testpaths =
    tests

In this example, minversion specifies the minimum pytest version required for the tests to run. addopts are additional options that pytest will use by default whenever it’s run.

Here, -ra means that pytest will print a summary of all non-passing tests at the end of a test session, and -q means that pytest will decrease verbosity.

testpaths is a list of directories that pytest will look in for tests.
This file is very useful as your test suite grows in size and complexity, as it can store default behaviors and configurations for pytest, making your testing process more efficient.

 

Generating test reports

Once you’ve run your tests with pytest, it’s often helpful to generate a report of the results. pytest allows you to do this using various plugins.
One such plugin is pytest-html, which generates an HTML report. You can install it using pip:

pip install pytest-html

Once installed, you can generate an HTML report using the --html option:

pytest --html=report.html

Running this command will create an HTML report named “report.html” in your current directory.

The report will provide a breakdown of the failure for each failing test case, the overall status report of the test session, and various other pieces of information.
Another useful plugin is pytest-cov, which provides test coverage reports:

pip install pytest-cov

Once installed, you can use it to generate a coverage report:

pytest --cov=myproject

This command will output a coverage report for the “myproject” directory, indicating the percentage of your code that was executed during the test session.
These are just a couple of examples of how pytest can generate useful reports.

 

Pytest Plugins

pytest plugins are a way to extend pytest’s functionality. These plugins can be installed from the Python Package Index (PyPI) using pip and used by simply importing them in your test code or by specifying them on the command line.
Here’s how you would install the pytest-xdist plugin, which allows for parallel and distributed testing:

pip install pytest-xdist

To use the plugin, you can specify it on the command line when running pytest:

pytest -n 4

In this command, -n is an option provided by pytest-xdist, and 4 is the number of CPU cores to use for testing.

Popular pytest plugins

Some popular pytest plugins include:

• pytest-xdist: for parallel and distributed testing.
• pytest-cov: for measuring test coverage.
• pytest-mock: for easier use of the unittest.mock module.
• pytest-html: for generating HTML reports.

You can also write your own pytest plugins. A pytest plugin is simply a Python package that defines one or more hook functions.

Here’s an example of a simple plugin that prints a message at the start of the test session:

# content of myplugin.py
def pytest_sessionstart():
    print("Starting test session")

To use this plugin, you would specify it on the command line using the -p option:

pytest -p myplugin

 

Conclusion

We’ve covered a wide range of topics from the basics of pytest, to more advanced features like fixtures, parameterization, mocking, and plugins.

I hope this tutorial has been helpful for you and that it will assist you in writing robust tests for your Python projects.

Remember, good tests are crucial for maintaining high-quality code and catching issues early.

Happy testing!