Python pytest tutorial ( Your Ticket to a Bug-Free Code)
pytest is a robust testing framework for Python that allows you to create small, simple tests easily, yet scales to support complex functional testing for applications and libraries.
It provides you with a platform to perform Python testing by writing test cases as functions, resulting in less code and minimal amount of boilerplate code.
It also makes it easy to create test suites, or collections of test cases that are meant to test specific aspects of your Python code.
Furthermore, pytest provides tools to handle the common tasks in testing such as setup and grouping of tests.
- 1 Installing Python pytest
- 2 Basic pytest terminology
- 3 Pytest Naming Conventions
- 4 Running your first test
- 5 Anatomy of a pytest function
- 6 Using assertions
- 7 Using pytest Markers
- 8 Handling Exceptions in pytest
- 9 pytest Fixtures
- 10 pytest conftest.py
- 11 Grouping tests in classes
- 12 Parameterizing tests
- 13 pytest Mocking
- 14 pytest Configuration (via pytest.ini)
- 15 Generating test reports
- 16 Pytest Plugins
- 17 Conclusion
Installing Python pytest
To start using pytest, you first need to install it. You can install pytest using pip, which is a package installer for Python that comes with the standard library.
Here is the command to install pytest:
pip install pytest
After running this command, pytest should be installed in your Python environment, and you can verify this by checking the installed version of pytest:
pytest --version
You might see output similar to:
pytest 7.3.1
This shows that pytest has been successfully installed, and the version that’s been installed is 7.3.1.
Now, you’re ready to begin using pytest for your Python testing needs.
Basic pytest terminology
Before we dive deeper into pytest, let’s first get to know some basic terminology:
- Test Case: This is a single unit of testing. It checks for a specific response to a particular set of inputs. In pytest, a test case is a Python function which starts with the keyword
test
. - Test Suite: This is a collection of test cases, test suites, or both. It is used to aggregate tests that should be executed together.
- Test Function: It’s a single function that tests a particular unit of code.
- Unit Testing: This is a level of software testing where individual units/ components of a software are tested. The purpose is to validate that each unit of the software performs as designed. pytest provides an easy way to do unit testing in Python.
Here is an example of a test suite, which consists of two test cases (test functions):
import pytest def test_addition(): assert 1 + 1 == 2 def test_subtraction(): assert 3 - 2 == 1
When running this test suite, pytest identifies two test functions and runs them. The output might be:
test_module.py::test_addition PASSED test_module.py::test_subtraction PASSED
This output indicates that both test functions in our test suite passed their respective tests.
Pytest Naming Conventions
The naming conventions in pytest are not just to keep the code consistent and clean, but also to ensure the functionality of the test discovery mechanisms.
The pytest framework makes use of these conventions to automatically identify and run the tests.
Here are some essential naming conventions used in pytest:
Test Files
The test discovery process in pytest begins with locating the test files. By convention, all test files should be named test_*.py
or *_test.py
. Pytest will automatically identify these as files containing tests.
Test Functions and Methods
Similarly, all test functions should start with test_
, for example, test_example()
. If you’re defining test methods within a class, they should also follow this naming convention.
Test Classes
When defining a class of tests, the class should begin with Test
, and should not have an __init__
method. For instance, TestClassExample
.
Fixture Functions
Fixture functions provide a fixed baseline so that tests execute reliably and produce consistent results. They are named as regular functions but should be descriptive of the specific state they provide. For example, def setup_database()
or def login_client()
.
Although pytest isn’t case sensitive, Python naming conventions suggest the use of lower case words separated by underscores. This is especially important in maintaining readability in your test files.
Remember, the importance of following these conventions isn’t simply about ‘rules’. Correctly naming your tests and classes helps ensure pytest can discover and run your tests, and fixtures appropriately.
Running your first test
After creating your test functions, you navigate to the directory that contains your test file and run the pytest command.
For instance, consider the following test case:
import pytest def test_addition(): assert 1 + 1 == 2
To run this test, navigate to the directory containing the test file and run:
pytest -v
You should see output similar to:
============================= test session starts ============================== ... test_module.py::test_addition PASSED ========================== 1 passed in 0.03 seconds ===========================
The “1 passed” message is an overall status report indicating the number of tests that passed.
The -v
command-line option stands for verbose, meaning that pytest will output more detailed reports.
Command-line options
- Specifying tests
-k EXPRESSION
: Run tests which match the given keyword expression.-m MARKEXPR
: Only run tests matching given mark expression.
- Test session
-x, --exitfirst
: Exit instantly on first error or failed test.
- Output
-s
: Display print statements in output (by default, pytest captures output).-v, --verbose
: Increase verbosity.-q, --quiet
: Decrease verbosity.
- Test execution
--ff, --failed-first
: Run all tests but run the last failures first.-n numprocesses
: Shortcut for ‘–dist=each –tx=NUM*popen’, where NUM refers to the number of cores to be used. This comes from thepytest-xdist
plugin for parallel execution.
- Reporting
--html=path
: Create an HTML report of test results. This requires thepytest-html
Anatomy of a pytest function
A pytest function is a Python function that begins with the word test_
.
This is how pytest recognizes its tests. Here’s a simple example of a pytest function:
import pytest def test_addition(): assert 1 + 1 == 2
In this test function, we use the assert
keyword to specify the condition we want to test. If the condition is True
, the test passes; if False
, the test fails and a breakdown of the failure is provided.
Running this function with pytest would output:
============================= test session starts ============================== ... test_module.py::test_addition PASSED ========================== 1 passed in 0.03 seconds ===========================
The output provides a status report that shows our test function test_addition
has passed.
The assert
keyword is one of the reasons pytest can write less code compared to other testing frameworks.
Using assertions
In pytest, assertions are made by the assert
statement, which checks if the given logical expression is true. If it is not true, the test will fail.
Here’s an example of using assertions in a pytest function:
import pytest def test_multiplication(): assert 2 * 3 == 6
Running this test function with pytest, the output would be:
============================= test session starts ============================== ... test_module.py::test_multiplication PASSED ========================== 1 passed in 0.03 seconds ===========================
In this test function, the assert
statement is used to check whether 2 multiplied by 3 equals 6.
As this is true, the test passes, and pytest gives us a “PASSED” output for that test.
However, if the assertion were false, pytest would provide a helpful failure explanation. For example, let’s change our test function:
import pytest def test_multiplication(): assert 2 * 3 == 7
Running this test function with pytest, the output would be:
============================= test session starts ============================== ... test_module.py::test_multiplication FAILED ================================ FAILURES ================================= ______________________________ test_multiplication ______________________________ def test_multiplication(): > assert 2 * 3 == 7 E assert (2 * 3) == 7 test_module.py:4: AssertionError ========================== 1 failed in 0.03 seconds ===========================
In this case, pytest shows exactly where the assertion failed and why, providing a breakdown of the failure. This makes it easier to debug and fix the test.
Using pytest Markers
Pytest markers allow us to easily set metadata on our test functions, such as marking tests as expected to fail, or that they should be skipped under certain conditions.
Skipping tests
For example, you can use the skip
marker to skip a particular test:
import pytest @pytest.mark.skip(reason="Skip this test") def test_addition(): assert 1 + 1 == 2
When you run this test with pytest, the output will be:
============================= test session starts ============================== ... test_module.py::test_addition SKIPPED ========================== 1 skipped in 0.02 seconds ===========================
Expected failures
The xfail
marker indicates that a test is expected to fail:
import pytest @pytest.mark.xfail def test_failed_addition(): assert 1 + 1 == 3
Running this with pytest, the output will be:
============================= test session starts ============================== ... test_module.py::test_failed_addition XFAIL ========================== 1 xfailed in 0.02 seconds ===========================
Marking and filtering tests
You can also create your own custom markers. For example, you can mark some tests to be run only on a specific platform:
import pytest import sys @pytest.mark.windows def test_windows_only_behavior(): assert sys.platform == 'win32' @pytest.mark.linux def test_linux_only_behavior(): assert sys.platform == 'linux'
You can then run only the tests that have a specific marker:
pytest -m windows
The -m
flag allows you to run tests which are marked with the given marker name.
Handling Exceptions in pytest
pytest provides several ways to assert that exceptions are raised. The main tool for this is the pytest.raises
function, which can be used to test that a certain piece of code raises a specific exception.
Let’s take an example:
import pytest def test_division_by_zero(): with pytest.raises(ZeroDivisionError): num = 1 / 0
In this test function, we are testing that dividing by zero raises a ZeroDivisionError
. Running this with pytest would output:
============================= test session starts ============================== ... test_module.py::test_division_by_zero PASSED ========================== 1 passed in 0.03 seconds ===========================
The test is successful because dividing by zero indeed raises a ZeroDivisionError
.
Testing warnings
To test warnings, you can use pytest.warns
. Here is an example:
import pytest import warnings def test_warning(): with pytest.warns(UserWarning): warnings.warn("This is a warning!", UserWarning)
When you run this test with pytest, the output will be:
============================= test session starts ============================== ... test_module.py::test_warning PASSED ========================== 1 passed in 0.03 seconds ===========================
In this test, we are checking that our code emits a UserWarning
. The test is successful because our code indeed emits a UserWarning
.
These are some of the ways pytest helps in testing for exceptions and warnings.
pytest Fixtures
pytest fixtures are functions that are run before each test function to which it is applied. Fixtures are used when we want to run some code before every test method.
Let’s start with a simple fixture:
import pytest @pytest.fixture def input_value(): input = 39 return input def test_division(input_value): assert input_value % 3 == 0
Here, we are defining a fixture function input_value()
that returns a value of 39. We are then using this fixture in test_division()
by passing it as an argument.
pytest knows that input_value
is a fixture and calls it, then it passes the return value to the test.
Running this with pytest will output:
============================= test session starts ============================== ... test_module.py::test_division PASSED ========================== 1 passed in 0.03 seconds ===========================
Using built-in pytest fixtures
pytest also provides several built-in fixtures. For example, the tmpdir
fixture provides a temporary directory unique to the test invocation:
def test_create_file(tmpdir): p = tmpdir.mkdir("sub").join("hello.txt") p.write("content") assert p.read() == "content"
Scope of fixtures
The scope of a fixture function can be controlled by providing a scope
parameter to the @pytest.fixture
decorator.
For example, if you want to create a fixture that is initialized once per module, you can use the module
scope:
import pytest @pytest.fixture(scope="module") def input_value(): input = 39 return input
This fixture will only be executed once per module. Other possible values for scope
are function
, class
, and session
.
pytest conftest.py
conftest.py
is a local plugin for shared fixture functions, hooks, plugins, and other configuration for pytest.
It gets automatically discovered by pytest, so you don’t need to import it in your test files. The name conftest
stands for configuration test.
conftest.py
should be located in the root directory of your project or in the directory containing your test files. pytest will find the conftest.py
file at test collection time.
One common use of conftest.py
is to store pytest fixture functions that can be used across multiple test files. For example, consider the following fixture function:
# conftest.py import pytest @pytest.fixture def csv_data(): return [ {"name": "John", "age": 30, "job": "developer"}, {"name": "Jane", "age": 25, "job": "designer"}, ]
This csv_data
fixture can now be used in any test file in the same directory as conftest.py
or any subdirectories. Here’s an example of a test file using this fixture:
# test_sample.py def test_csv_data(csv_data): for row in csv_data: assert isinstance(row["name"], str) assert isinstance(row["age"], int) assert isinstance(row["job"], str)
Running this test with pytest, you will find that the csv_data
fixture from conftest.py
is automatically used:
============================= test session starts ============================== ... test_sample.py::test_csv_data PASSED ========================== 1 passed in 0.03 seconds ===========================
Grouping tests in classes
pytest allows you to group several tests that all make similar assertions into a class. This helps keep your tests organized and allows you to share setup or fixture code.
Here is an example of how you can use a class to group tests:
import pytest class TestMathOperations: def test_addition(self): assert 1 + 1 == 2 def test_subtraction(self): assert 3 - 2 == 1
Running this with pytest, the output would look like this:
============================= test session starts ============================== ... test_module.py::TestMathOperations::test_addition PASSED test_module.py::TestMathOperations::test_subtraction PASSED ========================== 2 passed in 0.04 seconds ===========================
In the output, we see that both tests have been recognized and run under the TestMathOperations
test class. This way, as our test suite grows, we can keep our tests well-structured and maintainable.
Parameterizing tests
Parameterizing tests in pytest allows you to run a test function multiple times with different arguments. This is done using the @pytest.mark.parametrize
decorator.
Parameterization helps to test the behavior of a function for various inputs and expected results without writing multiple test cases.
Here’s a simple example of parameterized tests:
import pytest @pytest.mark.parametrize("test_input,expected", [(3, 9), (2, 4), (6, 36)]) def test_calc_square(test_input, expected): assert test_input**2 == expected
In this example, test_calc_square
is run three times. Each time, it uses a different set of values for test_input
and expected
.
Running this with pytest will output:
============================= test session starts ============================== ... test_module.py::test_calc_square[3-9] PASSED test_module.py::test_calc_square[2-4] PASSED test_module.py::test_calc_square[6-36] PASSED ========================== 3 passed in 0.04 seconds ===========================
Each line in the output represents one test function invocation with a different set of parameters.
This technique is powerful for reducing the amount of boilerplate code in your tests and making them more readable and maintainable.
pytest Mocking
Sometimes in unit testing, we want to replace parts of the system under test to isolate it from the rest of the system.
This process is called mocking, and Python’s standard library, unittest.mock
, provides a powerful framework for it.
Here’s an example of how you can use mocking in pytest:
from unittest.mock import MagicMock import pytest def test_magic_mock(): mock = MagicMock() mock.__str__.return_value = 'foobarbaz' assert str(mock) == 'foobarbaz'
In this example, we’re creating a MagicMock
instance and specifying a return value when its __str__
method is called.
The test then asserts that this behavior works as expected. Running this test with pytest, the output would be:
============================= test session starts ============================== ... test_module.py::test_magic_mock PASSED ========================== 1 passed in 0.03 seconds ===========================
pytest Configuration (via pytest.ini)
pytest can be configured to set default behaviors and variables through a configuration file named pytest.ini
.
This file needs to be located in the root directory of your project, and pytest will automatically discover and use it.
Here’s an example of a pytest.ini
file:
[pytest] minversion = 7.0 addopts = -ra -q testpaths = tests
In this example, minversion
specifies the minimum pytest version required for the tests to run. addopts
are additional options that pytest will use by default whenever it’s run.
Here, -ra
means that pytest will print a summary of all non-passing tests at the end of a test session, and -q
means that pytest will decrease verbosity.
testpaths
is a list of directories that pytest will look in for tests.
This file is very useful as your test suite grows in size and complexity, as it can store default behaviors and configurations for pytest, making your testing process more efficient.
Generating test reports
Once you’ve run your tests with pytest, it’s often helpful to generate a report of the results. pytest allows you to do this using various plugins.
One such plugin is pytest-html, which generates an HTML report. You can install it using pip:
pip install pytest-html
Once installed, you can generate an HTML report using the --html
option:
pytest --html=report.html
Running this command will create an HTML report named “report.html” in your current directory.
The report will provide a breakdown of the failure for each failing test case, the overall status report of the test session, and various other pieces of information.
Another useful plugin is pytest-cov, which provides test coverage reports:
pip install pytest-cov
Once installed, you can use it to generate a coverage report:
pytest --cov=myproject
This command will output a coverage report for the “myproject” directory, indicating the percentage of your code that was executed during the test session.
These are just a couple of examples of how pytest can generate useful reports.
Pytest Plugins
pytest plugins are a way to extend pytest’s functionality. These plugins can be installed from the Python Package Index (PyPI) using pip and used by simply importing them in your test code or by specifying them on the command line.
Here’s how you would install the pytest-xdist plugin, which allows for parallel and distributed testing:
pip install pytest-xdist
To use the plugin, you can specify it on the command line when running pytest:
pytest -n 4
In this command, -n
is an option provided by pytest-xdist, and 4
is the number of CPU cores to use for testing.
Popular pytest plugins
Some popular pytest plugins include:
pytest-xdist
: for parallel and distributed testing.pytest-cov
: for measuring test coverage.pytest-mock
: for easier use of theunittest.mock
module.pytest-html
: for generating HTML reports.
You can also write your own pytest plugins. A pytest plugin is simply a Python package that defines one or more hook functions.
Here’s an example of a simple plugin that prints a message at the start of the test session:
# content of myplugin.py def pytest_sessionstart(): print("Starting test session")
To use this plugin, you would specify it on the command line using the -p
option:
pytest -p myplugin
Conclusion
We’ve covered a wide range of topics from the basics of pytest, to more advanced features like fixtures, parameterization, mocking, and plugins.
I hope this tutorial has been helpful for you and that it will assist you in writing robust tests for your Python projects.
Remember, good tests are crucial for maintaining high-quality code and catching issues early.
Happy testing!
Mokhtar is the founder of LikeGeeks.com. He is a seasoned technologist and accomplished author, with expertise in Linux system administration and Python development. Since 2010, Mokhtar has built an impressive career, transitioning from system administration to Python development in 2015. His work spans large corporations to freelance clients around the globe. Alongside his technical work, Mokhtar has authored some insightful books in his field. Known for his innovative solutions, meticulous attention to detail, and high-quality work, Mokhtar continually seeks new challenges within the dynamic field of technology.