Posts Understanding pytest-Test Discovery and Naming Conventions
Post
Cancel

Understanding pytest-Test Discovery and Naming Conventions

Understanding How pytest Discovers and Runs Tests in Python

While building my RailOps AI Agent, a Python project that retrieves live UK train information and evaluates delays and cancellations, I started using pytest to test my code.

I had already written several unit tests, but one question came to mind:

How does pytest know where my test files are when I simply run python -m pytest -v?

I never explicitly told pytest which files or functions to execute.

The answer is a feature called test discovery.

In this post, I’ll explain how pytest discovers tests, its default naming conventions, and how to run specific tests.

1. What Is pytest?

pytest is a Python testing framework that helps developers verify whether their code behaves as expected.

For example, in my RailOps project, I created a function that calculates how many minutes a train is delayed.

If a train is scheduled for 15:45 but expected at 15:52, the function should return 7.0.

Instead of manually checking the output every time, I can write a test:

1
2
3
4
5
from app.train_utils import calculate_delay

def test_calculate_delay():
    result = calculate_delay("15:45", "15:52")
    assert result == 7.0

Here, assert checks whether the actual result matches the expected result.

If the condition is True, the test passes. If it is False, pytest reports a failure.

This makes it easier to catch bugs when modifying existing code.

2. How Does pytest Find Test Files Automatically?

When I run:

1
python -m pytest -v

pytest automatically searches for tests using predefined naming conventions.

This process is called test discovery.

By default, pytest recognises Python files that follow either of these patterns:

  • test_*.py
  • *_test.py

For example:

1
2
3
4
test_train_utils.py     ✓
rail_data_test.py       ✓
train_utils.py          ✗
testing_utils.py        ✗

A filename containing the word test is not necessarily enough. It must match one of the supported patterns.

The -v flag stands for verbose, which provides more detailed test results.

3. Do Test Files Have to Be Inside a tests/ Folder?

No.

This was something I initially misunderstood.

Although placing tests inside a tests/ directory is a common convention, pytest does not require it.

For example, my project has the following structure:

1
2
3
4
5
6
7
8
9
10
11
12
railops-ai-agent/
│
├── app/
│   ├── main.py
│   ├── rail_data.py
│   └── train_utils.py
│
├── tests/
│   ├── test_train_utils.py
│   └── test_rail_data.py
│
└── test_example.py

When running pytest from the project root with the default configuration, it can discover tests in both:

1
2
tests/test_train_utils.py
test_example.py

The folder name is not the important part. The filename convention is.

However, keeping tests in a dedicated tests/ directory makes a project easier to maintain.

4. How Does pytest Identify Test Functions?

Discovering a test file is only the first step.

Inside that file, pytest also looks for functions whose names begin with test_.

For example:

1
2
3
4
5
6
7
8
9
10
11
def test_delay():
    assert 7 == 7

def test_cancelled():
    assert True is True

def delay_test():
    assert 7 == 7

def calculate_something():
    return 10

Under the default rules, pytest automatically executes:

1
2
test_delay()
test_cancelled()

It does not collect delay_test() or calculate_something() as standalone tests.

This means the position of test matters.

For ordinary test functions, the name must start with test_.

pytest also supports class-based tests, typically using classes beginning with Test and methods beginning with test_.

These are default conventions and can be customised through pytest configuration.

5. Running All Tests vs. Specific Tests

Another useful thing I learned is that pytest allows different levels of test execution.

Run all discovered tests

1
python -m pytest -v

This discovers and runs tests according to the project’s pytest configuration.

Run tests from a specific file

1
python -m pytest tests/test_rail_data.py -v

This is useful when working on one module and wanting faster feedback.

Run a specific test function

1
python -m pytest tests/test_rail_data.py::test_get_trains_with_mock -v

The :: syntax identifies a particular test within a file.

I find this especially useful when debugging one failing test without repeatedly running the entire suite.

6. How Can I See Which Tests pytest Discovers?

Sometimes I want to check which tests pytest recognises without actually running them.

For that, I can use:

1
python -m pytest --collect-only -q

For example, the output might look like:

1
2
3
4
5
tests/test_rail_data.py::test_get_trains_with_mock
tests/test_rail_data.py::test_get_trains_connection_error
tests/test_rail_data.py::test_get_trains_empty_services

3 tests collected

This command collects tests without executing the test functions.

One thing worth noting is that pytest still imports test modules during collection, so import-related errors can occur even when using --collect-only.

7. How I Used pytest in My RailOps Project

In my RailOps AI Agent, I use pytest to verify different parts of the train information pipeline.

For example:

Delay calculation

Checking that a scheduled time of 15:45 and an estimated time of 15:52 produce a seven-minute delay.

Train status classification

Checking that trains are correctly classified as on_time, delayed, cancelled, or unknown.

API response handling

Using unittest.mock to simulate railway API responses without making real network requests.

Error handling

Simulating connection failures and verifying that the application returns the expected error response.

Empty results

Checking that the application returns an empty list when no train services are available.

These tests help me make changes to the application with more confidence.

8. Key Takeaways

The main thing I learned is that pytest uses naming conventions to automatically discover tests.

ComponentDefault convention
Test directoryNo specific name required
Test filetest_*.py or *_test.py
Test functionStarts with test_
Test classStarts with Test
Test methodStarts with test_

The three commands I want to remember are:

1
2
3
4
5
6
7
8
# Run all tests
python -m pytest -v

# Run one test file
python -m pytest tests/test_rail_data.py -v

# See discovered tests without running them
python -m pytest --collect-only -q

Understanding test discovery helped me realise that pytest is not magically finding my code. It follows a consistent set of conventions that make automated testing simple and scalable.

As I continue developing my RailOps AI Agent, I plan to use pytest not only for checking individual functions but also for testing API integrations and the reliability of the overall agent workflow.


Part of my learning journey in Python, software engineering, and Applied AI development.