There are two types of tests in pulp_core and in the plugins:
Unittests are meant to test the interface of a specific unit utilizing a test database.
Functional tests are meant to test certain workflows utilizing a running instance of pulp.
A pull request that has failing unit or functional tests cannot be merged.
New code is encouraged to have basic unit tests that demonstrate that units (function, method or class instance) are working correctly.
The unit tests for pulpcore are in pulpcore/tests.
Functional tests verify a specific feature. In general functional tests tend to answer the question “As an user can I do this?”
Functional tests for Pulp are written using Pulp Smash . Pulp smash is a test toolkit written to ease the testing of Pulp.
It is highly encouraged to accompany new features with functional tests in pulpcore/functional.
Only the tests for features related to pulpcore should live in this repository.
Functional tests for features related to a specific plugin should live in the plugin repository itself. For example:
Prerequisites for running tests¶
If you want to run the functional tests, you need a running Pulp instance that is allowed to be
mixed up by the tests (in other words, running the tests on a production instance is not
recommended). For example, using the development vm (see Developer Setup),
this can be accomplished by workon pulp; pulpcore-manager runserver 24817. The
pulpcore-manager command is
manage.py configured with the
Functional tests require a valid pulp-smash config file. This can be created with pulp-smash settings create.
When running one of the pulp3-source-* boxes in pulplift, the smash config is already in place. Also, all the services are running. They should be restarted with prestart if any pulp code (not test code) has been changed.
When testing S3 support, you can start and configure a local minio container with pminio.
Pulp functional tests use a set of upstream fixture repositories hosted on fixtures.pulpproject.org. In case you want serve those locally, you can run pfixtures which will execute a nginx container with a copy of those fixtures and configure pulp-smash to use it.
For more info about Pulp development specific helper commands, you can consult phelp.
In case pulp is installed in a virtual environment, activate it first (workon pulp).
All tests of a plugin (or pulpcore itself) are run with pulpcore-manager test <plugin_name>. This involves setting up (and tearing down) the test database, however the functional tests are still performed against the configured pulp instance with its production database.
To only perform the unittests, you can skip the prerequisites and call pulpcore-manager test <plugin_name>.tests.unit.
If you are only interested in functional tests, you can skip the creation of the test database by using pytest <path_to_plugin>/<plugin_name>/tests/functional.
Make sure, the task runners are actually running. In doubt, run prestart or systemctl restart pulpcore-worker@*.
You can be more specific on which tests to run by calling something like pulpcore-manager test pulp_file.tests.unit.test_models or py.test <path_to_plugin>/<plugin_name>/tests/functional/api/test_sync.py.
Contributing to tests¶
A new version of Pulp will only be released when all unit and functional tests are passing.
Contributing test is a great way to ensure that your workflows never regress.