Pull Request Checklist#
- Make sure your change does not break idempotency tests. See Testing (or let CI run the tests for you if you are certain it is idempotent.) If a task cannot be made idempotent, add the tag molecule-idempotence-notest.
- Unless a change is small or doesn't affect users, create an issue on GitHub.
- Add a changelog update.
- Write an excellent Commit Message. Make sure you reference and link to the issue.
- Push your branch to your fork and open a Pull request across forks.
- Add GitHub labels as appropriate.
The tests can be run as they are on GitHub Actions with tox, or they can run with various options using molecule directly.
Install Docker, and add yourself to the group that is authorized to administer containers, and log out and back in to make the permissions change take effect. The authorized group is typically the "docker" group:
gpasswd --add "$(whoami)" docker
NOTE: Docker containers can differ from bare-metal or VM OS installs. They can have different packages installed, they can run different kernels, and so on.
Install collection dependencies
ansible-galaxy collection install -r ./requirements.yml
- Run molecule commands.
Test the default scenario, release-static, on all hosts (linux distros.)
Test a specific scenario.
molecule test --scenario-name source
Test all scenarios on all hosts.
molecule test --all
Use debug for increased verbosity.
molecule --debug test
Create and provision, but don't run tests or destroy.
Explanation of Different Molecule Scenarios#
The molecule scenarios have names like
release-static, which we will refer to as
There is a 3 by 3 matrix of them.
The prefixes are:
release- Install stable release from PyPI
source- Install from git checkout. Different codepaths; preflight check does not exist.
packages- Install from distro (RPM) packages. Even more different codepaths; compatibility checks (preflight) is the job of the repo maintainer.
The suffixes are:
static- Statically specify the roles in the "roles:" syntax in the main playbook, such as in the example playbooks.
cluster- Runs as few roles as possible per host, creating a pulp cluster, rather than a set of independent pulp hosts. Catches any issues. Also, run roles 1 at a time via
tasks:in the main playbook. Catches undeclared dependencies between roles, and other "dynamic" include errors. This covers use cases such as users running a 3rd party role, setting vars, and later running our role.
upgrade- Upgrade an existing Pulp 3.y container, to test upgrading Pulp 3.y. Roles are applied statically. Depends on said containers existing on a registry, and having been built manually (See instructions below))
release-static is symlinked to
default, so that commands like
molecule test will use it.
There are other (intentional) differences between tests:
static- These include using unix sockets for the webserver to connect to pulp-api & pulp-content. The remainder use TCP connections (
upgradebecause that's what older installs only did,
clusterbecause containers communicate via TCP.)
release-upgradescenario uses its own
converge.ymlplaybook instead of the default one used by all other scenarios. This playbook upgrades a Pulp installation multiple times to ensure that all upgrade scenarios work correctly.
source-staticscenario defines paths different than the default for the following variables:
- Misc tests test different distros. This is intended to ensure that many distros are tested, but that tests do not take too long.
In order for
cluster scenarios to work on your local system, you must do the following to enable
firewall-cmd --zone=public --add-masquerade --permanent (assuming your firewall zone is
firewall-cmd --permanent --zone=public --add-rich-rule='rule family=ipv4 source address=172.27.0.0/16 accept'
(assuming your firewall zone is
1. Create /etc/sysctl.d/10-allow_docker_networking.conf with and reboot:
net.bridge.bridge-nf-call-iptables=0 net.bridge.bridge-nf-call-arptables=0 net.bridge.bridge-nf-call-ip6tables=0
To test both webserver solutions we testing
apache as webserver with
All others scenarios use
nginx as a webserver.
pip install mkdocs pymdown-extensions mkdocs-material mike mkdocs-git-revision-date-plugin
NFS & SELinux testing#
Here are example commands for testing the functionality prescribed for the variable
vagrant up --no-destroy-on-error pulp3-sandbox-centos9-stream (hit ctrl-c while ansible-galaxy is running, or early on in the ansible run) vagrant ssh pulp3-sandbox-centos9-stream sudo -i dnf install -y nfs-utils systemctl enable --now nfs-server.service mkdir -p /data/var/lib/pulp/pulpcore_static/ mkdir -p /var/lib/pulp/pulpcore_static/ echo '/data/var/lib/pulp 127.0.0.1(rw,no_root_squash)' >> /etc/exports exportfs -a echo '127.0.0.1:/data/var/lib/pulp /var/lib/pulp nfs defaults,_netdev,nosharecache 0 0' >> /etc/fstab echo '127.0.0.1:/data/var/lib/pulp/pulpcore_static /var/lib/pulp/pulpcore_static,nosharecache nfs defaults,_netdev,context="system_u:object_r:httpd_sys_content_rw_t:s0" 0 0' >> /etc/fstab mount -a exit exit vagrant provision pulp3-sandbox-centos9-stream
Creating Molecule Upgrade Test Containers#
- Identify a version from 3.14 or older (because the release-upgrade tests 1st test upgrading to 3.14 before the current version. This should be revisited in the future.)
- Identify how you will build the old version. Preferably using the old version of the installer. But if necessary, modifying the version variables with the current installer.
- Modify molecule/release-static/molecule.yml to test the desired distro if it is not in it. Add a host_vars if you do this.
- Modify molecule/release-static/group_vars (or host_vars) so that the webserver is nginx rather
than apache, and so that pulp_container is not installed. Also modify the version variables
pulpcore_version) if need be. Install release versions of pulpcore and plugins.
molecule create && molecule converge
- If that failed, be prepared to put workarounds on the branch. Such as modifying
docker exec -it <container-name> /bin/bash
dnf clean all
rm -rf /var/lib/pulp/.cache
docker commit <container-name <tag-name>with a tag like "quay.io/pulp/pulp_installer-pip-ci-f36:3.13.0"
- Modify molecule.yml for release-upgrade and source-upgrade (or packages-upgrade)
- Test the image locally (molecule create -s release-upgrade && molecule converge -s release-upgrade)
docker push <tag-name>
- If this is the 1st image of a new distro, go to https://quay.io/organization/pulp, set the descritpion of the repo based on existing ones, and set the repo to public.
- Create a PR.
Specifying Supported Distros#
We want to support the entire family of RHEL clones, even though we only test CentOS and Rocky in our CI.
We also want to workaround (undocumented) limitation of Ansible 2.9 not recognizing Rocky as a member of the "Red Hat" family.
Therefore, in order to support the entire family of RHEL clones, add logic like the following
when: - __pulp_os_family == 'RedHat - ansible_facts.distribution_major_version|int in [7,8,9,10]