Developer Setup¶

Prerequisites for libvirt¶

Follow these steps:

1. Install vagrant, ansible, and nfs-utils. NFS will be used to share your code directory with the deployed virtual machine:

   $ sudo dnf install ansible nfs-utils vagrant-libvirt
   

2. You will need to grant the nfsnobody user rx access to the folder that you check out your code under. Many developers check out code into $HOME/devel or similar. In Fedora, $HOME typically does not allow such access to other users. If your code is in $HOME, you will need to:

   $ setfacl -m user:nfsnobody:r-x $HOME  # Season to taste, as per above
   

   Warning

   Wherever you have hosted your code, it would be good to verify that nfsnobody is able to read it. If you experience an error message similar to “mount.nfs: access denied by server while mounting 192.168.121.1:/path/to/my/code/dir” during the vagrant up later, it is likely that nfsnobody is being blocked from reading your code directory by either filesystem permissions or SELinux.

3. Start and enable the nfs-server service:

   $ sudo systemctl enable nfs-server && sudo systemctl start nfs-server
   

4. You will need to allow NFS services through your firewall:

   $ sudo firewall-cmd --permanent --add-service=nfs
   $ sudo firewall-cmd --permanent --add-service=rpc-bind
   $ sudo firewall-cmd --permanent --add-service=mountd
   $ sudo firewall-cmd --reload
   

Prerequisites for docker¶

Follow these steps:

1. Install vagrant, ansible, and docker:

   $ sudo dnf install vagrant ansible docker
   

2. Enable and start the docker service:

   $ sudo systemctl enable docker
   $ sudo systemctl start docker
   

Creating the Vagrant environment¶

After preparing either the libvirt or docker prerequisites using the instructions above:

1. You are now prepared to check out the Pulp code into your preferred location. Change directories to that location, and check out the platform:

   $ cd $HOME/devel  # Season to taste
   $ git clone git@github.com:pulp/pulp.git
   

2. Check out the plugins you wish to develop or use as well:

   $ git clone git@github.com:pulp/pulp_deb.git
   $ git clone git@github.com:pulp/pulp_docker.git
   $ git clone git@github.com:pulp/pulp_openstack.git
   $ git clone git@github.com:pulp/pulp_ostree.git
   $ git clone git@github.com:pulp/pulp_puppet.git
   $ git clone git@github.com:pulp/pulp_python.git
   $ git clone git@github.com:pulp/pulp_rpm.git
   

   Note

   It is important to ensure that your repositories are all checked out to compatible versions. If you followed the instructions above, you have checked out master on all repositories which should be compatible.

3. Next, cd into the pulp directory. The Pulp project provides an example Vagrantfile that you can use as a starting point by copying it. After you’ve done that, you can begin provisioning your Vagrant environment. We will finish by running vagrant reload. This allows the machine to reboot after provisioning.:

   $ cd pulp
   $ cp Vagrantfile.example Vagrantfile
   # Choose ONE of the following, for your preferred provider:
   $ vagrant up --provider=libvirt
   $ vagrant up --provider=docker
   $ vagrant reload  # Reboot the machine at the end to apply kernel updates, etc.
   

   Note

   If you want to do a vagrant up without having to enter your sudo password, please follow the instructions mentioned in the ‘Root Privilege Requirement’ section of Vagrant docs.

4. Once you have followed the steps above, you should have a running deployed Pulp development machine. ssh into your Vagrant environment:

   $ vagrant ssh
   

Whenever you connect to your Vagrant environment, you will be greeted by a message of the day that gives you some helpful hints. All of the code is mounted in /home/vagrant/devel. Your development environment has been configured for virtualenvwrapper. If you would like to activate a virtualenv, you can simply type workon <repo_dir> to work on any particular Pulp repo. For example, workon pulp will activate the Pulp platform virtualenv and cd into the code directory for you. You can type workon pulp_rpm for pulp_rpm, workon pulp_python for pulp_python, and so forth. Any plugins in folders that start with pulp_ that you had checked out in your host machine’s code folder alongside the Pulp platform repository should have been installed and configured for virtualenv.

Remote Debugging¶

To use a remote debugger provided by PyCharm, ensure the PyCharm debug egg is installed in the Vagrant environment. This can be done in the Vagrant environment using easy_install so it is available in all virtualenv environments the Vagrantfile sets up.

When SSHing to Vagrant, use a reverse SSH tunnel to allow the Vagrant environment to connect back to your host system where the PyCharm remote debugger is listening. vagrant ssh allows you to specify arbitrary SSH commands using the -- syntax. Assuming a PyCharm remote debugger is listening on port 12345, connect to Vagrant with a reverse tunnel using:

$ vagrant ssh -- -R 12345:localhost:12345

You’ll also need to configure local to remote path mappings to allow PyCharm to treat your host code checkout corresponds with the remote Vagrant code. To do this, edit the PyCharm remote debugger instance and add the following path mapping configuration:

/home/<your_username>/devel=/home/vagrant/devel

Resolving References¶

With Vagrant, Pulp is not installed on your host system preventing PyCharm from knowing an object through static analysis. Practically speaking, this causes all Pulp objects to be shown as an unresolved reference and prevents jumping to the declaration (Ctrl + B).

To resolve this, configure your project with a Vagrant-aware, remote interpreter. In settings, find the ‘Project Interpreter’ area and add a Remote Interpreter. Select ‘Vagrant’ and give it the path to your vagrant file. In my case this is /home/<username>/devel/pulp.

Note

The remote interpreter copies the indexed remote code locally into PyCharm’s cache. Be aware, when you jump to a declaration (Ctrl + B), you are being shown PyCharm’s cached version. For reading code this is fine, but when applying changes, be sure you know if you are editing the actual code or a cached copy.

Source Code¶

Pulp’s code is stored on GitHub. The repositories should be forked into your personal GitHub account where all work will be done. Changes are submitted to the Pulp team through the pull request process outlined in Merging.

Follow the instructions on that site for checking out each repository with the appropriate level of access (Read+Write v. Read-Only). In most cases, Read-Only will be sufficient; contributions will be done through pull requests into the Pulp repositories as described in Merging.

Dependencies¶

The easiest way to download the other dependencies is to install Pulp through yum or dnf, which pulls in the latest dependencies according to the spec file.

1. Download the appropriate repository from https://repos.fedorapeople.org/repos/pulp/pulp/

   Example for Fedora:

   $ cd /etc/yum.repos.d/
   $ sudo wget https://repos.fedorapeople.org/repos/pulp/pulp/fedora-pulp.repo
   

2. Edit the repo and enable the most recent testing repository.

3. When using dnf, install the dependencies with this command. $ sudo dnf install -y $(rpmspec -q --queryformat '[%{REQUIRENAME}\n]' *.spec | grep -v "/.*" | grep -v "python-pulp.* " | grep -v "pulp.*" | uniq)

4. When using yum, install the main Pulp groups to get all of the dependencies. $ sudo yum install @pulp-server-qpid @pulp-admin @pulp-consumer

5. When using yum, remove the installed Pulp RPMs; these will be replaced with running directly from the checked out code. $ sudo yum remove pulp-\* python-pulp\*

6. Install some additional dependencies for development:

   $ sudo yum install python-setuptools redhat-lsb mongodb mongodb-server \
   qpid-cpp-server qpid-cpp-server-store python-qpid-qmf python-nose \
   python-mock python-paste python-pip python-flake8
   

The only caveat to this approach is that these dependencies will need to be maintained after this initial setup. Leaving the testing builds repository enabled will cause them to be automatically updated on subsequent yum update calls. Messages are sent to the Pulp mailing list when these dependencies are updated as well to serve as a reminder to update before the next code update.

Installation¶

Pulp can be installed to run directly from the checked out code base through setup.py scripts. Running these scripts requires the python-setuptools package to be installed. Additionally, it is also recommended to install python-pip for access to additional setup-related features.

This method of installation links the git repositories as the locally deployed libraries and scripts. Any changes made in the working copy will be immediately deployed in the site-packages libraries and installed scripts. Setup scripts are automatically run for you by pulp-dev.py.

Additionally, Pulp specific files such as configuration and package directories must be linked to the checked out code base. These additions are performed by the pulp-dev.py script located in the root of each git repository. The full command is:

$ sudo python ./pulp-dev.py -I

Uninstallation¶

The pulp-dev.py script has an uninstall option that will remove the symlinks from the system into the local source directory, as well as the Python packages. It is run using the -U flag:

$ sudo python ./pulp-dev.py -U

Permissions¶

The pulp-dev.py script links Pulp’s WSGI application into the checked out code base. In many cases, Apache will not have the required permissions to serve the applications (for instance, if the code is checked out into a user’s home directory).

One solution, if your system supports it, is to use ACLs to grant Apache the required permissions.

For example, assuming the Pulp source was checked out to ~/code/pulp, the following series of commands would grant Apache the required access:

$ cd $HOME
$ setfacl -m user:apache:rwx .
$ cd code
$ setfacl -m user:apache:rwx .
$ cd pulp
$ setfacl -m user:apache:rwx .

SELinux¶

Unfortunately, when developing Pulp SELinux needs to be disabled or run in Permissive mode. Most development environments will be created with pulp-dev.py, which deploys Pulp onto the system differently than a rpm based install. The SELinux policy of Pulp expects an RPM layout, and if SELinux is run in Enforcing mode your development to not function correctly.

To turn off SELinux, you can use sudo setenforce 0 which will set SELinux to permissive. By default, SELinux will be enabled on the next restart so make the change persistent by editing /etc/sysconfig/selinux.

SELINUX=permissive

mod_python¶

Pulp is a mod_wsgi application. The mod_wsgi and mod_python modules can not both be loaded into Apache at the same time as they conflict in odd ways. Either uninstall mod_python before starting Pulp or make sure the mod_python module is not loaded in the Apache config.

Start Pulp and Related Services¶

The instructions below are written to be a simple process to start pulp. You should read the user docs for more information on each of these services. Systemd shown below,see user docs for upstart commands.

Start the broker (Though qpid shown here, it is not your only option):

sudo systemctl start qpidd

Start the agent:

sudo systemctl start goferd

Install a plugin (the server requires at least one to start):

git clone https://github.com/pulp/pulp_rpm.git
cd pulp_rpm
sudo ./manage_setup_pys.sh develop
sudo python ./pulp-dev.py -I

Initialize the database:

sudo systemctl start mongod
sudo -u apache pulp-manage-db

Start the server:

sudo systemctl start httpd

Start pulp services:

sudo systemctl start pulp_workers
sudo systemctl start pulp_celerybeat
sudo systemctl start pulp_resource_manager

Login:

pulp-admin login -u admin

The default password is admin

Uninstallation¶

The pulp-dev.py script has an uninstall option that will remove the symlinks from the system into the local source directory. It is run using the -U flag:

$ sudo python ./pulp-dev.py -U

Each python package installed above must be removed by its package name.:

$ sudo pip uninstall <package name>