Pulp RPM Support¶
User Guide¶
Introduction¶
RPM Support for Pulp allows you to create and publish repositories of RPM packages (including RPM, SRPM, DRPM, errata, distributions, etc.).
- Automatically retrieve packages from external repositories and store them in local Pulp repositories, which are hosted by the Pulp server.
- Upload your own packages and errata into local Pulp repositories.
- Copy packages and errata from one local repository to another, enabling you to promote testing versions to a production repository.
- Push packages out to large numbers of consumers.
- Track from the server what packages are installed on each consumer.
How to Use This Guide¶
This guide documents features and concepts that are specific to RPM support. The Pulp User Guide (available here) has much more information about how to perform common operations like search repositories, copy packages from one repository to another, etc. As such, the Pulp User Guide should be used in conjunction with this guide.
You will also find that the pulp-admin
and pulp-consumer
command line
utilities have thorough help text available by appending --help
to any command
or section.
Release Notes¶
Contents:
Pulp 2.8 Release Notes¶
Pulp 2.8.0¶
New Behaviors¶
Repo sync cancellation now exists immediately. If the syncronization of the repo is cancelled, the worker process exits immediately with sys.exit(). A new worker process is created immediately, so further tasks are normally picked up and executed.
New Features¶
- The default Apache configuration has changed in order to support deferred content
downloading. All requests to
/pulp/repos
are now re-written to/pulp/content
.
Database Changes¶
The 2.8.0 release comes with strong validation enabled at the database layer. This required some database changes to be made. These changes should go unnoticed for most users, however, for those users integrating with Pulp in a deep way, this may affect you. This section recaps known changes.
The following collections and fields have had some data quality fixes applied to avoid validation issues.
Collection | Field Name | Data Quality Fix |
---|---|---|
units_package_category | translated_name | Previous values of “” are now {} |
units_package_category | translated_description | Previous values of “” are now {} |
units_package_environment | translated_name | Previous values of “” are now {} |
units_package_environment | translated_description | Previous values of “” are now {} |
units_package_group | translated_name | Previous values of “” are now {} |
units_package_group | translated_description | Previous values of “” are now {} |
units_erratum | pushcount | All int and floats converted to strings. All null values are unset. |
The database has renamed some field names for RPM unit models. This change should not be noticeable through the API, but it does come with a migration which drops some indexes. The following field names are renamed:
Collection | Old Field Name | New Field Name |
---|---|---|
units_distribution | id | distribution_id |
units_erratum | from | errata_from |
units_erratum | id | errata_id |
units_package_group | id | package_group_id |
units_package_category | id | package_category_id |
units_package_environment | id | package_environment_id |
The following indexes have been dropped:
Collection | Index Name |
---|---|
units_distribution | id_1 |
units_distribution | id_1_family_1_variant_1_version_1_arch_1 |
units_erratum | id_1 |
units_package_group | id_1 |
units_package_group | id_1_repo_id_1 |
units_package_category | id_1 |
units_package_category | id_1_repo_id_1 |
units_package_environment | id_1 |
units_package_environment | id_1_repo_id_1 |
Several indexes have been created, check your db to see what indexes are in place.
Pulp 2.7 Release Notes¶
Pulp 2.7.0¶
New Features¶
- The repo authentication functionality associated with pulp_rpm has been moved into platform. This allows other plugins to take advantage of this functionality.
- The RPM plugin supports basic auth when fetching data from upstream repos.
This can be configured via
basicauth-user
andbasicauth-pass
when creating or updating a repository. - The export distributor can optionally create a PULP_MANIFEST file in the directory where
ISOs are generated. This allows the pulp ISO importer to directly import the published product.
See the help text for
pulp-admin rpm repo sync run
, or the distributor documentation for details. - Comps.xml file can now be uploaded into pulp via CLI or API. The unit_type_id to use for import
of the content is
package_group
orpackage_category
. - The RPM plugin supports mirrorlist urls when syncing repo. This can be specified at repo creation
step via
feed
option.
Pulp 2.6 Release Notes¶
Pulp 2.6.2¶
New Behaviors¶
Install errata tasks will fail during unit translation when the consumer does not have an RPM profile.
The sync process checks the revision number in upstream metadata to determine if it has changed since the previous sync. If not, most steps in the sync will be skipped. On large repositories, this has reduced best-case sync times from an order of minutes to a few seconds.
Bug Fixes¶
See the list of bugs fixed in 2.6.2
Pulp 2.6.1¶
Bug Fixes¶
See the list of bugs fixed in 2.6.1
Pulp 2.6.0¶
New Behaviors¶
When doing a sync
operation, if the upstream “distribution” unit has not
changed since the last sync, pulp will no longer attempt to download additional
distribution-related files.
Bug Fixes¶
See the list of bugs fixed in 2.6.0
RHBZ #1153378 was addressed in 2.6.0. If
you have older (circa 2009) yum clients that fail in a way similar to what is described
in this bug, you may want to temporarily
re-enable SSLInsecureRenegotation
under /etc/httpd/conf.d/pulp_rpm.conf
until your client
systems have been updated.
Pulp RPM 2.5 Release Notes¶
Pulp RPM 2.5.2¶
New Features¶
The generation of sqlite files is now done with createrepo_c instead of createrepo.
Pulp RPM 2.5.1¶
This is an important bugfix release that contains the fix for #1165355 and #1171509.
Pulp RPM 2.5.0¶
New Features¶
The Alternate Content Source feature now works during an rpm sync.
Bug Fixes¶
You can see the list of bugs that were fixed.
Pulp RPM 2.4 Release Notes¶
Pulp RPM 2.4.4¶
This release requires the application of a database migration due to #1165355. In that bug report, it was learned that Pulp considered “sha” and “sha1” to be different checksums. As a result, some users have experienced duplicate packages. This data migration changes all “sha” checksums to “sha1”, and will remove packages with “sha” checksums if they are found to be duplicates of existing “sha1” packages.
The 2.4.4 release is a minor bugfix release. You can see the list of bugs that were fixed here.
Please see the Pulp Platform upgrade instructions for information on how to perform the upgrade.
Pulp RPM 2.4.3¶
No changes were introduced in the Pulp RPM 2.4.3 release. The Pulp build system requires Pulp RPM to match versions with Pulp server, which did have an update.
Pulp RPM 2.4.2¶
Pulp RPM 2.4.1¶
New Features¶
The 2.4.1 release drops the need for Pulp’s custom build of M2Crypto. You should remove the Pulp version of M2Crypto from your machine and downgrade to the version supplied by your operating system. As a result of this, the per-repo client certificate checks that are performed by our repository authorization system are inefficient, and you may notice degradation in performance if you are using this feature. The Pulp team has provided a workaround, but it can only be used if you don’t wish to use different CAs on different repositories. See the upgrade instructions below.
Bug Fixes¶
The 2.4.1 release is a minor bugfix release. You can see the list of bugs that were fixed here.
Upgrade Instructions¶
If you have the custom Pulp-provided M2Crypto package installed, you should downgrade to the M2Crypto package that is provided by your operating system:
$ sudo yum downgrade m2crypto
If you are using the repository protection feature and if you do not require different certificate
authorities on each repository, it is recommended that you configure your web browser to validate
client certificates against trusted certificate authorities instead of having Pulp do it. For
Apache, please see their documentation if
you wish to learn how to do this. You can set the new verify_ssl
setting to false
in
the [main]]
section of /etc/pulp/repo_auth.conf
if you wish to configure Pulp not to check
the certificate signatures. There is a performance advantage to configuring this setting this way if
you are able to use your web server to validate client certificates instead of Pulp.
Pulp RPM 2.4.0¶
New Features¶
- When a pulp_manifest.xml is added to kickstart repositories all of the additional files listed in the manifest will be downloaded as part of the repo. The command line utility available at pulp_rpm/playpen/yum_distributor/generate_distribution_manifest.py can be used to help with creating the manifest file. This file should be stored as a peer of the treeinfo file in a source repository that is being synced by Pulp.
- Added the ability to generate sqlite repository metadata files as part of an RPM repo publish.
API Changes¶
- The
generate_sqlite
metadata flag was added to the yum_distributor distributor type. Enabling that flag will turn on generation of sqlite repo metadata files. This is not enabled by default.- Removed the
use_createrepo
in favor of the more precisegenerate_sqlite
flag
When uploading RPMs, the unit key and metadata no longer need to be extracted client-side and passed to the server. The server will inspect the RPM and extract the necessary information.
Yum metadata files can now be uploaded to a repository. Information on valid metadata attributes can be found in the technical reference.
Upgrade Instructions¶
- pulp_rpm has added a dependency on python-lxml for xml parsing
Please see the Pulp Platform upgrade instructions for information on how to complete the upgrade.
Known Issues¶
There is a bug where if you have migrated from Pulp 2.0 or earlier of to any version prior to 2.4
and are performing a second migration to 2.4 or greater you may encounter an error that
Applying migration pulp_rpm.plugins.migrations.0003_custom_metadata_on_scratchpad failed
.
This is caused because of old data that is no longer needed.
The following command can be used to remove the data that is no longer needed:
find /var/lib/pulp/working/repos/ -path '*/yum_importer' -type d -exec rm -rf '{}' \;
Pulp 2.3 Release Notes¶
Pulp 2.3.0¶
New Features¶
- Users can now export repositories to a series of ISOs or to a location on disk that is convenient for transfer to another system.
- The consumer applicability API is vastly different and performs much faster. Please see the platform developer guide for details on the new API.
- The yum distributor now creates “listing” files in the parent directories of published yum repositories, all the way up the tree to the base publish path. These “listing” files are used by tools like Katello and Red Hat’s subscription-manager to discover available repositories within a particular path.
Client Changes¶
- The command
pulp-admin iso repo content
was renamed topulp-admin iso repo isos
Notable Bug Fixes¶
- In some cases, repo groups were displayed with a translated name only. This is fixed in 2.3, but a re-sync is required of any affected repository.
- Consumers are now notified when the relative path of a bound repository is changed.
- It is now far more intuitive
to update all packages on a consumer from
pulp-admin
.
All Bugs¶
You can see the complete list of over 100 bugs that were fixed in Pulp 2.3.0.
Known Issues¶
Fedora 19 repositories have a new data type called an “environment group” that Pulp does not yet support. As such, you cannot use Anaconda to install Fedora 19 from a repository published by Pulp.
Pulp 2.3.1¶
Bugs Fixed¶
- SRPM copy is now supported
by
pulp-admin
. - REST API clients could
see incorrect behavior
if using a
distributor_id
value different from thedistributor_type_id
when adding a distributor to a repository. This issue did not affect users who only usepulp-admin
to interact with the REST API.
Pulp 2.2 Release Notes¶
Pulp 2.2.0¶
New Features¶
- The yum importer is all-new. It performs sync operations in less time, and it uses much less RAM than in the past.
- The ISO Distributor now supports repository protection. See the API documentation for more details.
- The ISO Importer now allows ISO uploads.
- The ISO Importer now allows synchronization cancellation.
CLI Changes¶
- The
pulp-admin rpm repo uploads rpm
command now supports a flag (--skip-existing
) to skip re-uploading packages that are already in the destination repository. - The
pulp-admin rpm repo copy
command now has a--all
flag. - Many commands that used to display a task ID will now automatically poll the server and display progress until the task is complete.
- There is a new ISO section in the pulp-admin client. It currently allows syncing, creating, and updating repositories.
API Changes¶
Some of the configuration names for the yum importer have changed. See the table below for the mapping of old names to new names.
Old name | New Name |
---|---|
feed_url | feed |
ssl_verify | ssl_validation |
num_threads | max_downloads |
proxy_url | proxy_host |
proxy_user | proxy_username |
proxy_pass | proxy_password |
verify_checksum | validate |
remove_old | remove_missing |
num_old_packages | retain_old_count |
The following configuration values are no longer supported.
newest
- The idea of only downloading the newest RPM is handled by the
retain_old_count
value. verify_size
- Verifying the size and checksum have been consolidated into a single parameter called
validate
. During migration, the value forvalidate
will be set on existing repositories using theverify_checksum
value.
All existing yum importers will be migrated by pulp-manage-db, but any non-Pulp code that creates or updates yum importers with the API will need to be updated to use these new names.
Some of the configuration names for the ISO importer have changed. See the table below for the mapping of old names to new names.
Old name | New Name |
---|---|
feed_url | feed |
num_threads | max_downloads |
proxy_url | proxy_host |
proxy_user | proxy_username |
remove_missing_units | remove_missing |
validate_units | validate |
All existing ISO importers will be migrated by pulp-manage-db, but any non-Pulp code that creates or updates ISO importers with the API will need to be updated to use these new names.
Upgrade Instructions¶
Please see the Pulp Platform upgrade instructions for information on how to complete the upgrade.
The location of the global configuration file for the yum importer has changed. Any
custom changes to the pre-2.2.0 file
(/usr/lib/pulp/plugins/importers/yum_importer/yum_importer.conf
), such as those
suggested when configuring a proxy, are removed during upgrade.
The new location of this file is /etc/pulp/server/plugins.conf.d/yum_importer.json
.
Keep in mind the configuration property name changes above when re-adding proxy
information.
Pulp 2.2.1¶
Multiple proxy-related issues related to authentication and HTTPS to the proxy were fixed in RHBZ #1022662 and RHBZ #1014368.
A version comparison bug that caused recursive copies to not copy all dependencies was fixed.
A race condition with XML namespace parsing was fixed.
Several ISO-related bugs were fixed. They can be seen in the list of all fixed bugs.
Pulp 2.1 Release Notes¶
Pulp 2.1.1¶
Pulp 2.1.1 is a bugfix release that also comes with a few performance improvements.
Changes¶
A new config option may be passed when making a copy request between two
repositories. If copy_children=False
is passed as part of the override
config when copying errata, groups, or categories, then the child packages of
those units will not be copied. This saves time when copying an entire
repository, because for example, you can tell the group copy operation not to
worry about finding and retrieving the RPMs referenced by each group.
Noteworthy Bugs Fixed¶
RHBZ #873313 - Very high memory usage during repo sync
RHBZ #947208 - RFE: Add proxy server support to the ISO importer
RHBZ #949004 - The ISO Importer does not build the URL for the PULP_MANIFEST correctly
RHBZ #949008 - The ISO importer set the SSL_VERIFY_HOST value to 1, when it should be 2
RHBZ #953665 - Copying large repo uses tons of RAM and takes too long
Upgrade Instructions¶
Please see the Pulp Platform upgrade instructions for information on how to complete the upgrade.
Pulp 2.1.0¶
New Features¶
- Pulp 2.1 now supports Fedora 18 and Apache 2.4.
CLI Changes¶
- The pulp-consumer bind and unbind operations have been moved out of the Pulp project into this project. These operations can now be found under pulp-consumer rpm {bind,unbind}.
- The
pulp-admin rpm consumer [list, search, update, unregister, history]
commands from this project have been moved into the Pulp project, and can now be found underpulp-admin consumer *
.
Export Distributor Publishing Path¶
The export distributor now published to a different absolute URL path than it did in Pulp 2.0. Previously, the
exported ISOs were published on the Pulp server under /pulp/isos/
. They will now be published under
/pulp/exports/
. It is the user’s responsibility to move any ISOs they have exported out of
/pulp/isos/
before upgrading. This will be covered in the Upgrade Instructions.
Upgrade Instructions¶
Before upgrading, we will need to migrate any exported ISOs that were created using the export distributor to
their new location for Pulp 2.1. These ISOs can be in two different places in your filesystem, depending on
whether they were published over HTTP or HTTPS, or both. The HTTP published ISOs will be found in
/var/lib/pulp/published/http/isos/
. If there are any files or folders in that location, you can move them to
their new location, or remove them if you do not need them anymore. This command will move them:
$ sudo mv /var/lib/pulp/published/http/isos /var/lib/pulp/published/http/exports
Similarly, ISOs published over HTTPS will be found in /var/lib/pulp/published/https/isos/
. If you do not
wish to remove them, you can move them with this command:
$ sudo mv /var/lib/pulp/published/https/isos /var/lib/pulp/published/https/exports
Please see the Pulp Platform upgrade instructions for information on how to complete the upgrade.
Installation¶
Note
If you followed the Pulp installation instructions you already have RPM features installed. If not, this document will walk you through the installation.
Prerequisites¶
The only requirement is to meet the prerequisites of the Pulp Platform. Please see the `Pulp User Guide`_ for prerequisites including repository setup. Also reference that document to learn more about what each of the following components are for.
Server¶
If you followed the Pulp User Guide install instructions, you already have RPM support installed. If not, follow these steps.
Consider stopping Pulp. If you need Apache to keep running other web apps, or if you need Pulp to continue serving static content, it is usually sufficient to disable access to Pulp’s REST API. That will be left as an exercise for the reader. Otherwise, just stop Apache:
$ sudo apachectl stop
Next, install the package.
$ sudo yum install pulp-rpm-plugins
Then run pulp-manage-db
to initialize the new types in Pulp’s database.
$ sudo pulp-manage-db
Finally, restart Apache.
$ sudo apachectl restart
Admin Client¶
If you followed the Pulp User Guide install instructions, you already have RPM support installed. If not, just install the following package.
$ sudo yum install pulp-rpm-admin-extensions
Consumer Client¶
If you followed the Pulp User Guide install instructions, you already have RPM support installed. If not, just install the following package.
$ sudo yum install pulp-rpm-consumer-extensions
Agent¶
If you followed the Pulp User Guide install instructions, you already have RPM support installed. If not, just install the following package.
$ sudo yum install pulp-rpm-handlers
Then restart the Pulp agent.
$ sudo service pulp-agent restart
Configuration¶
Yum Importer Configuration¶
The yum importer is configured by editing
/etc/pulp/server/plugins.conf.d/yum_importer.json
. This file must be valid JSON.
The importer supports the settings documented in Pulp’s importer config docs.
ISO Importer Configuration¶
The ISO importer is configured by editing
/etc/pulp/server/plugins.conf.d/iso_importer.json
. This file must be valid JSON.
The importer supports the settings documented in Pulp’s importer config docs.
Protected Repositories¶
Repository authentication allows the creation of protected repositories in the Pulp server. Consumers attempting to access protected repositories with yum operations require some form of authentication in order to be granted access.
Two configuration file changes are necessary to enable repository authentication.
- Edit
/etc/pulp/server.conf
and set thessl_ca_certificate
option to the full path of the CA certificate that signed the Pulp server’s httpd SSL certificate. If this option is not set, it will default to/etc/pki/pulp/ssl_ca.crt
. This file must be readable by the apache user.
Note
If the default self signed certificate that is generated when mod_ssl
is installed is being used as the Pulp server’s certificate, copying that certificate
to /etc/pki/pulp/ssl_ca.crt
and making it apache readable will suffice.
The default location for that certificate is /etc/pki/tls/certs/localhost.crt
or /etc/pki/tls/certs/<hostname>.crt
.
- Edit
/etc/pulp/repo_auth.conf
and set theenabled
option totrue
. Save the file and restart Apache.
Validation With Your Web Server¶
If you are using the repository protection feature and if you do not require different certificate
authorities on each repository, it is recommended that you configure your web server to validate
client certificates against trusted certificate authorities instead of having Pulp do it. For
Apache, please see their documentation if
you wish to learn how to do this. You can set the new verify_ssl
setting to false
in
the [main]]
section of /etc/pulp/repo_auth.conf
if you wish to configure Pulp not to check
the certificate signatures. There is a performance advantage to configuring this setting this way if
you are able to use your web server to validate client certificates instead of Pulp.
Global Repo Authentication¶
Repository authentication may be configured globally for all repositories in the Pulp server or individually on a per repo basis. In the event that both are specified, only the individual repository authentication check will take place.
Global repository authentication is enabled by placing the authentication
credentials under /etc/pki/pulp/content/
. The following files are relevant:
pulp-global-repo.ca
- CA certificate used to validate inbound consumer certificates. If the consumer’s certificate cannot be validated by this CA, the consumer is automatically rejected as being unauthorized.
pulp-global-repo.cert
- Certificate to provide to consumers when they bind to repositories. If a repository overrides global repository authentication at the repository level, the certificate provided for the repository itself is used in place of this file. This file is optional; if unspecified, bound consumers will need to acquire a valid certificate for accessing the repository through other means.
pulp-global-repo.key
- If the private key for the consumer certificate above is not included in the certificate itself, it may be located in this file and will be sent to bound consumers at the same time as the certificate.
Individual Repository Authentication¶
Individual repositories can be setup to do SSL authentication. This allows you to use authentication on only specific repositories while leaving others unprotected, or to have different credentials for some repositories than others.
The three certificates listed above can be passed to the repository create
or update
command using the following options respectively:
--feed-ca-cert
--feed-cert
--feed-key
Quick Start¶
Sync and Publish a Repo¶
The following command creates a new repository and sets its upstream feed URL to that of the Pulp Project’s own repository. When we later run a synchronize operation, the contents of the remote repository will be downloaded and stored in our new repository.
$ pulp-admin rpm repo create --repo-id=zoo --relative-url=zoo \
--feed=http://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/
Successfully created repository [pulp]
--repo-id
is required and must be unique.--relative-url
is optional and was used here to make the path to the repository friendlier.--feed
is only required if you want to sync content from an external yum repository, which in this case we do.
Now let’s sync the repository, which downloads all of the packages from the remote repository and stores them in our new repository.
$ pulp-admin rpm repo sync run --repo-id=zoo
+----------------------------------------------------------------------+
Synchronizing Repository [zoo]
+----------------------------------------------------------------------+
This command may be exited via ctrl+c without affecting the request.
Downloading metadata...
[\]
... completed
Downloading repository content...
[==================================================] 100%
RPMs: 0/0 items
Delta RPMs: 0/0 items
... completed
Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed
Importing errata...
[-]
... completed
Importing package groups/categories...
[-]
... completed
Task Succeeded
Initializing repo metadata
[-]
... completed
Publishing Distribution files
[-]
... completed
Publishing RPMs
[==================================================] 100%
32 of 32 items
... completed
Publishing Delta RPMs
... skipped
Publishing Errata
[==================================================] 100%
4 of 4 items
... completed
Publishing Comps file
[==================================================] 100%
3 of 3 items
... completed
Publishing Metadata.
[-]
... completed
Closing repo metadata
[-]
... completed
Generating sqlite files
... skipped
Publishing files to web
[-]
... completed
Writing Listings File
[-]
... completed
Task Succeeded
Your repository is now available to browse at https://localhost/pulp/repos/pulp_beta/. (adjust the hostname as necessary)
Consumer Setup and Use¶
On a Pulp consumer, once you have completed the installation process, the next step is to register with the Pulp server. This allows the server to track what is installed on the consumer and initiate actions on the consumer, such as package install and system reboot.
Note
You must use login credentials for this command. Also note that this command must be run with root privileges.
$ sudo pulp-consumer register --consumer-id=con1
Enter password:
Consumer [con1] successfully registered
Now we can proceed with binding to a specific repository. Binding causes the Pulp
repository to be setup on the consumer as a normal yum repository. Bound repositories
are added to the file /etc/yum.repos.d/pulp.repo
. Binding also allows the
server to initiate the installation of packages from that repository onto the
consumer. In this case, repository “zoo” has already been created on the Pulp
server and contains packages.
$ pulp-consumer rpm bind --repo-id=zoo
Bind tasks successfully created:
Task Id: 44d64951-857a-4985-bfd9-dd6ead841065
Task Id: 14782cfa-bdb7-4307-b2b1-f1a0b4331d66
Note
The binding request is asynchronous and does not complete until the server has responded with binding information. This is why you see task IDs in the output above. That said, it happens very quickly and will almost certainly be done before you can type your next command.
At this point, the consumer is ready to install packages from the “zoo” repository. Let’s initiate a package install from the server.
$ pulp-admin rpm consumer package install run --consumer-id=con1 -n wolf
Install task created with id [ c89d4578-cb4e-451f-a87a-63272e77670e ]
This command may be exited via ctrl+c without affecting the request.
Refresh Repository Metadata [ OK ]
Downloading Packages [ OK ]
Check Package Signatures [ OK ]
Running Test Transaction [ OK ]
Running Transaction [ OK ]
Install Succeeded
+----------------------------------------------------------------------+
Installed
+----------------------------------------------------------------------+
Name: wolf
Version: 9.4
Arch: noarch
Repoid: zoo
Now the package “wolf” is installed on the consumer, and you can verify this by
running yum info wolf
on the consumer.
Next Steps¶
This guide documents features and concepts that are specific to RPM support. The Pulp User Guide has much more information about how to perform common operations like search repositories, copy packages from one repository to another, etc.
Please check out the Recipes section for more advanced use cases.
Recipes¶
Mirror a Remote Repository¶
This is an example of creating a local mirror of a remote repository. In this case, we will mirror the Foreman repository.
$ pulp-admin rpm repo create --repo-id=foreman --feed=http://yum.theforeman.org/releases/1.1/el6/x86_64/ --relative-url=foreman
Successfully created repository [foreman]
--repo-id
is required and must be unique.--relative-url
is optional and was used here to make the path to the repository friendlier.--feed
is only required if you want to sync content from an external yum repository, which in this case we do.
$ pulp-admin rpm repo sync run --repo-id=foreman
+----------------------------------------------------------------------+
Synchronizing Repository [foreman]
+----------------------------------------------------------------------+
This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.
Downloading metadata...
[/]
... completed
Downloading repository content...
[==================================================] 100%
RPMs: 98/98 items
Delta RPMs: 0/0 items
... completed
Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed
Importing errata...
[-]
... completed
Importing package groups/categories...
[-]
... completed
Publishing packages...
[==================================================] 100%
Packages: 98/98 items
... completed
Publishing distributions...
[==================================================] 100%
Distributions: 0/0 items
... completed
Generating metadata
[\]
... completed
Publishing repository over HTTPS
[\]
... completed
- The full URL to a published repository is derived from the following information:
- The server name of the Pulp server. This should be the same hostname used in the
CN of the server’s SSL certificate, otherwise SSL verification on the client
will likely fail. The configuration of that certificate is handled by Apache,
typically in the
ssl.conf
configuration file. - The Apache alias at which Pulp RPM repositories are hosted. This value is set
in the
pulp_rpm.conf
file located in Apache’sconf.d
directory. By default, this is set to/pulp/repos
. - The relative URL of the repository being published. This can be explicitly set when the repository is created in Pulp. If it is not explicitly set, Pulp will derive this value using the relative URL of the repository’s feed. For feedless repositories, the repository ID is used.
- The server name of the Pulp server. This should be the same hostname used in the
CN of the server’s SSL certificate, otherwise SSL verification on the client
will likely fail. The configuration of that certificate is handled by Apache,
typically in the
Applying these rules to the above example repository, the published repository can be found (adjusting the hostname as necessary) at: https://localhost/pulp/repos/foreman/.
Had the relative URL not been explicitly set in the repository, the hosted URL
would be:
https://localhost/pulp/repos/releases/1.1/el6/x86_64/
.
To keep the repository current, it might be desirable to synchronize it on a regular schedule. The following command sets a schedule of synchronizing once a day.
$ pulp-admin rpm repo sync schedules create -s '2012-12-15T00:00Z/P1D' --repo-id=foreman
Schedule successfully created
Use a Proxy¶
Using a web proxy is fairly straight-forward. Proxy details are specified when creating the repository, as in this example:
$ pulp-admin rpm repo create --repo-id=foo --proxy-host=http://bar.net \
--proxy-port=1234 --proxy-user=me --proxy-pass=letmein \
--feed=http://bar.net/repos/foo/
Successfully created repository [foo]
Warning
The password is stored in clear text and may be presented in clear text by the command line utility. Do not use sensitive credentials for your web proxy.
Alternatively, Pulp can be configured to use a specific proxy for all yum
repositories by adding the following settings to
/etc/pulp/server/plugins.conf.d/yum_importer.json
{
"proxy_host" : "<url>",
"proxy_port" : <port>,
"proxy_username" : "<username>",
"proxy_password" : "<password>"
}
Note
This is a JSON file, so care must be taken when editing it.
Sync a Protected Repo¶
Syncing against a protected repository requires specifying some SSL certificates.
The pulp-admin rpm repo create
command does a good job of documenting these
options, but the below example may help pull it all together.
This example was run on a RHEL6 server with an active subscription.
Note that you will need to adjust the file names for the certificate and key in
/etc/pki/
to match your own. Also note that this needs to run as root to
have permission to read the certificates and key.
$ sudo pulp-admin rpm repo create --repo-id=rhel-6-server \
--feed=https://cdn.redhat.com/content/dist/rhel/server/6/6Server/x86_64/os \
--feed-ca-cert=/etc/rhsm/ca/redhat-uep.pem --feed-cert=/etc/pki/entitlement/8435737662014631983.pem \
--feed-key=/etc/pki/entitlement/8435737662014631983-key.pem
Successfully created repository [rhel6server]
$ pulp-admin rpm repo sync run --repo-id=rhel6server
+----------------------------------------------------------------------+
Synchronizing Repository [rhel6server]
+----------------------------------------------------------------------+
This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.
Downloading metadata...
[/]
... completed
Downloading repository content...
[ ] 1%
RPMs: 91/8769 items
Delta RPMs: 0/0 items
If you would prefer not to use the entitlement certificates from an existing
RHEL installation, you can also acquire the entitlement certificate, key, and
CA certificate using
the Red Hat Customer Portal. To retrieve the entitlement certificate and key,
you will need to view your
Registered Consumers. On
that page, there is a “Systems” tab, and in that tab there is a link to
Register a system.
Fill out the form with the relevant details for your Pulp Server, and click
“Register”. Once you have registered your system, you must now attach a
subscription to it with the “Attach a subscription” link on the page for the
newly registered system. In the pop up, select the subscriptions that you want
to apply to the Pulp Server and click “Attach selected”. You will now see the
selected subscriptions in the “Attached Subscriptions” table, and you can use
the “Download” link from the “Entitlement Certificate” column to retrieve the
certificate and key, bundled into a single file. You can pass that same file as
the --feed-cert
and --feed-key
options when you create the repo.
It is also possible to sync a repo that is protected via basic authentication.
The --basicauth-user
and --basicauth-pass
options are used for this
during repo creation or update.
Export Repositories and Repository Groups¶
If you have a Pulp server that does not have access to the Internet, it is possible
to use a second Pulp server, which does have Internet access, to retrieve repositories and
repository updates for your disconnected server. The full list of options can be seen by
running pulp-admin rpm repo export run --help
.
The general workflow is as follows:
- Use the connected Pulp server to sync one or more repositories.
- Export these repositories to ISOs:
pulp-admin rpm repo export run --repo-id=demo-repo
$ pulp-admin rpm repo export run --repo-id=demo-repo
+----------------------------------------------------------------------+
Publishing Repository [demo-repo]
+----------------------------------------------------------------------+
This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.
Which, if publishing over HTTP, could be found at http://localhost/pulp/exports/repo/demo-repo/ (adjust hostname and repo-id as necessary.)
- Transport the ISOs to the disconnected Pulp server
- Mount each ISO and copy its contents to a directory on the disconnected Pulp server
$ cp -r /path/to/mounted/iso1/ /path/to/extracted/content
$ cp -r /path/to/mounted/iso2/ /path/to/extracted/content
- On the disconnected Pulp server, create a new repository with the feed pointing at
the directory containing the ISO contents:
pulp-admin rpm repo create --repo-id=demo-repo --feed=file:///path/to/extracted/content/
- Sync the repository using
pulp-admin rpm repo sync run --repo-id=demo-repo
The workflow for exporting repository groups is quite similar. The command is
pulp-admin rpm repo group export run
. Repository groups can contain any content type,
but this command will only export the yum repositories.
It is also possible to export all rpms and errata associated with a repository in a given
time frame using the --start-date
and --end-date
options. This is helpful if you have
already exported the repository and would like to only export updates. Be aware that since this
does not export package groups or categories, any updates to these will not be reflected on the
disconnected Pulp server. There is currently no support in the pulp-admin command-line utility
for uploading these incremental updates back into Pulp; you must use the REST API for these uploads.
Warning
It is very important keep track of the last time you performed an incremental export. If you fail use the correct date range, some dependencies may be missing from the export. It is recommended that you overlap the date ranges to be safe.
The default behavior is to create a set of ISO images and publish them over
HTTP or HTTPS to /pulp/exports/repo/<repo-id>/
, or if publishing a repo
group, /pulp/exports/repo_group/<group-id>/
. The default image size will
fit on a DVD (4308MB). However, if you would prefer to use an external hard drive
to transport the repositories, you can use the --export-dir
option, which will
export the repository to a directory on the Pulp server rather than creating a set
of ISOs and publishing them over HTTP or HTTPS. If you choose this option, simply
skip step 4.
Errata¶
Searching for Errata¶
Pulp has a very powerful search interface that can be used to search content
units. In this recipe, you will learn how to use it to search for errata that
have been issued on or after a date, and also how to search for errata by type.
Let’s start by defining a repo cleverly called repo
with a demo feed:
$ pulp-admin rpm repo create --repo-id=repo \
--feed=http://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/pulp_unittest/ \
--relative-url=repo
Successfully created repository [repo]
Now let’s sync the repo so it has some errata for us to search:
$ pulp-admin rpm repo sync run --repo-id=repo
The contents of our example repository are from a few years ago, but it includes
errata over a span of a few years. Suppose that I wanted to know which errata
were issued on or after December 1, 2009. For this example, I will include the
--fields=id
flag to limit the output to just be the IDs of the errata, but
you can season that flag to taste, or omit it if you want to see everything:
$ pulp-admin rpm repo content errata --filters='{"issued": {"$gte": "2009-12-01"}}' \
--repo-id=repo --fields=id
Id: RHBA-2010:0010
Id: RHBA-2010:0205
Id: RHBA-2010:0206
Id: RHBA-2010:0222
Id: RHBA-2010:0251
Id: RHBA-2010:0281
Id: RHBA-2010:0282
Id: RHBA-2010:0294
Id: RHBA-2010:0418
We already talked about the --fields=id
flag, so let’s focus on the
--filters='{"issued": {"$gte": "2009-12-01"}}'
flag. pulp-admin
has some built in simple filtering capabilities, but they aren’t as powerful as
the filtering we can achieve with the --filters
flag. We can use this flag
to pass a JSON filter
to MongoDB to have it apply any arbitrary filter we want. In our case, we want
to look for the “issued” field of our errata being greater than or equal to
2009-12-01.
There are three different types of errata: Security Advisories (RHSAs), Bug Fix Advisories (RHBAs), and Product Enhancement Advisories (RHEAs). Suppose we wanted to know which RHSAs were available in a repo. We would run this command:
$ pulp-admin rpm repo content errata --match type=security \
--repo-id=repo --fields=id
Id: RHSA-2007:0114
Id: RHSA-2007:0323
Id: RHSA-2008:0194
Id: RHSA-2008:0892
Id: RHSA-2009:0003
Id: RHSA-2009:0382
Id: RHSA-2009:1472
For this command we asked Pulp to find errata that had their type field set to “security”. We can also find these by applying a regex to the id field:
$ pulp-admin rpm repo content errata \
--match id=^RHSA --repo-id=repo
In this example, we asked MongoDB to look for errata that had an id
that
matched our supplied
Regular Expression.
The carat at the start of our regular expression will match the beginning of the
id
field, and we used RHSA after that to make sure the ID was an RHSA and
not an RHBA or RHEA.
Copy Errata From One Repository to Another¶
The pulp-admin utility can be used to copy errata from one repository to
another. In this recipe, we will create two repositories, sync one with a
sample upstream repository, and then copy an erratum from it to the other.
Let’s begin by creating our two repositories, repo_1
and repo_2
:
$ pulp-admin rpm repo create --repo-id=repo_1 \
--feed=http://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/pulp_unittest/ \
--relative-url=repo_1
Successfully created repository [repo_1]
$ pulp-admin rpm repo create --repo-id=repo_2 \
--relative-url=repo_2
Successfully created repository [repo_2]
Next, we will sync repo_1
, so that it will have some errata that we can
copy:
$ pulp-admin rpm repo sync run --repo-id=repo_1
+----------------------------------------------------------------------+
Synchronizing Repository [repo_1]
+----------------------------------------------------------------------+
This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.
Downloading metadata...
[|]
... completed
Downloading repository content...
[==================================================] 100%
RPMs: 3/3 items
Delta RPMs: 0/0 items
... completed
Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed
Importing errata...
[-]
... completed
Importing package groups/categories...
[-]
... completed
Publishing packages...
[==================================================] 100%
Packages: 3/3 items
... completed
Publishing distributions...
[==================================================] 100%
Distributions: 3/3 items
... completed
Generating metadata
[/]
... completed
Publishing repository over HTTPS
[-]
... completed
Publishing repository over HTTP
[-]
... skipped
Now repo_1
has errata and other units, and repo_2
has no units at all.
Suppose that we would like to pull all of the security updates from repo_1
to repo_2
. We can determine which errata are RHSA by using a match filter:
$ pulp-admin rpm repo content errata --match type=security \
--repo-id=repo_1 --fields=id
Id: RHSA-2007:0114
Id: RHSA-2007:0323
Id: RHSA-2008:0194
Id: RHSA-2008:0892
Id: RHSA-2009:0003
Id: RHSA-2009:0382
Id: RHSA-2009:1472
Running that same command for repo_2
doesn’t show any errata, so let’s use
the unit copy command to bring these RHSAs over, but not the RHBAs or the
RHEAs:
$ pulp-admin rpm repo copy errata --match type=security \
--from-repo-id=repo_1 --to-repo-id=repo_2
Progress on this task can be viewed using the commands under "repo tasks".
Note
Use the –recursive flag to copy any dependencies of units being copied from the source repo into the destination repo.
We can inspect the progress of this operation using
pulp-admin repo tasks list --repo-id=repo_1
. There are only a few
errata to be copied here so it should be complete shortly. Now we can inspect
the contents of repo_2
:
$ pulp-admin rpm repo content errata --repo-id=repo_2 --fields=id
Id: RHSA-2007:0114
Id: RHSA-2007:0323
Id: RHSA-2008:0194
Id: RHSA-2008:0892
Id: RHSA-2009:0003
Id: RHSA-2009:0382
Id: RHSA-2009:1472
Create Your Own Errata¶
You can also create your own errata on a repo using the Pulp client. In order to do this, you will need to create a few CSV files and provide a few data fields to the pulp-admin client.
Let’s begin by making a repo and syncing it:
$ pulp-admin rpm repo create --repo-id=repo \
--feed=http://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/pulp_unittest/
Successfully created repository [repo]
$ pulp-admin rpm repo sync run --repo-id=repo
Now let’s create a new errata that references one of the test packages from this repo called pulp-test-package. The first file that we will need to provide is a references CSV file. This CSV should have four columns: href, type, id, and description, giving a link to the referenced bug report or CVE, the type of the reference, the ID of the reference, and a brief description. Here is an example, named references.csv, wherein you can see that pulp-test-package-0.2.1 has some serious issues:
http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=123456,bugzilla,123456,pulp-test-package-0.2.1 prints mean error messages to users
http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=654321,bugzilla,654321,pulp-test-package-0.2.1 causes users' machines to run out of bits/bytes/whatever. The users must wait until the next supply comes next week
Next, we will need to provide a list of packages that the errata applies to. This CSV provides a list of packages that address the issue that the errata tracks with the following columns: name, version, release, epoch, arch, filename, checksum, checksum_type, and src. For example, let’s create package_list.csv for this:
pulp-test-package,0.3.1,1.fc11,0,x86_64,pulp-test-package-0.3.1-1.fc11.x86_64.rpm,6bce3f26e1fc0fc52ac996f39c0d0e14fc26fb8077081d5b4dbfb6431b08aa9f,sha256,pulp-test-package-0.3.1-1.fc11.src.rpm
Now that we have these two files, we can create our new errata like so:
$ pulp-admin rpm repo uploads erratum --erratum-id=DEMO_ID_1 \
--title="1: pulp-test-package bit conservation" \
--description="1: pulp-test-package now conserves your precious bits." \
--version=1 --release="el6" --type="bugzilla" --status="final" \
--updated="`date`" --issued="`date`" --reference-csv=references.csv \
--pkglist-csv=package_list.csv --from=pulp-list@redhat.com --repo-id=repo
+----------------------------------------------------------------------+
Unit Upload
+----------------------------------------------------------------------+
Extracting necessary metadata for each request...
... completed
Creating upload requests on the server...
[==================================================] 100%
Initializing upload
... completed
Starting upload of selected units. If this process is stopped through ctrl+c,
the uploads will be paused and may be resumed later using the resume command or
canceled entirely using the cancel command.
Importing into the repository...
... completed
Deleting the upload request...
... completed
And now we are able to see that our errata is part of the repo:
$ pulp-admin rpm repo content errata --repo-id=repo --match type=bugzilla
Description: 1: pulp-test-package now conserves your precious bits.
From Str: pulp-list@redhat.com
Id: DEMO_ID_1
Issued: Wed Dec 19 12:19:18 EST 2012
Pkglist:
Name: el6
Packages:
Arch: x86_64
Epoch: 0
Filename: pulp-test-package-0.3.1-1.fc11.x86_64.rpm
Name: pulp-test-package
Release: 1.fc11
Src: pulp-test-package-0.3.1-1.fc11.src.rpm
Sums: 6bce3f26e1fc0fc52ac996f39c0d0e14fc26fb8077081d5b4dbfb6431b08aa9f
Type: sha256
Version: 0.3.1
Short:
Pushcount: 1
Reboot Suggested: False
References:
Href: http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=123456
Id: 123456
Title: pulp-test-package-0.2.1 prints mean error messages to users
Type: bugzilla
Href: http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=654321
Id: 654321
Title: pulp-test-package-0.2.1 causes users' machines to run out of
bits/bytes/whatever. The users must wait until the next supply comes
next week
Type: bugzilla
Release: el6
Rights: None
Severity: None
Solution: None
Status: final
Summary: None
Title: 1: pulp-test-package bit conservation
Type: bugzilla
Updated: Wed Dec 19 12:19:18 EST 2012
Version: 1
Package Groups¶
Create Your Own Package Groups¶
You can easily define your own package groups with the pulp_admin utility. Let’s create and sync a repo:
$ pulp-admin rpm repo create --repo-id=repo_1 \
--feed=http://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/pulp_unittest/
Successfully created repository [repo_1]
$ pulp-admin rpm repo sync run --repo-id=repo_1
Now let’s build a package group for our demo repo test files:
$ pulp-admin rpm repo uploads group --repo-id=repo_1 --group-id=pulp_test \
--name="Pulp Test" --description="A package group of Pulp test files." \
--mand-name=pulp-dot-2.0-test --mand-name=pulp-test-package
+----------------------------------------------------------------------+
Unit Upload
+----------------------------------------------------------------------+
Extracting necessary metadata for each request...
... completed
Creating upload requests on the server...
[==================================================] 100%
Initializing upload
... completed
Starting upload of selected units. If this process is stopped through ctrl+c,
the uploads will be paused and may be resumed later using the resume command or
canceled entirely using the cancel command.
Importing into the repository...
... completed
Deleting the upload request...
... completed
We can see that the package group is now part of our repo:
$ pulp-admin rpm repo content group --repo-id=repo_1 --match id=pulp_test
Conditional Package Names:
Default: False
Default Package Names: None
Description: A package group of Pulp test files.
Display Order: 0
Id: pulp_test
Langonly: None
Mandatory Package Names: pulp-dot-2.0-test, pulp-test-package
Name: Pulp Test
Optional Package Names: None
Repo Id: repo_1
Translated Description:
Translated Name:
User Visible: False
Copying Package Groups Between Repos¶
Package groups can be copied from one repository to another, which will bring along the packages it references as well. For this example, we will assume you’ve performed the steps from the Create Your Own Package Groups section.
We’ll begin by creating a new empty repo, repo_2
:
$ pulp-admin rpm repo create --repo-id=repo_2
Successfully created repository [repo_2]
And now we will copy our package group, pulp_test
from repo_1
to
repo_2
:
$ pulp-admin rpm repo copy group --match id=pulp_test --from-repo-id=repo_1 \
--to-repo-id=repo_2
Progress on this task can be viewed using the commands under "repo tasks".
Note
Use the –recursive flag to copy any dependencies of units being copied from the source repo into the destination repo.
This task should complete fairly quickly since there isn’t much to do with our tiny example repo, but we can check on the progress to verify that it is finished:
$ pulp-admin repo tasks list --repo-id=repo_1
+----------------------------------------------------------------------+
Tasks
+----------------------------------------------------------------------+
Operations: associate
Resources: repo_2 (repository), repo_1 (repository)
State: Successful
Start Time: 2012-12-20T16:26:44Z
Finish Time: 2012-12-20T16:26:44Z
Result: N/A
Task Id: 9f1d0146-cc28-47a8-b0f4-b1b49f84e058
Now we can inspect repo_2
and see that the package group and its RPMs have
been copied there:
$ pulp-admin rpm repo content group --repo-id=repo_2
Conditional Package Names:
Default: False
Default Package Names: None
Description: A package group of Pulp test files.
Display Order: 0
Id: pulp_test
Langonly: None
Mandatory Package Names: pulp-dot-2.0-test, pulp-test-package
Name: Pulp Test
Optional Package Names: None
Repo Id: repo_1
Translated Description:
Translated Name:
User Visible: False
$ pulp-admin rpm repo content rpm --repo-id=repo_2
Arch: x86_64
Buildhost: gibson
Checksum: 435d92e6c09248b501b8d2ae786f92ccfad69fab8b1bc774e2b66ff6c0d83979
Checksumtype: sha256
Description: Test package to see how we deal with packages with dots in the
name
Epoch: 0
Filename: pulp-dot-2.0-test-0.1.2-1.fc11.x86_64.rpm
License: MIT
Name: pulp-dot-2.0-test
Provides: [[u'pulp-dot-2.0-test(x86-64)', u'EQ', [u'0', u'0.1.2',
u'1.fc11']], [u'pulp-dot-2.0-test', u'EQ', [u'0', u'0.1.2',
u'1.fc11']], [u'config(pulp-dot-2.0-test)', u'EQ', [u'0',
u'0.1.2', u'1.fc11']]]
Release: 1.fc11
Requires:
Vendor:
Version: 0.1.2
Arch: x86_64
Buildhost: gibson
Checksum: 6bce3f26e1fc0fc52ac996f39c0d0e14fc26fb8077081d5b4dbfb6431b08aa9f
Checksumtype: sha256
Description: Test package. Nothing to see here.
Epoch: 0
Filename: pulp-test-package-0.3.1-1.fc11.x86_64.rpm
License: MIT
Name: pulp-test-package
Provides: [[u'pulp-test-package(x86-64)', u'EQ', [u'0', u'0.3.1',
u'1.fc11']], [u'pulp-test-package', u'EQ', [u'0', u'0.3.1',
u'1.fc11']], [u'config(pulp-test-package)', u'EQ', [u'0',
u'0.3.1', u'1.fc11']]]
Release: 1.fc11
Requires:
Vendor:
Version: 0.3.1
Package Categories¶
Create Your Own Package Categories¶
You can also define your own package categories with the pulp_admin utility. Let’s create and sync a repo:
$ pulp-admin rpm repo create --repo-id=repo_1 \
--feed=http://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/pulp_unittest/
Successfully created repository [repo_1]
$ pulp-admin rpm repo sync run --repo-id=repo_1
Now let’s build two package groups for our demo repo test files:
$ pulp-admin rpm repo uploads group --repo-id=repo_1 \
--group-id=pulp_test_packages --name="Pulp Test Packages" \
--description="A package group of Pulp test files." \
--mand-name=pulp-dot-2.0-test --mand-name=pulp-test-package
$ pulp-admin rpm repo uploads group --repo-id=repo_1 \
--group-id=pulp_dotted_name_packages --name="Pulp Dotted Name Packages" \
--description="A group of packages that have dots in their names." \
--mand-name=pulp-dot-2.0-test
And now we can easily create a package category that is a collection of these two groups:
$ pulp-admin rpm repo uploads category --repo-id=repo_1 \
--category-id=example_category --name="Example Category" \
--description="An Example Category" --group=pulp_test_packages \
--group=pulp_dotted_name_packages
+----------------------------------------------------------------------+
Unit Upload
+----------------------------------------------------------------------+
Extracting necessary metadata for each request...
... completed
Creating upload requests on the server...
[==================================================] 100%
Initializing upload
... completed
Starting upload of selected units. If this process is stopped through ctrl+c,
the uploads will be paused and may be resumed later using the resume command or
canceled entirely using the cancel command.
Importing into the repository...
... completed
Deleting the upload request...
... completed
The package category details can be listed as well:
$ pulp-admin rpm repo content category --repo-id=repo_1 \
--match id=example_category
Description: An Example Category
Display Order: 0
Id: example_category
Name: Example Category
Packagegroupids: pulp_test_packages, pulp_dotted_name_packages
Repo Id: repo_1
Translated Description:
Translated Name:
Copying Package Categories¶
Like package groups, categories can be copied between repos, which will bring along their groups and packages. Assuming you’ve performed the steps from the Create Your Own Package Categories section, let’s begin by creating an empty second repo:
$ pulp-admin rpm repo create --repo-id=repo_2
Successfully created repository [repo_2]
Now let’s copy example_category
from repo_1
to repo_2
:
$ pulp-admin rpm repo copy category --match id=example_category \
--from-repo-id=repo_1 --to-repo-id=repo_2
Progress on this task can be viewed using the commands under "repo tasks".
Note
Use the –recursive flag to copy any dependencies of units being copied from the source repo into the destination repo.
We should check out the task to see when it’s done with the repo tasks command:
$ pulp-admin repo tasks list --repo-id=repo_1
+----------------------------------------------------------------------+
Tasks
+----------------------------------------------------------------------+
Operations: associate
Resources: repo_2 (repository), repo_1 (repository)
State: Successful
Start Time: 2012-12-20T20:41:12Z
Finish Time: 2012-12-20T20:41:12Z
Result: N/A
Task Id: b5139389-b985-40be-8ee5-10bc626a124a
And now we can see that repo_2
has the category, groups, and RPMs:
$ pulp-admin rpm repo content category --repo-id=repo_2
Description: An Example Category
Display Order: 0
Id: example_category
Name: Example Category
Packagegroupids: pulp_test_packages, pulp_dotted_name_packages
Repo Id: repo_1
Translated Description:
Translated Name:
$ pulp-admin rpm repo content group --repo-id=repo_2
Conditional Package Names:
Default: False
Default Package Names: None
Description: A group of packages that have dots in their names.
Display Order: 0
Id: pulp_dotted_name_packages
Langonly: None
Mandatory Package Names: pulp-dot-2.0-test
Name: Pulp Dotted Name Packages
Optional Package Names: None
Repo Id: repo_1
Translated Description:
Translated Name:
User Visible: False
Conditional Package Names:
Default: False
Default Package Names: None
Description: A package group of Pulp test files.
Display Order: 0
Id: pulp_test_packages
Langonly: None
Mandatory Package Names: pulp-dot-2.0-test, pulp-test-package
Name: Pulp Test Packages
Optional Package Names: None
Repo Id: repo_1
Translated Description:
Translated Name:
User Visible: False
$ pulp-admin rpm repo content rpm --repo-id=repo_2
Arch: x86_64
Buildhost: gibson
Checksum: 435d92e6c09248b501b8d2ae786f92ccfad69fab8b1bc774e2b66ff6c0d83979
Checksumtype: sha256
Description: Test package to see how we deal with packages with dots in the
name
Epoch: 0
Filename: pulp-dot-2.0-test-0.1.2-1.fc11.x86_64.rpm
License: MIT
Name: pulp-dot-2.0-test
Provides: [[u'pulp-dot-2.0-test(x86-64)', u'EQ', [u'0', u'0.1.2',
u'1.fc11']], [u'pulp-dot-2.0-test', u'EQ', [u'0', u'0.1.2',
u'1.fc11']], [u'config(pulp-dot-2.0-test)', u'EQ', [u'0',
u'0.1.2', u'1.fc11']]]
Release: 1.fc11
Requires:
Vendor:
Version: 0.1.2
Arch: x86_64
Buildhost: gibson
Checksum: 6bce3f26e1fc0fc52ac996f39c0d0e14fc26fb8077081d5b4dbfb6431b08aa9f
Checksumtype: sha256
Description: Test package. Nothing to see here.
Epoch: 0
Filename: pulp-test-package-0.3.1-1.fc11.x86_64.rpm
License: MIT
Name: pulp-test-package
Provides: [[u'pulp-test-package(x86-64)', u'EQ', [u'0', u'0.3.1',
u'1.fc11']], [u'pulp-test-package', u'EQ', [u'0', u'0.3.1',
u'1.fc11']], [u'config(pulp-test-package)', u'EQ', [u'0',
u'0.3.1', u'1.fc11']]]
Release: 1.fc11
Requires:
Vendor:
Version: 0.3.1
Comps¶
Upload comps.xml file¶
This is an example of creating a repo and uploading a comps.xml file into it.
$ pulp-admin rpm repo create --repo-id comps-repo
Successfully created repository [comps-repo]
$ pulp-admin rpm repo uploads comps --repo-id comps-repo --file ~/sample-comps.xml
+----------------------------------------------------------------------+
Unit Upload
+----------------------------------------------------------------------+
Extracting necessary metadata for each request...
[==================================================] 100%
Analyzing: sample-comps.xml
... completed
Creating upload requests on the server...
[==================================================] 100%
Initializing: sample-comps.xml
... completed
Starting upload of selected units. If this process is stopped through ctrl+c,
the uploads will be paused and may be resumed later using the resume command or
canceled entirely using the cancel command.
Uploading: sample-comps.xml
[==================================================] 100%
8407/8407 bytes
... completed
Importing into the repository...
This command may be exited via ctrl+c without affecting the request.
[\]
Running...
Task Succeeded
Deleting the upload request...
... completed
Now let’s list the repo and check its content.
$ pulp-admin rpm repo list --repo-id comps-repo
+----------------------------------------------------------------------+
RPM Repositories
+----------------------------------------------------------------------+
Id: comps-repo
Display Name: comps-repo
Description: None
Content Unit Counts:
Package Category: 2
Package Environment: 1
Package Group: 3
Chili¶
- 2 lb. Ground Beef
- Chili Powder
- Garlic
- 1 Large Onion
- 2 Cans of Tomatoes
- 4 Cans of beans (mix & match!)
- Habanero Peppers (be careful)
- Jalapeño Peppers
- 2 Bell Peppers
Put the meat, onion, powder, and tomatoes in a crock pot. Chop up all the vegetables. Put half the vegetables and put those in the crock pot, save the rest for later in the fridge. Turn the crock pot on for several (4-10) hours. After it is done, stir in the remaining vegetables and beans. Cook on high for 30 minutes.
Features¶
RPM Support includes a number of features that are not found in the generic Pulp platform, the most important of which are described below.
Types¶
Pulp RPM supports the following types:
- RPM
- DRPM
- SRPM
- Erratum
- Distribution
- Package Group
- Package Category
Errata¶
Red Hat provides security, bug fix, and enhancement updates for supported Red Hat Enterprise products. These security updates are provided through the Red Hat CDN, and are described by errata. Pulp supports these errata types with a number of related features.
Errata are synchronized from upstream repositories. Errata can also be copied from one repository to another. Administrators can also upload their own errata to a repository. Please see the Recipes documentation to learn how to perform these operations.
Protected Repositories¶
Red Hat protects its repositories with SSL-based entitlement certificates. Pulp supports both ends of that operation:
Each Pulp repository can be configured with a client entitlement certificate and key that it will use to retrieve packages from a remote repository. This is only required when the remote repository is protected, such as when connecting to the Red Hat CDN.
Pulp can be supplied a CA certificate that it will use to verify the authenticity of client certificates when clients try to access Pulp-hosted repositories. This is only required when you want to protect a Pulp-hosted repository. Repositories can have these protection settings specified individually, or they can be set globally for all RPM-related repositories.
For each Pulp-hosted repository that is protected, a consumer certificate can be supplied that will be distributed to consumers when they bind. That certificate will allow them to access the protected repository.
Export¶
In addition to publishing repositories as normal yum repositories over HTTP or HTTPS, it is also possible to export repositories to ISO images, which are published over HTTP or HTTPS, or to a directory on the Pulp server. Large repositories may be split into several ISOs.
Proxy Settings¶
When retrieving packages from a remote repository, Pulp can use a proxy and can supply basic authentication credentials to that proxy.
Bandwidth Throttling¶
When downloading packages from a remote source, Pulp can limit the speed at which data is transferred. The number of downloader threads can also be specified.
ISOs¶
New in Pulp RPM 2.2.0 is support in the admin client extensions for ISO content. The ISO client supports the following features around ISO repositories:
Features¶
- Create repositories
- Update repositories
- Delete repositories
- List repositories
- Copy ISOs between repositories
- Search for ISOs within a given repository
- Remove ISOs from a repository
- Upload ISOs to a repository
- Sync an ISO repository with a ISO feed
- Publish ISO repositories
We will not endeavor to document all of these feature thoroughly here, as they are pretty well
documented in the help text of the admin client’s new iso repo
section:
$ pulp-admin iso repo
Recipes¶
Syncing an ISO Repository¶
In this recipe, we will create an ISO repository with an upstream feed, and we will synchronize it.
Let’s begin by creating the repository:
$ pulp-admin iso repo create --repo-id example --feed http://pkilambi.fedorapeople.org/test_file_repo/ --serve-http true
Successfully created repository [example]
In this command, we’ve created an ISO repository that syncs with the given feed URL, and we’ve also instructed it to publish over HTTP.
Note
The ISO repository can only sync against feeds that publish a manifest file called
PULP_MANIFEST. Most ISO collections on the Internet do not publish a PULP_MANIFEST file
alongside their ISOs, and those collections cannot be consumed by the ISO Importer. The importer
will append a trailing slash to the --feed
setting if it doesn’t already have one, and then
will perform a URL join with the feed and the name PULP_MANIFEST
to determine where it
should look for the manifest. Please ensure that a PULP_MANIFEST is available at the URL you
give to the --feed
setting here.
Now that we’ve created the repository, let’s sync it as well:
$ pulp-admin iso repo sync run --repo-id example
+----------------------------------------------------------------------+
Synchronizing Repository [example]
+----------------------------------------------------------------------+
This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.
Downloading the Pulp Manifest...
The Pulp Manifest was downloaded successfully.
Downloading 3 ISOs...
[==================================================] 100%
ISOs: 3/3 Data: 10.2 MB/10.2 MB Avg: 1.7 MB/s
Successfully downloaded 3 ISOs.
The repository was successfully published.
ISO repositories auto-publish by default, so you can now browse to http://<your-server>/pulp/isos/example/ and view the downloaded ISOs.
Uploading ISOs to a Repository¶
You can also upload your own ISOs to a repository. Let’s begin by creating a repository:
$ pulp-admin iso repo create --repo-id uploads --serve-http true
Successfully created repository [uploads]
We didn’t give this one a feed, but we still instructed it to publish over HTTP. Let’s upload a file:
$ pulp-admin iso repo uploads upload --repo-id uploads -f ~/Desktop/Fedora-17-x86_64-Live-Desktop.iso
+----------------------------------------------------------------------+
Unit Upload
+----------------------------------------------------------------------+
Extracting necessary metadata for each request...
[==================================================] 100%
Analyzing: Fedora-17-x86_64-Live-Desktop.iso
... completed
Creating upload requests on the server...
[==================================================] 100%
Initializing: Fedora-17-x86_64-Live-Desktop.iso
... completed
Starting upload of selected units. If this process is stopped through ctrl+c,
the uploads will be paused and may be resumed later using the resume command or
canceled entirely using the cancel command.
Uploading: Fedora-17-x86_64-Live-Desktop.iso
[==================================================] 100%
676331520/676331520 bytes
... completed
Importing into the repository...
... completed
Deleting the upload request...
... completed
In this example, we uploaded the Fedora 17 ISO from our Desktop. We have not published the repository, and Pulp repositories do not auto publish after uploads, so let’s now publish the repository:
$ pulp-admin iso repo publish run --repo-id uploads
+----------------------------------------------------------------------+
Publishing Repository [uploads]
+----------------------------------------------------------------------+
This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.
The repository was successfully published.
You can now browse to http://<your-server>/pulp/isos/uploads/ and view the ISO you’ve uploaded, as well as the generated PULP_MANIFEST file.
FAQ¶
Why is a checksum used to calculate uniqueness of RPMs?¶
Pulp only stores one copy of each unique unit, no matter how many repositories that unit is associated with. Name-epoch-version-release-arch is not enough for us to guarantee that two RPMs are in fact the same. For example, they may be signed with different keys. Using the checksum to verify uniqueness is the best way for Pulp to accomplish this.
It may not make sense to have two RPMs with the same NEVRA in the same repository, but there might be a use case. As a rule, Pulp intentionally does very little to enforce what makes a repository “valid”. Pulp gives you the tools to manage collections of content, and you get to decide what constitutes a valid collection for your use case.
Troubleshooting¶
RPM with non UTF-8 fields results in error¶
Pulp does not support RPMs with non UTF-8 fields. When such an RPM is encountered, the following traceback will appear in the logs:
Content unit association failed [Unit [key={'name': 'ruby-zypptools', 'checksum': 'aa647a75db016962b72b8d7c1a328a2cf8cfd6a8d5827b58064ab383fde47231', 'epoch': '0', 'version': '0.2.0', 'release': '1.26', 'arch': 'x86_64', 'checksumtype': 'sha256'}] [type=rpm] [id=None]]
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/pulp/plugins/conduits/mixins.py", line 480, in save_unit
unit.id = self._update_unit(unit, pulp_unit)
File "/usr/lib/python2.6/site-packages/pulp/plugins/conduits/mixins.py", line 512, in _update_unit
return self._add_unit(unit, pulp_unit)
File "/usr/lib/python2.6/site-packages/pulp/plugins/conduits/mixins.py", line 534, in _add_unit
unit_id = content_manager.add_content_unit(unit.type_id, None, pulp_unit)
File "/usr/lib/python2.6/site-packages/pulp/server/managers/content/cud.py", line 35, in add_content_unit
collection.insert(unit_doc)
File "/usr/lib/python2.6/site-packages/pulp/server/db/connection.py", line 140, in retry
return method(*args, **kwargs)
File "/usr/lib64/python2.6/site-packages/pymongo/collection.py", line 357, in insert
continue_on_error, self.__uuid_subtype), safe)
InvalidStringData: strings in documents must be valid UTF-8
If you experience this problem, contact the RPM maintainer and ask them to change non UTF-8 fields to UTF-8.
Indices and tables¶
Technical Reference¶
Yum Plugins¶
Yum Types¶
The following are the supported unit types for the Yum plugins. Each unit has a unit type and metadata component. Unit type is a unique combination of fields, and metadata constitutes the rest of the information associated to the unit. The order of the unit fields is significant (and they are listed in order), as they together represent the unit key.
RPM¶
The RPM’s ID is rpm
.
Unit Key¶
name
- Name of the rpm package
version
- Version number of the rpm package
release
- Release of the rpm package
epoch
- Epoch of the rpm package
arch
- Arch of the rpm package
checksumtype
- Checksum type used to generate the rpm checksum value
checksum
- Checksum of the rpm package. This is the checksum of the package itself and not the rpm header checksum
Metadata¶
filename
- Filename of the rpm package
vendor
- The organization responsible for building this rpm package
description
- A more verbose description of the rpm package
buildhost
- The hostname of the build machine on which this package is built
license
- The license information of the vendor
requires
- Used to include required package dependencies for this rpm package
provides
- Used to include rpm provides information
repodata
- metadata xml snippets for rpm package. This includes primary, filelists and other xmls.
Example format:
{"primary" : <primary_xml>, "filelist" : <filelist_xml>, "other" : <other_xml> }
SRPM¶
The SRPM’s ID is srpm
.
Unit Key¶
name
- Name of the srpm package
version
- Version number of the srpm package
release
- Release of the srpm package
epoch
- Epoch of the srpm package
arch
- Arch of the srpm package
checksumtype
- Checksum type used to generate the srpm checksum value
checksum
- Checksum of the srpm package. This is not the srpm header checksum
Metadata¶
filename
- Filename of the srpm package
vendor
- The organization responsible for building this srpm package
description
- A more verbose description of the srpm package
buildhost
- The hostname of the build machine on which this package is built
license
- The license information of the vendor
requires
- Used to include the required package dependencies for this srpm package
provides
- Used to include srpm provides information
repodata
- metadata xml snippets for srpm package. This includes primary, filelists and other xmls.
Example format:
{"primary" : <primary_xml>, "filelist" : <filelist_xml>, "other" : <other_xml> }
DRPM¶
The DRPM’s ID is drpm
.
Unit Key¶
epoch
- Epoch of the rpm package
version
- Version of the rpm package
release
- Release of the rpm package
filename
- filename of the drpm package
checksum
- checksum of the drpm package
checksumtype
- checksum type of the drpm package
Metadata¶
size
- Size of the drpm
sequence
- delta rpm sequence
new_package
- new rpm package associated with the drpm package
Errata¶
The Erratum’s ID is erratum
.
Unit Key¶
id
- Erratum ID string
Metadata¶
title
- Title of the erratum
description
- A more detailed description of the erratum
version
- Version of the erratum
release
- Release of the erratum
type
- Type of erratum. Valid values include “security”, “bugfix” and “enhancement” erratum
status
- Status of the erratum. Example status: “final”
updated
- Updated date of the erratum. Expected format “YYYY-MM-DD HH:MM:SS”
issued
- Issued date of the erratum. Expected format “YYYY-MM-DD HH:MM:SS”
severity
- severity of the erratum. Valid values include “Low”, “Moderate”, “High”
references
- Reference information associated with this erratum
pkglist
- Includes package information associated with this erratum
rights
- Copyrights information associated for the erratum
summary
- Detailed summary information for this erratum
solution
- Detailed Solution information for this erratum
from
- Typically an email address of the erratum issuer
pushcount
- Number of times the erratum has been pushed
reboot_suggested
- Flag indicating if this erratum is installed it will require a reboot of the system
Distribution¶
The distribution type’s ID is distribution
.
Unit Key¶
id
- ID of the distribution to be inventoried
family
- Family of the distribution tree. For example: Red Hat Enterprise Linux
variant
- Variant of the distribution tree. For example: Workstation
version
- Version of the distribution tree. For example: 6Server
arch
- Arch of the distribution tree. For example: x86_64
Metadata¶
files
- Files associated with the distribution tree.
timestamp
- The
timestamp
value as taken from the treeinfo file.
Package Group¶
The Package Group’s ID is package_group
.
Unit Key¶
id
- Package group ID
repo_id
- Repository ID the package group ID is associated
Metadata¶
name
- Name of the package group
description
- Description of the package group
default
- Include this package group by default. Valid values are True and False
user_visible
- If the packagegroup should be visible when queried. Valid values are True and False
langonly
- Language support groups are selected based on this option
display_order
- Display order of the package group
mandatory_package_names
- Mandatory package names to include in the package group
conditional_package_names
- Conditional package names to include in the package group
optional_package_names
- Optional package names to include in the package group
default_package_names
- Default package names to include in the package group
Package Group Category¶
The Package Group Category’s ID is package_category
.
Unit Key¶
id
- Package group category ID
repo_id
- Repository ID to which the package group category ID is associated
Metadata¶
name
- Name of the package group category
description
- Description of the package group category
display_order
- Display order of the package group category
packagegroupids
- Package group IDs associated with the package category
Package Group Environment¶
The Package Group Environment’s ID is package_environment
.
Unit Key¶
id
- Package group Environment ID
repo_id
- Repository ID to which the package group category ID is associated
Metadata¶
name
- Name of the package group environment
translated_name
- Translated names of the package group environment. These are saved as a dictionary of locale
codes to translated names.
Example format:
{"zh_TW" : 'KDE Plasma 工作空間'}
description
- Description of the package group environment
translated_description
- Translated descriptions of the package group environment. These are saved as a dictionary of locale
codes to translated descriptions.
Example format:
{"ru" : 'KDE Plasma Workspaces - легко настраиваемый графический интерфейс пользователя, который содержит панель, рабочий стол, системные значки и виджеты рабочего стола, а также множество мощных приложений KDE.'}
display_order
- Display order of the package group environment
group_ids
- List of Package group IDs associated with the package environment
Example format:
['<group_id_1>','<group_id_2>']
options
- Package group IDs and whether they are default options. The default flag must be set to either
True or False.
Example format:
{"group" : <group_id>, "default" : True}
Note
Package_group, package_category and package environment elements can also be uploaded via comps file. For more info see :ref:` upload_comps_xml_file`.
Yum Repo Metadata File¶
The Yum Repo Metadata File’s ID is yum_repo_metadata_file
.
Unit Key¶
repo_id
- The repository id that this metadata file belongs to
data_type
- The type of the metadata file
Metadata¶
checksum
- The checksum of the metadata file
checksum_type
- The name of the algorithm used to calculate the
checksum
Yum Importer¶
The Yum Importer can be used to sync an RPM repository with an upstream feed. The Yum Importer ID is
yum_importer
.
Configuration Parameters¶
The following options are available to the yum importer configuration. All configuration values are optional.
feed
- URL where the repository’s content will be synchronized from. This can be either an HTTP URL or a location on disk represented as a file URL.
ssl_validation
- Indicates if the server’s SSL certificate is verified against the CA certificate
uploaded. The certificate should be verified against the CA for each client request.
Has no effect for non-SSL feeds. Valid values to this option are
True
andFalse
; defaults toTrue
. ssl_ca_cert
- CA certificate string used to validate the feed source’s SSL certificate (for feeds
exposed over HTTPS). This option is ignored if
ssl_verify
is false. ssl_client_cert
- Certificate used as the client certificate when synchronizing the repository. This is used to communicate authentication information to the feed source. The value to this option must be the full path to the certificate. The specified file may be the certificate itself or a single file containing both the certificate and private key.
ssl_client_key
- Private key to the certificate specified in
ssl_client_cert
, assuming it is not included in the certificate file itself. proxy_host
- Indicates the URL to use as a proxy server when synchronizing this repository.
proxy_port
- Port to connect to on the proxy server.
proxy_username
- Username to pass to the proxy server if it requires authentication.
proxy_password
- Password to use for proxy server authentication.
basic_auth_username
- Username to pass to the feed URL’s server if it requires authentication.
basic_auth_password
- Password to use for server authentication.
query_auth_token
- An authorization token that will be added to every request made to the feed URL’s server, which may be required to sync from repositories that use this method of authorization (SLES 12, for example). This mechanism only supports syncing RPM and deltarpm content.
max_speed
- The maximum download speed in bytes/sec for a task (such as a sync); defaults to None
validate
- If True, as the repository is synchronized the checksum of each file will be
verified against the metadata’s expectation. Valid values to this option are
True
andFalse
; defaults toFalse
. max_downloads
- Number of threads used when synchronizing the repository. This count controls
the download threads themselves and has no bearing on the number of operations
the Pulp server can execute at a given time; defaults to
1
. remove_missing
- If true, as the repository is synchronized, old rpms will be removed. Valid values
to this option are
True
andFalse
; defaults toFalse
retain_old_count
- Count indicating how many old rpm versions to retain; defaults to 0. This count
only takes effect when
remove_old
option is set toTrue
. skip
- List of content types to be skipped during the repository synchronization. If unspecified, all types will be synchronized. Valid values are: rpm, drpm, distribution, erratum; default is [].
checksum_type
- checksum type to use for metadata generation; defaults to source checksum type of
sha256
. num_retries
- Number of times to retry before declaring an error during repository synchronization;
defaults to
2
. copy_children
- Supported only as an override config option to a repository copy command, when this option is False, the copy command will not attempt to locate and copy child packages of errata, groups, or categories. For example, if it is already known that all of a group’s RPMs are available in the destination repository, it can save substantial time to set this to False and thus not have the importer verify the presence of each. default is True.
download_policy
- Set the download policy for a repository. By default this is
immediate
. The other options areon_demand
, where content is only downloaded when a client requests it, andbackground
, where the sync does not download content, but after the sync completes a task is dispatched to download all the content in the background. The content is available for client retrieval during this time.
Yum Distributor¶
The Yum Distributor ID is yum_distributor
.
Configuration Parameters¶
The following options are available to the Yum Distributor configuration.
Required Configuration Parameters¶
http
- Flag indicating if the repository will be served over a non-SSL connection.
Valid values to this option are
True
andFalse
. https
- Flag indicating if the repository will be served over an SSL connection. If
this is set to true, the
https_ca
option should also be specified to ensure consumers bound to this repository have the necessary certificate to validate the SSL connection. Valid values to this option areTrue
andFalse
. relative_url
- Relative path at which the repository will be served.
Optional Configuration Parameters¶
protected
- Protect the published repository with repo authentication. Valid values to this
option are
True
andFalse
. auth_cert
- Certificate that will be provided to consumers bound to this repository. This certificate should contain entitlement information to grant access to this repository, assuming the repository is protected. The value to this option must be the full path to the certificate file. The file must contain both the certificate itself and its private key.
auth_ca
- CA certificate that was used to sign the certificate specified in
auth-cert
. The server will use this CA to verify that the incoming request’s client certificate is signed by the correct source and is not forged. The value to this option must be the full path to the CA certificate file. https_ca
- CA certificate used to sign the SSL certificate the server is using to host this repository. This certificate will be made available to bound consumers so they can verify the server’s identity. The value to this option must be the full path to the certificate.
gpgkey
- GPG key used to sign RPMs in this repository. This key will be made available to consumers to use in verifying content in the repository. The value to this option must be the full path to the GPG key file.
generate_sqlite
- Boolean flag to indicate whether or not sqlite files should be generated during a repository publish. If unspecified it will not run due to the extra time needed to perform this operation.
checksum_type
- Checksum type to use for metadata generation
skip
- List of content types to skip during the repository publish. If unspecified, all types will be published. Valid values are: rpm, drpm, distribution, errata, packagegroup.
Export Distributors¶
There are two export distributors. One that exports a single repository, and another that exports
a repository group. The export distributors can export the repository or repository group as ISO
images, or to the directory of your choice as one or more yum repositories. The repository
distributor uses the ID export_distributor
. The repository group distributor uses the ID
group_export_distributor
. Exported repository ISOs will be published over HTTP or HTTPS at
the path /pulp/exports/repo/<repo-id>/
, and exported repository group ISOs can be found at
/pulp/exports/repo_group/<group-id>/
.
Configuration Parameters¶
The following options are available when configuring the export distributors.
Required Configuration Parameters¶
http
- Flag indicating if the generated ISO will be served over a non-SSL connection.
Valid values to this option are
True
andFalse
. This option is required. https
- Flag indicating if the generated ISO will be served over an SSL connection.
Valid values to this option are
True
andFalse
. This option is required.
Optional Configuration Parameters¶
start_date
- Any content that was associated with the repository before this date will be excluded in the generated ISO. Furthermore, the incremental export process exports errata and rpm metadata as JSON documents, and no repo metadata is generated. The date should be in standard ISO8601 format. For example, “2010-01-01T12:00:00”.
end_date
- Any content that was associated with the repository after this date will be excluded in the generated ISO. Furthermore, the incremental export process exports errata and rpm metadata as JSON documents, and no repo metadata is generated. The date should be in standard ISO8601 format. For example, “2010-01-01T12:00:00”.
iso_prefix
- Prefix to be used in naming the generated ISO. The ISO will be named like this:
<prefix>-<timestamp>-<disc_number>.iso
. The default is the repository or group id. The prefix should only contain alphanumeric characters, dashes, and underscores. skip
- List of content types to skip during the creation of the ISO.
If unspecified, all types will be published. Valid values are:
rpm
,drpm
,srpm
,distribution
,package_group
, anderratum
. iso_size
- An integer, which is the maximum size of the generated ISO images in megabytes. In this case, 1 megabyte is 1 * 1024 * 1024 bytes. This will default to 4380 megabytes (4380 * 1024 * 1024 bytes, to be exact) if it is not specified, which should fit on a single layer DVD.
export_dir
- A full path to an export directory. If this option is specified, the repositories are not placed in ISO images and published over HTTP or HTTPS. Instead, they are written to the export directory. This option is useful if exporting to an external hard drive, for example.
relative_url
- Relative path at which the repository will be served when exported. If this option is specified with
export_dir
, this will become the exported subdirectory name instead of the default, which is the- repository id.
manifest
- If this boolean is True, a PULP_MANIFEST file will be created in the directory where ISOs are created. This allows the ISO importer to directly import what was published. Defaults to False.
ISO Plugins¶
ISO Type¶
The ISO Plugins only have one type, the ISO. The ISO type’s ID is iso
. The following is a descripton of the
ISO unit type that is used by the ISO plugins. The unit key is a combination of fields that can be used to
uniquely identify an ISO. ISOs have no additional metadata.
Unit Key¶
The unit key for the ISO type is an ordered list of name
, checksum
, and size
. Each of
these attributes is described below.
name
- This is the filename of the ISO.
checksum
- This is the SHA-256 checksum of the ISO file.
size
- This is the size in bytes of the ISO file.
Metadata¶
ISOs have no additional metadata outside of the unit key.
ISO Importer¶
The ISO Importer can be used to sync an ISO repository with an upstream feed. The ISO Importer ID is
iso_importer
.
An ISO repository is a fairly basic type of repository. It should be accessible via feed
, and there
should be a file named PULP_MANIFEST
available by appending /PULP_MANIFEST
to the feed
. For
example, if the feed
is http://example.com/iso_repository, the ISO Importer will look for the manifest
at http://example.com/iso_repository/PULP_MANIFEST.
The manifest should be a CSV file with one row per ISO and these three columns, in order: name, checksum, and
size. The CSV file should not have a header row. The name should be the filenames of the ISO, and
the ISO should be accessible by appending the name to the feed
. The checksum should be the
SHA-256 checksum of the ISO, and the size column should represent the size of the ISO in bytes. Here is an
example PULP_MANIFEST
file:
example-1.0.iso,f02d5a72cd2d57fa802840a76b44c6c6920a8b8e6b90b20e26c03876275069e0,127346
example-1.1.iso,c7fbc0e821c0871805a99584c6a384533909f68a6bbe9a2a687d28d9f3b10c16,564830
Configuration Parameters¶
The following configuration parameters are all optional, and can be used to determine the behavior of the ISO importer.
feed
- This should be a string that represents the URL to an upstream ISO repository that you would like this importer to be able to synchronize with. This parameter is optional because it is valid to create an ISO importer that does not synchronize with an upstream feed, but is rather used to contain uploaded ISOs, or ISOs that are copied from other repositories. This parameter becomes required if any of the other parameters are provided.
max_speed
- This should be a numerical value, or a string that can be interpreted as a numerical value, representing the
maximum speed that the importer should be allowed to transfer ISOs at when synchronizing with
feed
. It should be specified in units of bytes per second. max_downloads
- This should be an integer, or a string that can be interpreted as an integer, representing the maximum number
of concurrent downloads that should be performed when synchronizing with
feed
. This parameter defaults to 5. proxy_password
- A string representing the password that should be used to authenticate with the proxy server specified in
proxy_host
. This parameter is required if theproxy_username
is provided. proxy_port
- An integer, or a string that can be interpreted as an integer, representing the port that should be used when
connecting to
proxy_host
. proxy_host
- A string representing the URL of the proxy server that should be used when synchronizing with
feed
. This parameter is required if any of the other proxy setting are provided. proxy_username
- A string representing the username that should be used to authenticate with the proxy server at
proxy_host
. This parameter is required if theproxy_password
is provided. remove_missing
- This is a boolean value, or a string “True” or “False”. If set to “True”, the importer will remove any ISOs
that are currently in the local Pulp repository that are not found in the manifest at
feed
. If “False”, missing ISOs will not be removed. This parameter defaults to False. ssl_ca_cert
- This is a string representing the SSL certificate authority certificate that should be used to validate the
server responding at
feed
. It should be provided in PEM format. ssl_client_cert
- This is a string representing the SSL client certificate that should be used to authenticate the importer to
the upstream repository at
feed
. It should be provided in PEM format. This parameter is required if thessl_client_key
is provided. ssl_client_key
- This is a string representing the private key for
ssl_client_cert
. It should be provided in PEM format. ssl_validation
- This is a boolean that indicates whether we should verify the remote
feed
against the givenssl_ca_cert
. validate
- This is a boolean value, or a string “True” or “False”. If set to “True”, the importer will check the
downloaded ISOs’ file sizes and checksums against the expected values in the manifest when downloading from
feed
. If “False”, no validation will be performed. This parameter defaults to True.
ISO Distributor¶
The ISO Distributor can be used to publish available ISOs in an ISO repository over http or https. It is
distinct from the export_distributor. The ISO Distributor ID is iso_distributor
.
Configuration Parameters¶
The following configuration parameters can be used to determine the behavior of the ISO Distributor. Both configuration parameters are required.
serve_http
- This is a boolean value, or a string “True” or “False”. If set to True, the distributor will publish the ISO repository over plain HTTP, port 80. If False, it will not be published over plain HTTP.
serve_https
- This is a boolean value, or a string “True” or “False”. If set to True, the distributor will publish the ISO repository over SSL protected HTTP, port 443. If False, it will not be published over HTTPS.
ssl_auth_ca_cert
- If the distributor is configured with an authorization CA certificate and the repository protection WSGI app is enabled, the distributed repository will become a protected repository. The given CA certificate will be used to verify the clients’ entitlement certificates. If this certificate is not provided, the repository will be an unprotected repository.
Sort Indexes¶
Overview¶
Version numbers for RPMs and SRPMs aren’t sortable by normal string comparisons. Take the following example:
- 3.1
- 3.9
- 3.10 (read: three point ten)
- 3.11 (read: three point eleven)
The above versions are sorted from oldest to newest. However, when sorting according to string sorting rules, the order is determined to be:
- 3.1
- 3.10
- 3.11
- 3.9
The rules become more complex when letters are added to the version string. More information on sorting RPM versions can be found on the Fedora wiki and the rpmvercmp source.
Pulp¶
This behavior affects both sorting RPMs as well as querying for RPMs relative to a specific
version (i.e. “RPMs newer than version 3.9”). It applies to both the version
and release
attributes on an RPM.
To work around this issue, two extra attributes are added to the RPM’s metadata that is stored
in Pulp’s database: version_sort_index
and release_sort_index
. When sorting or querying against
either an RPM’s version or release, the query should be done against the sort index attributes
instead.
Calculation¶
In order to use simple string sorting in the database, the original values for version and release are encoded for their sort index values. The encoding algorithm is as follows:
- Each version is split apart by periods. We’ll refer to each piece as a segment.
- If a segment only consists of numbers, it’s transformed into the format
dd-num
, where:- dd - number of digits in the value, including leading zeroes if necessary
- num - value of the int being encoded
- If a segment contains one or more letters, it is:
- Split into multiple segments of continuous letters or numbers. For example, 12a3bc becomes 12.a.3.bc
- All of these number-only subsegments is encoded according to the rules above.
- All letter subsegments are prefixed with a dollar sign ($).
- Any non-alphanumeric characters are discarded.
Examples:
3.9 -> 01-3.01-9
3.10 -> 01-3.02-10
5.256 -> 01-5.03-256
1.1a -> 01-1.01-1.$a
1.a+ -> 01-1.$a
12a3bc -> 02-12.$a.01-3.$bc
2xFg33.+f.5 -> 01-2.$xFg.02-33.$f.01-5
\ Sort by:\ best rated\ newest\ oldest\
\\
Add a comment\ (markup):
\``code``
, \ code blocks:::
and an indented block after blank line