Changelog

3.2.0 (2020-02-26)

REST API

Features

  • Added a pulpcore-manager script that is django-admin only configured with DJANGO_SETTINGS_MODULE="pulpcore.app.settings". This can be used for things like applying database migrations or collecting static media. #5859

  • Resolve DNS faster with aiodns #6190

Bugfixes

  • Considering base version when removing duplicates #5964

  • Renames /var/lib/pulp/static/ to /var/lib/pulp/assets/. #5995

  • Disabled the trimming of leading and trailing whitespace characters which led to a situation where a hash of a certificate computed in Pulp was not equal to a hash generated locally #6025

  • Repository.latest_version() considering deleted versions #6147

  • Stopped HttpDownloader sending basic auth credentials to redirect location if domains don’t match. #6227

Improved Documentation

  • Updated docs to suggest to use pulpcore-manager command instead of django-admin directly. #5859

Deprecations and Removals

  • Renaming Repository.last_version to Repository.next_version #6147

Misc

Plugin API

Features

  • Adding not equal lookup to model field filters. #5868

Improved Documentation

  • Adds plugin writer docs on adding custom url routes and having the installer configure the reverse proxy to route them. #6209


3.1.1 (2020-02-17)

REST API

Bugfixes

  • Content with duplicate repo_key_fields raises an error #5567

  • Resolve content app errors django.db.utils.InterfaceError: connection already closed. #6045

  • Fix a bug that could cause an inability to detect an invalid signing script during the validation #6077

  • Fixing broken S3 redirect #6154

  • Pin idna==2.8` to avoid a version conflict caused by the idna 2.9 release. #6169

Plugin API

Features

  • A new method _reset_db_connection has been added to content.Handler. It can be called before accessing the db to ensure that the db connection is alive. #6045


3.1.0 (2020-01-30)

REST API

Features

  • Allow administrators to add a signing service #5943

  • Adds pulpcore.app.authentication.PulpDoNotCreateUsersRemoteUserBackend which can be used to verify authentication in the webserver, but will not automatically create users like django.contrib.auth.backends.RemoteUserBackend does. #5949

  • Allow Azure blob storage to be used as DEFAULT_FILE_STORAGE for Pulp #5954

  • Allow to filter publications by repository_version and pulp_created #5968

  • Adds the ALLOWED_IMPORT_PATHS setting which can specify the file path prefix that file:/// remote paths can import from. #5974

  • Allow the same artifact to be published at multiple relative paths in the same publication. #6037

Bugfixes

  • Files stored on S3 and Azure now download with the correct filename. #4733

  • Adds operation_summary to the OpenAPI schema definition of repository modify operation #6002

  • Temporarily pinned redis-py version to avoid a task locking issue. #6038

Improved Documentation

  • Rewrote the Authentication page for more clarity on how to configure Pulp’s authentication. #5949

Deprecations and Removals

  • Removed the django.contrib.auth.backends.RemoteUserBackend as a default configured backend in settings.AUTHENTICATION_BACKENDS. Also removed pulpcore.app.authentication.PulpRemoteUserAuthentication from the DRF configuration of DEFAULT_AUTHENTICATION_CLASSES. #5949

  • Importing from file:/// now requires the configuration of the ALLOWED_IMPORT_PATHS setting. Without this configuration, Pulp will not import content from file:/// locations correctly. #5974

Misc

Plugin API

Features

  • Allow awaiting for resolution on DeclarativeContent. #5668

  • Add a previous() method to RepositoryVersion. #5734

  • Enable plugin writers to sign selected content with signing scripts provided by administrators #5946

  • Add a batching content iterator content_batch_qs() to RepositoryVersion. #6024

Deprecations and Removals

  • The `Handler._handle_file_response` has been removed. It was renamed to ``_serve_content_artifact and has the following signature:

    def _serve_content_artifact(self, content_artifact, headers):
    

    #4733

  • Remove get_or_create_future and does_batch from DeclarativeContent. Replaced by awaiting for resolution on the DeclarativeContent itself. #5668


3.0.1 (2020-01-15)

REST API

Bugfixes

  • Fix bug where content shows as being added and removed in the same version. #5707

  • Fix bug where calling Repository new_version() outside of task raises exception. #5894

  • Adjusts setup.py classifier to show 3.0 as Production/Stable. #5896

  • Importing from file:/// paths no longer destroys the source repository. #5941

  • Webserver auth no longer prompts for csrf incorrectly. #5955

Deprecations and Removals

  • Removed pulpcore.app.middleware.PulpRemoteUserMiddleware from the default middleware section. Also replaced rest_framework.authentication.RemoteUserAuthentication with pulpcore.app.authentication.PulpRemoteUserAuthentication in the Django Rest Framework portion of the config. #5955

Plugin API

Features

  • Added an optional parameter base_version to RepositoryVersion add() and removed() methods. #5706

Deprecations and Removals

  • Saving an Artifact from a source that is outside of settings.MEDIA_ROOT will copy the file instead of moving the file as it did in previous versions. This causes data imported from file:/// sources to be left in tact. #5941


3.0.0 (2019-12-11)

Note

Task names, e.g. pulpcore.app.tasks.orphan.orphan_cleanup, are subject to change in future releases 3.y releases. These are represented in the Task API as the “name” attribute. Please check future release notes to see when these names will be considered stable. Otherwise, the REST API pulpcore provides is considered semantically versioned.

REST API

Features

  • Pulp will do validation that a new repository version contains only content which is supported by the Repository type. Using the same a-priori knowledge of content types, increase performance of duplicate removal. #5701

Bugfixes

  • Improve speed and memory performance. #5688

Improved Documentation

  • Fix an incorrect license claim in the docs. Pulp is GPLv2+. #4592

  • Labeling 3.0 features as tech preview. #5563

  • Simplified docs index page. #5714

  • Add text to Promotion page. #5721

  • Fixes and updates to the glossry page. #5726

Plugin API

Features

  • Added a new required field called CONTENT_TYPES to the Repository model. #5701


3.0.0rc9 (2019-12-03)

REST API

Features

  • Multiple resource-managers can be started and only one will be active. #3707

  • Create an initial repo version when repos get created. #5757

  • Workers no longer require names, and auto-name as {pid}@{fqdn}. This allows easy finding of processes from the Status API. Custom names still work by specifying the -n option when starting a worker. Any worker name starting with resource-manager is a resource-manager, otherwise it’s assumed to be a task worker. #5787

Bugfixes

  • Release reservations for tasks when cleaned up by another worker. #5673

  • Delete the repository version if an exception is raised during finalize_new_version(). #5712

  • Fix uncast Repository bug when modifying repository content. #5728

  • Fix the pulp_type field output in __str__ for MasterModels. #5733

  • Fix path parameter in OpenAPI schema for Repoistory Version endpoints. #5760

  • Fix old references to ssl_ca_certificate and ssl_client_key. #5770

  • Only online workers are shown in the /pulp/api/v3/status/ causing environments where worker names change to not accumulate workers endlessly. #5786

Improved Documentation

  • Added info about resource-manager High Availability to the docs. #3707

  • Fixing a broken link to the Plugin API docs on the homepage. #5660

  • Added content to the pulp2-to-3 documentation page. #5715

  • Move the section “Static Content” to “Architecture and Deploying” #5716

  • Remove the empty Migration page under installation. The migration tool will add this content back as it becomes available. #5717

  • Removes the empty Distributed Installation page. #5718

  • Extract the section “Settings” from the section “Configuration” #5719

  • Adding dedicated Rest API left-navigation section. #5722

  • Move Client Bindings to their own page and fix links. #5723

  • Rearranged contributing documentation. #5724

  • Removes the empty Troubleshooting page. #5725

Deprecations and Removals

  • Resource managers must now have the name resource-manager. For example:

    /path/to/python/bin/rq worker -n 'resource-manager' -w 'pulpcore.tasking.worker.PulpWorker' -c 'pulpcore.rqconfig'
    

    #3707

  • Remove “spawned_tasks” and “parent” field from tasks. #5710

  • The /pulp/api/v3/status/ had the missing_workers section removed. Also the online_workers key had the online and missing keys removed. #5786

  • Remove Publishers from pulpcore

    Now that all plugins use Publications instead of Publishers, remove Publisher model from pulpcore. #5814

Misc

Plugin API

Features

  • Added artifact path overlap checks for repo versions and publications. Plugin writers should call validate_version_paths() or validate_publication_paths() during the finalize step when creating RepositoryVersions or Publications (respectively). #5559

  • Add a new finalize_new_publication() hook for plugin writers to call before a Publication is finalized. #5827

Bugfixes

  • Adds entries to all intended plugin API endpoints to import through pulpcore.plugin. This allows all plugins to safely use the plugin API as long as they import from pulpcore.plugin. #5693

  • Fix the pulp_type field output in __str__ for MasterModels. #5733

Deprecations and Removals

  • The pulpcore.plugin.models.Model is renamed to pulpcore.plugin.models.BaseModel. Also the following objects have been removed from the plugin API:

    • pulpcore.plugin.serializers.NestedIdentityField

    • pulpcore.plugin.serializers.SingleContentArtifactField

    • pulpcore.plugin.serializers.relative_path_validator

    • pulpcore.plugin.viewsets.RemoteFilter

    #5693


3.0.0rc8 (2019-11-13)

REST API

Features

  • New repository version is not created if no content was added or removed. #3308

  • Change relative_path from CharField to TextField #4544

  • Create Master/Detail models, serializers, viewsets for FileSystemExporter. #5086

  • Adds ability to view content served by pulpcore-content in a browser. #5378

  • Adds ability to view distributions served by pulpcore-content in a browser. #5397

  • Users specify Pulp settings file locaiton and type using PULP_SETTINGS environment variable. #5560

  • Added CONTENT_ORIGIN setting, which is now required. #5629

  • Add storage information to the status API. Currently limited to disk space information. #5631

Bugfixes

  • Raise meaningful error for invalid filters. #4780

  • Fix bug where ‘ordering’ parameter returned 400 error. #5621

  • Handling write_only fields on OpenAPISchema. #5622

  • Updated our package version requirements to be compatible with CentOS 7. #5696

Deprecations and Removals

  • Repository version number is no longer incremented if content has not changed. #3308

  • The /pulp/api/v3/repositories/ endpoint has been removed and Repositories have made a “typed” object. They now live at /pulp/api/v3/repositories/<plugin>/<type>, e.g. /repositories/file/file/.

    The convention for sync is that it will now be performed by POSTing to {repo_href}/sync/ remote={remote_href} instead of by POSTING to {remote_href}/sync/ repository={repo_href}. The latter convention will break due to the aforementioned change. #5625

  • Remove plugin managed repos #5627

  • Removed CONTENT_HOST variable and replace its functionality with CONTENT_ORIGIN. #5649

  • Renamed ssl_ca_certificate to ca_cert, ssl_client_certificate to client_cert, ssl_client_key to client_key, and ssl_validation to tls_validation. #5695

Plugin API

Features

  • Added Repository.finalize_new_version(new_version) which is called by RepositoryVersion.__exit__ to allow plugin-code to validate or modify the RepositoryVersion before pulpcore marks it as complete and saves it.

    Added pulpcore.plugin.repo_version_utils.remove_duplicates(new_version) for plugin writers to use. It relies on the definition of repository uniqueness from the repo_key_fields tuple plugins can define on their Content subclasses. #3541

  • Create Master/Detail models, serializers, viewsets for FileSystemExporter. #5086

  • Added the CONTENT_ORIGIN setting which can be used to reliably know the scheme+host+port to the pulp content app. #5629

Improved Documentation

  • Be more explicit about namespacing ref_name in plugin serializers. #5574

  • Add Plugin API section to the changelog. #5628

Deprecations and Removals

  • Renamed the Content.repo_key to be Content.repo_key_fields. Also the calling of remove_duplicates no longer happens in RepositoryVersion.add_content and instead is intended for plugins to call from Repository.finalize_new_version(new_version). Also the pulpcore.plugin.RemoveDuplicates Stage was removed. #3541

  • models.RepositoryVersion.create() is no longer available, it has been replaced by {repository instance}.new_version().

    The convention for sync is that it will now be performed by POSTing to {repo_href}/sync/ remote={remote_href} instead of by POSTING to {remote_href}/sync/ repository={repo_href}. The latter will break due to becoming a typed resource, so plugins will need to adjust their code for the former convention.

    Make repositories “typed”. Plugin writers need to subclass the Repository model, viewset, and serializer, as well as the RepositoryVersion viewset (just the viewset). They should also remove the /sync/ endpoint from their remote viewset and place it on the repository viewset. #5625

  • Remove plugin managed repos #5627


3.0.0rc7 (2019-10-15)

Features

  • Setting code on ProgressReport for identifying the type of progress report. #5184

  • Add the possibility to pass context to the general_create task. #5403

  • Filter plugin managed repositories. #5421

  • Using ProgressReport for known and unknown items count. #5444

  • Expose exclude_fields the api schema and bindings to allow users to filter out fields. #5519

Bugfixes

  • PublishedMetadata files are now stored in artifact storage. #5304

  • Fix 500 on Schemas. #5311

  • /etc/pulp/settings.py override default settings provided by plugins. #5425

  • Fixing error where relative_path was defined on model but not serializer #5445

  • Fixed issue where removing all units on a repo with no version threw an error. #5478

  • content-app sets Content-Type and Content-Encoding headers for all responses. #5507

  • Fix erroneous namespacing for Detail viewsets that don’t inherit from Master viewsets. #5533

Improved Documentation

  • Update installation docs since mariadb/mysql is no longer supported. #5129

Deprecations and Removals

  • By default, html in field descriptions filtered out in REST API docs unless ‘include_html’ is set. #5009

  • Remove support for mysql/mariadb making postgresql the only supported database. #5129

  • Creating a progress report now requires setting code field. #5184

  • Rename the fields on the ContentSerializers to not start with underscore. #5428

  • Removing ProgressSpinner and ProgressBar models. #5444

  • Change _type to pulp_type #5454

  • Change _id, _created, _last_updated, _href to pulp_id, pulp_created, pulp_last_updated, pulp_href #5457

  • Remove custom JSONField implementation from public API #5465

  • Delete NamePagination class and use sorting on the queryset instead. #5489

  • Removing filter for plugin_managed repositories. #5516

  • Renamed fields! to exclude_fields since exclamation mark is a special char in many languages. #5519

  • Removed the logic that automatically defines the namespace for Detail model viewsets when there is no Master viewset. #5533

  • Removing non_fatal_errors from Task. #5537

  • Remove “_” from _versions_href, _latest_version_href #5548

  • Removing base serializer field: _type . #5550


3.0.0rc6 (2019-10-01)

Features

  • Setting code on ProgressReport for identifying the type of progress report. #5184

  • Add the possibility to pass context to the general_create task. #5403

  • Filter plugin managed repositories. #5421

  • Using ProgressReport for known and unknown items count. #5444

Bugfixes

  • PublishedMetadata files are now stored in artifact storage. #5304

  • Fixing error where relative_path was defined on model but not serializer #5445

  • Fixed issue where removing all units on a repo with no version threw an error. #5478

  • content-app sets Content-Type and Content-Encoding headers for all responses. #5507

Improved Documentation

  • Update installation docs since mariadb/mysql is no longer supported. #5129

Deprecations and Removals

  • By default, html in field descriptions filtered out in REST API docs unless ‘include_html’ is set. #5009

  • Remove support for mysql/mariadb making postgresql the only supported database. #5129

  • Creating a progress report now requires setting code field. #5184

  • Rename the fields on the ContentSerializers to not start with underscore. #5428

  • Removing ProgressSpinner and ProgressBar models. #5444

  • Remove custom JSONField implementation from public API #5465

  • Delete NamePagination class and use sorting on the queryset instead. #5489


3.0.0rc5 (2019-09-10)

Features

  • Allow users to filter tasks by created resources #4931

  • Enable users to filter tasks by reserved resources #5120

  • Add CharInFilter that allows filtering CharField by mutiple values #5182

  • Pinning pulpcore dependencies to y releases #5196

Bugfixes

  • Adding fields parameter to OpenAPI schema. #4992

  • Improved the OpenAPI schema for RepositoryVersion.content_summary. #5210

  • Switch default DRF pagination to use LimitOffset style instead of Page ID. #5324

Improved Documentation

  • Update REST API docs for uploads_commit. #5190

  • Removed beta changelog entries to shorten the changelog. #5208

Deprecations and Removals

  • Removing code from task errors. #5282

  • All previous bindings expect a different pagination style and are not compatible with the pagination changes made. Newer bindings are available and should be used. #5324


3.0.0rc4 (2019-07-25)

Features

  • Allow users to pass sha256 with each chunk to have Pulp verify the chunk. #4982

  • Users can view chunks info for chunked uploads in the API #5150

Bugfixes

  • Setting missing fields on orphan cleanup tasks. #4662

  • Allow user to filter created resources without providing _href in a query #4722

  • GET of a Distribution without configuring the CONTENT_HOST setting no longer causes a 500 error. #4945

  • Increased artifact size field to prevent 500 errors for artifacts > 2GB in size. #4998

  • Allow artifacts to be created using json #5016

  • Have the commit endpoint dispatch a task to create artifacts from chunked uploads #5087

  • Allow user to delete uploaded content from a local file system when the artifact creation fails #5092

Improved Documentation

  • Fix broken urls in the /installation/configuration.html#settings area. #5160

Deprecations and Removals

  • Switched the default of the CONTENT_HOST setting from None to ''. #4945

  • Removed upload parameter from artifact create endpoint and converted upload commit to return 202. #5087


3.0.0rc3 (2019-06-28)

Features

  • Pulp now works with webserver configured authentication that use the REMOTE_USER method. Also a new setting REMOTE_USER_ENVIRON_NAME is introduced allowing webserver authentication to work in reverse proxy deployments. #3808

  • Changing chunked uploads to use sha256 instead of md5 #4486

  • Adding support for parallel chunked uploads #4488

  • Each Content App now heartbeats periodically, and Content Apps with recent heartbeats are shown in the Status API /pulp/api/v3/status/ as a list called online_content_apps. A new setting is introduced named CONTENT_APP_TTL which specifies the maximum time (in seconds) a Content App can not heartbeat and be considered online. #4881

  • The task API now accepts PATCH requests that update the state of the task to ‘canceled’. This replaces the previous task cancelation API. #4883

  • Added support for removing all content units when creating a repo version by specifying ‘*’. #4901

  • Added endpoint to delete uploads. Also added complete filter. #4988

Bugfixes

  • Core’s serializer should only validate when policy=’immediate’ (the default). #4990

Improved Documentation

  • Adds an authentication section to the installation guide. Also add two documented settings: AUTHENTICATION_BACKENDS and REMOTE_USER_ENVIRON_NAME. #3808

  • Switch to using towncrier for better release notes. #4875

  • Adds documentation about the CONTENT_APP_TTL setting to the configuration page. #4881

  • The term ‘lazy’ and ‘Lazy’ is replaced with ‘on-demand’ and ‘On-Demand’ respectively. #4990

Deprecations and Removals

  • The migrations are squashed, requiring users of RC3 to deploy onto a fresh database so migrations can be applied again. This was due to alterations made to migration 0001 during the upload work. #4488

  • All the string fields in the REST API no longer accept an empty string as a value. These fields now accept null instead. #4676

  • The Remote.validate field is removed from the database and Remote serializer. #4714

  • The task cancelation REST API has been removed. #4883


3.0.0rc2

Comprehensive list of changes and bugfixes for rc 2.

Breaking Changes

Default port changes happened in the Ansible Installer for Pulp and pulpcore was updated to match with this PR. Existing installs are unaffected. This was done to avoid conflicts that would prevent Pulp from starting by default in many environments; the previous ports (8000 & 8080) are commonly used by management webGUIs, development webservers, etc.

Publications are now Master/Detail which causes any Publication URL endpoint to change. To give an example from pulp_file see the URL changes made here as an example. See plugin docs compatible with 3.0.0rc2 for more details.

Distributions are now Master/Detail which causes the Distribution URL endpoint to change. To give an example from pulp_file see the URL changes made in this PR as an example. See plugin docs compatible with 3.0.0rc2 for more details.

The semantics of Remote attributes ssl_ca_certificate, ssl_client_certificate, and ssl_client_key changed even though the field names didn’t. Now these assets are saved directly in the database instead of on the filesystem, and they are prevented from being read back out to users after being set for security reasons. This was done with these changes.