Changelog

0.17.0 (2022-09-20)

Bugfixes

  • Fixed issue where migration.py passes a Content object to remove_content(), which ends up breaking pulpcore’s remove_content() further down the line with a traceback stating, ‘Content’ object has no attribute ‘count’.

    Pulpcore’s remove_content() will not face this issue anymore as it will now always receive a QuerySet object from migration.py. #550

  • Migration may stuck for many days when migrating an environment with massive number of rpms. This commit fixed the issue. #568

  • When pre-migrating errata, make query in batch to prevent the BSON too large error. #572

  • Fix invalid BSON size error when migrating docker contents

    The size limit for a single document in Mongodb is 16MB so migrating many docker tags could exceed this limit. This commit fixes this issue by fetching the docker tags in a specified batch size instead of fetching all and returning a large results into a single document. #578

Improved Documentation

  • Added steps to remove the plugin and announced its EOL. Pulp-2to3-migration plugin reaches its EOL on December 31, 2022. The last supported pulpcore version is 3.19. #534

0.16.0 (2022-03-15)

Features

  • Added the DEB_COMPONENT_BATCH_SIZE setting, so users can individually control the batch size for this one to many type. #9564

Bugfixes

  • Try parsing advisory dates as unix timestamps before giving up. #483

  • Fixed a bug in the deb migration that caused migrated repos to lack content needed for structured publications in some circumstances. #484

  • Fixed DistributionTree (kickstart tree) migration for pulp_rpm 3.17+.

    Alma Linux 8 or CentOS 8 repos migration will no longer fail with: No declared artifact with relative path “images/boot.iso” (or “.treeinfo”). #491

  • Escape django template syntax in xml when rendering filelist #496

  • Fixed local variable ‘item’ referenced before assignment #497

  • Taught mongo-connection how to deal with Really Big Queries. This addresses the cause of exceptions like pymongo.errors.OperationFailure: Sort exceeded memory limit of 104857600 bytes when migrating large collections. #511

  • Fixed 'NoneType' object has no attribute 'delete' error during migration re-runs. #8968

  • deb_components now use a default batch size of 50 to avoid CursorNotFound errors during pre_migration for this one to many type. #9564

0.15.0 (2021-11-11)

Features

  • Added signing_service for (apt-)repository in Complex-MigrationPlan. This allows for a signing-service created before the migration to automatically be attached to the migrated repositories. #9517

Bugfixes

  • Fixed distibution tree migration issue “‘NoneType’ object has no attribute ‘url’”. #8862

  • Add batch_size to a pulp 2 query for content pre-migration of errata. #9451

Improved Documentation

  • Added a note about migration when Pulp 3 runs on a FIPS-enabled machine. #7992

Misc

0.14.0 (2021-08-31)

Bugfixes

  • Filtered out incomplete publications for the pulp2repositories/ endpoint. #9242

  • Fixed migration of sles_auth_token #9254

  • Fixed bug where sync tasks would open a lot of DB connections #9283

  • Fixed remigration of publications after an unsuccessful or interrupted run. #9295

Misc

0.13.0 (2021-08-06)

Bugfixes

  • Fixed modular errata migration. #8874

  • Add migration of gpgkeys-field for verification of debian-repos. #8970

  • Fix migration for any clients that have applied the fix for #8893. #8982

  • Strip leading slashes from Relative URL for deb-migrations. #9184

Deprecations and Removals

  • Dropped support for Python 3.6 and 3.7. #9038

Misc

0.12.1 (2021-07-02)

Bugfixes

  • Fixed a bug causing deb migrations to fail if there are Pulp 2 importers without a releases field. #8928

0.12.0 (2021-06-17)

Features

  • Started migrating relationship between repo and importer to Pulp 3. #8626

Improved Documentation

  • Taught 2to3 migration about changes in task-dispatching and Distributions.

    NOTE: To minimize any potential impact from this change, please make sure you’ve done a 2to3 migration after any Pulp2 changes to distributors before upgrading to this release. #8522

0.11.11 (2022-06-06)

Bugfixes

  • Fixed issue where migration.py passes a Content object to remove_content(), which ends up breaking pulpcore’s remove_content() further down the line with a traceback stating, ‘Content’ object has no attribute ‘count’.

    Pulpcore’s remove_content() will not face this issue anymore as it will now always receive a QuerySet object from migration.py. #550

0.11.10 (2022-03-18)

Bugfixes

  • Taught mongo-connection how to deal with Really Big Queries. This addresses the cause of exceptions like pymongo.errors.OperationFailure: Sort exceeded memory limit of 104857600 bytes when migrating large collections. #511

0.11.9 (2022-01-14)

Bugfixes

  • Escape django template syntax in xml when rendering filelist #496

  • Fixed local variable ‘item’ referenced before assignment #497

0.11.8 (2022-01-07)

Bugfixes

  • Fixed distribution tree migration for repositories with partial .treeinfo (e.g. most of CentOS 8 repositories). #489

0.11.7 (2021-12-07)

Bugfixes

  • Fixed 'NoneType' object has no attribute 'delete' error during migration re-runs. (backported from #8968) #9612

0.11.6 (2021-11-17)

Bugfixes

  • Fixed migration of sles_auth_token (backported from #9254) #9456

  • Add batch_size to a pulp 2 query for content pre-migration of errata. (backported from #9451) #9482

0.11.5 (2021-09-10)

Bugfixes

  • Fixed distibution tree migration issue “‘NoneType’ object has no attribute ‘url’”. (backported from #8862) #9372

0.11.4 (2021-08-26)

Bugfixes

  • Fixed remigration of publications after an unsuccessful or interrupted run. (backported from #9295) #9296

  • Filtered out incomplete publications for the pulp2repositories/ endpoint. (backported from #9242) #9297

Misc

0.11.3 (2021-08-02)

Bugfixes

  • Fixed a bug causing deb migrations to fail if there are Pulp 2 importers without a releases field. (backported from #8928) #8945

  • Added migration of gpgkeys-field for verification of debian-repos. (backported from #8970) #9017

  • Fixed migration for any clients that have applied the fix for #8893. (backported from #8982) #9109

  • Fixed modular errata migration. (backported from #8874) #9173

  • Strip leading slashes from Relative URL for deb-migrations. (backported from #9184) #9188

0.11.2 (2021-06-14)

Bugfixes

  • Fixed migration of CentOS8 distribution trees. #8566

  • Fixed a problem when migrating one plugin after another will remove publications/distributions of the first one. #8686

  • Ensure a checksum type of a package is used for publications when a checksum type was not explicitly configured in Pulp 2.

    If you plan to perform sync from the migrated Pulp 3 to a Pulp 2 instance, this fix is important, otherwise you can ignore it. If you’ve already started migration of the RPM plugin to Pulp 3, reset the migration for it and start again. #8725

  • Fix migration of on_demand distribution (kickstart) trees when they do no have any images, e.g. CentOS 8 High Availability repo. #8817

  • Pulp2 Content that does not have downloaded flag is marked as downloaded. #8863

0.11.1 (2021-05-04)

Bugfixes

  • Stopped migrating unsupported metadata, like .zck, which could have been imported into some old Pulp 2 version. #8400

  • Handled overlapping paths which might come from old pulp2 repositories. #8582

  • Handled properly skipping of corrupted or missing docker content. #8594

  • Fixed a bug in the deb pipeline that was preventing successfull skipping of corrupted content for migrations with skip_corrupted=True. #8612

  • Fixed migration of Pulp 2 deb importers without configured components or architectures #8613

  • Fixed total counters for content migration. #8621

  • Fixed Debian importer migration on a re-run. #8653

0.11.0 (2021-04-12)

Features

  • Added a new configuration option CONTENT_PREMIGRATION_BATCH_SIZE to be able to adjust the content batch size if the system is slow. #8470

Misc

0.10.0 (2021-03-23)

Bugfixes

  • Fixed the case when some Pulp 2 content was removed and cleaned up between migration re-runs. #7887

  • Fixed a migraiton failure when no importer was specified in the migration plan. #8382

  • Fixed errata re-migration when it’s been copied to a new repo in Pulp2 between migration re-runs. #8417

  • Fixed the case when listing pulp2content/ endpoint using bindings would fail if content didn’t have a storage path in Pulp 2. #8436

0.9.1 (2021-03-11)

Bugfixes

  • Fixed a migration failure when no importer was specified in the migration plan. #8389

0.9.0 (2021-03-09)

Bugfixes

  • Fixed re-migration cases when a distributor on an importer changed in the migration plan between re-runs. #7889

  • Ensure trailing slash is present when assembling the url. #8321

  • Fixed pulp2content unique-constraint to correctly handle a null repo-id field. #8329

Improved Documentation

  • Added docs on the value of fast storage volumes and adjusting the worker timeout values for both Pulp 2 and Pulp 3. #7847

  • Updated docs to pulp-cli. #8254

Misc

0.8.0 (2021-02-18)

Features

  • Added the ability to migrate additional Debian content types needed for structured publishing. #7865

  • The default configuration now contains the ALLOWED_CONTENT_CHECKSUMS setting with all checksum types supported in Pulp 2. #8266

Bugfixes

  • Fixed the re-run times when repositories/importers/distributors haven’t changed much since the last run. #7779

  • Fixed an edge-case failure in erratum-migration when doing repeated migrations. #8166

  • Fixed distributor re-migration case when it was changed in Pulp 2 between migration plan runs. #8195

  • Fixed openapi schema for reset/ endpoint. Bindings no longer require plan parameter. #8211

Improved Documentation

  • Added a note that ALLOWED_CONTENT_CHECKSUMS is strongly recommended to allow all supported checksum types, and can be adjusted after the migration. #8266

Misc

0.7.0 (2021-02-04)

Bugfixes

  • Multi-artifact content aren’t (not) skipped properly when some artifacts are unavailable. #7681

  • Taught pre-migration to order content by last-updated.

    This lets a migration recover reliably from fatal errors during migration attempts. NOTE: this fix assumes the Pulp2 instance is at least at 2.21.5. Earlier versions are missing an index in the Mongo database that makes the ordering possible. #7781

  • Fix an error migrating module content with no “stream” or “profile” information specified, as is allowed by the spec. #7846

  • No longer generate sqlite metadata when publishing unless the Pulp 2 configuration specified to do so. #7851

  • Fixed Pulp2Content serialization when filters are applied. #7994

  • Taught rpm to warn and continue if a Distribution is missing a treeinfo file. #8084

Misc

0.6.0 (2020-12-04)

Features

  • Added an option to skip corrupted or missing Pulp 2 content. #7538

  • Added a reset/ endpoint to be able to run migration from scratch. #7714

  • Added support to migrate Debian packages (tech preview). #7863

Bugfixes

  • Fixed distribution serialization. #7809

Misc

0.5.1 (2020-10-27)

Bugfixes

  • Fixed a bug where RPM content metadata is not properly migrated to Pulp 3. #7625

0.5.0 (2020-10-13)

Bugfixes

  • Publications and Distributions are re-created on migration re-run for repos that contain mutable content, and pre-migrated mutable content is no longer deleted and recreated every time. #7280

  • Fixed a bug where PULP_MANIFEST was being created outside of the worker’s working directory. #7693

  • Sped up repository pre-migration by skipping the repository content relations pre-migration when nothing changed in a repository. #7694

  • Made content migration significantly faster on low-spec machines w/ HDD backed database storage. #7699

0.4.1 (2020-10-09)

Bugfixes

  • Fix the bindings for publication and distribution hrefs fields on pulp2repositories API. #7679

0.4.0 (2020-10-07)

Bugfixes

  • Fixed a distribution migration case when a repository in Pulp 2 has been recreated. #7080

  • Stopped logging warnings if at least one LCE per content migrated. #7193

  • Fixed metadata checksum type configuration re-migration. #7417

  • Fixed re-migration issue when pulp 2 importer changed a feed. #7418

  • Fixed validation of the distributor missing resources in the migration plan. #7488

  • Fix custom metadata migration when the same metadata is present under different paths in different repositories. #7489

  • Fixed high memory usage when migrating large amounts of content (300,000+). #7490

  • Removed comps content types from future_types. #7518

  • Fixed migration of lazy multi-artifact content not present in a repository in the plan. #7562

0.3.0 (2020-08-26)

Features

  • Added GroupProgressReport tracking during the migration. #6769

  • Make the migration plugin compatible with pulp_container 2.0 #7365

Bugfixes

  • Significantly improved performance of partial migrations (when some content / repos has been migrated already). #6111

  • Fixed migration of a distribution tree if it has a treeinfo and not .treeinfo #6951

  • Fixed cause of view_name warnings during (re)start of Pulp services. #7154

  • Marked all Pulp2LCEs as migrated for distribution tree migration. #7260

Misc

0.2.1 (2020-08-26)

Bugfixes

  • Updated migration of file remote url to point to the Manifest. #7264

0.2.0 (2020-08-20)

Bugfixes

  • Fix exceptions thrown by content migration not being bubbled up through the task. #6469

0.2.0b6 (2020-07-24)

Features

  • Add support for migrating SLES12+ repos which require auth token. #6927

Bugfixes

  • Fixed distribution tree migration when a distribution tree is present in multiple repositories. #6950

  • Fix a bug where errata were not always migrated for new repositories. #7092

  • Fix yum metadata files not being migrated. #7093

  • Fix an issue causing extremely high memory usage as # of content scale up. #7152

  • Fixed a bug where migrated repositories could have multiple different copies of an errata. #7165

Misc

0.2.0b5 (2020-07-03)

Bugfixes

  • Fixed distribution tree re-migration. #6949

  • Fixed RPM migration when its remote is not migrated. #7078

Misc

0.2.0b4 (2020-06-23)

Features

  • Migrate checksum_type configuration for an RPM publication. #6813

Bugfixes

  • Fixed Ruby bindings generation. #7016

0.2.0b3 (2020-06-17)

Features

  • Slightly improve performance by allowing repos to be migrated in parallel. #6374

  • As a user, I can track Remotes and not remigrate them on every run. #6375

  • Track Publications and Distributions, recreate if needed and not on every run. #6376

Bugfixes

  • Expose pulp3_repository_version on pulp2content if it is available. #6580

  • Ensure that only one migration plan can be run at a time. #6639

  • Fixed UnboundLocalError during migration of a repo with a custom name. #6640

  • Fix an issue where a migration with many plugin types would crash on execution. #6754

  • Fixed distribution creation when a distributor is from a repo which is not being migrated. #6853

  • Fixed migration of a sub-set of previously migrated repos. #6886

  • Handle already-migrated ‘re-created’ pulp2 repos #6887

  • Fixed marking of old distributors, when distributor only is migrated without the repo. #6932

  • Fixed case when a publication is shared by multiple distributions. #6947

  • Set pulp3_repo relation for all the cases, including remigration. #6964

  • Fixed incorrect pulp3_repo_version href for advisories after remigration. #6966

  • Fix comps migration when repo is recreated between the migration runs. #6980

0.2.0b2 (2020-04-22)

Features

  • Migrate errata content. #6178

  • As a user I can migrate comps content into pulp3. #6358

  • As a user I can migrate SRPMS. #6388

  • Improve performance by looking only at lazy content types and not through all the migrated content. #6499

Bugfixes

  • Set properly relative_path Pulp2YumRepoMetadataFile content_artifact. #6400

Misc

0.2.0b1 (2020-03-24)

Features

  • Migrate RPM packages to Pulp 3. #6177

  • Add custom repo metadata migration. #6283

  • As a user I can migrate modules and modules-defaults #6321

Bugfixes

  • Add awaiting for docker DC resolution and do not use does_batch. #6084

Misc

0.1.0 (2020-03-24)

Bugfixes

  • Do not pre-migrate schema1 docker tags when there are 2 tags with same name witin a repo. #6234

Improved Documentation

  • Moved README to readthedocs website. #6145

0.1.0rc1 (2020-02-28)

Bugfixes

  • Migrating large repository leads to unmigrated units. #6103

  • Migrate mutable content. #6186

0.0.1rc1 (2020-02-11)

Features

  • Add pulp3_repository_href to pulp2repositories api. #6053

  • Make pulp2 importer optional. #6056

  • Migrate empty repos if the migration plan specifies them. #6070

Bugfixes

  • Handling missing plugin modules #5820

  • Fix migration of multiple plugins. #5978

  • Add error message for the importers that cannot be migrated. #5984

  • Fix the bindings for publication and distribution hrefs fields on pulp2repositories API. #6049

  • Fix rendering of the pulp2repositories after a failed migration. #6058

  • Handle case when repos are removed and re-created. #6062

  • Fix docker repo migration with a custom distributor. #6097

  • Fix blobs and manifests relations on migration re-run. #6099

Misc

0.0.1b1 (2020-01-25)

Features

  • As a user, I can provide a Migration Plan.

  • Migrate iso content.

  • Migration plan resources are validated against MongoDB (i.e. that they exist). #5319

  • Migrate on_demand content. #5337

  • Migrate Pulp 2 repositories into Pulp 3 repo versions. #5342

  • As a user, I can migrate Pulp 2 distributor into publication/distribution in Pulp 3 #5343

  • Migrate docker content. #5363

  • Migration plans are respected. #5450

  • Mark and take into account changed or removed pulp2 resources. #5632

  • Adding a new endpoint to query the Pulp2-Pulp3 mapping for resources. #5634

  • Update get_pulp3_repository_setup so repos are grouped by plugin type. #5845

Bugfixes

  • Migrate only those repo types that belong to the plugin that is being migrated #5485

  • Fix bug preventing the serializer from accepting non-JSON data #5546

  • Prevent migration of importers/distributors with an empty config. #5551

  • Specify pulp2_distributor_repository_ids instead of distributor_ids #5837

  • Importer or distributor can be migrated even if their repository is not. #5852

  • Fix “local variable ‘pulp2repo’ referenced before assignment”. #5899

  • Fix repository type identification. #5957

  • All requested repositories are migrated regardless of the time of the last run or a migration plan change. #5980

Improved Documentation

  • Switch to using towncrier for better release notes. #5501

  • Add examples of a Migraiton plan. #5849

Deprecations and Removals

  • Change __id, __created, __last_updated, __href to pulp_id, pulp_created, pulp_last_updated, pulp_href #5457

Misc