Synchronize a Repository

Users can populate their repositories with content from an external source like Docker Hub by syncing their repository.

Create a Repository

#!/usr/bin/env bash
REPO_NAME=$(head /dev/urandom | tr -dc a-z | head -c5)

echo "Creating a new repository named $REPO_NAME."
REPO_HREF=$(http POST $BASE_ADDR/pulp/api/v3/repositories/container/container/ name=$REPO_NAME \
  | jq -r '.pulp_href')

echo "Inspecting repository."
http $BASE_ADDR$REPO_HREF

Repository GET Response:

{
    "pulp_created": "2019-09-05T14:29:43.424822Z",
    "pulp_href": "/pulp/api/v3/repositories/container/container/fcf03266-f0e4-4497-8434-0fe9d94c8053/",
    "latest_version_href": null,
    "versions_href": "/pulp/api/v3/repositories/container/container/ffcf03266-f0e4-4497-8434-0fe9d94c8053/versions/",
    "description": null,
    "name": "codzo"
}

Reference: Container Repository API Usage

Create a Remote

Creating a remote object informs Pulp about an external content source. In this case, we will be using Docker Hub, but pulp-container remotes can be anything that implements the registry API, including quay, google container registry, or even another instance of Pulp.

Note

Container plugin supports both Docker and OCI media types.

#!/usr/bin/env bash
REMOTE_NAME=$(head /dev/urandom | tr -dc a-z | head -c5)

echo "Creating $REMOTE_NAME remote that points to an external source of container images."
REMOTE_HREF=$(http POST $BASE_ADDR/pulp/api/v3/remotes/container/container/ \
    name=$REMOTE_NAME \
    url='https://registry-1.docker.io' \
    upstream_name='pulp/test-fixture-1' | jq -r '.pulp_href')

echo "Inspecting new Remote."
http $BASE_ADDR$REMOTE_HREF

Remote GET Response:

{
    "pulp_created": "2019-09-05T14:29:44.267406Z",
    "pulp_href": "/pulp/api/v3/remotes/container/container/1cc699b7-24fd-4944-bde7-86aed8ac12fa/",
    "pulp_last_updated": "2019-09-05T14:29:44.267428Z",
    "download_concurrency": 20,
    "name": "my-hello-repo",
    "policy": "immediate",
    "proxy_url": null,
    "ssl_ca_certificate": null,
    "ssl_client_certificate": null,
    "ssl_client_key": null,
    "ssl_validation": true,
    "upstream_name": "library/hello-world",
    "url": "https://registry-1.docker.io",
    "include_tags": null,
    "exclude_tags": null
}

Note

Use the fields include/exclude_tags when a specific set of tags is needed to be mirrored instead of the whole repository. Note that it is also possible to filter a bunch of tags that matches defined criteria by leveraging wildcards.

Some registries contain signed images. Such registries provide signatures in different ways. If a registry provides signatures via a dedicated SigStore, a URL to it should be specified in the sigstore field when creating a Remote.

Note

Some registries provide docker API extensions for atomic container signature type only, or have cosign type signatures that are stored as a separate OCI image in a registry. Pulp will automatically sync signatures provided via the docker API extension or cosign signatures stored as an OCI image.

Reference: Container Remote Usage

Sync Content

Use the remote object to kick off a synchronize task by specifying the repository to sync with. You are telling pulp to fetch content from the remote and add to the repository.

#!/usr/bin/env bash

echo "Create a task to sync the repository using the remote."
TASK_HREF=$(http POST $BASE_ADDR$REPO_HREF'sync/' remote=$REMOTE_HREF mirror=False \
  | jq -r '.task')

# Poll the task (here we use a function defined in docs/_scripts/base.sh)
wait_until_task_finished $BASE_ADDR$TASK_HREF

# After the task is complete, it gives us a new repository version
echo "Set REPOVERSION_HREF from finished task."
REPOVERSION_HREF=$(http $BASE_ADDR$TASK_HREF| jq -r '.created_resources | first')

echo "Inspecting RepositoryVersion."
http $BASE_ADDR$REPOVERSION_HREF

Note

In the above example, the payload contains the field mirror=False. This means that the sync will be run in the additive mode only. Set mirror to True and Pulp will pull in new content and remove content which was also removed from upstream. The same logic will be applied when include/exclude_tags are specified together with the mirror command, but only on the subset of tags.

Note

It is not posible to push content to a repository that has been used to mirror content.

Reference: Container Sync Usage

Every time you change a repository, a new repository version is created.

To retrieve a list of repository versions, use the following example request:

http $BASE_URL/pulp/api/v3/repositories/container/<uuid>/versions/

Repository Version GET Response (when complete):

{
    "pulp_created": "2019-09-05T14:29:45.563089Z",
    "pulp_href": "/pulp/api/v3/repositories/container/container/ffcf03266-f0e4-4497-8434-0fe9d94c8053/versions/1/",
    "base_version": null,
    "content_summary": {
        "added": {
            "container.blob": {
                "count": 31,
                "href": "/pulp/api/v3/content/container/blobs/?repository_version_added=/pulp/api/v3/repositories/container/container/fcf03266-f0e4-4497-8434-0fe9d94c8053/versions/1/"
            },
            "container.manifest": {
                "count": 21,
                "href": "/pulp/api/v3/content/container/manifests/?repository_version_added=/pulp/api/v3/repositories/container/container/fcf03266-f0e4-4497-8434-0fe9d94c8053/versions/1/"
            },
            "container.tag": {
                "count": 8,
                "href": "/pulp/api/v3/content/container/tags/?repository_version_added=/pulp/api/v3/repositories/container/container/fcf03266-f0e4-4497-8434-0fe9d94c8053/versions/1/"
            }
        },
        "present": {
            "container.blob": {
                "count": 31,
                "href": "/pulp/api/v3/content/container/blobs/?repository_version=/pulp/api/v3/repositories/container/container/fcf03266-f0e4-4497-8434-0fe9d94c8053/versions/1/"
            },
            "container.manifest": {
                "count": 21,
                "href": "/pulp/api/v3/content/container/manifests/?repository_version=/pulp/api/v3/repositories/container/container/fcf03266-f0e4-4497-8434-0fe9d94c8053/versions/1/"
            },
            "container.tag": {
                "count": 8,
                "href": "/pulp/api/v3/content/container/tags/?repository_version=/pulp/api/v3/repositories/container/container/fcf03266-f0e4-4497-8434-0fe9d94c8053/versions/1/"
            }
        },
        "removed": {}
    },
    "number": 1
}

Reference: Container Repository Version API Usage

Note

To set up a regular sync task, use one of the external tools that deal with periodic background jobs. Learn more about scheduling tasks here.