Pull-Through Caching

Pull-through caching enables plugins to use remotes on their distributions that will act as an upstream fallback source when an user requests content from Pulp. The content will be streamed from the remote and saved in Pulp to be served again in future requests. This feature requires plugins to provide implementations for the methods below on the subclasses of their Remote and Content objects.

pulpcore.app.models.Remote.get_remote_artifact_url(relative_path=None, request=None)

Get the full URL for a RemoteArtifact from relative path and request.

This method returns the URL for a RemoteArtifact by concatenating the Remote’s url and the relative path. Plugin writers are expected to override this method when a more complex algorithm is needed to determine the full URL.

Parameters:

• relative_path (str) – The relative path of a RemoteArtifact

• request (aiohttp.web.Request) – The request object for this relative path

Raises:

ValueError – If relative_path starts with a ‘/’.

Returns:

A URL for a RemoteArtifact available at the Remote.

Return type:

str

pulpcore.app.models.Remote.get_remote_artifact_content_type(relative_path=None)

Get the type of content that should be available at the relative path.

Plugin writers are expected to implement this method. This method can return None if the relative path is for metadata that should only be streamed from the remote and not saved.

Parameters:

relative_path (str) – The relative path of a RemoteArtifact

Returns:

The optional Class of the content type that should be available at the

relative path.

Return type:

Optional[Class]

static pulpcore.app.models.Content.init_from_artifact_and_relative_path(artifact, relative_path)

Return an instance of the specific content by inspecting an artifact.

Plugin writers are expected to override this method with an implementation for a specific content type. If the content type is stored with multiple artifacts plugin writers can instead return a tuple of the unsaved content instance and a dictionary of the content’s artifacts by their relative paths.

For example:

if path.isabs(relative_path):
    raise ValueError(_("Relative path can't start with '/'."))
return FileContent(relative_path=relative_path, digest=artifact.sha256)

Parameters:

• artifact (Artifact) – An instance of an Artifact

• relative_path (str) – Relative path for the content

Raises:

ValueError – If relative_path starts with a ‘/’.

Returns:

An un-saved instance of Content sub-class. Or a tuple of an un-saved instance of Content and a dict of form [relative_path:str, Optional[artifact:~pulpcore.plugin.models.Artifact]]

Finally, plugin writers need to expose the remote field on their distribution serializer to allow users to add their remotes to their distributions. The remote field is already present on the base distribution model, so no new migration is needed.

class GemDistributionSerializer(DistributionSerializer):
    """A Serializer for GemDistribution."""

    ...

    remote = DetailRelatedField(
        required=False,
        help_text=_("Remote that can be used to fetch content when using pull-through caching."),
        view_name_pattern=r"remotes(-.*/.*)?-detail",
        queryset=Remote.objects.all(),
        allow_null=True,
    )

    class Meta:
        fields = DistributionSerializer.Meta.fields + ("publication", "remote")
        model = GemDistribution