Storage

Pulp uses django-storages to support multiple storage backends. If no backend is configured, Pulp will by default use the local filesystem. If you want to use another storage backend such as Amazon Simple Storage Service (S3), you’ll need to configure Pulp.

You can also configure Pulp to use Amazon S3 and Azure storage using the Pulp installer. For more information see the Pulp installer documentation

Local Filesystem

This is the default storage backend Pulp will use if another is not specified. By default, Pulp will set the MEDIA_ROOT to /var/lib/pulp/media as the location where Pulp will store its files. There are three other settings that can be modified, MEDIA_URL, FILE_UPLOAD_PERMISSIONS and FILE_UPLOAD_DIRECTORY_PERMISSIONS. Pulp leaves these settings on their default Django values. See Django docs for more information.

SFTP

Warning

Using SFTP storage is not recommended in Pulp’s current state, and doing so can lead to file corruption. This is because Pulp currently uses coroutines that seem to be incompatible with Django’s SFTPStorage implementation.

Configuring Pulp to use SFTP storage

To use an SFTP server for pulp storage, complete the following steps:

Install the optional dependencies for using sftp storage:
```
pip install pulpcore[sftp]
```
Set the REDIRECT_TO_OBJECT_STORAGE option to False.
Set the MEDIA_ROOT configuration option to "".
Set the DEFAULT_FILE_STORAGE configuration option to "pulpcore.app.models.storage.PulpSFTPStorage".
Configure the remaining options for SFTPStorage according to the django-storages documentation.

Example

We assume that your storage server is set up to serve sftp at the hostname “sftp-storage-host”. It provides a user named “foo” with an ssh keypair stored in “/etc/pulp/certs/storage_id_ed25519”. In its sftp account there should be a directory named “storage” with write access for that user. Varying names would need to be adjusted in the example below.

The configuration would look like:

REDIRECT_TO_OBJECT_STORAGE = False
DEFAULT_FILE_STORAGE = "pulpcore.app.models.storage.PulpSFTPStorage"
MEDIA_ROOT = ""
SFTP_STORAGE_HOST = "sftp-storage-host"
SFTP_STORAGE_ROOT = "/storage/"
SFTP_STORAGE_PARAMS = {
    "username": "foo",
    "key_filename": "/etc/pulp/certs/storage_id_ed25519",
}

Amazon S3

Setting up S3

Before you can configure Amazon S3 storage to use with Pulp, ensure that you complete the following steps. To complete these steps, consult the official Amazon S3 documentation.

Set up an AWS account.
Create an S3 bucket for Pulp to use.
In AWS Identity and Access Management (IAM), create a user that Pulp can use to access your S3 bucket.
Save the access key id and secret access key.

Configuring Pulp to use Amazon S3

To have Pulp use S3, complete the following steps:

Install the optional django-storages and boto3 Python packages in the pulp virtual environment:
```
pip install django-storages[boto3]
```
Depending on which method you use to install or configure Pulp, you must set DEFAULT_FILE_STORAGE to storages.backends.s3boto3.S3Boto3Storage in Pulp Settings. For example, if you use the Pulp installer, the default_file_storage is part of the pulp_settings Ansible variables you can define in your Ansible playbook.
In that same way, add your Amazon S3 configuration settings to AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_STORAGE_BUCKET_NAME. For more S3 configuration options, see the django-storages documents.

Here is an example configuration that will use a bucket called pulp3 that is hosted in region eu-central-1:
AWS_ACCESS_KEY_ID = 'AKIAIT2Z5TDYPX3ARJBA'
AWS_SECRET_ACCESS_KEY = 'qR+vjWPU50fCqQuUWbj9Fain/j2pV+ZtBCiDiieS'
AWS_STORAGE_BUCKET_NAME = 'pulp3'
AWS_DEFAULT_ACL = "@none None"
S3_USE_SIGV4 = True
AWS_S3_SIGNATURE_VERSION = "s3v4"
AWS_S3_ADDRESSING_STYLE = "path"
AWS_S3_REGION_NAME = "eu-central-1"
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
MEDIA_ROOT = ''
If the system that hosts Pulp is running in AWS and has been configured with an instance profile that provides access to the S3 bucket you can omit the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY parameters as the underlying boto3 library will pick them up automatically.

It is only necessary to set AWS_DEFAULT_ACL to "@none None" if you have set the BlockPublicAcls option in the Block Public Access settings of your bucket or of your AWS account. The default setting in the latest version of django-storages is public-read, which will get blocked. This is set to change in a future release.

Azure Blob storage

Setting up Azure Blob storage

Before you can configure Azure Blob storage to use with Pulp, ensure that you complete the following steps. To complete these steps, consult the official Azure Blob documentation.

Set up an Azure account and create a storage account.
In your storage account, create a container under Blob service.
Obtain the access credentials so that you can later configure Pulp to access your Azure Blob storage. You can find the access credentials at the storage account level, at Access keys (these are automatically generated).

Configuring Pulp to use Azure Blob storage

Install the optional django-storages[azure] package in the pulp virtual environment:
```
pip install django-storages[azure]
```
Depending on which method you use to install or configure Pulp, you must set DEFAULT_FILE_STORAGE to storages.backends.azure_storage.AzureStorage in Pulp Settings. For example, if you use the Pulp installer, the default_file_storage is part of the pulp_settings Ansible variables you can define in your Ansible playbook.

In the same way, configure the following parameters:

AZURE_ACCOUNT_NAME = 'Storage account name'
AZURE_CONTAINER = 'Container name (as created within the blob service of your storage account)'
AZURE_ACCOUNT_KEY = 'Key1 or Key2 from the access keys of your storage account'
AZURE_URL_EXPIRATION_SECS = 60
AZURE_OVERWRITE_FILES = 'True'
AZURE_LOCATION = 'the folder within the container where your pulp objects will be stored'
MEDIA_ROOT = ''

For a comprehensive overview of all possible options for the Azure Blob storage backend see the django-storages[azure] documents.