.. _rq: http://python-rq.org
.. _deployment:
Architecture and Deploying
==========================
Pulp's architecture has three components to it: a REST API, a content serving application, and the
tasking system. Each component can be horizontally scaled for both high availability and/or
additional capacity for that part of the architecture.
REST API
--------
Pulp's REST API is a Django application which runs under any WSGI server. It serves the following
things:
* The REST API hosted at ``/pulp/api/v3/``
* The browse-able documentation at ``/pulp/api/v3/docs/``
* Any viewsets or views provided by plugins
* Static content used by Django, e.g. images used by the browse-able API. This is not Pulp content.
.. note::
A simple, but limited way to run the REST API as a standalone service using the built-in Django
runserver. The ``pulpcore-manager`` command is ``manage.py`` configured with the
``DJANGO_SETTINGS_MODULE="pulpcore.app.settings"``. Run the simple webserver with::
$ pulpcore-manager runserver 24817
The REST API can be deployed with any any WSGI webserver like a normal Django application. See the
`Django deployment docs `_ for more
information.
Content Serving Application
---------------------------
An aiohttp.server based application that serves content to clients. The content could be
:term:`Artifacts` already downloaded and saved in Pulp, or
:term:`on-demand content units`. When serving
:term:`on-demand content units` the downloading also happens from within this
component as well.
.. note::
Pulp installs a script that lets you run the content serving app as a standalone service as
follows:::
$ pulp-content
The content serving application can be deployed like any aiohttp.server application. See the
`aiohttp Deployment docs `_ for more
information.
Tasking System
--------------
Pulp's tasking system has two components: a resource manager and workers, all of which are run using
`rq`_.
Worker
Pulp workers perform most tasks "run" by the tasking system including long-running tasks like
synchronize and short-running tasks like a Distribution update. Each worker handles one task at a
time, and additional workers provide more concurrency. Workers auto-name and are auto-discovered,
so they can be started and stopped without notifying Pulp.
Resource Manager
A different type of Pulp worker that plays a coordinating role for the tasking system. You must
run exactly one of these for Pulp to operate correctly. The ``resource-manager`` is identified by
configuring using exactly the name ``resource-manager`` with the ``-n 'resource_manager'`` option.
*N* ``resource-manager`` rq processes can be started with 1 being active and *N-1* being passive.
The *N-1* will exit and should be configured to auto-relaunch with either systemd, supervisord, or
k8s.
.. note::
Pulp serializes tasks that are unsafe to run in parallel, e.g. a sync and publish operation on
the same repo should not run in parallel. Generally tasks are serialized at the "repo" level, so
if you start *N* workers you can process *N* repo sync/modify/publish operations concurrently.
Static Content
--------------
When browsing the REST API or the browsable documentation with a web browser, for a good experience,
you'll need static content to be served.
In Development
^^^^^^^^^^^^^^
If using the built-in Django webserver and your settings.yaml has ``DEBUG: True`` then static
content is automatically served for you.
In Production
^^^^^^^^^^^^^
Collect all of the static content into place using the ``collectstatic`` command. The
``pulpcore-manager`` command is ``manage.py`` configured with the
``DJANGO_SETTINGS_MODULE="pulpcore.app.settings"``. Run ``collectstatic`` as follows::
$ pulpcore-manager collectstatic