PACER dashboard¶

The PACER Dashboard is an optional monitoring component for PACER. When deployed, it provides real-time visibility into message processing activities across the platform. Its primary purpose is to offer a user-friendly interface that enables participating teams to monitor data ingestion workflows, trace message processing, and efficiently diagnose and troubleshoot errors.

Architecture¶

The dashboard is built using Django¹ and uses PostgreSQL² for persistent data storage. The web application is served through both uWSGI and ASGI: uWSGI handles the main web interface, while ASGI provides support for real-time features such as WebSockets.

After processing a message, each PACER consumer publishes processing results and operational metrics to RabbitMQ. On the dashboard side, a dedicated worker consumes these messages, processes the reported information, and stores it in the database. This data is then made available through the dashboard interface for monitoring, analysis, and debugging purposes.

architecture-beta
    group api(cloud)[PACER deployment]

    service db(database)[PostgreSQL] in api
    service uwsgi(server)[uWSGI server] in api
    service asgi(server)[ASGI server] in api
    service worker(server)[Worker] in api
    service rabbitmq(server)[RabbitMQ] in api
    service pacer(server)[PACER] in api

    db:R -- L:uwsgi
    db:R -- L:asgi
    db:R -- L:worker

    pacer:R -- L:rabbitmq

    rabbitmq:T -- R:worker

Partitioned database tables¶

Since all messages processed by PACER are stored in the database, partitioning the underlying tables is essential for maintaining efficient query performance and simplifying database maintenance. The dashboard application leverages the built-in partitioning capabilities provided by django-postgres-extra³ to manage data effectively, enabling the system to scale to large message volumes without significant performance degradation.

By default, partitions for the Message model span six-month intervals. The system maintains the current partition along with two future partitions at all times. This behavior can be customized through the model configuration if smaller partition sizes are required.

Partitions are created automatically using the command below. To ensure that future partitions are available when needed, it is recommended to run this command periodically as part of a scheduled maintenance task.

python /django/app/manage.py pgpartition --yes --skip-delete || true

Investigation checks¶

The dashboard provides a dedicated command that executes specific workflows after an investigation’s end date has passed.

By default, this command verifies that every non-industrial investigation whose end date has elapsed has both a DOI and a PSS item created. If either is missing, it sends a request to PACER to trigger their creation.

This check is performed for up to 30 days after the investigation’s end date to account for potential delays in data ingestion. The process stops earlier if both the DOI and the PSS item are successfully created, whichever occurs first.

python /django/app/manage.py run_investigation_checks 

Role-Based Access to Messages¶

Access to message data can be controlled through the application’s user group system. In particular, access can be restricted based on message type or message content. This enables fine-grained permissions, such as allowing members of a given instrument to view only messages associated with that instrument.

This can be configured through the Django admin interface at /admin.

Configuration¶

The configuration of the dashboard is mainly done through environment variables.

Database¶

Authentication¶

If authentication is enabled, anonymous users will not be able to see the details of the messages that can contain sensible information.

RabbitMQ¶

ICAT¶

PaNOSC¶

This is optional, you do not need to set this unless your PSS server has basic authentication enforced.

Django¶