Waldur Site Agent
Project for Waldur integration with a service provider's site. The main purpose of the agent is data syncronization between Waldur and a service backend (for example SLURM or MOAB cluster). The agent uses order information from Waldur to manage accounts in the site (backend) and accounting info from the site to update usage data in Waldur. For now, the agent supports only SLURM and MOAB clusters as a service backend.
Architecture
Agent is a stateless application, which is deployed on a machine with access to backend data. It consists of 4 sub-applications:
agent-order-process
, which fetches ordering data from Waldur and updates a state of backend object correspondingly; (e.g. creates/updates/deletes SLURM accounts);agent-report
, which reports computing usage and limits info from the backend to Waldur (e.g. update of resource usages);agent-membership-sync
, which syncronizes membership info between Waldur and the backend (e.g. adds users to a SLURM allocation);agent-event-process
, which uses event-based approach to do the same asagent-order-process
andagent-membership-sync
; requires an MQTT-system as an event delivery queue between Waldur and the agent.
Integration with Waldur
The agent uses a Python-based Waldur client
communicating with Waldur backend
via REST interface.
Agent-order-process
application pulls data of orders created
in Waldur and creates/updates/removes backend resources based on this info.
Agent-report
fetches usage data pushes it to Waldur.
Agent-membership-sync
fetches associations from
a backend and syncronizes it with remote ones.
Agent-event-process
manages Waldur orders and membership in event-based way.
Integration with the site
SLURM cluster
The agent relies on SLURM command-line utilities (e.g. sacct
and sacctmgr
)
and should run on a headnode of the SLURM cluster.
MOAB cluster
The agent relies on MOAB command line utilities (e.g. mam-list-accounts
and mam-create-account
)
and should run on a headnode of the MOAB cluster as a root user.
Agent configuration
The application supports the following CLI arguments:
-m
,--mode
- mode of agent; supported values:order_process
,report
,membership_sync
andevent_process
; default isorder_process
.-c
,--config-file
- path to the config file with provider settings.
Optional environment variables:
WALDUR_SITE_AGENT_ORDER_PROCESS_PERIOD_MINUTES
- trigger period fororder_process
mode in minutes (default is 5);WALDUR_SITE_AGENT_REPORT_PERIOD_MINUTES
- trigger period forreport
mode in minutes (default is 30);WALDUR_SITE_AGENT_MEMBERSHIP_SYNC_PERIOD_MINUTES
- trigger period formembership_sync
mode in minutes (default is 5).REQUESTS_VERIFY_SSL
- flag for SSL verification for Waldur client, default istrue
.SENTRY_ENVIRONMENT
- name of the Sentry environment.
The main config source for the agent is waldur-site-agent-config.yaml
file.
Using it, the agent can serve multiple offerings
and setup backend-related data, for example settings of computing components.
File example and reference.
NB: for MOAB, the only acceptable backend component is deposit
.
All other specified components are ignored by the agent.
Deployment
A user can deploy 4 separate instances of the agent. The first one (called agent-order-process) fetches data from Waldur with further processing, the second one (called agent-report) sends usage data from the backend to Waldur the third one syncs membership information between Waldur and the backend and the optional fourth one processes order and membership info in event-based way. All the instances must be configured with provider config file and CLI variables.
To deploy them, you need to setup and start the systemd services.
Prerequisite: offering configuration in Waldur
SLURM and MOAB
Agents require a pre-created offering in Waldur. As a service provider owner, you should create a new offering in the marketplace:
- Go to
Service Provider
section of the organization and open offering creation menu - Input a name, choose a category, select
SLURM remote allocation
from the drop-down list on the bottom and clickCreate
button
- Open the offering page, choose
Edit
tab, clickAccounting
section, chooseAccounting plans
from the drop-down list and create a plan: clickAdd plan
and input the necessary details;
- In the same page, click
Integration
section chooseUser management
from the drop-down list and set theService provider can create offering user
option toYes
;
- Activate the offering using the big green button
Activate
.
Note: You will need to set the offering UUID in the agent config file.
For this, you can copy the UUID from the Integration -> Credentials
section on the same page:
Setup
Firstly, install the waldur-site-agent
module:
1 |
|
Secondly, create the provider config file and adjust the content for your needs.
1 |
|
Please use the waldur_site_load_components
command
to load computing components into Waldur.
This step is necessary for correct setup of the offering in Waldur.
1 |
|
Thirdly, put systemd unit and provider config files to the corresponding locations.
-
agent-order-process systemd unit: waldur-agent-order-process.service
-
agent-report systemd unit: waldur-agent-report.service
-
agent-membership-sync systemd unit: waldur-agent-membership-sync.service
-
agent-event-process systemd unit: waldur-agent-event-process.service
1 2 3 4 5 6 7 8 9 10 11 |
|
After these preparation steps, run the following script to apply the changes.
1 2 3 4 5 6 7 8 9 10 11 12 13 |
|
Event-based processing
Each offering from the config file can have mqtt_enabled
set to true
or false
(false
by default).
If this setting set to true
, the offering is ignored
by agent-order-process
and agent-membership-sync
,
instead, the agent-event-process
takes care of it.
Older systemd versions
If you want to deploy the agents on a machine with systemd revision older than 240, you should use files with legacy configuration:
1 2 3 4 5 6 7 8 |
|
Provider config file reference
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 |
|