Waldur Site Agent
Agent for Mastermind integration with a provider's site. The main purpose of the agent is data syncronization between Waldur and an application (for example SLURM or MOAB cluster). The agent uses order-related information from Waldur to manage accounts in the site and accounting-related info from the site (backend) to update usage data in Waldur. For now, the agent supports only SLURM and MOAB clusters as a site.
Architecture
Agent is a stateless application, which is deployed on a machine having access to a backend data. It consists of three sub-applications:
agent-order-process
, which fetches ordering data from Waldur and updates a state of a backend correspondingly; (e.g. creates/updates/deletes SLURM accounts);agent-report
, which reports computing usage and limits info from a backend to Waldur (e.g. update of resource usages);agent-membership-sync
, which syncs membership info between Waldur and a backend (e.g. adds users to a SLURM allocation).
Integration with Waldur
For this, the agent uses Waldur client
based on Python and REST communication with Waldur backend.
Agent-order-process
application pulls data of orders created
in Waldur and creates/updates/removes backend resources based on this info.
Agent-report
fetches usage data pushes it to Waldur.
Agent-membership-sync
fetches associations from
a backend and syncronizes it with remote ones.
Integration with the site
SLURM cluster
The agent relies on SLURM command line utilities (e.g. sacct
and sacctmgr
)
and should run on a headnode of the SLURM cluster.
MOAB cluster
The agent relies on MOAB command line utilities (e.g. mam-list-accounts
and mam-create-account
)
and should run on a headnode of the MOAB cluster as a root user.
Agent configuration
The application supports the following CLI arguments:
-m
,--mode
- mode of agent; supported values:order_process
,report
andmembership_sync
; default isorder_process
.-c
,--config-file
- path to the config file with provider settings.
Optional environment variables:
WALDUR_SITE_AGENT_ORDER_PROCESS_PERIOD_MINUTES
- trigger period fororder_process
mode in minutes (default is 5);WALDUR_SITE_AGENT_REPORT_PERIOD_MINUTES
- trigger period forreport
mode in minutes (default is 30);WALDUR_SITE_AGENT_MEMBERSHIP_SYNC_PERIOD_MINUTES
- trigger period formembership_sync
mode in minutes (default is 5).REQUESTS_VERIFY_SSL
- flag for SSL verification for Waldur client, default istrue
.SENTRY_ENVIRONMENT
- name of the Sentry environment.
The primary config for the agent is a waldur-site-agent-config.yaml
.
Using it, the agent can serve several offerings
and setup backend-related data, for example computing component settings.
File example and reference.
NB: for MOAB, the only acceptable component is deposit
.
All other specified components are ignored by the agent.
Deployment
A user should deploy 3 separate instances of the agent. The first one (called agent-order-process) fetches data from Waldur with further processing, the second one (called agent-report) sends usage data from the backend to Waldur and the third one syncs membership information between Waldur and the backend. All the instances must be configured with CLI variables and provider config file.
To deploy them, you need to setup and start the systemd services.
Prerequisite: offering configuration in Waldur
SLURM and MOAB
The agents require existing offering data in Waldur. As a service provider owner, you should create an offering in the marketplace:
- Go to
Service Provider
section of the organization and open offering creation menu - Input a name, choose a category, select
SLURM remote allocation
from the drop-down list on the bottom and clickCreate
button
- Open the offering page and create a plan in the
Accounting
section: clickAdd plan
and input the necessary details - Go to
Integration
section, clickShow integration steps
and ensure they are completed within your SLURM/MOAB cluster.
Setup
Firstly, install the waldur-site-agent
module:
1 |
|
Secondly, create the provider config file and adjust the content for your needs.
1 |
|
Please use the waldur_site_load_components
command
to load computing components into Waldur.
1 |
|
Thirdly, put systemd unit and provider config files to the corresponding locations.
-
agent-order-process systemd unit: waldur-agent-order-process.service
-
agent-report systemd unit: waldur-agent-report.service
-
agent-membership-sync systemd unit: waldur-agent-membership-sync.service
1 2 3 4 5 6 7 8 |
|
After these preparation steps, run the following script to apply the changes.
1 2 3 4 5 6 7 |
|
Older systemd versions
If you want to deploy the agents on a machine with systemd revision older than 240, you should use files with legacy configuration:
- systemd legacy unit file for agent-pull: waldur-site-agent-pull-legacy.service
- systemd legacy unit file for agent-push: waldur-site-agent-push-legacy.service
1 2 3 4 5 6 |
|
Provider config file reference
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
|