Waldur Docker-compose deployment
Prerequisites
- at least 8GB RAM on Docker Host to run all containers
- Docker v1.13+
Prepare environment
1 2 3 4 5 |
|
Booting up
1 2 3 4 5 6 7 8 9 10 11 12 |
|
Waldur HomePort will be accessible on https://localhost. API will listen on https://localhost/api.
Healthcheck can be accessed on https://localhost/health-check.
Tearing down and cleaning up:
1 |
|
Logs
Logs emitted by the containers are collected and saved in the waldur_logs
folder. You can change the location by
editing environment variable (.env
) and updating LOG_FOLDER
value.
Known issues
When Waldur is launched for the first time, it applies initial database migrations. It means that you may need to wait few minutes until these migrations are applied. Otherwise you may observe HTTP error 500 rendered by REST API server. This issue would be resolved after upgrade to Docker Compose 1.29.
To use a custom script offering type, it should be possible to connect to /var/run/docker.sock
from
within the Waldur containers. If you are getting a permission denied error in logs, try setting more
open permissions, for example, chmod 666 /var/run/docker.sock
. Note that this is not a secure
setup, so make sure you understand what you are doing.
Upgrading Waldur
1 2 3 |
|
Upgrade Instructions for PostgreSQL Images
Automated Upgrade (Recommended)
To simplify the upgrade process, an upgrade script db-upgrade-script.sh
is included in the root directory. This script automates the entire upgrade process.
Usage Instructions
- Ensure Waldur is running with the current (old) PostgreSQL version that you wish to upgrade from:
1 |
|
-
Update the PostgreSQL versions in
.env
file:1 2
WALDUR_POSTGRES_IMAGE_TAG=<your_version> KEYCLOAK_POSTGRES_IMAGE_TAG=<your_version>
-
Ensure the script has execution permissions:
1 |
|
- Run the upgrade script:
1 |
|
Important: The script needs the containers to be running with the old PostgreSQL version first so it can back up the existing data before upgrading.
The script will automatically:
- Back up both databases
- Shut down all containers
- Remove old data directories and volumes
- Pull new PostgreSQL images
- Start containers with new PostgreSQL versions
- Restore data from backups
- Create SCRAM tokens for PostgreSQL 14+ compatibility
- Start all containers
Manual Upgrade (Alternative)
If you prefer to perform the upgrade manually, follow these steps:
Manual Prerequisites
- Backup existing data (if needed)
Backup Commands
You can back up the database using pg_dumpall
.
For Waldur DB:
1 |
|
For Keycloak DB:
1 |
|
Manual Upgrade Steps
-
Update PostgreSQL Versions
Update the
WALDUR_POSTGRES_IMAGE_TAG
andKEYCLOAK_POSTGRES_IMAGE_TAG
in the.env
file to the required versions.1 2
WALDUR_POSTGRES_IMAGE_TAG=<your_version> KEYCLOAK_POSTGRES_IMAGE_TAG=<your_version>
-
Shut down containers
1
docker compose down
-
Remove old data directories
Note: The waldur-db uses a bind mount (
./pgsql
) while keycloak-db uses a named volume (keycloak_db
). Both need to be removed before upgrading. Warning: This action will delete your existing PostgreSQL data. Ensure it is backed up before proceeding.Remove the pgsql directory (waldur-db data):
1
sudo rm -r pgsql/
Remove the keycloak_db volume:
1
docker volume rm waldur-docker-compose_keycloak_db
-
Pull the New Images
1
docker compose pull
-
Start database containers
1
docker compose up -d waldur-db keycloak-db
-
Restore Data (if backups have been made)
For Waldur DB:
1
cat waldur_upgrade_backup.sql | docker exec -i waldur-db psql -U waldur
For Keycloak DB:
1
cat keycloak_upgrade_backup.sql | docker exec -i keycloak-db psql -U keycloak
-
Create SCRAM tokens (for PostgreSQL 14+)
If the new PostgreSQL version is 14 or later, create SCRAM tokens for existing users:
1 2 3 4
export $(cat .env | grep "^POSTGRESQL_PASSWORD=" | xargs) docker exec -it waldur-db psql -U waldur -c "ALTER USER waldur WITH PASSWORD '${POSTGRESQL_PASSWORD}';" export $(cat .env | grep "^KEYCLOAK_POSTGRESQL_PASSWORD=" | xargs) docker exec -it keycloak-db psql -U keycloak -c "ALTER USER keycloak WITH PASSWORD '${KEYCLOAK_POSTGRESQL_PASSWORD}';"
-
Start all containers
1
docker compose up -d
-
Verify the Upgrade
Verify the containers are running with the new PostgreSQL version:
1
docker ps -a
Check container logs for errors:
1 2
docker logs waldur-db docker logs keycloak-db
Using TLS
This setup supports following types of SSL certificates:
- Email - set environment variable TLS to your email to register Let's Encrypt account and get free automatic SSL certificates.
Example:
1 |
|
- Internal - set environment variable TLS to "internal" to generate self-signed certificates for dev environments
Example:
1 |
|
- Custom - set environment variable TLS to "cert.pem key.pem" where cert.pem and key.pem - are paths to your custom certificates (this needs modifying docker-compose with path to your certificates passed as volumes)
Example:
1 |
|
Custom Caddy configuration files
To add additional caddy config snippets into the caddy virtual host configuration add .conf files to config/caddy-includes/
Keycloak
Keycloak is an Identity and Access Management software bundled with waldur-docker-compose.
To create a keycloak admin account set KEYCLOAK_ADMIN
env variable in docker-compose.yaml
and KEYCLOAK_ADMIN_PASSWORD
in .env
file.
After this, you can login to the admin interface at https://localhost/auth/admin and create Waldur users.
To use Keycloak as an identity provider within Waldur, follow the instruction here. The discovery url to connect to Keycloak from the waldur-mastermind-api container is:
1 |
|
Integration with SLURM
The integration is described here.
Whitelabeling settings
To set up whitelabeling, you need to define settings in ./config/waldur-mastermind/whitelabeling.yaml
.
You can see the list of all whitelabeling options below.
General whitelabeling settings
- site_name
- site_address
- site_email
- site_phone
- short_page_title
- full_page_title
- brand_color
- hero_link_label
- hero_link_url
- site_description
- currency_name
- docs_url
- support_portal_url
Logos and images of whitelabeling
The path to a logo is constructed like so: /etc/waldur/icons - is a path in the container (Keep it like it is) + the name of the logo file from config/whitelabeling directory.
All-together /etc/waldur/icons/file_name_from_whitelabeling_directory
- powered_by_logo
- hero_image
- sidebar_logo
- sidebar_logo_mobile
- site_logo
- login_logo
- favicon
Readonly PostgreSQL user configuration
In order to enable /api/query/ endpoint please make sure that read-only user is configured both in PostgreSQL and in the environment variables.
1. Create PostgreSQL readonly user
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
|
2. Configure environment variables
Add the following environment variables to your .env
file:
1 2 |
|
Note: Replace {readonly_password}
with the actual password you used when creating the readonly user, and {readonly_username}
with your chosen readonly username (e.g., "readonly").
Migration from bitnami/postgresql to library/postgres DB image
After migration from the bitnami/postgresql to library/postgres DB image, you might notice a working in logs like this:
1 2 3 4 |
|
In this case, you can simply update the collaction version and reindex the Waldur DB and the public schema:
1 2 3 4 5 6 7 8 9 |
|