Skip to content

Ad-hoc scripts

This section describes different ad-hoc scripts used by Waldur developers.

Restore CPU/GPU/RAM usages for a month

Sometimes, Waldur can have issues with data correctness. They can be caused by different circumstansies, e.g. problems with data syncronization between Waldur backend and service provider software. In order to resolve these issues, a user can restore usage data with restore_usages.py script.

Generate openmetrics file with usages for several months

If a user needs to create openmetrics file with usages, e.g. for historical data backfilling in Prometheus, the generate_openmetrics_usages.py can be used.

Prolong expired projects

If a user needs to prolong an expired project with remote resources, this script should be used. It can:

  1. automatically detect projects with pending update requests
  2. approve the requests
  3. set state OK for terminating resources both in a local Waldur and remote one

Note: the script must be executed in a local Waldur instance

If a user needs to remove a tenant with all related objects (instances, volumes, backups, etc.) manually, this script should be used. It removes all the tenants with names from tenant_names list using an OpenStack administrator project. Please, replace <main-tenant-uuid> (line 13) with the corresponding UUID of the admin tenant in Waldur.

Import projects with slug info

A CSV-file with the following format is required:

1
2
"id","unix_gid","valid_from","valid_to","accessible_from","accessible_to"
"abc","10000","01/1/2024","31/3/2024","05/1/2024","28/3/2024"

Put this file to the /tmp/projects-info.csv location and execute this script with your Waldur shell.

Migrate legacy SLURM allocations to a new SLURM-remote offering

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
from waldur_mastermind.marketplace import models as marketplace_models

COMPONENT_TYPE_MAP = {
    "cpu": "cpu",
    "gpu": "gres/gpu",
    "ram": "mem",
}


def migrate_legacy_allocation(
    resource: marketplace_models.Resource,
    offering_new: marketplace_models.Offering,
    dry_run=True,
):
    plan_old = resource.plan
    if dry_run:
        print("Dry-run migrating legacy allocation %s" % resource)
    else:
        print("Migrating legacy allocation %s" % resource)
        resource.offering = offering_new
        print("Updating resource plan to %s" % plan_new)
        resource.plan = plan_new
        resource.save(update_fields=["offering", "plan"])
    try:
        for usage in resource.usages.all():
            new_component = offering_new.components.get(
                type=COMPONENT_TYPE_MAP[usage.component.type]
            )
            if dry_run:
                print("Dry-run switching component usage %s" % usage)
            else:
                print("Switching usage %s" % usage)
                usage.component = new_component
                usage.save(update_fields=["component"])
    except Exception as e:
        if dry_run:
            print("Dry-run Rolling back resource offering and plan because of %s" % e)
        else:
            print("Rolling back resource offering and plan because of %s" % e)
            resource.offering = offering_old
            resource.plan = plan_old
            resource.save(update_fields=["offering", "plan"])

offering_old_uuid = "<LEGACY_OFFERING_UUID>"
offering_new_uuid = "<NEW_OFFERING_UUID>"
offering_old = marketplace_models.Offering.objects.get(uuid=offering_old_uuid)
offering_new = marketplace_models.Offering.objects.get(uuid=offering_new_uuid)
plan_new = offering_new.plans.first()

legacy_allocations = marketplace_models.Resource.objects.filter(offering=offering_old)

i = 1
legacy_allocations_count = legacy_allocations.count()
for legacy_allocation in legacy_allocations:
    print("Processing resource %s/%s" % (i, legacy_allocations_count))
    try:
        migrate_legacy_allocation(legacy_allocation, offering_new, dry_run=False)
    except Exception as e:
        print("Failed to migrate allocation %s: %s" % (legacy_allocation, e))
    i += 1