Skip to content

Arrow (ArrowSphere) Integration

Overview

The Arrow integration connects Waldur with the ArrowSphere cloud marketplace platform. It imports billing data, tracks consumption, and reconciles costs for IaaS subscriptions (Azure, AWS, etc.) managed through Arrow as a channel partner.

The integration provides:

  • Customer mapping between Arrow references (XSP...) and Waldur organizations
  • Vendor-to-offering mapping (e.g., Microsoft to a Waldur Azure offering)
  • Billing export sync with invoice item creation
  • Real-time consumption tracking with finalized billing reconciliation
  • Resource import from Arrow subscriptions

Prerequisites

  • Active ArrowSphere partner account with API access
  • Arrow API key with permissions for billing, customers, subscriptions, and consumption endpoints
  • Feature flag reseller.arrow enabled in Waldur

Configuration

Feature Flag

Enable the Arrow integration in Waldur features:

1
2
# Via Django admin or API
FEATURES["reseller.arrow"] = True

Constance Settings

All settings are managed via Constance (runtime-configurable):

Setting Default Description
ARROW_AUTO_RECONCILIATION False Auto-apply compensations when Arrow validates billing
ARROW_SYNC_INTERVAL_HOURS 6 Billing sync interval in hours
ARROW_CONSUMPTION_SYNC_ENABLED False Enable real-time consumption sync from Arrow API
ARROW_CONSUMPTION_SYNC_INTERVAL_HOURS 1 Consumption sync interval in hours
ARROW_BILLING_CHECK_INTERVAL_HOURS 6 Billing export check interval for reconciliation

Arrow Settings Record

Arrow API credentials are stored in the ArrowSettings model, not in Constance. Create settings via the API or setup wizard (see Setup Workflow below). Key fields:

Field Description
api_url Arrow API base URL
api_key API key for authentication
export_type_reference Billing export template reference (discovered from API)
classification_filter Filter for IaaS/SaaS classification (default: IAAS). If the Arrow API rejects requests with this filter, the system automatically retries without it.
invoice_price_source Which price to use for invoice items: sell (default) or buy
sync_enabled Whether automatic billing sync is enabled
is_active Whether this settings record is active

Only one active settings record should exist per deployment.

Setup Workflow

Step 1: Enable Feature Flag

Enable reseller.arrow via the features API or Django admin.

Step 2: Validate and Save API Credentials

Use the setup wizard endpoints to configure credentials:

1
2
3
4
5
POST /api/admin/arrow/settings/validate_credentials/
{
    "api_url": "https://xsp.arrow.com/api/",
    "api_key": "your-api-key"
}

This validates the credentials and returns partner info and available export types.

Then save the settings:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
POST /api/admin/arrow/settings/save_settings/
{
    "api_url": "https://xsp.arrow.com/api/",
    "api_key": "your-api-key",
    "export_type_reference": "DJ284LDZ-standard",
    "classification_filter": "IAAS",
    "sync_enabled": true,
    "customer_mappings": [
        {"arrow_reference": "XSP661245", "waldur_customer_uuid": "..."}
    ]
}

Step 3: Discover and Map Customers

Discover Arrow customers and get mapping suggestions:

1
2
3
4
5
POST /api/admin/arrow/settings/discover_customers/
{
    "api_url": "https://xsp.arrow.com/api/",
    "api_key": "your-api-key"
}

The response includes fuzzy-matched suggestions between Arrow company names and Waldur organizations. Create mappings via:

1
2
3
4
5
6
7
POST /api/admin/arrow/customer-mappings/
{
    "settings": "<settings-uuid>",
    "arrow_reference": "XSP661245",
    "arrow_company_name": "Example Corp",
    "waldur_customer": "<customer-uuid>"
}

Alternatively, use sync_from_arrow to bulk-sync customer data:

1
POST /api/admin/arrow/customer-mappings/sync_from_arrow/

Step 4: Map Vendors to Offerings

Map Arrow vendor names to Waldur marketplace offerings:

1
2
3
4
5
6
POST /api/admin/arrow/vendor-offering-mappings/
{
    "settings": "<settings-uuid>",
    "arrow_vendor_name": "Microsoft",
    "offering": "<offering-uuid>"
}

Use the vendor_choices endpoint to list available vendor names:

1
GET /api/admin/arrow/vendor-offering-mappings/vendor_choices/

Step 5: Sync Billing Data

Trigger a billing sync for a specific period:

1
2
3
4
5
POST /api/admin/arrow/billing-syncs/trigger_sync/
{
    "year": 2026,
    "month": 1
}

Step 6: Enable Consumption Tracking (Optional)

For real-time consumption tracking, set ARROW_CONSUMPTION_SYNC_ENABLED to True in Constance. This enables hourly consumption sync from the Arrow Consumption API.

Resources must have their backend_id field set to the Arrow license reference (e.g., XSP12345). The license reference is stored in the resource's arrow_license_reference attribute. Use the discover_licenses and link_resource actions on customer mappings to link resources.

During consumption sync, matching uses both License Reference and ARS Subscription ID fields from the Arrow billing export, so resources are found regardless of which field the export type populates.

API Endpoints

All endpoints require staff permissions and are located under /api/admin/arrow/.

Settings (/api/admin/arrow/settings/)

Standard CRUD plus:

Action Method Description
validate_credentials POST Test API credentials without saving
discover_customers POST Fetch Arrow customers with mapping suggestions
preview_settings POST Preview settings before saving
save_settings POST Save settings and optionally create customer mappings

Customer Mappings (/api/admin/arrow/customer-mappings/)

Standard CRUD plus:

Action Method Description
sync_from_arrow POST Bulk-sync customer data from Arrow API
billing_summary GET (detail) View billing summary for a mapped customer
fetch_arrow_data GET (detail) Fetch fresh billing and consumption data from Arrow
discover_licenses GET (detail) Discover Arrow licenses and suggest resource links
link_resource POST (detail) Link a Waldur resource to an Arrow license
import_license POST (detail) Import an Arrow license as a new Waldur resource
available_customers GET List unmapped Arrow customers with suggestions

Vendor Offering Mappings (/api/admin/arrow/vendor-offering-mappings/)

Standard CRUD plus:

Action Method Description
vendor_choices GET List available Arrow vendor names

Billing Syncs (/api/admin/arrow/billing-syncs/)

Read-only list/retrieve plus:

Action Method Description
trigger_sync POST Trigger billing sync for a specific month
reconcile POST Trigger reconciliation for a specific month
sync_resources POST Sync Arrow subscriptions to Waldur resources
trigger_consumption_sync POST Trigger consumption sync for a specific month
sync_resource_historical_consumption POST Sync historical consumption for a resource
trigger_reconciliation POST Trigger billing export check and reconciliation
cleanup_consumption POST Clean up consumption records
pause_sync POST Pause automatic sync
resume_sync POST Resume automatic sync
consumption_status GET View consumption sync status
consumption_statistics GET View consumption statistics
pending_records GET List pending consumption records
fetch_consumption POST Fetch raw consumption data from Arrow API
fetch_billing_export POST Fetch raw billing export from Arrow API
fetch_license_info POST Fetch license details from Arrow API

Consumption Records (/api/admin/arrow/consumption-records/)

Read-only list/retrieve. Filterable by resource_uuid, customer_uuid, project_uuid, license_reference, billing_period, is_finalized, and is_reconciled.

Billing Sync Items (/api/admin/arrow/billing-sync-items/)

Read-only list/retrieve. Filterable by billing_sync_uuid, report_period, vendor_name, classification, and has_compensation.

Periodic Tasks

The following Celery tasks run automatically:

Task Default Interval Description
sync-arrow-billing Every 6 hours Syncs billing export for the current month
check-arrow-validated-billing Every 12 hours Checks for validated billing and triggers reconciliation
sync-arrow-consumption Every 1 hour Syncs real-time consumption data (requires ARROW_CONSUMPTION_SYNC_ENABLED)
check-arrow-billing-export Every 6 hours Checks billing export and reconciles consumption records

All tasks check ArrowSettings.sync_enabled before running. Consumption tasks additionally check ARROW_CONSUMPTION_SYNC_ENABLED.

Management Commands

sync_arrow_resources

Sync Arrow IAAS subscriptions to Waldur resources from the command line:

1
waldur sync_arrow_resources

Arguments:

Argument Description
--period-from Start period in YYYY-MM format (default: 6 months ago)
--period-to End period in YYYY-MM format (default: current month)
--customer-uuid Waldur Customer UUID to create resources under
--project-uuid Waldur Project UUID to create resources under
--dry-run Preview changes without modifying the database
--create-offering Create an Arrow Azure offering if none exists
--force-import Auto-create Waldur Customers and Projects from Arrow data

Example - dry run:

1
waldur sync_arrow_resources --dry-run --period-from 2025-07 --period-to 2026-01

Example - force import:

1
waldur sync_arrow_resources --force-import

In force-import mode, each Arrow customer gets a Waldur Customer with an "Arrow Azure Subscriptions" project automatically created.

Invoice Price Source

The invoice_price_source setting controls which Arrow price is used for Waldur invoice items:

  • sell (default): Uses sell/customer prices from Arrow billing data
  • buy: Uses buy/wholesale prices from Arrow billing data

This affects:

  • Provisional invoice items created during consumption sync
  • Reconciliation adjustments when finalized billing arrives
  • Billing export invoice item creation

Billing Export Field Mapping

Different Arrow export types use different column names. The system handles this with fallback chains — it tries the first field name and falls back to alternatives:

Waldur Field Primary Column Fallback Column(s)
Line reference Line Reference Sequence → Order Id
Sell price Sell Total Price Customer Total Price
Buy price Buy Total Price Total Wholesale Price
Quantity Quantity Qty (defaults to 1)
Vendor Vendor Name Service Name
Product Product Name Friendly Name → Description
License reference License Reference ARS Subscription ID
Customer grouping Customer Reference End User Company Name (matched via arrow_company_name)

Classification Filter Fallback

When classification_filter is set (e.g., IAAS), the system first requests billing data filtered by classification. If the Arrow API rejects the filter (HTTP 400), the system automatically retries the request without the classification filter. This applies to:

  • Billing sync (sync_arrow_billing)
  • Billing reconciliation (reconcile_arrow_billing)
  • Consumption reconciliation (check_and_reconcile_billing)

Customer Grouping

Billing lines are grouped by customer using two strategies:

  1. By Customer Reference: Lines with a Customer Reference field are matched to ArrowCustomerMapping.arrow_reference
  2. By company name (fallback): If no lines match by reference, lines with an End User Company Name are matched to ArrowCustomerMapping.arrow_company_name

Billing Sync Lifecycle

Arrow billing sync records follow a finite-state machine:

graph LR
    A[Pending] --> B[Synced]
    B --> C[Validated]
    C --> D[Reconciled]
State Description
Pending Billing sync record created, waiting for data
Synced Billing data fetched from Arrow and invoice items created
Validated Arrow has validated the billing statement
Reconciled Compensation items applied for any price differences

Reconciliation

When billing moves from Synced to Validated, the system can automatically apply compensations if ARROW_AUTO_RECONCILIATION is enabled. Otherwise, use the reconcile action to trigger reconciliation manually.

During reconciliation, license references are matched using a fallback chain: first by License Reference, then by ARS Subscription ID. The price fields used for comparison depend on which columns are present in the export (see Field Mapping above).

Consumption-Based Reconciliation

When consumption tracking is enabled, the flow is:

  1. Hourly consumption data is synced from Arrow's Consumption API (provisional amounts)
  2. Invoice items are created/updated with consumed amounts (using the configured invoice_price_source)
  3. When Arrow's finalized billing export arrives, final amounts are compared
  4. If final amounts differ from consumed amounts, compensation items are created
  5. Licenses with zero consumption (both sell and buy are 0) are automatically skipped

Troubleshooting

No active Arrow settings found

Ensure an ArrowSettings record exists with is_active=True. Create one via the API or setup wizard.

Billing sync runs but creates no items

  • Verify customer mappings exist and are active
  • Check that export_type_reference is set correctly (use validate_credentials to discover types)
  • Confirm the classification_filter matches your subscriptions (default: IAAS). The system retries without the filter if the Arrow API rejects it, but verify in logs.
  • Check that arrow_company_name is set on customer mappings — some export types don't include Customer Reference and rely on company name matching instead
  • Different export types use different column names (e.g., Customer Total Price vs Sell Total Price). The system handles this with fallback chains, but check logs for warnings.

Consumption sync not running

  • Set ARROW_CONSUMPTION_SYNC_ENABLED to True in Constance
  • Ensure ArrowSettings.sync_enabled is True
  • Verify resources have backend_id set to Arrow license references

Resources not linked to Arrow licenses

Use the discover_licenses action on a customer mapping to find unlinked licenses, then use link_resource to set the backend_id on existing resources or import_license to create new ones.

API authentication failures

  • Verify the API key is valid using validate_credentials
  • Check that the Arrow API URL includes the correct base path
  • Review Waldur logs for detailed error messages from the Arrow API
  • Models: src/waldur_mastermind/waldur_arrow/models.py
  • Backend client: src/waldur_mastermind/waldur_arrow/backend.py
  • Views: src/waldur_mastermind/waldur_arrow/views.py
  • Tasks: src/waldur_mastermind/waldur_arrow/tasks.py
  • Extension: src/waldur_mastermind/waldur_arrow/extension.py
  • CLI command: src/waldur_mastermind/waldur_arrow/management/commands/sync_arrow_resources.py