Quotas Application
Overview
The Quotas application is a Django app that provides generic implementation of quotas tracking functionality for Waldur:
- Store and query resource limits and usages for project, customer or any other model
- Aggregate quota usage in object hierarchies
- Provide concurrent-safe quota updates using delta-based storage
- Support multiple quota field types for different use cases
Architecture
Core Models
QuotaLimit
- Stores quota limit values for different scopes
- Uses generic foreign key to relate to any model instance
- Unique constraint on (name, content_type, object_id)
- Default value of -1 indicates unlimited quota
QuotaUsage
- Stores quota usage deltas instead of absolute values
- Enables concurrent updates without deadlocks
- Aggregated using SUM queries to get current usage
- Uses generic foreign key pattern for scope association
QuotaModelMixin
- Base mixin for models that need quota functionality
- Provides core quota management methods
- Defines abstract Quotas inner class for field definitions
- Includes property accessors for quotas, quota_usages, quota_limits
ExtendableQuotaModelMixin
- Extends QuotaModelMixin for runtime quota field addition
- Disables field caching to support dynamic fields
- Used when quota fields need to be added programmatically
Quota Field Types
QuotaField
- Base quota field class
- Configurable default limits and backend flags
- Optional creation conditions for conditional quota assignment
- Supports callable default values:
QuotaField(default_limit=lambda scope: scope.attr)
CounterQuotaField
- Automatically tracks count of target model instances
- Increases/decreases usage on target model creation/deletion
- Configurable delta calculation via
get_delta
function - Example:
nc_resource_count
tracks total resources in a project
TotalQuotaField
- Aggregates sum of specific field values from target models
- Useful for tracking total storage size, RAM allocation, etc.
- Extends CounterQuotaField with field-specific aggregation
- Example:
nc_volume_size
sums all volume sizes in a project
UsageAggregatorQuotaField
- Aggregates quota usage from child objects with same quota name
- Enables hierarchical quota tracking (customer ← project ← resource)
- Configurable child quota name mapping
- Example: Customer's
nc_resource_count
aggregates from all projects
Signal Handling and Automation
Quota Handlers
count_quota_handler_factory
: Creates handlers for CounterQuotaField automationhandle_aggregated_quotas
: Manages usage aggregation across hierarchiesget_ancestors
: Safely traverses object relationships with depth limitsdelete_quotas_when_model_is_deleted
: Cleanup on model deletion
Signal Registration
- Automatically registers signals for CounterQuotaField instances
- Connects aggregation handlers to QuotaUsage model signals
- Handles project customer changes for quota recalculation
Concurrency Safety
Delta-Based Storage
The quota system uses INSERT operations instead of UPDATE to avoid deadlocks:
- Usage deltas are stored in QuotaUsage records
- Current usage calculated via SUM aggregation
- Multiple concurrent requests can safely add usage deltas
- Prevents shared write deadlocks in high-concurrency scenarios
Transaction Safety
set_quota_usage
uses@transaction.atomic
decorator- Quota validation can be enabled per operation
- Safe quota changes through
apply_quota_usage
method
Define Quota Fields
Models with quotas should inherit QuotaModelMixin
and define a Quotas
inner class:
1 2 3 4 5 6 7 |
|
Real-World Examples
Customer Quotas
1 2 3 4 5 6 7 8 9 10 11 |
|
Project Quotas
1 2 3 4 5 6 |
|
Quota Operations
Basic Operations
get_quota_limit(quota_name)
- Get current limit (returns -1 for unlimited)set_quota_limit(quota_name, limit)
- Set new quota limitget_quota_usage(quota_name)
- Get current usage (SUM of deltas)set_quota_usage(quota_name, usage)
- Set absolute usage valueadd_quota_usage(quota_name, delta, validate=False)
- Add delta to usage
Bulk Operations
apply_quota_usage(quota_deltas)
- Apply multiple quota deltas atomicallyvalidate_quota_change(quota_deltas)
- Validate quota changes before applying
Property Access
quotas
- List of all quotas with name, usage, limitquota_usages
- Dictionary of current usage valuesquota_limits
- Dictionary of current limit values
Quota Validation
Use validate_quota_change()
to check if quota changes would exceed limits:
1 2 3 4 5 |
|
Shared Quota Resources
For resources that affect multiple quota scopes, implement SharedQuotaMixin
:
1 2 3 4 5 6 7 8 9 10 |
|
Background Tasks
Celery Tasks
update_custom_quotas()
- Triggers custom quota recalculation signalupdate_standard_quotas()
- Recalculates all standard quota fields
These tasks enable periodic quota synchronization and can be scheduled via cron.
Performance Considerations
Hierarchy Traversal
get_ancestors()
includes depth limits (max_depth=10) to prevent infinite recursion- Handles deletion scenarios gracefully with ObjectDoesNotExist catching
- Uses sets to eliminate duplicate ancestors in complex hierarchies
Deletion Optimization
- Skips aggregation during bulk deletion (project deletion scenarios)
- Uses
_deleting
flag to avoid timeout issues - Automatically cleans up quota records on model deletion
Query Optimization
- Uses
Sum()
aggregation for efficient usage calculation - Generic foreign keys enable single tables for all quota types
- Field caching can be disabled for dynamic quota scenarios
Error Handling
Exception Types
QuotaError
- Base quota system exceptionQuotaValidationError
- Extends DRF ValidationError for quota limit violations
Graceful Degradation
- Missing relationships during deletion are safely ignored
- Invalid scopes return empty quota collections
- Failed quota operations don't break primary workflows
Integration Points
Structure Integration
- Customer and Project models include standard quota definitions
- Project movement between customers triggers quota recalculation
- User count and resource count quotas are tracked automatically
Plugin Integration
recalculate_quotas
signal allows plugin-specific quota logic- Backend quota synchronization through plugin-specific handlers
- Resource-specific quota fields defined in individual plugins
Usage Workflow
Standard Workflow
- Quota Allocation: Increase usage when resource allocation begins
- Validation: Check quota limits before proceeding with operations
- Backend Sync: Pull actual usage from backends periodically
- Cleanup: Decrease usage only when backend deletion succeeds
Error Recovery
- Frontend quota not modified if backend API calls fail
- Quota pulling (sync) handles discrepancies
- Manual recalculation available via management commands
Sort Objects by Quotas
Inherit your FilterSet
from QuotaFilterMixin
and add quota ordering:
1 2 |
|
Ordering can be done only by one quota at a time.