Bucket Rate Limits
A bucket rate limit caps how many S3 requests, and how many bytes, the object gateway serves for an Object Bucket each minute. This page covers applying limits to buckets and keeping reusable rate-limit policies.
How bucket rate limits work
QuantaStor sets the limits on the object gateway with the Ceph RGW rate limiter, at bucket scope. There are four limits, each counted per minute:
| Limit | Counts |
|---|---|
| Read Operations/min | GET requests |
| Write Operations/min | PUT requests |
| Read Bandwidth/min | Bytes read |
| Write Bandwidth/min | Bytes written |
A limit of 0 means unlimited. When all four are 0, QuantaStor turns rate limiting off for the bucket rather than leaving it on with no limits.
Each object gateway enforces the limits on its own. A bucket served through three gateways behind a load balancer can take up to three times the configured rate in total. Divide the rate you want by the number of gateways that serve the bucket.
A client that exceeds a limit has its requests refused by the gateway until the next minute begins.
Applying a rate limit to buckets
Select one or more buckets in the grid, then either pick a saved policy in Policy Preset or set the four limits directly under Rate Limit Settings. With a preset selected, it is the policy's saved values that are applied, whatever the four fields show: changes made to the fields while a preset is selected are ignored. To apply different values, edit the policy first with Manage Bucket Rate Policies..., or reopen the dialog and set the fields without choosing a preset.
| Field | Default | Notes |
|---|---|---|
| Ceph Cluster and bucket search | first cluster | Narrows the bucket grid to one Ceph cluster. All the buckets in one apply must be on the same cluster. |
| Bucket grid | Select one or more buckets. The grid shows each bucket's current limits. | |
| Policy Preset | (none) | A saved rate-limit policy. Manage Bucket Rate Policies... opens the policy manager without closing this dialog. |
| Read Operations/min | 0 | GET requests per minute. The slider runs to 1000. |
| Write Operations/min | 0 | PUT requests per minute. The slider runs to 1000. |
| Read Bandwidth/min | 0 | Bytes read per minute, with a unit selector. The slider covers 1 to 1024 of the selected unit; type a larger value. |
| Write Bandwidth/min | 0 | Bytes written per minute, as above. |
The limits are copied to each bucket when you apply them. The bucket does not keep a link to the policy they came from, so changing or deleting a policy later does not change any bucket; apply it again to update them.
To remove a bucket's limits, apply all four as 0.
Current limits are also read back from the gateway by Rescan Object Storage Buckets with Deep Scan and Enable Rate Limit Updates selected, which picks up limits set outside QuantaStor.
Managing rate-limit policies
A policy is a named set of the four limits, kept so the same limits can be applied to many buckets consistently. In Policy, choose an existing policy to edit it or < Create New Policy > to make one, then set its Name and the four limits. Apply saves the policy and keeps the dialog open; Delete removes the selected policy.
Policy names may contain letters, digits, -, _ and .. Policies are held in the grid, not on a cluster, so one policy can be applied to buckets on any Ceph cluster in the grid.
Managing rate limits from the CLI
| Command | Purpose |
|---|---|
| qs bucket-ratelimit-apply | Apply a policy (--ratelimit-policy) or inline limits to the buckets in --bucket-list
|
| qs bucket-ratelimit-policy-create | Create a policy |
| qs bucket-ratelimit-policy-modify | Change a policy's name or limits |
| qs bucket-ratelimit-policy-get | Show one policy |
| qs bucket-ratelimit-policy-list | List the policies |
| qs bucket-ratelimit-policy-delete | Delete a policy |
The limits are --max-read-ops, --max-write-ops, --max-read-bytes and --max-write-bytes, all per minute, with bandwidth in bytes. Without --ratelimit-policy, bucket-ratelimit-apply applies the limits given on the command line.
qs bucket-ratelimit-policy-create --name=tenant-standard --max-read-ops=6000 --max-write-ops=3000 --max-read-bytes=6442450944 --max-write-bytes=3221225472 qs bucket-ratelimit-apply --ratelimit-policy=tenant-standard --bucket-list=media,backups qs bucket-ratelimit-apply --bucket-list=media --max-read-ops=0 --max-write-ops=0 --max-read-bytes=0 --max-write-bytes=0
The last command removes the limits from the bucket.
Related pages
- Scale-out Object Setup (ceph) -- object gateways, users and buckets
- Bucket Lifecycle Policy -- moving and expiring objects by age
- Object Storage Classes and Cloud Tiering -- storage classes and tiering objects to the cloud
Verified against QuantaStor 6.9.0.