Scale, add, remove racks¶

How scaling works¶

Each rack in a ScyllaDB cluster maps to a single Kubernetes StatefulSet. Scaling changes the replica count of that StatefulSet:

• Scale up — new pods are appended at the end of the ordinal sequence (highest index). After the new node joins the token ring, the Operator automatically triggers a data cleanup on affected nodes.

• Scale down — the Operator decommissions the highest-ordinal pod first, streams its data to the remaining nodes, reduces the replica count, and then deletes the PVC and Service. Only one node is decommissioned at a time.

Because StatefulSets maintain contiguous pod ordinals and scale down from the highest ordinal, you cannot remove an arbitrary node from the middle of a rack. If a specific node is unhealthy, use node replacement instead.

For background on the StatefulSet-per-rack architecture, see StatefulSets and racks.

Sequential and parallel node provisioning¶

The Operator controls how it provisions new nodes with parallel node operations. They determine whether Operator starts ScyllaDB nodes one at a time or all at once: when you create a cluster, add, or scale-out a rack.

With parallel node operations disabled, Operator starts ScyllaDB nodes one at a time. Within a rack, each Pod must become ready before Operator starts the next one. Operator creates racks one at a time. Bringing up a cluster takes as long as the sum of every node’s startup time.

With parallel node operations enabled, Operator starts all ScyllaDB nodes at once. Operator starts Pods of a rack without waiting for the previous ones to become ready. Operator creates all racks at the same time. Bringing up a cluster is faster, and the difference grows with the number of nodes.

Parallel node operations provide better performance when your keyspaces are backed by tablets (the default since ScyllaDB 2025.2), as opposed to vnodes. Therefore, we strongly recommend that you only use tablets for your data keyspaces. This is because with vnode-based keyspaces, a joining node streams its data before it finishes joining, which slows down bringing up new nodes. With tablets, the data is moved in the background after the node joins, so the time to bring up new nodes is not affected by data streaming. Consider setting tablets_mode_for_new_keyspaces to enforced in your ScyllaDB configuration to prevent individual keyspaces from opting out of tablets.

apiVersion: scylla.scylladb.com/v1
kind: ScyllaCluster
metadata:
  name: scylla
  namespace: scylla
spec:
  enableParallelNodeOperations: true

Bootstrap synchronisation¶

In Kubernetes, Pods can start simultaneously, and a new node could attempt to bootstrap while another node is still restarting and appears down to its peers. ScyllaDB denies such a join request and leaves the new node in a state that is not recoverable automatically.

Enabling the BootstrapSynchronisation feature gate protects against this by holding each node’s startup until all nodes in the cluster are UP. It is recommended that you enable it whether or not parallel node operations are enabled. See Bootstrap synchronisation for details on the mechanism and Feature gates for instructions on enabling feature gates.

Scale a ScyllaCluster¶

Change spec.datacenter.racks[].members to the desired node count and apply:

apiVersion: scylla.scylladb.com/v1
kind: ScyllaCluster
metadata:
  name: scylla
  namespace: scylla
spec:
  datacenter:
    name: us-east-1
    racks:
      - name: us-east-1a
        members: 3          # was 1, now 3
        storage:
          capacity: 500Gi

apiVersion: scylla.scylladb.com/v1
kind: ScyllaCluster
metadata:
  name: scylla
  namespace: scylla
spec:
  datacenter:
    name: us-east-1
    racks:
      - name: us-east-1a
        members: 1          # was 3, now 1
        storage:
          capacity: 500Gi

Wait for the operation to complete:

kubectl -n scylla wait --timeout=10m --for='condition=Available' scyllaclusters.scylla.scylladb.com/scylla

Verify with nodetool status:

kubectl -n scylla exec -it scylla-us-east-1a-0 -c scylla -- nodetool status

Add a rack to a ScyllaCluster¶

Append a new entry to the spec.datacenter.racks array. The Operator creates racks in the order they appear and waits for each rack to be fully ready before creating the next.

apiVersion: scylla.scylladb.com/v1
kind: ScyllaCluster
metadata:
  name: scylla
  namespace: scylla
spec:
  datacenter:
    name: us-east-1
    racks:
      - name: us-east-1a
        members: 3
        storage:
          capacity: 500Gi
      - name: us-east-1b         # new rack
        members: 3
        storage:
          capacity: 500Gi

kubectl -n scylla patch scyllacluster scylla --type=json \
  -p='[{"op":"replace","path":"/spec/datacenter/racks/<index>/members","value":0}]'

kubectl -n scylla wait --timeout=30m \
  --for='condition=Available=True' scyllacluster/scylla

kubectl -n scylla get pods -l scylla/rack=<rack-name>

kubectl -n scylla edit scyllacluster scylla

kubectl -n scylla wait --timeout=5m \
  --for='condition=Available=True' scyllacluster/scylla

Key considerations¶

Related pages¶

• StatefulSets and racks — how StatefulSets map to racks and why mid-set removal is not possible

• Bootstrap synchronisation — the mechanism gating bootstrap on the cluster’s node statuses

• Replace nodes — replacing a specific unhealthy node without scaling

• Migrate a rack to a new node pool — scaling up a new rack and scaling down the old one to migrate infrastructure

• Perform a rolling restart — restarting all nodes without changing the cluster size

• Data distribution with tablets — how tablets distribute data and why they speed up topology changes