Was this page helpful?
This document explains how ScyllaDB Operator exposes ScyllaClusters in different network setups. A ScyllaCluster can be exposed in various network configurations, independently to clients and nodes.
Note
ScyllaClusters can be only exposed when the ScyllaDB version used version is >=2023.1 ScyllaDB Enterprise or >=5.2 ScyllaDB Open Source.
exposeOptions specifies configuration options for exposing ScyllaCluster’s.
A ScyllaCluster created without any exposeOptions is equivalent to the following:
apiVersion: scylla.scylladb.com/v1
kind: ScyllaCluster
spec:
exposeOptions:
nodeService:
type: ClusterIP
broadcastOptions:
clients:
type: ServiceClusterIP
nodes:
type: ServiceClusterIP
The following sections cover what every field controls and what the configuration options are.
nodeService serves as a template for a node-dedicated Service managed by the Scylla Operator for each node within a ScyllaCluster.
The properties of the Services depend on the selected type.
Additionally, there’s an option to define custom annotations, incorporated into each node’s Service,
which might be useful for further tweaking the Service properties or related objects.
For Headless type, Scylla Operator creates a Headless Service with a selector pointing to the particular node in the ScyllaCluster.
Such Service doesn’t provide any additional IP addresses, and the internal DNS record resolves to the PodIP of a node.
This type of Service is useful when ScyllaCluster nodes broadcast PodIPs to clients and other nodes.
Example:
apiVersion: scylla.scylladb.com/v1
kind: ScyllaCluster
spec:
exposeOptions:
nodeService:
type: Headless
For ClusterIP type, Scylla Operator creates a ClusterIP Service backed by a specific node in the ScyllaCluster.
These IP addresses are only routable within the same Kubernetes cluster, so it’s a good fit, if you don’t want to expose them to other networks.
Example:
apiVersion: scylla.scylladb.com/v1
kind: ScyllaCluster
spec:
exposeOptions:
nodeService:
type: ClusterIP
For the LoadBalancer type, Scylla Operator generates a LoadBalancer Service that directs traffic to a specific node within the ScyllaCluster.
On platforms with support for external load balancers, this Service provisions one.
The accessibility of this load balancer’s address depends on the platform and any customizations made; in some cases it may be reachable from the internal network or public Internet.
LoadBalancer Service is a superset of ClusterIP Service, implying that each LoadBalancer Service also contains an allocated ClusterIP. They can be configured using the following fields, which propagate to every node Service:
externalTrafficPolicy
internalTrafficPolicy
loadBalancerClass
allocateLoadBalancerNodePorts
Check Kubernetes Service documentation to learn more about these options.
Example:
apiVersion: scylla.scylladb.com/v1
kind: ScyllaCluster
spec:
exposeOptions:
nodeService:
type: LoadBalancer
loadBalancerClass: my-custom-load-balancer-class
Broadcast options control what is the source of the address being broadcasted to clients and nodes. It’s configured independently for clients and nodes because you may want to expose these two types of traffic on different networks. Using different networks can help manage costs, reliability, latency, security policies or other metrics you care about.
Address broadcasted to clients/nodes is taken from Pod.
By default, the address is taken from Pod’s status.PodIP field.
Because a Pod can use multiple address, you may want to provide source options by specifying podIP.source.
Example:
apiVersion: scylla.scylladb.com/v1
kind: ScyllaCluster
spec:
exposeOptions:
broadcastOptions:
clients:
type: PodIP
podIP:
source: Status
Address broadcasted to clients or nodes is taken from spec.ClusterIP field of a node’s dedicated Service.
In order to configure it, the nodeService template must specify a Service having a ClusterIP assigned.
Example:
apiVersion: scylla.scylladb.com/v1
kind: ScyllaCluster
spec:
exposeOptions:
broadcastOptions:
clients:
type: ServiceClusterIP
Address broadcasted to clients/nodes is taken from the node dedicated Service, from status.ingress[0].ipAddress or status.ingress[0].hostname field.
In order to configure it, the nodeService template must specify the LoadBalancer Service.
Example:
apiVersion: scylla.scylladb.com/v1
kind: ScyllaCluster
spec:
exposeOptions:
broadcastOptions:
clients:
type: ServiceLoadBalancerIngress
podIP:
source: Status
The following section contains several specific examples of various network scenarios and explains how nodes and clients communicate with one another.
ScyllaCluster definition:
apiVersion: scylla.scylladb.com/v1
kind: ScyllaCluster
spec:
exposeOptions:
nodeService:
type: ClusterIP
broadcastOptions:
clients:
type: ServiceClusterIP
nodes:
type: ServiceClusterIP
Both client and nodes are deployed within the same Kubernetes cluster. They talk through ClusterIP addresses taken from the Service. Because ClusterIP Services are only routable within the same Kubernetes cluster, this cluster won’t be reachable from outside.
ScyllaCluster definition:
apiVersion: scylla.scylladb.com/v1
kind: ScyllaCluster
spec:
exposeOptions:
nodeService:
type: ClusterIP
broadcastOptions:
clients:
type: PodIP
nodes:
type: ServiceClusterIP
In this scenario, we assume that the Pod IP subnet is routable within a VPC. Clients within the VPC network can communicate directly with ScyllaCluster nodes using PodIPs. Nodes communicate with each other exclusively within the same Kubernetes cluster.
ScyllaCluster definition:
apiVersion: scylla.scylladb.com/v1
kind: ScyllaCluster
spec:
exposeOptions:
nodeService:
type: Headless
broadcastOptions:
clients:
type: PodIP
nodes:
type: PodIP
In this scenario, we set up two separate Kubernetes clusters in distinct VPCs. These VPCs are interconnected to facilitate inter-VPC connectivity. We operate on the assumption that the Pod IP subnet is routable within each VPC.
Both ScyllaClusters use the same exposeOptions, nodes broadcast their Pod IP addresses, enabling them to establish connections with one another.
****Check other documentation pages to know how to connect two ScyllaClusters into one logical cluster.
Clients, whether deployed within the same Kubernetes cluster or within a VPC, have the capability to reach nodes using their Pod IPs.
Since there is no requirement for any address other than the Pod IP, the Headless service type is sufficient.
ScyllaCluster definition:
apiVersion: scylla.scylladb.com/v1
kind: ScyllaCluster
spec:
exposeOptions:
nodeService:
type: LoadBalancer
broadcastOptions:
clients:
type: ServiceLoadBalancerIngress
nodes:
type: ClusterIP
We assume that a Kubernetes cluster has been deployed in a cloud provider environment that supports external load balancers. By specifying the LoadBalancer type in the nodeService template, the Scylla Operator generates a dedicated LB Service for each node. The cloud provider then establishes an external load balancer with an internet-accessible address. ScyllaDB nodes broadcast this external address to clients, enabling drivers to connect and discover other nodes. Since all ScyllaDB nodes reside within the same Kubernetes cluster, there is no need to route traffic through the internet. Consequently, the nodes are configured to communicate via ClusterIP, which is also accessible within LoadBalancer Services.
Other more complex scenarios can be built upon these simple ones.
Was this page helpful?