Was this page helpful?
Caution
You're viewing documentation for a deprecated version of ScyllaDB Operator. Switch to the latest stable version.
Exposing ScyllaDB cluster¶
This document explains how ScyllaDB Operator exposes ScyllaDB clusters in different network setups. A ScyllaDB cluster can be exposed in various network configurations, independently to clients and nodes.
Expose Options¶
Note
exposeOptions are immutable, they cannot be changed after a ScyllaDB cluster is created.
exposeOptions specifies configuration options for exposing ScyllaCluster’s.
A ScyllaCluster created without any exposeOptions is equivalent to the following:
The following sections cover what every field controls and what the configuration options are.
Node Service Template¶
nodeService serves as a template for a node-dedicated Service managed by the Scylla Operator for each node within a ScyllaDB cluster.
The properties of the Services depend on the selected type.
Additionally, there’s an option to define custom annotations, incorporated into each node’s Service,
which might be useful for further tweaking the Service properties or related objects.
Headless Type¶
For Headless type, Scylla Operator creates a Headless Service with a selector pointing to the particular node in the ScyllaDB cluster.
Such Service doesn’t provide any additional IP addresses, and the internal DNS record resolves to the PodIP of a node.
This type of Service is useful when ScyllaDB cluster nodes broadcast PodIPs to clients and other nodes.
Example:
ClusterIP Type¶
For ClusterIP type, Scylla Operator creates a ClusterIP Service backed by a specific node in the ScyllaDB cluster.
These IP addresses are only routable within the same Kubernetes cluster, so it’s a good fit, if you don’t want to expose them to other networks.
Example:
LoadBalancer Type¶
For the LoadBalancer type, Scylla Operator generates a LoadBalancer Service that directs traffic to a specific node within the ScyllaCluster.
On platforms with support for external load balancers, this Service provisions one.
The accessibility of this load balancer’s address depends on the platform and any customizations made; in some cases it may be reachable from the internal network or public Internet.
Customizations are usually managed via Service annotations, key-value pairs provided in annotations field are merged into each Service object.
LoadBalancer Services should be configured to pass through entire traffic.
For example, to expose LoadBalancer only to internal network use the following annotations:
Check platform-specific documentation regarding LoadBalancer configuration to learn more about available options.
LoadBalancer Service is a superset of ClusterIP Service, implying that each LoadBalancer Service also contains an allocated ClusterIP. They can be configured using the following fields, which propagate to every node Service:
externalTrafficPolicy
internalTrafficPolicy
loadBalancerClass
allocateLoadBalancerNodePorts
Check Kubernetes Service documentation to learn more about these options.
Example:
Broadcast Options¶
Broadcast options control what is the source of the address being broadcasted to clients and nodes. It’s configured independently for clients and nodes because you may want to expose these two types of traffic on different networks. Using different networks can help manage costs, reliability, latency, security policies or other metrics you care about.
PodIP Type¶
Address broadcasted to clients/nodes is taken from Pod.
By default, the address is taken from Pod’s status.PodIP field.
Because a Pod can use multiple address, you may want to provide source options by specifying podIP.source.
Example:
ServiceClusterIP Type¶
Address broadcasted to clients or nodes is taken from spec.ClusterIP field of a node’s dedicated Service.
In order to configure it, the nodeService template must specify a Service having a ClusterIP assigned.
Example:
ServiceLoadBalancerIngress Type¶
Address broadcasted to clients/nodes is taken from the node dedicated Service, from status.ingress[0].ipAddress or status.ingress[0].hostname field.
In order to configure it, the nodeService template must specify the LoadBalancer Service.
Example:
Deployment Examples¶
The following section contains several specific examples of various network scenarios and explains how nodes and clients communicate with one another.
In-cluster only¶
Both client and nodes are deployed within the same Kubernetes cluster. They talk through ClusterIP addresses taken from the Service. Because ClusterIP Services are only routable within the same Kubernetes cluster, this cluster won’t be reachable from outside.
In-cluster node-to-node, VPC clients-to-nodes¶
In this scenario, we assume that the Pod IP subnet is routable within a VPC. Clients within the VPC network can communicate directly with ScyllaCluster nodes using PodIPs. Nodes communicate with each other exclusively within the same Kubernetes cluster.
Multi VPC¶
In this scenario, we set up two separate Kubernetes clusters in distinct VPCs. These VPCs are interconnected to facilitate inter-VPC connectivity. We operate on the assumption that the Pod IP subnet is routable within each VPC.
Clients, whether deployed within the same Kubernetes cluster or within a VPC, have the capability to reach nodes using their Pod IPs.
Since there is no requirement for any address other than the Pod IP, the Headless service type is sufficient.
Internet¶
We assume that a Kubernetes cluster has been deployed in a cloud provider environment that supports external load balancers. By specifying the LoadBalancer type in the nodeService template, the Scylla Operator generates a dedicated LB Service for each node. The cloud provider then establishes an external load balancer with an internet-accessible address. ScyllaDB nodes broadcast this external address to clients, enabling drivers to connect and discover other nodes. Since all ScyllaDB nodes reside within the same Kubernetes cluster, there is no need to route traffic through the internet. Consequently, the nodes are configured to communicate via ClusterIP, which is also accessible within LoadBalancer Services.
Other more complex scenarios can be built upon these simple ones.