# K8s - Features

There are 2 possible topologies of K8s cluster provided by **cegedim.cloud**:

* **Standard**: workloads are deployed in a single data center, but protected against data center disaster using a secondary data center as failover.
* **High Availability**: workloads are deployed amongst two datacenters. If multi-replica is used and workload is well distributed, no service interruption occurs when a datacenter is down.

## Topologies <a href="#kubernetesarchitecture-topologies" id="kubernetesarchitecture-topologies"></a>

### Compute topology <a href="#kubernetesarchitecture-computetopology" id="kubernetesarchitecture-computetopology"></a>

**cegedim.cloud** provides a compute topology based on :

* Region : a pair of data centers
* Area : infrastructure network isolation between tenants
* Availability Zones : inside an area, isolated infrastructure for Compute and Storage

<figure><picture><source srcset="/files/pcoiqoZm9tXqRJn1EvE3" media="(prefers-color-scheme: dark)"><img src="/files/PbTeWMXwg18t8SU204Zc" alt="" width="563"></picture><figcaption><p>Compute topology</p></figcaption></figure>

### Kubernetes clusters topologies <a href="#kubernetesarchitecture-kubernetesclusterstopologies" id="kubernetesarchitecture-kubernetesclusterstopologies"></a>

Kubernetes clusters can be deployed using 2 topologies :

<table data-full-width="true"><thead><tr><th>Topology</th><th>Datacenters of Masters</th><th>Datacenters of workers</th><th>Worker Availability Zones</th><th>Cluster Availability Zones</th><th data-type="checkbox">Disaster Recovery Protection</th><th>Recovery Time Objective (RTO)</th></tr></thead><tbody><tr><td>Standard</td><td>1</td><td>1</td><td>2</td><td>2</td><td>true</td><td>4h</td></tr><tr><td>High Availability</td><td>3</td><td>2</td><td>3</td><td>4</td><td>true</td><td>0 - 15 min</td></tr></tbody></table>

Based on your requirements in terms of RTO and costs, you can choose the best topology for your needs.

### Availability of topologies

<table data-full-width="true"><thead><tr><th>Topology</th><th data-type="checkbox">EB-EMEA</th><th data-type="checkbox">EB-HDS</th><th data-type="checkbox">ET-EMEA</th><th data-type="checkbox">ET-HDS</th></tr></thead><tbody><tr><td>Standard</td><td>true</td><td>true</td><td>true</td><td>true</td></tr><tr><td>High Availability</td><td>true</td><td>false</td><td>true</td><td>false</td></tr></tbody></table>

For more details about the High Availability topology, please follow this page [High Availability](/compute/containers-k8s/k8s-get-started/high-availability.md#title-text).

### Topology Keys <a href="#kubernetesarchitecture-topologykeys" id="kubernetesarchitecture-topologykeys"></a>

**cegedim.cloud** uses standard topology keys :

<table><thead><tr><th width="454">Key</th><th>Component</th></tr></thead><tbody><tr><td>topology.kubernetes.io/region</td><td>Region</td></tr><tr><td>topology.kubernetes.io/zone</td><td>Availability Zone</td></tr><tr><td>kubernetes.io/hostname</td><td>FQDN of node</td></tr></tbody></table>

## Components and Versions <a href="#kubernetesarchitecture-componentsandversions" id="kubernetesarchitecture-componentsandversions"></a>

**cegedim.cloud** uses **RKE2** (Rancher Kubernetes Engine 2) as the Kubernetes distribution. RKE2 is a fully conformant Kubernetes distribution that focuses on security and compliance within the U.S. Federal Government sector.

Here is the list of the components and tools that are deployed in a standard delivered cluster:

| Features                | Versions                                          |
| ----------------------- | ------------------------------------------------- |
| Kubernetes Distribution | RKE2                                              |
| Rancher                 | 2.12                                              |
| Kubernetes              | 1.33                                              |
| Ingress controllers     | ingress-nginx 1.12.1, traefik 3.3.4, istio 1.24.1 |
| Prometheus              | 2.53.1                                            |
| Grafana                 | 11.1.0                                            |
| Helm                    | 3.17.0                                            |
| CSI Ceph                | 3.14.0                                            |
| Node OS                 | Ubuntu 24.04                                      |

## Network Architecture <a href="#kubernetesarchitecture-networkarchitecture" id="kubernetesarchitecture-networkarchitecture"></a>

{% hint style="info" %}
The following network architecture is described using **Nginx ingress controller**. The configuration is slightly different with Traefik or Istio, but the overall architecture and concepts remain the same.
{% endhint %}

Here is a figure with all network components explained:

### Outbound flow <a href="#kubernetesarchitecture-outboundflow" id="kubernetesarchitecture-outboundflow"></a>

<figure><img src="/files/fvelPWRzWxSpw4i0Hjk9" alt=""><figcaption></figcaption></figure>

* Two pods of 2 namespaces that belong to the same Rancher Project can fully communicate between them.
* Two pods of 2 namespaces that belong to two different Rancher Project cannot communicate unless user defines a Network Policy dedicated for this need.
* Pods from Rancher Project named System can communicate to pods from other Rancher Projects.
* Pods can only send requests to servers of the same VLAN, unless a specific network opening rule is configured between the two VLANs.
* Pods cannot send requests to Internet unless, a proxy is setup inside the pod or specific network opening rule is configured for the related VLAN.

### Inbound flow <a href="#kubernetesarchitecture-inboundflow" id="kubernetesarchitecture-inboundflow"></a>

<figure><img src="/files/xxGwNAWYh8gACD8CqakG" alt=""><figcaption></figcaption></figure>

* Requests toward kube api-server can be reverse-proxied by Rancher URL.
* Workload hosted by pods cannot be directly accessible from outside of K8S cluster, but via ingress layer for HTTP protocol or via a NodePort service for TCP protocol with a respective Load Balancer.

### Ingress Controller : nginx <a href="#kubernetesarchitecture-ingresscontroller-nginx" id="kubernetesarchitecture-ingresscontroller-nginx"></a>

nginx is the ingress controller deployed to expose your workloads. You can find relevant documentation on official Github.

{% embed url="<https://github.com/nginxinc/kubernetes-ingress>" %}

Two ingress controllers are deployed:

* One exposing to internal Cegedim Network
  * Workload name: nginx-int-ingress-nginx-controller
  * Listen on every ingress node at the port :80
  * Ingress class: "nginx" (default, so no ingress class needs to be specified)
* One exposing to internet
  * Workload name: nginx-ext-ingress-nginx-controller
  * Listen to every ingress node at the port :8081
  * Ingress class: nginx-ext
    * using the annotation: kubernetes.io/ingress.class: "nginx-ext"

### Load Balancing, DNS and Certificates <a href="#kubernetesarchitecture-loadbalancing-dnsandcertificates" id="kubernetesarchitecture-loadbalancing-dnsandcertificates"></a>

A K8s cluster comes with :

* An Elastic Secured Endpoint, managed by F5 appliances, exposing the K8s workload to the cegedim internal network (once you're connected to Cegedim LAN, either physically or through VPN)
* A \*.\<yourclustername>.ccs.cegedim.cloud DNS resolution to this endpoint
* A \*.\<yourclustername>.ccs.cegedim.cloud SSL certificate configured

### Requesting specific configuration <a href="#kubernetesarchitecture-requestingspecificconfiguration" id="kubernetesarchitecture-requestingspecificconfiguration"></a>

You can use ITCare in case of a need of a specific configuration :

* Exposing your workloads to the Internet or private link
* Using a specific FQDN to deploy your workload
* Using a specific certificate to deploy your workload
* Using Traefik as Ingress Provider instead of nginx
* Adding other Ingress Providers
* Accessing resources outside of cluster

## Cluster Customization Options <a href="#kubernetesarchitecture-clustercustomizationoptions" id="kubernetesarchitecture-clustercustomizationoptions"></a>

When creating a new Kubernetes cluster, **cegedim.cloud** provides you with the flexibility to customize key networking components according to your specific requirements and workload characteristics.

### CNI Provider Selection <a href="#kubernetesarchitecture-cniproviderselection" id="kubernetesarchitecture-cniproviderselection"></a>

You can select from the following Container Network Interface (CNI) providers when provisioning your cluster:

| CNI Provider | Description                                                                               | Maximum Nodes |
| ------------ | ----------------------------------------------------------------------------------------- | ------------- |
| **Canal**    | Combination of Calico and Flannel, providing policy enforcement and simplified networking | 200 nodes     |
| **Calico**   | Advanced networking and network policy solution with high scalability                     | 2,000 nodes   |
| **Cilium**   | eBPF-based networking, observability, and security with high performance                  | 2,000 nodes   |

{% hint style="info" %}
The CNI provider selection should be based on your cluster size requirements and specific networking needs. For clusters requiring more than 200 nodes, Calico or Cilium is recommended.
{% endhint %}

{% hint style="warning" %}
When using **Cilium** as the CNI provider, **kube-proxy** is not deployed. Cilium replaces kube-proxy functionality with its eBPF-based implementation, providing enhanced performance and efficiency.
{% endhint %}

#### When to Choose Each CNI Provider

**Canal (Calico + Flannel)**

* Standard deployments requiring up to 200 nodes
* Proven stability with simplified networking
* FIPS 140-2 compliance requirements
* Default choice for most use cases

**Calico**

* Large-scale deployments (200-2,000 nodes)
* Advanced network policy requirements
* High scalability requirements

**Cilium**

* High-performance workloads requiring maximum throughput
* Advanced observability and monitoring needs
* eBPF-based networking and security features
* Large-scale deployments (200-2,000 nodes) with enhanced performance
* Clusters where eliminating kube-proxy overhead is beneficial

### Ingress Provider Selection <a href="#kubernetesarchitecture-ingressproviderselection" id="kubernetesarchitecture-ingressproviderselection"></a>

You can choose your preferred Ingress controller to manage external access to services within your cluster:

| Ingress Provider | Description                                                                                           |
| ---------------- | ----------------------------------------------------------------------------------------------------- |
| **Nginx**        | Industry-standard ingress controller with robust features and wide community support (default option) |
| **Traefik**      | Modern cloud-native ingress controller with automatic service discovery                               |
| **Istio**        | Service mesh providing advanced traffic management, security, and observability capabilities          |

{% hint style="info" %}
To request a specific CNI or Ingress provider during cluster creation, please specify your requirements through ITCare when ordering your Kubernetes cluster.
{% endhint %}

## Cluster Hardening <a href="#kubernetesarchitecture-clusterhardening" id="kubernetesarchitecture-clusterhardening"></a>

For more information regarding the hardening of Kubernetes, please follow this page [Hardening](/compute/containers-k8s/k8s-features/hardening.md).

## Persistent Storage <a href="#kubernetesarchitecture-persistantstorage" id="kubernetesarchitecture-persistantstorage"></a>

For more information regarding the persistant solution available for **cegedim.cloud**'s Kubernetes clusters, please follow this page [Persistent Storage](/compute/containers-k8s/k8s-features/persistent-storage.md).


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://academy.cegedim.cloud/compute/containers-k8s/k8s-features.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.