Kubernetes is becoming an essential infrastructure for IT operations and DevOps teams. It’s used to run all types of workloads, spanning everything from development and testing environments to mission-critical production applications.

The API Server is the “front gate” of Kubernetes, allowing authorized users to perform any action on a Kubernetes cluster, and potentially gain access to any sensitive data it manages. This makes the Kubernetes API Server a highly valuable target for attackers and a top priority for your open source security strategy.

Read on to better understand how the Kubernetes API Server works and what is required to limit the attack surface to protect it from cyber threats.

What Is the Kubernetes API Server?

The API server (kube-apiserver) is the frontend of Kubernetes’ core control plane. It exposes an HTTP API that enables communication between end-users, external components, and different parts of a cluster. The Kubernetes API allows you to query and manipulate the state of API objects, such as pods, ConfigMaps, events, and namespaces.

In Kubernetes, you can perform most operations through the kubectl command-line interface (CLI) or other tools like kubeadm, all of which use the Kubernetes API. You can also use REST calls to access the API directly.

The Importance of Securing the Kubernetes API Server

The Kubernetes API server is responsible for establishing communication between end-users, external elements, and cluster components. Once compromised, the API server can allow attackers to take over this communication and perform virtually any action in a Kubernetes cluster.

For example, attackers could use API server communication to manipulate components into communicating with malicious resources hosted externally. Attackers can also exploit this communication to spread malware that threatens the availability of applications and services.

Therefore, organizations must lock down the Kubernetes API server to strengthen Kubernetes security and take additional steps to secure this API from unauthorized access.

Kubernetes API Concepts

To better understand API Server security best practices, let’s take a quick look at the Kubernetes API architecture.

The Kubernetes API is a resource-based (RESTful) interface, which is why Kubernetes uses common RESTful terminology for API concepts. Here are key concepts you should know:

• Resource type: The name described in the URL. For example, Kubernetes pods, persistent storage volumes, and namespaces.
• Kind: The object schema (concrete representation) of resource types.
• Collection: This term refers to a resource’s list of instances.
• Resource: This term refers to a single instance of a resource type. It usually also represents an object.
• Sub-resources: This option is available for some resources. The API can include one or more sub-resources, represented as URI paths.

In Kubernetes, most API resource types are objects representing a concrete instance of a concept on a cluster, like a namespace. Here are key characteristics of object resource types:

• These objects have unique names to enable idempotent retrieval and creation.
• Virtual resource types that are not retrievable or rely on idempotency may not necessarily have unique names.
• In each namespace, you can use only one object of a kind and a certain name at a given time. You can create a new object with the same name once you delete the object.

Most object resource types support common HTTP verbs. Kubernetes also uses unique verbs, typically written in lowercase, to distinguish them from standard HTTP verbs.

Here is how the API generally works:

• HTTP verbs: The Kubernetes API uses standard HTTP verbs, such as POST, PATCH, PUT, GET, and DELETE, to delete, retrieve, update, and create primary resources.
• Fine-grained authorization: The Kubernetes API provides additional subresources for some resources. It can also accept and serve these resources in different representations for efficiency and convenience.
• Change notification: Kubernetes uses watches to detect and notify you of changes occurring on resources. It also provides a list of operations to enable API clients to cache, track, and synchronize the state of your resources.

Controlling Access to the Kubernetes API

You can use client libraries, or kubectl, or make a REST call to access the Kubernetes API. Kubernetes supports the authorization of both service accounts and users for API access. Here are some built-in ways for controlling access to the Kubernetes API.

Transport Layer Security (TLS)

TLS typically protects Kubernetes clusters served by the API on port 443. The API server provides a certificate signed by a certificate authority (CA) or using a public key infrastructure associated with a recognized CA.

If you use a private CA for your cluster, you need to configure a copy of the CA certificate into the ~/.kube/config on your client. This ensures the connection is trustworthy and will not be intercepted.

Authentication

After establishing TLS, the HTTP request progresses to authentication. The whole request serves as the input for authentication, although the authentication system usually only examines the client certificate or headers. The authentication step involves modules like passwords, client certificates, and tokens (i.e., plain, bootstrap, or JSON Web Tokens).

If authentication is successful, the user can use the username for the next steps. If the system cannot authenticate the HTTP request, it generates a 401 status code. In some cases, the authenticator provides the user’s group memberships.

Authorization

The API must authorize requests once it authenticates them as coming from a legitimate user. Each request must contain the requester’s username, the action requested, and the affected object. The system authorizes the request if the access control policy allows the user to perform the specific action. However, it denies authorization if the user requests a restricted action. For example, a user might have Read but not Write permissions.

Kubernetes authorization should interact with your established access control systems using REST attributes. You need to use the REST format to enable interaction with other APIs. Kubernetes supports several authorization modes, including RBAC, ABAC, and Webhook. When you create a cluster, you need to configure the API server’s authorization type. If you configure multiple authentication modes, Kubernetes will check requests against all of them.

Admission Control

Admission control uses software modules to modify or reject access requests. Admission controllers can access the attributes used by authorization modules and the contents of objects being built or modified.

Admission control modules respond to requests that create, connect to, modify, or delete objects (but not read requests). If you configure multiple controllers, the system calls them in order. Admission controllers differ from authentication and authorization systems in that they immediately reject requests denied by a single module.

Advanced Kubernetes API Server Security Best Practices

The first step to securing the API Server is to establish basic access control. But this is not enough for production clusters. These additional best practices can help you harden the Kubernetes API Server for mission-critical environments.

Use Third-Party Authentication for the API Server

Kubernetes offers several built-in API server authentication mechanisms, which we described above, but these are usually unsuitable for large production clusters. In this scenario, you can use a third-party authentication method such as OpenID Connect (OIDC) or managed Kubernetes services (like GKE, AKS, or EKS). You can also externalize API server authentication (for clusters without configuration access) using Kubernetes impersonation.

Treat API access as a high privilege and implement multi-factor authentication to increase security.

Use a Single Authentication Method for Each User

Kubernetes supports multiple authentication methods, but it grants access to the user based on the first successful authentication request. If a request passes one authenticator but not another, Kubernetes will go with the weaker one. You cannot know the order in which the authenticators run.

Ensure that each user account has a single authentication method to prevent malicious actors from exploiting weaker authenticators.

Protect etcd

The etcd component stores state information and secrets and requires special protection. Users with Write access to the etcd have root access to the whole cluster. A malicious user could use Read access to escalate privileges. The Kubernetes scheduler searches the etcd for pod definitions without a node and sends any pods it identifies to a kubelet. The API server validates submitted pods before writing them to etcd.

A malicious user can bypass security mechanisms like pod security policies by writing directly to the etcd. Thus, it’s best to use strong API server credentials for etcd — for example, use TLS client certificates for mutual authentication. Protect your etcd servers with a firewall to restrict access to the API servers.

Conclusion

In this article, we examined the basics of Kubernetes API security and provided the following best practices for hardening your API Server:

• Enable Transport Layer Security (TLS).
• Set up basic authentication, authorization, and admission control.
• For production environments, consider a more secure third-party authentication mechanism.
• Use a single authentication method for each user.
• Protect the etcd configuration database.

I hope this will be useful as you improve the security posture of your mission-critical Kubernetes assets.