Introduction
Setting up a Kubernetes environment can be a complex task, and installing one from scratch can be very time consuming. One of the easier ways of getting started with a new Kubernetes cluster is using Amazon Elastic Kubernetes Service along with eksctl which can set up a new cluster for you within minutes.
In some cases, we will want to use the Kubernetes API to read the configuration of a cluster or for automation of tasks. The issue with EKS clusters is that this is not trivial. If you have ever tried to access an EKS cluster through a Kubernetes Java client, you have likely encountered this error: io.kubernetes.client.openapi.ApiException: Unauthorized. Authenticating with Amazon EKS is a little different than in many other clusters where just using a kubeconfig file can be enough to gain access.
In this blog, we will try to shed some light on how this works and how to get it working.
Accessing an EKS cluster using kubectl
First, let’s try to take a look at an authentication method that does work. AWS offers an easy way to get set up to use kubectl with your new cluster through the command line. The update-kubeconfig command is available to generate a kubeconfig file that will allow you to access the cluster. Let’s run this and take a look at the config file that it generates:
We can see that in the clusters section, we have the certificate information as well as the URL of the API server endpoint of the cluster. In the contexts section, we are using the Cluster ARN to identify the cluster, and then in the users section we can see something interesting. Instead of the user name, we can see that there is an exec command which is actually running an AWS Cli command: aws –region us-east-1 eks get-token –cluster-name test-cluster. Let’s run this command and see what we get:
This seems to be generating a token that we can use to access the cluster from our program, right? We can just call the eks get-token command and pass the token on to the Kubernetes client for authentication.
Well, apparently, this command is not available on the AWS SDK. It can only be run through the CLI. This means we will need to dig deeper…
Looking at the output, we can see this command is generating a token that starts with the string “k8s-aws-v1.” and then has a Base64 encoded value. If we strip the prefix and decode the Base64 string, we can see what this token really is:
This is actually a signed call to the AWS STS API function GetCallerIdentity. All calls to the AWS API are signed, in order to authenticate the request. If you want details on why and how these requests need to be signed, you can see the AWS documentation here. From this, we can already understand how the authentication is actually performed. The aws eks get-token command is actually generating a signed request to AWS STS GetCallerIdentity which is then run in the custom Kubernetes authenticator. This allows the authenticator to know which user is trying to access the cluster and then it can either allow or deny the request based on the user permissions.
If we look at the documentation for the AWS IAM Authenticator for Kubernetes, under the section for API Authorization from Outside a Cluster, we can see some details on generating this token. In addition, we can see that we need to add an additional HTTP header (x-k8s-aws-id) to the request containing the name of the cluster we want to access.
Now, we can try to replicate this behavior in code.
Accessing the Kubernetes cluster using the official java client
In our example here, we will be using the official Kubernetes Java client and version 1 of the AWS SDK. This can also be done with any other supported Kubernetes client and can be performed with version 2 of the SDK (perhaps in a future blog post).
The first step in authenticating with the cluster is creating the request:
URI uri = new URI(“https”, “sts.amazonaws.com”, null, null);
defaultRequest.setResourcePath(“/”);
defaultRequest.setEndpoint(uri);
defaultRequest.setHttpMethod(HttpMethodName.GET);
defaultRequest.addParameter(“Action”, “GetCallerIdentity”);
defaultRequest.addParameter(“Version”, “2011-06-15”);
defaultRequest.addHeader(“x-k8s-aws-id”, clusterName);
Here, we are creating a request to the GetCallerIdentity function, and adding the cluster name in the HTTP Header.
We then need to get the credentials:
BasicAWSCredentials basicCredentials = new BasicAWSCredentials(accessKey, secretKey);
AWSStaticCredentialsProvider credentials = new AWSStaticCredentialsProvider(basicCredentials);
Then use them to sign the request:
Signer signer = SignerFactory.createSigner(SignerFactory.VERSION_FOUR_SIGNER,
new SignerParams(“sts”, region));
AWSSecurityTokenServiceClient stsClient = (AWSSecurityTokenServiceClient) AWSSecurityTokenServiceClientBuilder
.standard().withRegion(region).withCredentials(credentials).build();
SignerProvider signerProvider = new DefaultSignerProvider(stsClient, signer);
PresignerParams presignerParams = new PresignerParams(uri, credentials, signerProvider,
SdkClock.STANDARD);
PresignerFacade presignerFacade = new PresignerFacade(presignerParams);
URL url = presignerFacade.presign(defaultRequest, new Date(System.currentTimeMillis() + 60000));
Note that for the expiration time, we are using the current timestamp + 60 seconds. This means that the signed URL will only be valid for the next 60 seconds (this is also the maximum time allowed by the API).
Finally, we encode the url using Base64 and prepend the k8s-aws-v1. string at the beginning:
String encodedUrl = Base64.getUrlEncoder().withoutPadding().encodeToString(url.toString().getBytes());
return “k8s-aws-v1.” + encodedUrl;
Here is the full function that generates the authentication token:
public String generateEksToken(String clusterName, String region, String accessKey, String secretKey) throws URISyntaxException {
DefaultRequest defaultRequest = new DefaultRequest<>(
new GetCallerIdentityRequest(), “sts”);
URI uri = new URI(“https”, “sts.amazonaws.com”, null, null);
defaultRequest.setResourcePath(“/”);
defaultRequest.setEndpoint(uri);
defaultRequest.setHttpMethod(HttpMethodName.GET);
defaultRequest.addParameter(“Action”, “GetCallerIdentity”);
defaultRequest.addParameter(“Version”, “2011-06-15”);
defaultRequest.addHeader(“x-k8s-aws-id”, clusterName);
BasicAWSCredentials basicCredentials = new BasicAWSCredentials(accessKey, secretKey);
AWSStaticCredentialsProvider credentials = new AWSStaticCredentialsProvider(basicCredentials);
Signer signer = SignerFactory.createSigner(SignerFactory.VERSION_FOUR_SIGNER,
new SignerParams(“sts”, region));
AWSSecurityTokenServiceClient stsClient = (AWSSecurityTokenServiceClient) AWSSecurityTokenServiceClientBuilder
.standard().withRegion(region).withCredentials(credentials).build();
SignerProvider signerProvider = new DefaultSignerProvider(stsClient, signer);
PresignerParams presignerParams = new PresignerParams(uri, credentials, signerProvider,
SdkClock.STANDARD)
PresignerFacade presignerFacade = new PresignerFacade(presignerParams);
URL url = presignerFacade.presign(defaultRequest, new Date(System.currentTimeMillis() + 60000));
String encodedUrl = Base64.getUrlEncoder().withoutPadding().encodeToString(url.toString().getBytes());
return “k8s-aws-v1.” + encodedUrl;
}
Now that we have the token, we can generate a client to access the cluster like this:
String url = “[cluster-api-server-endpoint]”;
String token = generateEksToken(“test-cluster”, “us-east-1”, “AKIAIOSFODNN7EXAMPLE”, “wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY”);
ApiClient client = Config.fromToken(url, token);
Conclusion
As we saw here, it is possible to connect to a Kubernetes cluster on Amazon EKS programmatically. However, this process is not well documented and has not been added to the AWS SDK in a user-friendly way. Hopefully Amazon and Kubernetes will better address this in the future, but for now, I hope this has helped you to get started.
