Instana Blog

Date: July 2, 2019

Writing a Kubernetes Operator in Java: Part 2

Category: Engineering

Part 2: Getting Started with the Quarkus Kubernetes Client Extension

This is the second Blog post in our series on writing a Kubernetes operator in Java. The first post gave a general overview of what an operator is. In this post, we show how to set up the a Quarkus application that interacts with the Kubernetes API server. In the third post we will add typical operator functionality, like watching a custom resource.

Initializing a Quarkus Project

Quarkus provides Maven archetypes for initializing projects. In order to create our project, we simply take the command from the official Quarkus getting started guide:

mvn io.quarkus:quarkus-maven-plugin:0.18.0:create \
    -DprojectGroupId=com.instana \
    -DprojectArtifactId=operator-example \
    -DclassName="com.instana.operator.example.GreetingResource" \
    -Dpath="/hello"

This command generates a hello world project. Running ./mvnw package will create an executable JAR file, which will run hello world service on http://localhost:8080/hello.

Creating a Native Executable

In addition to the default profile, the generated pom.xml defines an optional native profile that can be used to create a native executable:

./mvnw package -Pnative -Dnative-image.docker-build=true

The native profile uses GraalVM to compile Java to a native binary. If you don’t have GraalVM installed locally, you can use the -Dnative-image.docker-build=true flag as shown above. This makes maven download a GraalVM Docker image and use it to compile the project. That way, you don’t need to have GraalVM installed.

As a result, an executable ./target/operator-example-1.0-SNAPSHOT-runner is created. If you run it, it opens a hello world REST service on http://localhost:8080/hello.

Adding the Kubernetes Client Extension

GraalVM can be a bit tricky to use, because it is not 100% compatible with Java. If you add a random Maven dependency to a GraalVM project, there might be unexpected errors, both at compile time and at runtime.

Quarkus leverages this by providing extensions. Quarkus extensions are like Maven dependencies, but they are guaranteed to work in native mode. We will use the Fabric8 Kubernetes client to talk to the Kubernetes API server, but we will not add it as a regular Maven dependency, but as a Quarkus extension.

First, list all available extensions to make sure that quarkus-kubernetes-client is available:

mvn quarkus:list-extensions

If it is available, install it as follows:

mvn quarkus:add-extension -Dextensions=quarkus-kubernetes-client

This will modify the pom.xml and add the following dependency:

<dependency>
  <groupId>io.quarkus</groupId>
  <artifactId>quarkus-kubernetes-client</artifactId>
</dependency>

Listing Pods

Quarkus supports dependency injection with CDI, so we are going to use CDI annotations like @Inject to initialize our code. First, let’s create a provider class to initialize the Kubernetes client:

package com.instana.operator.example;

import io.fabric8.kubernetes.client.DefaultKubernetesClient;
import io.fabric8.kubernetes.client.KubernetesClient;

import javax.enterprise.inject.Produces;
import javax.inject.Named;
import javax.inject.Singleton;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;

public class ClientProvider {

  @Produces
  @Singleton
  @Named("namespace")
  private String findNamespace() throws IOException {
    return new String(Files.readAllBytes(Paths.get("/var/run/secrets/kubernetes.io/serviceaccount/namespace")));
  }

  @Produces
  @Singleton
  KubernetesClient newClient(@Named("namespace") String namespace) {
    return new DefaultKubernetesClient().inNamespace(namespace);
  }
}

The interesting part is: In order to learn the namespace we are running in, we read the magic file /var/run/secrets/kubernetes.io/serviceaccount/namespace. This works well if the operator runs within a Kubernetes cluster, because Kubernetes makes sure that this file exists in our container. However, if you intend to run the operator outside of a cluster, you should implement an alternative mechanism here.

The example we create in this Blog post will list all Pods and print the list of Pods to stdout. In order to print this list on startup, we put it in a method that @Observes the CDI StartupEvent.

package com.instana.operator.example;

import io.fabric8.kubernetes.api.model.Pod;
import io.fabric8.kubernetes.client.KubernetesClient;
import io.quarkus.runtime.StartupEvent;

import javax.enterprise.event.Observes;
import javax.inject.Inject;
import java.util.List;

public class PodLister {

  @Inject
  private KubernetesClient client;

  void onStartup(@Observes StartupEvent _ev) {
    List<Pod> podList = client.pods().list().getItems();
    System.out.println("Found " + podList.size() + " Pods:");
    for (Pod pod : podList) {
      System.out.println(" * " + pod.getMetadata().getName());
    }
  }
}

Creating a Docker Image

First of all, we must use -DskipTests to compile the code above, because the tests will initialize all CDI beans, and our beans will try to get the namespace and list pods, which will fail outside the cluster.

./mvnw package -Pnative -DskipTests -Dnative-image.docker-build=true

This should create the executable file ./target/operator-example-1.0-SNAPSHOT-runner.

The next step is to create a Docker image for running this executable. We need to modify the generated ./src/main/docker/Dockerfile.native file to make that work.

The Kubernetes client talks to the Kubernetes API server via HTTPS. In order to do that, it needs a library called libsunec.so, implementing elliptic curve cryptography. This library is missing in the default ubi-minimal Docker image referenced in ./src/main/docker/Dockerfile.native.

libsunec.so is provided as part of GraalVM. After compiling the application with the Maven command above, you should already find a Docker image containing GraalVM in your local repository. The GraalVM image name is quay.io/quarkus/centos-quarkus-native-image. You can simply copy the library out of this docker image:

id=$(docker create quay.io/quarkus/centos-quarkus-native-image)
docker cp "$id:/opt/graalvm/jre/lib/amd64/libsunec.so" ./src/main/docker/

Now you should have the file libsunec.so next to Dockerfile.native in the ./src/main/docker/ directory.

Modify ./src/main/docker/Dockerfile.native and:

  • add a COPY command to copy libsunec.so into the image
  • add the -Djava.library.path parameter to CMD

The resulting ./src/main/docker/Dockerfile.native should look like this:

FROM registry.access.redhat.com/ubi8/ubi-minimal
WORKDIR /work/
COPY target/*-runner /work/application
COPY src/main/docker/libsunec.so /work/library/
RUN chmod 775 /work
EXPOSE 8080
CMD ["./application", "-Dquarkus.http.host=0.0.0.0", "-Djava.library.path=/work/library"]

The generated project has a file .dockerignore that prevents libsunec.so from being added to the Docker image. Edit .dockerignore and add an exception as follows:

*
!src/main/docker/libsunec.so
!target/*-runner
!target/*-runner.jar
!target/lib/*

Now, create the Docker image:

docker build -f src/main/docker/Dockerfile.native -t quarkus-quickstart/getting-started .

Deploying the Operator

By default, Pods in Kubernetes do not have the permission to list other pods. Therefore, we need to create a cluster role, a service account, and a cluster role binding.

Create a file operator-example.clusterrole.yaml with the following content:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: operator-example
rules:
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - list

Create a file operator-example.serviceaccount.yaml with the following content:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: operator-example

Create a file operator-example.clusterrolebinding.yaml with the following content:

kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: operator-example
subjects:
- kind: ServiceAccount
  name: operator-example
  namespace: default
roleRef:
  kind: ClusterRole
  name: operator-example
  apiGroup: rbac.authorization.k8s.io

Now apply all three files:

kubectl apply -f operator-example.clusterrole.yaml
kubectl apply -f operator-example.serviceaccount.yaml
kubectl apply -f operator-example.clusterrolebinding.yaml

The demo lists all pods in the same namespace on startup. So it’s good to have some pods running to see something. For a quick test, just apply the replica set example at the top of the [previous Blog post]((https://www.instana.com/blog/writing-a-kubernetes-operator-in-java-part-1).

Once you have some Pods running, you are ready to deploy the operator. Create a file operator-example.deployment.yaml like this:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: operator-example
spec:
  selector:
    matchLabels:
      app: operator-example
  replicas: 1
  template:
    metadata:
      labels:
        app: operator-example
    spec:
      serviceAccountName: operator-example
      containers:
      - image: quarkus-quickstart/getting-started
        name: operator-example
        imagePullPolicy: IfNotPresent

Apply the file:

kubectl apply -f operator-example.deployment.yaml

With kubectl get pods you should see that an operator-example Pod was started. With kubectl logs -f you can view the Pod list printed to stdout:

> kubectl logs -f operator-example-577bc9448-cdjjg 
Found 4 Pods:
 * kuard-8s5t4
 * kuard-kvbbm
 * kuard-x69mx
 * operator-example-577bc9448-cdjjg

Summary

This Blog post showed how to get a Quarkus application using the Kubernetes client extension up and running. As an example interaction with the Kubernetes API server, we listed all Pods and printed the list of Pods to stdout.

The next post shows how to extend this example to add some real operator functionality, like watching a custom resource.

14 days, no credit card, full version

Free Trial

Sign up for our blog updates!
|
Category: Engineering
Part 3: Writing a Kubernetes Operator in Java In the previous post of this series, we created an example Quarkus...
|
Category: Engineering
Part 1: What is a Kubernetes Operator? We recently published the first alpha version of our Instana Agent Operator, built...
|
Category: Events, Featured
Instana Showcases APM for DevOps Around the World This week, Instana is actively participating at 3 great industry events. Personally,...

Start your FREE TRIAL today!

Free Trial

About Instana

As the leading provider of Automatic Application Performance Monitoring (APM) solutions for microservices, Instana has developed the automatic monitoring and AI-based analysis DevOps needs to manage the performance of modern applications. Instana is the only APM solution that automatically discovers, maps and visualizes microservice applications without continuous additional engineering. Customers using Instana achieve operational excellence and deliver better software faster. Visit https://instana.com to learn more.