Spring Cloud integration with Kubernetes
This project provides an implementation of Discovery Client for Kubernetes. This allows you to query Kubernetes endpoints (see services) by name. A service is typically exposed by the Kubernetes API server as a collection of endpoints which represent http
, https
addresses that a client can access from a Spring Boot application running as a pod. This discovery feature is also used by the Spring Cloud Kubernetes Ribbon or Zipkin projects to fetch respectively the list of the endpoints defined for an application to be load balanced or the Zipkin servers available to send the traces or spans.
This is something that you get for free just by adding the following dependency inside your project:
<dependency>
<groupId>org.springframework.cloudgroupId>
<artifactId>spring-cloud-starter-kubernetesartifactId>
<version>${latest.version}version>
dependency>
Then you can inject the client in your code simply by:
@Autowire
private DiscoveryClient discoveryClient;
If for any reason you need to disable the DiscoveryClient
you can simply set the following property in application .properties
:
spring.cloud.kubernetes.discovery.enabled=false
Some Spring Cloud components use the DiscoveryClient
in order to obtain info about the local service instance. For this to work you need to align the service name with the spring.application.name
property.
The most common approach to configure your Spring Boot application is to create an application.properties|yaml
or an application-profile.properties|yaml
file containing key-value pairs providing customization values to your application or Spring Boot starters. Users may override these properties by specifying system properties or environment variables.
Kubernetes provides a resource named ConfigMap to externalize the parameters to pass to your application in the form of key-value pairs or embedded application.properties|yaml
files. The Spring Cloud Kubernetes Config project makes Kubernetes ConfigMap
s available during application bootstrapping and triggers hot reloading of beans or Spring context when changes are detected on observed ConfigMap
s.
ConfigMapPropertySource
will search for a Kubernetes ConfigMap
which metadata.name
is either the name of your Spring application (as defined by its spring.application.name
property) or a custom name defined within thebootstrap.properties
file under the following key spring.cloud.kubernetes.config.name
.
If such a ConfigMap
is found, it will be processed as follows:
yaml
the content of any property named application.yaml
application.properties
Example:
Let's assume that we have a Spring Boot application named demo
that uses properties to read its thread pool configuration.
pool.size.core
pool.size.maximum
This can be externalized to config map in yaml
format:
kind: ConfigMap
apiVersion: v1
metadata:
name: demo
data:
pool.size.core: 1
pool.size.max: 16
Individual properties work fine for most cases but sometimes embedded yaml
is more convenient. In this case we will use a single property named application.yaml
to embed our yaml
:
kind: ConfigMap
apiVersion: v1
metadata:
name: demo
data:
application.yaml: |-
pool:
size:
core: 1
max:16
Spring Boot applications can also be configured differently depending on active profiles and it is possible to provide different property values for different profiles using an application.properties|yaml
property, specifying profile-specific values each in their own document (indicated by the ---
sequence) as follows:
kind: ConfigMap
apiVersion: v1
metadata:
name: demo
data:
application.yml: |-
greeting:
message: Say Hello to the World
---
spring:
profiles: development
greeting:
message: Say Hello to the Developers
---
spring:
profiles: production
greeting:
message: Say Hello to the Ops
To tell to Spring Boot which profile
should be enabled at bootstrap, a system property can be passed to the Java command launching your Spring Boot application using an env variable that you will define with the OpenShift DeploymentConfig
or Kubernetes ReplicationConfig
resource file as follows:
apiVersion: v1
kind: DeploymentConfig
spec:
replicas: 1
...
spec:
containers:
- env:
- name: JAVA_APP_DIR
value: /deployments
- name: JAVA_OPTIONS
value: -Dspring.profiles.active=developer
Notes:
To access ConfigMap
s on OpenShift the service account needs at least view permissions i.e.:
oc policy add-role-to-user view system:serviceaccount:$(oc project -q):default -n $(oc project -q)
Properties:
Name | Type | Default | Description |
---|---|---|---|
spring.cloud.kubernetes.config.enabled | Boolean | true | Enable Secrets PropertySource |
spring.cloud.kubernetes.config.name | String | ${spring.application.name} | Sets the name of ConfigMap to lookup |
spring.cloud.kubernetes.config.namespace | String | Client namespace | Sets the Kubernetes namespace where to lookup |
spring.cloud.kubernetes.config.paths | List | null | Sets the paths were ConfigMaps are mounted |
spring.cloud.kubernetes.config.enableApi | Boolean | true | Enable/Disable consuming ConfigMaps via APIs |
Kubernetes has the notion of Secrets for storing sensitive data such as password, OAuth tokens, etc. This project provides integration with Secrets
to make secrets accessible by Spring Boot applications. This feature can be explicitly enabled/disabled using the spring.cloud.kubernetes.secrets.enabled
property.
The SecretsPropertySource
when enabled will lookup Kubernetes for Secrets
from the following sources:
spring.application.name
)Please note that by default, consuming Secrets via API (points 2 and 3 above) is not enabled for security reasons and it is recommend that containers share secrets via mounted volumes.
If the secrets are found their data is made available to the application.
Example:
Let's assume that we have a spring boot application named demo
that uses properties to read its database configuration. We can create a Kubernetes secret using the following command:
oc create secret generic db-secret --from-literal=username=user --from-literal=password=p455w0rd
This would create the following secret (shown using oc get secrets db-secret -o yaml
):
apiVersion: v1
data:
password: cDQ1NXcwcmQ=
username: dXNlcg==
kind: Secret
metadata:
creationTimestamp: 2017-07-04T09:15:57Z
name: db-secret
namespace: default
resourceVersion: "357496"
selfLink: /api/v1/namespaces/default/secrets/db-secret
uid: 63c89263-6099-11e7-b3da-76d6186905a8
type: Opaque
Note that the data contains Base64-encoded versions of the literal provided by the create command.
This secret can then be used by your application for example by exporting the secret's value as environment variables:
apiVersion: v1
kind: Deployment
metadata:
name: ${project.artifactId}
spec:
template:
spec:
containers:
- env:
- name: DB_USERNAME
valueFrom:
secretKeyRef:
name: db-secret
key: username
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
name: db-secret
key: password
You can select the Secrets to consume in a number of ways:
By listing the directories were secrets are mapped:
-Dspring.cloud.kubernetes.secrets.paths=/etc/secrets/db-secret,etc/secrets/postgresql
If you have all the secrets mapped to a common root, you can set them like:
-Dspring.cloud.kubernetes.secrets.paths=/etc/secrets
By setting a named secret:
-Dspring.cloud.kubernetes.secrets.name=db-secret
By defining a list of labels:
-Dspring.cloud.kubernetes.secrets.labels.broker=activemq
-Dspring.cloud.kubernetes.secrets.labels.db=postgresql
Properties:
Name | Type | Default | Description |
---|---|---|---|
spring.cloud.kubernetes.secrets.enabled | Boolean | true | Enable Secrets PropertySource |
spring.cloud.kubernetes.secrets.name | String | ${spring.application.name} | Sets the name of the secret to lookup |
spring.cloud.kubernetes.secrets.namespace | String | Client namespace | Sets the Kubernetes namespace where to lookup |
spring.cloud.kubernetes.secrets.labels | Map | null | Sets the labels used to lookup secrets |
spring.cloud.kubernetes.secrets.paths | List | null | Sets the paths were secrets are mounted (example 1) |
spring.cloud.kubernetes.secrets.enableApi | Boolean | false | Enable/Disable consuming secrets via APIs (examples 2 and 3) |
Notes:
spring.cloud.kubernetes.secrets.labels
behaves as defined by Map-based binding.spring.cloud.kubernetes.secrets.paths
behaves as defined by Collection-based binding.Example of application using secrets (though it hasn't been updated to use the new spring-cloud-kubernetes
project):spring-boot-camel-config
Some applications may need to detect changes on external property sources and update their internal status to reflect the new configuration. The reload feature of Spring Cloud Kubernetes is able to trigger an application reload when a related ConfigMap
or Secret
changes.
This feature is disabled by default and can be enabled using the configuration property spring.cloud.kubernetes.reload.enabled=true
(eg. in the application.properties file).
The following levels of reload are supported (property spring.cloud.kubernetes.reload.strategy
):
refresh
(default): only configuration beans annotated with @ConfigurationProperties
or @RefreshScope
are reloaded. This reload level leverages the refresh feature of Spring Cloud Context.restart_context
: the whole Spring ApplicationContext is gracefully restarted. Beans are recreated with the new configuration.shutdown
: the Spring ApplicationContext is shut down to activate a restart of the container. When using this level, make sure that the lifecycle of all non-daemon threads is bound to the ApplicationContext and that a replication controller or replica set is configured to restart the pod.Example:
Assuming that the reload feature is enabled with default settings (refresh
mode), the following bean will be refreshed when the config map changes:
@Configuration
@ConfigurationProperties(prefix = "bean")
public class MyConfig {
private String message = "a message that can be changed live";
// getter and setters
}
A way to see that changes effectively happen is creating another bean that prints the message periodically.
@Component
public class MyBean {
@Autowired
private MyConfig config;
@Scheduled(fixedDelay = 5000)
public void hello() {
System.out.println("The message is: " + config.getMessage());
}
}
The message printed by the application can be changed using a ConfigMap
as follows:
apiVersion: v1
kind: ConfigMap
metadata:
name: reload-example
data:
application.properties: |-
bean.message=Hello World!
Any change to the property named bean.message
in the ConfigMap
associated to the pod will be reflected in the output. More generally speaking, changes associated to properties prefixed with the value defined by the prefix
field of the @ConfigurationProperties
annotation will be detected and reflected in the application. Associating a ConfigMap
to a pod is explained above.
The full example is available in spring-cloud-kubernetes-reload-example.
The reload feature supports two operating modes:
view
role on the service account is required in order to listen for config map changes. A higher level role (eg. edit
) is required for secrets (secrets are not monitored by default).spring.cloud.kubernetes.reload.period
and defaults to 15 seconds. It requires the same role as the monitored property source. This means, for example, that using polling on file mounted secret sources does not require particular privileges.Properties:
Name | Type | Default | Description |
---|---|---|---|
spring.cloud.kubernetes.reload.enabled | Boolean | false | Enables monitoring of property sources and configuration reload |
spring.cloud.kubernetes.reload.monitoring-config-maps | Boolean | true | Allow monitoring changes in config maps |
spring.cloud.kubernetes.reload.monitoring-secrets | Boolean | false | Allow monitoring changes in secrets |
spring.cloud.kubernetes.reload.strategy | Enum | refresh | The strategy to use when firing a reload (refresh, restart_context, shutdown) |
spring.cloud.kubernetes.reload.mode | Enum | event | Specifies how to listen for changes in property sources (event, polling) |
spring.cloud.kubernetes.reload.period | Long | 15000 | The period in milliseconds for verifying changes when using the polling strategy |
Notes:
Spring Boot uses HealthIndicator to expose info about the health of an application. That makes it really useful for exposing health related information to the user and are also a good fit for use as readiness probes.
The Kubernetes health indicator which is part of the core module exposes the following info:
All of the features described above will work equally well regardless of whether your application is running inside Kubernetes or not. This is really helpful for development and troubleshooting. From a development point of view, this is really helpful as you can start your Spring Boot application and debug one of the modules part of this project. It is not required to deploy it in Kubernetes as the code of the project relies on the Fabric8 Kubernetes Java client which is a fluent DSL able to communicate using http
protocol to the REST API of Kubernetes Server.
When the application runs as a pod inside Kubernetes a Spring profile named kubernetes
will automatically get activated. This allows the developer to customize the configuration, to define beans that will be applied when the Spring Boot application is deployed within the Kubernetes platform (e.g. different dev and prod configuration).
Spring Cloud client applications calling a microservice should be interested on relying on a client load-balancing feature in order to automatically discover at which endpoint(s) it can reach a given service. This mechanism has been implemented within the spring-cloud-kubernetes-ribbon project where a Kubernetes client will populate a Ribbon ServerList
containing information about such endpoints.
The implementation is part of the following starter that you can use by adding its dependency to your pom file:
<dependency>
<groupId>org.springframework.cloudgroupId>
<artifactId>spring-cloud-starter-kubernetes-netflixartifactId>
<version>${latest.version}version>
dependency>
When the list of the endpoints is populated, the Kubernetes client will search the registered endpoints living in the current namespace/project matching the service name defined using the Ribbon Client annotation:
@RibbonClient(name = "name-service")
You can configure Ribbon's behavior by providing properties in your application.properties
(via your application's dedicated ConfigMap
) using the following format:
where:
corresponds to the service name you're accessing over Ribbon, as configured using the@RibbonClient
annotation (e.g. name-service
in the example above)
is one of the Ribbon configuration key defined by Ribbon's CommonClientConfigKey classAdditionally, the spring-cloud-kubernetes-ribbon
project defines two additional configuration keys to further control how Ribbon interacts with Kubernetes. In particular, if an endpoint defines multiple ports, the default behavior is to use the first one found. To select more specifically which port to use, in a multi-port service, use the PortName
key. If you want to specify in which Kubernetes' namespace the target service should be looked up, use the KubernetesNamespace
key, remembering in both instances to prefix these keys with your service name and ribbon
prefix as specified above.
Examples that are using this module for ribbon discovery are:
Remark : The Ribbon discovery client can be disabled by setting this key within the application properties filespring.cloud.kubernetes.ribbon.enabled=false
.
Zipkin is a distributed tracing system which is supported by the project Spring Cloud Sleuth which allows to collect traces or spans from microservice applications.
A Discovery client has been implemented top of Kubernetes in order to fetch the Zipkin service (e.g. zipkin
). This client is provided by the spring-cloud-kubernetes-zipkin project that you can use by adding this starter to your maven pom file:
<dependency>
<groupId>org.springframework.cloudgroupId>
<artifactId>spring-cloud-starter-kubernetes-zipkinartifactId>
<version>${latest.version}version>
dependency>
This works as an extension of the spring-cloud-sleuth-zipkin project. The name of the Zipkin service and the target Kubernetes namespace/project where the service runs can be specified using the following application.properties
properties:
spring.cloud.kubernetes.zipkin.discovery.serviceName=my-zipkin
spring.cloud.kubernetes.zipkin.discovery.serviceNamespace=tracing
By default, the discovery client will look for a Zipkin service named zipkin
within the current namespace.
Examples of application that are using Zipkin discovery in Kubernetes:
The section ConfigMap PropertySource introduced how to configure a spring boot application via Kubernetes ConfigMap
containing your configuration properties file.
If you prefer to use the configuration management library Netflix Archaius instead of using the Spring application.properties|"yaml file, then you can also leverage the ConfigMap feature
by using the spring-cloud-kubernetes-archaius project.
To use it, add the following starter spring-cloud-starter-kubernetes-all
to your pom file definition.
This module allows you to annotate your application with the @ArchaiusConfigMapSource
and archaius will automatically use the Kubernetes configmap
as a watched source (get notification on changes).
The current version of Spring Cloud Kubernetes is using version 2.2.x of the Fabric8 Kubernetes Client and is expected to work with version 1.x of Kubernetes and 1.x of Openshift. Note, that Kubernetes and Openshift are for the most part backwards compatible so, its expected that this framework is compatible with both the latest and earlier versions.
Most of the components provided in this project need to know the namespace. For Kubernetes (1.3+) the namespace is made available to pod as part of the service account secret and automatically detected by the client. For earlier version it needs to be specified as an env var to the pod. A quick way to do this is:
env:
- name: "KUBERNETES_NAMESPACE"
valueFrom:
fieldRef:
fieldPath: "metadata.namespace"
For distros of Kubernetes that support more fine-grained role-based access within the cluster, you need to make sure a pod that runs with spring-cloud-kubernetes has access to the Kubernetes API. For example, OpenShift has very comprehensive security measures that are on by default (typically) in a shared cluster. For any service accounts you assign to a deployment/pod, you need to make sure it has the correct roles. For example, you can add cluster-reader
permissions to your default
service account depending on the project you're in:
oc policy add-role-to-user cluster-reader system:serviceaccount::default
You can just use maven to build it from sources:
mvn clean install
The project provides a "starter" module, so you just need to add the following dependency in your project.
<dependency>
<groupId>org.springframework.cloudgroupId>
<artifactId>spring-cloud-starter-kubernetesartifactId>
<version>x.y.zversion>
dependency>