The Infinispan Operator provides operational intelligence and reduces management complexity for deploying Infinispan on Kubernetes clusters.

1. Manually Deploying the Infinispan Operator

You can manually deploy the Infinispan Operator as an alternative to installing it from OperatorHub.io.

Prerequisites
  • OKD 3.11 or later.

  • Kubernetes 1.11 or later.

  • Administrator access to the Kubernetes cluster.

1.1. Creating Infinispan Operator Resources

Add the custom resource definition and role-based access control (RBAC) resources for the Infinispan Operator.

Procedure
  1. Apply the custom resource definition.

    $ oc apply -f https://raw.githubusercontent.com/infinispan/infinispan-operator/master/deploy/crd.yaml
  2. Install RBAC resources.

    $ oc apply -f https://raw.githubusercontent.com/infinispan/infinispan-operator/master/deploy/rbac.yaml

1.2. Deploying the Infinispan Operator

Manually deploying the Infinispan Operator to Kubernetes involves applying the yaml template that defines metadata and container specifications.

Procedure
  • Apply the Infinispan Operator template as follows:

$ oc apply -f https://raw.githubusercontent.com/infinispan/infinispan-operator/master/deploy/operator.yaml

2. Creating Infinispan Clusters

You create, configure, and manage Infinispan clusters with the Infinispan Operator by defining custom resource objects in yaml format and applying them to Infinispan pods. Get started here and learn the basics for using the Infinispan Operator.

Prerequisites

2.1. Using Default Infinispan Operator Resources

Infinispan provides default resource definitions that are available in GitHub.

Procedure
  • Run oc apply -f to apply default Infinispan Operator resource definitions.

    For example, to use the default minimal Infinispan cluster:

    $ oc apply -f https://raw.githubusercontent.com/infinispan/infinispan-operator/master/deploy/cr/minimal/cr_minimal.yaml

2.2. Infinispan Custom Resources

The Infinispan Operator adds a resource type and custom resource definition (CRD) with recommended defaults for Infinispan pods.

You configure Infinispan pods dynamically by changing defaults and adding custom resources.

The minimum resources to define a Infinispan cluster are as follows:

apiVersion: infinispan.org/v1 (1)
kind: Infinispan (2)
metadata:
  name: (3)
spec:
  replicas: (4)
1 declares the version.
2 sets the resource type that the Infinispan Operator uses to handle the lifecycle of Infinispan clusters.
3 names the Infinispan cluster.
4 sets the number of pods in the Infinispan cluster.

2.2.1. Pod Resources

The Infinispan Operator lets you allocate resources and specify JVM options for Infinispan pods.

Currently, you cannot modify resources for existing pods. To change container resources you must delete Infinispan pods and create new ones.

spec:
  ...
  container:
    extraJvmOpts: "-XX:NativeMemoryTracking=summary" (1)
    cpu: "1000m" (2)
    memory: 1Gi (3)
1 specifies JVM options.
2 allocates CPU resources.
3 allocates memory resources.

2.3. Spinning Up Infinispan Clusters

Use the Infinispan Operator to create clusters of two or more Infinispan pods.

Procedure
  1. Create a custom resource that sets replicas: 3.

    For example, create a cr_minimal.yaml file as follows:

    $ cat > cr_minimal.yaml<<EOF
    apiVersion: infinispan.org/v1
    kind: Infinispan
    metadata:
      name: example-infinispan
    spec:
      replicas: 3
    EOF
  2. Apply your custom resources.

    $ oc apply -f cr_minimal.yaml
  3. Watch the Infinispan Operator create the Infinispan pods.

    $ oc get pods -w
    
    NAME                        READY  STATUS              RESTARTS   AGE
    example-infinispan-1        0/1    ContainerCreating   0          4s
    example-infinispan-2        0/1    ContainerCreating   0          4s
    example-infinispan-3        0/1    ContainerCreating   0          5s
    infinispan-operator-0       1/1    Running             0          3m
    example-infinispan-3        1/1    Running             0          8s
    example-infinispan-2        1/1    Running             0          8s
    example-infinispan-1        1/1    Running             0          8s
Next Steps

Try changing the value of replicas: and watching the Infinispan Operator scale the cluster up or down.

2.4. Verifying that Infinispan Pods Receive Clustered Views

Review log messages to verify that Infinispan pods successfully form clusters.

Procedure
  • Do either of the following:

    • Retrieve the cluster view from pod logs.

      $ oc logs example-infinispan-0 | grep ISPN000094
      
      INFO  [org.infinispan.CLUSTER] (MSC service thread 1-2) \
      ISPN000094: Received new cluster view for channel infinispan: \
      [example-infinispan-0|0] (1) [example-infinispan-0]
      
      INFO  [org.infinispan.CLUSTER] (jgroups-3,{example_crd_name-0) \
      ISPN000094: Received new cluster view for channel infinispan: \
      [example-infinispan-0|1] (2) [example-infinispan-0, example-infinispan-1]
    • Retrieve the custom resource type for the Infinispan Operator.

      $ oc get infinispan -o yaml

      The response indicates that Infinispan pods have received clustered views:

      conditions:
          - message: 'View: [example-infinispan-0, example-infinispan-1]'
            status: "True"
            type: wellFormed

3. Configuring Authentication

Application users must authenticate with Infinispan pods to access data.

The Infinispan Operator generates default credentials and stores them in secrets. You can use the default credentials or add custom authentication secrets.

Prerequisites
  • A kubectl client in your $PATH.

3.1. Retrieving Credentials

Get base64-encoded credentials from the authentication secret.

Default credentials

example-infinispan-generated-secret is the default authentication secret.
developer is the default application user.
operator is an internal user that interacts with Infinispan clusters.

Procedure
  • Retrieve credentials from the default authentication secret as follows:

    $ oc get secret example-infinispan-generated-secret -n my_namespace

    Do the following to base64 decode credentials:

    $ oc get secret example-infinispan-generated-secret -n my_namespace \
    -o jsonpath="{.data.identities\.yaml}" | base64 -D
    
    credentials:
    - username: developer
      password: dIRs5cAAsHIeeRIL
    - username: operator
      password: uMBo9CmEdEduYk24

3.2. Using Custom Authentication Secrets

Create a custom authentication secret and configure Infinispan pods to use it.

Procedure
  1. Create an authentication secret.

    For example, to create connect_secret.yaml see Credentials Secret.

  2. Add the authentication secret to your OpenShift namespace.

    $ oc apply -f connect_secret.yaml
  3. Define a custom resource for the authentication secret.

    spec:
      ...
      security:
        endpointSecret: connect-secret (1)
    1 specifies the authentication secret.
  4. Apply the custom resource with oc apply -f.

  5. Use oc get pods -w to watch the Infinispan Operator apply the custom resources.

3.2.1. Credentials Secrets

apiVersion: v1
kind: Secret
metadata:
  name: connect-secret (1)
type: Opaque (2)
stringData:
  identities.yaml: |- (3)
    credentials (4)
    - username: (5)
      password: (6)
1 names the authentication secret.
2 authentication secrets must be Opaque.
3 contains application users in YAML format.
4 lists credentials for application users.
  • You must include the operator user and specify a password. The Infinispan Operator requires these credentials to interact with Infinispan clusters.

  • Do not use example passwords from this documentation. You should always replace example credentials with ones that conform to the relevant security guidelines.

4. Securing Infinispan Endpoints

Configure Infinispan pods to encrypt network traffic between clients and endpoints with TLS certificates.

You can generate Red Hat OpenShift service certificates or use custom TLS certificates.

4.1. Generating Service Certificates for Encryption

The Infinispan Operator can generate Red Hat OpenShift service certificates that:

  • Consist of a certificate, tls.crt, and key, tls.key, in PEM format that OpenShift stores in a secret.

  • Are signed by the OpenShift CA, are valid for one year, and are automatically replaced before expiration.

Procedure
  1. Define custom resources to use service certificates for encryption.

    spec:
      ...
      security:
        endpointEncryption: (1)
                type: service (2)
                certService: service.beta.openshift.io (3)
                certSecretName: tls-secret (4)
    1 encrypts traffic to and from Infinispan endpoints.
    2 configures Infinispan to service certificates.
    3 adds the OpenShift annotation.
    4 names the encryption secret.
  2. Apply the custom resource with oc apply -f.

  3. Use oc get pods -w to watch the Infinispan Operator apply the custom resources.

  4. Retrieve tls.crt from the secret to create truststores for clients.

    $ oc get secret tls-secret -o jsonpath='{.data.tls\.crt}' |  \
    base64 -d > tls.crt

4.2. Adding TLS Certificates to Encryption Secrets

Add a PKCS12 keystore or TLS certificate/key pair to a secret and configure Infinispan to use it for endpoint encryption.

Procedure
  1. Create an encryption secret.

    For example, to create tls_secret.yaml see one of the following:

  2. Add the encryption secret to your OpenShift namespace.

    $ oc apply -f tls_secret.yaml
  3. Define a custom resource for the encryption secret.

    spec:
      ...
      security:
        endpointEncryption: (1)
                type: secret (2)
                certSecretName: tls-secret (3)
    1 encrypts traffic to and from Infinispan endpoints.
    2 configures Infinispan to use secrets that contain encryption certificates.
    3 names the encryption secret.
  4. Apply the custom resource with oc apply -f.

  5. Use oc get pods -w to watch the Infinispan Operator apply the custom resources.

4.2.1. Certificate Secrets

apiVersion: v1
kind: Secret
metadata:
  name: tls-secret
type: Opaque
data:
    tls.key:  "LS0tLS1CRUdJTiBQUk ..." (1)
    tls.crt: "LS0tLS1CRUdJTiBDRVl ..." (2)
1 adds a base64 encoded TLS key.
2 adds a base64 encoded TLS certificate.

4.2.2. Keystore Secrets

apiVersion: v1
kind: Secret
metadata:
  name: tls-secret
type: Opaque
stringData:
    alias: server (1)
    password: password (2)
data:
    keystore.p12:  "MIIKDgIBAzCCCdQGCSqGSIb3DQEHA..." (3)
1 specifies an alias for the keystore.
2 specifies a password for the keystore.
3 adds a base64 encoded keystore.

5. Reference

Find useful information for Infinispan clusters that you create with the Infinispan Operator.

5.1. Services

The Infinispan Operator automatically creates services to route network traffic. These services use the names that you give to your Infinispan clusters.

Internal services
  • Allow Infinispan nodes to discover each other and form clusters.

  • Provide access to Infinispan endpoints from clients in the same Kubernetes namespace.

    Service Port Protocol Description

    ${cluster_name}

    11222

    TCP

    Infinispan endpoints

    ${cluster_name}-ping

    8888

    TCP

    Cluster discovery

External services

Provide access to Infinispan endpoints from clients outside the Kubernetes namespace.

Service Port Protocol Description

${cluster_name}-external

11222

TCP

Infinispan endpoints