We moved our workloads to Kubernetes and now want to run our tests in the cluster. In this series I describe our journey with Testkube. This setup works for us, your milage may vary. You can view all posts in this series by filtering on tag testkube-journey

Azure Devops Server

We use Azure Devops Server to build and deploy our software. The deployment is done with GitOps by committing an updated kustomize file, This can also be used to create a complete new environment just for the tests by committing some more files. In this test environment we will deploy a TestWorkflowExecution that triggers the testworkflow.

After the test environment is created we have to make Azure Devops Server wait for the tests to complete. For this we use the asynchronous check in an approval step of the pipeline. A lot of technical info can be found on the microsoft learn site: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/invoke-checks?view=azure-devops-2022

The pipeline will look like this

Testrelay

The service that synchronises Azure Devops Server and Testkube we call Testrelay. It is a webapi that is called from the approval step with a payload containing information about the tag of the testworkflowexection in testkube and a webhook to report the result back to Azure Devops Server.

Testrelay polls testkube for completion of the testworkflow execution with the tag from the payload. When the testworkflow execution is completed the result is posted back to Azure Devops Server.

Azure Devops Server will then request the artifacts from the testworkflow execution to report the test results. For this Testrelay gets the artifacts from Testkube. When the tests succeed the software is deployed into production.

┌── azure devops server──┐    ┌── kubernetes ────────────────────┐
│                        │    │                                  │
│┌────────┐ ┌──────────┐ │    │  ┌──────────┐    ┌─────────────┐ │
││pipeline│ │          │ │    │  │         ─┼────► application │ │
││       ─┼─►   GIT   ─┼─┼────┼──►  Gitops ─┼──┐ │ under test  │ │
││        │ │          │ │    │  │         ─┼─┐│ └─────────────┘ │
││        │ └──────────┘ │    │  └──────────┘ ││ ┌─────────────┐ │
││        │              │    │               │└─► testworkflow│ │
││        │              │    │  ┌──────────┐ │  └─────────────┘ │
││        │              │    │  │          │ │  ┌─────────────┐ │
││        │              │    │  │ testkube │ └──► testworkflow│ │
││        │              │    │  │          ◄────┐ execution   │ │
││        │              │    │  └────▲─────┘    └─────────────┘ │
││        │              │    │       │                          │
││        │ ┌──────────┐ │    │  ┌────┼─────┐                    │
││        │ │          │ │    │  │          │                    │
││       ─┼─► approval ┼─┼────┼──► testrelay│                    │
││        │ │          │ │    │  │          │                    │
│└───▲────┘ └──────────┘ │    │  └─┬────────┘                    │
│    └───────────────────┼────┼────┘                             │
└────────────────────────┘    └──────────────────────────────────┘

Open source

We plan to open source the Testrelay webapi. Would you be interested? Leave a comment.