I’m Eric Tummers, Software Architect at Statistics Netherlands, and this is how I work

My jobs description says Software Architect. I’m guiding my team through the technical hurdles and do some programming/testing/analysing myself. We follow the scrum guide and are working towards continuous integration and deployment. We do this in small steps. Every sprint we complete means the retrospective is done in the pub.

I expect the best and I give the best. Here’s the beer. Here’s the entertainment. Now have fun. That’s an order!
Rasczak

Location: Sittard, NL
Current Gig: Software Architect, CBS
Word that best describes how you work: remote
Current mobile device: iPhone SE 2nd gen
Current computer: MacBook Air M1

What apps/software/tools can’t you live without? Why?

Evernote: I’m a fan for years now.
Spotlight on Mac and Ueli on Windows: for quick start apps and other things.
Microsoft To Do: work tasks, chores, groceries, planning, every task is there.

What’s your workspace setup like?

My home workspace is a Macbook Air M1 connected to a Benq 24 inch monitor (BL2420PT). For input I use the logitech keyboard and mouse (MX800). There is a wireless charger for my iPhone SE or Airpods and a desk lamp (like the Pixar luxo Jr.) This is my main workspace.

When I go to the office I’m free to work from a coffee table with my MacBook Air M1 or hook it up to an external monitor. We’re a hybrid working organisation so the laptop moves with me in my rainbow backpack.

We communicate with each other via zoom and slack. So those apps are my virtual workspace. Does that count?

What’s your best time-saving shortcut/life hack?

Timebox. Start on a task and spent the time you’ve got to get the best result. Get someone to look at it, get feedback. Then decide if the result is final or to spent some more time.

Besides your phone and computer, what gadget can’t you live without and why?

I replaced my Garmin FR235 with the Garmin FR745. It has smart watch features, an optical heart rate monitor and music. I now go on a run without my phone.
My Apple Airpods 2. Easy to use, small and good sound. Replaced my old first gen with the second gen with wireless charging case – still the best wireless earbuds for me.

What everyday thing are you better at than everyone else? What’s your secret?

Learning new things. My current project lets me implement new things (joy) Also I try to learn the things I know to my team or anyone who listens.
I have a basic understanding of how things work and try to map new things on there. For the details I have a Pluralsight subscription and black belt google skills.

What do you listen to while you work?

My alarm clock plays classical music to wake me up in the morning. The car stereo plays about everything (grunge, rock, kids stories) driving to work. When I need some focus I play drum and bass on my headphones. My ringtone is still Run riot by Camo & Krooked, although it is muted since I got a Garmin.

What are you currently reading?

Nerds. A fun view of the things a nerd does, likes and hates. Good read to get me out of the working mindset.

How do you recharge? What do you do when you want to forget about work?

Spending quality time with my wife and daughters. Phone on silent, no screens, no work. 
Also sports like running, fitness, climbing and snowboarding to keep me fit and healthy.

Fill in the blank: I’d love to see _________ answer these same questions.

Eric Rosen a chess streamer I follow on YouTube. This is someone who does what he loves and loves what he does.

What’s the best advice you’ve ever received?

someecards.com - Make a shit first draft you cannot edit a blank page
I believe this is a variant on a Hemingway quote.

Is there anything else you’d like to add that might be interesting to readers?

Learn Powershell. There is so much possible with Powershell. If you can learn one thing this year pick Powershell.

Original idea from Lifehacker.com.

Posted in Uncategorized | Tagged , , , , , , , | Leave a comment

From ingress to gateway api

We use ingress for traffic into our Kubernetes clusters. The next step is to migratie to gateway api. This requires some attention since not all implementations of gateway api support the features we need. On the k8s website there is a list: https://gateway-api.sigs.k8s.io/implementations/

We use nginx ingress for the timeout and sticky session features. The gateway api should support these. We decided to go for the envoy gateway (https://gateway.envoyproxy.io) implementation.

Installation

Envoy gateway offers a helm chart that we deploy using argocd. The initial sync failed because the CRD files surpass the max size. After setting syncOption ServerSideApply everything synced.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: envoy-gateway
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    chart: gateway-helm
    # note: the oci:// syntax is not included.
    repoURL: registry-1.docker.io/envoyproxy
    targetRevision: 1.7.1
  destination:
    name: "in-cluster"
    namespace: envoy-gateway-system
  syncPolicy:
    automated: 
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true
      - ServerSideApply=true

With envoy gateway installed, we now can create the GatewayClass and gateway resources. For simplicity only the unsecured listener is shown here.

apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: envoy-gateway
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: envoy-gateway-local
spec:
  gatewayClassName: envoy-gateway
  listeners:
    - name: http
      protocol: HTTP
      port: 80
      allowedRoutes:
        namespaces:
          from: All

HTTPRoute

Adding the httproute will replace the ingress. Below is an example with no timeout and sticky sessions.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: my-route
spec:
  parentRefs:
    - name: envoy-gateway-local
      namespace: envoy-gateway-system
      sectionName: http
  hostnames:
    - "my-route.localtest.me"
  rules:
    - backendRefs:
        - name: my-service
          port: 80
      matches:
        - path:
            type: PathPrefix
            value: /
      # no timeout = 0s
      timeouts:
          request: 0s
          backendRequest: 0s
      # sticky sessions
      sessionPersistence:
        sessionName: my-route-cookie
        type: Cookie

To activate the gateway api we change the dns record to use the gateway api ip-address and we’re done.

Posted in Uncategorized | Tagged , , , , | Leave a comment

Testkube journey – Sync with azure devops pipeline

We moved our workloads to Kubernetes and now want to run our tests in the cluster. In this series I describe our journey with Testkube. This setup works for us, your milage may vary. You can view all posts in this series by filtering on tag testkube-journey

Azure Devops Server

We use Azure Devops Server to build and deploy our software. The deployment is done with GitOps by committing an updated kustomize file, This can also be used to create a complete new environment just for the tests by committing some more files. In this test environment we will deploy a TestWorkflowExecution that triggers the testworkflow.

After the test environment is created we have to make Azure Devops Server wait for the tests to complete. For this we use the asynchronous check in an approval step of the pipeline. A lot of technical info can be found on the microsoft learn site: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/invoke-checks?view=azure-devops-2022

The pipeline will look like this

Testrelay

The service that synchronises Azure Devops Server and Testkube we call Testrelay. It is a webapi that is called from the approval step with a payload containing information about the tag of the testworkflowexection in testkube and a webhook to report the result back to Azure Devops Server.

Testrelay polls testkube for completion of the testworkflow execution with the tag from the payload. When the testworkflow execution is completed the result is posted back to Azure Devops Server.

Azure Devops Server will then request the artifacts from the testworkflow execution to report the test results. For this Testrelay gets the artifacts from Testkube. When the tests succeed the software is deployed into production.

┌── azure devops server──┐ ┌── kubernetes ────────────────────┐
│ │ │ │
│┌────────┐ ┌──────────┐ │ │ ┌──────────┐ ┌─────────────┐ │
││pipeline│ │ │ │ │ │ ─┼────► application │ │
││ ─┼─► GIT ─┼─┼────┼──► Gitops ─┼──┐ │ under test │ │
││ │ │ │ │ │ │ ─┼─┐│ └─────────────┘ │
││ │ └──────────┘ │ │ └──────────┘ ││ ┌─────────────┐ │
││ │ │ │ │└─► testworkflow│ │
││ │ │ │ ┌──────────┐ │ └─────────────┘ │
││ │ │ │ │ │ │ ┌─────────────┐ │
││ │ │ │ │ testkube │ └──► testworkflow│ │
││ │ │ │ │ ◄────┐ execution │ │
││ │ │ │ └────▲─────┘ └─────────────┘ │
││ │ │ │ │ │
││ │ ┌──────────┐ │ │ ┌────┼─────┐ │
││ │ │ │ │ │ │ │ │
││ ─┼─► approval ┼─┼────┼──► testrelay│ │
││ │ │ │ │ │ │ │ │
│└───▲────┘ └──────────┘ │ │ └─┬────────┘ │
│ └───────────────────┼────┼────┘ │
└────────────────────────┘ └──────────────────────────────────┘

Open source

We plan to open source the Testrelay webapi. Would you be interested? Leave a comment.

Posted in Tooling | Tagged , , , | Leave a comment

Testkube journey – Powershell pester

We moved our workloads to Kubernetes and now want to run our tests in the cluster. In this series I describe our journey with Testkube. This setup works for us, your milage may vary. You can view all posts in this series by filtering on tag testkube-journey

Pester

When we deploy our documentation we test for broken links before deploying to production. These tests are written in PowerShell using the Pester module.

We build a docker image with the tests and the Pester module.

FROM mcr.microsoft.com/powershell AS base

# install pester module
COPY <<EOF /tmp/install.pester.ps1
Register-PSRepository -Name 'internal' -SourceLocation 'https://internal-psgallery' -InstallationPolicy 'Trusted'
Install-Module -Name 'Pester' -Repository 'internal' -Scope 'AllUsers'
EOF
RUN ["pwsh", "/tmp/install.pester.ps1"]

# copy tests
WORKDIR /src/tests
COPY . ./

We run the tests by creating a run script in the testworkflow. Important line is the Throw on failing tests so Testkube knows some files failed.

apiVersion: testworkflows.testkube.io/v1
kind: TestWorkflow
metadata:
  name: broken-links
spec:
  content:
    files:
    - path: /src/runtests.ps1
      content: |
        Import-Module Pester
        $config = New-PesterConfiguration
        $config.TestResult.OutputFormat = 'NUnitXml'
        $config.TestResult.OutputPath = '/data/broken-links-output.xml'
        $config.Run.Path = '/src/tests'
        # make sure testkube knows about failing tests
        $config.Run.Throw = $true
        Invoke-Pester -Configuration $config

  steps:
  - name: pester-tests
    run:
      image: internal-registry/company/broken-links-pester
      command: [ "pwsh" ]
      args: [ "/src/runtests.ps1" ]

See Testkube journey – GitOps about updating the image version with gitops.

The file runtests.ps1 is created in the TestWorkflow with the content block. This way we can change the output format or other settings without rebuilding the image. Happy devs 🚀

Posted in Tooling | Tagged , , | Leave a comment

Testkube journey – GitOps

We moved our workloads to Kubernetes and now want to run our tests in the cluster. In this series I describe our journey with Testkube. This setup works for us, your milage may vary. You can view all posts in this series by filtering on tag testkube-journey

GitOps

TLDR: add configurations so Kustomize understands the TestWorkflows CRD.

We use GitOps for kubernetes deployments. This means files in a git repository that can be edited from every tool that has access. On a regular interval (or by git triggers) the GitOps tool will pull the latest version from git and apply it.

Our weapon of choice is ArgoCD and the files in Git are Kustomize file(s). This means we can update the tag for our tests container from the command line.

Kustomize edit set image internal-registry-address/company/e2etests:NEW_TAG

This line actually adds or changes some lines in the kustomization.yaml file.

apiVersion: Kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- playwright-selfservice-testworkflow.yaml
- rbac.yaml

images:
- name: internal-registry-address/company/e2etests
  newTag: "NEW_TAG"

But when ArgoCD deploys this the image tag is not updated. The default implementation can update the image tag in known kubernetes resources like deployments, jobs and statefulsets. It does not know about the image for a step in TestWorkflows. This is where configurations are used.

In the stepimages.yaml below we tell Kustomize where the images must be applied when the kind is TestWorkflow.

images:
- path: spec/steps/container/image
  kind: TestWorkflow
- path: spec/steps/run/image
  kind: TestWorkflow

Add the stepimages.yaml to the configurations of the Kustomization and the image tag is updated.

apiVersion: Kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- playwright-selfservice-testworkflow.yaml
- rbac.yaml

configurations:
- stepimages.yaml

images:
- name: internal-registry-address/company/e2etests
  newTag: "NEW_TAG"

Whenever there is a new build of the e2etests image the pipeline can update the tag in our TestWorkflow. With GitOps this change is applied in kubernetes and we are using the latest version of our tests.

Posted in Tooling | Tagged , , | 2 Comments

Testkube journey – artifacts

We moved our workloads to Kubernetes and now want to run our tests in the cluster. In this series I describe our journey with Testkube. This setup works for us, your milage may vary. You can view all posts in this series by filtering on tag testkube-journey

Artifacts

When we run our tests we hope everything is :mrgreen: but prepare for ❗ . The output of the tests is placed on a location that is shared with all steps in the TestWorkflow. Testkube can collect these files in an artifact.

TLDR: artifacts is a wildcard path filter relative to the workingDir.

We added the artifacts to the TestWorkflow with “/data/**/*” as path, but gitops kept synchronising. The artifact entry never showed up in the resource. Turns out the path must be relative to the workingDir. After changing to “**/*” all files from “/data” are saved as artifact with the TestExecution.

To download the artifacts we used the cli. Only to be presented with a failure of EOF.

testkube get artifact 1234abcde1234abcde --client direct --api-url http://testkube-api-server:8088
testkube download artifacts 1234abcde1234abcde --download-dir /tmp --client direct --api-url http://testkube-api-server:8088

The logs from the Testkube-api showed the execution was not found. Printing the artifacts would show no execution is registered with the artifacts. We think this was caused by setting up Testkube without Minio first and adding it later. After removing MongoDB and Minio to get a clean install again, it still would not work. We think this was caused by enabling the legacyTests. So we disabled it in the values.yaml without any change in behaviour . Turns out this was a bug that was fixed in a later release. After the upgrade we were able to download all artifacts.

Another way to get the artifacts is by downloading them from the api.

https://testkube-api-server/v1/test-workflow-executions/1234abcde1234abcde/artifact-archive

We struggled some time to get the api working. The v1 in the url was missing at first. Not sure why this is not on the docs, but we got it from slack.

The api will be used to get our pipeline connected to the tests. As described in Testkube journey – where we start we want to automate the decision to release software to production. More about that in a future post.

Posted in Tooling | Tagged , , | 1 Comment