Commit to git from classic release pipeline

Posted on December 22, 2023 by erictummers

We have Azure Devops Server to manage our solutions. The version we currently run comes with yaml build pipelines and classic release pipelines. It is “easy” to commit and push to git from a yaml pipeline with the repository resource definition (details on microsoft learn) You need to do some extra lifting in the classic pipeline.

First enable the “Allow scripts to access the OAuth token” – this makes the $env:SYSTEM_ACCESSTOKEN available to scripts. It is located on the Agent job part inside the Additional options.

Now you can clone the repo with a task like powershell. Make sure to set the working directory.

Write-Host 'Clone repo'
# create base64 encoded version of the access token
$B64Pat = [Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes("`:$env:SYSTEM_ACCESSTOKEN"))
# git with extra header
git -c http.extraHeader="Authorization: Basic $B64Pat" clone  https://AZURE_DEVOPS_URL/_git/REPOSITORY
# move into the repository folder
push-location REPOSITORY
git checkout main

Make your change and commit + push with a task. Make sure to set the same working directory – like the folder with the artefact from the build.

Write-Host 'Add, commit and push file'
$B64Pat = [Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes("`:$env:SYSTEM_ACCESSTOKEN"))
# move into the repository folder
push-location REPOSITORY
# create / update file
Copy-Item -Path ../deployment.yaml -Destination overlays/dev/deployment.yaml -Force
# stage file
git add overlays/dev/deployment.yaml
# provide use info
git config user.email "info@erictummers.com"
git config user.name "build agent"
# commit using the build number
git commit -m "updated deployment to $(Build.BuildNumber)"
# pull and push
git -c http.extraHeader="Authorization: Basic $B64Pat" pull
git -c http.extraHeader="Authorization: Basic $B64Pat" push

In the clean up we remove the repository as a good practice. Again make sure to set the same working directory.

Write-Host 'Clean up'
Remove-Item REPOSITORY -Recurse -Force

Now the file is changed in git from a classic pipeline.

References

Posted in Development | Tagged , | Leave a comment

Rancher Desktop and K3s allowed-unsafe-sysctls

Posted on December 1, 2023 by erictummers

Today I’ve got this error

SysctlForbidden kubelet forbidden sysctl: "net.ipv6.conf.all.disable_ipv6" not whitelisted

Rancher desktop will not accept all sysctls. Some are allowed, but most options are not. To get this working you have to add the setting to the allowed-unsafe-sysctls list.

podSecurityContext:                        
  sysctls:                                 
    - name: net.ipv6.conf.all.disable_ipv6 
      value: '1'

⭐️ Tip: you can see hidden files in Finder on Mac by pressing CMD + Shift + .

Open ~/library/Application Support/rancher-desktop/lima/_config/override.yaml and add the extra options to K3S_EXEC. Be sure to specify the complete value since the setting is overwritten not appended. Use commas with multiple values.

In the example below I’ve added the option to specify net.ipv6.conf.all.disable_ipv6 next to the default setting values net.ipv4.ip_forward and net.ipv6.conf.all.forwarding. Scroll horizontal to see everything.

env:
  K3S_EXEC: --kubelet-arg=allowed-unsafe-sysctls=net.ipv4.ip_forward,net.ipv6.conf.all.forwarding,net.ipv6.conf.all.disable_ipv6

You can validate the change has been applied by looking at the logfile ~/Library/Logs/rancher-desktop/k3s.log. Here you can find the “default” value before you change it with the override.yaml edit above.

time="2023-12-01T11:35:33Z" level=info msg="Running kubelet --address=0.0.0.0 --allowed-unsafe-sysctls=net.ipv4.ip_forward,net.ipv6.conf.all.forwarding,net.ipv6.conf.all.disable_ipv6 --anonymous-auth=false

References

https://docs.rancherdesktop.io/how-to-guides/provisioning-scripts/

Posted in Development, Security | Tagged , , | Leave a comment

Show container info on hosted static webpage with nginx server side includes

Posted on October 13, 2023 by erictummers

We are creating a CICD pipeline for apps hosting on Kubernetes. One of the things we like to do is to have the build number or any other version information visible in the UI. To achieve this with static webpages hosted in nginx we are using the server side include module and an environment variable.

Setup

The static webpages are hosted in nginx. In the html file(s) we add a special tag for server side include to print a variable.

<p>
    Hello there you are running container version 
    <!--# echo var="version" default="unknown" -->
</p>

To enable the server side include module we provide a template that nginx processes before starting. This template will override the default website configuration to enable the server side include module and expand an environment variable.

# short version of the template
server {
    listen       80;
    location / {
        root   /usr/share/nginx/html;
        index  index.html index.htm;
        ssi    on;
        set    $version "${CONTAINER_VERSION}";
    }
}

Both the html file(s) and the template are copied into the container during docker build. When the container starts nginx will proces the template, enable ssi and expand the environment variable CONTAINER_VERSION. When hosting a html file with a server side include it will be executed and we see the version as part of the webpage.

For a detailed description and working code see my GitHub https://github.com/erictummers/nginx_k8s_ssi_demo

Posted in Development | Tagged , , , | Leave a comment

Unittest with RestSharp

Posted on July 17, 2023 by erictummers

We’re developing an aspnet core website with webapi backend all on a cloud platform. The auth part is implemented with openidconnect and cookies. Every tab is a new application to reduce release and test times. For the website / applications we have a razor class library that contains the main layout. See all posts in this series cloudnative

Our adapters use IRestClient that is injected during construction. Only using the interface is hard, since it is very limited. Ease of use is with the extension methods, but needs some extra setup. The magic is in the SetupRestClient helper method that configures the use of json and returns the response (with data) constructed with the GetContent* helper methods. Below an example for testing the get of 6 records.

using RestSharp;    // see https://restsharp.dev
using AutoMoqCore;  // see https://github.com/thomashfr/AutoMoqCore

private static void SetupRestclient(AutoMoqer autoMoqer, RestResponse response)
{
  var defaultSerializer = new SerializerConfig();
  defaultSerializer.UseDefaultSerializers();
  var serializers = new RestSerializers(defaultSerializer);
  var fakeClient = autoMoqer.GetMock<IRestClient>();
  fakeClient.SetupGet(x => x.Serializers).Returns(serializers);
  fakeClient.Setup(x => x.ExecuteAsync(
    It.IsAny<RestRequest>(),
    It.IsAny<CancellationToken>()))
  .ReturnsAsync(response);
}

private static string GetContentFor6Records()
{
  var items = Enumerable.Repeat(new Record(), 6);
  return JsonSerializer.Serialize(items);
}

[TestMethod]
public async Task GetRecords_returns_6_items()
{
  var autoMoqer = new AutoMoqer();
  // arrange
  SetupRestclient(autoMoqer, new RestResponse { Content = GetContentFor6Records() });
  //act
  var testObject = autoMoqer.Create<RecordApiAdapter>();
  // assert
  var items = await testObject.GetRecords();
  Assert.AreEqual(6, items.Length);
}
Posted in Development | Tagged , | Leave a comment

Call upstream api with token from jQuery

Posted on July 14, 2023 by erictummers

We’re developing an aspnet core website with webapi backend all on a cloud platform. The auth part is implemented with openidconnect and cookies. Every tab is a new application to reduce release and test times. For the website / applications we have a razor class library that contains the main layout. See all posts in this series cloudnative

In Call upstream api with token we described how to get the token in the controller – the backend. Now we need to call the api from jQuery – the frontend. For this we implemented methods on the controller that can be called from jQuery. Because the request is for the samen site, the cookie is send with the request and the controller can access the token for us and do the upstream api request.

// controller = backend
public class RecordController : Controller {
  // other methods for aspnet core mvc like Index
 
  // called from frontend with jQuery
  [Authorize]
  public async Task<IActionResult> CallupstreamApi(
        [Required(AllowEmptyStrings = false)] string recordid)
  {
    if (!ModelState.IsValid) return BadRequest(ModelState);
    // call upstream api with IRestClient and recordid
    var request = new RestRequest("record/{id}");
    request.AddUrlSegment("id", recordid);
    var response = await _restClient.ExecuteAsync<record>(request, default);
    // return json for easy parsing in jQuery
    return Json(response.data);
  }
}

// index.cshtml = frontend
function RequestRecordFromUpstreamApi () {
  $.ajax({
    type: 'GET',
    url: '@Url.Action("CallupstreamApi")',
    datatype: 'json',
    data: { 
      recordid: 'my_id_for_record_1'
    },
    success: function (record) { console.log(record.id); },
    error: function (xhr, status, error) { console.log(xhr.responseText); }
  });
}
Posted in Development | Tagged , , | Leave a comment