TFS tips #1 – Install Java runtime on local server

💡 This post is part of my TFS tips where I write about how we make the most of TFS (on premise)

We are currently working on a project that needs the Java runtime to be installed on our local servers. As lazy developers we want to script this once and let TFS Release handle it from then. This way we can expand our OTAP or scale out in productie with ease.

Artifact

The default proces of Release is get agent from pool, collect artifacts, install artifacts on target. So the first thing we did was add the Java runtime installation to an artifact on the build.
The build makes one archive (zip) from the installation files and scripts. That zip file is published as an artifact on the server. This way everything is available to TFS, no hassle with access rights to fileshares.

DSC

Our standard for script installation (and checking) of software is Desired State Configuration.
The complexity with this DSC is the required reboot after the Java runtime installation. There is a way to do this with DSC by allowing the DscLocalConfigurationManager to reboot if needed and to signal that a reboot is needed.
For this we create one Dsc script and one powershell script that applies this Dsc. The important part of powershell script is below:

# create mof
DscJavaServer -ConfigurationData $ConfigData -Verbose
# allow the server to be rebooted
Set-DscLocalConfigurationManager .\DscJavaServer -Verbose -Force
# apply DSC (install java if needed)
Start-DscConfiguration DscJavaServer -Wait -Verbose -Force

The contents of the Dsc file:

configuration DscJavaServer
{       
  Import-DscResource -ModuleName 'PSDesiredStateConfiguration'
      
  Node $AllNodes.NodeName
  {
    Write-Verbose "Configuration for $NodeName"
    Write-Verbose "Installation files should be in $DeployLocation"
 
    # This allows the reboot
    LocalConfigurationManager
    {
      RebootNodeIfNeeded = $true
    }
 
    Script Java
    {
      GetScript  = { return @{} }
      TestScript = { return Test-Path 'HKLM:\SOFTWARE\JavaSoft\Java Runtime Environment' }
      SetScript  = {   
        $installer = "$($using:Node.DeployLocation)\jre-8u144-windows-x64.exe"
        Start-Process $installer -ArgumentList '/s' -Wait
        # signal reboot
        $global:DSCMachineStatus = 1
      }
    }   
  }   
}

Release

The release definition will push the artifact to the target server, unpack the archive (zip) and execute the powershell script.
Now we have a release pipeline that can make sure the Java runtime is on a target machine. If it is not the installation will be done including the required reboot.

Posted in Development, Tooling | Tagged , , , | Leave a comment

SDN event June 2018 – My scrum experience

The SDN – Software Development Network – is a special interest group for dutch developers. Four times a year they organise an event where people present and talk about their passion.

As a member of the SDN you are aware of the latest developments. You are part of a network of professional developers who assist each other in word and deed. This means there is a technical helpdesk at your fingertips so you can book considerable time savings in solving problems.
sdn.nl with Google translate

I’ve been working in a scrum team for two years now at the Statistics Netherlands. Today I shared my experience with the audience. Slides direct download (dutch).
Eric presenting

Slides and recordings of all talks available at SDN.nl.

Posted in Conference | Tagged | Leave a comment

Adding SonarQube to TFS build

We’re moving toward DEVOPS with our team and want to automate code quality checks. Another team already installed a SonarQube server for their python development and advised us to try it for our dotnet development. Now we know why the other team is stocked about SonarQube. We are sure you will be setting this up for your own team when you see how easy the setup is and the huge quality gain it brings.

Service Endpoint

First add a new service endpoint to your TFS by going to settings > services. Put in the url (root of the sonarqube server: http://your_sonar_server) and your token (generated in sonarqube, hover over the icon at the end of the input) and give it a suitable name.

Build

After installing the sonarqube build tasks on TFS and adding javaruntime to your build agents you can add sonarqube to your builds. There are three tasks you should add: prepare, run and publish. Only the first task (prepare) needs to be configured to the service endpoint you created before. The project key and project name are up to you and will be created if they don’t exist (like the first time you run the build) We’ve set the project version to the buildnumber which is not the case in this screenshot.

Report

Now queue a build and watch sonarqube do it’s magic in the logging. You’ll notice the actual analysis is done on the build agent, not on the sonarqube server. This is why you need the java runtime on the build agent. After the publish you can browse to the sonarqube server and find your project in the list of projects. Open the details page and find something like below:

The bugs are mostly caused by not validating parameters like assuming not null.
Vulnerabilities are about known security issues.
Code smells tell you about to much nesting of to much if statements.
Code coverage from unittests is reported.
Duplication reports duplicate code.
Everything combined gives you the technical debt.
You can setup rules (or use the default) that determined the quality gate result. Make sure you’re in the green :mrgreen:

References

SonarQube VSTS/TFS documentation
SonarQube build tasks

Posted in Tooling | Tagged , , , , | Leave a comment

Powershell extension method

We are testing our Powershell Module with Pester. Since the data is some sort of graph we need to work with indexers / lists all the time. After a small brainstorm session we decided on an Extension Method approach.

Reading the post Extension Methods in Windows PowerShell from Bart de Smet we started to develop. After some attempts we ended up with an XML containing the scriptmethod we needed. Important tip is to handle the $args in a special way: assign to local variable to prevent getting lost …

<?xml version="1.0" encoding="utf-16"?>
<Types>
 <Type>
  <Name>Company.RootNode</Name>
   <Members>
    <ScriptMethod>
     <Name>GetById</Name>
     <Script>
      switch ($args.Count) {
       1 { $req = $args[0];[System.Linq.Enumerable]::First($this.Children, [System.Func[Company.Node,System.Boolean]]{ param($x) $x.Id -eq $req })}
 default { throw "No overload for Uses takes the specified number of parameters." }
      }
    </Script>
   </ScriptMethod>
  </Members>
 </Type>
</Types>

You can load the extension method into powershell with the following command, where RootNodeExtensions.ps1xml is the name of the file containing the xml

Update-TypeData -prependPath RootNodeExtensions.ps1xml

Now the RootNode has the extension method GetById, so we can use $root.GetById(“12345″) that returns the first node in the Children property that has the Id==”12345” or throws an exception. Sounds like testing 😉

Posted in Development | Tagged , , , | Leave a comment

Powershell module

Windows PowerShell is a Windows command-line shell designed especially for system administrators. Windows PowerShell includes an interactive prompt and a scripting environment that can be used independently or in combination.
docs.microsoft.com

powershell-jonatan-pie-234237-unsplash

We are building a system and need to transform-and-load data. This feature must be implemented with data imported from and published to webservices. Time to dust off my Powershell skills from 2015 (open your application to Powershell) and get to work.

With one commandlet we load and transform the data (import-webdata) This writes the new object to the output. Next on the pipeline will be another commandlet that sends the transformed data to the other webservice (publish-structureddata)

To test this we use unittest for the (internal) C# code and Pester for the powershell part and the integration testing. Got to love automation 🙂

Want to start building a Powershell module too? Here are some links:

Posted in Development | Tagged , , | 1 Comment

Elasticsearch

Elasticsearch is a distributed, RESTful search and analytics engine capable of solving a growing number of use cases. As the heart of the Elastic Stack, it centrally stores your data so you can discover the expected and uncover the unexpected.
https://www.elastic.co/products/elasticsearch

marten-newhall-226045-unsplash

We are building a system to search for information. Searching (and finding) is what Elasticsearch is made for. It uses apache lucene to store and index documents and runs on java.

The same metadata we put into Neo4J we want to find with Elasticsearch. The heavy lifting of syncing the data is done by the neo4j-to-elasticsearch plugin.

So far we’ve learned that controlling the number of shards in DEV and TEST is a must to get predictable search results. This is caused by the limited amount of documents (only one million) in these environments. Read more

Want to start searching too? Here are some links:

Posted in Tooling | Tagged , | Leave a comment

Neo4j

I am added to a team that is using Neo4j as a datastore. After my last Neo4j experience I had to dust of my skills 🙄 I still don’t think in graphs but am getting there.

monkey string

We store metadata about datasets in a graph. Using some smart logic we try to couple different datasets. For this we needed special functions like shortest path between nodes. Neo4j is great at these.

Neo4j is developed in Java and offers extensibility trough plugins. One of the plugins we plan to use is the UUID, unmutable unique identifier. This adds a solution for the evil ID used by Neo4j.

Looking forward to learning more about graphdatabases? Here are the resources I used:

Posted in Tooling | Tagged , | 1 Comment