Powershell module

Posted on May 11, 2018 by erictummers

Windows PowerShell is a Windows command-line shell designed especially for system administrators. Windows PowerShell includes an interactive prompt and a scripting environment that can be used independently or in combination.
docs.microsoft.com

powershell-jonatan-pie-234237-unsplash

We are building a system and need to transform-and-load data. This feature must be implemented with data imported from and published to webservices. Time to dust off my Powershell skills from 2015 (open your application to Powershell) and get to work.

With one commandlet we load and transform the data (import-webdata) This writes the new object to the output. Next on the pipeline will be another commandlet that sends the transformed data to the other webservice (publish-structureddata)

To test this we use unittest for the (internal) C# code and Pester for the powershell part and the integration testing. Got to love automation 🙂

Want to start building a Powershell module too? Here are some links:

Posted in Development | Tagged , , | 1 Comment

Elasticsearch

Posted on April 17, 2018 by erictummers

Elasticsearch is a distributed, RESTful search and analytics engine capable of solving a growing number of use cases. As the heart of the Elastic Stack, it centrally stores your data so you can discover the expected and uncover the unexpected.
https://www.elastic.co/products/elasticsearch

marten-newhall-226045-unsplash

We are building a system to search for information. Searching (and finding) is what Elasticsearch is made for. It uses apache lucene to store and index documents and runs on java.

The same metadata we put into Neo4J we want to find with Elasticsearch. The heavy lifting of syncing the data is done by the neo4j-to-elasticsearch plugin.

So far we’ve learned that controlling the number of shards in DEV and TEST is a must to get predictable search results. This is caused by the limited amount of documents (only one million) in these environments. Read more

Want to start searching too? Here are some links:

Posted in Tooling | Tagged , | Leave a comment

Neo4j

Posted on April 10, 2018 by erictummers

I am added to a team that is using Neo4j as a datastore. After my last Neo4j experience I had to dust of my skills 🙄 I still don’t think in graphs but am getting there.

monkey string

We store metadata about datasets in a graph. Using some smart logic we try to couple different datasets. For this we needed special functions like shortest path between nodes. Neo4j is great at these.

Neo4j is developed in Java and offers extensibility trough plugins. One of the plugins we plan to use is the UUID, unmutable unique identifier. This adds a solution for the evil ID used by Neo4j.

Looking forward to learning more about graphdatabases? Here are the resources I used:

Posted in Tooling | Tagged , | 1 Comment

Colouring with Google and Bing

Posted on February 21, 2018 by erictummers

We all know google is the synonym to searching (and finding) stuff on the internet. Microsoft has bing for this. Both platforms offer custom searching as an API. So developers can take advantage of the massive resources both companies have and build that into their next awesome new product.

My kids use google to search for images they can print and colour. They just want to search, print and colour. A simple workflow that can be automated using the custom search API.

Print

One of the features is the full page printing of the image. Sometimes the image fills up multiple pages or just a small portion when printed. Using google I found some samples to scale and print an image.

Google vs Bing

Comparing the search API provided by Google and Bing they are very similar. I created the Colouring app with both API’s and my kids were happy with the results from both platforms. Happy customers 😉

Google Cloud platform Bing
Documentation CSE:list Bing Images Search
Pricing €5.00 per 1000 calls €3.37 per 1000 calls (S3)
Free? 100 calls per day 30 days
Other Max results is 10 Max results is 150

As you can see the free option is only available on google. So I went that way first. After hitting the 100 calls limit on google I migrated to the Bing Api with a trial license. Nobody noticed the change since the customer is not interested in the technical details, but in the finished product.

Colouring Surfer

References

Posted in Development | Tagged , , , , , , , | Leave a comment

Setup Boot Camp with hyper-V support

Posted on January 31, 2018 by erictummers

For development I’m using Windows in a Parallels VM on my MacBook Pro. To use docker on Windows I need to enable Hyper-V.

Parallels has an option called “nested virtualisation” where the guest OS can do virtualisation. For this you’ll need a Pro subscription, which I don’t have. This means I cannot use nested virtualisation.


My only option is setting up Boot Camp and boot native into Windows. My hurdles steps are listed below.

Disk space

The first step in the Boot Camp Assistant is setting up the Boot Camp partition. I want to assign 80Gb since I’m installing Windows, Visual Studio, Docker and some other development stuff. My disk reports 105Gb of free space so that’s okay .. not ..

The assistant lets me assign max 48 Gb. What is going on? Seems that local time machine backups secretly eat into your available disk space. Luckily there is a way of removing these hidden files. The assistant now lets me create the partition of 80Gb.

Remove partition

The ISO file I downloaded from the msdn subscriptions page was incompatible and resulted in an error. Looks like only the commercial version of the ISO is supported. Now I have a corrupted Boot Camp partition. To remove these I used Disk Util as described here.


After downloading the supported ISO file and rebooting (have you tried turning it off and on again) the Boot Camp assistant did the job.

Installation

After a reboot I was presented the Windows 10 installation setup. Boot Camp installed the drivers after the first login. Another reboot and Visual Studio, Docker, … all installed without issues. Happy developer 😉

Cold boot

The next day I started my MacBook into Windows and docker crashed…
The logfile presented me with this:

Virtual machine ‘MobyLinuxVM’ could not be started because the hypervisor is not running (Virtual machine ID ~some GUID~).
The following actions may help you resolve the problem:
1) Verify that the processor of the physical computer has a supported version of hardware-assisted virtualization.
2) Verify that hardware-assisted virtualization and hardware-assisted data execution protection are enabled in the BIOS of the physical computer. (If you edit the BIOS to enable either setting, you must turn off the power to the physical computer and then turn it back on. Resetting the physical computer is not sufficient.)
3) If you have made changes to the Boot Configuration Data store, review these changes to ensure that the hypervisor is configured to launch automatically.

Google suggested to run this command to check if Hypervisor was activated/present. The answer was False.

(gcim Win32_ComputerSystem).HypervisorPresent

There seems to be a virtualisation bug that disables hyper-v on a cold boot. For now I’ll be booting into macOS and then booting into Windows. The command above will return True this way and everything works.

Aftermath

The option of running Windows in Parallels still interested me. By installing a trial version of Parallels Desktop I was able to start the Boot Camp Windows installation from within macOS. In the trial version all options are available, so I used the nested virtualisation option. The machine creation took some time while installing the Parallels Tools. But right after the startup (again) docker crashed.

Docker also crashed when native booted into Windows. This was resolved after uninstallation of the Parallels Tools. No Parallels for now.

Posted in Development, Tooling | Tagged , , , , , | 2 Comments