Parse IIS logs with powershell

With the log4j CVE we’re checking our IIS logs in detail. To automate this we’ve created a powershell script that parses the logs and provides query access. Below a script that searches for 502 responses and prints the fields we need for investigation.

$logFolder = 'C:\inetpub\logs\LogFiles\W3SVC1'

# sort with oldest file first
$files = Get-ChildItem -Path $logFolder -Filter '*.log' | sort name
foreach($file in $files) {
   # skip the 3 header lines and remove the #Fields: part to be able to use the first line as headers 
    $log = get-content "$logFolder\$file" | select-object -skip 3 | foreach-object { $_ -replace '#Fields: ', ''} | convertfrom-csv -Delimiter ' '
   # now search for 502 status and print the fields we need in a table
    $log | where sc-status -eq '502' | Select-Object -Property date, time, s-ip, cs-method, cs-uri-stem, cs-username, sc-status | format-table
}

No screenshot – because of security đŸ˜‰

Posted in Uncategorized | Tagged | Leave a comment

R Shiny dashboard inspection with profvis

We’re building a R Shiny dashboard to display big datasets. In our production environment we sometimes experience slow performance. Profiling with the profvis package is our plan to fix this.

The profvis package needs to be installed. Then it can be used by encapsulating the startcode in a profvis call.

# only once
install.packages("profvis")

# in your startcode
library(profvis)
profvis({
   shiny::runApp(app)
})

After the R Shiny dashboard starts we followed a test scenario and stopped the app. Then R Studio showed us the profile data in two tabs (images below). One tab with a flame graph so you can see the spikes in (wait) time / memory during the test scenario. The other tab shows the same data in a table per file / function.

Flame Graph tab with memory and time per code line
Data tab with memory and time per file/function

We now have a better understanding where our dashboard is performing less optimal. After changing the code we can start another profvis session to see if the changes fix or add to the problem.

References

Posted in Uncategorized | Tagged , | Leave a comment

Developing inside a Container

I’m contributing to Sonarqube-Sql-Plugin (https://github.com/erictummers/sonar-sql-plugin) and creating pull requests to merge back my changes. Getting my environment setup was easy using homebrew (https://brew.sh) but still required some work. Here is how VSCode makes this much easier with developing inside a container.

Container

With docker you can start an isolated environment that is created to do one thing – host the software. The container is setup with all the dependencies and settings needed during the initial build, so we can just use them. Installation is easy with docker desktop (https://www.docker.com/products/docker-desktop)

VSCode now has an extension that you can use to run the development environment inside a container. I installed the remote containers extension (https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) and added the .devcontainer folder. The dockerfile contents I copied from Microsoft’s sample container for java (https://github.com/microsoft/vscode-dev-containers/tree/main/containers/java/.devcontainer) Now I can dev – build – repeat in a container.

VSCode project

After reopening the project in the container (F1 > Remote-Containers: Rebuild and Reopen in Container) VSCode started loading everything I needed to develop. Me being a Microsoft developer, I didn’t know the pom.xml contained the build target information. The maven build targets are offered with a run button next to them for easy use. I was able to build the jar file and test it within minutes.

The .devcontainer folder can be added to Github so everyone can use it. No more “works on my machine” đŸ˜‰

References

developing inside a container – vscode (https://code.visualstudio.com/docs/remote/containers)

Posted in Development, Tooling | Tagged , , , , | Leave a comment

Code Notes snippet manager

Like many developers I have some snippets with code. These snippets are dear to me and I use them all the time. Without a real place to store them I would be lost. This is where Code Notes comes in.

image from https://lauthieb.github.io/code-notes/

Code Notes (https://lauthieb.github.io/code-notes/) is a standalone application that stores my snippets. It is simple and intuitive to use and deploy – just put the files in your My Documents and use it from there. There is the option of connecting to github for their gist feature – but I have no internet connection from work :(. Maybe someday …

You never miss this tool until you’ve used it. Now I can’t work without it. Go download it, use it and contribute to it (https://github.com/lauthieb/code-notes).

Posted in Development, Tooling | Tagged , | Leave a comment

macOS Big Sur update without issues

My macbook became my work-from-home device and I was hesitant to update to Big Sur. The question today is “when to update, not if”. So I made a backup, had my trusty macbook 2009 ready for emergencies and started the installer.

TLDR: no issues on my MacBook Pro (Retina, 15-inch, Mid 2015) except for Little Snitch 4 which is incompatible and I had to upgrade to version 5 for €25

Hardware

MacBook Pro (Retina, 15-inch, Mid 2015)
Processor 2,2 GHz Intel Core i7
Memory 16 GB 1600 MHz DDR3
SSD 256 Gb
Graphics Intel Iris Pro 1536 MB

I’ve written about my update to 10.14 (Mojave update without issues) and updated to 10.15 sometime ago without any issues.

Installation

After the download of 10+ Gb the installation completed in about an hour. I left the scene and don’t have an exact number. It was ready when I returned.

Success

After the first login I was presented with some privacy pop-ups and some blocked extensions. Again I had to upgrade little snitch just like with Mojave. I allow the app store to just update everything when available (like on iOS) and all other apps were up-to-date.

All other apps just worked. I was most happy about Camtasia 2 and Alfred 3 still working. Check https://roaringapps.com for compatibility of your favourite apps. But keep an eye on the versions – since Little Snitch is supported on Big Sur but not version 4.

Failure

Unfortunately Parallels Desktop 11 had stopped working. Never really used it, so I won’t be buying the upgrade.

Work around

Logitech Control Center was a disappointment. I only needed my Logitech Performance MX mouse extra button(s) configured to trigger Mission Control and that was not possible (for me). Turns out that is part of macOS settings > Mission Control > Mouse Button xxx.

On Monday I can work-from-home with an updated macbook. Everything just works.

Why upgrade?

1. I want to have my macbook running the latest and greatest macOS (patches)
2.New control centre and other improvements – https://www.apple.com//macos/big-sur/

Posted in Tooling | Tagged , | Leave a comment