Add MySql to dotnet core app in Cloud Foundry

In this post you’ll add MySql from the marketplace to a dotnet core mvc app. Mysql was added to the marketplace in this post.

Service

We could use the cli to add MySql and then hook it up to the core app. But since we’ve installed Stratos, we can do this with a wizard:

  1. Select Service: choose p-mysql
  2. Select Plan: choose 10mb (or whatever applies to your needs)
  3. Binding Params (Optional): leave empty
  4. Service Instance:
    • Create and Bind to a new Service Instance
    • Name: mydatastore

You can wait for the changes to be applied or move to Code Changes below.

Code changes

We’re going to edit the dotnet core app.

First add some nugets:

dotnet add package Steeltoe.CloudFoundry.ConnectorCore
dotnet add package Steeltoe.Extensions.Configuration.CloudFoundryCore
dotnet add package MySql.Data
dotnet restore

Then make the changes below.

In the program.cs we need to load the environment variable VCAP_SERVICES and parse it into the configuration. For this we’ll use the Steeltoe extension:

using Steeltoe.Extensions.Configuration.CloudFoundry;

public static IWebHost BuildWebHost(string[] args) =>
WebHost.CreateDefaultBuilder(args)
.AddCloudFoundry()
.UseStartup<Startup>()
.Build();

Then we need to create a MySqlConnection and inject it into the IoC container. Again with a Steeltoe extension:

using Steeltoe.CloudFoundry.Connector.MySql;

public void ConfigureServices(IServiceCollection services)
{
   services.AddMvc();
   // get the connection to the mysql service
   services.AddMySqlConnection(Configuration);
}

We plan to connect to MySql on the About page. The code below runs a query that returns some text (no mysql objects needed) but only when the connection succeeds.

using MySql.Data.MySqlClient;

public IActionResult About([FromServices] MySqlConnection dbConnection)
{
   var cmd = new MySqlCommand("SELECT \"hello world from mysql \"; ", dbConnection);
   dbConnection.Open();
   ViewData["Message"] = "Your application description page. " + cmd.ExecuteScalar().ToString() ;
   dbConnection.Close();
   return View();
}

The code changes are done. Now cf push the application.

Security

After the cf push completes the application throws an exception:

fail: Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware[0]
An unhandled exception has occurred: Unable to connect to any of the specified MySQL hosts.
MySql.Data.MySqlClient.MySqlException (0x80004005): Unable to connect to any of the specified MySQL hosts. 
---> System.AggregateException: One or more errors occurred. (Connection refused 10.244.0.145:3306) 
---> System.Net.Internals.SocketExceptionFactory+ExtendedSocketException: Connection refused 10.244.0.145:3306

This means the security is not setup for the app to communicate with the MySql endpoint. With a json file we allow communication from our app to the MySql machine over port 3306.

[
   {
      "protocol": "tcp",
      "destination": "10.244.0.145",
      "ports": "3306",
      "log": true,
      "description": "Allow traffic to mysql"
    }
  ]

With the cf statements below we install the security profile in the demo organisation, dev space.

cf create-security-group mysql mysql-asg.json
cf bind-security-group mysql demo dev
cf restart core

After the app is restarted the about page shows us “hello from mysql”.

References

https://github.com/amithn/cloudfoundry-steeltoe-mysql-app
https://steeltoe.io/docs/steeltoe-configuration/#1-2-usage

Posted in Development | Tagged , , | Leave a comment

MySql in marketplace of your Cloud Foundry

This post is a combination of information available elsewhere (links are included) and can be used to get mysql in the marketplace on your local Cloud Foundry.

Tip! Run startupcf.sh from my quickly set environment for local cloud foundry post.

MySql

Follow http://operator-workshop.cloudfoundry.org/labs/services. But first peek below 😉

git clone https://github.com/cloudfoundry/cf-mysql-release.git ~/workspace/cf-mysql-release
cd ~/workspace/cf-mysql-release
git checkout tags/v36.10.0
./scripts/update
bosh create-release
bosh upload-release

git clone https://github.com/EngineerBetter/cf-mysql-deployment.git ~/workspace/cf-mysql-deployment
cd ~/workspace/cf-mysql-deployment

# edit operations/register-proxy-route.yml manually
# edit cloud-config manually

bosh -d cf-mysql deploy \
  cf-mysql-deployment.yml --vars-store mysql-creds.yml \
  -o ./operations/add-broker.yml \
  --vars-file bosh-lite/default-vars.yml \
  --var cf_mysql_external_host=p-mysql.$SYSTEM_DOMAIN \
  --var cf_mysql_host=$BOSH_ENVIRONMENT \
  --var cf_admin_password=$CF_ADMIN_PASSWORD \
  --var cf_api_url=https://api.$SYSTEM_DOMAIN \
  --var cf_skip_ssl_validation=true

I ran into some trouble with the deployment. The url in the register-proxy-route.yml was not working; so I downloaded the file and changed the url in register-proxy-route.yml to my local copy.

In the lab you must find out how to edit the cloud-config. Here’s our take:

# download the cloud-config to a local file
bosh cloud-config > cloud.yml
# edit the local file with vs code
# add massive to vm_types (see cloud.yml below)
code cloud.yml
# upload the cloud-config
bosh update-cloud-config cloud.yml
...
vm_types:
- name: minimal
- name: small
- name: small-highmem
- name: default
- name: massive

Cloud Foundry

After deployment of MySql to Bosh you’ll need to register it in Cloud Foundry. It is in the lab, but we’ve added it for completeness.

bosh deployments
bosh errands -d cf-mysql
bosh run-errand -d cf-mysql broker-registrar 

Now you’ll have MySql in the Marketplace

Posted in Development, Tooling | Tagged , | Leave a comment

Quickly set environment for local Cloud Foundry

We’ve been playing with our local Cloud Foundry installation.

After “save state” and system shutdown some settings are lost. To get our environment quickly setup for local Cloud Foundry we run a script:

source ~/startcf.sh

The contents of the script file is based on the posts on my blog. This sets up the bosh and cf command.

export BOSH_CLIENT=admin
export BOSH_CLIENT_SECRET=`bosh int ~/deployments/vbox/creds.yml --path /admin_password`
export BOSH_ENVIRONMENT=192.168.50.6
export CF_ADMIN_PASSWORD=`bosh interpolate --path /cf_admin_password ~/workspace/cf-deployment/deployment-vars.yml`
export SYSTEM_DOMAIN=bosh-lite.com

sudo route add -net 10.244.0.0/16     192.168.50.6 # Mac OS X

It’s the first command we run in a new terminal window.

Posted in Development | Tagged | Leave a comment

Time drift on Cloud Foundry

On April 24th the ssl certificate of cloudfoundry is renewed.
buildpacks.cloudfoundry.org ssl certificate valid-from april 24th

Directly after that I got errors on the certificate:

**ERROR** Unable to install Libunwind: Get https://buildpacks.cloudfoundry.org/dependencies/manual-binaries/dotnet/libunwind-1.2.1-linux-x64-80af276a.tgz: 
x509: certificate has expired or is not yet valid

Workaround

My workaround was to create an offline buildpack.

Fix

On their Slack channel I got the tip to look at the time setting on Cloud Foundry.
https://starkandwayne.com/blog/bosh-director-time-drift-2/

bosh ssh -d cf -c date

Listing confirms time drift: CF thinks it is still April 17th. Fixed it by setting the date to April 26th:

bosh ssh -d cf -c 'sudo date --set="2019-04-26 08:37:00.000"'

Now the buildpacks work again.

Posted in Development | Tagged , | Leave a comment

Offline buildpack for Cloud Foundry

This post is a combination of information available elsewhere (links are included) and can be used to get you’re own offline buildpack on Cloud Foundry.

Prerequisites

install go (https://ahmadawais.com/install-go-lang-on-macos-with-homebrew/)

Dotnet-core buildpack

Instructions on https://github.com/cloudfoundry/dotnet-core-buildpack/tree/v2.0.1

cd ~/workspace
git clone https://github.com/cloudfoundry/dotnet-core-buildpack
cd dotnet-core-buildpack
git checkout v2.0.1 #I'm on v2.0.1, you may need another version
export GOPATH="${HOME}/.go"
export GOROOT="$(brew --prefix golang)/libexec"
export PATH="$PATH:${GOPATH}/bin:${GOROOT}/bin"
source .envrc
(cd src/*/vendor/github.com/cloudfoundry/libbuildpack/packager/buildpack-packager && go install)
buildpack-packager --cached
cf create-buildpack offline-dotnet-core dotnet-core_buildpack-cached-v2.0.1.zip 1

Warning! buildpack is 841MB.

Now you can use you’re offline buildpack with the -b parameter:

cf push core -b offline-dotnet-core
Posted in Development | Tagged , | Leave a comment

Deploy your first dotnet core app to Cloud Foundry

After installing Cloud Foundry it is time to deploy your first dotnet core app.

Make sure you’re using the right framework/sdk version. If the version is not supported by Cloud Foundry you’ll get logging like below

-----> Dotnet-Core Buildpack version 2.0.1
-----> Supplying Dotnet Core
-----> Installing libunwind 1.2.1
       Download [https://buildpacks.cloudfoundry.org/dependencies/manual-binaries/dotnet/libunwind-1.2.1-linux-x64-80af276a.tgz]
-----> Installing dotnet 2.0.3
       Download [https://buildpacks.cloudfoundry.org/dependencies/dotnet/dotnet.2.0.3.linux-amd64-b56d13fc.tar.xz]
-----> Finalizing Dotnet Core
-----> Restore dotnet dependencies
       /tmp/contents738113418/deps/0/dotnet/sdk/2.0.3/Sdks/Microsoft.NET.Sdk/build/Microsoft.NET.TargetFrameworkInference.targets(135,5): 
error : 
The current .NET SDK does not support targeting .NET Core 2.2.  
Either target .NET Core 2.0 or lower, or use a version of the 
.NET SDK that supports .NET Core 2.2. [/tmp/app/newcore.csproj]
       /tmp/contents738113418/deps/0/dotnet/sdk/2.0.3/Sdks/Microsoft.NET.Sdk/build/Microsoft.NET.TargetFrameworkInference.targets(135,5): error : The current .NET SDK does not support targeting .NET Core 2.2.  Either target .NET Core 2.0 or lower, or use a version of the .NET SDK that supports .NET Core 2.2. [/tmp/app/newcore.csproj]
       **ERROR** Unable to run dotnet restore: exit status 1
Failed to compile droplet: Failed to run finalize script: exit status 12
Exit status 223
Stopping instance 872072e8-a3db-4b10-94ba-9dd207587dcc
Destroying container
Error staging application: App staging failed in the buildpack compile phase
FAILED

You can force the use of a supported dotnet core version by adding a global.json to the root of your project. This can be generated by the dotnet cli.

{
   "sdk": {
      "version": "2.0.3"
   }
}

Now create a simple website with the cli and push it to cloud foundry. You should be greeted with success messages 😉

mkdir core
cd core
dotnet new global --sdk-version 2.0.3
dotnet new mvc
cf push core

In Stratos the new app is visible. There you can view logging, scale up/down and configure routes.

The new app is available on http://core.bosh-lite.com (default route)

Posted in Development | Tagged , | Leave a comment

Install Cloud Foundry on macOS with Virtualbox

We’re investigating Cloud Foundry for our PAAS solution. At home I’ve installed it on my mac with virtualbox. This post is a combination of information available elsewhere (links are included) and can be used to get you’re own local Cloud Foundry up and running.

Prerequisites

Install VirtualBox (https://www.virtualbox.org/wiki/Download_Old_Builds_5_2)
Install VirtualBox extension pack
Install xcode

xcode-select --install

Install Homebrew (https://brew.sh) and use it to install bosh-cli, openssl and node.js

brew install cloudfoundry/tap/bosh-cli
brew install openssl
brew install node

Now start installation …

Bosh-lite

Follow https://bosh.io/docs/bosh-lite/ and you’ll have bosh-lite running. JUST DON’T INSTALL ZOOKEEPER! The script I’ve used is below. The last line adds a route to the system that is crucial for the cloud foundry cli to work.

git clone https://github.com/cloudfoundry/bosh-deployment ~/workspace/bosh-deployment
mkdir -p ~/deployments/vbox
cd ~/deployments/vbox
bosh create-env ~/workspace/bosh-deployment/bosh.yml \
  --state ./state.json \
  -o ~/workspace/bosh-deployment/virtualbox/cpi.yml \
  -o ~/workspace/bosh-deployment/virtualbox/outbound-network.yml \
  -o ~/workspace/bosh-deployment/bosh-lite.yml \
  -o ~/workspace/bosh-deployment/bosh-lite-runc.yml \
  -o ~/workspace/bosh-deployment/uaa.yml \
  -o ~/workspace/bosh-deployment/credhub.yml \
  -o ~/workspace/bosh-deployment/jumpbox-user.yml \
  --vars-store ./creds.yml \
  -v director_name=bosh-lite \
  -v internal_ip=192.168.50.6 \
  -v internal_gw=192.168.50.1 \
  -v internal_cidr=192.168.50.0/24 \
  -v outbound_network_name=NatNetwork
bosh alias-env vbox -e 192.168.50.6 --ca-cert <(bosh int ./creds.yml --path /director_ssl/ca)
export BOSH_CLIENT=admin
export BOSH_CLIENT_SECRET=`bosh int ./creds.yml --path /admin_password`
sudo route add -net 10.244.0.0/16     192.168.50.6 # Mac OS X

Congratulations you’ve now got a working bosh-lite installation. You can use the bosh-cli to interact with it. Now install/deploy Cloud Foundry.

Cloud Foundry

Read up on http://operator-workshop.cloudfoundry.org/labs/deploy-cf/ but use the script below as it fills some holes.

git clone https://github.com/cloudfoundry/cf-deployment ~/workspace/cf-deployment
cd ~/workspace/cf-deployment
git checkout tags/v1.15.0
export BOSH_ENVIRONMENT=192.168.50.6
bosh update-cloud-config ~/workspace/cf-deployment/iaas-support/bosh-lite/cloud-config.yml
export STEMCELL_VERSION=$(bosh int cf-deployment.yml --path '/stemcells/alias=default/version')
bosh upload-stemcell "https://bosh.io/d/stemcells/bosh-warden-boshlite-ubuntu-trusty-go_agent?v=$STEMCELL_VERSION"
export SYSTEM_DOMAIN=bosh-lite.com
bosh -d cf deploy ~/workspace/cf-deployment/cf-deployment.yml \
-o ~/workspace/cf-deployment/operations/bosh-lite.yml \
-o ~/workspace/cf-deployment/operations/use-compiled-releases.yml \
--vars-store deployment-vars.yml \
-v system_domain=$SYSTEM_DOMAIN

Now login to you’re Cloud Foundry environment. The password for the admin user is revealed with the first statement of the script below.
More info on (http://operator-workshop.cloudfoundry.org/labs/cf-basics/):

bosh interpolate --path /cf_admin_password ~/workspace/cf-deployment/deployment-vars.yml
cf login -a https://api.$SYSTEM_DOMAIN --skip-ssl-validation
cf create-org demo
cf target -o demo
cf create-space dev
cf target -o demo -s dev

Stratos

A nice GUI for your Cloud Foundry is Stratos. You deploy Stratos to CF with details from https://github.com/cloudfoundry-incubator/stratos/tree/v2-master/deploy/cloud-foundry This is where node.js is needed.

cd ~/workspace
git clone https://github.com/cloudfoundry-incubator/stratos
cd stratos
git checkout tags/stable -b stable
npm install
npm run prebuild-ui
cf push

Now logon to Stratos on https://console.bosh-lite.com with Admin and the password from the first line in the previous script.

Save state the VirtualBox machine before shutdown to preserve your environment. After a restart create the route again.

Extra

If you’ve already installed node, make sure to update it to the latest version:

brew update
brew upgrade node
npm update -g npm
Posted in Development | Tagged , , , , | 1 Comment