Using log4net

Posted on September 9, 2015 by erictummers

Image courtesy of KROMKRATHOG / FreeDigitalPhotos.netWe are using log4net in our project. It works great. Here are some links and tips to get you started.

In this tutorial on codeproject you can read all the details and even see a video about it. Very good starting point for getting to know log4net.

We plan to use MEF for IoC. In Little Trick to generate ILogger with MEF based on type you can read how to get it to work. It involves a wrapper around the log4net.ILog interface and an implementation that creates the log4net logger and redirects the calls to log4net.

As last a reference to the Log4net Layout Patternlayout documentation. With the PatternLayout you can configure the information in the output.

* Image courtesy of KROMKRATHOG / FreeDigitalPhotos.net

Posted in Development | Tagged | Leave a comment

XML data type for storing messages

Posted on September 7, 2015 by erictummers

Image courtesy of Stuart Miles / FreeDigitalPhotos.net We want to store xml message in a database for batch processing, logging and retention. For some time now Sql Server offers the xml data type for columns. This way we could index the xml, validate it with an xsd and write a blogpost about the data type :mrgreen:

Image courtesy of Stuart Miles / FreeDigitalPhotos.net

TLDR

After some testing we decided not to use the xml data type but nvarchar(max).

Pros

Microsoft has written a msdn article about when to consider the xml data type.

We can index the xml and query a group of messages with the same xml elements.

With xsd validation the columns would only accept xml that we can process.

Cons

Entity Framework recognizes the xml column as a string. There are ways to get EF to play ball, but we’re only interested in the out-of-the-box solution. This would mean conversion to and from string on every database action.

While putting some xml documents into the column (using EF) we got errors. When changing the encoding in the xml header between UTF8 and UTF16 the database would complain.

All our processing is done in dotNET code. The xsd validation in Sql Server would be an extra guard for inserts from other applications or manual scripts. This means some extra code in our program to validate the xml (again) after reading.

Conclusion

We decided to go for the nvarchar(max) data type and do the xml stuff in dotNET. This was the best solution for our problem, you might come to a different conclusion.

References

XML Data Type and Columns (SQL Server) on msdn

Posted in Development | Tagged , , , | Leave a comment

Week roundup

Posted on September 4, 2015 by erictummers

Last week recap and links:
Image courtesy of kanate / FreeDigitalPhotos.net

Image courtesy of kanate / FreeDigitalPhotos.net

What are your best reads this week? Leave them in the comments below.

Posted in Uncategorized | Tagged | Leave a comment

XPath for robust message handling

Posted on September 2, 2015 by erictummers

Consider the xml message below

<document xmlns="http://www.valid.nl/2015/test">
  <meta>
    <documentid>1234</documentid>
    <documentlocation>c:\temp\data.xml</documentlocation>
  </meta>
  <pages>
    <page id="1">
      <text bold="true">This is my text</text>
    </page>
    <page id="2">
      <text>Hello world</text>
    </page>
    <page id="3">
      <text bold="true">Next information</text>
    </page>
  </pages>
</document>

Mapping this to a class would make it tightly coupled. Any change to the message would mean an update to the class. We expect some minor changes to the XSD / XML message but only want to recompile on “real” impact like extra (needed) information. Changing the naming of the root node or grouping nodes should not result in a new deployment.

We plan to do this by querying the message with XPath. That searches for the information in the xml and does not need a complete mapping of the message to a class. Code says more than a thousand words.

var doc = new XmlDocument();
using(var reader = new XmlTextReader(path)) {
    // needed to ignore the namespace
    reader.Namespaces = false;
    doc.Load(reader);
}
var docid = doc.SelectSingleNode("//documentid");
var doclocation = doc.SelectSingleNode("//documentlocation");
var boldTexts = doc.SelectNodes("//text[@bold='true']");

Console.WriteLine("DocumentId: {0}", docid.InnerText);
Console.WriteLine("DocumentLocation: {0}", doclocation.InnerText);
foreach(XmlNode boldText in boldTexts) {
    Console.WriteLine("Found bold text: {0} on Page: {1}",
        boldText.InnerText,
        boldText.ParentNode.Attributes["id"].Value);
}

Changing the tags <document>, <meta>, <pages>, <page> or adding things like <headings> has no impact and the code would output the same.

What do you think?

References

W3Schools XPath Tutorial
Ignore namespace answer on stackoverflow

Posted in Development | Tagged , | Leave a comment

Data Access to MsAccess database

Posted on August 31, 2015 by erictummers

msaccess
We’re developing an adapter to connect to a legacy application. The connection is a MsAccess database that we create as an export. The legacy application will import the database.

My initial thought was to use Entity Framework to write to the MsAccess database. It turns out that OleDB is not supported by EF. Since the solution is temporary we decided to create our own Data Access Layer.

We only export the data and create a new database for every export. This means insert-only and no other operations. Most of the work is writing the OR-mapping, but that is a one time exercise. During testing we discovered that working in batch (array of records) improved the throughput from 2 minutes to 2 seconds for 1500 records. A simplified implementation of the Save method is below

public void Save(object[] records) {
    var builder = new OleDbConnectionStringBuilder {
        // provider present from Windows 2000 and up
        Provider = "Microsoft.ACE.OLEDB.12.0",
        DataSource = @"path_to_database_file"
    };
    using(var connection = new OleDbConnection(builder.ConnectionString)) {
        connection.Open();
        foreach (var record in records) {
            using (var command = connection.CreateCommand()) {
                command.CommandText = "INSERT INTO Table1 (Field1) VALUES (@Field1)";
                command.Parameters.Add("Field1", OleDbType.Integer).Value = 1;
                command.ExecuteNonQuery();
            }
        }
        connection.Close();
    }
}

As Dick Moffat stated on SDC last year: MsAccess is here to stay.

Posted in Uncategorized | 1 Comment