Most .NET developers are going to need to access relational data at some point. For somebody approaching .NET for the first time, the data access story in the .NET world can be a little overwhelming to say the least. Existing .NET developers looking to update their data access techniques can face a similar overwhelming experience. This article will help you understand the currently shipping mainstream technologies and how you can choose the one that is right for your particular needs.

There is a wide array of choices from Microsoft as well as from within the .NET ecosystem. There are many considerations to keep in mind when evaluating these technologies such as type safety, developer productivity, maintainability, concurrency, database agnosticism, performance, and scalability. One element that is almost universally accepted as a best (and only) practice these days is optimistic concurrency. Just about every data access technology that you’ll be interested in is going to use this concurrency model. That is where the similarities end.

Traditional ADO.NET

ADO.NET is a .NET brand name for all things data access from Microsoft. Typically using ADO.NET means using SqlCommand objects. Using a SqlCommand, we can pass a native T-SQL statement directly to the database. This could be a SELECT, INSERT, UPDATE, DELETE, a stored proc, or even a DDL (Data Definition Language) statement. Because ADO.NET doesn’t know too much about the database syntax, this statement is simply stored in a SqlCommand as a string and passed off directly to the database. You must use one of the Execute* methods on the SqlCommand to invoke the command. Using ExecuteNonQuery will fire the statement and assume it returns no result sets. This is useful in the Insert, Update, and Delete scenarios. To bring a result set back, we use the ExecuteReader method which will return a DataReader. A DataReader looks a little bit like a cursor which will allow us in .NET code to loop through the result set one row at a time and look at the columns. The DataReader is somewhat stupid in that it doesn’t know much about the shape or data types of the results coming back so you must use untyped access to get the columns as such in the code below:

It is also important to note that the connection to the database remains open during this looping. You may want to reduce the amount of work you do in the loop to minimize the amount of time the database connection is held open. This will make your application more scalable since database connections are a finite resource.

If you are only returning a single value (like a count or exactly one column from one row) you can use the ExecuteScalar method which is a bit easier to use than a DataReader.

SqlCommands and DataReaders in general let you be very precise about the SQL that is sent to the server and control exactly when the connection is opened and closed. You have to provide a bit more muscle in getting the data out of the SQL and coerced into .NET data types, but for performance sensitive areas of your applications, this could pay off. The other thing to be aware of is that there are a different set of objects (Commands, Connections, DataReaders) for each relational database provider. So if you need to move from SQL Server to ODBC to Oracle, you’ll need different objects. Fortunately each of these objects are based on an interface and you can program through the interface by using a DbProviderFactory class to create the concrete types based on configuration data. You will have to be sure to use ANSI compatible SQL in your attempt to be database agnostic, which may run counter to providing high speed data access which often requires developers to use native & unique syntax to a particular relational database engine. Obviously your mileage might vary here.

DataSets, DataTables, Data & Table Adapters

With DataReaders you need to be careful about how long you keep your connection open while iterating. Sometimes it is better to pull back your entire result set into memory, release your connection to the database, and then process the result set afterwards. This is where DataSets can be helpful. If you have a batch processing scenario that requires the entire result set in memory or you need to provide end user editing in a grid control, then DataSets are a good place to hold that data.

One of the primary benefits of a DataSet is that it not only caches the data in an efficient memory structure, but it can provide change tracking (what records were added? Deleted? Changed? What were the original values of each column, etc). DataSets work collaboratively with a SqlDataAdapter object, which is simply a composite of 4 SqlCommands (InsertCommand, SelectCommand, UpdateCommand, and DeleteCommand) mapping to the appropriate CRUD behavior (create, read, update, and delete).

DataSets can contain multiple result sets, each mapped into its own DataTable. Tables can be related to each other with Relations which mimic foreign keys. DataSets take care of master-detail linkages and keep everything "hooked up" when inserting records into parent and child tables with identity columns. Each table will require its own SqlDataAdapter. If you are doing a mix of insert, updates, and deletes and your database has referential constraints, you’ll need to orchestrate the changes, making sure parent records are inserted before children and that deletions happen in the reverse order.

DataSets/DataTables come in two flavors: typed and untyped. Untyped DataSets simply infer the table and column definitions after executing the SqlDataAdapter.Fill method. Accessing the columns must be done similarly to DataReaders using column names in quotes or by positional reference. Typed Datasets on the other hand use wizards and a design surface to generate a strongly typed object. The typed dataset is defined by an XSD document. The XSD is then turned into a strongly typed/named class that essentially sits on top of an untyped dataset. A table mappings collection on the SqlDataAdapter class allows you to use different names for your strongly typed object than that of your database table and column names.

Many people in the .NET 1.0-2.0 era used typed datasets as business entities and wrapped some additional business logic around them. Although you could map column names, you were typically tied tightly to your table design in the database. DataSets can also be converted to and from XML, including a diffgram format that retains pending change information. In many circles, DataSets have and continue to be serialized between distributed layers of an application. This makes for a pretty productive development environment, but has drawn architectural criticisms compared to properly modeled services with messages and contracts.

In .NET 1.0, DataAdapters and DataSets were different objects that really didn’t know about each other until you executed the DataAdapter Fill or Update method and passed the DataSet/DataTable to the respective method. It was possible for the table mappings in the SqlDataAdapter to get out of sync with the strongly typed dataset. To improve this situation, Microsoft introduced TableAdapters in .NET 2.0, which are really just DataAdapters that are strongly typed and embedded into the definition of the typed dataset.

Object Relational Mapping

With .NET 3.0/3.5, MS has finally released not one but two ORM toolkits. I say finally because MS previously promised an ORM toolkit to developers in the form of a project called ObjectSpaces. ObjectSpaces was never released and its demise is clouded in a series of potentially bad integrations with WinFS, Project Green, or the MS Business Framework, all of which were eventually cancelled.

The spark that reignited MS ambitions to produce an ORM was the Language Integrated Query (LINQ) project. LINQ’s goal as part of C# 3.0 and Visual Basic 9.0 was to provide a set based metaphor for dealing with heterogeneous data types including object collections, XML, and last but not least, relational data. For the relational component of LINQ, the language team came up with a project called LINQ to SQL and shipped it as part of .NET Framework 3.5.

During the same time period, the Data Programmability Group in the SQL Server team had been working on a new layer to sit over top of SQL Server’s logical data model. This new "Entity Data Model" (EDM) would allow developers to interact with their relational database using a more abstract conceptual model based on the ideas of Dr. Peter Chen. For this team, the promise of Language Integrated Query was what really brought the notion of the EDM to life, and so began the "Entity Framework" project. The Entity Framework did not ship until several months after LINQ to SQL as part of the .NET 3.5 Service Pack 1 release.

On the outside, LINQ to SQL and the Entity Framework ORMs appear very similar. Both tools use a graphical design tool and wizard to map a relational database into an object model. They can both use LINQ queries to project relational data into objects using composable queries. Both provide change tracking so that changes made to the objects can be persisted back to the database with a single method call.

Given these similarities, Microsoft has spent some considerable energy in explaining when to use each of these technologies. While these two ORMs have strong similarities in programming models, their internal architectures are quite different. These key differences are:

• LINQ to SQL has been locked directly to SQL Server, whereas the Entity Framework includes a provider model that allows it to work with any relational engine. As mid 2009 the list of available providers in the ecosystem included SQL Server, Oracle, Sybase, DB2, Informix, MySQL, PostgresSQL, SQLite, OpenLink Virtuoso, and Firebird.
• LINQ to SQL does not provide a mechanism for streaming records one at a time. Using Entity SQL, the Entity Framework can provide Data Reader style access to results using the same mapping layer used for object queries.
• LINQ to SQL provides an out of the box experience for lazy loading. Lazy loading allows the engine to automatically (by default) go out and retrieve new data as it is needed based on code that access different elements of the object graph. In the Entity Framework, you have to be explicit when you make trips to the database. Both ORMs allow you to load deep object graphs eagerly as well.
• Perhaps the largest difference between Entity Framework and LINQ to SQL is the conceptual modeling layer that EF provides. EF provides a much richer mapping layer than LINQ to SQL, allowing you to join tables together, provide elaborate class hierarchies with abstract and concrete types, and retain many-to-many relationships in your model. Future plans will see the conceptual model being used in other MS products such as SQL Server Reporting Services and SQL Server Analysis Services.
• Finally, expect to see significant investment in the Entity Framework ORM from Microsoft. Both ORMs are now managed by the same product team at MS and they have stated that while LINQ to SQL will continue to be supported and maintained, future innovation will be focused on the Entity Framework.

Other ORM Technologies

You can’t talk about ORMs in the .NET world without mentioning NHibernate. NHibernate or "NH" is a mature yet free open source framework which is a port of the popular Java ORM Hibernate. Perhaps one of the biggest distinctions of NH is its support for a Domain Driven Design workflow which purports a code-first or test-first approach to building applications. Typical NH developers build their classes first and then use an XML file to map those to a database. Unlike NH, both LINQ to SQL and Entity Framework advocate a model-first approach, and one could argue a database driven approach. Both LINQ to SQL and Entity Framework designers assume you have an existing database that needs mapping to some objects that will be derived out of your model.

Although there are other commercial and open source ORM tools available, NHibernate, LINQ to SQL, and Entity Framework are by far the most popular technologies in use today.

Enter Data Services

When it comes to building distributed applications, your mileage will vary with the array of data access technologies. Dealing with serialization and concurrency issues in those environments is beyond the scope of this article. However, it is important to mention what appears to be yet another data access technology available from Microsoft called ADO.NET Data Services, formerly code-named "Astoria". Astoria was made available with .NET 3.5 SP1.

Astoria is an HTTP service layer built on top of WCF that provides a REST-style API to your data, giving each of your elements of data a unique URI (for example, http://host/northwind.svc/Products(1) ). Out of the box, this service can be enabled for an Entity Framework model or any other IUpdateable/IQueryable data source in just a few lines of code. Data is queryable and updateable via pure HTTP verbs (PUT, POST, DELETE, and GET) using query strings and HTTP payload. Data can be serialized in either AtomPub or JSON format. The net effect is that your data model is widely interoperable with a dizzying array of potential clients and technologies.

Although you are free to build up complex query strings and HTTP payloads for just about any type of operation, the Astoria team has created client libraries to assist in these endeavors. Firstly, an ASP.NET Ajax library is available on CodePlex to allow JavaScript developers to easily work with Astoria Services. Secondly, as part of the core installation, there is a .NET Client Library which provides a natural query model using LINQ and projecting data into client side .NET objects for use by your .NET or Silverlight projects.

A new project, currently named "RIA Services" (code-named "Alexandria"), builds on top of ADO.NET Data Services by also providing rich validation and UI cues on top of your data model in a client/server model. This technology probably won’t be released until around the .NET 4.0 timeframe, but it is definitely something you should keep your eye on.

Application Frameworks

As you can see, the trend in Data Access is to abstract into higher and higher levels within your applications. As data access libraries are generating entity classes, the question around locating data validation logic can become a slippery slope. If a data access generated entity can validate correct data types, maybe it should also validate that postal codes and phone numbers are of the correct format. Should they also cross-validate postal codes with states/provinces? Should these error messages be managed inside of my entities? The waters can be muddied quite quickly.

It is therefore worthy to briefly mention a few of the more popular application frameworks that embrace a holistic view of data access in the scope of an application.

CSLA.NET (Component-based Scalable Logical Architecture) is a framework developed principally by author Rocky Lhotka in conjunction with his popular series of Business Objects books. When first released, CSLA was focused on Visual Basic 6, but over the years it has remained very current on the latest Microsoft technologies and is now principally maintained in C# and ported to VB. In CSLA, data is fully encapsulated by rich business objects that manage all behavior including persistence. CSLA facilitates the re-use of business objects in many possible client technologies including ASP.NET, WPF, WinForms, and Silverlight including distributed architectures using Web Services.

DevForce from IdeaBlade is a commercially available framework for building rich, distributed applications in Silverlight, WPF, WinForms, and ASP.NET. DevForce builds on top of the Entity Framework for its persistence layer and includes a Business Objects server for distributed architectures. DevForce is a popular application framework and has been around since 2001.

But All I Wanted Was My Data

If you can believe it, this article has been a short list of the current and popular data access technologies. I think I could safely say that the one piece of universally accepted guidance is that if the data access technology you are considering is not in the above list, then it is likely out dated or obscure and perhaps you should think twice about using it.

If you are comfortable with open source projects and you are a strong believer in Domain-Driven-Development, NHibernate is an obvious choice. There is a strong community to back you up on this decision.

If you prefer to stick with Microsoft Technologies, and you have a legacy database to start from, or prefer a model-driven or data-driven approach, then the Entity Framework is likely where you should end up.

If you prefer to work with a more structured and complete application framework, both CSLA and DevForce are worthy choices at this level and you will have community and paid-support available to back you up on each of these respectively.

The last key piece of advice to offer is that you don’t have to pick just one of these. There are pros and cons and you may have to use multiple technologies, even within the same application. If you feel the need for performance, there are no rules against dropping out of your ORM and opting for more direct access. Keep an open mind and use the best tool for the job.

—–

Barry Gervin is a founding Partner of ObjectSharp in Toronto, Canada. As a Principal Consultant, Barry provides technical leadership to his valued clients, staff, and the development community. Over his 19 year career in the IT industry, he has led many development teams to successfully deliver large software projects within tight schedules and budgets that consistently perform for their customers. Barry currently serves as a MS Regional Director in Southern Ontario and has received the Microsoft MVP Award for Solutions Architecture for the past 5 years.

digiTweet is an open source desktop Twitter client for Windows. DigiTweet provides Twitter functionality within a rich user interface built with Windows Presentation Foundation (WPF).

Listen to our interview with Mano Kulasingam (@digi_tweet) using the embedded audio player or download the MP3.

—–

CodePlex is Microsoft’s open source project hosting web site. Start a new project, join an existing one, or download software created by the community. Learn More

This edition of Practical Programming covers a set of three useful classes / namespaces from the .NET Framework that you probably aren’t familiar with (yet).

Let’s make some noise: System.Media

Producing sound from a computer is something you may have learned in Programmer Playschool and promptly forgot (like me), but it is a useful little feature to add to all types of software, not just desktop games. Playing a system sound or other sound files is very useful in monitoring environments where the users are not seated in front of a computer 24×7 (for example, in server rooms or manufacturing environments).

System.Media is a namespace that contains three useful classes for playing system sounds and .wav files. System sounds are defined in the Sounds applet of the Windows Control Panel.

What MSDN says:

The System.Media namespace contains classes for playing sound files and accessing sounds provided by the system.

The following classes are included in this namespace:

• A SoundPlayer class for loading and playing sounds in various file formats.
• A SystemSound class for representing and playing a system sound.
• A SystemSounds class that retrieves sounds associated with a set of Windows operating system sound-event types.

Playing a System sound couldn’t be easier:

This will play the .wav file associated with the asterisk system sound. SystemSounds is a simple wrapper for the five most common sounds – asterisk, beep, question, exclamation and hand.

If you have your own attention-grabbing, eardrum-busting sound file then you can play this just as easily:

Loading of large .wav file can take a while so, thoughtfully, Microsoft added support for loading a sound file asynchronously.

This simple C# console application will play the sound file after it is loaded. Not only will the loading of the sound occur asynchronously, but the player will also play the .wav file in a new thread.

Additionally, the player can load a wave file from various sources such as a URL, application resources, or a stream.

Brain the size of a planet? MemoryFailPoint

In these multi-gigabyte days it’s not often that an application runs out of memory – virtual or otherwise – but there are few things more frustrating to a user than an ‘Insufficient Memory’ exception. Applications that don’t handle these exceptions can potentially corrupt system data (and possibly damage your career!).

Thankfully, there is a simple solution: MemoryFailPoint.

What MSDN says:

Check for sufficient memory resources prior to execution.

MemoryFailPoint throws an InsufficientMemoryException prior to your operation beginning, rather than an OutOfMemoryException at some unpredictable point during the operation. Here’s the actual program that is used to launch the Space Shuttle Microsoft (j/k).

Hopefully, the real NASA application is a little more complicated than this, but you could imagine a situation where running out of available memory just after ignition could be somewhat problematic. It’s better to know before you begin that your countdown is likely to fail. Adding a MemoryFailPoint around the critical code will help with this:

When using MemoryFailPoint for more realistic purposes, you can make intelligent decisions about the execution path. For example, fast sorting algorithms can use more memory than slower but less resource hungry versions.

Secret Squirrel – System.Security.SecureString

What MSDN says:

Represents text that should be kept confidential. The text is encrypted for privacy when being used, and deleted from computer memory when no longer needed.

This sounds like something useful to do, and indeed it is! To use a SecureString you need to append characters to it like so:

Clearly, this would be a tad pointless! The unsecure password is right there in clear text for all to see (and discover). To understand this, you need to understand show strings in the .NET Framework work.

Strings are immutable. That is, once you create a string in memory, it can’t change – it can only be replaced. For example:

Looking at this you might think that there is some chunk of memory somewhere that starts off as "fred" and later becomes "bob". This is not the case. In actuality, both "fred" and "bob" will be in memory somewhere within the process space of the application. Every time you change a string, a new copy of the string is made and the old one is left for later cleanup by the garbage collector. A sniffer type application on a compromised machine could read through your unprotected memory quite easily.

So, when should you use SecureString? There are a few places in the framework that require a SecureString but the best time to use this is anywhere in your application that you prompt for something you don’t want to risk exposing to unauthorized users or processes.

SecureString should be used in conjunction with other secure mechanisms such as an encrypted configuration file or database record, SSL/TLS for web applications to network transport, etc.

References:

B# .NET Blog – http://bartdesmet.net/blogs/bart/archive/2006/03/31/3851.aspx

.NET Security Blog – http://blogs.msdn.com/shawnfa/archive/2004/05/27/143254.aspx

—–

Peter Jones is a 25 year veteran (vintage?) programmer and Microsoft MVP (ASP.NET) from New Zealand. He currently works for Intergen – a leading Microsoft Gold Partner. Peter has worked on and created systems for many industries such as Telecommunications, Manufacturing, Government, Education and more. He cut his programming teeth on dBase 3, Clipper 5 and C before moving to Windows development tools such as Visual Basic, Delphi and .NET. Currently he is working with content management systems (SharePoint and EPiServer) in Sydney, Australia. You can contact Peter and read his occasionally updated blog at http://jonesie.net.nz.

Developers solve problems. Good developers solve problems efficiently. The reward for that efficiency is [insert your preferred reward here]. Let’s assume you’re simply motivated by the satisfaction of a job well done (or maybe a promotion, a raise, longer lunch breaks, or just some extra time with your family). Whatever the motivation, the secret to solving problems efficiently is not inventing everything from scratch. Certainly the secret is not spinning your mental wheels hoping for inspiration. And most definitely the secret is not using Intellisense to arbitrarily try classes, methods, and properties until something works.

The secret to solving development problems efficiently is having the right information and the means to find the information you don’t have.

If you have been developing software for more than a week, you know that a working knowledge of language syntax and program construction (that is, "programming") is only a small part of the necessary skill set for solving problems with software. As a developer, you need to understand the technologies, processes, and methods available to you. You also need to have a set of reliable resources you can turn to when you encounter problems.

Below you will find some resource recommendations to help you find the information you need to help solve development problems more efficiently so you can get on with living the rest of your life.

Finding Answers to Specific Problems

Search Engines

I have heard colleagues muse about the changing role of "software developer," postulating that development has become a job mostly concerned with research and discovery. While that is no doubt an overstatement, the ability to find information quickly is an important skill. The place many of us turn to first is our search engine of choice whether it’s Live Search, Google, or a meta search site like Dogpile.

The appeal of most search engines is the beautiful simplicity of a single search box, but the needs of a developer are not simple when you factor in multiple versions of frameworks, programming languages with similar constructs, and behavior differences between competing products (I’m looking at you, Structured Query Language and Cascading Style Sheets). With just a short time investment, you can learn some advanced search operators and tricks to help narrow search results to exactly the information you need. For example, search for "live advanced search keywords" or "google advanced operators" and you’ll find a bunch of operators to help you become a search ninja.

In addition to your search engine of choice, Wikipedia can be an invaluable problem solving tool. I have found Wikipedia to be especially useful when searching for general topics such as common computer science algorithms and concepts. Although I often end up at Wikipedia through search engine results, Wikipedia has its own search box right on the home page (http://wikipedia.org/).

Of course not all problems will have solutions waiting to be discovered via a search engine. Some problems have already been solved in another part of your organization and you simply have no idea about the pre-existing solution. There are many tools designed to help companies deal with code asset management and search behind the firewall. One interesting product is Krugle, a specialty search engine built for indexing and searching code.

Stack Overflow

Searching is great when you know exactly what to search for and somebody else has previously solved a similar problem. When you are dealing with a more difficult issue, you may want to pose your specific problem to other developers for advice. There are plenty of discussion boards and forums for developers on the Internet to ask and answer questions.

One intriguing and useful site is Stack Overflow, a "collaboratively edited question and answer site for programmers." This innovative developer Q&A site allows users to post responses to questions and vote answers up or down based on usefulness, accuracy, etc. In addition to posting questions, you can search Stack Overflow or browse posted questions by tag. When you are posting a question on Stack Overflow, the application automatically searches existing questions and shows you matches that might already contain the answer you seek.

For obscure topics and scenarios such as bizarre error codes, Stack Overflow can be a job saver.

Twitter

Microblogging seems to be all the rage these days and some developers find services like Twitter to be an extremely effective way to share tips and send out requests for help. Twitter and similar services can be a two-edged sword, though, as they can be as much a distraction as a tool for information exchange.

If you are not familiar with Twitter, it lets you follow posts or "tweets" from other Twitter users. Tweets can be no more than 140 characters. You choose who you want to follow.

There are plenty of ways to use Twitter that will surely reduce your efficiency, but with a bit of intentionality in your approach, you can build a reasonable size network of trusted colleagues with whom you can swap questions, answers, tips, and tricks. (When your friends hear you are "on Twitter," they may demand to know your Twitter username. You might need to maintain separate Twitter accounts for work and personal use.)

Officemates, Coworkers, and IM Contacts

If you are anything like me, you find it tempting to just ask an officemate or IM contact for an answer. Of course this means solving my problem at the expense of my colleague who is distracted from doing his/her own work. With your recently acquired ninja search engine skills, take a look at some of the research on the impact of interruptions on productivity and ponder how a work culture that accepts interruptions as common practice could ultimately make you a less effective developer. Just something to think about before you start picking the brains of your cubicle mates.

Other Options

What sources do you rely on for finding answers to specific problems? Leave a comment below.

Keeping Up with New Technologies

With the rapid and continuous rate of change in the software development space, keeping up with the latest and greatest technologies could be a full-time job – in fact it is in many organizations! It really has become impossible to keep up with everything, but you should not let the fear of information overload prevent you from staying informed about significant changes directly related to your area of expertise. Being informed about new technologies helps you make informed choices about the tools, platforms, and libraries you will use to get from problem to solution.

Blogs

One easy way to stay informed about topics that might affect you is to subscribe to at least a few blogs in your area. Scott Guthrie’s blog is a good choice if you only want to get the big headline news like a new release of the .NET Framework (http://weblogs.asp.net/scottgu/). If you’re a Silverlight developer, Tim Heuer’s blog would be a good choice (http://timheuer.com/blog/). If you’re an ASP.NET developer, perhaps try the Visual Web Developer Team Blog (http://blogs.msdn.com/webdevtools/).

If checking blogs via a blog reader is not your thing, you might want to use a service that will deliver new blog posts to you via email.  FeedBlitz (http://www.feedblitz.com/) is a service I use for several blogs that I want to track even when I’m too busy to spend time reading my larger blog subscription list. Or you might want to add a few feeds to a customized browser home page through a service like Pageflakes or Netvibes.

Podcasts

Another way to stay current on technology is by listening to audio podcasts like .NET Rocks (http://dotnetrocks.com/) and Hanselminutes (http://hanselminutes.com/). Internet audio shows like these are generally packed with great technical content that helps keep you up to date. Podcasts can be an easy way to transform a tedious commute into a serious professional development opportunity. Personally I use the Zune desktop software (http://zune.net/) to automatically download new podcast episodes and sync them to my Zune device. If you don’t have a Zune, you can still grab the MP3 files from the Zune download folder and copy them to any portable media player.

User Groups

User groups are another great way to stay current on new technologies. User groups are community-run technical groups that hold events (both in-person and virtual) for members to learn about technologies related to the focus area(s) of the groups. User group events often happen outside of normal work hours, which can admittedly make it difficult for some people to attend. In addition to learning opportunities, user group events can be a great place to discuss technical issues you are tackling and increase the size of your network of colleagues that you can turn to for help.

Go to http://www.ineta.org/ to find details (including meeting times and places) for .NET-focused user groups in your neck of the woods.

Challenging Your Thinking

When building software, your existing methods for thinking about problems are used to design a solution based on the tools and information you have available. Too many times I have seen developers stuck in a mental rut and developing solutions inefficiently because of outdated or misguided thinking about the problem at hand. Case in point: I used to spend far too much time building and deploying simple tools and utilities as compiled executables until I learned to embrace scripting as a fully legitimate form of programming (after having turned my back on my past as a UNIX scripting hack). In part my thinking was influenced by the experiences of other developers who were sharing their experiences online.

If you do not have people challenging your think, I believe you will stagnate as a developer. That means talking to or reading the musings of other developers. Ideally you will find yourself disagreeing with or questioning their thoughts on a regular basis. You aren’t being challenged if you only read what likeminded people write.

Below are some sources that I find useful in challenging my thinking and assumptions. They might not work for you, which is fine, but I encourage you to find your own trusted sources that will regularly challenge your thinking.

CodingHorror.com is written by the brilliant and prolific Jeff Atwood. It has been a constant trove of insight and reasoned perspective, ranging from discussions of technical details to choosing a programming chair. (Jeff is one of the founders of the abovementioned Stack Overflow.)

JoelOnSoftware.com is written by the equally brilliant Joel Spolsky. He is the source of such gems as "The Joel Test: 12 Steps to Better Code" and "Things You Should Never Do." In addition to development topics, Joel writes about management and business issues. (Joel is the other founder of Stack Overflow.  Hmmm… perhaps I have a bias.)

Ted Neward writes a technical blog (http://blogs.tedneward.com/) that I regularly enjoy. Although I don’t read every post, I always find his pontifications worth digesting.

Daniel Crenna (http://dimebrain.com/blog) is another technical blogger whose musings I enjoy reading. Daniel is a great community champion and the founder of several open source projects. His blogging pace has slowed down in 2009 as he’s been working on some useful projects, but I keep hoping he’ll starting writing again soon. He’s a web / RIA guy, an area I’m interested in, so he stays in the ol’ blog list.

Join the Conversation

What do you do to be more efficient as a developer? What are your favorite resources for challenging your thinking? How do you keep up on technology changes in your area of expertise? How do you find solutions to specific problems? Leave a comment to share YOUR tips with us.

—–

Derek Hatchard is the content editor for http://microsoft.com/youshapeit/msdn and http://devshaped.com. He is also the founder of Crowd Space. You can find him online at http://derekh.com, http://ardentdev.com, and http://twitter.com/derekhat.

The MSBuild Extension Pack provides a collection of over 240 tasks for MSBuild, including things like Environment Variables, Event Logs, Performance Counters, Registry, Services, Code Signing, Zip Files, Email, IIS6, IIS7, Twitter, and WMI.

Listen to our interview with project coordinator Mike Fourie using the embedded audio player or download the MP3.

—–

CodePlex is Microsoft’s open source project hosting web site. Start a new project, join an existing one, or download software created by the community. Learn More

Continuous Integration is a development practice that can help improve your company’s ability to deliver quality software. It can reduce the time to market and the deployment cycle from functional complete products to having the solutions deployed on desktop or servers.

Continuous Integration, or CI, can start with a lone developer desktop and mature closer towards production. The ultimate goal is to have a CI environment that can take you from checked-in code to production ready installation packages.

The instructions in this article will help you get started with CI even if you’re a single developer or a small team.

This is a process that takes time, but don’t be afraid of the road ahead, I will walk you through the first baby-steps towards a well-tuned machine. CI-practice is one of many important elements for the software development industry to mature to levels of other types of manufacturing.

Introduction to Continuous Integration

Martin Fowler has written a very good introduction to CI which you can read this on his website: http://martinfowler.com/articles/continuousIntegration.html [1]

Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily – leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly. This article is a quick overview of Continuous Integration summarizing the technique and its current usage.

The clean and simple essence of CI is to avoid the situation “It works on my machine”. This has caused a lot of pain between developer and users, but also between developers. Most of us have experienced that our projects stops compiling properly after a Get Latest operation on the source control system. Another important aspect of CI is to have a cycles of automatic compilation and testing of your projects. We will get deeper into automatic testing later in this article.

Source Control Systems

If you are not familiar with source control systems (SCM), I suggest reading a good introduction by Eric Sink. While the first CI example I will give in this article doesn’t use SCM, it is an essential piece of the puzzle if you’re doing computer software development. Even as a single developer, it’s advisable to use a SCM. SCM will give you better control and management of your projects, with history and the ability to restore deleted code.

Introduction to SCM: http://www.ericsink.com/scm/scm_intro.html [2]

There are many providers of SCM software; for the purpose of this article we will use Microsoft Visual SourceSafe. When your team grows and you have the finances to purchase licenses, I suggest upgrading to Microsoft Visual Studio Team System, which is a complete Application Life-Cycle Management (ALM) suite.

Setting up your environment

There are various CI solutions available for .NET developers, one of the most popular is Cruise Control .NET (http://ccnet.thoughtworks.com/ [3]).

[Please note that all examples are from a 64-bit system where the Program Files paths includes (x86) for 32-bit applications. If you have a 32-bit environment, please make sure you remove any reference to (x86).]

Cruice Control .NET (CC.NET) comes in various forms, for beginners the setup alternative is the simplest to get you started (CruiseControl.NET-1.4.2-Setup.exe).

After the installation has completed, a new folder will appear in the Windows start menu. From here we can access the main configuration file and launch the console application.

Before we continue with updates to our configuration file, we need to learn a little bit about build tools…

Build Tools

To compile the source code, you will need a tool that does the build for you. One alternative is NAnt, which is a free tool (http://nant.sourceforge.net/ [4]).

With Visual Studio 2005 Microsoft introduced MSBuild, which is a standalone build engine for .NET. This has made it more accessible for third parties and made it simpler to automate and script.

The simplest way to get started with a build process is to utilize MSBuild, which is already installed if you have .NET Framework 2.0. We will configure CC.NET to start the MSBuild process which takes care of the compilation of our Visual Studio Solution.

New Project or Existing Solutions

It is much less effort to include guidance, frameworks, and tools in the beginning of a new project. This applies to CI as well. The further you get in the development process, the harder it gets to make everything work properly in a CI-environment.

Sample Projects

For the purpose of this article, we’ll create a new Visual Studio solution called IntegrationSample. Within this solution we will create 3 projects: IntegrationSample (Windows Forms), Integration.Common (Class Library) and Integration.Test (Class library).

Add a Person.cs class to the Common project with the following code:

This class represents part of our business domain model and has certain business logic applied to it that is important to our business, namely the method that tells us if a person is more than 60 years old. The next step is to apply a test for our domain object.

Automated Testing with Unit Tests

Being able to automate the testing process of your projects is an important feature of a CI-environment. Having source code that works as tests for your application logic will help you when you’re later going back to modify, extend and fix existing code. With good tests you can ensure that your code changes won’t break the whole application. There are different frameworks available for unit testing, the most widely used with .NET is NUnit (www.nunit.org [5]). To learn more about test-driven development, visit Wikipedia: http://en.wikipedia.org/wiki/Test-driven_development [6]

When the installation of NUnit is complete, you can explore the included samples to learn more about writing unit tests. You can use the NUnit GUI to run your unit tests outside of the CI-environment.

To start writing unit tests, you need to add a reference to the following assembly in your Integration.Test project: "C:\Program Files (x86)\NUnit 2.4.8\bin\nunit.framework.dll". Additionally you should create a reference between the Test and the Common project, so we can access our business domain model in the tests project.

When you’re done adding the reference, let’s create a class named PersonTest.cs which will contain all the unit tests for our specific domain object.

Tests are done through assertions. There are many types of assert methods included in the NUnit framework and you should use them to validate your business logic. Writing good unit tests is hard and it’s a practice in which you will continually improve as you get more experienced with it.

You can now begin to run and validate the unit tests on your local machine, open the NUnit GUI application and open the Integration.Test.dll file.

As you can see in my screenshot, both of the tests for the Person object validates. When we later go back to change the business logic, we can use the unit tests as a means to validate that our changes doesn’t break anything.

Bringing it all together

Let’s go back to the CC.NET configuration file and start modifying it so we get our CI-environment up and running. It’s advisable to have a dedicate computer that act as a build-computer, but it’s not a requirement.

The first thing we’re going to do is modify the default configuration file to run NUnit and our tests, here is the script so far:

<cruisecontrol xmlns:cb="urn:ccnet.config.builder">
 <project name="IntegrationSample">
  <tasks>
   <nunit>
    <path>C:\Program Files (x86)\NUnit 2.4.8\bin\nunit-console.exe</path>
    <assemblies>
     <assembly>C:\Source\Samples\IntegrationSample\Integration.Test\bin\Debug\Integration.Test.dll</assembly>
    </assemblies>
   </nunit>
  </tasks>
 </project>
</cruisecontrol>

Save the configuration and start the CruiceControl.NET application in the start menu. This will open a console application that will run your configuration. This is useful for beginners and when you’re setting up a new environment. When the configuration is complete, you should use the Windows Service for a more stable runtime.

When you ran the CC.NET installation, it should have created a virtual directory for you on the local Internet Information Services. If you happen to have any problems with this, make sure you create a virtual directory called webdashboard and map it to C:\Program Files (x86)\CruiseControl.NET\webdashboard.

Opening up the web dashboard should show us our IntegrationSample project and hopefully a green status, indicating that our unit tests has completed successfully. With the current configuration, we need to click the Force button to run tests.

This dashboard will be your viewport into the status and history of integration builds. You can drill down into the details of a build and you can have multiple CC.NET projects specified in the configuration file.

Our next step is to make CC.NET compile our source code using MSBuild.exe.

Within the <tasks> element in the configuration, add the following script above the NAnt task, which will tell CC.NET to execute MSBuild.exe with our Visual Studio solution file.

<msbuild>
 <executable>C:\WINDOWS\Microsoft.NET\Framework\v3.5\MSBuild.exe</executable>
 <workingDirectory>C:\Source\Samples\IntegrationSample\</workingDirectory>
 <projectFile>IntegrationSample.sln</projectFile>
 <buildArgs>/noconsolelogger /p:Configuration=Debug /v:m</buildArgs>
 <targets>Build</targets>
 <timeout>720</timeout>
 <logger>C:\Program Files (x86)\CruiseControl.NET\server\ThoughtWorks.CruiseControl.MsBuild.dll</logger>
</msbuild>

To validate that CC.NET is working correctly, we can go ahead and make a quick modification to the source code. If you reverse the operator for the age method, we should see the IntegrationSample fail on the dashboard. Drilling down into the details, we should see that our unit test fails.

The last and final step is to add support for a source control system. As you have now seen, you can setup an CI-environment without source control, but it makes sense to use source control. A source control system adds history and backup support and the ability to make heavy and local changes and not disrupt the active codebase which is inside the source control database.

Visual SourceSafe 2005

If you don’t have a source control system yet, an easy way to get started is with Microsoft Visual SourceSafe 2005. With an MSDN Subscription, you can download it from the downloads page.

When you go beyond a small team of programmers I suggest upgrading the source control with a more modern provider, like Visual Studio Team System or SourceGear [7].

Going back to Visual Studio and our IntegrationSample project, we will go ahead and add it to our source control provider of choice.

When the solution and all the projects is added to the source control, we can open the Visual SourceSafe Explorer and see how our newly added project is added and we can begin to check-out and check-in code changes.

We’re now ready for the final changes to our ccnet.config file. Open up the configuration file from the start menu and add the following new elements to the top beneath the <project> element.

<triggers>
 <intervalTrigger seconds="60" />
</triggers>

 <sourcecontrol type="vss" autoGetSource="true" applyLabel="true">
 <executable>C:\Program Files (x86)\Microsoft Visual SourceSafe\ss.exe</executable>
 <project>$/IntegrationSample</project>
 <username>build</username>
 <password>Password123</password>
 <ssdir>C:\Source\VSS</ssdir> <!-- Path to your SourceSafe database -->
 <workingDirectory>C:\Source\Samples\IntegrationSample\Build\</workingDirectory>
 <cleanCopy>false</cleanCopy>
</sourcecontrol>

Make sure you use a different working directory than your normal project folder, if you are running CC.NET on the same machine that you’re developing on. You also need to modify the <workingDirectory> and <assembly> elements we created earlier. See the bottom of the article for a full version of our build script.

When you start CruiceControl.NET again, you should see something similar to this output. Please beware that it can take a short while for the process to start.

As you can see, the script finds that there has been modification in the project inside our source control, which triggers a get process of the source code and later compilation and test-run.

You can specify your own trigger interval in the configuration file, depending on the size of your development team and your software, you can tune it to make sure you build often, but not so much that your build server starts having trouble.

You should create a special user account that your build process can run as and access the source control, as you can see in the example above I created a user called build.

Now it’s time to test the whole cycle of editing source code, check-in, watch the build server figure out there has been a modification and trigger a compilation and test-run. Let’s check out the Person.cs file and modify the age check to make our unit tests fail.

When you’re checking code back into the source control system, it’s good practice to add an informative comment. This is useful for historical purposes and for other team members to get an overview of changes in the history view.

After checking in our changes, we only need to wait a few seconds to see that CC.NET discovers our change and runs our script.

Congratulations, you now have a CI-environment up and running!

Going Forward

We have looked at the base tools in a CI-environment, going forward I suggest taking a look at frameworks and tools that works good together with CC.NET. FxCop [8] is a tool from Microsoft that analyzes your assemblies and reports anything that is not according to the industry standards for naming conventions, etc. NCover [9] is a tool which will generate code coverage reports (that means it will highlight which application logic in your application is being covered by unit tests). If you’re a component shop, API documentation is important. Using tools like SandCastle [10] you can automatically generate help files from your XML comments.

There is a whole world to explore and the extension ability to your CI-environment is endless and everything is there to help you improve the process of building good software with high quality.

You should now understand how to setup your own CI-environment using free tools. Best of luck in setting up your own environment!

References:

[1] http://martinfowler.com/articles/continuousIntegration.html

[2]http://www.ericsink.com/scm/scm_intro.html

[3] http://ccnet.thoughtworks.com/

[4] http://nant.sourceforge.net/

[5] http://www.nunit.org/

[6] http://en.wikipedia.org/wiki/Test-driven_development

[7] http://www.sourcegear.com/

[8] http://msdn.microsoft.com/en-us/library/bb429476(VS.80).aspx

[9] http://www.ncover.com/

[10] http://www.codeplex.com/Sandcastle

Sample Soure Code (includes the ccnet.config build script):

IntegrationSample.zip

Build Script

<cruisecontrol xmlns:cb="urn:ccnet.config.builder">
 <project name="IntegrationSample">

  <triggers>
   <intervalTrigger seconds="60" />
  </triggers>

  <sourcecontrol type="vss" autoGetSource="true" applyLabel="true">
   <executable>C:\Program Files (x86)\Microsoft Visual SourceSafe\ss.exe</executable>
   <project>$/IntegrationSample</project>
   <username>build</username>
   <password>Password123</password>
   <ssdir>C:\Source\VSS</ssdir> <!-- Path to your SourceSafe database -->
   <workingDirectory>C:\Source\Samples\IntegrationSample\Build\</workingDirectory>
   <cleanCopy>false</cleanCopy>
  </sourcecontrol>

  <tasks>

   <msbuild>
    <executable>C:\WINDOWS\Microsoft.NET\Framework\v3.5\MSBuild.exe</executable>
    <workingDirectory>C:\Source\Samples\IntegrationSample\Build\IntegrationSample\</workingDirectory>
    <projectFile>IntegrationSample.sln</projectFile>
    <buildArgs>/noconsolelogger /p:Configuration=Debug /v:m</buildArgs>
    <targets>Build</targets>
    <timeout>720</timeout>
    <logger>C:\Program Files (x86)\CruiseControl.NET\server\ThoughtWorks.CruiseControl.MsBuild.dll</logger>
   </msbuild>

   <nunit>
    <path>C:\Program Files (x86)\NUnit 2.4.8\bin\nunit-console.exe</path>
    <assemblies>
     <assembly>C:\Source\Samples\IntegrationSample\Build\IntegrationSample\Integration.Test\bin\Debug\Integration.Test.dll</assembly>
    </assemblies>
   </nunit>

  </tasks>
 </project>
</cruisecontrol>

—–

Sondre is a Microsoft Regional Director with 11 years of experience in software and web development. He currently works as Technology Leader for the Microsoft department at Capgemini in Oslo, Norway. Sondre is additionally the leader of the Norwegian .NET User Group in Oslo and a speaker at various technology and software development events. Active blogger, writer and technology enthusiast with a special passion for robotics, intelligent systems and open source.

So you are almost done with implementing a ground breaking, state of the art, super duper plugin-enabled calculator application with syntax highlighting and now you are wondering how to get it deployed? In case you are already trying to put some arguments together to convince your boss for buying a suite to building Microsoft Installer packages worth several thousand dollars: Stop! Don’t do that. Well, at least not if your application is not as complex as let’s say your development environment.

Using Microsoft Visual Studio you should be able to create an MSI in literally no time by using the predefined Setup Project template and if you need to accomplish additional tasks that Windows Installer does not support out of the box, read on to find out how to extend your installer with own custom actions.

The Vision

A couple of years ago market researcher Evans Data noticed a massive drop of C++ developers and also forecasted a continuous decrease. Bjarne Stroustrup on the other hand said that the number of C++ developers is higher than ever (he’s the inventor of C++ by the way). Whoever you trust, you are most probably reading this because you want to know how to use C# or VB.NET for writing custom actions in Visual Studio instead of using good old C++. So let’s dive into that topic by creating a not so fictional case scenario: The first task you would want to accomplish is creating an eventlog source for your application (because you read that the installation of an application is the suggested moment for doing just that)

Note: We will do a lot more than that, so I have included a small sample solution for you to easily reproduce what I am talking about. You can download it from http://devshaped.com/files/MyCalcSln.zip.

The Solution

As I said earlier, you could use C++ to implement a native custom action. However, the easiest solution for getting this done using C# or VB.NET is deriving a class from the Installer Class and overriding the Install method. Then you would add your custom action to the Custom Action Table of the installer.

So let’s start!

Create the Custom Action

Add an Installer by right clicking your project and select Add > New Item > Installer Class. Name it CreateEventSource.cs for now.

Figure 1- Adding an Installer Class

Next right click the CreateEventSource.cs file in the solution explorer and select View Code. Replace it with the following (you may want to adjust the namespace and the actual event source name):

As you can see, deriving from the Installer base class is not enough. You need to add the attribute RunInstaller to the class and initialize it with true as shown above. This basically marks the CreateEventSource class as an Installer class and makes it visible to reflection.

Create the setup project

Simply add a new Setup Project to your solution by right clicking your solution in the solution explorer and then Add > New Project > Other Project Types > Setup Project as shown in the following figure:

Figure 2 – Adding a Setup Project

To let the Setup Project know what to install, right click the project and select Add > Project Output… Make sure your application project is selected as Project and choose Primary Output. Click OK.

Figure 3 – Adding Project Output to Setup Project

With these simple steps you just created a setup that is able to install your application. Now you would make sure your custom actions are called and configure some simple settings. Right click the Setup Project in the solution explorer and select View > Custom Actions.

Now right click the Custom Actions node and select Add Custom Action. Look in File System on Target Machine and double click Application Folder. Select the Primary Output from your application project and click OK.

Figure 4 – Custom Action View

Easy. This will create the eventlog source for you by making use of the Installer Class you created earlier. If you would build your setup now and install it, the application will be defaulted to [ProgramFilesFolder][Manufacturer]\[ProductName]. Of course it is up to you to change the default target path. In my case, C:\Program Files\TOP TECHNOLOGIES CONSULTING GmbH\MyCalc would be a little like breaking a fly on the wheel. So I right click the Setup Project in the solution explorer and select View > File System and choose Application Folder. In the properties pane, I can configure the DefaultLocation to something like [ProgramFilesFolder][ProductName]. That’s better.

Install your application

Now let’s finally make a real job of it. Right click the Setup Project in the solution explorer and select Build. After being built successfully, right click the project again and select Install.

You should be familiar with the dialogs that follow. However, after your setup successfully installed your top notch calculator, have a look at the event viewer by clicking Start and entering eventvwr.

You will find the following entry in the application log:

Figure 5 – Eventlog Entry

Observations

When using a fully featured environment for creating and using Custom Actions such as InstallShield from Acresso or Windows Installer Xml from Microsoft, you have absolute control over when and how your custom action is actually called. You can start it during the interview phase of your setup (aka UI sequence), during the immediate phase of the execution sequence when the actual installation script is generated or use it as a deferred custom action that is run in within the system context while the system changes are really applied.

Custom Actions implemented as Installer Classes are only run deferred in the system context. That means that your application is already successfully installed at that time. You have full access to all your files and also the configuration. You could implement a Custom Action for instance that alters configuration settings of your application based upon properties set by the user during the UI sequence (I will show you how later).

If you want to dive into the details of the Microsoft Windows Installer installation process, go ahead and read Alex Shevchuck’s TechNet entry here. This is a great resource for a decent understanding about what’s going on under the hood.

Potentialities (or: What the heck is CustomActionData?)

When looking at the choices in the View context menu of your Setup Project, you already noticed the following entries:

• File System
• Registry
• File Type
• User Interface
• Custom Actions
• Launch Conditions

What most certainly caught your attention is the User Interface entry. Yes! You can really add custom UI dialogs to a Visual Studio Setup Project. Although it is possible creating real custom dialogs, you can instantly start using own dialogs by choosing from the provided templates. Since we already agreed that we are talking about relatively simple applications here, in most cases that will get you started immediately (if you really really need more than that, look here).

Changing Application Settings During Installation

Let’s have some fun with custom dialogs to do something really useful here you most probably won’t instantly find by poking keywords into a search engine. Looking into the functional specification of your calculator you notice that one of your bosses (the one who usually has the most intriguing ideas of all) demanded that the calculator should welcome the user with his/her name. You heard him mumbling something about “Unique Selling Point” while he left the conference room…

So here is what you do:

Create a splash screen

Right click your calculator application and select Add > Windows Form. Name the form Welcome.cs and set its properties to the following values:

Property	Value
FormBorderStyle	None
ShowInTaskbar	False
TopMost	True
StartPosition	CenterScreen

Drag and drop three label controls into the form like this:

Figure 6 – Borderless splash screen

Create and bind an application setting to a label

Now configure the UserNameLabel label. In the property pane expand the ApplicationSettings node and click the ellipsis button (…) to the right of the Property Binding setting. Select the Text property, expand the drop down list and select New… Enter UserName as Name and Application as Scope (we’ll assume that the user who installs the calculator won’t let anyone else start it).

At this moment, Visual Studio automatically creates an app.config file for you that should basically look like this:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <configSections>
      <sectionGroup name="applicationSettings" type="System.Configuration.ApplicationSettingsGroup,
          System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" >
          <section name="MyCalcWin.Properties.Settings"
              type="System.Configuration.ClientSettingsSection, System, Version=2.0.0.0,
              Culture=neutral, PublicKeyToken=b77a5c561934e089" requirePermission="false" />
      </sectionGroup>
  </configSections>
  <applicationSettings>
      <MyCalcWin.Properties.Settings>
          <setting name="UserName" serializeAs="String">
              <value />
          </setting>
      </MyCalcWin.Properties.Settings>
  </applicationSettings>
</configuration>

If you have a close look at the Welcome.Designer.cs file you will find the following line that does the binding for you:

Let the splash screen appear at startup

To make the splash screen appear, adjust the Load method of your main form like this:

Now let’s finally wire things up with the setup project.

Create a custom dialog

Right click the Setup Project in the solution explorer and select View > User Interface. Right click the Start node in the Install tree and select Add dialog. Choose Textboxes (A) and click OK.

Figure 7 – Adding a Custom Dialog

Drag and drop the new dialog in front of the Confirm Installation node. Select it and change its properties to the following values:

Property	Value
BannerText	Enter your name
BodyText	MyCalc is the only calculator that will welcome you personally on each startup! Just tell it who you are.
Edit1Label	Your name:
Edit1Property	USERNAME
Edit2Visible	False
Edit3Visible	False
Edit4Visible	False

Now right click again the Setup Project in the solution explorer and select View > Custom Actions. Select Primary Output from MyCalcWin (Active) right under the Install node and change its CustomActionData property value to /userName="[USERNAME]".

Create a new Custom Action

Now comes the tricky part. We are talking here of at least a .NET 2.0 application. As we already saw, we are working with real type safe and supposedly easy to use settings utilizing the data bound Settings.settings file. Fortunately, .NET already generated a type safe wrapper for us. Unfortunately you cannot access the settings as easily as from within the application due to the fact that the installer simply does not have a mapper for it (and we won’t supply it with one).

What you can do though is this:

Create another custom action as you already did before, call it RegisterUser and replace the generated code with the following:

As you can see, we can access the Custom Action Data we prepared earlier through the InstallContext Parameters collection. Also the InstallContext already contains a couple of default parameters such as the assemblypath (which we need to access the applications’ configuration file).

What we are doing here is loading the .exe.config file of our application, digging through to the UserName setting, creating a new Xml node containing the username value and finally putting the value into the configuration before we are saving the changes. It is really as easy as that. Wasn’t that obvious?

Now compile and install your application.

The setup will ask you for your username:

Figure 8 – User registration dialog

When starting the calculator, the splash screen welcomes you:

Figure 9 – Actual splash screen

Alternatives

Over and above the samples you saw so far there are several alternatives to using the Installer Class provided by Visual Studio. One is using native C++ (the languageWindows Installer is actually developed in, which is why it is the most powerful way of implementing custom actions). Explaining how that works is out of the scope of this article, but you can read about it for example on Steven Bone’s Blog who published three absolutely awesome articles about it here.

A native custom action that accesses the MSI log for instance basically looks like this:

extern "C" UINT __stdcall Install(MSIHANDLE hInstall)
{
  PMSIHANDLE hRecord = MsiCreateRecord(0);
  MsiRecordSetString(hRecord, 0, TEXT("Native C++ is better than Installer Class!"));
  MsiProcessMessage(hInstall, INSTALLMESSAGE(INSTALLMESSAGE_INFO), hRecord);
  return ERROR_SUCCESS;
}

Using DTF (Deployment Tools Foundation), which is part of the Windows Installer Xml Toolset, enables you to implement Custom Actions with managed code. However, you need to make sure .NET Framework is already installed on the computer you are running the setup on and also have a couple of prerequisites to fulfill.

Action	Installer Class	Native C++ DLL	WiX DTF
Run in UI sequence		X	X
Run immediate in execute sequence		X	X
Run deferred in execute sequence	X	X	X
Read MSI properties	X[1]	X	X
Write MSI properties		X	X
Debuggable	X	X	X [2]
Access MSI Logfile		X	X
Access Installstate log	X
.NET Framework must be installed	X		X
Windows Installer Xml must be installed			X

Conclusion

In many cases, installation needs are straightforward enough that it is not worth the effort to implement anything more complex than the Custom Actions that run deferred in system context. Although you gain more control over the installation process when you choose to implement Windows Installer Xml based Custom Actions or native CA’s, Installer class based Custom Actions can be a great way to extend your Visual Studio based Setup Project without the need to invest in additional installer solutions.

[1] Through stateSaver dictionary collection

[2] Simple unit testing possibility via wrapping the MSI session

—–

Christian Jacob is an IT Consultant with TOP TECHNOLOGIES CONSULTING, a Microsoft Gold Partner with focus on infrastructures, security and business solutions. He primarily uses C# and VB.NET in nowadays development and  has his roots in C++ based game development. He spends a lot of time in researching and creating innovative solutions around team development, Windows Installer, and TOP TECHNOLOGIES’ main software products. He currently resides nearby Hamburg in North Germany.

Recently we asked 6 technology leaders to provide some advice on what developers should be doing to make themselves more valuable at work.

You might find their perspectives surprising:

(There’s an embed code if you want to share this video with others on your own site.)

Since there has been software, there has been the dream of the big green button.

Predictions of the future usually involve elaborate thought-capturing machines that deduce the true desire of an end user and effortlessly create their ideal application, ready for orders, often in the form of a simple form and a big green button promising magic with a label like ‘Go’. This is great for the futurist daydreamer, but terrible for the software developer who lives a life of continuous improvement, watching in horror as their profession is reduced to an afterthought somewhere between waking and the business application someone dreamt up before lunch. This is right around the time that the very emotional craftsmanship debate kicks in, and the humans vs. machines themes spring to awkward, robotic life.

Since the term "software factories" was coined in the late 60’s by Hitachi, it has been linked inexorably to the manufacturing process it is meant to emulate: people fear that the automation of software components will put developers out of work, or reduce their capacity down to mere assemblers of components rather than authors of code. We hear the buzz in technical communities about the emergence of powerful new software factories, developed by Microsoft and others, and can’t help but wonder if the intended goal of these new innovations is exactly along those lines. Yet, the people most involved in the software factory movement see it as just the opposite, a way to harness factory processes simply to meet the surging global demand for software at all, let alone with chairs to spare.

"We see a capacity crisis looming. The industry continues to hand-stitch applications distributed over multiple platforms housed by multiple businesses located around the planet, automating business processes like health insurance claim processing and international currency arbitrage, using strings, integers and line by line conditional logic. Most developers build every application as though it is the first of its kind anywhere.

Without significant changes in our methods and practices, global demand for software development and maintenance will vastly exceed the pace at which the industry can deliver in the very near future." (from http://softwarefactories.com)

Why should you care about software factories? After all they have that irksome quality of sounding like something meant to replace you. The reality is that software factories are a potential answer to a problem that you live with on a daily basis: change. When a software system changes, it creates risk which often hits your desk in the form of bugs to fix. When businesses change, they adopt new technologies, abandon others, or leverage what they already have in new ways. For developers on the ground, this means you either need to predict the future, or jump hoops in the present to keep up with this change and maintain or acquire the skills that make it work.

The future of computing, arguably, is a repeated pattern from its past: enabling technologies are introduced, adopted, and eventually reach maturity in the wake of the next round of capabilities introduced by innovators. Developers working with bleeding edge technologies knows it’s their blood. Developers working on mature or sunset systems feel a little out in the cold when they hear what’s on the horizon. The software factory is rising in popularity as a way to preempt the technology adoption cycle by introducing automation where a developer would spend time on low yield tasks, and encapsulating domain knowledge where a developer would need lengthy, dedicated time with domain experts. Today, factories are even used to allow business experts themselves to change core behavior of their software, without interrupting the software development life cycle.

You already understand the value of re-usability (think open source), maintainability (think unit testing), and replaceable architecture (think layers or seams); what’s a software factory if only more of the same on a grand, possibly unworkable scale, and what’s in it for you? Who cares?

As a developer, you should care because the goal of a software factory is to clear a path for you to get to "the good bits"; the business strategy that changes markets, the social networking feature that changes how people communicate, the custom solution that runs a small business like clockwork. In the end you’re remembered for how you solved a problem, not how separated your concerns were. The sooner you can get to writing the code that matters to your customer, the better.

Within Microsoft itself, a quiet revolution is forming in the labs and in upcoming updates to our most central development tools. Let’s look at some of the software factories that make up the future of computing on the Microsoft stack.

Factories for Business Problems

Domain Specific Language Tools (http://msdn.microsoft.com/en-us/library/bb126235.aspx )

A domain-specific language, or DSL, is all the rage at the moment and is likely to crop up in many new development projects in the near future. The power of a DSL lies in its ability to define a specific domain (such as the shipping business, or indoor pool maintenance) in a small utility language that is either used by a developer to quickly script up new modules for a large software package, or taught to business users so they can adapt their software to meet their changing needs. Want to change the business logic around preferred customer treatment? It’s one very human readable line in a script, as opposed to a code change, a check-in, and a redeploy across an organization. A DSL is a software factory because it automates the processes of a specific domain, and the artifacts it creates, in this case scripts for customer treatment behaviors, are as reusable as the language definition itself. Microsoft’s Domain Specific Language Tools are a collection of visual designers that let you model a DSL visually, similar to the Visual Studio Class Designer or the LINQ to SQL mapping design surface.

M Grammar (http://msdn.microsoft.com/en-us/library/dd129870.aspx )

Destined to ship as a feature of Visual Studio 2010 and available today as a technical preview, Mg is first class support for DSL language design. Originally intended to work with other tools for defining and parsing schema, such as XML, directly to databases, Mg is useful in its own right for declarative development of DSLs and it will become a popular tool and topic for the future of DSL modelling with Microsoft technologies.

A video detailing Mg from Paul Vick, Microsoft Architect
(http://www.informit.com/podcasts/episode.aspx?e=E8D94BA1-1B77-4F4D-99CC-BFCE8CE36489)

Factories for Architecture

While a DSL would provide a factory for a generalized language, more robust needs require generalized solutions. Several evolutions of full-scale software factories exist in the Microsoft ecosystem, here are a few established and emerging technologies that will allow you to re-brand, repackage, or reconfigure code you’ve already written, for customers you will have down the line.

Microsoft Enterprise Library (http://msdn.microsoft.com/en-us/library/cc467894.aspx )

Microsoft’s Patterns & Practices team developed a true software factory long ago when it introduced the Microsoft Enterprise Library. Designed for composing applications out of blocks of clearly defined enterprise features, MEL reduces the number and scale of repetitive tasks that you face every day on a new software project, such as logging, messaging, services, and to some extent, even the flow and functionality of UI. This library requires a fairly low commitment in terms of education as it builds on existing Microsoft technologies but provides a unified set of repeatable features that every enterprise project needs.

Microsoft Managed Extensibility Framework (http://www.codeplex.com/MEF)

A lighter-weight library for building composite applications is available with MEF. Borrowing the term composition from the manufacturing domain, the Managed Extensibility Framework is designed to make it easy for a developer to define application features based on components and configuration rather than reinvent the spinning wheel with one-off software implementations for products that exist in many different instances to satisfy specific customer needs. Still in development, look to MEF to be the application block strategy of the future.

T4 Templates (http://msdn.microsoft.com/en-us/library/bb126445.aspx)

Revisiting the manufacturing paradigm in software factories, the end result of any factory process is a collection of artifacts. Without artifacts, there is no output and no value to use once the smokestack clears. Arguably the lynch pin behind all software factory approaches, template generation is the artifact concept that what you define at a high level can be broken down and generated into working source code at a lower level. In fact, the very same template engine behind the DSL tools developed by Microsoft is available for you, today, with Visual Studio T4 templates. After defining a T4 template, dropping them into your Visual Solution awakens them, using meta-data to create entire frameworks, if necessary. One well-known use of T4 templates is within the SubSonic 3.0 database technology (http://code.google.com/p/subsonicthree/), developed by Rob Conery of Microsoft; the entire framework for database persistence is created using T4 templates dynamically built around your database’s connection string.

Prism: Composite Application Guidance for WPF and Silverlight (http://www.codeplex.com/CompositeWPF/)

Need a software factory for differentiated, rich user experiences with the latest Microsoft stack on desktops and the web? Another software factory making a big splash now and in the next few years will undoubtedly be the evolution of Prism. The future of application interfaces involves dynamically assembling behaviors, interactions, effects, and transitions together while being able to reuse each piece in new views. Composition is the buzz word for this new breed of applications that favor declaring what the UI could do, over coding what the UI must do.

Factories for Code Quality

Five years ago, the idea that code testing tools could "predict", "discover", or "experiment" with source code to find fatal flaws and defend against future quality problems would sound like science fiction, but a new breed of heuristic software factory is emerging from Microsoft Labs in the form of intelligent tools for breaking, and mending, software source code. These will be exciting, cutting-edge projects in your near future, and they’re available for preview use right now.

PeX (http://research.microsoft.com/en-us/projects/Pex)

The goal of PeX is to automate "white box" unit testing. You already know that unit testing is a safeguard, not a guarantee, of quality programs. You try very hard to achieve 100% code "coverage", or all possible paths through an application hunting for bugs both glaring and devious. PeX promises the precision of a computer with the craftiness of a human when looking for software programming flaws. It won’t be a stand-in for writing real unit tests by hand, but will help catch what you miss.

CHESS (http://research.microsoft.com/en-us/projects/Chess)

To paraphrase the immortal Wesley in The Princess Bride, "Concurrent programming is painful, Highness. Anyone who says otherwise is selling something". Well, in this case, Microsoft Research is on track to improving detecting threading complications in an automated fashion, and you can try it out for free. CHESS looks for "Heisenbugs", a pet term for uncertain, undefinable, intermittent software defects. You know these as the accounting glitch that only happens on Tuesdays, or the dynamically loaded assembly that crashes every one thousand successful operations. CHESS attempts to bring tooling to finding, and more importantly reproducing the conditions that lead to these strange bugs, so you can squash them.

Conclusion

We’re very far from the big green button, and thankfully so, but the next five to ten years of software development will bring increasing factory-mindedness to the tasks that return the least value-to-investment ratio, accelerate the time to working software iterations, and assist human innovation with computer precision. Developers should embrace anything that allows them to move from the menial to the meaningful, and should not fear the software factory. Life is too short for one developer to deliver all of the technology that his or her world demands, but software factories in the hands of skilled developers can; factories are the past, present, and future of computing.

—–

Daniel Crenna is a Microsoft MVP and creator of TweetSharp, an open-source Twitter API development library. Daniel is focused on improving the landscape for .NET developers building social software through founding www.dimebrain.com. Daniel was a contributor to the Microsoft AJAX Control Toolkit, and is a Microsoft Certified Professional Developer.

Be sure to read Ted’s article "Beyond VB and C#".