Most .NET developers are going to need to access relational data at some point. For somebody approaching .NET for the first time, the data access story in the .NET world can be a little overwhelming to say the least. Existing .NET developers looking to update their data access techniques can face a similar overwhelming experience. This article will help you understand the currently shipping mainstream technologies and how you can choose the one that is right for your particular needs.

There is a wide array of choices from Microsoft as well as from within the .NET ecosystem. There are many considerations to keep in mind when evaluating these technologies such as type safety, developer productivity, maintainability, concurrency, database agnosticism, performance, and scalability. One element that is almost universally accepted as a best (and only) practice these days is optimistic concurrency. Just about every data access technology that you’ll be interested in is going to use this concurrency model. That is where the similarities end.

Traditional ADO.NET

ADO.NET is a .NET brand name for all things data access from Microsoft. Typically using ADO.NET means using SqlCommand objects. Using a SqlCommand, we can pass a native T-SQL statement directly to the database. This could be a SELECT, INSERT, UPDATE, DELETE, a stored proc, or even a DDL (Data Definition Language) statement. Because ADO.NET doesn’t know too much about the database syntax, this statement is simply stored in a SqlCommand as a string and passed off directly to the database. You must use one of the Execute* methods on the SqlCommand to invoke the command. Using ExecuteNonQuery will fire the statement and assume it returns no result sets. This is useful in the Insert, Update, and Delete scenarios. To bring a result set back, we use the ExecuteReader method which will return a DataReader. A DataReader looks a little bit like a cursor which will allow us in .NET code to loop through the result set one row at a time and look at the columns. The DataReader is somewhat stupid in that it doesn’t know much about the shape or data types of the results coming back so you must use untyped access to get the columns as such in the code below:

It is also important to note that the connection to the database remains open during this looping. You may want to reduce the amount of work you do in the loop to minimize the amount of time the database connection is held open. This will make your application more scalable since database connections are a finite resource.

If you are only returning a single value (like a count or exactly one column from one row) you can use the ExecuteScalar method which is a bit easier to use than a DataReader.

SqlCommands and DataReaders in general let you be very precise about the SQL that is sent to the server and control exactly when the connection is opened and closed. You have to provide a bit more muscle in getting the data out of the SQL and coerced into .NET data types, but for performance sensitive areas of your applications, this could pay off. The other thing to be aware of is that there are a different set of objects (Commands, Connections, DataReaders) for each relational database provider. So if you need to move from SQL Server to ODBC to Oracle, you’ll need different objects. Fortunately each of these objects are based on an interface and you can program through the interface by using a DbProviderFactory class to create the concrete types based on configuration data. You will have to be sure to use ANSI compatible SQL in your attempt to be database agnostic, which may run counter to providing high speed data access which often requires developers to use native & unique syntax to a particular relational database engine. Obviously your mileage might vary here.

DataSets, DataTables, Data & Table Adapters

With DataReaders you need to be careful about how long you keep your connection open while iterating. Sometimes it is better to pull back your entire result set into memory, release your connection to the database, and then process the result set afterwards. This is where DataSets can be helpful. If you have a batch processing scenario that requires the entire result set in memory or you need to provide end user editing in a grid control, then DataSets are a good place to hold that data.

One of the primary benefits of a DataSet is that it not only caches the data in an efficient memory structure, but it can provide change tracking (what records were added? Deleted? Changed? What were the original values of each column, etc). DataSets work collaboratively with a SqlDataAdapter object, which is simply a composite of 4 SqlCommands (InsertCommand, SelectCommand, UpdateCommand, and DeleteCommand) mapping to the appropriate CRUD behavior (create, read, update, and delete).

DataSets can contain multiple result sets, each mapped into its own DataTable. Tables can be related to each other with Relations which mimic foreign keys. DataSets take care of master-detail linkages and keep everything "hooked up" when inserting records into parent and child tables with identity columns. Each table will require its own SqlDataAdapter. If you are doing a mix of insert, updates, and deletes and your database has referential constraints, you’ll need to orchestrate the changes, making sure parent records are inserted before children and that deletions happen in the reverse order.

DataSets/DataTables come in two flavors: typed and untyped. Untyped DataSets simply infer the table and column definitions after executing the SqlDataAdapter.Fill method. Accessing the columns must be done similarly to DataReaders using column names in quotes or by positional reference. Typed Datasets on the other hand use wizards and a design surface to generate a strongly typed object. The typed dataset is defined by an XSD document. The XSD is then turned into a strongly typed/named class that essentially sits on top of an untyped dataset. A table mappings collection on the SqlDataAdapter class allows you to use different names for your strongly typed object than that of your database table and column names.

Many people in the .NET 1.0-2.0 era used typed datasets as business entities and wrapped some additional business logic around them. Although you could map column names, you were typically tied tightly to your table design in the database. DataSets can also be converted to and from XML, including a diffgram format that retains pending change information. In many circles, DataSets have and continue to be serialized between distributed layers of an application. This makes for a pretty productive development environment, but has drawn architectural criticisms compared to properly modeled services with messages and contracts.

In .NET 1.0, DataAdapters and DataSets were different objects that really didn’t know about each other until you executed the DataAdapter Fill or Update method and passed the DataSet/DataTable to the respective method. It was possible for the table mappings in the SqlDataAdapter to get out of sync with the strongly typed dataset. To improve this situation, Microsoft introduced TableAdapters in .NET 2.0, which are really just DataAdapters that are strongly typed and embedded into the definition of the typed dataset.

Object Relational Mapping

With .NET 3.0/3.5, MS has finally released not one but two ORM toolkits. I say finally because MS previously promised an ORM toolkit to developers in the form of a project called ObjectSpaces. ObjectSpaces was never released and its demise is clouded in a series of potentially bad integrations with WinFS, Project Green, or the MS Business Framework, all of which were eventually cancelled.

The spark that reignited MS ambitions to produce an ORM was the Language Integrated Query (LINQ) project. LINQ’s goal as part of C# 3.0 and Visual Basic 9.0 was to provide a set based metaphor for dealing with heterogeneous data types including object collections, XML, and last but not least, relational data. For the relational component of LINQ, the language team came up with a project called LINQ to SQL and shipped it as part of .NET Framework 3.5.

During the same time period, the Data Programmability Group in the SQL Server team had been working on a new layer to sit over top of SQL Server’s logical data model. This new "Entity Data Model" (EDM) would allow developers to interact with their relational database using a more abstract conceptual model based on the ideas of Dr. Peter Chen. For this team, the promise of Language Integrated Query was what really brought the notion of the EDM to life, and so began the "Entity Framework" project. The Entity Framework did not ship until several months after LINQ to SQL as part of the .NET 3.5 Service Pack 1 release.

On the outside, LINQ to SQL and the Entity Framework ORMs appear very similar. Both tools use a graphical design tool and wizard to map a relational database into an object model. They can both use LINQ queries to project relational data into objects using composable queries. Both provide change tracking so that changes made to the objects can be persisted back to the database with a single method call.

Given these similarities, Microsoft has spent some considerable energy in explaining when to use each of these technologies. While these two ORMs have strong similarities in programming models, their internal architectures are quite different. These key differences are:

• LINQ to SQL has been locked directly to SQL Server, whereas the Entity Framework includes a provider model that allows it to work with any relational engine. As mid 2009 the list of available providers in the ecosystem included SQL Server, Oracle, Sybase, DB2, Informix, MySQL, PostgresSQL, SQLite, OpenLink Virtuoso, and Firebird.
• LINQ to SQL does not provide a mechanism for streaming records one at a time. Using Entity SQL, the Entity Framework can provide Data Reader style access to results using the same mapping layer used for object queries.
• LINQ to SQL provides an out of the box experience for lazy loading. Lazy loading allows the engine to automatically (by default) go out and retrieve new data as it is needed based on code that access different elements of the object graph. In the Entity Framework, you have to be explicit when you make trips to the database. Both ORMs allow you to load deep object graphs eagerly as well.
• Perhaps the largest difference between Entity Framework and LINQ to SQL is the conceptual modeling layer that EF provides. EF provides a much richer mapping layer than LINQ to SQL, allowing you to join tables together, provide elaborate class hierarchies with abstract and concrete types, and retain many-to-many relationships in your model. Future plans will see the conceptual model being used in other MS products such as SQL Server Reporting Services and SQL Server Analysis Services.
• Finally, expect to see significant investment in the Entity Framework ORM from Microsoft. Both ORMs are now managed by the same product team at MS and they have stated that while LINQ to SQL will continue to be supported and maintained, future innovation will be focused on the Entity Framework.

Other ORM Technologies

You can’t talk about ORMs in the .NET world without mentioning NHibernate. NHibernate or "NH" is a mature yet free open source framework which is a port of the popular Java ORM Hibernate. Perhaps one of the biggest distinctions of NH is its support for a Domain Driven Design workflow which purports a code-first or test-first approach to building applications. Typical NH developers build their classes first and then use an XML file to map those to a database. Unlike NH, both LINQ to SQL and Entity Framework advocate a model-first approach, and one could argue a database driven approach. Both LINQ to SQL and Entity Framework designers assume you have an existing database that needs mapping to some objects that will be derived out of your model.

Although there are other commercial and open source ORM tools available, NHibernate, LINQ to SQL, and Entity Framework are by far the most popular technologies in use today.

Enter Data Services

When it comes to building distributed applications, your mileage will vary with the array of data access technologies. Dealing with serialization and concurrency issues in those environments is beyond the scope of this article. However, it is important to mention what appears to be yet another data access technology available from Microsoft called ADO.NET Data Services, formerly code-named "Astoria". Astoria was made available with .NET 3.5 SP1.

Astoria is an HTTP service layer built on top of WCF that provides a REST-style API to your data, giving each of your elements of data a unique URI (for example, http://host/northwind.svc/Products(1) ). Out of the box, this service can be enabled for an Entity Framework model or any other IUpdateable/IQueryable data source in just a few lines of code. Data is queryable and updateable via pure HTTP verbs (PUT, POST, DELETE, and GET) using query strings and HTTP payload. Data can be serialized in either AtomPub or JSON format. The net effect is that your data model is widely interoperable with a dizzying array of potential clients and technologies.

Although you are free to build up complex query strings and HTTP payloads for just about any type of operation, the Astoria team has created client libraries to assist in these endeavors. Firstly, an ASP.NET Ajax library is available on CodePlex to allow JavaScript developers to easily work with Astoria Services. Secondly, as part of the core installation, there is a .NET Client Library which provides a natural query model using LINQ and projecting data into client side .NET objects for use by your .NET or Silverlight projects.

A new project, currently named "RIA Services" (code-named "Alexandria"), builds on top of ADO.NET Data Services by also providing rich validation and UI cues on top of your data model in a client/server model. This technology probably won’t be released until around the .NET 4.0 timeframe, but it is definitely something you should keep your eye on.

Application Frameworks

As you can see, the trend in Data Access is to abstract into higher and higher levels within your applications. As data access libraries are generating entity classes, the question around locating data validation logic can become a slippery slope. If a data access generated entity can validate correct data types, maybe it should also validate that postal codes and phone numbers are of the correct format. Should they also cross-validate postal codes with states/provinces? Should these error messages be managed inside of my entities? The waters can be muddied quite quickly.

It is therefore worthy to briefly mention a few of the more popular application frameworks that embrace a holistic view of data access in the scope of an application.

CSLA.NET (Component-based Scalable Logical Architecture) is a framework developed principally by author Rocky Lhotka in conjunction with his popular series of Business Objects books. When first released, CSLA was focused on Visual Basic 6, but over the years it has remained very current on the latest Microsoft technologies and is now principally maintained in C# and ported to VB. In CSLA, data is fully encapsulated by rich business objects that manage all behavior including persistence. CSLA facilitates the re-use of business objects in many possible client technologies including ASP.NET, WPF, WinForms, and Silverlight including distributed architectures using Web Services.

DevForce from IdeaBlade is a commercially available framework for building rich, distributed applications in Silverlight, WPF, WinForms, and ASP.NET. DevForce builds on top of the Entity Framework for its persistence layer and includes a Business Objects server for distributed architectures. DevForce is a popular application framework and has been around since 2001.

But All I Wanted Was My Data

If you can believe it, this article has been a short list of the current and popular data access technologies. I think I could safely say that the one piece of universally accepted guidance is that if the data access technology you are considering is not in the above list, then it is likely out dated or obscure and perhaps you should think twice about using it.

If you are comfortable with open source projects and you are a strong believer in Domain-Driven-Development, NHibernate is an obvious choice. There is a strong community to back you up on this decision.

If you prefer to stick with Microsoft Technologies, and you have a legacy database to start from, or prefer a model-driven or data-driven approach, then the Entity Framework is likely where you should end up.

If you prefer to work with a more structured and complete application framework, both CSLA and DevForce are worthy choices at this level and you will have community and paid-support available to back you up on each of these respectively.

The last key piece of advice to offer is that you don’t have to pick just one of these. There are pros and cons and you may have to use multiple technologies, even within the same application. If you feel the need for performance, there are no rules against dropping out of your ORM and opting for more direct access. Keep an open mind and use the best tool for the job.

—–

Barry Gervin is a founding Partner of ObjectSharp in Toronto, Canada. As a Principal Consultant, Barry provides technical leadership to his valued clients, staff, and the development community. Over his 19 year career in the IT industry, he has led many development teams to successfully deliver large software projects within tight schedules and budgets that consistently perform for their customers. Barry currently serves as a MS Regional Director in Southern Ontario and has received the Microsoft MVP Award for Solutions Architecture for the past 5 years.

Recently we asked 6 technology leaders to provide some advice on what developers should be doing to make themselves more valuable at work.

You might find their perspectives surprising:

(There’s an embed code if you want to share this video with others on your own site.)

Within the .NET community, a low-grade battle rages between two groups, those who use Visual Basic and those who use C#, and never has the world of Computer Science seen a more virulent and angry battle… except maybe for those who argue where the curly-braces should go in C++ and C# code. Not that there’s really anything to argue about, anyway; as any right-thinking C# developer knows, they go on the next line. But that’s not the point of this article, so we’ll move on.

(Besides, I’m right, anyway.)

The problem, of course, is that a viewpoint that limits itself to just those two languages misses out on a much larger world, one which contains a much larger set of tools than just those two languages. And while those two languages, like other object-oriented languages before them, are each genuinely useful languages, the unfortunate fact of each is that they are, at heart, just object-oriented languages. And while that may hardly seem to be a limiting factor, considering how much we’ve been able to accomplish with those languages to date, much of that viewpoint stems from the fact that most .NET developers have never seen anything else.

You know the old saying: When all you have is a hammer….

Back in 1999 and 2000, during the incubation before Microsoft’s first .NET PDC, a curious thing took shape. Microsoft invited a number of researchers and academics to participate in a project called "Project 7": as part of its efforts to ensure that the CLR was not just another proprietary technology, Microsoft wanted to see more than just its own efforts running on this new platform. The CLR was at the heart of it, certainly, but the langauges that developers used would be of far more lasting concern; while Microsoft was reasonably certain that its two lead players (an adaptation of its classic Visual Basic and the brand-new C#) would be a hit, even back then the lead architects on the CLR had a vision that encompassed a larger viewpoint.

Consider, for example, Microsoft’s most recent entry into the CLR family of languages, F# (a.k.a Visual F#). Like C# and VB, F# has a number of object- oriented features baked into it, but, unlike C# and VB, F# is primarily a functional language. The nomenclature here is deliberate: just as an object- oriented language drives its users to think primarily in terms of objects, a functional language drives its users to think primarily in terms of functions. This means that certain kinds of programming–such as programming that deals with mathematics or computations, like what might be seen in simulations, financial forecasting, or even game mechanics–becomes much easier than in a language where functions are almost an afterthought.

At this point, if you’re like most .NET developers, particularly those who "grew up" on a steady diet of object-oriented languages, trying to see where a functional approach would be non-detrimental, much less beneficial, is probably somewhat hard. After all, "functions" seems to imply "global functions", and didn’t the High Priests of Good Design decry global functions almost two decades ago?

Here’s a simple example, drawn from a blog post by Chris Smith, of the F# team. In it, Chris creates a simple "artillery game", in which two tanks positioned randomly on a flat plain take turns taking pot shots at one another; the challenge is in selecting the correct angle, velocity, and "shot mass" to reach the other tank before they get you.

In itself, this is a fairly simple (yet addictive) game, but the important part of the story is in its basic construction: like many (if not most) F# projects, while parts of the code are made for a functional language like F#, such as the mathematics involved in calculating the shot itself given the user’s inputs, other parts are pretty clearly not functional at all, such as the UI interaction with the Windows Presentation Foundation libraries.

CLR interoperability rides to the rescue. Because the F# language is a CLR-based language, it’s trivial to put the actual "guts" (the logic) of the artillery calculations into F# functions, then call them from C# code directly, or even indirectly via WPF data-bindings (as Chris does in his code).

Of course, the C# fanatic will quickly point out that C# has a number of features that make functional programming easier–anonymous methods, lambda expressions, basic delegate support, and so on. In the same vein, C# can be argued to be a dynamic language like Ruby, so long as the C# developer doesn’t mind programming via the System.Reflection APIs all the time. A vast gulf lies between what a language "can" support, and what a language "natively encourages".

And herein lies the crux of the situation: every language ever designed, from ancient Lisp to modern C#, was designed with particular ideas and concepts in mind. In other words, like every kind of tool designed throughout history, every programming language has a particular usage model in mind when created, and its feature set is carefully crafted around that usage model.

Consider the aforementioned Lisp, for example. While sporting a syntax that only a mother (or a really dedicated programmer) could love, Lisp maintains a very powerful relationship between code and data: that, at the heart of things, code *is* data, and Lisp programs often manipulate code structures every bit as much as they manipulate user-entered data. This allows Lisp programmers to write layers of abstraction on top of abstractions, eventually creating an entirely new layer on top of the programming language itself. (It’s no accident that many of the "domain-specific language" advocates look at Lisp as a shining beacon of things to come.) This means, then, that the business domain can be much more closely expressed in the language, rather than trying to mold the domain into a strictly object-oriented model.

Fortunately, these ideas aren’t lying fallow waiting for developers to come to Lisp to pick them up and play. Thanks to the foresight of those Microsoft lead architects and developers back in the early days of the CLR’s development, a wide range of languages have appeared on top of the CLR (and even more for the CLR’s twin brother, the Java Virtual Machine, thanks to its half-decade greater age), many of which explore these ideas in depth. And because they run on top of the CLR, all of these languages can take advantage of the rich ecosystem of .NET libraries and tools, like the .NET Framework Class Library, ASP.NET, WCF, WPF, Workflow, NHibernate, and so on.

For example, consider the language Boo, described as a "wrist-friendly language for the CLR", derived from the scripting language Python. A dynamic language, Boo ranks alongside IronPython and IronRuby as languages that make certain kinds of tasks far easier to complete because of its essentially scripted nature and lack of need for compile-time type-safety. (In other words, practically speaking, scripting languages make it easier to create higher-level code.) For that reason alone, Boo (and IronPython and IronRuby) merit investigation.

But Boo also provides the ability to "plug in" to the compiler itself, essentially offering developers the chance to extend the language in the same way that Lisp does: by having a look at the structure of the program after it has been parsed, in what compiler wonks refer to as the "abstract syntax tree", or "AST", but before it has been turned into IL. This gives the Boo programmer an opportunity to edit, validate and modify that AST in various ways, providing a clear "meta-programming" capability to the language that C# and VB both lack. In Boo, this is known as a syntactic macro.

For example, a simple macro that comes with the language is the ability to conditionally process certain snippets of code, depending on whether a particular string is provided during the compilation phase. In other words, this is Boo’s equivalent to the #if preprocessor directive from C/C++ or C#. (When Boo is interpreting the code in a more scripting-like fashion, the flag is assumed to be passed in to the Boo engine’s command-line.) Its usage looks pretty simple:

and the definition of the "ifdef" macro is remarkably simple as well, considering that we’re hooking into the compilation process itself:

Similar macros come predefined with the language, such as the "using" macro, which serves the same basic purpose as the "using" keyword does in C#:

This ability to extend the core language is a powerful feature, and one that shouldn’t be taken lightly — consider the pain and suffering that Visual Basic developers went through, waiting for the team at Microsoft to introduce the "using" keyword into their language. Had Visual Basic supported a syntactic macro system like Boo’s, developers could have added it themselves, alongside any other interesting and useful macros they’d cooked up along the way. Or, if you don’t particularly like the exact semantics of the "using" keyword–I personally have always wanted a diagnostic logging statement in the "finally" block generated by the compiler–then in a macro-based approach, the semantics can change without having to go back to Microsoft to ask for a patch.

The point of all this, in case you were concerned, is not that you need to give up your existing fondness for an object-oriented language; Lord knows, I’m not going to any time soon, either. C# and Visual Basic are both languages that serve a useful purpose, and nobody should be telling you to give up something that serves a useful purpose. (Though you’ll never get me to admit it, Perl probably does, too, but let’s keep that a secret between us, OK?) In fact, the point is precisely the opposite: to make use of *all* the tools available to you–F#, Boo, Nemerle, C# and/or VB, as the problem dictates.

After all, who wants to own just a hammer? Sounds boring, to me.

—

Ted Neward is a Principal Consultant with ThoughtWorks, an international consultancy developing agile solutions. He is fluent in Java, C#, Scala, F#, C++, and a few other languages, and spends a lot of time studying programming languages, virtual execution engines, and scalable enterprise systems. He currently resides in the Pacific Northwest.

During our careers as developers we sit in training courses, conference sessions and vendor demos in which we hear about how easy and great technology XYZ is for creating applications. And almost every one of them assumes that we’re working on new development. I’m not sure about you, but I find that new (or greenfield) application development has been fairly rare in my career.

More often than not, we developers are working in companies that already have a suite of custom software. If you’re brought in to work on a specific project, it’s rare that the project hasn’t had something already started on it. As a result, most developers spend a large part of their careers working with or on someone else’s code. I don’t think this is a bad thing. Instead I think that it is something that we should embrace with good practices, strong tooling and a strong desire to succeed.

If these projects aren’t classified as greenfield, how should we refer to them? One of the most common terms in application development is ‘legacy’. In his highly regarded book "Working Effectively with Legacy Code", Michael Feathers defines legacy code as:

"…code from the past maintained because it works"

I completely agree with this definition, but it does leave us with a gap. Partially completed applications fall squarely in between legacy (maintained and working) and greenfield (not inhibited by past technical decisions or debt). They have incurred some amount of technical debt and are probably constrained by some design or architectural decisions. They also are only partially completed so we’re not working with the code base solely to maintain an already functioning application. Heck, none of the application may function at all yet. It’s these applications that we developers often find ourselves working on.

What is Brownfield?

Brownfield is an increasingly common term being used to define application code bases that are neither greenfield nor legacy. My definition of brownfield is:

" a project, or codebase, that was previously created and may be contaminated by poor practices, structure, and design but has the potential to be revived through comprehensive and directed refactoring".

Brownfield applications exist all over the world, at all companies and in most project teams. Unfortunately they’re the norm, not the exception in our industry. The developer across the world, sitting next to you, and even you and I are all creating brownfield applications on a daily basis.

The fact is that as soon as we write a line of code, we have accepted that we are going to maintain it in that form. No matter how well you write code, you are, at the point of finishing that line of code, incurring some level of technical debt and some level of technical constraint going forward. Every single line of code you write is adding to or otherwise modifying the level at which those debts exist.

These statements shouldn’t scare you. With today’s tools (Visual Studio, Resharper, CodeRush and others) modifying code after it has been created is easier than ever. Refactoring code to lessen the debt or constraints that it places on the project and team is something that is actively encouraged in development circles. There are some things that are important to ensuring that you can do this efficiently and effectively.

Confidence

The largest factor that you need to take into account when working with brownfield code is the confidence level that you, your testers and your client will have with the act of modifying code, without breaking functionality. Nothing scares testers and clients more than the prospect of failing regression tests and bugs introduced to already functional code. It’s not just frustrating for them; it’s also very demoralizing for a development team. As soon as defects are being introduced from what should be innocuous code refactorings, the development team starts to lose confidence in their abilities to manage the code, their abilities to work with the code in its current state, and the their ability to deliver a product to the client. Those things are a well baked recipe for disaster.

I have seen this on projects which I’ve joined part way through their completion. In one case I was working with a team where the developers and the business analysts would physically break into cold sweats when there was mention of having to alter one component within the application. They did anything that they could to avoid having to fix bugs or rework that code. That was a team that had completely lost its confidence.

There are some tools that you can use to build or maintain a team’s confidence. One of them is automated testing. Even if your team is currently confident about its ability to refactor code, creating a suite of unit and/or integration tests will prove to them that their confidence isn’t misplaced. When a team is lacking confidence, adding tests prior to making changes and ensuring that they’re still passing after the changes have been effected will work to rebuild that confidence.

I know that some of you will now be saying that you don’t have the time, resources or budget to build a comprehensive test suite for your brownfield application. You’re probably right. Creating tests after the code has been completed is a laborious task, more so if your project has a significant amount of code. Instead of worrying about comprehensive application-wide test suites, let’s think about them in a more localized and granular fashion.

If you’re looking to rework a small portion of an application to increase its maintainability, work first at creating a good test suite for that small portion of the application. This approach of creating confidence-building tests only for the portions of the application that are going to be changed will allow you to minimize the effort required for creating tests while maximizing the confidence the team has in changes it makes. This approach will possibly never see your application have a comprehensive test suite, but any changes that you have made will be covered by tests.

The advice for creating tests prior to refactoring code also stands true when the team is faced with modifying code to fix defects or to add new functionality. In the case of fixing defects, first creating a test that exposes the bug (and fails as a result of that bug) allows you to have full confidence that you’ve fixed it. Likewise creating tests that confirm new functionality prior to creating the code to implement those capabilities within the application will give you a strong understanding of when you’ve completed the required task.

All Shall Build

Confidence in implementing, fixing and refactoring code is only one part of the brownfield project experience. Confidence on projects comes in many forms. For instance, do you have full confidence that anyone on your project team could correctly compile and deploy your application to any of your testing, staging or production environments? My experience is that there is usually one person on the team who can perform this task and it usually involves a combination of black magic, voodoo and chicken sacrifices.

Why not work towards a scenario where any one of the developers on the team can, and does, regularly create a release package? Traditional methods of compiling applications in the .NET sphere have been limited to "Rebuild All" within the Visual Studio IDE. While there can be some automation included through the use of pre- and post-build events, "Rebuild All" is a largely manual process when you include tasks such as test execution and release package construction. While this can work, it doesn’t necessarily offer the consistency, or flexibility, of using a build script. There are many build scripting tools and languages available to .NET developers. Each offers the same fundamental features, but each also offers a twist.

If you create a compilation, testing and release script that is available to every developer on the team, it becomes one of your projects mechanisms for creating confidence in release management. If the script is available to all developers at all times, they should be encouraged, through simple tooling and fast execution, to run it as many times per day as possible. Every time that you run a script like this you are testing a technically complex portion of your release management process. If that process is being executed tens of times per day, the confidence level of the team and its management and clients should soar.

Social Considerations

We could continue this article with a long and detailed discussion on the merits of using fundamental OO code practices to ensure that your code is flexible, decoupled and robust. Instead I’d like to finish with a brief discussion on the social aspect of working with brownfield applications.

Joining an already started project can present interesting issues both in personal and team dynamics. We all join a new project or job thinking that we can bring greatness to the team or job. As a result we often head into these new engagements bringing suggestions about changes to techniques, tools or practices that we’ve had success with in the past. Unfortunately this usually happens without us taking the time to properly and completely assess the current situation. This lack of understanding around past decisions and current situations, combined with the vocal desire to try to improve things, often leads new team members to be seen in a negative light.

When joining a team we also have to be aware that the currently serving team members probably have an emotional attachment to the current code base. While many developers believe that they are able to openly take constructive criticism of their past or present endeavours, often they can’t. When suggesting changes to a code base, new team members need to be cognizant of the potential negative impact that they may be inflicting on the team’s morale. While this is a delicate situation to have to step around while working to improve code, it is one that is vitally important if the team, as a whole, is going to improve productivity, maintainability and quality going forward. Getting team members to believe in the reasons behind the changes that you’re proposing instead of the changes themselves can be an effective technique.

Many projects that we developers encounter should be categorized as brownfield. While each of these projects encounters its own set of unique issues, there are some underlying problems that are seen on many. Projects might suffer from a lack of confidence in code, releases and quality. They can also have severe maintainability issues caused by poor coding practices. Those technical issues can be some of the easiest to solve, but each technical problem can expose a series of potentially debilitating social and team cohesion issues.

Don’t let any of these issues scare you though. You’re probably already working on a brownfield project and having to deal with many of them. Remember that much of what we do in this industry is deal in areas of confidence between developers and management, developers and clients, and developers amongst themselves.

—

Donald Belcham is an independent contractor in Edmonton, Alberta, Canada who specializes in software development with the .NET platform.  With more than 7 years experience delivering Web and Smart Client applications to government and Fortune 100 clients in North America and the South Pacific, he has experience and knowledge in the entire software development life cycle.  Combining that experience with his passion for agile development practices and solid OO fundamentals, Donald works to provide the client with software that works for their business.

Recognized by Microsoft for his technical skill and community contribution with the Microsoft MVP award in C#, Donald is a notable leader in the developer community.  In addition to being a founding member, and current President, of the Edmonton .NET User Group, Donald regularly speaks for .NET User Groups and Code Camps across North America on topics ranging from development practices to the intricacies of different technologies.

His thoughts on software development can be found on his blog at www.igloocoder.com.