continuous integration

Continuous Integration Enters the Cloud

Posted on March 4, 2010 by Scott

I came across this blog post in Google Reader and thought I’d share it. The idea of being able to outsource the care and feeding of a continuous integration system to someone else is a very interesting one. Having implemented and maintained such systems (which I’ve blogged about in the past), I know it can be a lot of work (though using a product like TeamCity lightens the load considerably compared with CruiseControl.NET). Stelligent isn’t the first company to come up the idea of CI in the cloud, but they may be the first using all free/open source tools to implement it.

I’ve read Paul Duvall’s book on continuous integration and highly recommend it to anyone who works with CI systems on a regular basis. If anyone can make a service like this successful, Mr. Duvall can.

Build Server Debugging

Posted on August 24, 2009 by Scott

Early in June, I posted about inheriting a continuous integration setup from a former colleague. Since then, I’ve replaced CruiseControl.NET and MSTest with TeamCity and NUnit 2.5.1, added FxCop, NCover, and documentation generation (with Doxygen). This system had been running pretty smoothly, with the exception of an occasional build failure due to SQL execution error. Initially, I thought the problem was due to the build restoring a database for use by some of our integration tests. But when replacing the restore command with a script for database creation didn’t fix the problem, I had to look deeper.

A look at the error logs for SQL Server Express 2005 revealed a number of messages that looked like:

SQL Server has encountered <x> occurrence(s) of cachestore flush …

Most of what I found in my initial searches indicated that these could be ignored. But a bit more googling brought me to this thread of an MSDN SQL Server database forum. The answer by Tom Huleatt that recommended turning off the Auto-Close property seemed the most appropriate. After checking in database script changes that included the following:

ALTER DATABASE <database name> SET AUTO_CLOSE OFF

GO

none of the builds have failed due to SQL execution errors. We’ll see if these results continue.

MSBuild Transforms, Batching, Well-Known Metadata and MSTest

Posted on June 12, 2009 by Scott

Thanks to a comment from Daniel Richardson on my previous MSTest post (and a lot more research, testing, & debugging), I’ve found a more flexible way of calling MSTest from MSBuild. The main drawback of the solution I blogged about earlier was that new test assemblies added to the solution would not be run in MSBuild unless the Exec call to MSTest.exe was updated to include them. But thanks to a combination of MSBuild transforms and batching, this is no longer necessary.

First, I needed to create a list of test assemblies. The solution is structured in a way that makes this relatively simple. All of our test assemblies live in a “Tests” folder, so there’s a root to start from. The assemblies all have the suffix “.Test.dll” too. The following CreateItem task does the rest:

The task above creates a TestAssemblies element, which contains a semicolon-delimited list of paths to every test assembly for the application. Since the MSTest command line needs a space between each test assembly passed to it, the TestAssemblies element can’t be used as-is. Each assembly also requires a “/testcontainer:” prefix. Both of these issues are addressed by the combined use of transforms, batching, and well-known metadata as shown below:

<Exec Command=””$(VS90COMNTOOLS)..IDEmstest.exe” @(TestAssemblies->’%(TestContainerPrefix)%(FullPath)’,’ ‘) /runconfig:localtestrun.testrunconfig” />

Note the use of %(TestContainerPrefix) above. I defined that metadata element in the CreateItem task. Because it’s part of each item in TestAssemblies, I can refer to it in the transform. The %(FullPath) is well-known item metadata. For each assembly in TestAssemblies, it returns the full path to the file. As for the semi-colon delimiter that appears by default, the last parameter of the transform (the single-quoted space) replaces it.

The end result is a MSTest call that works no matter how many test assemblies are added, with no further editing of the build script.

Here’s a list of the links that I looked at that helped me find this solution:

Calling MSTest from MSBuild or The Price of Not Buying TFS

Posted on June 3, 2009 by Scott

When one of my colleagues left for a new opportunity, I inherited the continuous build setup he built for our project. This has meant spending the past few weeks scrambling to get up to speed on CruiseControl.NET, MSTest and Subversion (among other things). Because we don’t use TFS, creating a build server required us to install Visual Studio 2008 in order to run unit tests as part of the build, along with a number of other third-party tasks to make MSBuild work more like NAnt. So the first time a build failed because of tests that had passed locally, I wasn’t looking forward to figuring out precisely which of these pieces triggered the problem.

After reimplementing unit tests a couple of different ways and still getting the same results (tests passing locally and failing on the build server), we eventually discovered that the problem was a bug in Visual Studio 2008 SP1. Once we installed the hotfix, our unit tests passed on the build server without us having to change them. This hasn’t been the last issue we’ve had with our “TFS-lite” build server.

Build timeouts have proven to be the latest hassle. Instead of the tests passing locally and failing on the build server, they actually passed in both places. But for whatever reason, the test task didn’t really complete and build timed out. Increasing the build timeout didn’t address the issue either. Yesterday, thanks to the Microsoft Build Sidekick editor, we narrowed the problem down to the MSTest task in our build file. The task is the creation of Nati Dobkin, and it made writing the test build target easier (at least until we couldn’t get it to work consistently). So far, I haven’t found (or written) an alternative task, but I did find a blog post that pointed the way to our current solution.

The solution:

<!– MSTest won’t work if the tests weren’t built in the Debug configuration –>
<Target Name=”Test:MSTest” Condition=” ‘$(Configuration)’ == ‘Debug'”>
<MakeDir Directories=”$(TestResultsDir)” />
<MSBuild.ExtensionPack.FileSystem.Folder TaskAction=”RemoveContent” Path=”$(TestResultsDir)” />

<Exec Command=””$(VS90COMNTOOLS)..IDEmstest.exe” /testcontainer:$(TestDir)<test assembly directory>bin$(Configuration)<test assembly>.dll /testcontainer:$(TestDir)<test assembly directory>bin$(Configuration)<test assembly>.dll /testcontainer:$(TestDir)<test assembly directory>bin$(Configuration)<test assembly>.dll /runconfig:localtestrun.testrunconfig” />

</Target>

TestDir and TestResultsDir are defined in a property group at the beginning of the MSBuild file. VS90COMNTOOLS is an environment variable created during the install of Visual Studio 2008. Configuration comes from the solution file. Actual test assembly directories and names have been replaced with <test assembly> and <test assembly directory>. The only drawback to the solution so far is that we’ll have to update our MSBuild file if we add a new test assembly.

TeamCity 3.0

Posted on January 18, 2008 by Scott

Now there’s a freeware version of it that supports up to 20 users and build configurations. We were looking at setting up CruiseControl.NET again for continuous integration at work, but this will be much easier.

Posted on October 11, 2006 by Scott

The practice is well-defined in a couple of articles on Wikipedia and on Martin Fowler’s website. But as long as I’ve been reading about this best practice, I’ve never seen it implemented at any of my past jobs (or my current one for that matter). Fortunately, one of the consultants I’m currently working with not only has it implemented, but has the necessary software and test projects on a USB key that he carries with him from job-to-job.

Before I demonstrate it to the broader software team as a practice, I’m trying to get it working on my own machine. Because he uses MbUnit instead of NUnit as part of his implementation, it took me a little longer to get the second of his six test projects working. A little googling for Nant and MbUnit yielded an article that listed 5 files to be copied to the bin directory of Nant. Once I did that, the second test project worked fine.

Strangely, I only saw 4 of the 5 files in this list:

MbUnit.Core.dll
MbUnit.Framework.dll
MbUnit.Tasks.dll
Quickgraph.dll
QuickGraph.Algorithms.dll

The core dll was missing, but copying the other 4 dlls did the trick.