So our friend David Burela is back at it again with Developer Blog Banter #2: How do you test your applications?
How do you organize your tests. Do you separate your unit tests, integration tests and UI tests into separate projects? Do you do anything specific to keep track of your tests? What naming conventions do you use? Do you run them before a check in or is that what the build server is for?
If you are not testing, then how would you like to test your apps if given the opportunity?
This post is my response to the above question.
I tend to have a Specs assembly for each major component of a project, these usually include some form of integration tests using in memory SQLite databases. The first thing most people will think is that I am probably mixing integration tests with my specs/unit tests and getting everything all mishmashed together. That is probably a good observation. As I find myself almost exclusively using MSpec for testing I see no real reason to separate the tests into any other grouping other than by their system components.
An example set of Test assemblies would be :
I did learn a few tricks from my friend @cbilson that I really liked and continue to use. That is naming the actual files in the test assemblies about the feature or part of the system we are testing. So for a group of specs that test the calculation of a bunch of distribution dates for a retirement fund based on some frequency (monthly, quarterly, annually etc) the name of the file would be “creating_distribution_dates_for_funds.cs”. This name is also used for the namespace that all the tests live in since MSpec tests are each a separate class. Groups of related tests can be found quickly and helps others that may come on to the project find the tests that describe how something works.
Okay so about those in-memory database integration tests. Well this is another thing that @cbilson and I worked on together (ok mostly him but I helped). Its certainly not a new idea , I got it from some blog post of ayende’s that I read, but it was a major breakthrough in helping us move quickly with testing and be extremely accurate. We were able to not only have nice _FAST_ tests for mappings in nHibernate but also were able to test queries quickly and accurately. Having this ability helps a lot when you want to be able to test with not only the database for your application but also for test versions of other databases you may need to access (most of our apps use at least 3-4 databases). This can make repository testing a no brainer and helps eliminate the kludgy methods people have had to use in the past. @NotMyself, @codereflection and I have even gone so far as to integrate NBuilder into the process for some scenarios making tests clean, expressive and to the point. I’m getting sidetracked I think…. testing is exciting stuff damnit!
Of course all our tests are run on the CI server and we “always” run them all before committing…. right? 😉
I would like to one day soon find a really elegant way to add some more integration testing, maybe even at the UI level, into the process. As this is very painful and hard to maintain today we do our best to test as much as we can. There is no substitution for great comprehensive QA we just hope we make their jobs a lot easier by building well designed rock solid software… that works.
Hope this covers the question and hope someone finds some value in it.
See also my response to the first Developer Blog Banter : My Technology Stack