We're hard at work on CSV Manager 1.2 - coming out Real Soon Now! But in the meantime we're bringing out a bug fix release of 1.1. This release (1.1.11) fixes bug #0013: comment characters at the start of a data field were causing parse errors.

This was a real nasty bug. We added comment support recently as it was a common feature request from customers. And this bug is a really classic example of the law of unintended consequences. It's a great new feature, but it introduced a hard-to-find bug. We had no test cases for the standard comment character '#', in the case where it was just ordinary data. In fact we have no test cases to check that all valid characters can in fact be parsed. We sould have had them. So this is a classic lesson: think hard about regression testing, and try really really hard to check against all input conditions.

One easy way to perform this type of test is something called fuzz-testing. Basically you chuck a load of random data at your program and see it it breaks. We actually do have a test like this. We use massive randomly generated CSV files to check performance and parsing. But here's the thing. At no point did any of those files ever have a '#' character in just the right place to trigger this bug. The laws of probability are against you on this one.

I think this shows the importance of a combined testing approach. You must apply all the techniques. None of them are silver bullets. You must create tests based on really hard thinking about test conditions, and you should also have randomised testing. And don't forget your test coverage either. Or multi-platform testing. etc. etc. Erik Sink had some good points on this stuff recently. Anyway, the long and the short is, it's not easy to cover all the bases and you have to put in the leg work. No wonder software products take ten times more work than software projects.

Back to the bug fix release. This is an important one to download and install. If your input data ever contains a '#' you could be in trouble with older versions of CSV Manager. So login to your user account and grab the latest release. We strongly encourage you to do this. And let me apologise for introducing such a nasty bug. Data integrity and loss prevention is a design mantra for CSV Manager, so we're not happy campers at all about messing that up.

Finally, thanks to Dan for discovering the bug. He also got our first bug-bounty – a $15 ThinkGeek gift cert.

tag-gen:Technorati Tags: ricebridge java csv manager jar Del.icio.us Tags: ricebridge java csv manager jar