Versioning Cistern

My versioning system is broken. It's painful and I'm not entirely sure what to do about it. Time for a blog entry so!

Previously I wrote about my vision for the Ricebridge Versioning System. All Ricebridge components have three version numbers. For example, 1.2.3 means that you have major version 1, minor version 2, and build number 3.

Major versions allow for incompatible changes. Minor versions add new functionality but keep compatibility. Build numbers track bug fixes and minor changes. I think it works very nicely. It's pretty clear and easy to follow. And you know where you stand and what you can upgrade to. You know that a major version change will probably break your code and you'll have some extra work to do. But you also know that you can upgrade to a bug fix release or take advantage of a new method in a minor release by just dropping the jar in and continuing on your way.

I'm very attached to the three number versioning system. And so are my customers who've got very used to it. Changing versioning system is in itself a major version change really (given that you are breaking semantic compatibility), and I don't really feel like it at the moment!

So what's broken? Well I have new version of CSV Manager coming out soon. This will add a few new things, including Java Beans support. Nothing that will affect the existing code, so no problems there.

However, based on the experience gained building XML Manager, and my future plans for the product set, it looks like some of the interfaces in CSV Manager will have to change.

I want to create a coherent set of components that all work the same way. That's very important. There has to be interop, both at the code level and at the user level. That means that the APIs should be the same, Once you learn the API of one Ricebridge component, you can then apply it to all Ricebridge components (pretty much).

But the original CSV Manager API is not right for this. It needs to be modified. That breaks compatibility. So fine. We go to CSV Manager 2.0.

Except, not so fine. Not at all. You see, Ricebridge customers get an upgrade path. When you buy from us, you get the right to upgrade, for free, to a release that has the same major version. Right now CSV Manager is at major version 1. If I bump it up to major version 2, then all those customers will loose out. Ouch. Not very nice at all. Definitely not the right way to go. I want existing customers to get the full benefit of the new API as well.

It looks like we can’t bump the major version. So let's drop the compatibility restriction on minor versions. That could work. Except now, when you look at a version number, you can't tell right away whether it will work with your current setup. Not so good either.

How about using four version numbers? We can add an extra one for compatibility: So we bump the release number every time there's an incompatibility change. Except this is not very user-friendly. Four version numbers is really pushing it. Three is just about as much as anyone can take. In any case, it's a change I don’t want, as noted above.

Another option is to stick with the old API until the real version 2.0. We can add the new stuff, but keep the old stuff in and deprecate it. This is the standard way of doing things and it is the approach that I use normally. It works especially well for adding and removing methods from API classes. But it doesn't work for interfaces that the user implements.

If you want to change an interface that a user has implemented, then you have to force the user to change their implementation class. There's no easy way round that (sadly we're in Javaland, so dynamic solutions are awkward). This happens because the interaction model between the interface and client code changes. The implementation class must provide the new services required by the client code. In some cases it might be possible to work around this, but mostly it's very messy and there is no way you can ask people to do this.

In the end, we are forced to choose between a new implementation interface and an old one. So if we want to wait until 2.0 for the big change, then we must stick with the old API for now. This works, but it increases the amount of people using the old API and makes the changeover much more painful for many more people. Microsoft has been badly stung by this many times: The long and sad story of the Shell Folders key. It seems better on the whole to take the hit now and impact the smallest number of people.

So where does this leave us? The next release of CSV Manager will include the new API. It will break old code. I'm really sorry to have to do this to my customers. It's very nasty. But the benefit will be the many ways in which you will be able to use Ricebridge components together to solve all sorts of tricky problems. It seems like a good tradeoff in the longterm.

Actually it's not even as bad as all that. The main CSV Manager methods will not be changing much and 90% of existing code should still work fine. The change will be occurring in a part of the code that is not even used by most customers (LineProviders). So maybe all this agonising isn't even necessary!

That still leaves the problem that version 1.1 and version 1.2 will be incompatible. We still need a way to communicate this and to allow users to control versioning in their own projects. It looks like an additional stream of version information is needed. We need to track API changes separately.

One idea is to have API revisions. This means publishing a detailed description of the API and assigned it a revision number. All CSV Manager releases with the same API revision number are going to be API compatible. That means you can copy in the new jar and things will still just work.

When incompatible changes have to be made to the API, then the revision number changes. A new API description is published, showing the changes, so that users can track what happened. So each version of CSV Manager has an API revision. You can show this in a table and it should then be easy for people to follow.

API revisions would not change very frequently. In fact, as we move further along with building Ricebridge components, things should start to settle down very solidly. I don't think that there will be many API revisions.

In this system, the minor version number then means: new functionality and possibly a new API revision. This creates extra work in that API revision changes have to be made clear to users, and handled carefully.

What really gets me about all this is that I had actually put in place a system to deal with it. After reading David Bau's posts about a theory of compatibility, I created a design for the user-implemented interfaces that could accommodate certain types of changes.

When you implement a Ricebridge interface, you are actually advised to extend a designated abstract support class. This class has the job of insulating you from future API changes by translating changes into the older version of what you expect. Except that this only works in some cases. To solve my existing problem it looks like I would have to write a lot of reflection code and that would probably have a bad effect on the performance of the system. It also means that new customers would find the support class to be a confusing mess. I have in fact used this technique already on one minor change to CSV Manager.

One problem that it does not solve however is that I made a mistake with the method signatures when I first released CSV Manager. Some *Impl methods which should be protected are actually public. Bugger!

So the way forward is still not clear. I am inclined to take the hit now and offer free support to all customers who need it to make the change.

What do you guys and gals think I should do?

In proving foresight may be vain;
The best-laid schemes o' mice an 'men
Gang aft agley,
An'lea'e us nought but grief an' pain,
For promis'd joy!

Robert Burns

tag-gen:Technorati Tags: Tags:

Posted in Java | Leave a comment

Make $97 With One Blog Post

If Jonathan can give away free servers, then I can give away free licenses! I've been inspired by Sun's T2000 promotion to try something like it myself.

Here's the deal. You blog about one of our products, after trying the trial version out, and you get a free single-developer license.

You can write whatever you like. Tear us to shreds or sing our praises. It's all good. We just want links :)

Well, you should make sure that your audience is OK with doing something like this. Full disclosure is a good idea. So don't do anything you're not comfortable with.

Before you start writing, here's the full details of the blog promotion.

Our products are data-munging widgets for Java: CSV Manager (for CSV files, surprisingly), and XML Manager (for XML files, again, a surprise there). The single dev licenses are worth $47 and $97 dollars respectively.

Oh, and the $15 gift cert you get for every bug found, that's still valid. So you might even end up with something nice from Think Geek.

No idea when I'll end this promotion, so don't hang around if you want one, get writing!

And no, I won't apologise for the blatantly linkbaiting title! :)

tag-gen:Technorati Tags: Tags:

Posted in Business | Leave a comment

Rant: Maven Muppetry

Hani was right. Maven really is a pain in the arse.

I have just managed to install the cobertura plugin. Not by following the instructions mind. On no.

The install instructions are to run the following command:

maven plugin:download

Well shucks ain't that nice and easy. And it used to work, because I used that very same command a few months ago and it just worked. Tip for the documentation writer: put this command all on one line and then it's much easier to cut-and-paste into a command window.

But now it doesn't work anymore. Wanna know why? Because it references a load of dependencies inside that do not exist. Did someone delete them? Huh?

In order to install you have to drop the cobertura plugin jar into your repository manually, and then run maven. Then it picks up the dependencies from berlios.

Am I missing something here?

If you're going to release open source stuff, make sure your install just works. No arsing around. People lose interest pretty quickly when you do that sort of thing.

Posted in General | 1 Comment

CSV Manager 1.1.10 Released

This is a bug fix release for bug report 0012: the TableModelLineListener object was dropping the first row of data when the headers setting was true. If you are using Swing TableModels please check if this bug affects you. As usual, all Ricebridge customers get a free upgrade with this fix.

Thanks to Jøgen for catching this one!

tag-gen:Technorati Tags: Tags:

Posted in General | Leave a comment

10 Reasons Not To Use Commission Junction

Ooh, a blog list! So commission junction (no capitals for you, ya messers!) are off my site. Here's why:

10. When they deactivate your account, they don't send you any email to let you know about it. The first time you find out is when you try to login. Bummer for you!

9. There is no way to contact them by email. No addresses on their site. Nix, Nada, Zippity-Doo-Da. Customer support is provided by international telephone numbers. Oh yeah! Do it to me baby!

8. The only way to get in touch using this new-fangled interwebnet thing is by using a teeny-tiny itsy-bitsy help form for lost passwords. No, I'm not linking to it, but it's not hard to find from the login page. Ah go on, go find it, it's hilarious. Just remember to write really small. Oh, and I wouldn't try to have a longish domain name – cos it won't fit! Muppets! /shakes head

7. If you're not making the numbers after six months, they cut you off. Hello? Ever heard of the long tail fellas? Hey I know it's just a little old personal blog, but do you have to be so mean? Not everyone can be Cory or the Doc, you know. Some us of just have our own wee little corner of cyberspace we call home. Sob …. bwwahhhaahha…

6. If your account is deactivated it can't be reactivated. Ah Holy God! databases … yesss … beeg ma-jeek! veree scaree … yesss …

5. After kicking you in the metaphorical nuts, they say, and I quote:

Thank you for allowing us to be of service to you. Client Services Commission Junction, A Valueclick Company

ROFL dudes.

4. All their mails are Roach-Motel mails. As in, you can receive 'em, but you can never reply… stimulus, no-response, stimulus, no-reponse, ...

3. Their online reporting sucks. This one is from memory since I can't, um, login anymore.

2. They deactivate your account, without confirming your final balance. Nasty. Any word on how I'm supposed to get my few pence out guys (if indeed there is any)?

And the number one reason not to use commission junction (still no caps for you!) …

1. Their site is … puke-green. Oh my poor dear eyes.

Well CJ, it's been great, but it's time to move on. It's not you, it's me. I'm sure you'll find someone who loves you.

Wait. Actually…

Go gcreime maorlathaí­ m­í­thrócaireach do chuid infheistí­ochtaí­!
The curse of Cromwell on you!
Go scriosa cúnna ifrinn do chuid fo-éadaigh!
May you be afflicted with the itch and have no nails to scratch with!

Me? Bitter? Nah…

tag-gen:Technorati Tags: Tags:

Posted in General | Leave a comment

Friday Fun: Fancy an Island?

Feeling a little despotic? Need to impose your vision on the world?

Look no further than … a private island!

Sure Charlie did it. So can you!

Posted in General | Leave a comment

Jostraca 0.4.1 Released!

I have just released the latest version of my open–source code–generator: Jostraca. I first released Jostraca in late 2000 and I have been unable to get rid of it since, despite repeated kicking!

So what does Jostraca do? It takes a template and generates a lot of repetitive code. Anytime you have loads of repeated code, you can use it. Now maybe with EJB3 that's not such a big deal anymore,
but if you still want just one definition of your data objects, templates can keep them DRY.

The other thing about Jostraca is that is uses JSP-style syntax, so you don't have to learn any new-fangled semi-language. You just use the language you know. And you can also use regex macros to completely define your own template syntax if you really want to. Jostraca is also language agnostic. Currently you write templates in Java, JavaScript, Python, Jython, Ruby, C, Rebol and Perl (of course!). And you can even define your own code generator formats so you're not stuck with the standard one (which just wants to dump out a load of files).

That said, Jostraca is warty. It's a work in progress and the project iterations are about 1 year(!) in length. In this business it can be quite rare to work with a piece of code for such a long time. Most projects come and go. You move on, others take over (and curse you), and the merry-go-round continues.

The nice thing about running an open-source project is that you get to step out of the coding rat-race once in while. Jostraca has been through many refactorings, and is a lot better for it. Of course, there's still a lot to do and there's a bit pile of user stories in the unassigned bucket. But no harm. I've got years to get it right. You don't often have that luxury as a coder.

One last thing. I'd like to welcome Morten Christensen on board. He's done some great work on the Eclipse Plugin for Jostraca, which was sorely lacking. He's doing some good stuff for code-generation in Eclipse.

By the way, you may have noticed that I love competitions. So here's another one. If you can figure out how I came up with the name “Jostraca”, you can get a free XML Manager license. BUT DON'T POST THE ANSWER HERE! To qualify you have to send me the answer in a private mail. That means everyone has a fair chance. (The standard competition rules apply, blah-de-blah etc.)

Oh yeah. The Sphinx is kind of a clue…

tag gen:Technorati Tags: Tags:

Posted in Java | Leave a comment

CodeGenClipse Released!

Jostraca now has an Eclipse plugin: CodeGenClipse! (Jostraca is a code-generator, in case you're interested.)

Open source is great. The plugin is all thanks to Morten Christensen. And he found a load of bugs in Jostraca while he was writing it, so they're all fixed now.

And that's not all folks. CodeGenClipse is not just Jostraca specific. It's a general plugin framework for code-generation. So look out for some more cool stuff from Morten in the future.

This post probably sounds a little over-excited. Sorry about that. It's just cool when open-source works out. It really is a great way to get things done. Of course, I still think there's a place for commercial stuff, but especially when it comes to developer tools, you can't beat a shared solution.

And for all you Jostraca fans out there, the long wait is nearly over — 0.4.1 is oh–so nearly, nearly ready… (and will be announced right here on this blog of course)

Posted in Java | Leave a comment

Munster Wins!

Woohoo! Munster wins the Heineken Cup Final!

I'd only have a casual interest in Rugby, but I'd have to say I really enjoyed the final. Nail-biting stuff at the end.

Posted in General | Leave a comment

Most People Are Below Average

The almighty Seth Godin says: “In every category, in every profession, half the people are below average.“.

Bzzt! Wrong! Na-ah. Half the people in any industry are below the median. As in, half the people are below the half-way point (which is what the median is).

But I still think his main point is right. Let's fix the math.

In any industry you're going to have star performers. Programmers ten times better than the rest, etc. We all know the cliche. That drags the average up and beyond the main group of code-monkeys or office-slaves or whatever-you-call-your-grunts. And it drags the average higher than the median. Ability is a skewed distribution:

So Seth's argument about marketing to the below average applies even more he realises. More than half of the people in any industry are below average.

But there is a counter-point: the average is a lot higher than you think. Most jobs look easier on the outside than on the inside. Assuming you are more clever than most people is a dangerous game.

As for me, hey, I'm with the huddled masses. Who wants to be a Distinguished-Engineer anyway!

tag-gen: Technorati Tags: Tags:

Posted in General | Leave a comment