Friday Fun: Yoda, Yo, You Da Man!

Thanks to Adam Stiles for this one.

I fell off my chair the first time.

Yoda, telling it like it is.

Rant: Functions Deserve Brackets Dammit!

I know brackets are parentheses, but bear with me, we're on the other side of the pond here.

So what is the story with the

functionName (param1, param2, ...)

style? Darlings, you are disassociating context!

Huh? Well look, one of the basics of design, of style, is put things in context. To put those things that are associated with one another, together. That way the eye places them together for the brain, aiding understanding and bringing clarity to the mind.

In languages of the C heritage, you often have variables standing alone:

int    count = 0;
String foo   = "bar";

A name standing by itself is a visual clue that what you are dealing with is a variable.

What about methods? Where's the visual clue? It's the brackets! A name attached to brackets is a method name. It goes into the cerebral cortex straight through the eyes when you scan the code, short circuiting the hard, "thinking" part of understanding code.

functionName( param1, param2, ... )

puts it all in perspective.

People complain about code being hard to read. Well schucks, you're making it hard for yourself! Get with the program and starting applying the principles of visual design to your code.

Big Data Dump at the Small Office

I'm a big fan of BigPictureSmallOffice, mostly because I don't work in one anymore! Apparently the IT department managed to entirely bork the company's database, including the list of oustanding orders. Yee Haa!

Hey, I've been there. One month into starting my first job I deleted a live customer website. Now, we were transfering it to a new server and there was a new design, so no permanent harm done. But the old site was still live (or rather, recently deceased after my act of vandalism), and due to the vagaries of DNS updates, still needed to be live for at least a week. Oh Joy. Let's just say that the client's ISP still had the old IP, and the client was having a hard time believing that other people could see their new site even though they couldn't.

Never make the same mistake twice. Well I have never deleted a live website since. You might think the lesson I learned was: always check twice. Oh no. The lesson I learned was: if you mess up a client and your boss has to handle the angry phone call because you were late in that morning, you are in deep deep doodaa.

Damned if You Document, Damned if You Don't

My recent post about documentation writing and technical writers was a little off the cuff and quite easy to misinterpret. I suppose the phrase "throw some technical writers at it" didn't help either.

To clarify: I think technical writers are a very valuable part of the software development process. I have worked with some really great ones and I know that collaboration can be good fun and produce excellent results. But it has always been for user level documentation, not API level documentation. The problem I see with API docs is that you nearly have to write the documentation to describe the functionality.

So after thinking about this for a while, and thinking about what makes O'Reilly books so good, I think that the approach to take for API docs is one of a more editorial nature. The developer should write the actual text, given that this is the most efficient way to get the information down, and given the enormous benefit of generating a feedback loop in the developer's mind. But then the technical writer can step in to act as mentor and editor – to enhance the writing skills of the developer and to provide those things that only a professional can: style, grammar and textual flow.

So to repeat: I do not see the technical writer as someone who simply takes a set of bullet points and a demo application and mechanically hacks out a lot of repetitive pages. I do understand the value of technical writing. But my question was: how can we apply this skill to API documentation, which is so close to the code? In the end, it must be through even greater collaboration – the technical writer must be part of the team, not off in a separate department. And the technical writer is not a writer in this role, they are an editor and an educator.

Sudoku, Part Three

With the sudoku program I hacked together in the last post, I was able to solve the Irish Independent's Super Sudoku for three weeks running. But then, disaster! It turns out that sudoku puzzles can be pretty fiendish - they are still solvable with a very small set of given numbers.

So my initial logic, just continuously removing numbers that could not possibly be in a cell, did not solve all sudokus. Time for a rethink. After staring at the unsolved output for a while, it occurred to me that sometimes you end up with a set of cells in a vertical, horizontal or home square that are all unsolved, but where only one cell contains a certain number. So lets say the top three cells on the left are 123, 234, 234, with all others in the top row solved. Well then the top left cell must be 1. There are no other possibilities, it's the only place for that number. Now we're suckin' diesel.

For some reason, I think of this new operation as "reducing" the cell possibilities, so henceforth to the code:

sub reduceHome {
  my @cell = @{shift()};
  my $prow = shift;
  my $pcol = shift;

  my $valbin = $cell[$prow][$pcol];
  if( 1 == keys(%{$valbin}) ) {
    return;
  }

  my $foundval;
  for my $val (keys(%{$valbin})) {

    my $found = 0;
    for( my $r = $sqsize*floor($prow / $sqsize); 
         $r < $sqsize*floor(($prow+$sqsize) / $sqsize); 
         $r++ ) {

      for( my $c = $sqsize*floor($pcol / $sqsize); 
           $c < $sqsize*floor(($pcol+$sqsize) / $sqsize); 
           $c++ ) {

        if( $prow != $r || $pcol != $c ) {
          my $testbin = $cell[$r][$c];

          if( "" ne ${$testbin}{$val} ) {
            $found = 1;
          }
        }
      }
    }
    if( !$found ) {
      $foundval = $val;
      last;
    }
  }

  if( "" ne $foundval ) {
    for my $val (keys(%{$valbin})) {
      if( $val ne $foundval ) { 
        delete(${$valbin}{ $val });
      }
    }
  }
}

Yikes. Well what does this monstrosity do? Again I plead guilty to all charges of hacking and offer this code as final proof that Perl is meant to be written, not read. So this just runs through all the cells in the home square of a given cell: $prow, $pcol. The ugly nested for loop in the middle does that by keeping us within the bounds of the home square. So if we're cell 2,2 in a 3x3 sudoku, the home square is defined as all the cells in the range 0,0 - 3,3.

Now, as we run through the cells, if we come across a value $foundval, that is only in one cell, then we can delete all the other numbers from that cell. That's what the last for loop does in a rather pointless way - why not just create a new map with just one entry? I don't know, I was cut-n-pasting I guess.

This operation is also performed horizontally and vertically, so our main loop now does all this:

excludeHome(\@cell,$rowI,$colI);
excludeVertical(\@cell,$rowI,$colI);
excludeHorizontal(\@cell,$rowI,$colI);
reduceHome(\@cell,$rowI,$colI);
reduceVertical(\@cell,$rowI,$colI);
reduceHorizontal(\@cell,$rowI,$colI);

I'd say that's efficient alright. So did this help? You bet! I got another two weeks out of this baby. But, yeah, you guessed, still not enough. Even this code won't solve all sudokus. There's another trick that I came up with though, and we'll look at that the next time.

This post is part of a series on a Perl Suduko solver.

Documentation Damnation

I've been writing a lot (and I mean a whole lot) of JavaDoc lately. It's all for the new Ricebridge product, to be announced Real Soon Now.

One of the things that I like to do with my components is to document pretty much every method. This really makes things easier for users as you can grab usage information out of context just by reading one method description. So it's a big part of what makes Ricebridge components special – pretty good docs.

The reason most JavaDoc documentation is not very good is that it is a real pain to write. Technical writing in general is very hard. It's one of those things that "disappears" when it's good. You only notice bad documentation. Sun is quite good at it. But in fairness a lot of their documentation is still fairly sparse. And don't get me started on most open source Java components (even my own!). There's just not enough payback to really put the hours in.

But for paying customers, you just gotta do it. It's part of what people pay for. I certainly expect proper and complete documentation with anything I buy. I am often disappointed, but when the documentation is good it really makes a big impression on me. So that's why I have to document pretty much everything when it comes to Ricebridge components.

Right now, I can do this myself. It's time consuming and hard work, but it still makes sense to do it myself. However, I am beginning to wonder how the documentation process can be optimised. I'm sorry, but you can't just throw technical writers at it. First of all, it's not usage documentation in the traditional sense. We're talking hard-core techie stuff here. Secondly, you really have to be a developer to have a feeling for the pain points. Thirdly, forcing developers to write creates a nice feedback loop that actually increases code quality. But still, we do need to actually produce something that describes all the salient information about a given method, in good writing. I'm thinking text that is of the O'Reilly standard.

So developers can't write well (as a rule), but they know the facts. And technical writers can write, but a deep understanding of the API is something they will struggle with. Where do we go from here? I don't know. I know there are some folks out there who can mix these disciplines. I know that a lot of technical writers would say that they are up to job. But I want to know how it can be done with ordinary people in a small company. That's useful. Star performers are not a real solution. How does the average developer/technical writer produce great API documentation?

Rolling On

Roller is working out pretty well. No road blocks so far for anything that I wanted to do.

However, my approach is probably not the easiest way to do things. I have set up the main layout of the site in the _decorator.vm template. I edit this directly in my own theme and then reselect the theme to apply the changes. You don't actually have to reselect the theme each time – you can use the theme preview window to see your latest changes by just hitting F5.

I have my own theme just to see what maintaining a theme involves. The trouble is that you need to be able to edit the theme files directly on the server. No bother to me as I just use Emacs for that. But you do have to login to the server and use it from there.

The ideal setup is to edit locally and deploy remotely. To this end I have imported my theme files into a subversion repository and I will set up a local installation of Roller on my development machine. This should make things a little easier.

The only outstanding issue that I have at the moment is that my hacked atom.vm template does not format paragraphs properly, which is a bit naff – all the text just flows together. I will have to fix this as I do intend to post the full text of each entry in my feeds.

One word of warning, turn off the Textile Formatter plugin when you are posting source code, as it really arses it up.

Friday Fun: Cool Illusion

Take some time out to confuse your eyes: stare at the black + in the center.

Hey, I got all the pink dots to disappear! More like this at: Visual Illusions · Optische Täuschungen.

Yeah I know, now I have a headache too…

Sudoku, Part Two

Solving Sudoku algorithmically is actually a bit harder than it looks - as a previous commenter pointed out. But I'm with Richard Feynman on this one: If you want to learn something, make a mess of it yourself first, then read up on the solution.

Let's restrict ourselves to 3x3 sudoku for the purposes of this series, with the numbers 1-9. My Perl program works the same way for 4x4 grids, so we're not losing anything. OK, each cell in the grid, and there are 9x9 of them, can contain the numbers 1-9. Now we are given certain cells, so we can immediately remove all other numbers in the given cells.

The given cells in turn mean that unknown cells can't contain given numbers. If we are given 1 in the top left corner, then the top left square cannot contain a 1 in any other cell, so we can remove 1 from the list of possibilities for the other unknown cells.

So this suggests are very simple approach - for each "solved" cell (a cell with only one possible number left), check all cells in the same horizontal, and all cells in the same vertical, and all cells in the same home square. If the number occurs in the list of possible numbers for a cell that we are checking, remove it. In this way we can reduce the list of possibilities in each unknown cell.

How does this help us? Well if we keep repeating this operation, we eventually reduce the list of possibilities in some cells to one number, and that means we have solved those cells. These cells then used to solve further cells and so on. At first glance you'd think that would be it. I sure did! In fact this simple algorithm did solve a number of 4x4 sudokus all by itself (I was not drawn out of the hat on those ones, and did not win anything...).

But it can't solve all sudokus. Sometimes the set of initial given numbers does not give enough information to exclude all possibilities, and further loops of the algoritm fail to produce any changes in the sudoku data structure. In keeping with this simple minded approach, see if you can think up anything to reduce the list of possibilities in each cell further. We'll talk about that in the next post.

Now for some code. The data structure that I used to store the sudoku puzzle is a two-dimensional array of associative arrays. The associative arrays don't actually associate anything - they are just bins for the list of possibilities. Initially they contain the pairs { 1=>1, 2=>2, 3=>3, ... }. This setup makes is easy to test if a given number is in a cell or not.

Here's the main loop. This just keeps running the number exclusion logic until no more changes are seen:

sub solve {
  my @cell = @{shift()};
  my $pass = 0;
  my $changes = 1;

  while( $changes ) {
    $pass++;
    $changes = 0;
    my $rowI = 0;
    for my $row ( @cell ) {
      my $colI = 0;
      for my $col (@{$row}) {
        my $origsize = keys(%{$col});
        if( 1 < $origsize ) {
          excludeHome(\@cell,$rowI,$colI);
          excludeVertical(\@cell,$rowI,$colI);
          excludeHorizontal(\@cell,$rowI,$colI);
        }
        my $size = keys(%{$col});
        $changes = $changes 
                   || ($origsize > $size);
        $colI++;
      }
      $rowI++;
    }
  }
  print "passes:$pass\n";
}

I have made no attempt to prettify this code - this is exactly how I hacked it up. Since it uses Perl references, here's a quick decompile: \@cell = reference to entire sudoku, @{$row} = list of cells for given row, %{$col} = number bin for a given cell.

Let's look at what happens inside the exclude subroutines. We'll use excludeVertical as an example.

sub excludeVertical {
  my @cell = @{shift()};
  my $prow = shift;
  my $pcol = shift;

  my $valbin = $cell[$prow][$pcol];
  if( 1 == keys(%{$valbin}) ) {
    return;
  }

  for( my $r = 0; $r < ($sqsize*$sqsize); $r++ ) {
    if( $r != $prow ) {
      my $testbin = $cell[$r][$pcol];
        if( 1 == keys(%{$testbin}) ) {
        my $remove = (keys(%{$testbin}))[0];
        if( 1 > keys(%{$valbin}) ) {
          delete(${$valbin}{ $remove });
        }
      }
    }
  }
}

We do a quick check to make sure that there's still more than one number possibility (if there is only one number left, then this cell is solved), and then we run through all the cells in the same vertical as our current cell (indicated by $prow, $pcol). $sqsize is 3 for a 3x3 grid, and 4 for a 4x4 grid, and so on. We also make sure to ignore the current cell ( if( $r != $prow ) ).

This subroutine inverts the informal algorithm described above. It searches for all cells in the vertical that contain only one number, and removes all such numbers from the current cell. This is equivalent to removing the current cell from all others if it only contains one number. The keys(%{$testbin}) expression tells you how many numbers are still in the bin to be tested. We delete the solved number from the current bin using the Perl delete function, which has the rather strange idiom of using the value of the key, rather than the key itself. Oh well, it's Perl.

I know, I know, the code is ugly and messy, but I was trying to win €150 as quickly as possible before I got better and had to go back to real work.

This post is part of a series on a Perl Suduko solver.

Steve Pavlina's Non-Objective Reality

Steve Pavlina points out that objective reality, well, ain't. Actually he's on even firmer ground than he thinks.

Now I'm no Sapir-Whorf fanboy, but as it turns out, classical objective reality is also a logical impossibility. Good old Hilary Putnam showed us that.
That doesn't mean there's no there there, just that we can't talk about it without an inevitable subjectivity creeping in. The disembodied AI mind has no experiential frame of reference for creating meaning, but we do, because we experience stuff like up and down, hot and cold, red and blue, in a way that cannot be modelled with set theory (which you kinda need for the objective reality viewpoint). And no, this doesn't mean I think Berkeley was right (now he was one wacky chappy).

In case you think I can think for myself (ha ha, fooled ya!), I'm getting all this from George Lakoff's Women, Fire and Dangerous Things. A bit of a tough read, but right on the money. The main point is that we can choose to make our meanings in the world, and they reflect back on us, determining how we can think. So yes, Steve is right, your beliefs really do create reality.

Really really.

Sudoku, Part One

A while back I was stuck in bed for a week or two due to illness, and after I got bored with mobile games (well your reactions aren't the best on pain killers), I decided to have a go at a few sudoku puzzles.

Actually, my interest in sudoku was just a little bit mercantile. The Irish Independent newspaper runs a weekly "Super Sudoku" competition, with a prize of €150. Problem is, the sudoku variant that they use is not your normal 3x3 grid, with the numbers 1-9. Ah no, if ye are going to be giving away the money, ya have got ta make it a wee bit of a co-nun-drum. "Super Sudoku" is a 4x4 grid with the numbers 0-9 and the letters A-F. Now how anyone can solve that manually I do not know - I can't even solve the 3x3 ones! But I know Perl can…

So I resolved to write a little perl script to tackle this important scientific research project. Nothing like a few associative arrays to solve a puzzle… more tomorrow.

Sudoku, Part two

Bloglines Strangeness

In order to test my hacked atom 1.0 feed, I created a bloglines account and subscribed to the feed. That was all fine, but then things got funky. Bloglines only lists the first two posts. Later posts just aren't there. It doesn't seem to be cache related, as the posts are now over 24 hours old.

I also subscribed to the RSS feed for my blog to check, and that was fine. Both feeds, atom and RSS work fine in FeedDemon. Very wierd. I'll have to register with a few more online aggregators to check this out further…stay tuned.

Dealing with Bugs

If you sell software and a customer discovers a bug in your code, it's not a nice feeling. In fact, you get a very nasty sinking feeling in your gut. And the first thing that your gut says is: "shh! don't tell anyone else!".

If you listen to your gut, you can avoid some temporary pain. But your gut is wrong this time. It's always better to disclose bugs to all your customers. Why? Because most of the time, you are doing them a big favour. Think about it, up till now, all other customers have been getting along just fine. The bug probably doesn't even affect them. Yet. But sooner or later it will and they need to know what their options are.

Imagine you're the customer, you've downloaded a software product, integrated it into your system, (development or otherwise), and now a new release comes along. If things are working fine, you'd rather not bother with the hassle of upgrading. In fact, if your system uses multiple external products and components, keeping up with upgrade cycles is a real pain. So you need to know whether the upgrade is really necessary. The only way to do that is to know what bugs it fixes. If the bug is bad for you, you know you need to sort it out. If not, you can wait till the next major release.

By providing full disclosure of all bugs, fixed and open, you give your customers a little bit of help to manage their own complex environments. If you keep the bugs secret, things tend to blow up nastily every so often.

So that's what I do at Ricebridge. All bugs are accounted for. It means that customers can tell at a glance which version fixes what. And they know right away if they need to upgrade or not.

Hacking atom.vm
Well my little hack of atom.vm to produce valid atom 1.0 output was buggy: I got caught out by the atom content element and the existing showEntryDescription($entry) macro (in weblog.vm). This escapes the content which is not at all the atom way. Instead you have to declare a HMTL namespace. A quick trip over to the source of Tim Bray's atom feed shows how to do it. So I replaced the entry content code with this:

<content type="xhtml">
<div xmlns="http://www.w3.org/1999/xhtml">
$entry.text
</div>
</content>

Now everthing validates in feedvalidator. Of course, all this messing around makes me a moron. This sort of hacking is actually a really bad idea and pretty fragile, so bring on roller 1.3.

Velocity Templates

I've never really used Velocity templates before.
Sure I checked them out, but I have written my own template engine, Jostraca, so I've never seen the need to use Velocity.

Well, Velocity templates are really pretty easy to pick up. One thing that I did find was that they are rather messy. Now, I know, JSPs and PHP are even messier. In fact, it's all rather ugly. That's one of the reasons I added a general replacement function to Jostraca, so that the template code would look natural, and not much like a template at all - you can code things like: FirstName LastName instead of <%=firstName%> <%=lastName%>. It's not so important for HTML, but it makes source code templates a lot easier to read. But hey, Velocity's not so bad! (Maybe I'll add Jostraca support to Roller, [evil laugh]).

I decided to create my own theme, instead of customising via the web interface. For a start, I get to edit the files in a normal editor, and secondly, it seems like the best way to have a more permanent record of the design. Anyway, it seems to work for the moment – you just hit F5 to reload on the theme preview page and you can check your changes right away. For content however (such as the text of the about page) I am sticking with the web interface, and thus storing that in the database.

I've started with Roller 1.2, which does not have Atom 1.0 support. So I went over to the Roller source and grabbed the latest version of the atom.vm flavor and used that after a bit of tweaking. I'm hoping it will all be OK once I upgrade to 2.0