Saturday, January 14, 2006

Fuseki Module: Status Report

I discovered a few wrong bits in all new database updates (the ones containing Fuseki data), so I removed them from my site. My apologies to people who downloaded them already. Because they contain some new sorting fields they will ultimately all have to be redone and re-uploaded, but that will not affect the Fuseki module.

I had to remove them (temporarily) anyway, because I was getting rather scary bandwidth bills of 22,000 USD (even though they were a mistake, I prefer not to exceed the monthly maximum).

A much better solution would be to build the new databases on the user's machine, if that would not take too much processing time. This appears to be feasible. Extracting the Fuseki is very fast, but sorting the index files could need some more work (can be twice as fast with a better sorting algorithm). So the next update will build its own Fuseki index files for 410,000 games, without the need to download new databases!

The main part of the Fuseki module is working: It displays a list of matching games with average rank & winning percentage. I designed the system to take almost no time for the search. Which means that when you have a position with a black stone on a 4-4 point, there will be a list of a few hundred thousand games, and you won't notice that this took any time, on a reasonably modern PC. Now I just have to do the boring stuff like column-size/position persistence, column sorting, finishing & debugging of the GUI integration etc. And of course the part that creates the Fuseki index files on the user's PC.

But I also want it to display all variations and their winning percentage and average player rank, so that will take quite some work. I might do that at a slightly later stage, as an additional auto-update.

I want to have it well-rounded and tested rigorously, so please bear with me.