Saturday, September 09, 2006

51,000+ Pro Games Amidst Organized Chaos

Thanks to the substantial intellectual efforts of a very talented and helpful person - Jean-Pierre Vesinet, I am able to add a thousand new pro-pro games and almost two thousand pro-ama games and/or handicap games between pros.

Heaven permitting, tomorrow, Moyo Go Studio will be the first Go software that breaks the 50,000 pro-game barrier with more than 51,000 games where at least one player is a pro!

We have spent quite some time (several days) ensuring the quality of the games and each and every one is unique and has all important data. I want to remind again that the Chinese, Korean and Japanese player's names can only be displayed when support for East-Asian languages has been enabled in Windows as explained in the Help file.

I added quite a few robustness filters to the game import wizard, to ensure that games with missing moves will not end up in the database. Other improvements include a "wrong-date-format-normalizer". Infinately many wrong ways to specify a date in SGF are used, and Moyo Go needs to know the date to be able to calculate the average playing date for fuseki, for example. So it is important that every crazy date format is recognized and normalized. One Chinese Go server uses outrageously wrong SGF and I had to make a converter for that as well, but Jean-Pierre did the real detective work. Things are still not perfect with Moyo Go's pro database, but it is bigger and better than any other pro database in existence, and it keeps getting better, thanks to half a dozen hard-core contributors. And it's not just size that matters - it's what you do with it. Moyo Go does things with its game databases that no other Go software can repeat, whether complex SQL queries or Statistical Move Likelihood queries (have you seen the new Ctrl + E clipboard feature for that? The next update will improve upon it - I forgot to include the SML percentage).

Building the new databases came as an interrupt on an interrupt (or a subroutine in a subroutine), because originally, I was working on the TsumeGo module, which became interupted by a publishing module that I want to have ready in early october (at least a "technology preview"). The reason I started a printing module "in between" has something to do with something that still is a secret :-)

Running a μISV is challenging stuff - you have to be able to manage interruptions within interruptions within interruptions. Of course one shouldn't let this get out of control, but bugfixing and support issues sometimes, or even often, have a higher priority than anything else. The bigger Moyo Go gets, the higher the demands on program architecture, overall stability, Help file etc.

To give an example of what goes on behind the scenes and what's involved in producing shrinkwrap software: I got a report of the selection in result lists not being properly visible under a certain XP theme. It turned out that several of the components that I use rather follow the theme's selection colors, and although one could argue that the user should be responsible for the color schemes he/she selected, it appears that the user typically does not realize that XP themes are more than just another color window. So I went deep into the source of a handful components and made sure they always look as I intended them to look.

Another issue was with a reported slowdown during Fuseki matching. My computer is a mega-monster with 4 GB RAM, four 64-bit Opteron cores and dual 15,000 RPM SCSI harddisks*, so (you feel it coming), I do not have many slowdown problems :-)

Yet, it would be too easy to dismiss those reports with "I do not see your problem here". Me being a diver and all, I went to the bottom of the problem (I hope it wasn't a false bottom) and decided it was caused by memory fragmentation. The code pointed to serious fragmentation, so - without going into the gory details - I fixed that for the next update.

Another support issue was about the precedence of the and and or operator in a database search, and how to get a list of games between certain players. That proved to be a non-intuitive one, but possible due to the power of the SQL engine I use. I found a way to do it and updated the Help file.

Norway still has a few nice late-summer days left and after that and the upcoming updates, it's back to the publishing (printing) module!

*Will be obsolete junk before I have gray hair