Multiprocessing

Due to a technicality, the past two days of number crunching for the Fuseki module have to be redone. I decided to give multiprocessing a go: Four threads (each CPU its own) will load SGF files, process them and write the results to a database. This should increase speed about fourfold. And if I manage to make this in a day, the gain will be half a day.

However I don't think it's realistic to do this in a day - Multithreaded number crunching is non-trivial for a beginner in these matters like myself. It will become increasingly important to do so: More and more customers will expect that their multi-core CPU's be put to good use. The goal is to process SGF (add to database, including index generation for pattern recognition) with all available processors, even when there are dozens, as will be the case ten years from now.

"Ethics" Witchhunts in Comp. Go also in Cloning, Cold Fusion

Hwang Woo-Suk made impressive progress in cloning, something that will make Big Pharma loose Big Money, because when you can cure diseases by injecting the affected organ with cloned stem cells, many cash-cow symptom relievers become obsolete.

Upon the breaking of the news, some corporations and competing scientists launched an all-out attack: They said his research was unethical because one of his colleagues (voluntarily!) donated eggs for the experiments (for which she received proper credit). Nevertheless, that colleague was influenced into doing a 180-degree turn and quit her job in South Korea and returned to her native country, the United States.

Not just spokespeople of Big Pharma and the Religious Right shouted "unethical": Mr. Hwang Woo-Suk's scientific competitors were most vociferous as well. No wonder, with their potential Nobel Prizes - and funding - at stake. Nobody cares that Mr. Hwang Woo-Suk is a veterinarian, and that veterinarians are not instructed on the PC notion that donating an egg for common fame and the progress of science is "unethical".

Meanwhile, press releases of dubious origin, shamelessly promulgated by Western media, spreaded FUD about his research with only flimsy circumstantial evidence of any wrongdoing, let alone that there was anything wrong with the cultures themselves. Hwang Woo-Suk, after allegedly attempting suicide due to being accused of having besmirched the honor of his nation, was recently released from hospital. In spite of thoroughly peer-reviewed publications in Nature and other respected scientific media, allegations of "fabricated results" persist. Meanwhile, independent tests on his cloned puppy "Snuppy" confirmed that indeed he had succeeded to clone the world's hardest-to-clone dog breed.

Not the slightest evidence remains to suggest that he ever fabricated results or acted unethically, but the harm to his reputation has been done. He denied the allegations of fraud, and said his basic research was sound. Dr. Hwang said his research team made 11 patient-specific embryonic stem cells, and that they still have the technology to produce them. Unfortunately, while he was in hospital, they closed his lab and contaminated some of his cultures, making it hard to prove his innocence.

1-0 for Big Pharma, the Established Order and the Powers That Be. However, Hwang Woo-Suk, after release from hospital, vowed to fight back via the courts. Let's hope the Rifkin-Kristol petition won't pass: It would make cloning human cells a criminal offense - punishable by up to 10 years in prison - effectively denying us to modify our body at the genetical or cellular level.

Massive FUD campaigns lauched by multinational corporations or even government institutions are nothing new. Remember the cold fusion "hoax"? For the past two decades, they made us think that cold fusion was a lie and the discoverers (Martin Fleischmann and Stanley Pons) were publicly disgraced. But nowadays cold fusion has become so reproducable that some US universities hand out cold fusion kits to students, to prove, with a calorimeter, that the phenomenon of cold fusion indeed exists.

Most scientists who discover something important see their work - at least initially - being viciously discredited, and the most interesting bona-fide inventions are getting "exposed" as "hoaxes". Examples that come to mind are the increased half time of Tritium in a Titanium crystal matrix (my theory is that this works similarly as why cold fusion in a metal matrix works, by "conduction" of neutrons from one nucleus to the other) , the ballotechnic material commonly referred to as "Red Mercury" and the discovery that 95% of stomach ulcers are caused by Heliobacter Pylorii. Interesting aside: Due to the taboo on antibiotics in the West, it took ages and courage for this to be discovered. Antibiotics are my favorite interest. Recently, two dozen Norwegians died from Legionaire's Disease. An ID specialist told me that Clarithromycin and Azithromycin are "not used" here. Sadly, those are exactly the most effective against Legionella.. I can go on: Amongst the real experts, there is no more doubt that ME, MS and Fibromyalgia are of bacteriological etiology (spiroplasma / mycoplasma), but the PTB want to hear nothing of it. Of course not, because cheap generic antibiotics work at least as well as costly, patented symptom relievers.

Relativity Theory of Ethics

Some folks have a peculiar sense of ethics. They think that against someone who is in their opinion unethical, any retaliation is permitted, including an unethical one.

A not-to-be-mentioned Austrian thought I was unethical, so he violated my Copyright and the rules of ethics by putting a private letter to him online, and by removing my postings and people who posted in my defence from a public forum. A not-to-be-mentioned German thought I was unethical, so he violated Moyo Go's EULA and put up his copy of Moyo Go for sale (no better than software piracy, as I can't verify that he deletes his own copy and since he never notified me of his EULA violation, why would I trust him?).

Interestingly, the people who shout hardest about ethics, are the ones who are the sneakiest in their attacks. I haven't had a single official complaint about anything I have ever done, but in the War Against Bad Ethics, anything goes. DOS attacks in the form of bogus complaints to my ISP or web provider, backroom conspiracies to deny me a retail channel or a review platform, anything it takes to prevent "unethicalness" in the Go community.

Claiming ownership of those pro games is like the bedouins who forbade me to take people to Bir Swair ("small spring"), the semi-secret oasis. They wanted a piece of the action, even though they were too lazy to take folks there themselves. "It's our desert", they said.

Genetic Evolution

While slowly progressing on the new Fuseki module (due to deteriorating health and the Christmas period), I have firmly decided to go for "genetics" for the future Go playing part of Moyo Go Studio. This is not an impulsive decision - four years ago I already publicly mulled "GenetiGo" as a possible name for my future Go program.

My girlfriend went cross-country skiing, and I am cooking beans and reindeer stew.

Interestingly, it is due to my health problems that Moyo Go exists at all, because otherwise I would still have been a freelance Divemaster/desert guide here:



Nothing focuses the mind better than being constantly aware of one's mortality.

This picture is where I have lived for two years, one year most literally, when I got permission to build a hut on a beach "owned" by a Tarabin bedouin family. My water came from their basin, filled every few months by a truck. I washed myself in the sea and walked barefoot for a year until the callus on the soles resembled hooves.

Usually when I get "firm resolves", I am so convinced that it will work that my first thought is: "Keep it secret", but I will blog about progress on this. I have a pretty good idea of how things are going to work, and I'm glad I have secured ownership of that dual-CPU dual-core 275, even though I recently resigned and I did not get it as cheaply as I had planned. The beast will be the key to efficient genetic evolution of a Go program. I'll let each CPU pit a (64-bit) evolving version against the other.

Later I'll write about the general workings of such a system, because it's always nice to get some feedback by email.

The (S)AGA Continues

AGA has kicked me off their mailing list, but someone brought this (from the AGA E-Journal) to my attention:

"HALF-PRICE SALE ON MASTERGO: The creator of MasterGo is offering a special half-price New Year sale: $50 instead of the usual $100 cost. The software for studying joseki and fuseki includes tens of thousands of professional games. Available at www.slateandshell.com for delivery in early January."

Hm.

Wasn't Slate & Shell a spin-off of AGA's board members? And isn't MasterGo a competitor of Moyo Go Studio? Hasn't Moyo Go Studio been refused a review in the AGA journal? Isn't this a "conflict of interest"?

Who's who on IGS

Alexandre Dinerchtein at Go4Go.net knows the real names of the highest ranking players on IGS, and many are pros. Since Moyo Go includes those IGS games, I decided to make an import filter in which you can specify the real name for an alias, and re-import them.

Tomorrow I hope to make the "reprocessed" IGS games available for download, and the auto-update with the import filter wil come soon.

Yet another new website

As you see, the new site looks very different. Nothing much works yet, except the blog and the auto-update system. The site is both registered and hosted in Russia, and yet the downloads will be very fast indeed. I had to change hosting providers twice in ten days so yet again the Forum has to be re-installed, all updates uploaded etc.

Heaven permitting, there will be a major Moyo go update out in a week or so, which will have 40-move deep Fuseki matching and -statistics for all 410,000 games in Moyo Go's databases.
Half of the data has been gathered and is already uploading as database updates, now it's just the quarter of a million CyberKiwon games that are being reprocessed.

I still have to code the Fuseki matching system after the CyberKiwon games have been processed, so all in all it will take about a week. My girlfriend will visit me from Czechia this Thursday (she'll stay over for Xmas) and I have to clean this place up, so it will be a hectic week.

Full Out

OK, the "all games" window is done and so far no bug reports. Now I'm working on Fuseki-search. Moyo Go already has Fuseki search, but only in the context of the "pattern urgency" system. Many less-common Fuseki simply isn't "statistically urgent" enough to be in Moyo Go's pattern database so what I'm making (for starters) is a 40-move Fuseki database as Kombilo implements it: Utilizing hashes for fast searching.

See here what happens when you add thirty fields and thirty indices to seven SQL databases simultaneously on a machine with 4 CPU's and 4 GB RAM.

I hope some of the databases will be restructured by tomorrow afternoon ..

Need For Speed

Some progress on the "All games" list. What I wanted was:

- No delay in loading the game list (SmartGo needs three seconds for it, quite acceptable but I prefer that things do not take any time),

- Be able to sort on player's names, ranks, date etc. without any delay. (it takes SmartGo a bit over a second). Without any delay with super-huge game collections as well.

It looks like I succeeded :)
Well, not that hard, with a full-fledged SQL database engine beneath the surface..
Moyo Go looks simple on the outside but what makes it tick is approximately 0.95 MLOC (million lines of code).

Now it's just to do the finishing touches and get this baby out the door!

Most popular at Sensei's:

Go in the Army

A Korean customer told me that if a Korean hasn't learned to play Go before age 18, he would learn it in the army because there it's the favorite passtime.. In the Netherlands, it's drinking beer, playing billiards and watching porn movies!

This is what I mean when I say "I am starting to fall in love with Korean culture".

The AGA Strikes Back

The battle Between The Go Software Vendors heats up: AGA is ratcheting up their FUD campaign:

..highly distrustful of running any of his code on my computer," writes Andrew Jackson. "If he has no problems doing unethical things that are legal, who knows what else his code might do?

It seems that the Go software millionaires are not afraid I will install a virus onto their computer. Without naming any names, the authors of the top-three bestselling Go playing programs all own a copy of Moyo Go Studio! One of those colleagues ordered his copy today, a few minutes after the AGA published their latest smear article on me.

PS AGA tacitly admitted that their first article was intended to deny my product sales: "Bad press can be just as good as good press, I think it would have been best to do like everyone else and just keep mum on the issue."

Too late. And I have to go to the postoffice now.

Open-Sourcing the data

Open-sourcing the pro games I collected from various sources and "cleaned up" a little (pro ranks should have a "p", for example, to be able to automatically distinguish them from "d" amateur ranks) was not just beneficial for computer Go researchers in countries where the GoGoD CD costs the equivalent of a week's salary. Or students of the game of Go in those countries, for that matter. (My POV is that not only is there no Copyright on Go game records, it is also unethical to claim Copyright on them in an attempt to make money of them greatly exceeding the cost of producing the actual collection).

Immediately after I put the games online, people started to report bugs in the SGF, as well as quite a number of duplicates. There turned out to be a bug in my "duplicates detection" code, when games were 180 degrees rotated! Strangely enough, these games exist. Many of them.
My duplicates-detection code works with rotational/mirror/color reversed end-of-game hashes, but there was a bug there.

The "illegal SGF" never became known to me either, because Moyo Go follows the principle of "Be liberal in what you accept as input and conservative in what you produce as output". I should have tested with other Go software as well!

So now I have put online a new pro collection, which is now pretty OK.

Moyo Go's auto-update system

Sometimes, being paranoid is just having all the facts. For example. Moyo Go downloads its own updates automaticaly, when enabled. This is done by FTP. In order for FTP to work, there needs to be an FTP account with an FTP adress, username and password.

And let's say I hard-code the FTP address into Moyo Go. For example, the FTP address for my old server was ftp08.powweb.com. This works, until ftp08.powweb.com does not exist any more (you never know with those Weapons of Mass Destruction of the Axis of Evil out there). In fact, no weapons of mass destruction are neccessary to deny my customers automatic updates if I would have hard-coded the update FTP address into Moyo Go. A frivolous Copyright infriction claim will do, when the server is located in that bastion of Freedom, the US of A.

Therefore, I spent considerable effort into designing MoyoGo's automatic update system. Without exaggeration, this update system is so advanced that I could sell it instead of Moyo Go Studio, and I would probably make more money out of it too.

To make a long story short: I have been kicked off the powweb server, and MoyoGo's automatic updates work just fine again.

Sore Losers

I was quietly minding my own business yesterday, when suddenly the lights went out for Moyo Go. They pulled the plug on it, so to say.

Damn! So I Skyped my spineless hosting provider PowWeb in the good ol' US of A, and hit a brick wall. They told me that someone had told them I am a criminal, and for the rest they had no comment. They did not want to connect me to anyone else, neither did they return my emails.

Some helpful folks on the Internet told me that this happens all the time. In the US, any competitor or random person can make a frivolous Copyright infriction claim and Bob's not your uncle any more.

Boy oh boy.. Imagine you're having some kind of eCommerce business hosted in the US!

I abhor things that break easily so I decided to move not only my hosting provider somewhere else, but the domain registration as well. You never know.

I am now hosted by a sweet little hosting company in the idyllic little country of Russia. Yes, the one with the nukes. And they told me that people like Anders Kierulf and John Fairbairn and Roy Laird would have to come to Moscow and win a courtcase, if they wanted to fuck me over.

We really talked this through thoroughly. The folks in Moscow told me I was going to be hosted in Moscow instead of on one of their US-based servers. When physically hosted in Russia, the rug could not be yanked from underneath me any more.

Still going strong

A few days after the AGA scandal, and I still can barely handle the orders. I ran out of stamps and I had to order 100 more bubble envelopes. A lot of good has come out of this. Not just the pro games - I will make a free trial version of Moyo Go as well.

Someone wrote me that the AGA has been suspected of fraud.

Alledgedly, they have been misappropriating substantial parts of their annual 100,000 USD Ing foundation grant to fund their own private Go paraphernalia company. Meanwhile, the AGA would endorse exclusively the products of their own company. More about this here.

PS

I just hear that the Ing foundation had already cut their funding to AGA before I wrote this article.

Jewel case cover

The first 50-odd customers did not get a printed paper jewel case cover, because there wasn't one yet :)

Mail me, and I'll send you one!

Win - Win

Yesterday, AGA published a nasty bunch of lies about my software. Or, to be exact, not about my software per-sé - it was a FUD article. Fear, uncertainty, doubt. They said that nobody in the cyber-Go world was interested in my software, and that in the eyes of some of my competitors, I am a criminal. The situation is that the GoGoD dudes are friends with Roy Laird, an AGA board member. AGA has refused a Moyo Go review in the past, with the remark that GoGoD objects to using games from their CD, and that GoGoD thinks this is "criminal".

Of course, this is completely legal according to the Berne Convention on Copyright, and it became even ethical, after John Fairbairn of GoGoD said a few times in public (before I used the games on the GoGoD CD) that my software was not only very lousy, he also was afraid it would mess up his computer, that he lost all trust in my software etc. etc. I guess he was pissed off about my refusal to give my software to him, to be sold as part of his GoGoD CD.

I am not well-versed in the politics of the Go world, but I understand that the folks I am the biggest competitor to (e.g. Gobase and GoGod) won't link to me - fair enough - even though GoBase links to every other Go software - giving the impression that Moyo Go doesn't exist. Power-politics of asking distributors not to sell my software is illegal. Colluding with the Go-media to libel and slander a competitor is illegal. And Arno Hollosi's and Morten Pahle's decision of banning me from advertizing, taking the position as judges as to who is right and who is wrong in the dispute between myself and Go game record "appropriators" and punishing me is totally uncalled for. I had half a dozen online retailers eagerly promising to do business with me, Janice Kim even promised to purchase an initial minimum of 30 copies, after reviewing Moyo Go Studio. GoGoD's criminal operations of trying to dominate and monopolize the Go game record market and all software that uses it remind me of a case in Norway recently. A large diary company offered a supermarket a better price if they would stop selling a smaller competitors' products. This backfired in a spectacular way - their sales dropped overnight by 20% and the smaller one's doubled. And a police investigation was launched. GoGoD has told their distributors that I am a criminal (of course, only in their opinion), and that nobody should sell my products or link to me, and they have secured the cooperation of journaille like AGA to "spreak the word".

The FUD GoGoD spread amongst Go software retailers has proven very effective, which presumably has less to do with the "ethical" or "legal" side, but much more with the fact that the GoGoD CD is a hot seller, and that, when given the choice, retailers preferred to sell GoGoD than Moyo Go.

I feared that the AGA article - a vile pack of lies if there ever was one, several people have lodged official complaints with them about the article - would affect sales in a negative manner. Well.. I sold four copies yesterday! That is not "normal" by a long shot. Mr. Laird: The picture above is one I took just now, before going to the post office. On a very good day nowadays, I sell two copies a day. The average is about two or three a week.

I am grateful for the trust put in me by my new customers and I will do my best not to shame it.

Moyo Go's own Sensei's page :)

After four years of work on Moyo Go Studio, I thought it was time to give it its own Sensei's Library page.

It's a Wiki page, so if you want you can say something nice about Moyo Go in the User Comments :)

40,000 pro games!

One of the reasons I wanted to have Unicode capability was the fact that I recently found 4,000 professional games not yet in Moyo Go Studio's collection. But the players' names are in Chinese.

Moyo Go's next update (in 24 hours, heaven permitting) will be capable of displaying the CJK languages (when enabled in the Windows setup). Tomorrow I'll make a new pro database update available here which includes those four thousand games. Later, there might be a function available that transliterates Chinese, Korean and Japanese names to their corresponding phoneticized equivalents. I am working on more advanced database- adding filters, like keeping either the phonetic or the native names.

Moyo Go Studio now has by a very wide margin the most pro games on the market.

Tygem, Cyberoro & IGS

On of my customers (his name is Neil Mclean) made importers for the games for the most popular Go servers. I can't thank him enough for that.

I hope he doesn't give his sources to the "competition" ;-D

The only thing left to do for me was to figure out which encoding is used, and to display this correctly in Moyo Go Studio. Behold the result!

Neil also downloaded and donated 21,000 of the strongest Tygem games, so you can imagine how happy I am.

You Can Only Think Outside The Box When You Know What Lies Outside That Box

Sometimes, the seemingly most logical solution is not the best. I can't think of a software-related example right now but listen to this:

I use a cordless mouse and recently the signal became too weak for it to work properly. My PC stands behind the door in the hallway, so I moved the receiver as much forward as its cables would allow (about a metre), and that helped. The big drawback to that solution was that the thing obstructed the door to the bathroom, and it would be only a matter of time before I would crush it.

So, I moved it half a metre backwards again. But that caused the reception problem to re-appear. The only other option, it seemed, was to change the angle of the thing, so that the radio signals might arrive more advantageously. I know some stuff about antennea and RF signals and I thought it was worth a try. Unfortunately, if didn't help much.

And now we arrive at the "thinking outside the box". The changing of the angle was already a little bit of TOTB, but the real TOTB was when I, in spite of the fact that the empirical facts supported the premisse that "further away equals worse signal", started to remember some more stuff about RF signals. One of the things I remembered was that they have a rather undesirable tendency to inject themselves into other wires. This is why the FDC requires shielding on just about every electronic applicance that might produce some RF (Radiofrequent) signals, so that it can't interfere with other applicances.

Behind my PC (and as far away from my desk as can be) is a bundle of cables that lead all the way to my desk. I thought that if I would place the mouse's receiver on top of that bundle, it would be in an advantageous position to pick up some of the RF energy absorbed by the cables. My theory was that the mouse would radiate some of its energy into the cables near my desk, that the cables would carry this signal to the coil behind the PC, and that putting the receiver inside that coil would allow the receiver to pick up enough signal.

This indeed worked. Instead of placing the transmitter a metre forward, I moved it quite a bit further away (and behind a wall as well) and it solved all problems perfectly. The device wasn't even visible any more.

Of course anyone could have arrived at the same result simply by "exhaustive search" (trial and error) but often a programmer doesn't try a certain thing because it simply doesn't occur to him. It's not that you have all the time in the world to try everything, usually you design something and then implement it.

My point is that in order to achieve true excellence (= solving the problems best), you have to think outside the box, and this cliché can only be fulfilled if one actually knows what's going on outside that box.

Quest for Speed

This is how my CPU utilization graph looks when doing a time-consuming database query. CPU usage is 30% on a 4-CPU machine, meaning that there is some multithreading occuring.

Corrupted Custom Cursor Conundrum

It's a pity I don't have champagne, because two nasty bugs have been squashed.

Moyo Go uses custom cursors to show what you're doing. When annotating board labels with green letters for example, and you have just put an "A" and a "B" on the board, the cursor is a little hand with a green "C" underneath an outstretched finger, because the next time you click the board, a green "C" will appear.

This is of course highly cool.

For some weird reason however, the cursor, at startup, looked "corrupted". Sometimes. It became partially a fuchsia color, or it got a photo-negative kind of appearance. I looked at my code and it seemed OK.

Little did I know that Delphi's Bitmap Canvas objects were not threadsafe, and that right in the middle of a loop that made the custom cursor's transparency mask, its handle would become invalid!

I solved it by putting a "canvas lock" around the loop.

x64

I can now confirm that Moyo Go Studio works on 64-bit Windows™. I was a little nervous about that - quite a few well-known software packages don't yet. Another thing I can confirm is that MoyoGo works on multi-CPU machines. Also this is a non-trivial thing for multithreaded applications, programs that use multiple threads of execution can work fine on a single CPU but if no care is taken by the programmer, they might "hang" when multiple CPU's are present.

Yet another thing that has been verified to work is running on a machine with more than 3 GB of memory. The only thing I had to do was to get rid of the "Moyo Go needs at least 360 MB to work comfortably" warning, because for some reason having 4 GB of RAM made it think it had almost no RAM to work with. Google Earth has yet to fix this: When I tried to install it it told me that my computer did not have enough memory or a powerful enough CPU, and it refused to install. I suspect it is not an API issue, but due to an assymmetric allocation of NUMA resources.

Kimchi

I am falling in love with Korean culture.

I watched two Korean films recently, Park Chan-Wook's "Old Boy" (or "Oldboy") and Kim Ki-Duk's "Spring, Summer, Autumn, Winter and Spring". These films were awesome.

So I decided to buy a kilo of kimchi (-: Oslo recently got its first Korean toko/restaurant. I'm afraid the peasants here won't give it the time of day though.

Kimchi has been shown in peer-reviewed medical literature to help cure H5N1, Avian Influenza. Nobody knows why. It's "Sauerkraut++", spiced-up fermented cabbage. Lots of good Acidophillus bacteria, and I need those because I have been taking incredibly high doses of antibiotics since the previous millenium. I have a chronic infection in my brain (neuroborreliosis) and anything I can do to ameliorate the bad effects of the antibiotics - especially when it tastes as good as kimchi or natto - is welcome.

Wrong SGF

What's wrong in this picture? (from a commented game by Cornel - no idea which SGF editor was guilty of this). Every time I think I have seen it all, something new comes up. I was wondering why my "automatic branch annotation" put five letters on top of eachother. Must be a bug, right?

Nope, there are in fact five branches with the same starting move! (Q12). This is a perversion of the whole concept of branching - and most likely even illegal SGF, a pity there are SGF editors out there that allow this sort of thing.

Moyo Go Studio gracefully handles bad SGF. It never gives an error when it reads stuff that's invalid SGF, as long as it's unambiguous and renderable. But this is the weirdest thing I've seen in a while.. Of course, those five branches should start on move Q12, not M3! Not Cornel's fault by a long shot - it's us SGF editor writers that have to shape up.

Almost Famous

The idea-thieving scum at Microsoft managed to get some major press coverage on my work:

http://www.theregister.co.uk/2005/07/04/silicon_chess/

*Silicon beats carbon in chess battle*

By Lucy Sherriff
Published Monday 4th July 2005 14:16 GMT

.."Computers have yet to master the game of Go, a much trickier proposition with its branching factor of 250. Brute force is no good here, because the number of moves gets really big, really fast. Researchers at Microsoft's labs in Cambridge have made some improvements to computer Go, using Bayesian ranking to calculate probable good moves. But the researchers concede that even the best programs are pretty easily dispatched by good human players."..

Unconventional Silence

As you can see, after I bought that super-monster PC, there was no money left for paint. :)

I didn't manage to make it silent, it seems the case fans produce a lot of noise as well. So I made a "cable snake", and put the PC next to the door, in the corridor of my microscopic 1-room apartment. This works (especially with the door closed), but the mouse- and keyboard cables didn't work. They simply are too long. Weird. So I bought a wireless keyboard/mouse combo and now I can work in relative quietness!

Longhauling

Interesting. I had a big spike in sales for a week, and now I have no sales whatsoever for almost a week. I am pretty sure why: Every new customer found one bug and two new features they would like to see implemented, and kindly informed me so on the forum.

New customers of course look at the forum to see how existing customers like Moyo Go, and see a lot of: "BUG found!" and "We really desperately need feature XYZ". So they think: "Why not wait a little until the bugs are ironed out and feature XYZ is implemented".

Hm, in case of Moyo Go, this is a bad idea. Because customers get lifetime free updates, which means you're loosing nothing when you buy now. But as soon as I have straightened out major bugs and implemented major features, I'll raise the price. Master Go, for example, can't do what Moyo Go does, but it costs 100 USD. This means that Moyo Go will most likely be doubled in price, over the coming years.

So it's expensive to wait.

Of course it's not that bad with bugs and lacking features, it's just the fact that because I offer free updates and I quickly fix bugs and implement new features, that people "avalanche" me with them :) Blake Haber asked for a much more powerful SQL database search and he got it 24 hours later. Of course this encourages to request more :) It doesn't mean that Moyo Go is a stinking pile of crap, on the contrary.

I do not care that I am not selling anything, for the moment. Because things go in waves. All the stuff people talked about will be fixed/implemented, they will rave about it, there will be a sales wave again, they'll find bugs in the new functionality and sales will slump again, etc. etc. until Moyo Go Studio has achieved Full World Domination. While all the time being under-priced, when you look at MoyoGo's commercial counterparts.

Bug movie

Some of my customers are geniusses. Adrian Petrescu is one.

Today I received a movie of a bug in Moyo Go, made with the trial version of Camtasia.

"It's easier to show you the bug this way", he said. Awesome idea. Of course there is nothing better than a bug report where you see the stuff go wrong, and hear the customer exclaim: "See? It's a bug!.."

(Well, I just have to guess for the moment because the movie was in Flash format, and I make a point of uninstalling Flash as soon as Windoze is installed. You wouldn't believe the tranquility of the browsing experience w/o Flash and with pictures set to "do not animate" (Mozilla has that option). Whenever I look at other's browsers I have to suppress an epileptic seizure.

Update: I can watch the movie now because Adrian made an AVI version and I am starting to think that I should give priority to bug reports that come on video :) A picture is worth a thousand words but a video outperforms that by yet again a few orders of magnitude :)

Fake Virus

As soon as I designed the blazingly fast index system for MoyoGo's instantaneous pattern search, I thought it would only be a matter of time before faux virus-alerts would be generated. It has remained silent surprisingly long, but my fear has become reality:

A dodgy virus checker ("F-Secure") thinks that one file might have a virus, namely:

moyo go studio\databases\nngs\3053.dat
Infection: Possibly a destructive program [F-PROT]

..this is nonsense, and I will take it up with F-Secure. I wrote "dodgy", because I was amazed that this company does not have an email address. Instead, they require me to fill out a dozen fields, and I'll end up having no copy of the correspondence, because a bug in Mozilla causes such fields to be empty, when the page is printed. Their support number in Norway is dead, on Sundays. I guess people don't use their computer on weekends. The bigger the company, the worse the service. My phone number is +47-98680219 and any customer can call me, any day of the week.

It is possible that even less-dodgy virus checkers will find "virusses" in other files. The reason is simple: Moyo Go installs twenty thousand files with semi-random (encoded and compressed) data. Virus checkers work with "signatures", which are byte sequences. You can imagine that 1.5 GB of semi-random byte sequences has quite a chance of having, just by coincidence, one of those "signatures" somewhere.

Anti-virus software makers could avoid this problem, but this would penalize their scanning speed and the ease of adding virus definitions.

Automatic Updates Now With Safety Belt

In a post of a few days ago I mentioned the potential risk of automatic updates: You might distribute something that doesn't work any more, irritating customers, which is A Very Bad Thing™.

Now, before an auto-update is installed, everything it modifies is backed up, allowing the user to revert to the previous version with a click of the mouse. This should not just make the customer relaxed about installing updates, it gives me peace of mind as well.

Why Moyo Go can't be Translated Right Now

I get offers to translate Moyo Go Studio into Spanish, Portugese, Chinese, German, Dutch, French and so on.

And I would like to take people up on those offers, of course.
But not now.

The thing is: I am still adding so much new stuff to Moyo Go, that I would need a few new translation strings all the time. The delay to release an update could run into weeks. A chain is only as strong as its weakest link - if the person who translates into Xhosa happens to be on a hunting expedition, releasing an update would have to be delayed until he came back (if not eaten by a lion).

The power of SQL

I'm so glad I chose an SQL solution to manage MoyoGo's games!

Many cool things can be done. For example, whenever Moyo Go, during game importing, sees a player without a rank, it simply asks its database whether it already knows the rank of that player, and the player's rank will be automagically assigned.

An Example of an Insane GUI

In spite of me pushing forty, I just pulled an all-nighter. At 08:15 my phone's alarm (the phone is the cheapest Sony Ericsson) went off. When I pushed a button, it went on bleeping louder and louder, but asked: "Turn off alarm?".

Like, if you get woken up by the thing, manage to half-drunkenly grab it and press its "Yes" button, that you would possibly want it to go on waking you up..

Something Irrational

I just bought SmartGo v. 1.5.4 and this is what I found at hex 1901B3:


tsk, tsk..

Spanish Ad

This is going to be an ad in the Spanish Go Journal. With some help, I finally got it right.

Gossip

My biggest (potential) competitor, Anders Kierulf, is a really nice guy. The first thing he did when I announced my entry in the computer Go field, was sending me his thesis on computer Go. He's probably a lot brighter than I am, too. When I just started with my old SGF Reader, he offered me advice and submitted bug reports. Yesterday I was having some trouble implementing something that SmartGo has as well, and when I finally managed, I thought: "Let's blog about this". I wanted to mention his employment history at Microsoft (arguing that he might hold an advantage over me, because the particular problem I wrestled with was a typical "Microsoft Registry Hell" thing.) So I did a search on his name.

Then I read this here:

In and around California's Silicon Valley, the part-time directors of a number of small foundations - all the organizations sharing the same accountant - have continued to collect benefits even as their foundations' assets plummeted.

One of them is the Sara and Anders Kierulf Family Foundation, which was established in 1999 in San Carlos, Calif., but relocated to Utah when its sole trustees, Sara and Anders Kierulf, moved to Salt Lake City. The Kierulfs run the foundation out of their home, contributing to a variety of charities. They estimate they each spend 20 hours a week on foundation business for the $120,000 in salary that each of them takes.

They have also used foundation holdings to cover escalating medical costs for themselves and their three children. Last year, the foundation paid nearly $11,000 for the family's health care, up from $7,800 the previous year and $4,100 the year before that. Meanwhile, the foundation's assets dropped by almost a third - from nearly $6 million in 2000 to slightly more than $4 million last year. The foundation's accountant, Martin Logies of Sunnyvale, Calif., defended the benefits, saying they had been approved by the foundation's board of directors. But he acknowledged that Sara and Anders Kierulf are the board's only members, and that they approved the benefits for themselves. As to the work the Kierulfs perform for their pay, Logies demurred. ``I couldn't give you that information,'' he said.

But it isn't as bad as it looks! I had been wondering where he got the money to buy the source code of GoTools, Thomas Wolfs' TsumeGo solver. I had assumed Anders was a millionaire, due to Microsoft stock options or something. Because Thomas had alledged to me that he had sold it for hundreds of thousands of USD. He never wanted to be specific, but one of the figures he named was five hundred thousand dollars. I kid you not. So I had no chance of aquiring it. I tried persuading him to let me have it in return for me making it run much faster, to no avail. If I have to believe the article, Anders spends half of his working week on accountancy, instead of working on SmartGo. Perhaps I still stand a chance catching up with him. Of course this is just a clever tax trick - Anders is not misappropriating funds but has merely put his 6 million saved dollars in a SmartTax haven.

Yes, there is (potentially) a lot of money to be made with computer Go. Take GoGoD, for example. They say that their business doesn't make a profit. It's a hobby, a service to the Go community. But.. They are well-known, have been at it for a long time. They offer an interesting product. And it's pretty expensive, if you consider the fact that it's almost exclusively freeware and non-copyrightable game records.

They say it's not making them any profit - All I can say is that I earned 4,000 USD in the first three months of selling Moyo Go. Profit is defined as what you have left after expenses, which makes it a meaningless metric to indicate how much you have earned. By the way: Everybody wants a copy of my software, everybody wants to review it, but nobody wants to publish any reviews.. (That might have something to do with GoGoD having told the Go publishing world that I am a bad boy for including "their" games in Moyo Go without offering them compensation - a consequence of them spreading FUD on rec.games.go about ZenHacker, my earlier SGF Reader. GoBase, the Western world's largest Go website, doesn't want to link to me, and mentions on their website that it's a very bad thing to copy "GoGoD's" games, that the poor GoGoD guys are "non-profitably" selling. Well, I don't know. If GoGoD sells as much as I do, they make 16,000 USD/year. But I am pretty sure they sell an order of magnitude more than I do. Everybody knows them, also in Asia. Everybody endorses them. Everybody links to them. They are all over the web. I think they make about a hundred thou a year, and that was exactly the reason they immediately vilified my first excellent freeware - because it was a clear threat to their business.

The same with GoBase, their business is to give people access to Go games and let them perform searches on them, just like you can with Kombilo on the GoGoD game collection CD. Of course, if folks would have GoBase's games on their local machine and would be able to do much more powerful (and faster!) searches on them, Jan van der Steen's plans to charge his - at least - ten thousand customers money for his site will be frustrated. If he asks 10 USD/year he'll make a cool hundred thousand USD/year - no wonder he refuses to include MoyoGo in his list of SGF Editors!

Some think that the solution to their problem is not innovation, but FUD, boycots and libel. In the short term this will have some effect, but in the long term it will derail them even faster because of course in business, one focuses on competing with one's most vociferous competitors first. Therefore, Moyo Go soon will exceed GoGoD's and GoBase's capabilities, for the sole reason that they chose to make my life difficult. Otherwise I would have been busy making an IGS client or something.

Moyo Go, The Movie

I've just made an off-the-cuff demo in which I convey the cool capabilities of my creation. The movie comes as a humongous 175 MB EXE, made with Camtasia. The reason it's an EXE is to avoid the need to install their CODEC.

The (20 minute or so) audio/screen recording presentation gives a fair general impression, although the docking system's guiding widgets don't appear because Camtasia can't handle "overlays". And MoyoGo's (annotation/splitting) cursors are replaced by a stock Camtasia hand cursor instead. And I had to keep the window small, though MoyoGo likes to have as much screen real estate as possible. But you'll get the idea.

Smaller movies that give a good impression of certain features are here.

There are a few slips of the tongue and you''ll notice that I suffer from absentmindedness, but hey, I'm a hacker, not a Hollywood director :)

The movie.

Hype & Reality

The new Zalman CNPS 9500 is a CPU air cooler with the best noise/cooling performance. The web is full of praising words. Zalman's propaganda machine has been very effective (so far, no review of Moyo Go has been published anywhere, I'm envious).

Anyway, I bought two because after years of living and sleeping in my rented 1-room appt., the noise was getting on my nerves. My PC is on 24/7 because that increases reliability (thermal considerations - I am an electronic engineer by education and we learn that it's better to have a constant temperature and constant current instead of on/off/on/off/on..). My supplier sent me only one, but billed me for two. So far not the end of the world. One costs 75 USD here, BTW. Okay. So I mounted the thing and I was surprized how little contact it made with the CPU.

I decided to STW for that model and hey presto, I found a bunch of German folks on a forum who had quite some trouble with the thing. The mounting clip broke. I thought: "Let's look at mine" and sure enough, my clip was half way through..

Zalman has perhaps the world's best CPU cooling solutions. They are creative, innovative and they have a proven track record of producing kick-ass coolers. Yet they managed to produce this utter piece of crap. (If it would have broken off during operation, my damage would have been 1,500 USD for the overheated CPU plus 1,000 USD for the shattered graphics card underneath - without the slightest chance of compensation apart from another one of thouse lousy clips.) And that would be best-case, because that brittle clip broke straight through the middle as well, into three pieces. If a piece would have short-circuited something, the disaster would have been complete.

Automatic Updates - No Gray Hairs Yet

The system of automatically updating software is similar, in some respects, to the system of periodic abstention: It's very nice and it usually works, but when something goes wrong, it usually goes quite wrong.

So far, so good.

If I would have to roll out bug-free or feature-complete software, my customers would have to wait until all my hairs were gray. So auto-updates it is. This way, Moyo Go can grow strongest fastest. It is a concept of XP, (eXtreme Programming), that you should release bugfixes and new features often. This speeds up the customer-feedback cycle.

I am not sure whether my customers think I should tone down the updates a little. I don't really know how fast their average internet connection is, for example. But there is always one customer waiting for that one fix or feature. I guess that when they don't want to be bothered too often, they just turn them on when they feel like it. Those who always want the latest version have it on all the time, even if that means they'll sometimes get an update twice a day, or run the risk that they'll have to cope a day with a build in which something is broken that wasn't broken before.

All in all, auto-updating works extremely well, knock on wood. In the ten weeks I've been at it, I've managed to introduce many new features and fixed at least a dozen bugs. It's much better to work with small increments (and get instant feedback), than to hack about for months and release a major new version with zero "real world" usability-testing in the meantime.

For me, the programmer, the update system actually is a complex affair. Moyo Go has no copy-protection, but I have implemented a sort-of anti-piracy measure: Every customer has their own, uniquely encrypted pattern database and the decryption key depends on the checksum of the executable. And the executable itself is packed. I can know which copies update a hundred times per week and are therefore pirated en-masse. I can block automatic updates of that version. I believe that potential pirates who are really serious about some software, will still prefer to pay for software if pirating it means they will not be able to get any updates of the software. And the original owners will not likely allow their software to be pirated, because that means their own updates (which have become identical to the pirated ones) will be jeopardized.

Of course, all this only works when there are updates. But there are.

Some software developers jump through hoops to safeguard their intellectual property, and I'm not one of those. The guys that sold me the installer I use (Ghost Installer, 600 USD) actually phoned me to verify that I wasn't a Chinese hacker or something.. And they do that to all their customers!

The Tree

One of the solutions in Moyo Go that I am proud of is the treeview. This treeview is unique: None of the other SGF Editors displays the moves like Moyo Go does. It looks so natural that it isn't even noticeable at first sight, but there is something highly cool going on here. The way the moves are shown is totally user-centric instead of programmer-centric.

Moves 1 - 5 still look like you would expect in an ordinary tree. But look at moves 9 - 13. They don't sprawl out diagonally, they are below eachother. Because why would you display moves to the user as if the user is interested in the way the programmer has represented the tree structure in memory? The user only wants to have the clearest possible representation of a game and its variations.

Trees can be daunting, so I tried to make things clearer by having the main branch always in white, and the darker the background, the "deeper" in the tree (= the further from the main branch). It's very rare to have the move numbers displayed on the (3D) stones.

There are many more conveniences for SGF authors: Simply click on a node and edit its title. You can zoom the tree. You can change move numbers with a spinner button and immediately see them change on the stones.

No effort has been spared to make this tree a joy to use and it's packed with those little details that make a difference. When you rotate the boards, the move coordinates in the tree update as well.

Like all other windows in Moyo Go, the tree can be "docked" anywhere you want, and it can contain any toolbar, like the navigation toolbar.

Navigation in the tree is done with the cursor keys, but you can also use the mouse wheel to navigate through its branches - as long as the mouse hovers over the board, not the tree itself. (When the mouse is above the tree, the mouse wheel scrolls through it.)

Lie on the envelope

This is the bubble envelope that Moyo Go Studio comes in. To keep the cost down to customers, it's not sent by registered mail because only in less than 1% of cases, a letter doesn't arrive in time and in that case I send a new copy but for the contents I write something like "cardboard sample demo". Because my post office asks me 10.75 USD extra to send MoyoGo registered. That's more expensive than sending another copy. That's the disadvantage of living in one of the world's most expensive cities. A beer across the street is 9 USD. One would almost be relieved that summer lasts only two weeks here.

The customs declaration specifies "CD", even when it's a DVD. This is to minimize the risk of theft. It is sent as a "gift" and the value is "none".

Standards, which standards?

The computer Go world suffers from a lack of standards, or, to be more exact, enforced standards.

Take player's names, for example. Jan van der Steen has invented a marvellous method of uniquely identifying players. He assigns numbers to them, the socalled PID or "player ID".

I would love to know what his mechanism for assigning them is. I would love to have a list of those PID's. Why? There are so many alternative methods to transliterate Chinese, Japanese and Korean names into English. Hypothetical case: When, in Gobase for example, someone is called "Fu Chien" but in GoGoD "Fu Kien", Moyo Go will only be able to find "Fu Kien" and not "Fu Chien".

In such cases it would be a solution to know the PID, and be able to search on it. Unfortunately, Jan van der Steen sees me as a competitor and he declined to share his PID system with me.

There are more of these problems. Similarly as we are currently unable to uniquely identify players because we somehow don't seem to get along, we also are unable to uniquely identify games. There is the "Dyer Signature" - Moyo Go implements it - but Dyer Signatures are neither universally supported, nor suitable to distinguish between games with minor endgame variations.

Then there is the issue of identifying good and bad moves. Take the marvellous Kogo Joseki Dictionary. Gary Odom has done a terriffic job and permitted me to include it with Moyo Go. Yet I found something to bitch about :)

Namely, the way bad moves are indicated leaves something to be desired: They are indicated simply by a "bad move" comment, instead of the standard SGF property "BM". That's not really Gary's fault, we SGF Editor publishers should make it easier for the user to click a "bad move" button! For computer Go scientists, it is wonderful to have a standardized way of indicating bad moves (especially with omitted comments), so that automatic learning modules know they are bad.

Another point is Unicode. Some SGF contains Unicode without explicitly indicating this. Almost all SGF with Unicode that does indicate this with the proper SGF property does not contain a Byte Order Mark, so that ordinary editors still don't know that the file contains Unicode.

Etcetera, etcetera. Write an SGF parser, and you'll see what I mean. There is so much malformed, downright illegal SGF out there. Not to mention abuse of certain SGF properties, like adding a circle marker after each move (KGS), instead of leaving it to the user, to have the SGF reader show something on the last move or not.

In the meantime, the SGF standard has not really been brought into the 21st century. Where are the properties that cater to multimedia? How do I encode Rich Text, Sound and movies from my webcam? It's left to the individual programmers to invent new properties and encoding standards, if they want to support stuff like that.

I did that, I made up a new SGF property that supports Rich Text, including images and tables.
So far so good. No rocket science. Just take some Rich Text (*.rtf), compress it and then BASE64-encode it because some SGF reader makers complained that their code can't handle bytes that have their MSB set. Fine. So I followed all the rules for well-formed SGF to the letter and put some nice pictures of Go Seigen in my newly defined SGF property. What happened when I tried to load it into [censored] and [censored]? Not just a crash, much worse, a complete lockup of the computer! The programmers never anticipated long SGF properties. (And of course I provided a compatible SGF property as well with the plain-text equivalent).

I informed the perps and hopefully, the newer versions of [censored] and [censored] do support SGF properies of arbitrary length. Since I introduced Rich Text in Moyo Go and sold 100 copies, I have not heard of a single person who has actually used it (the feature is disabled by default, that might have something to do with it). But if any other programmer is interested, the spec of how I encode RTF into BASE64 is open, send me an email and I'll be more than happy to share it with you.

Secret Opcodes

Every serious hacker sooner or later needs the popcount instruction.

This "population count" instruction counts the set bits in a register, and is so useful that the NSA demands that all computers they purchase implement it in hardware.

Decryption, uncompression (decompression is something divers do :), pattern recognition, Zobrist hashing (generating keys in a certain Hamming distance range) and cellular automata all profit from a "popcount" (better to be called BC) instruction, and the time and effort it saves is very substantial, compared to a kludge in MMX or SSE2, where it's done on 64 or 128 bits in parallel.

But even with hundreds of millions of transistors in current CPU's, there isn't a single mainstream desktop CPU that implements this so-called "canonical NSA instruction".

This reeks of a conspiracy.

The Itaniums and some less prolific CPU's implement it (V9 SPARC and Seymore Cray's Cyber, both at the explicit request of the NSA - as well as Sun's UltraSparc, the PowerPC and Dec's Alpha 21264, heck - even the T800 Transputer had it), but neither the latest Pentiums, nor the latest Opterons support it.

Conspicuously absent, in spite of a billion transistors in real estate.

It wouldn't be the first time that a microprocessor manufacturer (Intel) would implement undocumented opcodes, only to "licence" them to major - sworn to secrecy - software producers.
Those opcodes can have aliases, so that when they leak, they can be traced back to the licencee.

That way, Intel could provide Microsoft with a competitive advantage in producing Codecs and unZIPpers (to give but an example), whilst Microsoft would, for Intel's benefit, delay their 64-bit OS for the Opteron for more than a year, to give Intel time to catch up with AMD. Of course this is all conjecture, but it's inspired by the historical fact that a certain CPU manufacturer has sold the specs to undocumented opcodes before, whilst we all know that Microsoft is getting convicted for illegal monopolistic practices on a regular basis.

In the past there was COCOM, which was a set of export restrictions to prevent the Cold War adversary from getting their hands on powerful CPU's, amongst other things. But nowadays the "enemy" is everywhere, the new enemy uses encryption and the NSA would like to keep the edge.
One of the ways to maintain an edge is to either keep certain capabilities of current CPU's a secret, or to prevent these capabilities from being implemented in affordable mainstream PC's altogether.

If mainstream CPU's would have a popcount instruction, I could make a Go position evaluation function run twice as fast as it does now.
In Go, chains of stones have liberties, and when chains are merged or captured, liberties need to be re-counted.
Merging chains can be done very efficiently using bitboards (as used in chess), and it would be ideal to count liberties using this much missed instruction.
At the moment I need to use a parallellized algorithm that uses MMX or SSE2.

Random Thoughts

When people say: "I built my own computer", they mean they buy a motherboard, drives, CPU, cooling system, RAM, power supply, case and screw and plug it all together.

So I always have to explain that when I say: "I built my first computer myself" I mean that I had to actually solder all components onto the motherboard. (The empty motherboard of my Acorn Atom came with a bag of resistors, IC's, capacitors and connectors).

Still I should not say: "I built my computer myself", for I know a guy that has really done so. He lives in Tbilisi, Republic of Georgia and he showed me his self-made computer. It was unique. He designed the hardware using "Soviet" IC's, and he hand-designed and etched the motherboard in an acid bath. He drilled the holes for the component's wires with a dentist's drill. He wrote the BIOS. He wrote the OS. He wrote the compiler. He wrote the system utilities. Tbilisi had rationed electricity in the early nineties so he powered his computer from old car batteries. Those batteries were refurbished. He had emptied them of battery acid, collected the best plates, washed and cleaned them, hammered them flat again and put them back in the battery casing and filled it up with new acid.

If I would really build my own computer today, I would add a very important thing for computer scientists: A crappy carbon resistor or a crappy Germanium diode. In fact I am prepared to pay more for a system that has it.

The thing is, crappy carbon resistors or crappy Germanium diodes noise (verb). The DAI had one. (The DAI hails from the same era as the Acorn Atom, the TRS 80 and the Exidy Sorcerer, around 1980). Such "Johnson" noise is produced by the random movement (Brownian motion largely caused by thermal effects) of electrons in the component (PN junction noise in case of a [Zener]diode or transistor). This is real quantum-randomness. Mathematicians will start to drool at this point. People that use Zobrist hashing will salivate as well.

John von Neumann said: "Anyone who considers arithmetic methods of producing random digits is, of course, in a state of sin."

The DAI used quantum noise to generate truly random numbers. You do this by amplifying the noise, Schmitt-triggering it, feeding the pulse train into a shift register and clocking the resulting machine word out. Highly cool. Strange that this is not integrated into modern CPU's. Hundreds of millions of transistors, but nobody thinks of using a handful of them (downgraded to be "noisy") to generate real random numbers fast.

But yet, really good hardware random generators are unwieldy and expensive.

Thermal noise-based random is slow compared to generating "random" numbers with a shift register, but this can be augmented by giving each bit its own random generator, hashing the bits to compensate for the fact that some generators will have an asymmetric duty cycle, and having a constantly filled-up "random cache" with a capacity of a few thousand 64-bit random numbers, to be used as either integers or floats by the CPU. Random numbers would be really random and load into a register in a single clock cycle! This would be near-trivial to do for AMD, Motorola, IBM or Intel. And I want royalties.

Why would you need a "perfect" random? I am not a statistician or some kind of dogmatic purist, but I do need one. I make pattern recognizers for "Artificial Intelligence" purposes, excusez le mot ;)

I started with the random generator in Delphi. That one sucked. When you plotted the random on a 1024 x 1024 field, entire patches remained empty. So my pattern recognizer sometimes shot an old lady because it mistook it for a tank (this is a metaphor, it would go too far to explain what my stuff actually does, and why some forms of pattern recognition need really random random).

I then decided to generate random numbers myself, ensuring they all were in a certain Hamming distance range from each other. That's allegedly very useful for Zobrist (xor) hashing, to get an even spread (minimize collisions). But then a discussion on the computer Go newsgroup made me have doubts, and I now use a Mersenne Twister. The verdict on that one is still pending.

<tinfoil hat> Note how none of the modern mainstream CPU's/PC's provide functions to decrypt quickly (popcount) or encrypt well (true random), but that older CPU's/PC's did provide this functionality.. </tinfoil hat>

The Case For 64 bits

Hardly anyone has a clue as to why 64-bit computing is almost just as cool as the step from 16 bits to 32 bits.

The hard core of the database weenies can tell you that you get a larger address space, and that this is nice.
Nothing much to make the hacker's heart beat faster.

To make matters worse, there have been some benchmarks recently where folks took a 32-bit source, compiled it for 64-bits, ran it, and noticed that it ran slower (one of the reasons is the fact that 64-bit code is bigger and therefore puts more strain on the cache, another reason is that bigger code & data takes longer to fetch from memory).

It is clear that not many really understand what 64-bit code can do for software, apart from being able to address more memory.

So I had a look at my code and I will show you where I use 64-bit variables, and why this makes the code simpler and sometimes extremely much faster (a factor of 4 is not uncommon). Just to be clear: Of course my code only runs that fast when it's compiled by a 64-bit capable compiler, and running on a 64-bit machine running a 64-bit OS.

The point in using 64-bit variables is speed. Unadulterated, blazing speed. What I used to do with hand-optimized MMX assembly, I now do with 64-bit variables. MMX is severely limited (read: slower) compared to full-fledged 64-bit code.

My software for the Asiatic board game of Go could well use an address space larger than 2 GB, in fact I am at that limit right now. During pattern harvesting, I use an array of 134,217,728 64-bit variables. I need more space for other data structures, and some spare RAM to be able to work on my computer while the thing spends weeks number crunching. (I have another PC but it's a bit slower and it doesn't have 2 GB, only 1.5 GB which is just too little for pattern harvesting). So much for the boring part of 64-bits, the increased address space.

64-bit operations afford very major speedups in applications like encryption/decryption, video/audio encoding/decoding, Chess programs, image manipulation and data compression/decompression and I leave it to others to explain the details. I will give concrete examples on how 64 bit variables occur "naturally" in my computer Go program.

1. Patterns
Moyo Go Studio has a pattern matcher. Patterns are, for efficiency's sake, stored as 64-bit hash values. 64 bits is needed to achieve the required reliability. The pattern matcher uses 64-bit random values, it XOR's those values with each other, it looks those values up in an array with 64-bit values, and so on and so forth.

2. Bitboards
Chess programs use them: for every piece-class, they use a 64-bit variable that has a set bit for a piece of that class on the corresponding chessboard coordinate. In Go, bitboards are highly useful as well, but unfortunately they do not fit into a single 64-bit variable. That does not mean we should revert to a bunch of 32-bit variables - of course we want the largest width available that the compiler supports. Our code will be smaller, simpler and faster. Moyo Go's bitboards are records consisting of seven 64-bit variables, and operations on those bitboards go about three times faster when they are done in native 64-bit as opposed to 32-bit (or 32-bit opcodes simulating 64-bit variables by combining two 32-bit values).

3. Operations on bitfields
Moyo Go uses all kinds of databases (games, patterns, players) and the amount of data is so massive that every trick in the book is used to compress the data. Otherwise the software would not fit on a CD. Of course you can use bitfields in 32-bit variables, but as soon as a unit of data exceeds 32 bits and you need to sort them, you need to do two separate swaps of two 32-bit values instead of a single swap of a 64-bit value. The same goes for manipulations on bitfields. Things go faster when everything is in a single CPU register, and you save registers as well. 64-bit CPU's have not only the ability to operate on two 32-bit values in a single instruction, they also have more registers, something the optimizer can take advantage of. I use 64-bit variables to hold data collections. a 64-bit variable can hold the (max.) four coordinates of any adjacent chains and their color. So instead of using four variables, I use a single 64-bit variable.

4. Copying state
Search algorithms can be sped up by clever "state"-bookkeeping and that involves the copying back of previous states. Memory is most efficiently copied 64 bits or more at a time. Often, the OS should be able to do this most efficiently but whenever you use a loop to copy memory, 64-bit variables are faster than 32-bits.

There are many more places where 64-bit variables are the most natural and fastest solution, and I am simply flabberghasted that so few programmers realize it. Apart perhaps from simple databases, I can't think of any kind of software that would not benefit of their use. (In fact, people have told me that even "trivial" databases would benefit).

Big Iron vs. the µISV

We small Independent Software Developers (micro-ISV's) should not be afraid of large software companies. Look at Microsoft, for example:
A while ago, three of their boffins spent quite some time trying to copy my research, but they admit in their cover letter that they fell short. I am not a Ph.D. so I didn't understand their paper much. In fact I have been a beach bum while they were doing their homework:

Now don't I look like a "terrorist".. The areas where I dwelled the longest (Taba, Sharm el Sheikh and Ras Satan) have, in the meantime, all been bombed by "Al Qaida". But to return to the topic: Large software companies by definition are unable to produce good software. It's pretty sad that even after years of development and aquisition by Google, Blogger's "Upload Image" still doesn't work. And don't get me started on MS products. Simple things like typing a word in a certain font does not work. Copying files does not work. Cut & paste doesn't work (Why would I want to copy a trailing space when I double-click a word?) No Microsofties, I am neither impressed nor afraid.

"I don't play Go myself"

..Yet, I am utterly fascinated by the game and if it weren't for the fact that my talent lies more in programming Go rather than playing it, I would spend a lot of time playing online. Above are some notes I made last year, when trying to design an algorithm to recognize Seki.

Of course I'm not totally ignorant about the game - I have played it at least a dozen times and even managed to win few times from a 12 Kyu. I intend to become a regular player in the near future because it's important to use one's software oneself. Besides, Playing Go is good for the soul :)

Skinned

This surrealistic goban is a MoyoGo board skin. You can make your own skins - simply use Google Image Search for a nice picture and let MoyoGo do the rest.

Natto

After trying several brands, I found the perfect natto (bacterially fermented soya beans). To stay on the topic of "multi- threading": It is not too slimy that it pulls permanent threads, made with 100% whole beans and with mustard in addition to soya sauce.

I like natto, but it also helps against osteoporosis. Due to neuroborreliosis, I have been taking megadosed antibiotics for the past five years. This kills the gut flora (which produces vitamin K, which is needed for healthy bones). Natto contains a lot of vitamin K.

The Birth of a New Machine

Finally!

It has been quite a challenge to build a new PC. The problem was not putting all the parts together, but getting them to work properly.


The machine in this picture is a dual Opteron dual-core 275, one of the fastest personal workstations on the planet. Sporting dual 20.3" LCD's, 4 GB RAM, several 15,000 rpm SCSI disks, 43,000 MIPS, 18 GFLOPS and 10 GB/s RAM bandwidth, its price tag is a whopping 11,000 USD - an investment into Moyo Go Studio's future. The next major development (apart from bugfixing and implementing features suggested by users) will be a massively multithreaded, 64/128-bit TsumeGo module, and this machine is where it will all come together. Now the only thing I still need is a gun, so that nobody will come and take it away from me :)

Why such an enormous expense? Ordinary desktop PC's of the future will have dozens of "cores", which are separate processors, the powerhouses of the computer. And they will all be 64-bit as opposed to 32-bit. Go software that does not anticipate this, will be doomed to play a minor role. Compared to what the average Go player will have on their desktop in a mere five years, my Colossus is not even special. Add to this the generous oil-wealth inspired Norwegian "HjemmePC Ordningen" - my employer lends me half and the government pays the other half - and I couldn't resist. UPDATE: I recently resigned my job so I had to pay almost the full price..

A major problem in creating a strong Go program is the speed of current computers, so taking full advantage of the (near) future's multi-core 64 bit desktop PC's is essential to staying ahead and coming out on top, eventually.