Saturday, December 31, 2005

Multiprocessing

Due to a technicality, the past two days of number crunching for the Fuseki module have to be redone. I decided to give multiprocessing a go: Four threads (each CPU its own) will load SGF files, process them and write the results to a database. This should increase speed about fourfold. And if I manage to make this in a day, the gain will be half a day.

However I don't think it's realistic to do this in a day - Multithreaded number crunching is non-trivial for a beginner in these matters like myself. It will become increasingly important to do so: More and more customers will expect that their multi-core CPU's be put to good use. The goal is to process SGF (add to database, including index generation for pattern recognition) with all available processors, even when there are dozens, as will be the case ten years from now.