The Tradeoff - Sat, 14 Nov 2009

I’ve made great performance improvements that you will see when the Beta is released. It is all due to virtualizing Richview and not generating the entire Everything Report at program startup. Instead, only the piece you need to see is generated.

For my large test GEDCOM (24 MB, 100,000 people), it has reduced loading time and memory from 116 seconds and 994 MB down to 14 seconds and 260 MB. That’s a pretty good improvement in both time and memory.

But there is a tradeoff involved. The entire Everything Report is no longer resident in memory, so operations that need to go through the whole document need to have it generated.

That problem manifests itself in my last 3 tasks before Beta: Searching, Printing and Exporting. In each case, you may need to go through the whole file to get your result.

In the last alpha version, after waiting the 116 seconds for the file to load, if I searched for a text string that wasn’t in the file, it took less than 2 seconds to come back with “string not found”. In the new version, it takes about 10 seconds.

This is only a problem with very large files such as this one. It is something that I can optimize and make somewhat faster. It will never be as fast as searching though a pre-generated in-memory file. And the larger the file gets, the longer it will take. I may be able to get it to work fast enough for this size file, but one 3 times bigger will still take 3 times longer.

So for now, this will be the tradeoff. A few operations that will take longer than before in return for much faster loading and a big reduction in memory use.

The result is I will have to change the cursor to indicate when Behold is busy with these operations. For now, that will have to do.