TreeProjects in the 64-bit world

Author: admin  |  Category: Development

The consumer desktop world is moving quickly towards 64-bit computing, and 64-bit Windows installations may already outnumber 32-bit ones. When TreeProjects development was started in mid-2008, 64-bit systems were exotic, but today they are mainstream. Therefore, in TreeProjects 2, I had to address some issues arising on 64-bit Windows. This post will describe what was going on.

The hidden bliss of Windows on Windows

64-bit Windows runs 32-bit applications just fine. Note that this capability doesn’t come for free to Windows. Running 32-bit on 64-bit called for an entire new Windows subsystem called WoW64 (Windows on Windows). This subsystem virtualizes the environment for 32-bit applications and provides compatible access to system services.  Sometimes, 64-bit Windows goes as far as providing two copies of services or programs – for example, there are two Internet Explorers on 64-bit Windows systems, one 32-bit and one 64-bit.

Whereas 32-bit and 64-bit programs coexist and function on 64-bit systems, there is an important limitation to their cooperation. 32-bit applications cannot use 64-bit libraries, and 64-bit applications cannot use 32-bit libraries. (A library is a small sub-program that is used by an application; several applications may share a single library.) This is not a problem for TreeProjects per se, because both its libraries and itself are 32-bit programs. However,  an issue arose in a very important area of TreeProjects: attachment search.

Indexing blues

When TreeProjects creates a file item, it indexes the content of the file, if possible. Indexing means extracting the plain text from a file and remembering word positions, to enable instant search later. Extracting text from a file is a non-trivial task, and there are myriads of file formats in existence.

Fortunately, Windows provides a way to “outsource” text extraction to another installed program that supports the file format in question. For instance, when TreeProjects indexes a .doc file, it goes to Microsoft Word and says: “Hey, you can read and write .doc files, so you can as well tell what text there is in this .doc file. Please do me the favor, thanks!” This is great, because TreeProjects doesn’t have to know anything about a file format, therefore it potentially supports any current or future file format for indexing.

In order for TreeProjects to communicate with it, a program that supports a file format has to volunteer and register an “IFilter” component that actually does the text extraction. So when TreeProjects wants to extract text from a file, it in fact calls the corresponding IFilter. Now, in 64-bit Windows programs may install either 32-bit or 64-bit IFilters.

But, as I mentioned, 32-bit programs cannot use 64-bit libraries. And 64-bit IFilters are not usable by 32-bit TreeProjects. This is why TreeProjects version 1 would often fail to index files on 64-bit computers, despite the supporting software being installed on the PC.

Solving the problem

There seems to be the obvious solution to this problem: make a 64-bit version of TreeProjects for 64-bit users. Unfortunately, this is not a solution at all. As you remember, 64-bit programs cannot use 32-bit libraries, either. And if you happen to have 32-bit IFilters on your shiny 64-bit Windows (and you do have them), they will in turn become inaccessible.

So this is an either-or situation: you either have 32-bit TreeProjects without access to 64-bit IFilters, or you have 64-bit TreeProjects without access to 32-bit IFilters. You can’t win, can you?

Fortunately, there is a technique that allows TreeProjects to use both 32-bit and 64-bit IFilters. Whereas programs cannot call libraries of different “bitness”, they can communicate with other programs of different “bitness” using inter-process communication. Inter-process communication is much more cumbersome than a simple call into a library, but it builds the vital bridge between the 32-bit and 64-bit kingdoms.

So, when TreeProjects needs to index a file, it first tries to index it by directly calling an appropriate 32-bit IFilter. If it fails, it runs a tiny 64-bit exe file called “srgprc.exe” (srgprc stands for “surrogate process”). This new short-lived 64-bit process talks to 64-bit IFilters on behalf of the main application, and returns the result to TreeProjects using inter-process communication.

This solution is more clunky that I’d like it to be, but it leaves us with all bases covered and is completely transparent to the users of TreeProjects.

A case for 64-bit TreeProjects

Why don’t I make a 64-bit build of TreeProjects and use a 32-bit surrogate process? 64-bit is cool, after all!

The question is good, but the answer is: there will not be a 64-bit build of TreeProjects in the near, or not-so-near, future. And the reasons for that are:

  • Among the libraries that TreeProjects uses, there are libraries that are provided by third-party vendors, which I cannot recompile for 64-bit architectures, and which the vendors don’t plan to release for 64-bit platforms.
  • Contrary to an intuitive conception, a 64-bit build of TreeProjects would not noticeably improve the speed or resource usage on 64-bit computers. It might even become slower, and would certainly become somewhat bigger. So even if I could battle the first bullet point and make a 64-bit build, it would not be worth the significant overhead of keeping and managing two separate builds.

To sum it up, there will not be a 64-build of TreeProjects for a while, but that will not hurt its users in the slightest.

Thanks for reading, take care and have fun!

–Yaroslav

One Response to “TreeProjects in the 64-bit world”

  1. NeoSmereka Says:

    Thank you for writing this latest blog entry. Very informative, as usual, and quite educational.

    I appreciate you taking the time to write it.

Leave a Reply