I am happy to announce that Compass now supports integration with Terracotta. This integration allows for applications that use Compass or pure Lucene to store the index in an optimized, memory based storage, that uses Terracotta as a “network attached memory”.

The latest nightly build supports it. The Artifacts tab allows to download the nightly build as well as view the terracotta integration documentation.

In a nutshell, Compass jar is now a Terracotta Integration Module (TIM), which allows to drop it into the terracotta installation and enable it simply within the application tc-config.xml file. Compass can be configured to store the index in a Terracotta based connection using a one line change in the configuration and easily distribute your Lucene index.

A common question that might be raised (for people who are not familiar with Terracota) is how does a large index will be stored in memory. The answer is that Terracotta takes care of this by fetching and evicting data on demand from the server to the different clients. The TerracottaDirectory (an implementation of Lucene Directory) is built in a way that breaks the file content into one or more byte arrays (their size is configured) which allows Terracotta to easily fetch parts of the file and evict other parts based on certain policies (LRU, LFU, …).

Another question, which is answered in the reference docs, is what will be the Terracotta “root”. When using pure Lucene, the root will need to be defined and configured. When using Compass, the root is already defined (as Compass holds the directories) within the TIM configuration (as well as the locks), and there is no need to configure special Compass terracotta configuration except for listing the Compass module within the application tc-config.xml.

Enjoy!

Well, its time for me to move on. I am leaving GigaSpaces and joining Delver to help in creating the best socially connected search engine out there.

This has not been a simple decision, and I think my “leaving GigaSpaces email” describes it best:

I am sad to say that I am leaving GigaSpaces. After almost 2 years with GigaSpaces, its time for me to move on. As someone who strongly believes in GigaSpaces, this has not been a simple decision, especially since I am sure that GigaSpaces is going to succeed. But, a chance to go and work on my primordial love, which is search, in a very young startup, is not something that I thought that I can allow myself to pass on. It has been great working with all of you, the DNA pool of GigaSpaces is one of the best one can wish to find in a workplace.