Compass 1.2.2 maintenance version has just been released fixing major bug when working with a Jdbc directory as well as adding minor features. Check out the release notes for more information.

I am happy to announce that Compass now supports integration with Terracotta. This integration allows for applications that use Compass or pure Lucene to store the index in an optimized, memory based storage, that uses Terracotta as a “network attached memory”.

The latest nightly build supports it. The Artifacts tab allows to download the nightly build as well as view the terracotta integration documentation.

In a nutshell, Compass jar is now a Terracotta Integration Module (TIM), which allows to drop it into the terracotta installation and enable it simply within the application tc-config.xml file. Compass can be configured to store the index in a Terracotta based connection using a one line change in the configuration and easily distribute your Lucene index.

A common question that might be raised (for people who are not familiar with Terracota) is how does a large index will be stored in memory. The answer is that Terracotta takes care of this by fetching and evicting data on demand from the server to the different clients. The TerracottaDirectory (an implementation of Lucene Directory) is built in a way that breaks the file content into one or more byte arrays (their size is configured) which allows Terracotta to easily fetch parts of the file and evict other parts based on certain policies (LRU, LFU, …).

Another question, which is answered in the reference docs, is what will be the Terracotta “root”. When using pure Lucene, the root will need to be defined and configured. When using Compass, the root is already defined (as Compass holds the directories) within the TIM configuration (as well as the locks), and there is no need to configure special Compass terracotta configuration except for listing the Compass module within the application tc-config.xml.

Enjoy!

I asked a while back if Compass should include the alias within the all field or not. Here is the question and the decision was to remove it. After getting some emails and based on some recent forum posts, I am not sure that this is the best “out of the box” experience for Compass users. I am considering turning this back and currently leaning towards it. What do you say? I wish to nail this before 2.0 RC1.

It started with a thread in Compass forum. This turned out to be a new feature request in Compass, allowing for Compass to automatically set the index setting for a property mapping based on the type (if it is not explicitly set). Compass tries to adhere to “Conventions over Configuration” (which I like to name: “COC” :) ), but I guess it can always do better.

For example, most, if not all times, a number (such as int, float, …), should be stored in UN_TOKENIZED form. By default in Compass, if not setting the “index”, everything will be TOKENIZED. Now, with the latest Compass trunk, number types will be TOKENIZED. This also applies to Boolean, URI, URL types.

Last but not least, for numeric types, a specific format has been added named: sortable. This means that if you put it in the format tag for specific mapping definition, it will be stored in an internal sortable format, which makes thing much simpler.

Compass supports the ability to completely index external content (for example, a database mapped using an ORM) using very simple API:

1
2

ComapssGps gps = // ...
gps.index();

This indexing process is a parallel indexing process (by default) based on the number of sub indexes, and will go and fetch all the data mapped using the ORM tool and index it using Compass. The old content of the index (for all types) will be replaced by the new content.

Some Compass users have suggested a (very) valid feature request of being able to just index specific types. So now, this feature is available in trunk (todays nightly will have it). Here is what it looks like:

1
2

ComapssGps gps = // ...
gps.index(Person.class, Account.class);

This will only index the Person class and Account class data (in parallel if possible) and replace only their respective sub indexes. The rest of the index will remain the same.

Now, the only thing left is for the Grails Searchable Plugin to enable it so people will be able to this: Person.index() in groovy.

Finally I can go to bed ;)

Compass 2.0.0 M3 version includes a really nice feature which is called searchable cascading. Compass support the ability to cascade operations when using the component and reference mappings. What Compass users wanted was the ability to defined cascade definitions for properties that do not necessarily have a component/reference mappings.

This feature is really handy when integrating with an ORM, which, because of a result of Account, the User object will be reindexed. More information can be found in the OSEM docs.

Compass version 2.0.0 M3 has just been released. This release includes several major features, among them are: boosting with all property, spell check support as well as many bug fixes and improvements. For a complete list, please check the release notes. Enjoy!

It seems like most people prefer that the alias will not be included in the “all” property by default. As of latest 2.0 M3 builds, it is not. Check CMP-580 for more information.

In Compass, currently, the way to create a Resource (usually when using RSEM) is using the CompassSesssion API. Here is a quick code example of how it is done:

1
2
3

Resource resource = session.createResource("alias");
resource.addProperty("id", 1);
session.create(resource);

This is a bit confusing, especially when it comes to resource creation. Some people confuse the createResource method (which instantiates a resource) with the create method which creates it in the search engine index.

This is why, in version 2.0 (its committed on trunk), this has been abstracted to a ResourceFactory, which is accessible through the Compass instance. Here is a revised example that makes use of it:

1
2
3
4
5
6

Resource resource = compass.getResourceFactory().createResource("alias");
resource.addProperty("id", 1);
 
// open session, begin transaction
session.create(resource);
// commit transaction, close session

This breaks backward compatibility at the expense of greater clarity. It is fairly simple to keep the old behavior as well, but it really cleans Compass up not supporting the old method. What do you say? If it really messes your application, just comment or send me an email.

Mathieu posted that he created a really nice Compass server based on a set of libraries he developed and has it working with php. Check it out here.