Compass version 2.1.0 M1 released. The release includes several features including a much improved XSEM support (namespace xpath, better stax integration, jdom integration), Dynamic mappings removal and additions, JSEM - JSON to Search Engine Mapping support, and Collocated integration and distributed search with GigaSpaces. It also includes several bug fixes (with the important ones backported to 2.0 branch). Enjoy!.

Compass 2.0 already comes with an integration between GigaSpaces and Lucene/Compass. The integration allows to store the Lucene index (using Lucene Directory) abstraction on top of GigaSpaces. It also allows to use GigaSpaces mirror services allowing to automatically index the content of the data grid (Space) in an asynchronous reliable manner.

The current integration provides great value for both Lucene and Compass users, and it is already used by several users. But, there is still something missing. A big part of GigaSpaces data grid solution is the ability to run collocated services within cluster members, and query them in a distributed manner. This type of integration fits very nicely to the indexing and search requirements by Compass users. Here is a diagram of how this should work:

Lets explain how the integration works by following the flow of operations. The first operation is writing POJOs to the Space (data grid):

1. A POJO is written to the Space using a clustered proxy of the Space.
2. The POJO is directed to one of the partitions (automatically by GigaSpaces).
3. The POJO is written to the collocated cluster member.
4. The indexing service is notified that something changed in the Space (based on POJO templates).
5. The indexing services applies the changes (assuming they have Compass mappings) using Compass API.
6. Compass applies the change (save/delete). The Compass instance stores the Luceen index on the collocated Space (the partition, which might or might not have a backup).

The search operation works as follows:

1. Using a simple POJO search service (CompassSearchService) the search is performed on the clietn side.
2. The search is automatically broadcast to all active partitions and executed on each in an embedded manner.
3. Results that match the query are returned to the client from each partition.
4. The internal client implementation automatically reorders and reduce the results and returns them to the client.

This type of integration takes collocation of indexing and searching to a new level. Indexing and Search operations are performed in a collocated manner in memory making them extremely fast. Scalability is easily handled by adding more partitions, and high availability is provided by adding backups to each partition.

The integration itself can be used both by Compass users and GigaSpaces users. Compass users can use the integration to scale their search integration (use GigaSpace API to store/delete the domain model, and the search API to search). GigaSpaces users can, seamlessly, add google like search support to their current integration with GigaSpaces.

More information can be found in the reference docs (here is a link to the nightly build docs) and this feature is available from tonight nightly builds. The feature in Compass Jira is CMP-666, muhahaha :).

Upcoming Compass 2.1 M1 will support native JSON indexing (on top of Object, XML, and Resource). This means that using simple mappings, JSON strings can be indexed using Compass API as well as provide search capabilities over them.

As an example, lets take the following simple JSON:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

{   
    "id": 1,
    "name": "Mary Lebow",
    "address": {
      "street": "5 Main Street"
      "city": "San Diego, CA",
      "zip": 91912,
    },
    "phoneNumbers": [
      "619 332-3452",
      "664 223-4667"
    ]
  }
}

We can then map using using the following (fully explicit) mappings:

1
2
3
4
5
6
7
8
9
10
11
12

<root-json-object alias="addressbook">
    <json-id name="id" />
    <json-property name="name" />
    <json-object name="address">
        <json-property name="street" />
        <json-property name="city" />
        <json-property name="zip" index="un_tokenized" />
        <json-array name="phoneNumbers" index-name="phoneNumber">
            <json-property />
        </json-array>
    </json-object>
</root-json-object>

Or use the less explicit mapping and let Compass recursively add JSON elements to the index:

1
2
3

<root-json-object alias="addressbook" dynamic="true">
    <json-id name="id" />
</root-json-object>

Once we mapped the JSON elements under a given alias, we can then index it and search on it. Here is an example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

JsonObject jsonObject = new DefaultAliasedJSONObject("addressbook", "json string goes here");
// this will index the provided JSON
session.save(jsonObject);
 
// now we can load the Resource that represents it
Resource resource = session.loadResource("addressbook", 1);
resource.getValue("name"); // will get Mary Lebow
 
// we can also get back the JSON content and actual object when using content mapping (see later)
jsonObject = (JsonObject) session.load("addressbook", 1);
 
// Last, we can search
CompassHits hits = session.find("mary");
hits.lenght() // will print one
resource = hits.resource(0);
jsonObject = (JsonObject) hits.data(0);

In terms of JSON object model implementations, Compass comes with a built in one based on json.org. It also integrates with Grails and Jettison implementations of JSON object model. Other object models can be easily implemented as well (any you would like to see?)

That was one of the main features I wanted to add to Compass 2.1, and am happy I manged to get it into Compass 2.1 first milestone release. 2.1 M1 will be released in the upcoming days.

One of the upcoming Compass 2.1 M1 features include a much requested feature by users which is the ability to dynamically remove and add mappings using the Compass instance. So, Compass instance now allows to get its internal ComapssConfiguration object and change it dynamically (not just mappings, but other settings as well). One changes, Compass#rebuild() should be called in order to take the changes into account. Here is an example:

1
2
3
4
5

Compass compass = // create the compass instance
 
compass.getConfig().addClass(NewClass.class);
compass.getConfig().removeMappingByClass(OldClass.class);
compass.rebuild();

From now on, any operations on the new Compass instance will use the updated mappings (and optional settings). Underneath, Compass shuts down the “old Compass instance” gracefully (waiting for current open sessions to be closed).

TSS at Prague was great. Great sessions, always great meeting up with people from all over the world. Sadly, I did not manage to get to too many sessions, had to work a bit on GigaSpaces upcoming 6.5 GA release, but the ones I went to were really good.

I also did a tech talk with Jason Carriera. I was talking about next generation application servers and Jason was talking about his experience working with GigaSpaces (he is working on a really cool app, I won’t spoil the surprise) and also about the OpenSpaces developer contest where he won the third place.

I also gave a presentation titles “Beyond a Data Grid” where I try to talk about things that can be done with a DataGrid beyond your typical Cache#put and Cache#get mainly through examples. Here is the presentation (mostly pictures, but still ;) ):

| View | Upload your own

Compass version 2.0.1 released. This is a bug fix released for Compass 2.0 and is recommended for all Compass users. Full change log can be found here.

Matt pointed out an integration between Compass and AppFuse. Cool.

Compass 2.0.0 GA has just been released. This is the final release of 2.0 and another milestone in the Compass project. Nothing much changed from RC2, just fixing some bugs as indicated in the release notes. More information regarding 2.0 release can be found in the M1, M2, M3, RC1 and RC2 posts.

Compass version 2.0.0 RC2 has been released. The release include minor bug fixes and improved support for polymorphic operations both in the API and the query string level. Check out the release notes for more information.

Ohh, and one last thing for all you Maven lovers out there, Compass now has a maven repository at repo.compass-project.org hosting both 1.2.2 and 2.0.0 RC2 versions. Hopefully later on it will get synced with maven central repository.

Compass version 2.0.0 RC1 has just been released. The release includes several major features including: Terracotta integration, Specific Gps indexing, Automatic scanning for mappings, and many many more.

This version will be the final one where major features will get into the 2.0.0 release train. The current plan is to release 2.0.0 RC2 in a week or two, and soon after (a week) release 2.0.0 GA. Both releases will only include bug fixes. This has been a long road, and 2.0.0 has become a really nice release. So please, if you are using old Compass version, take the time to upgrade to 2.0.0 RC1 and give us feedback (make sure you read the upgrade.txt file).