Compass 2.0 already comes with an integration between GigaSpaces and Lucene/Compass. The integration allows to store the Lucene index (using Lucene Directory) abstraction on top of GigaSpaces. It also allows to use GigaSpaces mirror services allowing to automatically index the content of the data grid (Space) in an asynchronous reliable manner.

The current integration provides great value for both Lucene and Compass users, and it is already used by several users. But, there is still something missing. A big part of GigaSpaces data grid solution is the ability to run collocated services within cluster members, and query them in a distributed manner. This type of integration fits very nicely to the indexing and search requirements by Compass users. Here is a diagram of how this should work:

Lets explain how the integration works by following the flow of operations. The first operation is writing POJOs to the Space (data grid):

1. A POJO is written to the Space using a clustered proxy of the Space.
2. The POJO is directed to one of the partitions (automatically by GigaSpaces).
3. The POJO is written to the collocated cluster member.
4. The indexing service is notified that something changed in the Space (based on POJO templates).
5. The indexing services applies the changes (assuming they have Compass mappings) using Compass API.
6. Compass applies the change (save/delete). The Compass instance stores the Luceen index on the collocated Space (the partition, which might or might not have a backup).

The search operation works as follows:

1. Using a simple POJO search service (CompassSearchService) the search is performed on the clietn side.
2. The search is automatically broadcast to all active partitions and executed on each in an embedded manner.
3. Results that match the query are returned to the client from each partition.
4. The internal client implementation automatically reorders and reduce the results and returns them to the client.

This type of integration takes collocation of indexing and searching to a new level. Indexing and Search operations are performed in a collocated manner in memory making them extremely fast. Scalability is easily handled by adding more partitions, and high availability is provided by adding backups to each partition.

The integration itself can be used both by Compass users and GigaSpaces users. Compass users can use the integration to scale their search integration (use GigaSpace API to store/delete the domain model, and the search API to search). GigaSpaces users can, seamlessly, add google like search support to their current integration with GigaSpaces.

More information can be found in the reference docs (here is a link to the nightly build docs) and this feature is available from tonight nightly builds. The feature in Compass Jira is CMP-666, muhahaha :).

Upcoming Compass 2.1 M1 will support native JSON indexing (on top of Object, XML, and Resource). This means that using simple mappings, JSON strings can be indexed using Compass API as well as provide search capabilities over them.

As an example, lets take the following simple JSON:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

{   
    "id": 1,
    "name": "Mary Lebow",
    "address": {
      "street": "5 Main Street"
      "city": "San Diego, CA",
      "zip": 91912,
    },
    "phoneNumbers": [
      "619 332-3452",
      "664 223-4667"
    ]
  }
}

We can then map using using the following (fully explicit) mappings:

1
2
3
4
5
6
7
8
9
10
11
12

<root-json-object alias="addressbook">
    <json-id name="id" />
    <json-property name="name" />
    <json-object name="address">
        <json-property name="street" />
        <json-property name="city" />
        <json-property name="zip" index="un_tokenized" />
        <json-array name="phoneNumbers" index-name="phoneNumber">
            <json-property />
        </json-array>
    </json-object>
</root-json-object>

Or use the less explicit mapping and let Compass recursively add JSON elements to the index:

1
2
3

<root-json-object alias="addressbook" dynamic="true">
    <json-id name="id" />
</root-json-object>

Once we mapped the JSON elements under a given alias, we can then index it and search on it. Here is an example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

JsonObject jsonObject = new DefaultAliasedJSONObject("addressbook", "json string goes here");
// this will index the provided JSON
session.save(jsonObject);
 
// now we can load the Resource that represents it
Resource resource = session.loadResource("addressbook", 1);
resource.getValue("name"); // will get Mary Lebow
 
// we can also get back the JSON content and actual object when using content mapping (see later)
jsonObject = (JsonObject) session.load("addressbook", 1);
 
// Last, we can search
CompassHits hits = session.find("mary");
hits.lenght() // will print one
resource = hits.resource(0);
jsonObject = (JsonObject) hits.data(0);

In terms of JSON object model implementations, Compass comes with a built in one based on json.org. It also integrates with Grails and Jettison implementations of JSON object model. Other object models can be easily implemented as well (any you would like to see?)

That was one of the main features I wanted to add to Compass 2.1, and am happy I manged to get it into Compass 2.1 first milestone release. 2.1 M1 will be released in the upcoming days.

One of the upcoming Compass 2.1 M1 features include a much requested feature by users which is the ability to dynamically remove and add mappings using the Compass instance. So, Compass instance now allows to get its internal ComapssConfiguration object and change it dynamically (not just mappings, but other settings as well). One changes, Compass#rebuild() should be called in order to take the changes into account. Here is an example:

1
2
3
4
5

Compass compass = // create the compass instance
 
compass.getConfig().addClass(NewClass.class);
compass.getConfig().removeMappingByClass(OldClass.class);
compass.rebuild();

From now on, any operations on the new Compass instance will use the updated mappings (and optional settings). Underneath, Compass shuts down the “old Compass instance” gracefully (waiting for current open sessions to be closed).

TSS at Prague was great. Great sessions, always great meeting up with people from all over the world. Sadly, I did not manage to get to too many sessions, had to work a bit on GigaSpaces upcoming 6.5 GA release, but the ones I went to were really good.

I also did a tech talk with Jason Carriera. I was talking about next generation application servers and Jason was talking about his experience working with GigaSpaces (he is working on a really cool app, I won’t spoil the surprise) and also about the OpenSpaces developer contest where he won the third place.

I also gave a presentation titles “Beyond a Data Grid” where I try to talk about things that can be done with a DataGrid beyond your typical Cache#put and Cache#get mainly through examples. Here is the presentation (mostly pictures, but still ;) ):

| View | Upload your own

Compass version 2.0.1 released. This is a bug fix released for Compass 2.0 and is recommended for all Compass users. Full change log can be found here.

While waiting (yes, another sleepless night) for the second game in the Lakers Vs. Celtics NBA finals, I remembered one of my all time favorite arcade games that I played when I was a child. It was an NBA arcade game called Lakers versus Celtics and it was one of the main reasons for my love for the NBA (and the Lakers).

So, while waiting for the game, I managed to run it on my macbook pro. Here is what I needed to do: First, download DOSBox. Next, head over and download the game here. Using DOSBox is pretty simple. You start it, and then mount c ~/path/where/you/undzip/the/game. So much fun!. Warcraft 2 anyone? :).