Compass 2.0 already comes with an integration between GigaSpaces and Lucene/Compass. The integration allows to store the Lucene index (using Lucene Directory) abstraction on top of GigaSpaces. It also allows to use GigaSpaces mirror services allowing to automatically index the content of the data grid (Space) in an asynchronous reliable manner.
The current integration provides great value for both Lucene and Compass users, and it is already used by several users. But, there is still something missing. A big part of GigaSpaces data grid solution is the ability to run collocated services within cluster members, and query them in a distributed manner. This type of integration fits very nicely to the indexing and search requirements by Compass users. Here is a diagram of how this should work:
Lets explain how the integration works by following the flow of operations. The first operation is writing POJOs to the Space (data grid):
The search operation works as follows:
This type of integration takes collocation of indexing and searching to a new level. Indexing and Search operations are performed in a collocated manner in memory making them extremely fast. Scalability is easily handled by adding more partitions, and high availability is provided by adding backups to each partition.
The integration itself can be used both by Compass users and GigaSpaces users. Compass users can use the integration to scale their search integration (use GigaSpace API to store/delete the domain model, and the search API to search). GigaSpaces users can, seamlessly, add google like search support to their current integration with GigaSpaces.
More information can be found in the reference docs (here is a link to the nightly build docs) and this feature is available from tonight nightly builds. The feature in Compass Jira is CMP-666, muhahaha :).