The tech talk I did with Jason Carriera talking about data grids and beyond is online. Check it out here.

Compass 2.0 already comes with an integration between GigaSpaces and Lucene/Compass. The integration allows to store the Lucene index (using Lucene Directory) abstraction on top of GigaSpaces. It also allows to use GigaSpaces mirror services allowing to automatically index the content of the data grid (Space) in an asynchronous reliable manner.

The current integration provides great value for both Lucene and Compass users, and it is already used by several users. But, there is still something missing. A big part of GigaSpaces data grid solution is the ability to run collocated services within cluster members, and query them in a distributed manner. This type of integration fits very nicely to the indexing and search requirements by Compass users. Here is a diagram of how this should work:

Lets explain how the integration works by following the flow of operations. The first operation is writing POJOs to the Space (data grid):

1. A POJO is written to the Space using a clustered proxy of the Space.
2. The POJO is directed to one of the partitions (automatically by GigaSpaces).
3. The POJO is written to the collocated cluster member.
4. The indexing service is notified that something changed in the Space (based on POJO templates).
5. The indexing services applies the changes (assuming they have Compass mappings) using Compass API.
6. Compass applies the change (save/delete). The Compass instance stores the Luceen index on the collocated Space (the partition, which might or might not have a backup).

The search operation works as follows:

1. Using a simple POJO search service (CompassSearchService) the search is performed on the clietn side.
2. The search is automatically broadcast to all active partitions and executed on each in an embedded manner.
3. Results that match the query are returned to the client from each partition.
4. The internal client implementation automatically reorders and reduce the results and returns them to the client.

This type of integration takes collocation of indexing and searching to a new level. Indexing and Search operations are performed in a collocated manner in memory making them extremely fast. Scalability is easily handled by adding more partitions, and high availability is provided by adding backups to each partition.

The integration itself can be used both by Compass users and GigaSpaces users. Compass users can use the integration to scale their search integration (use GigaSpace API to store/delete the domain model, and the search API to search). GigaSpaces users can, seamlessly, add google like search support to their current integration with GigaSpaces.

More information can be found in the reference docs (here is a link to the nightly build docs) and this feature is available from tonight nightly builds. The feature in Compass Jira is CMP-666, muhahaha :).

TSS at Prague was great. Great sessions, always great meeting up with people from all over the world. Sadly, I did not manage to get to too many sessions, had to work a bit on GigaSpaces upcoming 6.5 GA release, but the ones I went to were really good.

I also did a tech talk with Jason Carriera. I was talking about next generation application servers and Jason was talking about his experience working with GigaSpaces (he is working on a really cool app, I won’t spoil the surprise) and also about the OpenSpaces developer contest where he won the third place.

I also gave a presentation titles “Beyond a Data Grid” where I try to talk about things that can be done with a DataGrid beyond your typical Cache#put and Cache#get mainly through examples. Here is the presentation (mostly pictures, but still ;) ):

| View | Upload your own

Well, I am moving back to GigaSpaces. After a short two weeks I simply felt that something was missing (which has nothing to do with the new workplace). The people and the technology (which I have been doing for such a long time) were simply things that I could not leave behind. Anyhow, back to work…

GigaSpaces announced a week ago that it started its Startup Program. It basically allows individuals and start-up companies (which we define as companies with less than US $5 million in revenues) to use the GigaSpaces products for any purpose, including production, indefinitely - for free. There are no time, functionality or CPU limitations to this program.

This is a revolutionary move for a commercial product. GigaSpaces has been doing really well (we had, by far, out best year ever, and the year is not over yet…), especially within markets that can afford us. What we realized was that there is a big portion of innovative market that can’t afford us, but would still love to use our product, namely - statups. So, we decided to share our wealth with startups, allowing them to get the product for free, and once they get to a stage where they actually make money, and only then, pay us back.

I really think the following post at this TSS thread nails it:

This is a good move. I don’t mind paying for a product … the problem is paying upfront (when you have zero revenue and don’t know if you will ever have some).

If one day I have revenues, I think it is logical to share the wealth generated in part thanks to the software used.

I think that many startups used free/open source products because of the cost. If you get a great product and only pay license fees if you make enough $$$ I think that many people will focus on choosing the best product (not only the best product among free/open source products).

This move is going (actually already) to have many startup companies (and I hope each and every one of them is successful) to be able to use GigaSpaces. This, together with our ongoing effort of integrating with popular Open Source projects (Spring, Hibernate) is going to make us a very viable solution to such startups. On that front, we are hard at work at having Mule integration within our OpenSpaces effort (at the same level of our Spring integration). And I have been working (as a side project) at having GigaSpaces integrated with Compass and Lucene, allowing to store Lucene index within GigaSpaces distributed grid (and integrated with Compass) as well as having the ability to automatically index objects stored in the grid indexed and searched upon (using Compass OSEM support). This effort is at a rough beta stages, but if you are interested in it, drop me a line.

I was watching Steve Jobs keynote and during the keynote Steve was showing off a new widget in the new Mac OS X Leopard version called Web Clip. Here is a video demonstrating it:

And then it hit me. Web sites (and as a result the developers) have been having a lot of problems with syndication (RSS) causing difficulties both in terms of scalability and bandwidth for web sites that expose syndication (the refresh intervals on rss readers). Now, with Apple providing a web clip widget, and probably many other widget system will follow shortly after, our web sites are going to have to handle much more frequent and bigger load. Note, with RSS there are relatively simple solution, but with Web Clips there is no option to load only part of the web site, and every web clip refresh causes the whole web page to download. Also, different than RSS, there is no option to turn it off as MSDN did with RSS.

Naturally, current JEE web sites won’t be able to handle such load without caching. But as noted many times, just caching does solve the whole problem. With web clips (different than RSS), the rate of cache misses is going to be very high.

So, when such web clips widgets are going to come out, web sites are better to get ready for it. The load on a typical web site is going to grow exponentially and developers will need to find a way to solve this (remember, there is no way to turn this off!). This is, in part, what we at GigaSpaces have been working on and thinking about with our latest release, especially with our OpenSpaces solution, but it should be on all of our minds how things like that are going to be handled and solved.

p.s. And I am not even talking about the iPhone and what it brings to the table (in terms of more web enabled devices) or Ajax.

It looks like SpringOne is going to be really exciting this year.

I will be presenting Compass and also be around Nati GigaSpaces session. With Compass, we will see how simply we can add Google like search to any Spring based application. With GigaSpaces, we will be showing off our next version which one of its main feature, Open Spaces, I have been responsible for. Open Spaces is really exciting (the early access will be out very soon), based on Spring, and is going to evolve and simplify the way people do high performance computing or any typical EDA(+DataGrid) applications. Personally I believe that we are doing to EDA/DataGrid what Spring did to parts of J2EE, so come over to our booth/session and take an early peek.