Quantum Physics and Data Grids

2009 May 6
tags:
by Shay Banon

One of quantum physics’ crazier notions is that two particles seem to communicate with each other instantly, even when they’re billions of miles apart. Albert Einstein, arguing that nothing travels faster than light, dismissed this as impossible “spooky action at a distance.”

The great man may have been wrong. A series of recent mind-bending laboratory experiments has given scientists an unprecedented peek behind the quantum veil, confirming that this realm is as mysterious as imagined.

quantum1
quantum2

Based off this theory, there has been several computer science related experiments, especially revolving around Quantum Computer and different encryption algorithms that I won’t get into in this blog post. What I would like to suggest is how Quantum Physics can revolutionize the area of (In Memory ;) ) Data Grids.

One of the main problems of Data Grids is the ability to replicate state changes from one instance to the other, especially when using WAN. This highly ties into Brewer’s CAP Theorem, which states that:

When designing distributed web services, there are three properties that are commonly desired: consistency, availability, and partition tolerance. It is impossible to achieve all three. In this note, we prove this conjecture in the asynchronous network model, and then discuss solutions to this dilemma in the partially synchronous model.

What I am suggesting is that once the ability (and it very close, there are already companies building highly secure computer systems using Quantum Physics), CAP theorem will no longer be applicable.

As a thought experiment, imagine that a photon has an up spin, and that represents binary 1 (I know, it can get much more advance than that, I am simplifying things). A down spin represent binary 0. Once photons are “entangled” (we bring up our data grid), and then we separate them (across the building or across the ocean, does not really matter), we can get “instantaneous replication”. Once we change the state of one photon, the other will change its state instantaneously (only when we check its state, but that is when we really care about it ;) ). By exhibiting this behavior, we actually can get all three properties of CAP theorem.

Imagine as well the ability to store a “local cache” of the data. Since the size of data that can be “store” is exponentially bigger than current technology, and the fact that “state change” is not bounded by current technology (no need for wires), most people can have most of the data locally most of the time (which in itself, is relative). Once local data is changed, there is no need for 2PC or something like that in order to update the master data. For all intent and purposes, we hold the master data :).

Quantum Physics is going to revolutionize the way we go about and use technology. What I talked about is just the tip of the iceberg, but I personally believe that once the technology starts maturing, the impact it will have on the world will dwarf the arrival of computers, the industrial revolution, or any other major event that occurred in our not so long history.

One of Einstein famous quotes regarding Quantum physics is “God doesn’t play dice with the universe”. I personally like better Neils Bohr, a big proponent of quantum uncertainty, rebuttal: “Quit telling God what to do.”

Compass moves back form Google Group

2009 May 5
tags:
by Shay Banon

Seems like several Compass users did not like the move to Google groups (can’t access it from work and so on), so we are back to the old forums. Sorry for the trouble…

Cheers,
Shay

Searchable Google AppEngine with Compass

2009 April 9
by Shay Banon

appengine Google App Engine was released yesterday for Java. It provides the ability to deploy Java based web applications onto the App Engine, and provides a data store service (with JPA and JDO on top of it), memcached, and others.

So, what I decided to do is try and get Compass integrated with Google App Engine to allow for easy full text search of Google App Engine apps (yea, the irony). Basically, it took me a couple of hours (most of them spent on this bug in gae), and we have something ready to use. Here is a screencast showing how the Guestbook app can be made searchable easily (use full screen mode when code is shown):

What are the steps needed?

  • Make your domain model searchable.
  • Create a Compass instance. Configure it to store the index using GAE data store (there is also support for native Lucene applications). Configure it not to use threads.
  • Create a Compass Gps, have it automatically mirror changes done using the JDO/JPA API to the search engine. Call index operation to completely reindex your application.

Thats it. Pretty simple no? Especially since these are the steps you use to enable any Java application with full text search using Compass :).

Of course, there are some leftovers. The first is better support for JPA (built on top of Compass generic support for JPA). Another enhancement is to use memcache service to cache heavily used index portions.

Enjoy!

Compass 2.2.0 GA Released

2009 April 7
tags:
by Shay Banon

breaking-news Compass version 2.2.0 GA released. There are many features included in the release which I covered in the different milestone and release candidates announcements. Here is a link to the user group announcement.

I already have several things lined up for Compass 3.0, with the first thing on the plan is a complete restructuring of the codebase to a more modular one (should not be a breaking change, maybe some package renaming, will try to keep that to a minimum). Compass integrates with many libraries now, and its time to make the structure/build more modular (it is already mostly modular internally). Will report back on how it goes.

Enjoy!

p.s. Compass is now on maven repo central, all maven lovers rejoice!.

Compass 2.1.4 Released

2009 April 6
tags:
by Shay Banon

hot-off-the-press1 Compass version 2.1.4 released. The version fixes one major bug revolving around how Compass manages read committed transactions, it is highly recommended to upgrade to it.

Enjoy!

Compass Forum Moving to Google Group

2009 April 4
tags:
by Shay Banon

Compass forums have moved to Google Groups, the group name is compass-project-user, and the link is http://groups.google.com/group/compass-project-user.

Why? Well, basically, Jive is no more, and it seems like Clear Space or something like that is the new way for the company to go. Also, no more open source license. So, I thought it was a good time to move to Google groups.

I managed to migrate most of the forum posts to Google groups, but, if you want to be sure that I will answer a question, please repost a question you have again. Sorry for the inconvenience.

Cheers,
Shay

Compass 2.2.0 RC2 Released

2009 March 27
tags:
by Shay Banon

hot-off-the-press1 Compass version 2.2.0 RC2 released. The version fixes several bugs, including the two bugs described in the 2.1.3 post.

Full release notes can be found here.

Enjoy!

Compass 2.1.3 Released

2009 March 26
tags:
by Shay Banon

hot-off-the-press1 Compass version 2.1.3 released. The version fixes two important bugs. The first fixes a problem when using Compass support unmarshall flag set to false (OSEM) and using analyzer property controller or boost property controller (basically, it did not work…).

The second one revolves around Compass integration with TopLink and EclipseLink. Basically, with complex hierarchy of mappings, objects would be created twice in the index. It boiled down to how TopLink and EclipseLink fire up events for such cases. An event on class B that extends A is also fired for A…, so Compass needs to take that into account when registering event listeners. Surprisingly, there was an extra fix required for EclipseLink, as the logic of firing events changed a bit between EclipseLink and TopLink.

Full release notes can be found here.

Enjoy!.

Compass 2.2.0 RC1 Released

2009 March 10
tags:
by Shay Banon

breaking-news Compass version 2.2.0 RC1 released. Main features include simplified index and search session, mt transaction processor, and dynamic properties. Full release notes can be found here.

Enjoy!

Compass 2.1.2 Released

2009 March 9
tags:
by Shay Banon

hot-off-the-press1 Compass version 2.1.2 released. Mainly includes bug fixes and upgrade to Lucene 2.4.1 (from Lucene 2.4.0). Full release notes can be found here.

Enjoy!.