JSON Mappings with Compass

2008 June 27

by Shay Banon

Upcoming Compass 2.1 M1 will support native JSON indexing (on top of Object, XML, and Resource). This means that using simple mappings, JSON strings can be indexed using Compass API as well as provide search capabilities over them.

As an example, lets take the following simple JSON:

{   
    "id": 1,
    "name": "Mary Lebow",
    "address": {
      "street": "5 Main Street"
      "city": "San Diego, CA",
      "zip": 91912,
    },
    "phoneNumbers": [
      "619 332-3452",
      "664 223-4667"
    ]
  }
}

We can then map using using the following (fully explicit) mappings:

<root-json-object alias="addressbook">
    <json-id name="id" />
    <json-property name="name" />
    <json-object name="address">
        <json-property name="street" />
        <json-property name="city" />
        <json-property name="zip" index="un_tokenized" />
        <json-array name="phoneNumbers" index-name="phoneNumber">
            <json-property />
        </json-array>
    </json-object>
</root-json-object>

Or use the less explicit mapping and let Compass recursively add JSON elements to the index:

1
2
3

<root-json-object alias="addressbook" dynamic="true">
    <json-id name="id" />
</root-json-object>

Once we mapped the JSON elements under a given alias, we can then index it and search on it. Here is an example:

JsonObject jsonObject = new DefaultAliasedJSONObject("addressbook", "json string goes here");
// this will index the provided JSON
session.save(jsonObject);
 
// now we can load the Resource that represents it
Resource resource = session.loadResource("addressbook", 1);
resource.getValue("name"); // will get Mary Lebow
 
// we can also get back the JSON content and actual object when using content mapping (see later)
jsonObject = (JsonObject) session.load("addressbook", 1);
 
// Last, we can search
CompassHits hits = session.find("mary");
hits.lenght() // will print one
resource = hits.resource(0);
jsonObject = (JsonObject) hits.data(0);

In terms of JSON object model implementations, Compass comes with a built in one based on json.org. It also integrates with Grails and Jettison implementations of JSON object model. Other object models can be easily implemented as well (any you would like to see?)

That was one of the main features I wanted to add to Compass 2.1, and am happy I manged to get it into Compass 2.1 first milestone release. 2.1 M1 will be released in the upcoming days.

9 Responses leave one →

2008 June 27

Kenny MacLeod permalink

Upcoming Compass 2.0 M1 will support native JSON indexing

2.1 M1, shurely….

Reply
2008 June 27

Shay Banon permalink

Yep :), I will fix this…

Reply
2008 June 28

Lukas Vlcek permalink

This is interesting. It makes me think that Compass could be easily used in similar context like Solr. Users could push new content into index via http. The only thing one has to add would be some simple web server which can accept JSON objects (these will be sent to Compass) and user queries. I am not an expert on Solr but do you see any reason why Compass should not be use in similar manner?

Reply
2008 June 28

Shay Banon permalink

Actually no :). I was planning to blog exactly about this. This, with the support for dynamic mappings additions/removal, allows to build a generic Compass server that accepts JSON/XML queries. It would be a really nice addition to Compass, and I am looking for volunteers …

Reply
2008 June 28

Lukas Vlcek permalink

Then why don’t you go ahead and blog about this ;-) Or even better, let’s create a new CMP ticket. You probably have some kind of design concept in your head already so let’s write it down. I am really curious about indexing/searching performance of such service.

Reply
2008 August 27

Lukas Vlcek permalink

Shay, one more question. I was thinking about this this morning and I don’t see a simple way how I can set boosting factors for individual properties in JSON. Are all properties handled equaly (i.e. boost=1.0f)? And how about other parameters? In your example there is explicitly stated that the ZIP code is un_tokenized (in XML mapping) but I don’t see it in dynamic JSON mapping. (Note: I haven’t check the latest Compass documentation so ignore my comment if it is explained there)

Reply
2008 August 27

Shay Banon permalink

Hi Lukas,

I suggest asking this questions in the forum so other people would benefit this information. You can use the json-property mapping, and there, you can specify all the things you’ve come to know from the meta-data mapping with OSEM. Look at the reference docs for all the things you can configure.

Reply
2009 February 16

Tushar Khairnar permalink

Hi,

I tried the code above : but last line

jsonObject = (JsonObject) hits.data(0);

does not work by default. you have to add to back get exact json dynamic string.
At First, it did not work but after reading reference guide it worked.
Nice that it was documented.

Regards,
Tushar

Reply