Comments on: Lucene and Amazon S3

By: Troy Tolle

Troy Tolle — Wed, 28 Nov 2007 16:54:24 +0000

If you are trying to use Lucene and it’s indexing there are a couple of things that I would suggest. You will want to check out JSR 170 and possibly writing your own Node type. There are already built in Node types for files, but they are easily extended to persist to other places than the file system. This has several advantages. JSR 170 will allow you to change as the underlying S3 api changes and it will allow you to move away from S3 if Amazon really starts to jack up their prices without changing the business logic. You also might want to check the numbers again on the “free” from EC2. You don’t have to pay for the bandwidth used, but you do have to pay for every request under the new pricing scheme. It is still cheap, but not free. If you are planning on indexing often or having a ton of requests this is something to account for when in design.

By: Emmanuel Bernard

Emmanuel Bernard — Mon, 19 Nov 2007 13:26:05 +0000

There is no out of the box pessimistic locking API for JGroups (though it is doable to write one). But the JGroups / JBoss Cache guys want to add such an API soonish. I don’t know if it will sit in JGroups, JBoss Cache or both but that will definitely make life easier for people who have such needs :)

By: Shay Banon

Shay Banon — Mon, 19 Nov 2007 08:09:35 +0000

I can build a distributed lock system, for example using JGroups or something like that. I am not sure how this is going to hold within EC2. Would you be interested in having the ability to store things on Amazon S3 and have a distributed locks using JGroups?

By: ursuletzu

ursuletzu — Sun, 18 Nov 2007 21:55:47 +0000

Hi, interesting stuff, i intend to use s3 also.
But, why not using other storage for locks and simulating a distributed transaction for both the storage and s3?
Hm…Don’t shelve it.