[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Linux-cluster] gfs+nfs+lucene, anyone had tried?



HI, 

look at the error that happens when two nodes are writing to the index:

 java.io.IOException: Stale NFS file handle
        at java.io.RandomAccessFile.writeBytes(Native Method)
        at java.io.RandomAccessFile.write(RandomAccessFile.java:466)
        at org.apache.lucene.store.FSIndexOutput.flushBuffer(FSDirectory.java:503)
        at org.apache.lucene.store.BufferedIndexOutput.flush(BufferedIndexOutput.java:84)
        at org.apache.lucene.store.BufferedIndexOutput.close(BufferedIndexOutput.java:98)
        at org.apache.lucene.store.FSIndexOutput.close(FSDirectory.java:506)
        at org.apache.lucene.index.FieldsWriter.close(FieldsWriter.java:48)
        at org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:191)
        at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:88)
        at org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:709)
        at org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:686)
        at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:543)
     
I think in this case one node has writen to the index and the other don´t. Could it be a problem? does GFS anything with this situation?

Thanks, 

ESG


2010/4/21 Jeff Sturm <jeff sturm eprize com>

We use Lucene over GFS (no NFS), but the design of our application updates Lucene from only one node at a time.

 

In general applications that utilize POSIX locking can handle concurrent updates safely on GFS even with multiple nodes.  It wasn't clear to us whether Lucene supports this, however, and in your case NFS adds a layer to the mix.

 

From: linux-cluster-bounces redhat com [mailto:linux-cluster-bounces redhat com] On Behalf Of ESGLinux
Sent: Wednesday, April 21, 2010 8:22 AM
To: linux clustering
Subject: [Linux-cluster] gfs+nfs+lucene, anyone had tried?

 

Hi All, 

 

I´m mounting a cluster using NFS over GFS and I´m going to store a lucene index on it. 

 

There are two nodes that write in this index, and I´m worried about the index corruption.

 

So anyone have implemented something like this? any problem I can find?

 

Thanks in advance, 

 

ESG

 

 


--
Linux-cluster mailing list
Linux-cluster redhat com
https://www.redhat.com/mailman/listinfo/linux-cluster


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]