Hadoop RawLocalFileSystem and getPos
I've found that the getPos in the RawLocalFileSystem's input stream can
throw a null pointer exception if its underlying stream is closed.
I discovered this when playing with a custom record reader.
to patch it, I simply check if a call to "stream.available()" throws an
exception, and if so, I return 0 in the getPos() function.
The existing getPos() implementation is found here:
https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20/src/examples/org/apache/hadoop/examples/MultiFileWordCount.java
What should be the correct behaviour of getPos() in the RecordReader?
No comments:
Post a Comment