C# (CSharp) Lucene.Net.Store 네임스페이스

중첩된 네임스페이스

Lucene.Net.Store.Azure

클래스들

이름 설명
BufferedIndexInput Base implementation class for buffered IndexInput.
BufferedIndexOutput Base implementation class for buffered {@link IndexOutput}.
ByteArrayDataOutput DataOutput backed by a byte array. WARNING: this class omits most low-level checks, so be sure to test heavily with assertions enabled. @lucene.experimental
ChecksumIndexOutput
CompoundFileDirectory Class for accessing a compound stream. this class implements a directory, but is limited to only read operations. Directory methods that would normally modify data throw an exception.

All files belonging to a segment have the same name with varying extensions. The extensions correspond to the different file formats used by the Codec. When using the Compound File format these files are collapsed into a single .cfs file (except for the LiveDocsFormat, with a corresponding .cfe file indexing its sub-files.

Files:

  • .cfs: An optional "virtual" file consisting of all the other index files for systems that frequently run out of file handles.
  • .cfe: The "virtual" compound file's entry table holding all entries in the corresponding .cfs file.

Description:

  • Compound (.cfs) --> Header, FileData FileCount
  • Compound Entry Table (.cfe) --> Header, FileCount, <FileName, DataOffset, DataLength> FileCount, Footer
  • Header --> CodecUtil#writeHeader CodecHeader
  • FileCount --> DataOutput#writeVInt VInt
  • DataOffset,DataLength --> DataOutput#writeLong UInt64
  • FileName --> DataOutput#writeString String
  • FileData --> raw file data
  • Footer --> CodecUtil#writeFooter CodecFooter

Notes:

  • FileCount indicates how many files are contained in this compound file. The entry table that follows has that many entries.
  • Each directory entry contains a long pointer to the start of this file's data section, the files length, and a String with that file's name.
@lucene.experimental
CompoundFileDirectory.FileEntry Offset/Length for a slice inside of a compound file
CompoundFileDirectory.IndexInputSlicerAnonymousInnerClassHelper
DataInput Abstract base class for performing read operations of Lucene's low-level data types.

{@code DataInput} may only be used from one thread, because it is not thread safe (it keeps internal state like file position). To allow multithreaded use, every {@code DataInput} instance must be cloned before used in another thread. Subclasses must therefore implement #clone(), returning a new {@code DataInput} which operates on the same underlying resource, but positioned independently.

DataOutput Abstract base class for performing write operations of Lucene's low-level data types.

{@code DataOutput} may only be used from one thread, because it is not thread safe (it keeps internal state like file position).

FSDirectory Base class for Directory implementations that store index files in the file system. There are currently three core subclasses: SimpleFSDirectory is a straightforward implementation using java.io.RandomAccessFile. However, it has poor concurrent performance (multiple threads will bottleneck) as it synchronizes when multiple threads read from the same file. NIOFSDirectory uses java.nio's FileChannel's positional io when reading to avoid synchronization when reading from the same file. Unfortunately, due to a Windows-only Sun JRE bug this is a poor choice for Windows, but on all other platforms this is the preferred choice. Applications using System.Threading.Thread.Interrupt() or Future#cancel(boolean) (on Java 1.5) should use SimpleFSDirectory instead. See NIOFSDirectory java doc for details. MMapDirectory uses memory-mapped IO when reading. This is a good choice if you have plenty of virtual memory relative to your index size, eg if you are running on a 64 bit JRE, or you are running on a 32 bit JRE but your index sizes are small enough to fit into the virtual memory space. Java has currently the limitation of not being able to unmap files from user code. The files are unmapped, when GC releases the byte buffers. Due to this bug in Sun's JRE, MMapDirectory's IndexInput.Close is unable to close the underlying OS file handle. Only when GC finally collects the underlying objects, which could be quite some time later, will the file handle be closed. This will consume additional transient disk usage: on Windows, attempts to delete or overwrite the files will result in an exception; on other platforms, which typically have a "delete on last close" semantics, while such operations will succeed, the bytes are still consuming space on disk. For many applications this limitation is not a problem (e.g. if you have plenty of disk space, and you don't rely on overwriting files on Windows) but it's still an important limitation to be aware of. This class supplies a (possibly dangerous) workaround mentioned in the bug report, which may fail on non-Sun JVMs. Applications using System.Threading.Thread.Interrupt() or Future#cancel(boolean) (on Java 1.5) should use SimpleFSDirectory instead. See MMapDirectory java doc for details. Unfortunately, because of system peculiarities, there is no single overall best implementation. Therefore, we've added the Open(System.IO.DirectoryInfo) method, to allow Lucene to choose the best FSDirectory implementation given your environment, and the known limitations of each implementation. For users who have no reason to prefer a specific implementation, it's best to simply use FSDirectory.Open(System.IO.DirectoryInfo) . For all others, you should instantiate the desired implementation directly.

The locking implementation is by default NativeFSLockFactory , but can be changed by passing in a custom LockFactory instance.

FSDirectory.FSIndexOutput Writes output with RandomAccessFile#write(byte[], int, int)
FileSwitchDirectory Expert: A Directory instance that switches files between two other Directory instances.

Files with the specified extensions are placed in the primary directory; others are placed in the secondary directory. The provided Set must not change once passed to this class, and must allow multiple threads to call contains at once.

@lucene.experimental
IOContext IOContext holds additional details on the merge/search context. A IOContext object can never be initialized as null as passed as a parameter to either Lucene.Net.Store.Directory#openInput(String, IOContext) or Lucene.Net.Store.Directory#createOutput(String, IOContext)
IndexInput Abstract base class for input from a file in a {@link Directory}. A random-access input stream. Used for all Lucene index input operations.
Lock An interprocess mutex lock.

Typical use might look like:

 new Lock.With(directory.makeLock("my.lock")) { public Object doBody() { ... code to execute while locked ... } }.run(); 
Lock.With Utility class for executing code with exclusive access.
MMapDirectory File-based Directory implementation that uses mmap for reading, and {@link FSDirectory.FSIndexOutput} for writing.

NOTE: memory mapping uses up a portion of the virtual memory address space in your process equal to the size of the file being mapped. Before using this class, be sure your have plenty of virtual address space, e.g. by using a 64 bit JRE, or a 32 bit JRE with indexes that are guaranteed to fit within the address space. On 32 bit platforms also consult #MMapDirectory(File, LockFactory, int) if you have problems with mmap failing because of fragmented address space. If you get an OutOfMemoryException, it is recommended to reduce the chunk size, until it works.

Due to this bug in Sun's JRE, MMapDirectory's IndexInput#close is unable to close the underlying OS file handle. Only when GC finally collects the underlying objects, which could be quite some time later, will the file handle be closed.

this will consume additional transient disk usage: on Windows, attempts to delete or overwrite the files will result in an exception; on other platforms, which typically have a "delete on last close" semantics, while such operations will succeed, the bytes are still consuming space on disk. For many applications this limitation is not a problem (e.g. if you have plenty of disk space, and you don't rely on overwriting files on Windows) but it's still an important limitation to be aware of.

this class supplies the workaround mentioned in the bug report (see #setUseUnmap), which may fail on non-Sun JVMs. It forcefully unmaps the buffer on close by using an undocumented internal cleanup functionality. #UNMAP_SUPPORTED is true, if the workaround can be enabled (with no guarantees).

NOTE: Accessing this class either directly or indirectly from a thread while it's interrupted can close the underlying channel immediately if at the same time the thread is blocked on IO. The channel will remain closed and subsequent access to MMapDirectory will throw a ClosedChannelException.

MMapDirectory.AnonymousClassPrivilegedExceptionAction
MMapDirectory.IndexInputSlicerAnonymousInnerClassHelper
MMapDirectory.MMapIndexInput
MMapDirectory.MultiMMapIndexInput
MockDirectoryWrapper this is a Directory Wrapper that adds methods intended to be used only by unit tests. It also adds a number of features useful for testing:
  • Instances created by LuceneTestCase#newDirectory() are tracked to ensure they are closed by the test.
  • When a MockDirectoryWrapper is closed, it will throw an exception if it has any open files against it (with a stacktrace indicating where they were opened from).
  • When a MockDirectoryWrapper is closed, it runs CheckIndex to test if the index was corrupted.
  • MockDirectoryWrapper simulates some "features" of Windows, such as refusing to write/delete to open files.
MockDirectoryWrapper.BufferedIndexOutputWrapper
MockDirectoryWrapper.Failure Objects that represent fail-able conditions. Objects of a derived class are created and registered with the mock directory. After register, each object will be invoked once for each first write of a file, giving the object a chance to throw anSystem.IO.IOException.
MockDirectoryWrapper.IndexInputSlicerAnonymousInnerClassHelper
MockIndexInputWrapper Used by MockDirectoryWrapper to create an input stream that keeps track of when it's been closed.
MockRAMDirectory
MockRAMDirectory.Failure
MockRAMInputStream Used by MockRAMDirectory to create an input stream that keeps track of when it's been closed.
MockRAMOutputStream Used by MockRAMDirectory to create an output stream that will throw an IOException on fake disk full, track max disk space actually used, and maybe throw random IOExceptions.
NIOFSDirectory An FSDirectory implementation that uses java.nio's FileChannel's positional read, which allows multiple threads to read from the same file without synchronizing.

this class only uses FileChannel when reading; writing is achieved with FSDirectory.FSIndexOutput.

NOTE: NIOFSDirectory is not recommended on Windows because of a bug in how FileChannel.read is implemented in Sun's JRE. Inside of the implementation the position is apparently synchronized. See here for details.

NOTE: Accessing this class either directly or indirectly from a thread while it's interrupted can close the underlying file descriptor immediately if at the same time the thread is blocked on IO. The file descriptor will remain closed and subsequent access to NIOFSDirectory will throw a ClosedChannelException. If your application uses either Thread#interrupt() or Future#cancel(boolean) you should use SimpleFSDirectory in favor of NIOFSDirectory.

NIOFSDirectory.IndexInputSlicerAnonymousInnerClassHelper
NIOFSDirectory.NIOFSIndexInput Reads bytes with FileChannel#read(ByteBuffer, long)
NativeFSLock
NativeFSLockFactory

Implements {@link LockFactory} using native OS file locks. Note that because this LockFactory relies on java.nio.* APIs for locking, any problems with those APIs will cause locking to fail. Specifically, on certain NFS environments the java.nio.* locks will fail (the lock can incorrectly be double acquired) whereas {@link SimpleFSLockFactory} worked perfectly in those same environments. For NFS based access to an index, it's recommended that you try {@link SimpleFSLockFactory} first and work around the one limitation that a lock file could be left when the JVM exits abnormally.

The primary benefit of {@link NativeFSLockFactory} is that lock files will be properly removed (by the OS) if the JVM has an abnormal exit.

Note that, unlike {@link SimpleFSLockFactory}, the existence of leftover lock files in the filesystem on exiting the JVM is fine because the OS will free the locks held against these files even though the files still remain.

If you suspect that this or any other LockFactory is not working properly in your environment, you can easily test it by using {@link VerifyingLockFactory}, {@link LockVerifyServer} and {@link LockStressTest}.

RAMDirectory A memory-resident Directory implementation. Locking implementation is by default the SingleInstanceLockFactory but can be changed with #setLockFactory.

Warning: this class is not intended to work with huge indexes. Everything beyond several hundred megabytes will waste resources (GC cycles), because it uses an internal buffer size of 1024 bytes, producing millions of {@code byte[1024]} arrays. this class is optimized for small memory-resident indexes. It also has bad concurrency on multithreaded environments.

It is recommended to materialize large indexes on disk and use MMapDirectory, which is a high-performance directory implementation working directly on the file system cache of the operating system, so copying data to Java heap space is not useful.

RateLimiter Abstract base class to rate limit IO. Typically implementations are shared across multiple IndexInputs or IndexOutputs (for example those involved all merging). Those IndexInputs and IndexOutputs would call #pause whenever they want to read bytes or write bytes.
RateLimiter.SimpleRateLimiter Simple class to rate limit IO.
SimpleFSDirectory A straightforward implementation of {@link FSDirectory} using java.io.RandomAccessFile. However, this class has poor concurrent performance (multiple threads will bottleneck) as it synchronizes when multiple threads read from the same file. It's usually better to use {@link NIOFSDirectory} or {@link MMapDirectory} instead.
SimpleFSDirectory.IndexInputSlicerAnonymousInnerClassHelper
SimpleFSDirectory.SimpleFSIndexInput
SlowClosingMockIndexInputWrapper hangs onto files a little bit longer (50ms in close). MockDirectoryWrapper acts like windows: you can't delete files open elsewhere. so the idea is to make race conditions for tiny files (like segments) easier to reproduce.
SlowOpeningMockIndexInputWrapper Takes a while to open files: gives testThreadInterruptDeadlock a chance to find file leaks if opening an input throws exception
TestBufferedIndexInput
TestBufferedIndexInput.MockFSDirectory
TestBufferedIndexInput.MyBufferedIndexInput
TestDirectory
TestDirectory.TheThread
TestDirectory.TheThread2
TestFileSwitchDirectory
TestHelper this class provides access to package-level features defined in the store package. It is used for testing only.
TestHugeRamFile
TestHugeRamFile.DenseRAMFile Fake a huge ram file by using the same byte buffer for all buffers under maxint.
TestLock
TestLock.LockMock
TestLockFactory
TestLockFactory.MockLockFactory
TestLockFactory.MockLockFactory.MockLock
TestLockFactory.SearcherThread
TestLockFactory.WriterThread
TestMockDirectoryWrapper
TestMultiMMap
TestNRTCachingDirectory
TestRAMDirectory
TestRAMDirectory.AnonymousClassThread
TestRAMDirectory.ThreadAnonymousInnerClassHelper
TestWindowsMMap
TrackingDirectoryWrapper A delegating Directory that records which files were written to and deleted.