leveldb的運(yùn)行涉及到很多文件,包括manifest文件,WAL log文件,sst文件,日志文件等,為了方便進(jìn)行文件io,leveldb抽象了幾個(gè)接口
// A file abstraction for reading sequentially through a file
class SequentialFile {
public:
SequentialFile() { }
virtual ~SequentialFile();
virtual Status Read(size_t n, Slice* result, char* scratch) = 0;
// Skip "n" bytes from the file. This is guaranteed to be no
// slower that reading the same data, but may be faster.
virtual Status Skip(uint64_t n) = 0;
private:
// No copying allowed
SequentialFile(const SequentialFile&);
void operator=(const SequentialFile&);
};
// A file abstraction for randomly reading the contents of a file.
class RandomAccessFile {
public:
RandomAccessFile() { }
virtual ~RandomAccessFile();
// Safe for concurrent use by multiple threads.
virtual Status Read(uint64_t offset, size_t n, Slice* result,
char* scratch) const = 0;
private:
// No copying allowed
RandomAccessFile(const RandomAccessFile&);
void operator=(const RandomAccessFile&);
};
// A file abstraction for sequential writing. The implementation
// must provide buffering since callers may append small fragments
// at a time to the file.
class WritableFile {
public:
WritableFile() { }
virtual ~WritableFile();
virtual Status Append(const Slice& data) = 0;
virtual Status Close() = 0;
virtual Status Flush() = 0;
virtual Status Sync() = 0;
private:
// No copying allowed
WritableFile(const WritableFile&);
void operator=(const WritableFile&);
};
在具體實(shí)現(xiàn)這些接口的時(shí)候,除了RandomAccessFile類型外,leveldb都是使用標(biāo)準(zhǔn)IO庫來實(shí)現(xiàn),也就是<stdio>提供的一套函數(shù),標(biāo)準(zhǔn)IO庫有自己的緩存,leveldb沒有對磁盤文件做額外的緩存。
對RandomAccessFile 這個(gè)類型,leveldb使用了mmap來進(jìn)行內(nèi)存映射,以此提高文件IOD的性能。同時(shí)控制了能夠同時(shí)進(jìn)行內(nèi)存映射的文件數(shù)目。(為了防止內(nèi)存消耗過度?)
對于日志打印,leveldb的實(shí)現(xiàn)比較簡單。首先定義了一個(gè)接口:
// An interface for writing log messages.
class Logger {
public:
Logger() { }
virtual ~Logger();
// Write an entry to the log file with the specified format.
virtual void Logv(const char* format, va_list ap) = 0;
private:
// No copying allowed
Logger(const Logger&);
void operator=(const Logger&);
};
然后根據(jù)具體環(huán)境來實(shí)現(xiàn)這個(gè)接口,在POSIX下,實(shí)現(xiàn)的class是:
class PosixLogger : public Logger {
private:
FILE* file_;
uint64_t (*gettid_)(); // Return the thread id for the current thread
public:
PosixLogger(FILE* f, uint64_t (*gettid)()) : file_(f), gettid_(gettid) { }
virtual ~PosixLogger() {
fclose(file_);
}
virtual void Logv(const char* format, va_list ap);
}
打印日志的過程使用了變長參數(shù),這是日志的基本需求。值得注意的是,對于寫日志整個(gè)過程沒有使用額外的并發(fā)控制,主要是因?yàn)闃?biāo)準(zhǔn)IO庫<stdio>提供的讀寫函數(shù)都是線程安全的,多個(gè)線程可以對同一個(gè)FILE同時(shí)調(diào)用fwrite()。 每次寫完一條日志后,立即調(diào)用fflush()將緩存刷到操作系統(tǒng),但是并沒有調(diào)用fsync來落盤。