ES 使用Lucene 修復(fù)分片問題

Elasticsearch分片包含一個(gè)Lucene索引,因此我們可以使用Lucene的CheckIndex工具,該工具使我們能夠掃描并修復(fù)有問題的片段,而且通常只會(huì)造成最小的數(shù)據(jù)丟失。

Lucene CheckIndex工具包含在默認(rèn)的Elasticsearch發(fā)行版中,不需要額外的下載。

# change this to reflect your shard path, the format is
# {path.data}/{cluster_name}/nodes/{node_id}/indices/{index_name}/{shard_id}/index/

$ export SHARD_PATH=data/elasticsearch/nodes/0/indices/foo/0/index/
$ java -cp lib/elasticsearch-*.jar:lib/*:lib/sigar/* -ea:org.apache.lucene... org.apache.lucene.index.CheckIndex $SHARD_PATH

如果CheckIndex檢測(cè)到問題并且其修復(fù)建議看起來(lái)很明智,則可以通過添加-fix命令行參數(shù)來(lái)告訴CheckIndex應(yīng)用修補(bǔ)程序。

實(shí)際是用 -exorcise 命令


 java -cp lib/elasticsearch-*.jar:lib/*:lib/sigar/* -ea:org.apache.lucene... org.apache.lucene.index.CheckIndex

ERROR: index path not specified
Usage: java org.apache.lucene.index.CheckIndex pathToIndex [-exorcise] [-crossCheckTermVectors] [-segment X] [-segment Y] [-dir-impl X]

  -exorcise: actually write a new segments_N file, removing any problematic segments
  -fast: just verify file checksums, omitting logical integrity checks
  -crossCheckTermVectors: verifies that term vectors match postings; THIS IS VERY SLOW!
  -codec X: when exorcising, codec to write the new segments_N file with
  -verbose: print additional details
  -segment X: only check the specified segments.  This can be specified multiple
              times, to check more than one segment, eg '-segment _2 -segment _a'.
              You can't use this with the -exorcise option
  -dir-impl X: use a specific FSDirectory implementation. If no package is specified the org.apache.lucene.store package will be used.

**WARNING**: -exorcise *LOSES DATA*. This should only be used on an emergency basis as it will cause
documents (perhaps many) to be permanently removed from the index.  Always make
a backup copy of your index before running this!  Do not run this tool on an index
that is actively being written to.  You have been warned!

Run without -exorcise, this tool will open the index, report version information
and report any exceptions it hits and what action it would take if -exorcise were
specified.  With -exorcise, this tool will remove any segments that have issues and
write a new segments_N file.  This means all documents contained in the affected
segments will be removed.

This tool exits with exit code 1 if the index cannot be opened or has any
corruption, else 0.

修復(fù)

當(dāng)出現(xiàn)分片損壞時(shí),應(yīng)該是分片中的某些 segment 出問題了。

解決辦法是刪掉這些 segment 用新的代替,但是這些segment中保存的數(shù)據(jù)會(huì)丟失。

處理步驟:

  1. 關(guān)閉ES

  2. 執(zhí)行

$ export SHARD_PATH=data/elasticsearch/nodes/0/indices/foo/0/index/
$ java -cp lib/elasticsearch-*.jar:lib/*:lib/sigar/* -ea:org.apache.lucene... org.apache.lucene.index.CheckIndex $SHARD_PATH

會(huì)提示有幾個(gè)壞的segment,會(huì)丟失多少數(shù)據(jù)

  1. 使用 -exorcise命令進(jìn)行數(shù)據(jù)恢復(fù)
$ java -cp lib/elasticsearch-*.jar:lib/*:lib/sigar/* -ea:org.apache.lucene... org.apache.lucene.index.CheckIndex $SHARD_PATH -exorcise
  1. 完了啟動(dòng)es即可,只是會(huì)丟失上面所說(shuō)的X條數(shù)據(jù)
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成,瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點(diǎn),簡(jiǎn)書系信息發(fā)布平臺(tái),僅提供信息存儲(chǔ)服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容