hbase與mapduce集成

1.從HBase讀取數(shù)據(jù)
2.將數(shù)據(jù)寫入HBase
3.結(jié)合以上兩種,用于數(shù)據(jù)遷移

關(guān)聯(lián)jar包
問題:hadoop環(huán)境變量中缺少hbase的jar包
方法一:
全部copy進(jìn)去
可能會引起jar包之間的沖突
方法二:
將需要的hbase的jar包放入Hadoop的運行環(huán)境變量中
export HBASE_HOME=/opt/moduels/hbase-0.98.6-hadoop2
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HBASE_HOME/bin/hbase mapredcp
檢驗
/opt/moduels/hadoop-2.5.0/bin/yarn jar lib/hbase-server-0.98.6-hadoop2.jar rowcounter ns1:t1

模擬場景

場景構(gòu)建
表一:
create 'stu_info','info','addinfo','work'
put 'stu_info','20180219_14001','info:name','tom'
put 'stu_info','20180219_14001','info:age','18'
put 'stu_info','20180219_14001','info:sex','male'
put 'stu_info','20180219_14001','addinfo:height','180'
put 'stu_info','20180219_14001','work:job','worker'
put 'stu_info','20180219_14002','info:name','rose'
put 'stu_info','20180219_14002','info:age','22'
put 'stu_info','20180219_14002','info:sex','female'
put 'stu_info','20180219_14002','addinfo:height','175'
put 'stu_info','20180219_14003','info:name','lee'
put 'stu_info','20180219_14003','info:age','22'
put 'stu_info','20180219_14004','info:name','tony'
put 'stu_info','20180219_14004','info:age','18'
put 'stu_info','20180219_14005','info:name','jim'
put 'stu_info','20180219_14005','info:age','19'
put 'stu_info','20180219_14006','info:name','xiaoming'
put 'stu_info','20180219_14006','info:age','20'
put 'stu_info','20180219_14006','addinfo:height','200'
表二:
create 'stu_result','info'
環(huán)境變量
修改hadoop下hadoop——env.sh文件
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/opt/moduels/hbase-0.98.6-hadoop2/lib/*

?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成,瀏覽時請結(jié)合常識與多方信息審慎甄別。
平臺聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點,簡書系信息發(fā)布平臺,僅提供信息存儲服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容