安裝 Hadoop 集群

參考 Docker Hadoop 集群搭建

下載 Hive

Hive下載地址
本實(shí)例下載版本： apache-hive-3.1.1

root@suoron:~# tar zxvf apache-hive-3.1.1-bin.tar.gz -C /root/hive/apache-hive-3.1.1
#掛載到 Hadoop master 容器，腳本參考: http://www.itdecent.cn/p/4bdd888fd81f
root@suoron:~# ./dynamic_mount_docker_volume.sh master /root/hive/apache-hive-3.1.1 /opt/apache-hive-3.1.1

安裝 Mysql

參考: Linux 安裝 MySQL
mysql 主要用來(lái)存儲(chǔ) Hive 元數(shù)據(jù)

• 新建數(shù)據(jù)庫(kù) hive
• 開(kāi)通遠(yuǎn)程訪問(wèn)權(quán)限

export JAVA_HOME=/usr/local/jdk1.8.0_152
export HIVE_HOME=/opt/apache-hive-3.1.1
export HADOOP_HOME=/usr/local/hadoop-2.9.1
export PATH=$PATH:$HIVE_HOME/bin

• 復(fù)制 jdbc 驅(qū)動(dòng)到 hive

root@47ba702846f0:/opt/apache-hive-3.1.1# ll lib/mysql-connector-java-5.1.46.jar 
-rw-r--r-- 1 root root 1004838 Jun 23 12:30 lib/mysql-connector-java-5.1.46.jar

配置環(huán)境變量

• 登錄 master 容器

root@suoron:~#export JAVA_HOME=/usr/local/jdk1.8.0_152
root@suoron:~#export HIVE_HOME=/opt/apache-hive-3.1.1
root@suoron:~#export HADOOP_HOME=/usr/local/hadoop-2.9.1
root@suoron:~#export PATH=$PATH:$HIVE_HOME/bin

• 修改 hive-config.sh

root@47ba702846f0:/opt/apache-hive-3.1.1# vi bin/hive-config.sh
在文件末尾添加
export JAVA_HOME=/usr/local/jdk1.8.0_152
export HIVE_HOME=/opt/apache-hive-3.1.1
export HADOOP_HOME=/usr/local/hadoop-2.9.1
export PATH=$PATH:$HIVE_HOME/bin

• 修改 hive-site.xml

root@47ba702846f0:/opt/apache-hive-3.1.1/conf# cp hive-default.xml.template hive-site.xml
root@47ba702846f0:/opt/apache-hive-3.1.1/conf#vi hive-site.xml
# 在 </configuration> 前添加
  <property> 
    <name>javax.jdo.option.ConnectionURL</name>  
    <value>jdbc:mysql://192.168.0.1:3306/hive?characterEncoding=UTF-8</value> 
  </property>  
  <property> 
    <name>javax.jdo.option.ConnectionDriverName</name>  
    <value>com.mysql.jdbc.Driver</value> 
  </property>  
  <property> 
    <name>javax.jdo.option.ConnectionUserName</name>  
    <value>hive</value> 
  </property>  
  <property> 
    <name>javax.jdo.option.ConnectionPassword</name>  
    <value>123</value> 
  </property>  
  <property> 
    <name>datanucleus.schema.autoCreateAll</name>  
    <value>true</value> 
  </property>  
  <property> 
    <name>hive.metastore.schema.verification</name>  
    <value>false</value> 
  </property>  
  <property> 
    <name>hive.exec.local.scratchdir</name>  
    <value>/opt/apache-hive-3.1.1/tmp</value>  
    <description>Local scratch space for Hive jobs</description> 
  </property>  
  <property> 
    <name>hive.downloaded.resources.dir</name>  
    <value>/opt/apache-hive-3.1.1/tmp/resources</value>  
    <description>Temporary local directory for added resources in the remote file system.</description> 
  </property>  
  <property> 
    <name>hive.querylog.location</name>  
    <value>/opt/apache-hive-3.1.1/tmp</value>  
    <description>Location of Hive run time structured log file</description> 
  </property>  
  <property> 
    <name>hive.server2.logging.operation.log.location</name>  
    <value>/opt/apache-hive-3.1.1/tmp/operation_logs</value>  
    <description>Top level directory where operation logs are stored if logging functitonality is enabled</description> 
  </property> 

• 在hive目錄中創(chuàng)建tmp臨時(shí)目錄
  root@47ba702846f0: mkdir /opt/apache-hive-3.1.1/tmp

初始化 Hive 元數(shù)據(jù)庫(kù)

root@47ba702846f0: schematool -dbType mysql -initSchema

測(cè)試 Hive 是否安裝成功

root@47ba702846f0: hive
hive> show databases;
OK
default
Time taken: 0.019 seconds, Fetched: 1 row(s)
hive> 

能出現(xiàn)上面的提示信息，說(shuō)明 Hive 已經(jīng)成功安裝

Hive 遠(yuǎn)程訪問(wèn)

• 開(kāi)啟 Thrift 服務(wù)

root@47ba702846f0:# hive --service hiveserver2
# java 客戶端可以訪問(wèn) 10000 端口，瀏覽器可以訪問(wèn) 10002 端口

• 啟動(dòng)hiveWebInterface，通過(guò)網(wǎng)頁(yè)訪問(wèn) hive

root@47ba702846f0:#hive --service hwi
# 127.0.0.1:9999/hwi