Hadoop實(shí)驗(yàn)——NoSQL與關(guān)系型數(shù)據(jù)庫的比較

實(shí)驗(yàn)?zāi)康?/h1>

  1. 理解四種數(shù)據(jù)庫(MySQL,HBase,Redis,MongoDB)的概念以及不同點(diǎn)。
  2. 熟練使用四種數(shù)據(jù)庫操作常用的Shell命令。
  3. 熟悉四種數(shù)據(jù)庫操作常用的Java API。

實(shí)驗(yàn)平臺(tái)

  • 操作系統(tǒng):Ubuntu-16.04
  • Hadoop版本:2.6.0
  • JDK版本:1.8
  • IDE:Eclipse
  • HBase版本:1.2.3
  • MySQL版本:5.7.16
  • MongoDB版本:2.6.10
  • Redis:版本:3.0.6
  • IDE:Eclipse

數(shù)據(jù)庫的安裝

  1. MySQL的安裝

  2. 更新APT
    sudo apt-get update

  3. 打開終端，安裝mysql-server
    sudo apt-get install mysql-server

  4. 輸入密碼


  5. 安裝mysql-client
    sudo apt-get install mysql-client

  6. 安裝libmysqlclient-dev
    sudo apt-get install libmysqlclient-dev

  7. 測(cè)試是否安裝成功
    sudo netstat -tap | grep mysql
    通過上述命令檢查之后，如果看到有mysql 的socket處于 listen 狀態(tài)則表示安裝成功。

  8. Redis的安裝

  9. 安裝redis-server
    sudo apt-get install redis-server

  10. 測(cè)試是否安裝成功
    sudo netstat -tap|grep redis

  11. MongoDB的安裝

  12. 安裝mongodb-server
    sudo apt-get install mongodb-server

  13. 測(cè)試是否安裝成功
    sudo netstat -tap|grep mongod

  14. HBase已經(jīng)安裝過了(詳見http://www.itdecent.cn/p/9ac6a4878b07）

實(shí)驗(yàn)內(nèi)容和要求

一，MySQL數(shù)據(jù)庫操作：

<div align = center>student學(xué)生表 </div>

name English Math Computer
zhangsan 69 86 77
lisi 55 100 88
  1. 根據(jù)上面給出的表格,利用MySQL設(shè)計(jì)出student學(xué)生表格。
  2. 登陸MySQL(退出指令為quit)
    mysql -u root -p
    輸入密碼
  3. 創(chuàng)建數(shù)據(jù)庫
    create database test;
  4. 使用數(shù)據(jù)庫
    use test;
  5. 創(chuàng)建student表
create table student(
    name varchar(30) not null,
    English tinyint unsigned not null,
    Math tinyint unsigned not null,
    Computer tinyint unsigned not null
    );
  1. 初始化數(shù)據(jù)
    insert into student values("zhangsan",69,86,77);

    insert into student values("lisi",55,100,88);
  2. 查看student表
    select * from student;
  3. 查看zhangsan的Computer成績(jī)
    select name , Computer from student where name = "zhangsan";
  4. 修改lisi的Math成績(jī),改為95
    update student set Math=95 where name="lisi";
  5. 根據(jù)上面已經(jīng)設(shè)計(jì)出的student表,通過JDBC操作MySQL
  6. 添加數(shù)據(jù):Name:scofield English:45 Math:89 Computer:100

Eclipse的使用
1. 找到 File 菜單，選擇 New -> Java Project



1. 輸入 Project name,然后Finish



1. 點(diǎn)開項(xiàng)目,找到 src 文件夾，右鍵選擇 New -> Class

1. 輸入 Package 和 Name,然后Finish

1. 將jar包從主機(jī)拉到虛擬機(jī)中的Home



1. 右鍵工程,選擇 Properties ,然后在工程中導(dǎo)入外部jar包


1. 寫好Java代碼(填上密碼),右鍵選擇 Run As -> Java Application,就可以在Console里看到結(jié)果了

JAVA代碼: 
package com.mysql;
import java.sql.*;
public class MysqlTest {
   static final String DRIVER = "com.mysql.jdbc.Driver";
   static final String DB = "jdbc:mysql://localhost/test?useUnicode=true&characterEncoding=utf-8&useSSL=false";
   static final String USER = "root";
   static final String PASSWD = "";
   public static void main(String[] args) {
       Connection conn = null;
       Statement stmt = null;
       try {
           Class.forName(DRIVER);
           System.out.println("Connecting to a selected database...");
           conn = DriverManager.getConnection(DB, USER, PASSWD);
           stmt = conn.createStatement();
           String sql = "insert into student values('scofield',45,89,100)";
           stmt.executeUpdate(sql);
           System.out.println("Inserting records into the table successfully!");
       } catch (ClassNotFoundException e) {
           e.printStackTrace();
       } catch (SQLException e) {
           e.printStackTrace();
       } finally {
           if (stmt != null)
               try {
                   stmt.close();
               } catch (SQLException e) {
                   e.printStackTrace();
               }
           if (conn != null)
               try {
                   conn.close();
               } catch (SQLException e) {
                   e.printStackTrace();
               }
       }
   }
}

  1. 插入數(shù)據(jù)之后,MySQL客戶度查詢結(jié)果如下
  1. 獲取scofield的English成績(jī)信息

JAVA代碼:

package com.mysql;
import java.sql.*;
public class MysqlTest2 {

    static final String DRIVER = "com.mysql.jdbc.Driver";
    static final String DB = "jdbc:mysql://localhost/test?useUnicode=true&characterEncoding=utf-8&useSSL=false";
    static final String USER = "root";
    static final String PASSWD = "0822";

    public static void main(String[] args) {
        Connection conn = null;
        Statement stmt = null;
        ResultSet rs = null;
        try {
            Class.forName(DRIVER);
            System.out.println("Connecting to a selected database...");
            conn = DriverManager.getConnection(DB, USER, PASSWD);
            stmt = conn.createStatement();
            String sql = "select name,English from student where name='scofield' ";
            rs = stmt.executeQuery(sql);
            System.out.println("name" + "\t\t" + "English");
            while (rs.next()) {
                System.out.print(rs.getString(1) + "\t\t");
                System.out.println(rs.getInt(2));
            }
        } catch (ClassNotFoundException e) {
            e.printStackTrace();
        } catch (SQLException e) {
            e.printStackTrace();
        } finally {
            if (rs != null)
                try {
                    rs.close();
                } catch (SQLException e1) {
                    e1.printStackTrace();
                }
            if (stmt != null)
                try {
                    stmt.close();
                } catch (SQLException e) {
                    e.printStackTrace();
                }
            if (conn != null)
                try {
                    conn.close();
                } catch (SQLException e) {
                    e.printStackTrace();
                }
        }
    }
}

Eclipse控制臺(tái)輸出如下:


二，HBase數(shù)據(jù)庫操作：

<div align = center>student學(xué)生表 </div>

name score:English score:Math score:Computer
zhangsan 69 86 77
lisi 55 100 88
  1. 根據(jù)上面給出的表格，用Hbase Shell模式設(shè)計(jì)student學(xué)生表格。
  2. 啟動(dòng) Hadoop
    • 進(jìn)入 Hadoop 主文件夾
      cd /usr/local/hadoop/
    • 開啟 Hadoop 相關(guān)進(jìn)程
      sbin/start-dfs.sh

      sbin/start-yarn.sh
  3. 啟動(dòng) HBase
    • 進(jìn)入HBase主文件夾
      cd /usr/local/hbase/
    • 開啟HBase相關(guān)進(jìn)程
      bin/start-hbase.sh
  4. 進(jìn)入 Hbase Shell（退出指令為quit）

  5. 創(chuàng)建表student表


  6. 初始化student表
put 'student','zhangsan','score:English','69'
put 'student','zhangsan','score:Math','86'
put 'student','zhangsan','score:Computer','77'
put 'student','lisi','score:English','55'
put 'student','lisi','score:Math','100'
put 'student','lisi','score:Computer','88'
  1. 查看student表
    scan 'student'
  2. 查詢zhangsan 的Computer成績(jī)
    get 'student','zhangsan','score:Computer'
  3. 修改lisi的Math成績(jī)，改為95
    put 'student','lisi','score:Math','95'
  4. 根據(jù)上面已經(jīng)設(shè)計(jì)出的student表,用Hbase API操作MySQL
  5. 添加數(shù)據(jù):Name:scofield English:45 Math:89 Computer:100

    1. 點(diǎn)開項(xiàng)目,找到 src 文件夾，右鍵選擇 New -> Class


    2. 輸入 Package 和 Name,然后Finish


    3. 右鍵工程,選擇 Properties ,然后在工程中導(dǎo)入外部jar包



      JAVA代碼:

package com.hbase;
import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.Admin;
import org.apache.hadoop.hbase.client.Connection;
import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.client.Table;
public class HbaseTest {
   public static Configuration configuration;
   public static Connection connection;
   public static Admin admin;
   public static void main(String[] args) {
       configuration = HBaseConfiguration.create();
       configuration.set("hbase.rootdir", "hdfs://localhost:9000/hbase");
       try {
           connection = ConnectionFactory.createConnection(configuration);
           admin = connection.getAdmin();
       } catch (IOException e) {
           e.printStackTrace();
       }
       try {
           insertRow("student", "scofield", "score", "English", "45");
           insertRow("student", "scofield", "score", "Math", "89");
           insertRow("student", "scofield", "score", "Computer", "100");
       } catch (IOException e) {
           e.printStackTrace();
       }
       close();
   }
   public static void insertRow(String tableName, String rowKey,
           String colFamily, String col, String val) throws IOException {
       Table table = connection.getTable(TableName.valueOf(tableName));
       Put put = new Put(rowKey.getBytes());
       put.addColumn(colFamily.getBytes(), col.getBytes(), val.getBytes());
       table.put(put);
       table.close();
   }
   public static void close() {
       try {
           if (admin != null) {
               admin.close();
           }
           if (null != connection) {
               connection.close();
           }
       } catch (IOException e) {
           e.printStackTrace();
       }
   }
}

  1. 插入數(shù)據(jù)之后,HBase Shell查詢結(jié)果如下
  1. 獲取scofield的English成績(jī)信息

JAVA代碼:

package com.hbase;
import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.CellUtil;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.Admin;
import org.apache.hadoop.hbase.client.Connection;
import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.hadoop.hbase.client.Get;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.Table;
public class HbaseTest2 {
    public static Configuration configuration;
    public static Connection connection;
    public static Admin admin;

    public static void main(String[] args) {
        configuration = HBaseConfiguration.create();
        configuration.set("hbase.rootdir", "hdfs://localhost:9000/hbase");
        try {
            connection = ConnectionFactory.createConnection(configuration);
            admin = connection.getAdmin();
        } catch (IOException e) {
            e.printStackTrace();
        }
        try {
            getData("student", "scofield", "score", "English");
        } catch (IOException e) {
            e.printStackTrace();
        }
        close();
    }

    public static void getData(String tableName, String rowKey,
            String colFamily, String col) throws IOException {
        Table table = connection.getTable(TableName.valueOf(tableName));
        Get get = new Get(rowKey.getBytes());
        get.addColumn(colFamily.getBytes(), col.getBytes());
        Result result = table.get(get);
        showCell(result);
        table.close();
    }

    public static void showCell(Result result) {
        Cell[] cells = result.rawCells();
        for (Cell cell : cells) {
            System.out.println("RowName:" + new String(CellUtil.cloneRow(cell))
                    + " ");
            System.out.println("Timetamp:" + cell.getTimestamp() + " ");
            System.out.println("column Family:"
                    + new String(CellUtil.cloneFamily(cell)) + " ");
            System.out.println("row Name:"
                    + new String(CellUtil.cloneQualifier(cell)) + " ");
            System.out.println("value:" + new String(CellUtil.cloneValue(cell))
                    + " ");
        }
    }

    public static void close() {
        try {
            if (admin != null) {
                admin.close();
            }
            if (null != connection) {
                connection.close();
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

Eclipse控制臺(tái)輸出如下:


三，Redis數(shù)據(jù)庫操作：

student 鍵值對(duì):

zhangsan:｛
English: 69
Math: 86
Computer: 77
｝
lisi:｛
English: 55
Math: 100
Computer: 88
｝
  1. 根據(jù)上面給出的鍵值對(duì),利用哈希結(jié)構(gòu)設(shè)計(jì)出上述表格。(鍵值可以用student.zhangsan,student.lisi來表示兩個(gè)鍵值屬于同一個(gè)表格)
  2. 啟動(dòng)Redis客戶端(退出指令為quit)
    redis-cli
  3. 設(shè)計(jì)上述表格
hset student.zhangsan English 69
hset student.zhangsan Math 86
hset student.zhangsan Computer 77
hset student.lisi English 55
hset student.lisi Math 100
hset student.lisi Computer 88
  1. 輸出zhangsan和lisi的信息
    hgetall student.zhangsan

    hgetall student.lisi
  2. 查看zhangsan的Computer成績(jī)
    hget student.zhangsan Computer
  3. 修改lisi的Math成績(jī),改為95
    hset student.lisi Math 95
  4. 根據(jù)上面已經(jīng)設(shè)計(jì)出的student表,通過jedis操作Redis
  5. 添加數(shù)據(jù):
scofield:｛
English: 45
Math: 89
Computer: 100
｝

  1. 點(diǎn)開項(xiàng)目,找到 src 文件夾，右鍵選擇 New -> Class

1. 輸入 Package 和 Name,然后Finish



1. 將jar包從主機(jī)拉到虛擬機(jī)中的Home



1. 右鍵工程,選擇 Properties ,然后在工程中導(dǎo)入外部jar包

JAVA代碼: 
package com.redis;
import java.util.Map;
import redis.clients.jedis.Jedis;
public class RedisTest {
   public static void main(String[] args) {
       Jedis jedis = new Jedis("localhost");
       jedis.hset("student.scofield", "English", "45");
       jedis.hset("student.scofield", "Math", "89");
       jedis.hset("student.scofield", "Computer", "100");
       Map<String, String> value = jedis.hgetAll("student.scofield");
       for (Map.Entry<String, String> entry : value.entrySet()) {
           System.out.println(entry.getKey() + ":" + entry.getValue());
       }
   }
}

  1. 插入數(shù)據(jù)之后,Redis客戶度查詢結(jié)果如下
  1. 獲取scofield的English成績(jī)信息

JAVA代碼:

package com.redis;
import redis.clients.jedis.Jedis;
public class RedisTest2 {
    public static void main(String[] args) {
        Jedis jedis = new Jedis("localhost");
        String value = jedis.hget("student.scofield", "English");
        System.out.println("scofield's English score is:    " + value);
    }
}

Eclipse控制臺(tái)輸出如下:


四，MongoDB數(shù)據(jù)庫操作：

student文檔如下:

{
“name”: “zhangsan”,
“score”: {
“English”: 69,
“Math”: 86,
“Computer”: 77
}
}
{
“name”: “l(fā)isi”,
“score”: {
“English”: 55,
“Math”: 100,
“Computer”: 88
}
}
  1. 根據(jù)上面給出的文檔,用Mongo shell設(shè)計(jì)出student集合
  2. 啟動(dòng)MongoDB客戶端(退出指令為quit();)
    mongo
  3. 創(chuàng)建student數(shù)據(jù)庫,
    use student
  4. 定義數(shù)組
var stus=[
      {"name":"zhangsan","scores":{"English":69,"Math":86,"Computer":77}},        
      {"name":"lisi","score":{"English":55,"Math":100,"Computer":88}} ]
  1. 插入到數(shù)據(jù)庫
    db.student.insert(stus)
  2. 輸出student的信息
    db.student.find().pretty()
  3. 查詢zhangsan 的所有成績(jī)(只顯示score列)
    db.student.find({"name":"zhangsan"},{"_id":0,"name":0})
  4. 修改lisi的Math成績(jī),改為95
    db.student.update({"name":"lisi"}, {"$set":{"score.Math":95}} )
  5. 根據(jù)上面已經(jīng)設(shè)計(jì)出的student集合,通過JDBC操作MongoDB
  6. 添加數(shù)據(jù):
English:45  Math:89 Computer:100
{
“name”: “scofield”,
“score”: {
“English”: 45,
“Math”: 89,
“Computer”: 100
}
}

  1. 點(diǎn)開項(xiàng)目,找到 src 文件夾，右鍵選擇 New -> Class

1. 輸入 Package 和 Name,然后Finish



1. 將jar包從主機(jī)拉到虛擬機(jī)中的Home



1. 右鍵工程,選擇 Properties ,然后在工程中導(dǎo)入外部jar包

JAVA代碼: 
package com.mongo;
import java.util.ArrayList;
import java.util.List;
import org.bson.Document;
import com.mongodb.MongoClient;
import com.mongodb.client.MongoCollection;
import com.mongodb.client.MongoDatabase;
public class MongoTest {
   public static void main(String[] args) {
       MongoClient mongoClient = new MongoClient("localhost", 27017);
       MongoDatabase mongoDatabase = mongoClient.getDatabase("student");
       MongoCollection<Document> collection = mongoDatabase
               .getCollection("student");
       Document document = new Document("name", "scofield").append(
               "score",
               new Document("English", 45).append("Math", 89).append(
                       "Computer", 100));
       List<Document> documents = new ArrayList<Document>();
       documents.add(document);
       collection.insertMany(documents);
       System.out.println("文檔插入成功");
   }
}

  1. 插入數(shù)據(jù)之后,MongoDB客戶度查詢結(jié)果如下
  1. 獲取scofield的English成績(jī)信息

JAVA代碼:

package com.mongo;
import org.bson.Document;
import com.mongodb.MongoClient;
import com.mongodb.client.MongoCollection;
import com.mongodb.client.MongoCursor;
import com.mongodb.client.MongoDatabase;
import com.mongodb.client.model.Filters;
import static com.mongodb.client.model.Filters.eq;
public class MongoTest2 {
    public static void main(String[] args) {
        MongoClient  mongoClient=new MongoClient("localhost",27017);
        MongoDatabase mongoDatabase = mongoClient.getDatabase("student");
        MongoCollection<Document> collection = mongoDatabase.getCollection("student");
        MongoCursor<Document>  cursor=collection.find( new Document("name","scofield")).
                projection(new Document("score",1).append("_id", 0)).iterator();
        while(cursor.hasNext())
            System.out.println(cursor.next().toJson());
    }
}

Eclipse控制臺(tái)輸出如下:


最后編輯于 ：
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成，瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明：文章內(nèi)容（如有圖片或視頻亦包括在內(nèi)）由作者上傳并發(fā)布，文章內(nèi)容僅代表作者本人觀點(diǎn)，簡(jiǎn)書系信息發(fā)布平臺(tái)，僅提供信息存儲(chǔ)服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

  • import org.apache.hadoop.conf.Configuration; import org.a...
    蘇大鴻閱讀 666評(píng)論 0贊 1
  • 泛型 對(duì)象和實(shí)例是一個(gè)意思，類與對(duì)象的關(guān)系就像數(shù)據(jù)類型和變量一樣。 泛型的主要目的之一就是用來指定類（如：容器）要...
    yueyue_projects閱讀 632評(píng)論 0贊 0
  • 瑣事記10.6-10.8 昨天（10.5）下午，從荊州回來，下車之后，我走進(jìn)家族內(nèi)的一位哥哥家，想與他商量一件考慮...
    小棕櫚閱讀 280評(píng)論 0贊 0

友情鏈接更多精彩內(nèi)容