編譯

下載源碼

git clone https://github.com/apache/iceberg.git

切換到0.12分支

gco 0.12.x 

安裝gradle

由于iceberg采用gradle來(lái)管理項(xiàng)目, 在編譯之前需要安裝gradle
檢查jdk版本, gradle需要jdk8以及以上版本

java -version
java version "1.8.0_261"
Java(TM) SE Runtime Environment (build 1.8.0_261-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.261-b12, mixed mode)

安裝gradle

brew install gradle

修改環(huán)境變量

export GRADLE=/Users/home_name/opt/gradle-7.0

驗(yàn)證安裝結(jié)果

gradle -v

------------------------------------------------------------
Gradle 7.0
------------------------------------------------------------

Build time:   2021-04-09 22:27:31 UTC
Revision:     d5661e3f0e07a8caff705f1badf79fb5df8022c4

Kotlin:       1.4.31
Groovy:       3.0.7
Ant:          Apache Ant(TM) version 1.10.9 compiled on September 27 2020
JVM:          1.8.0_261 (Oracle Corporation 25.261-b12)
OS:           Mac OS X 10.14.6 x86_64

編譯

./gradlew build -x test -x integrationTest

部署

將jar包(iceberg-flink-runtime-0.12.0.jar)發(fā)送到遠(yuǎn)程服務(wù)器flink測(cè)試集群lib目錄下, 重啟集群

測(cè)試運(yùn)行

創(chuàng)建Hadoop Catalog

 CREATE CATALOG hadoop_catalog WITH (
  'type'='iceberg',
  'catalog-type'='hadoop',
  'warehouse'='hdfs://xxx-hdfs/flink/tmp/iceberg_test',
  'property-version'='1'
)
;

創(chuàng)建測(cè)試庫(kù)

CREATE DATABASE hadoop_catalog.iceberg_db;

創(chuàng)建format為v2的表

CREATE TABLE hadoop_catalog.iceberg_db.sample_test (
    id BIGINT COMMENT 'unique id',
    data STRING,
    PRIMARY KEY(id) NOT ENFORCED
)
WITH (
    'format-version'= '2',
    'write.upsert.enable'='true'
)
;

Insert 數(shù)據(jù)

INSERT INTO hadoop_catalog.iceberg_db.sample_test  VALUES (10, 'test10_U'), (11, 'test11'), (12, 'test12');

查看insert結(jié)果

select * from hadoop_catalog.iceberg_db.sample_test;