flinksql 實時查詢hudi 的數據

1.版本

組件 版本
hudi 10.0
flink 13.5

2. 場景:

在flink 中新建一張表(t1)插入數據, 然后同時用過另外一張表進行查詢(t2) 

場景如圖

hudi 實時寫入數據.png

3. t1 建表

    CREATE TABLE t1(
      uuid VARCHAR(20), 
      name VARCHAR(10),
      age INT,
      ts TIMESTAMP(3),
      `partition` VARCHAR(20)
    )
    PARTITIONED BY (`partition`)
    WITH (
      'connector' = 'hudi',
      'path' = 'hdfs://192.168.1.161:8020/hudi-warehouse/hudi-t1',
      'write.tasks' = '1',
      'compaction.tasks' = '1', 
      'table.type' = 'MERGE_ON_READ'
    );

3.1 插入數據

INSERT INTO t1 VALUES('id1','Danny',28,TIMESTAMP '1970-01-01 00:00:01','par1');
insert into t1 values ('id9','Danny',18,TIMESTAMP'1970-01-01 00:00:01','par9');
INSERT INTO t1 VALUES
('id2','Stephen',33,TIMESTAMP '1970-01-01 00:00:02','par1'),
('id3','Julian',53,TIMESTAMP '1970-01-01 00:00:03','par2'),
('id4','Fabian',31,TIMESTAMP '1970-01-01 00:00:04','par2'),
('id5','Sophia',18,TIMESTAMP '1970-01-01 00:00:05','par3'),
('id6','Emma',20,TIMESTAMP '1970-01-01 00:00:06','par3'),
('id7','Bob',44,TIMESTAMP '1970-01-01 00:00:07','par4'),
('id8','Han',56,TIMESTAMP '1970-01-01 00:00:08','par4');


4.另外開一個窗口創(chuàng)建表t2

CREATE TABLE t2(
uuid VARCHAR(20), 
name VARCHAR(10),
age INT,
ts TIMESTAMP(3),
`partition` VARCHAR(20)
)
PARTITIONED BY (`partition`)
WITH (
'connector' = 'hudi',
'path' = 'hdfs://192.168.1.161:8020/hudi-warehouse/hudi-t1',
'table.type' = 'MERGE_ON_READ',
'read.tasks' = '1', 
'read.streaming.enabled' = 'true',  -- this option enable the streaming read
'read.streaming.start-commit' = '20210316134557',  -- specifies the start commit instant time
'read.streaming.check-interval' = '4'   -- specifies the check interval for finding new source commits, default 60s.
);

5. 在t1 表插入數據 在t2表中會實時增加

hudi 動態(tài)查詢數據.png
?著作權歸作者所有,轉載或內容合作請聯系作者
【社區(qū)內容提示】社區(qū)部分內容疑似由AI輔助生成,瀏覽時請結合常識與多方信息審慎甄別。
平臺聲明:文章內容(如有圖片或視頻亦包括在內)由作者上傳并發(fā)布,文章內容僅代表作者本人觀點,簡書系信息發(fā)布平臺,僅提供信息存儲服務。

相關閱讀更多精彩內容

友情鏈接更多精彩內容