發(fā)簡信
IP屬地：加州
-
-
26-SparkStreaming02
用戶行為日志 信令數(shù)據(jù) cell mapWithState DStream 整合RDD == transform 數(shù)據(jù)一：日志信息 DStream domain,traff...
-
25-SparkStreaming01
Spark Streaming 基于Spark之上的流處理 流：source ==> compute ==> store 離線是特殊的流 letting you write ...
-
24-SparkSQL04
Function functions.scala hobbies.txt alice jogging,Coding,cooking 3 lina travel,danc...
-
22-SparkSQL03
External Data Source API 外部數(shù)據(jù)源 MapReduce Hive Spark 加載數(shù)據(jù) 格式：json、parquet、text、jdbc........
-
21-SparkSQL02
DataFrame python pandas R RDD MapReduce DataFrame vs Dataset(1.6) DS: Java Scala DF: 4 ...
-
19-Kafka02
1.核心概念 broker: 進程 producer: 生產(chǎn)者 consumer: 消費者 topic: 主題 partitions: 分區(qū) (副本數(shù)) consumergr...
-
20-SparkSQL01
Spark SQL IOE SQL：schema + file select ... from xxx where..... SQL on Hadoop Hive Impal...
-
18-Kafka01-Kafka部署及常用命令
下載地址: Zookeeper: http://mirror.bit.edu.cn/apache/zookeeper/current/ Scala: http://www.s...
-
18-Kafka01
Kafka: 消息中間件 -->分布式流式平臺 MQ Redis Kafka Flume 生產(chǎn)者 source Broker channel 消費者 sink 正常部...
-
17-SparkCore04
collect collect countByKey countByValue collectAsMap groupByKey vs reduceByKey val rdd=...
-
16-SparkCore03
Spark on YARN 將spark作業(yè)提交到y(tǒng)arn上去執(zhí)行 spark僅僅作業(yè)一個客戶端 ./spark-submit \ --class org.apache.sp...
-
15-SparkCore02
Application a driver program + executors SparkContext = application spark-shell ? appli...
-
14-SparkCore01
x.y.z 1.6.1 2.3.1 2.2.2 RDD transformation: lazy map filter union flatMap mapPartition ...
-
13-Hadoop高級
Hadoop的HDFS HA、Yarn HA集群部署 1.HDFS NN SNN(secondary) 熱備 NN(active) 掛了 NN(standby)--》acti...
-
12-Hive高級02
Hive高級第二部分： *****Hive：復(fù)雜數(shù)據(jù)類型、JDBC編程ZK： Compression壓縮比解壓速度1G的沒壓縮數(shù)據(jù)：1G的gzip壓縮數(shù)據(jù)：codec：我...
-
11-Zookeeper
ZK 1) 高可用： HDFS/HBase/Spark HA2) API：ZK/Curator開發(fā)：Java/Scala操作ZKKafka：offset可以存儲在ZK =...
-
10-Python02
python官網(wǎng)給出的編程規(guī)范 1.Use 4-space indentation, and no tabs. 2.Wrap lines so that they don’t...
-
09-Python01
anaconda3下載地址 官網(wǎng)：https://www.anaconda.com/download/ 百度云鏈接：https://pan.baidu.com/s/17jHe...
-
08-Flume02
.../page_views/201808082008 .... .../page_views/201808082009 .... ./flume-ng agent \ --...
個人介紹
I have traveled many countries. Be crazy in techniques and Tango! I think I should be a historian or racing driver！I live in Chicago but I'm a fun of Phila 76ers.