apache atlas項(xiàng)目搭建(1.0.0,2.0.0),centos7環(huán)境

一、前置條件

1、jdk1.8、maven3、nodejs

jdk、maven解壓、配置環(huán)境變量;

nodejs安裝,參考 https://www.cnblogs.com/jizhong/p/12102100.html

centos7自帶python2.7.5,無(wú)需安裝

2、配置環(huán)境變量 /etc/profile

JAVA_HOME=/usr/local/tools/jdk1.8.0_161
export export PATH=${JAVA_HOME}/bin:${PATH}
MAVEN_HOME=/usr/local/tools/apache-maven-3.6.3
export PATH=${MAVEN_HOME}/bin:${PATH}
NODE_HOME=/usr/local/tools/node
export PATH=${NODE_HOME}/bin:${PATH}

配置生效 source /etc/profile

3、查看版本信息:

node -v

v6.12.3

npm -v

3.10.10

python -V

Python 2.7.5

java -version

mvn –v

4、使用淘寶的npm鏡像cnpm

npm install -g cnpm --registry=https://registry.npm.taobao.org

5、maven 配置文件(阿里鏡像、倉(cāng)庫(kù)地址、jdk版本)

替換maven源,阿里云的源

打開(kāi)maven配置文件 settings.xml

<mirror>
    <id>alimaven</id>
    <name>aliyun maven</name>
    <url>http://maven.aliyun.com/nexus/content/groups/public/</url>
    <mirrorOf>central</mirrorOf>
</mirror>

指定下載資源位置

<localRepository>/usr/local/maven/repository</localRepository>

指定jdk版本(本座沒(méi)有配置)

<profile>    
    <id>jdk-1.8</id>    
    <activation>    
        <activeByDefault>true</activeByDefault>    
        <jdk>1.8</jdk>    
    </activation>    
    <properties>    
        <maven.compiler.source>1.8</maven.compiler.source>    
        <maven.compiler.target>1.8</maven.compiler.target>    
        <maven.compiler.compilerVersion>1.8</maven.compiler.compilerVersion>    
    </properties>
</profile>

二、apache atlas1.0.0

1、下載源系統(tǒng) http://atlas.apache.org/#/Downloads

企業(yè)微信截圖_15882361078763.png

2、編譯

tar -zxvf apache-atlas-1.0.0-sources.tar.gz cd apache-atlas-sources-1.0.0

配置堆大小,防止內(nèi)存溢出

export MAVEN_OPTS="-Xms2g -Xmx4g"

編譯,兩種方法選其一

1)、該方式編譯不會(huì)內(nèi)嵌HBase和Solr

mvn clean -DskipTests package -Pdist

2)、使用atlas內(nèi)嵌HBase和Solr,測(cè)試用這種方式,注意:embedded-hbase-solr前面沒(méi)有空格

mvn clean -DskipTests package -Pdist,embedded-hbase-solr

編譯完成后的jar包位于目錄:\apache-atlas-sources-1.1.0\distro\target 中,壓縮包有對(duì)應(yīng)的解壓文件,可直接部署;或者自己加壓到指定路徑

3、部署,進(jìn)入 apache-atlas-1.0.0 路徑

/usr/local/projects/apache-atlas-sources-1.0.0/distro/target/apache-atlas-1.0.0-bin/apache-atlas-1.0.0

用內(nèi)嵌的 hbase 和 solr 啟動(dòng),配置hbase和solr跟隨atlas啟動(dòng)和停止

export MANAGE_LOCAL_HBASE=true

export MANAGE_LOCAL_SOLR=true

上面的兩步已經(jīng)在 atlas-env.sh 文件默認(rèn)配置

4、啟動(dòng)

bin/atlas_start.py

5、查看進(jìn)程 jps

企業(yè)微信截圖_158823674822.png

6、添加自帶例子 bin/quick_start.py

Enter username for atlas :- admin Enter password for atlas :- admin

7、管理界面登錄入口 http://localhost:21000 用戶(hù)名:admin 密碼:admin

8、停止atlas,bin/atlas_stop.py

在hbase里配置了JAVE_HOME

三、apache atlas2.0.0

編譯過(guò)程和1.0.0一致,編譯時(shí)間較1.0更長(zhǎng)

編譯完成后使用

/usr/local/project/apache-atlas-sources-2.0.0/distro/target/apache-atlas-2.0.0-server/apache-atlas-2.0.0

1、修改配置文件 apache-atlas-2.0.0-server/apache-atlas-2.0.0/conf/

vim atlas-env.sh
export JAVA_HOME=/usr/local/tools/jdk1.8.0_161
export MANAGE_LOCAL_HBASE=true  #如果要使用外部的zk和hbase,則改為false
export MANAGE_LOCAL_SOLR=true  #如果要是用外部的solr,則改為false

vim atlas-application.properties
# Hbase地址(對(duì)應(yīng)的zk地址)配置(自帶hbase會(huì)根據(jù)此端口啟動(dòng)一個(gè)zk實(shí)例)
atlas.graph.storage.hostname=localhost:2181 # 如果使用外部hbase,則填寫(xiě)外部zookeeper地址

# Solr地址配置
atlas.graph.index.search.solr.http-urls=http://localhost:8984/solr

# Kafka相關(guān)配置
atlas.notification.embedded=true # 如果要使用外部的kafka,則改為false
# 內(nèi)嵌kafka會(huì)根據(jù)此端口啟動(dòng)一個(gè)zk實(shí)例
atlas.kafka.zookeeper.connect=localhost:9026 # 如果使用外部kafka,則填寫(xiě)外部zookeeper地址
atlas.kafka.bootstrap.servers=localhost:9027 # 如果使用外部kafka,則填寫(xiě)外部broker server地址

2、手動(dòng)啟動(dòng)hbase

# 進(jìn)入atlas自帶的hbase目錄
apache-atlas-2.0.0-server/apache-atlas-2.0.0/hbase 
# 啟動(dòng) hbase 
bin/start-hbase.sh

3、手動(dòng)啟動(dòng)solr,為solr創(chuàng)建初始化index庫(kù)

# 進(jìn)入atlas自帶的solr目錄,
apache-atlas-2.0.0-server/apache-atlas-2.0.0/solr
# 啟動(dòng)solr
bin/solr start -c -z localhost:2181 -p 8984 -force

# 創(chuàng)建初始化collections
bin/solr create -c vertex_index -shards 1 -replicationFactor 1 -force
bin/solr create -c edge_index -shards 1 -replicationFactor 1 -force
bin/solr create -c fulltext_index -shards 1 -replicationFactor 1 -force

然后打開(kāi)瀏覽器訪問(wèn)solr的web服務(wù)如下,則solr啟動(dòng)成功 http://localhost:8984/

4、啟動(dòng)atlas,apache-atlas-2.0.0-server/apache-atlas-2.0.0

bin/atlas_start.py

最后添加自帶例子,同1.0.0

四、訪問(wèn)路徑

atlas:http://192.168.198.129:21000/

solr(1.0.0):http://192.168.65.139:9838/

solr(2.0.0):http://localhost:8984/

五、問(wèn)題

1、

[ERROR] Java heap space -> [Help 1]

[ERROR]

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR]

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/OutOfMemoryError

解決:

沒(méi)有設(shè)置 MAVEN_OPTS="-Xms2g -Xmx4g", 造成堆內(nèi)存溢出

2、

[ERROR] [ERROR] Some problems were encountered while processing the POMs:

[FATAL] Non-resolvable parent POM for org.apache.atlas:apache-atlas:1.0.0: Could not transfer artifact org.apache:apache:pom:17 from/to alimaven (http://maven.aliyun.com/nexus/content/groups/public/): Transfer failed for http://maven.aliyun.com/nexus/content/groups/public/org/apache/apache/17/apache-17.pom and 'parent.relativePath' points at wrong local POM @ line 23, column 13

編譯錯(cuò)誤:

晚上下班突然就可以了

3、

[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.4:install-node-and-npm (install node and npm) on project atlas-dashboardv2: Could not extract the Node archive: Could not extract archive: '/usr/local/maven/repository/com/github/eirslett/node/4.4.2/node-4.4.2-linux-x64.tar.gz': EOFException -> [Help 1]

Failed to run task: 'npm install' failed

npm: command not found...

需要安裝nodejs,并使用淘寶的npm鏡像cnpm

npm install -g cnpm --registry=https://registry.npm.taobao.org

4、

Creating sample types:

Exception in thread "main" org.apache.atlas.AtlasServiceException: Metadata service API org.apache.atlas.AtlasClientV2$API_V2@491b9b8 failed with status 503 (Service Unavailable) Response Body (<html>

<head>

<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>

<title>Error 503 </title>

</head>

<body>

<h2>HTTP ERROR: 503</h2>

<p>Problem accessing /api/atlas/v2/types/typedefs/. Reason:

<pre> Service Unavailable</pre></p>

<hr /><a >Powered by Jetty:// 9.3.14.v20161028</a><hr/>

</body>

</html>

)

at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:395)

at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:323)

at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:211)

at org.apache.atlas.AtlasClientV2.createAtlasTypeDefs(AtlasClientV2.java:227)

at org.apache.atlas.examples.QuickStartV2.createTypes(QuickStartV2.java:185)

at org.apache.atlas.examples.QuickStartV2.runQuickstart(QuickStartV2.java:141)

at org.apache.atlas.examples.QuickStartV2.main(QuickStartV2.java:126)

No sample data added to Apache Atlas Server.

curl -v -u username:password http://localhost:21000/api/atlas/admin/version

解決:

solr啟動(dòng)失敗,還是配置的問(wèn)題

5、

hbase Error: JAVA_HOME is not set

解決:

/usr/local/project/apache-atlas-sources-2.0.0/distro/target/apache-atlas-2.0.0-bin/apache-atlas-2.0.0/hbase/conf

atlas 1.0.0 配置了JAVA_HOME

hbase-env.sh

6、

cannot create temp file for here-document: No space left on device

空間滿(mǎn)了 df -h

7、

Caused by: org.apache.solr.common.SolrException: Cannot connect to cluster at localhost:2181: cluster not found/not ready

at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:385)

at org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:141)

at org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:383)

at org.janusgraph.diskstorage.solr.Solr6Index.<init>(Solr6Index.java:218)

2181 zookeeper 默認(rèn)端口

還是配置的問(wèn)題,atlas2.0.0

最后編輯于
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成,瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點(diǎn),簡(jiǎn)書(shū)系信息發(fā)布平臺(tái),僅提供信息存儲(chǔ)服務(wù)。

友情鏈接更多精彩內(nèi)容