上篇已經(jīng)介紹了通過Spark Graphx構(gòu)建圖，在構(gòu)圖過程中要準(zhǔn)備一個Long類型的字段作為節(jié)點的VertexID，而對于上篇中用到的數(shù)據(jù)集，顯然增加了工作量(將oid[String類型]轉(zhuǎn)換為Long)。
GraphFrames能夠避免這樣的操作。

GraphFrames

GraphFrames是以DataFrame為基礎(chǔ)的構(gòu)圖組件。
(GraphFrames is a package for Apache Spark which provides DataFrame-based Graphs. )
它的目標(biāo)是保留GraphX功能的同時充分利用Spark DataFrames的優(yōu)勢來擴展GraphX的功能。其擴展功能包括motif 預(yù)測，DataFrame序列化以及高性能的圖查詢。
( It aims to provide both the functionality of GraphX and extended functionality taking advantage of Spark DataFrames. This extended functionality includes motif finding, DataFrame-based serialization, and highly expressive graph queries.)

利用GraphFrames進行構(gòu)圖

主要流程和上篇沒太大差異。主要是利用GraphFrames構(gòu)圖時，節(jié)點DataFrames需要指明某列為id列(可以使String類型)，邊DataFrames需要指明src、dst列(可以使String類型)。代碼如下：

    //將RDD轉(zhuǎn)為DataFrame
    val personsdf = sqlContext.createDataFrame(personsRDD).toDF("id", "name", "no", "groupno", "vertextype")
    val groupsdf = sqlContext.createDataFrame(groupsRDD).toDF("id", "name", "tags", "groupno", "vertextype").dropDuplicates("groupno")

    val groupsds = groupsdf.select("id", "groupno", "tags")
    val relation = personsdf.join(groupsds, personsdf("groupno") === groupsds("groupno")).toDF("src","personname","personno","groupno","personvertextype","dst","groupno","tags").select("src","dst","tags")

    val personsds = personsdf.dropDuplicates("id", "no").toDF("id", "name", "tags", "groupno", "vertextype")
    val unionds = personsds.union(groupsdf)

    val graph = GraphFrame(unionds, relation)

linkurious.js

linkurious.js是純Javascript開發(fā)的js庫。它利用Sigma.js進行結(jié)構(gòu)數(shù)據(jù)的解析及可視化，其提供Canvas、WebGL、SVG三種渲染模式來渲染圖的節(jié)點和邊。
(Linkurious.js is developed in pure Javascript. It uses Sigma.js for its graph data structure and visualization engine, which provides both Canvas, WebGL and SVG renderers for nodes and edges. )

利用linkurious.js進行圖的可視化

首先，需要將DataFrames輸出成gexf格式的數(shù)據(jù)結(jié)構(gòu)文件

    def graphFrametoGexf(g: GraphFrame) : String = {
      "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n" +
        "<gexf xmlns=\"http://www.gephi.org/gexf\" xmlns:viz=\"http://www.gephi.org/gexf/viz\">\n" +
        "  <graph mode=\"static\">\n" +
        "    <nodes>\n" +
        g.vertices.rdd.map(v => if(v.get(4)=="p")
        {"      <node id=\"" + v.get(0) + "\" label=\"" +
          v.get(1) + "\">\n        <viz:color b=\"51\" g=\"51\" r=\"255\"/>\n      </node>\n"} else
        {"      <node id=\"" + v.get(0) + "\" label=\"" +
          v.get(1) + "\">\n        <viz:color b=\"255\" g=\"51\" r=\"51\"/>\n      </node>\n"
        }).collect.mkString +
        "    </nodes>\n" +
        "    <edges>\n" +
        g.edges.rdd.map(e =>
        "    <edge source=\"" + e.get(0) +
          "\" target=\"" + e.get(1) + "\" label=\"" + e.get(2) +
          "\" />\n").collect.mkString +
        "    </edges>\n" +
        "  </graph>\n" +
        "</gexf>"

    val strgexf = graphFrametoGexf(graph)
    //println(strgexf)
    val writer = new PrintWriter(new File("D:\\Tomcat7.0\\webapps\\linkurious.js\\examples\\data\\4444.gexf"))
    writer.print(strgexf)
    writer.close()

其次，利用linkurious.js進行可視化

sigma.parsers.gexf('data/4444.gexf', {
  // container: 'graph-container',
  renderer: {
    container: document.getElementById('graph-container'),
    type: 'canvas'
  },
  settings: {
    edgeColor: 'default',
    defaultEdgeColor: '#ccc',
    animationsTime: 5000,
    drawLabels: true,
    labelThreshold: 5,
    scalingMode: 'inside',
    batchEdgesDrawing: true,
    hideEdgesOnMove: true,
    sideMargin: 1
  }
}, function(s) {
  s.graph.nodes().forEach(function (n) {
    if (!s.graph.degree(n.id)) {
      s.graph.dropNode(n.id);
    }
    else {
      n.x = Math.random();
      n.y = Math.random();
    }
  });
  s.refresh();

  // Configure the ForceLink algorithm:
  var fa = sigma.layouts.configForceLink(s, {
    worker: true,
    autoStop: true,
    background: true,
    scaleRatio: 30,
    gravity: 3,
    easing: 'cubicInOut'
  });

  // Bind the events:
  fa.bind('start stop', function(e) {
    console.log(e.type);
    document.getElementById('layout-notification').style.visibility = '';
    if (e.type == 'start') {
      document.getElementById('layout-notification').style.visibility = 'visible';
    }
  });

  // Start the ForceLink algorithm:
  sigma.layouts.startForceLink();
});

測試

利用200個用戶及400個組進行構(gòu)圖測試，顯示效果流暢。(注意：節(jié)點名字不能包括Unicode字符，否則linkurious.js無法解析)

關(guān)系圖.png