Storm中內(nèi)置了7種分組方式

Shuffle grouping

• 定義: Tuples are randomly distributed across the bolt's tasks in a way such that each bolt is guaranteed to get an equal number of tuples.

• 樣例
  此樣例由Storm的官方提供，通過下面這個例子可以對Shuffle grouping有更直觀的認識

  public class ExclamationTopology {
  
  public static class ExclamationBolt extends BaseRichBolt {
      OutputCollector _collector;
  
      @Override
      public void prepare(Map conf, TopologyContext context, OutputCollector collector) {
          _collector = collector;
      }
  
      @Override
      public void execute(Tuple tuple) {
          System.out.println(tuple.getString(0) + " is from task " + tuple.getSourceTask() + " of Spout/Bolt:" + tuple.getSourceComponent());
  
          _collector.emit(tuple, new Values(tuple.getString(0) + "!!!"));
          _collector.ack(tuple);
      }
  
      @Override
      public void declareOutputFields(OutputFieldsDeclarer declarer) {
          declarer.declare(new Fields("word"));
      }
  
  }
  
  public static void main(String[] args) throws Exception {
      TopologyBuilder builder = new TopologyBuilder();
  
      builder.setSpout("word", new TestWordSpout(), 10);
      builder.setBolt("exclaim1", new ExclamationBolt(), 3).shuffleGrouping("word");
      builder.setBolt("exclaim2", new ExclamationBolt(), 2).shuffleGrouping("exclaim1");
  
      Config conf = new Config();
      conf.setDebug(true);
  
      if (args != null && args.length > 0) {
          conf.setNumWorkers(30);
  
          StormSubmitter.submitTopologyWithProgressBar(args[0], conf, builder.createTopology());
      }
      else {
  
          LocalCluster cluster = new LocalCluster();
          cluster.submitTopology("test", conf, builder.createTopology());
          Utils.sleep(5000);
          cluster.killTopology("test");
          cluster.shutdown();
      }
  }
  

}
```

本地運行這個樣例，會有類似如下的日志打印，從這個打印中可以看到，Bolt exclaim1的數(shù)據(jù)來自于Spout word的10個task，即task[7-16]

```
jackson is from task 11 of Spout/Bolt:word
mike is from task 8 of Spout/Bolt:word
nathan is from task 12 of Spout/Bolt:word
nathan is from task 16 of Spout/Bolt:word
nathan is from task 13 of Spout/Bolt:word
```

Fields grouping

• 定義:The stream is partitioned by the fields specified in the grouping. For example, if the stream is grouped by the "user-id" field, tuples with the same "user-id" will always go to the same task, but tuples with different "user-id"'s may go to different tasks.

• 樣例

  對上面的樣例稍加改造

  builder.setSpout("word", new TestWordSpout(), 10);
  builder.setBolt("exclaim1", new ExclamationBolt(), 3).fieldsGrouping("word", new Fields("word"));
  builder.setBolt("exclaim2", new ExclamationBolt(), 2).shuffleGrouping("exclaim1");
  

  從運行的結(jié)果中可以看到類似如下的打印，說明相同的字符都來自于同一個task

  mike!!! is from task 2 of Spout/Bolt:exclaim1
  mike!!! is from task 2 of Spout/Bolt:exclaim1
  mike!!! is from task 2 of Spout/Bolt:exclaim1
  mike!!! is from task 2 of Spout/Bolt:exclaim1
  

  或者在execute方法中在加如下的打印System.out.println("Current thread is " + Thread.currentThread().getId() + " to emit " + tuple.getString(0) + "!!!");，可以看到類似如下的打印，所有的mike!!!都是由同一個線程處理的。

  Current thread is 124 to emit mike!!!
  Current thread is 124 to emit mike!!!
  

All grouping

• 定義:The stream is replicated across all the bolt's tasks. Use this grouping with care.

• 樣例

  對上面的樣例稍加改造

  builder.setSpout("word", new TestWordSpout(), 10);
  builder.setBolt("exclaim1", new ExclamationBolt(), 3).allGrouping("word");
  builder.setBolt("exclaim2", new ExclamationBolt(), 2).shuffleGrouping("exclaim1");
  

  從運行的結(jié)果中可以看到類似如下的打印，因為Bolt exclaim1的有3個task，所以下面的結(jié)果說明了，Bolt exclaim2要從每個task中都取一次

  Current thread is 124 to emit mike!!!
  Current thread is 128 to emit mike!!!
  Current thread is 150 to emit mike!!!
  
  [Thread-18-exclaim1-executor[2 2]] INFO  o.a.s.d.executor - TRANSFERING tuple [dest: 6 tuple: source: exclaim1:2, stream: default, id: {}, [mike!!!]]
  [Thread-22-exclaim1-executor[3 3]] INFO  o.a.s.d.executor - TRANSFERING tuple [dest: 5 tuple: source: exclaim1:3, stream: default, id: {}, [mike!!!]]
  [Thread-44-exclaim1-executor[4 4]] INFO  o.a.s.d.executor - TRANSFERING tuple [dest: 6 tuple: source: exclaim1:4, stream: default, id: {}, [mike!!!]]
  

Global grouping

• 定義:The entire stream goes to a single one of the bolt's tasks. Specifically, it goes to the
  task with the lowest id.

• 樣例

  對上面的樣例稍加改造

   builder.setSpout("word", new TestWordSpout(), 10);
   builder.setBolt("exclaim1", new ExclamationBolt(), 3).globalGrouping("word");
   builder.setBolt("exclaim2", new ExclamationBolt(), 2).shuffleGrouping("exclaim1");
  

  結(jié)果中會有類似如下的打印，說明mike!!!都來自于了同一個Bolt

  mike!!! is from task 2 of Spout/Bolt:exclaim1
  mike!!! is from task 2 of Spout/Bolt:exclaim1
  mike!!! is from task 2 of Spout/Bolt:exclaim1
  

None grouping

• 定義:This grouping specifies that you don't care how the stream is grouped. Currently, none
  groupings are equivalent to shuffle groupings.

Direct grouping

• 定義:This is a special kind of grouping. A stream grouped this way means that the producer
  of the tuple decides which task of the consumer will receive this tuple. Direct groupings can only be declared on streams that have been declared as direct streams. Tuples emitted to a direct stream must be emitted using one of the emitDirect methods.

Local or shuffle grouping

• 定義: If the target bolt has one or more tasks in the same worker process, tuples will be shuffled to just those in-process tasks. Otherwise, this acts like a normal shuffle grouping.