(K.hashcode & Integer.MAX_VALE) % (reducer number)
hashpartitioner 相同key的數(shù)據(jù)一定會在同一個reducer中，但一個reducer中不就只有一個key

class HostPartitioner(ins:Array[String]) extends Partitioner {
    val parMap = new mutable.HashMap[String, Int]()
    val count = 0
    for (i <- ins) {
        parMap += (i -> count)
        count += 1
    }

    override def numPartitions:Int = ins.length
    verride def getPartition(key: Any):Int =  {
         parMap.getOrElse(key.toString, 0)
    }
}

調(diào)用的時候

val hostPartitioner = new HostPartitioner()
rdd.partitionBy(hostPartitioner)


rdd.partitionBy(new HashPartitioner(6))