Spark Core 实现排序的三种方式

原文链接： https://my.oschina.net/dreamness/blog/3094234

一、普通的实现方式

---

    val data: RDD[String] = sc.parallelize(Array("aa 200 3000", "bb 200 4000", "cc 999 3000", "dd 10 20000"))
    data.map(x => {
      val splits: Array[String] = x.split(" ")
      val name: String = splits(0)
      val price: Int = splits(1).toInt
      val store: Int = splits(2).toInt
      (name, price, store)
    }).sortBy(x => (-x._2, -x._3)).foreach(println(_))

二、自定义类， 实现Order方法

---

    val data: RDD[String] = sc.parallelize(Array("aa 200 3000", "bb 200 4000", "cc 999 3000", "dd 10 20000"))
    data.map(x => {
      val splits: Array[String] = x.split(" ")
      val name: String = splits(0)
      val price: Int = splits(1).toInt
      val store: Int = splits(2).toInt
      ProduceCaseClass(name, price, store)
    }).sortBy(x => x).foreach(println(_))

case class ProduceCaseClass(val name:String, val price: Int, val store: Int)
extends Ordered[ProduceCaseClass] with  Serializable {
  override def compare(that: ProduceCaseClass): Int = {
    this.price - that.price
  }
  override def toString: String = s"$name $price $store"
}

三、利用隐式转换，对普通的、不带比较规则的class进行排序

---

    val data: RDD[String] = sc.parallelize(Array("aa 200 3000", "bb 200 4000", "cc 999 3000", "dd 10 20000"))
    implicit def produceClass2Ordered(produceClass: ProduceClass): Ordered[ProduceClass] = new Ordered[ProduceClass]{
      override def compare(that: ProduceClass): Int = {
        that.price - produceClass.price
      }
    }

      data.map(x => {
          val splits: Array[String] = x.split(" ")
          val name: String = splits(0)
          val price: Int = splits(1).toInt
          val store: Int = splits(2).toInt
          new ProduceClass(name, price, store)
        }).sortBy(x => x).foreach(println(_))

class ProduceClass(val name:String, val price: Int, val store: Int)  extends Serializable {
  override def toString: String = s"$name $price $store"
}

转载于:https://my.oschina.net/dreamness/blog/3094234