java、scala集合自定义排序异常:Comparison method violates its general contract!

cala 集合(array,seq)排序报错
User class threw exception: org.apache.spark.SparkException: Job aborted due to stage failure: Task 43 in stage 3.0 failed 4 times, most recent failure: Lost task 43.3 in stage 3.0 (TID 1479, node9): java.lang.IllegalArgumentException: Comparison method violates its general contract!
at java.util.TimSort.mergeHi(TimSort.java:868)
at java.util.TimSort.mergeAt(TimSort.java:485)
at java.util.TimSort.mergeCollapse(TimSort.java:408)
at java.util.TimSort.sort(TimSort.java:214)
at java.util.TimSort.sort(TimSort.java:173)
at java.util.Arrays.sort(Arrays.java:659)
at scala.collection.SeqLike$class.sorted(SeqLike.scala:615)
at scala.collection.AbstractSeq.sorted(Seq.scala:40)
at scala.collection.SeqLike$class.sortWith(SeqLike.scala:572)
at scala.collection.AbstractSeq.sortWith(Seq.scala:40)
at com.wk.dict.seo.SupplyAggregation$$anonfun$2$$anonfun$apply$4.apply(SupplyAggregation.scala:40)
at com.wk.dict.seo.SupplyAggregation$$anonfun$2$$anonfun$apply$4.apply(SupplyAggregation.scala:39)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)

代码:

  val supplyKeywordsSort = supplyKeywords.sortWith((a, b) => {
          var flag = true
          val i = a._3 - b._3
          if (i > 0) {
            flag = false
          } else if (i == 0) {
            val j = a._4 - b._4
            if (j > 0) {
              flag = false
            }
          }
          flag
        })
原因:底层的排序算法TimSort,如果两个元素相等,返回为true的话,就会报该异常Comparison method violates its general contract!

解决办法:将相等的时候设为false

val supplyKeywordsSort = supplyKeywords.sortWith((a, b) => {
          var flag = true
          val i = a._3 - b._3
          if (i > 0) {
            flag = false
          } else if (i == 0) {
            val j = a._4 - b._4
            if (j >= 0) {
              flag = false
            }
          }
          flag
        })


你可能感兴趣的:(那些年,趟过的坑)