2016 第四讲 Scala模式匹配、类型系统彻底精通与Spark源码阅读
20160104 19:00-21:00 课程笔录
Scala模式匹配：可以对值，类型，集合进行匹配
（1）值匹配
// a. data值为Spark,Hadoop或其他时
def bigData(data: String){
data match {
case "Spark" => println("Wow!!!")
case "Hadoop" => println("Ok")
case _ => println(" Something others")
}
}
bigData:(data: String)Unit
>bigData("Spark")
Spark
>bigData("Hadoop")
Hadoop
>bigData("else")
Something others
//b. 注意 data值为Flink时的写法, data_
def bigData(data: String){
data match {
case "Spark" => println("Wow!!!")
case "Hadoop" => println("Ok")
case
data_
if
data_ == "Flink" => println("Cool")
case _ => println ("Something else")
}
}
>bigData("Flink")
Cool
//c.
def bigData(data: String){
data match {
case "Spark" => println("Wow!!!")
case "Hadoop" => println("Ok")
case
_ if data == "Flink" => println("Cool")
case _ => println("Something else")
}
}
>bigData("Flink")
//d.
def bigData(data: String){
data match {
case "Spark" => println("Wow!!!")
case "Hadoop" => println("Ok")
case
data if data == "Flink" => println("Cool")
case _ => println("Something else")
}
}
>bigData("Flink")
(2)类型匹配
import java.io._
def exception(e:Exception) {
e match {
case fileException: FileNotFoundException => println("File not found :" + fileException)
case _: Exception => println("Exception getting thread dump from executor $executorId", e)
}
}
>exception(new FileNotFoundException(" Oops!!!")
(3)集合匹配
def data(array: Array[String]){
array match {
case Array("Scala") =>println("Scala")
case
Array(spark, hadoop, flink) =>println(spark + " : " + hadoop + " : " + flink)
case
Array("Spark", _*) => println("Spark ...")
case _ => println("Unknown")
}
}
>data(Array("Scala"))
Scala
>data(Array("hello","world","google"))
hello : world : google
>data(Array("Spark"))
Spark...
>data(Array("Spark","test"))
Spark...
类型系统
//
相当于java的Bean
case class
case class Person
//实例化
case class Person(name: String)
>class Person
>case class Worker(name: String, salary: Double) extends Person
>case class Student(name: String, score: Double) extends Person
def sayHi(person: Person){
person match {
case Student(name, score) => println(name + score)
case Worker(name, salary) => println(name + salary)
case _ => println("Unknown")
}
}
>sayHi(Worker("Spark ",6.4))
>sayHi(Student("Spark ",6.5))
>sayHi(Student("Hadoop ",6.6)
sayHi(Worker("Spark ",6.4))
类型体系
泛型类、泛型函数、类型界定（upperBound）、视图界定（View Bounds）、协变和逆变等
类型参数，主要用于类型限定，加强代码的健壮性
1. 泛型类和泛型函数
2. 类型界定
对类型也可以限定边界，upperBounds（<:表示，上边界，是它或它的子类），（>:表示，下边界，是某个类的父类或类本身）
3. 视图界定
//
Manifest -> ClassTag
abstract class RDD[T: ClassTag](
@transient private var _sc: SparkContext,
@transient private var deps: Seq[Dependency[_]]
)
@transient private var _sc: SparkContext,
@transient private var deps: Seq[Dependency[_]]
)
RDD[T: ClassTag]
T:ClassTag
//
{{{
* scala> def mkArray[T : ClassTag](elems: T*) = Array[T](elems: _*)
* mkArray: [T](elems: T*)(implicit evidence$1: scala.reflect.ClassTag[T])Array[T]
*
* scala> mkArray(42, 13)
* res0: Array[Int] = Array(42, 13)
*
* scala> mkArray("Japan","Brazil","Germany")
* res1: Array[String] = Array(Japan, Brazil, Germany)
* }}}
* scala> def mkArray[T : ClassTag](elems: T*) = Array[T](elems: _*)
* mkArray: [T](elems: T*)(implicit evidence$1: scala.reflect.ClassTag[T])Array[T]
*
* scala> mkArray(42, 13)
* res0: Array[Int] = Array(42, 13)
*
* scala> mkArray("Japan","Brazil","Germany")
* res1: Array[String] = Array(Japan, Brazil, Germany)
* }}}
作业：
阅读Spark源码 RDD、HadoopRDD、SparkContext、Master、Worker的源码，
并分析里面使用的所有的模式匹配和类型参数的内容,
从http://weibo.com/ilovepains获取代码 并完成作业。