Spark的One Stack to rule them all的特性，在Spark SQL即有显现。在传统的基于Hadoop的解决方案中，需要另外安装Pig或者Hive来解决类SQL的即席查询问题。
本文以Spark Shell交互式命令行终端简单的体验下Spark提供的类SQL的数据查询能力
上传数据到HDFS
首先将测试数据上传到HDFS中，本文用到的测试数据来自于Spark安装里面的people.txt文件，它位于spark-1.2.0-bin-hadoop2.4\examples\src\main\resources\people.txt。people.txt的文件内容是：
Michael, 29
Andy, 30
Justin, 19
使用如下命令将people.txt上传至HDFS(people.txt已经拷贝至当前目录
hdfs dfs -put people.txt /user/hadoop
Spark Shell操作
1. 创建SQLContext对象
val cxt = new org.apache.spark.sql.SQLContext(sc);
cxt: org.apache.spark.sql.SQLContext = org.apache.spark.sql.SQLContext@ab552b0
2. 引入隐式转化，用于把RDD转换为SchemaRDD
scala> import cxt._
import cxt._
3. 创建一个POJO类Person
scala> case class Person(name: String, age: Int)
defined class Person
4. 读取HDFS中的数据并ORM为Person集合
scala> val people = sc.textFile("people.txt").map(_.split(",")).map(p => Person(p(0),p(1).trim.toInt))
5. 查看people这个RDD的lineage的关系
scala> people.toDebugString
15/01/03 06:25:17 INFO mapred.FileInputFormat: Total input paths to process : 1
res0: String =
(1) MappedRDD[3] at map at :19 []
| MappedRDD[2] at map at :19 []
| people.txt MappedRDD[1] at textFile at :19 []
| people.txt HadoopRDD[0] at textFile at :19 []
6. 将people这个RDD注册为一个虚拟表People
scala> people.registerAsTable("People")
此时查看people的RDD lineage关系，结果同第5步一样
scala> people.toDebugString
res2: String =
(1) MappedRDD[3] at map at :19 []
| MappedRDD[2] at map at :19 []
| people.txt MappedRDD[1] at textFile at :19 []
| people.txt HadoopRDD[0] at textFile at :19 []
7. 对People表进行查询并查看查询计划和物理计划
scala> val teenagers = cxt.sql("select name from People where age < 20 and age > 10");
teenagers: org.apache.spark.sql.SchemaRDD =
SchemaRDD[6] at RDD at SchemaRDD.scala:108
== Query Plan ==
== Physical Plan ==
Project [name#0]
Filter ((age#1 < 20) && (age#1 > 10))
PhysicalRDD [name#0,age#1], MapPartitionsRDD[4] at mapPartitions at ExistingRDD.scala:36
scala> teenagers.toDebugString
res3: String =
(1) SchemaRDD[6] at RDD at SchemaRDD.scala:108
== Query Plan ==
== Physical Plan ==
Project [name#0]
Filter ((age#1 < 20) && (age#1 > 10))
PhysicalRDD [name#0,age#1], MapPartitionsRDD[4] at mapPartitions at ExistingRDD.scala:36 []
| MapPartitionsRDD[8] at mapPartitions at basicOperators.scala:43 []
| MapPartitionsRDD[7] at mapPartitions at basicOperators.scala:58 []
| MapPartitionsRDD[4] at mapPartitions at ExistingRDD.scala:36 []
| MappedRDD[3] at map at :19 []
| MappedRDD[2] at map at :19 []
| people.txt MappedRDD[1] at textFile at :19 []
| people.txt HadoopRDD[0] at textFile at :19 []
8. 提交查询作业，打印结果
teenagers.map(t => "Name:" + t(0)).collect().foreach(println)
///结果
Justin
参考：http://spark.apache.org/docs/latest/sql-programming-guide.html#getting-started