spark优化之临时目录

官方是这样说的：

Directory to use for "scratch" space in Spark, including map output files and RDDs that get stored on disk. This should be on a fast, local disk in your system. It can also be a comma-separated list of multiple directories on different disks. NOTE: In Spark 1.0 and later this will be overriden by SPARK_LOCAL_DIRS (Standalone, Mesos) or LOCAL_DIRS (YARN) environment variables set by the cluster manager.

大概就是MAP和RDD的临时数据可能存放在spark.local.dir（SPARK_LOCAL_DIRS）指定的目录中，默认是在/tmp中。

该目录的磁盘一定是快速的本地磁盘系统，我们可以考虑把这个目录来单独放成一个磁盘，或者以都好隔开放入多个磁盘，来提高i/o读写速度