pyspark和jupyter在mac osx上的配置和应用

0、mac osx ei capitain系统是10.11.3 

1、下载pyspark

https://spark.apache.org/downloads.html

2、安装虚拟环境

sudo pip install virtualenv

3、创建虚拟环境

 virtualenv ipython_notebook

4、进入虚拟环境ipython_notebook

source  ./ipython_notebook/bin/active

5、下载jupyter

pip install jupyter

6、环境变量配置

export SPARK_HOME=/Users/winsun/spark

export PATH=$SPARK_HOME/bin:$PATH

export PYSPARK_SUBMIT_ARGS="--master local[2]"

export PYTHONPATH=/usr/bin/python:$SPARK_HOME/python:$SPARK_HOME/python/build:$SPARK_HOME/python/lib/py4j-0.9-src.zip:$PYTHONPATH

7、进入交互式分析环境

方法一：IPYTHON_OPTS="notebook"$SPARK_HOME/bin/pyspark

方法二：ipython notebook

参考链接：

［1］jupyter http://npatta01.github.io/2015/08/01/pyspark_jupyter/

［2］virtualenv http://www.cnblogs.com/tk091/p/3700013.html