提交Python 代码到Spark

没有设置spark环境变量的话：

cd /spark路径/bin
./spark-submit /usr/qy/test_pyspark.py

test_pyspark.py:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

# Wicle Qian
# 2015.11.19
# test the python in Spark without pyspark

from pyspark import *

def test():
    sc = SparkContext('local','qy_test')
    print sc
    textFile = sc.textFile("file:///usr/local/cluster/spark/README.md")
    print textFile.count()
    print textFile.first()

if __name__ == '__main__':
    test()