anaconda中pyspark_自学大数据——9 Anaconda安装与使用pyspark

首先从Anaconda官网上下载Anaconda。

一、解压安装包

sudo bash Anaconda3-2020.07-Linux-x86_64.sh

chown -R hadoop:hadoop /opt/anaconda/

vi /etc/profile

export ANACONDA_HOME=/opt/anaconda

export PATH=$PATH:$ANACONDA_HOME/bin:

source /etc/profile

二、配置Jupyter Notebook远程访问添加anaconda清华源

conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/

conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/

conda config --set show_channel_urls yes

conda config --show

创建虚拟环境

conda create -n your_env_name python=X.X

conda activate ml

conda install jupyter

conda install pyspark

pip install modin spark-sklearn modin[dask] -i https://pypi.tuna.tsinghua.edu.cn/simple

配置Jupyter Notebook

jupyter notebook