PySpark Failed to locate the winutils binary

STEP 1: download right version of winutils.exe, I'm using spark 2.3 with hadoop 2.7, so I need to download Hadoop 2.7's winutils.exe.

https://github.com/steveloughran/winutils/blob/master/hadoop-2.7.1/bin/winutils.exe

STEP 2: put the winutils.exe under HADOOP_HOME/bin.

STEP 3: PyCharm -> Run -> Edit Configurations -> Environment Variables, add new variable HADOOP_HOME.

 

Reference: https://dzone.com/articles/working-on-apache-spark-on-windows