windows环境运行datax缺少本地hadoop环境报错：Could not locate executable null\bin\winutils.exe

 

目录

一、完整报错

二、解决

三、临时解决方案

四、其他

---

一、完整报错

十二月 07, 2023 10:01:05 上午 org.apache.hadoop.util.Shell getWinUtilsPath
严重: Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
        at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:356)
        at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:371)
        at org.apache.hadoop.util.Shell.(Shell.java:364)
        at org.apache.hadoop.util.StringUtils.(StringUtils.java:80)
        at org.apache.hadoop.fs.FileSystem$Cache$Key.(FileSystem.java:2807)
        at org.apache.hadoop.fs.FileSystem$Cache$Key.(FileSystem.java:2802)
        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2668)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170)
        at com.alibaba.datax.plugin.writer.hdfswriter.HdfsHelper.getFileSystem(HdfsHelper.java:74)
        at com.alibaba.datax.plugin.writer.hdfswriter.HdfsWriter$Job.init(HdfsWriter.java:49)
        at com.alibaba.datax.core.job.JobContainer.initJobWriter(JobContainer.java:704)
        at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:304)
        at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:113)
        at com.alibaba.datax.core.Engine.start(Engine.java:94)
        at com.alibaba.datax.core.Engine.entry(Engine.java:176)
        at com.alibaba.datax.core.Engine.main(Engine.java:246)

十二月 07, 2023 10:01:05 上午 org.apache.hadoop.util.NativeCodeLoader 
警告: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

二、解决

        2.1、下载本地hadoop环境

                地址（免费下载）：windows本地hadoop环境包

        2.2、解压并配置环境变量

        环境变量配置如下：

        然后重启pycharm或电脑再运行datax.py即可

三、临时解决方案

        利用Python的os模块中datax.py文件上面加上如下代码，加入临时环境变量也可。

os.environ['HADOOP_HOME'] = r"D:\Hadoop"

     

四、其他

        如发现运行datax后，日志有乱码情况，可以在终端或控制台输入以下命令来解决。

chcp 65001