(null) entry in command string exception in saveAsTextFile() on Pyspark
You are missing winutils.exe
a hadoop binary . Depending upon x64 bit / x32 bit System download the winutils.exe
file & set your hadoop home pointing to it.
1st way :
- Download the file
- Create
hadoop
folder in Your System, exC:
- Create
bin
folder inhadoop
directory, ex :C:\hadoop\bin
- paste
winutils.exe
inbin
, ex:C:\hadoop\bin\winutils.exe
- In User Variables in System Properties -> Advance System Settings
Create New Variable
Name: HADOOP_HOME
Path: C:\hadoop\
2nd Way :
You can set hadoop home directly in Your Java Program with the following Command like this :
System.setProperty("hadoop.home.dir","C:\hadoop" );
I had a similar exception saying permission issue when loading a model built in some other machine and copied in my Windows system although my HADOOP_HOME
was set.
I just ran the following command over my model folder:
winutils.exe chmod -R 777 model-path