Write a file in hdfs with Java

Solution 1:

an alternative to @Tariq's asnwer you could pass the URI when getting the filesystem

import org.apache.hadoop.fs.FileSystem
import org.apache.hadoop.conf.Configuration
import java.net.URI
import org.apache.hadoop.fs.Path
import org.apache.hadoop.util.Progressable
import java.io.BufferedWriter
import java.io.OutputStreamWriter

Configuration configuration = new Configuration();
FileSystem hdfs = FileSystem.get( new URI( "hdfs://localhost:54310" ), configuration );
Path file = new Path("hdfs://localhost:54310/s2013/batch/table.html");
if ( hdfs.exists( file )) { hdfs.delete( file, true ); } 
OutputStream os = hdfs.create( file,
    new Progressable() {
        public void progress() {
            out.println("...bytes written: [ "+bytesWritten+" ]");
        } });
BufferedWriter br = new BufferedWriter( new OutputStreamWriter( os, "UTF-8" ) );
br.write("Hello World");
br.close();
hdfs.close();

Solution 2:

Either define the HADOOP_CONF_DIR environment variable to your Hadoop configuration folder or add the following 2 lines in your code :

config.addResource(new Path("/HADOOP_HOME/conf/core-site.xml"));
config.addResource(new Path("/HADOOP_HOME/conf/hdfs-site.xml"));

If you don't add this, your client will try to write to the local FS, hence resulting into the permission denied exception.