Tuesday, November 16, 2010

How to write a file in HDFS using hadoop

The following is the code to write a file from HDFS file system

1. Open File Write.java and paste the following code

package org.myorg;
import java.io.*;
import java.util.*;
import java.net.*;
import org.apache.hadoop.fs.*;
import org.apache.hadoop.conf.*;
import org.apache.hadoop.io.*;
import org.apache.hadoop.mapred.*;
import org.apache.hadoop.util.*;

public class Write{
public static void main (String [] args) throws Exception{
try{
Path pt=new Path("hdfs://jp.seka.com:9000/user/jfor/out/abc");
FileSystem fs = FileSystem.get(new Configuration());
BufferedWriter br=new BufferedWriter(new OutputStreamWriter(fs.create(pt,true)));
// TO append data to a file, use fs.append(Path f)
String line;
line="Disha Dishu Daasha";
System.out.println(line);
br.write(line);
br.close();
}catch(Exception e){
System.out.println("File not found");
}
}
}


2. Compile the code

javac -classpath hadoop-0.20.1-dev-core.jar -d Write/ Write.java


3. Create jar

jar -cvf Write.jar -C Write/ .


4. Run

hadoop jar Write.jar org.myorg.Write

No comments:

Post a Comment