Tuesday, March 23, 2010

Compile hadoop eclipse-plugin 0.20.2 for Eclipse 3.5 on Mac

Since the original eclipse-plugin of Hadoop 0.20.2 doesn't work well, because the plugin is developed under eclipse europa, i.e., 3.3. The phenomenon of mine is that when I click the "Run on Hadoop", there will be nothing continue. Here (if you can read Chinese ^_^) offers a good way to ignore this, to write a self-write Makefile. This is easy, actually I am using this way finished my first little hadoop project, but also boring. So I want to make the eclipse-plugin work well. Then I found some other help from:
http://wiki.apache.org/hadoop/EclipsePlugIn
http://trac.nchc.org.tw/cloud/wiki/waue/2010/0211 (help me a lot)
http://hi.baidu.com/feng2211/blog/item/4ab8a2095eb1b9a42eddd48e.html

http://yjhexy.javaeye.com/blog/591769
All links above except the first one are written in Chinese.
However, this work is not so hard, I finished this in one hour, and I will describe my steps in the follow:

1. Modify two files

First modify this file,
$HADOOP_HOME/src/contrib/build-contrib.xml
add below property:

<property name="eclipse.home" location="$Eclipse_HOME" />

Here, the $Eclipse_HOME  is the application path of your own eclipse. For example, Since I am working on Mac OS X 10.5, so my $Eclipse_HOME is /Applications/eclipse .

The second file need to be modifed is,

$HADOOP_HOME/src/contrib/eclipse-plugin/src/java/org/apache/hadoop/eclipse/lauch/HadoopApplicationLaunchShortcut.java



comment the old import about JavaApplicationLaunchShortcut, and add a new one. Like this:



//import org.eclipse.jdt.internal.debug.ui.launcher.JavaApplicationLaunchShortcut; 
import org.eclipse.jdt.debug.ui.launchConfigurations.JavaApplicationLaunchShortcut;


2. Compile


The basic compile steps are as follows:




cd $HADOOP_HOME 
ant compile
ant pack


There are several problems I met when I executing the command ant pack, maybe it’s not usual, but I think it’s better to describe them all here:




  • First, I was told that:



  • build.xml:908: 'java5.home' is not defined. Forrest requires Java 5. Please pass -Djava5.home=&lt;base 5=&quot;&quot; distribution&gt; to Ant on the command-line.



It seems need to be indicate JDK5, since the Apache Forrest need use JDK5 to run. Fortunately, Mac OS X has several versions of JDK together by default, so I just set the pack command like this:



ant package -Djava5.home=/System/Library/Frameworks/JavaVM.framework/Versions/1.5.0



  • But, then I was told:



  • build.xml:904: 'forrest.home' is not defined. Please pass -Djava5.home=<base of Java 5 distribution> to Ant on the command-line.


That's because I don't have Apache Forrest in my computer. So I go to here to download it , then decompress it to $Forrest_HOME (self-defined) . Then use:




ant package -Djava5.home=/System/Library/Frameworks/JavaVM.framework/Versions/1.5.0 -Dforrest.home=$Forrest_HOME/apache-forrest-0.8



  • Then, the new problem is:



  • [exec] Error: JAVA_HOME is not defined correctly. 
    [exec] We cannot execute /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/bin/java



That's because the JDK in Mac OS X is different from Unix or Windows. I just create a link called bin under the JDK path to take a work around. Then everything left works very fine.





3. Result





The result jar file is inside:


$HADOOP_HOME/build/contrib/eclipse-plugin/hadoop-0.20.3-dev-eclipse-plugin.jar


It's very funny that file name is different from the default one. You can change it by your self interesting.

3 comments:

  1. Great instructions! I've been pulling my hair out for over a day trying to get it to work on eclipse.

    During your final step, how long does it hang on the:

    docs:
    [exec] Apache Forrest. Run 'forrest -projecthelp' to list options
    [exec]

    It seems to be running for ~20 minutes on my relatively new MBP.

    ReplyDelete
  2. Saved me a headache, thanks for the post!

    ReplyDelete
  3. Would you mind please tell me if there is a same procedure for hadoop 1.0.3?
    Since I receive this error:
    Target "pack" does not exist in the project "Hadoop".

    And do I have to install eclipse and hadoop on the same OS (Ubuntu desktop?)?
    Thanks.

    ReplyDelete