Skip to main content

Changing JAVA_HOME on a HDP 1.1 Hadoop on Windows Setup

We were setting up a small cluster of  Hadoop for some requirements, and decided to go with the HortonWorks release of Hadoop. (HDP 1.1 for Windows).

After installing, we realized that for some reason the Oozie service was not starting up . (we tried both the CLI command provided with the HDP installation, as well as trying to manually start the service via services.msc).

Looking through the log files (<HDP 1.1 installation path>\oozie-<version>\Service directory, specifically the oozieservice.out.log file) we found the error -
   and so realized that Oozie server is dependent on JDK. It does not work with only JRE.

To correct this we uninstalled JRE, and installed JDK. We also manually modified the JAVA_HOME environment variable to point to the new installation path. And then Oozie server started working all fine.
But all the rest of the services (hadoop jobtracker, hadoop datanode etc.) stopped starting.

Checking the logs of the services, we found that all the services seemed to be referring to the old JRE path which did not exist. Thinking that maybe the environment variables were being cached by the service, we restarted the machine in vain. Exploring this further we saw that there were many XML files which contained the initial JAVA_HOME folder as a part of the XML configuration, rather than picking the environemtn variable  dynamically at run time.

Finally we got all our hadoop services on our master node, by modifying the following XML files -

  • <hdp1.1 installation path>\hive-0.9.0\bin\metastore.xml
  • <hdp1.1 installation path>\hive-0.9.0\bin\hwi.xml
  • <hdp1.1 installation path>\hive-0.9.0\bin\hiveserver.xml
  • <hdp1.1 installation path>\hive-0.9.0\bin\derbyserver.xml
  • <hdp1.1 installation path>\hadoop-1.1.0-SNAPSHOT\bin\tasktracker.xml
  • <hdp1.1 installation path>\hadoop-1.1.0-SNAPSHOT\bin\secondarynamenode.xml
  • <hdp1.1 installation path>\hadoop-1.1.0-SNAPSHOT\bin\namenode.xml
  • <hdp1.1 installation path>\hadoop-1.1.0-SNAPSHOT\bin\jobtracker.xml
  • <hdp1.1 installation path>\hadoop-1.1.0-SNAPSHOT\bin\historyserver.xml
  • <hdp1.1 installation path>\hadoop-1.1.0-SNAPSHOT\bin\datanode.xml
Similarly for xml files within templeton, sqoop.

I suggest using TextPad's Search sub directory feature for finding all the locations where the JRE path is mentioned.


Labels:

Comments

SSI Education said…

if you are looking for the Big Data Training in indore i would highly recommened you the ssi , they are the one the best education institute in indore
10/07/2015 2:34 PM
srav7n said…
The information which you have provided is very good. It is very useful and easily understood. who is looking for Hadoop Online Training with Experts
11/18/2015 3:55 PM
Post a Comment

Popular posts from this blog

So you have your website deployed in PROD ... now what ??

Posting on behalf of Usr.Web.Speed - My previous job had been to architect and develop websites for various customers. During that time my team and I have architected and developed various web applications mainly for enterprises. (But below info is not restricted to enterprises) Other than the usual development and testing tasks involved, our focus area was to abide by multiple SLAs. One of the primary SLAs was to provide the users of our websites a very low (usually subsecond) response time (or page load time). To adhere to this SLA, we did multiple activities, in code, process as well as infrastructure. These include (but not limit to) - Using best practices including (http://developer.yahoo.com/performance/rules.html) Determining the optimum number of calls to the databases, open connections, etc. Providing the fastest mechanisms to download associated content (such as stylesheets, JS files, etc. over CDN) And debugging the reason for the slowness of the websites, when ...
Post a Comment
Read more

Theming in Whidbey using CSS

Continuing my blog titled Themes in Whidbey . I would like to mention some great points I did find out today about Whidbey featrures. How many times have you got sick of developing Sites which have themes???? I mean every page u create has to have, maybe in ASP, something like <link href="<%=Session["ThemeFile"]%>.css"/> I have been doing since a long time. Do mention if you have a simpler method of defining a variable CSS for your page!!! Whidbey simplifies this further. Now in ASP.NET, all you will have to do is Create Themes directory, if not aldready present, Create a directory for each theme required, maybe Black, Gray,etc. Dump your CSS files into this directory. in Page_PreInit method define the Theme as Page.Theme="<Theme directory Name>". Further can be made into Page.Theme=(string)Session["ThemeType"] and viola, your theme management is take care by ASP.NET When you want to change the theme, the...
Post a Comment
Read more