Skip to main content

Changing JAVA_HOME on a HDP 1.1 Hadoop on Windows Setup

We were setting up a small cluster of  Hadoop for some requirements, and decided to go with the HortonWorks release of Hadoop. (HDP 1.1 for Windows).

After installing, we realized that for some reason the Oozie service was not starting up . (we tried both the CLI command provided with the HDP installation, as well as trying to manually start the service via services.msc).

Looking through the log files (<HDP 1.1 installation path>\oozie-<version>\Service directory, specifically the oozieservice.out.log file) we found the error -
   and so realized that Oozie server is dependent on JDK. It does not work with only JRE.

To correct this we uninstalled JRE, and installed JDK. We also manually modified the JAVA_HOME environment variable to point to the new installation path. And then Oozie server started working all fine.
But all the rest of the services (hadoop jobtracker, hadoop datanode etc.) stopped starting.

Checking the logs of the services, we found that all the services seemed to be referring to the old JRE path which did not exist. Thinking that maybe the environment variables were being cached by the service, we restarted the machine in vain. Exploring this further we saw that there were many XML files which contained the initial JAVA_HOME folder as a part of the XML configuration, rather than picking the environemtn variable  dynamically at run time.

Finally we got all our hadoop services on our master node, by modifying the following XML files -

  • <hdp1.1 installation path>\hive-0.9.0\bin\metastore.xml
  • <hdp1.1 installation path>\hive-0.9.0\bin\hwi.xml
  • <hdp1.1 installation path>\hive-0.9.0\bin\hiveserver.xml
  • <hdp1.1 installation path>\hive-0.9.0\bin\derbyserver.xml
  • <hdp1.1 installation path>\hadoop-1.1.0-SNAPSHOT\bin\tasktracker.xml
  • <hdp1.1 installation path>\hadoop-1.1.0-SNAPSHOT\bin\secondarynamenode.xml
  • <hdp1.1 installation path>\hadoop-1.1.0-SNAPSHOT\bin\namenode.xml
  • <hdp1.1 installation path>\hadoop-1.1.0-SNAPSHOT\bin\jobtracker.xml
  • <hdp1.1 installation path>\hadoop-1.1.0-SNAPSHOT\bin\historyserver.xml
  • <hdp1.1 installation path>\hadoop-1.1.0-SNAPSHOT\bin\datanode.xml
Similarly for xml files within templeton, sqoop.

I suggest using TextPad's Search sub directory feature for finding all the locations where the JRE path is mentioned.


Labels:

Comments

SSI Education said…

if you are looking for the Big Data Training in indore i would highly recommened you the ssi , they are the one the best education institute in indore
10/07/2015 2:34 PM
srav7n said…
The information which you have provided is very good. It is very useful and easily understood. who is looking for Hadoop Online Training with Experts
11/18/2015 3:55 PM
Post a Comment

Popular posts from this blog

The fight is in the open

Been busy lately and so have not managed to post for quite long. But I just had to post this. This post is dedicated to all persons who dont know anything about the fight between MS and Google: Check out the links starting from: http://news.zdnet.com/2100-9588_22-5875433.html?tag=nl.e589 There are more links as you get thru the article. Be sure to read those too. And just as a side-note: Check out " Atlas ". These are client side and server side components, which easily (??) enable you to make Web calls. This is done by encapsulating AJAX. (please dont insult by telling me that you dont know what that is).
4 comments
Read more

Imagine...an internet based world

Was reading "The Google Story", when i got this wierd idea. Consider a stripped down version of Linux, meant only for doing simple things such as browsing the internet, to read from the various drives (which is primarily meant only for storage of personal data), to play music and some single-player games. All the rest provided by the internet. Information searching/Chatting/Mailing/Documentation editing/journal maintenance/daily schedule maintenance/news/rss/shopping, etc. What else will you use for your computer for? Ok, actually i have been working on mainframes for some time now and am getting adjusted to the "dumb terminal" concept. Then i started associating the concept with scenario we are observing all over the net. As a matter of fact this will reduce the power of processing required on the client side also!!! Looks like the world is changing.... for the bettter, yup, i am sure. technorati tags: New internet age , dumb terminal , internet based world
3 comments
Read more

Finally decided

I have been having these various itch since quite some time. Both about what i am doing here (where i work) and what am i meant to do. Finally, i have decided "I need a job change". I guess IT services industry was never meant to be the right industry for me. Don't get me wrong there, it's not that i wanna quit because i am not getting any interesting work, it's just that I have ideas about life which does not match with what my company is doing. (As amatter of fact, they have some really beautiful concepts which are great to work on). But i have believed web to be the future and now i have started realising that personal dreams do become quite a pain in the neck, when they are ignoired. Huh.... now that i put it on a log, i feel much better. Back to work...
4 comments
Read more