Skip to main content

Changing JAVA_HOME on a HDP 1.1 Hadoop on Windows Setup

We were setting up a small cluster of  Hadoop for some requirements, and decided to go with the HortonWorks release of Hadoop. (HDP 1.1 for Windows).

After installing, we realized that for some reason the Oozie service was not starting up . (we tried both the CLI command provided with the HDP installation, as well as trying to manually start the service via services.msc).

Looking through the log files (<HDP 1.1 installation path>\oozie-<version>\Service directory, specifically the oozieservice.out.log file) we found the error -
   and so realized that Oozie server is dependent on JDK. It does not work with only JRE.

To correct this we uninstalled JRE, and installed JDK. We also manually modified the JAVA_HOME environment variable to point to the new installation path. And then Oozie server started working all fine.
But all the rest of the services (hadoop jobtracker, hadoop datanode etc.) stopped starting.

Checking the logs of the services, we found that all the services seemed to be referring to the old JRE path which did not exist. Thinking that maybe the environment variables were being cached by the service, we restarted the machine in vain. Exploring this further we saw that there were many XML files which contained the initial JAVA_HOME folder as a part of the XML configuration, rather than picking the environemtn variable  dynamically at run time.

Finally we got all our hadoop services on our master node, by modifying the following XML files -

  • <hdp1.1 installation path>\hive-0.9.0\bin\metastore.xml
  • <hdp1.1 installation path>\hive-0.9.0\bin\hwi.xml
  • <hdp1.1 installation path>\hive-0.9.0\bin\hiveserver.xml
  • <hdp1.1 installation path>\hive-0.9.0\bin\derbyserver.xml
  • <hdp1.1 installation path>\hadoop-1.1.0-SNAPSHOT\bin\tasktracker.xml
  • <hdp1.1 installation path>\hadoop-1.1.0-SNAPSHOT\bin\secondarynamenode.xml
  • <hdp1.1 installation path>\hadoop-1.1.0-SNAPSHOT\bin\namenode.xml
  • <hdp1.1 installation path>\hadoop-1.1.0-SNAPSHOT\bin\jobtracker.xml
  • <hdp1.1 installation path>\hadoop-1.1.0-SNAPSHOT\bin\historyserver.xml
  • <hdp1.1 installation path>\hadoop-1.1.0-SNAPSHOT\bin\datanode.xml
Similarly for xml files within templeton, sqoop.

I suggest using TextPad's Search sub directory feature for finding all the locations where the JRE path is mentioned.


Labels:

Comments

SSI Education said…

if you are looking for the Big Data Training in indore i would highly recommened you the ssi , they are the one the best education institute in indore
10/07/2015 2:34 PM
srav7n said…
The information which you have provided is very good. It is very useful and easily understood. who is looking for Hadoop Online Training with Experts
11/18/2015 3:55 PM
Post a Comment

Popular posts from this blog

The fight is in the open

Been busy lately and so have not managed to post for quite long. But I just had to post this. This post is dedicated to all persons who dont know anything about the fight between MS and Google: Check out the links starting from: http://news.zdnet.com/2100-9588_22-5875433.html?tag=nl.e589 There are more links as you get thru the article. Be sure to read those too. And just as a side-note: Check out " Atlas ". These are client side and server side components, which easily (??) enable you to make Web calls. This is done by encapsulating AJAX. (please dont insult by telling me that you dont know what that is).
4 comments
Read more

Finally decided

I have been having these various itch since quite some time. Both about what i am doing here (where i work) and what am i meant to do. Finally, i have decided "I need a job change". I guess IT services industry was never meant to be the right industry for me. Don't get me wrong there, it's not that i wanna quit because i am not getting any interesting work, it's just that I have ideas about life which does not match with what my company is doing. (As amatter of fact, they have some really beautiful concepts which are great to work on). But i have believed web to be the future and now i have started realising that personal dreams do become quite a pain in the neck, when they are ignoired. Huh.... now that i put it on a log, i feel much better. Back to work...
4 comments
Read more

Movies common for 2 actors/actresses

One question: Which all movies did both George Clooney and Brad Pitt act in? Am sure you would be going to google. And what do you do after that? See all the links, and check each and every page, trying to find the common movies. Now why could the computer not do this? Why are you spending 5 minutes trying to find this out, when you should be able to request the computer to do this..... Enter Semantic search, a mechanism where you can query data for relationships which the human mind understands, and secondly the computer understands. There are many places where they are trying to build this semantic search. I build a small application which uses freebase . And built an application where you can provide the names of 2 actors/actresses and find the movies both worked on. Check the page http://www.nlighten.in/CommonMovies.html . Details about freebase: Freebase uses the databases provided by Wikipedia and multiple other providers, and provide all the data in OOP
2 comments
Read more