Skip to main content

Changing JAVA_HOME on a HDP 1.1 Hadoop on Windows Setup

We were setting up a small cluster of  Hadoop for some requirements, and decided to go with the HortonWorks release of Hadoop. (HDP 1.1 for Windows).

After installing, we realized that for some reason the Oozie service was not starting up . (we tried both the CLI command provided with the HDP installation, as well as trying to manually start the service via services.msc).

Looking through the log files (<HDP 1.1 installation path>\oozie-<version>\Service directory, specifically the oozieservice.out.log file) we found the error -
   and so realized that Oozie server is dependent on JDK. It does not work with only JRE.

To correct this we uninstalled JRE, and installed JDK. We also manually modified the JAVA_HOME environment variable to point to the new installation path. And then Oozie server started working all fine.
But all the rest of the services (hadoop jobtracker, hadoop datanode etc.) stopped starting.

Checking the logs of the services, we found that all the services seemed to be referring to the old JRE path which did not exist. Thinking that maybe the environment variables were being cached by the service, we restarted the machine in vain. Exploring this further we saw that there were many XML files which contained the initial JAVA_HOME folder as a part of the XML configuration, rather than picking the environemtn variable  dynamically at run time.

Finally we got all our hadoop services on our master node, by modifying the following XML files -

  • <hdp1.1 installation path>\hive-0.9.0\bin\metastore.xml
  • <hdp1.1 installation path>\hive-0.9.0\bin\hwi.xml
  • <hdp1.1 installation path>\hive-0.9.0\bin\hiveserver.xml
  • <hdp1.1 installation path>\hive-0.9.0\bin\derbyserver.xml
  • <hdp1.1 installation path>\hadoop-1.1.0-SNAPSHOT\bin\tasktracker.xml
  • <hdp1.1 installation path>\hadoop-1.1.0-SNAPSHOT\bin\secondarynamenode.xml
  • <hdp1.1 installation path>\hadoop-1.1.0-SNAPSHOT\bin\namenode.xml
  • <hdp1.1 installation path>\hadoop-1.1.0-SNAPSHOT\bin\jobtracker.xml
  • <hdp1.1 installation path>\hadoop-1.1.0-SNAPSHOT\bin\historyserver.xml
  • <hdp1.1 installation path>\hadoop-1.1.0-SNAPSHOT\bin\datanode.xml
Similarly for xml files within templeton, sqoop.

I suggest using TextPad's Search sub directory feature for finding all the locations where the JRE path is mentioned.


Labels:

Comments

SSI Education said…

if you are looking for the Big Data Training in indore i would highly recommened you the ssi , they are the one the best education institute in indore
10/07/2015 2:34 PM
srav7n said…
The information which you have provided is very good. It is very useful and easily understood. who is looking for Hadoop Online Training with Experts
11/18/2015 3:55 PM
Post a Comment

Popular posts from this blog

B.O.F.H.

Was browsing the net, and found some old link to BOFH. Not sure as to how many used to read PCQuest some time back when it used to come, but i remember reading them, with of course re-reads and re-re,etc...i can go on you know. Anyway, it's about a system operator who is bugged by no-brainers asking silly computer related queries, and this guy repling with wierd answers. I had never known the origin of this character, until i saw this . Anyway for those who have never read BOFH, here are two of them.
Post a Comment
Read more

So you have your website deployed in PROD ... now what ??

Posting on behalf of Usr.Web.Speed - My previous job had been to architect and develop websites for various customers. During that time my team and I have architected and developed various web applications mainly for enterprises. (But below info is not restricted to enterprises) Other than the usual development and testing tasks involved, our focus area was to abide by multiple SLAs. One of the primary SLAs was to provide the users of our websites a very low (usually subsecond) response time (or page load time). To adhere to this SLA, we did multiple activities, in code, process as well as infrastructure. These include (but not limit to) - Using best practices including (http://developer.yahoo.com/performance/rules.html) Determining the optimum number of calls to the databases, open connections, etc. Providing the fastest mechanisms to download associated content (such as stylesheets, JS files, etc. over CDN) And debugging the reason for the slowness of the websites, when ...
Post a Comment
Read more