I was doing some work in AWS and I decided to stop my free tier instance and start up a new medium tier instance. I was going through the usual drill of selecting the Amazon AMI, configuring and tagging the instance, and then launching it. Once in installing Anaconda and setting up Hadoop except this time I ran into a problem I didn’t have in the free instance so I thought I would write about it and how I solved it in case someone else hits a similar snag.
After running the start scripts for hdfs, yarn, and history server you should run jps to get the list of java processes running to make sure things are working. Unlike the last time in the other instance where there was no problem when I ran it this time I got an error message:
[ec2-user@ip ~]$ jps
-bash: jps: command not found
That is strange since the OpenJDK is installed already as part of the AMI so my first thought was that there must be a path problem which seemed reasonable especially when I found a StackExchange thread with the same problem/answer. Unfortunately this did not solve the problem so I began to wonder why. I checked the contents of openjdk’s bin and I noticed there was no jps.
[ec2-user@ip ~]$ ls /usr/lib/jvm/jre-1.7.0-openjdk.x86_64/bin
java keytool orbd pack200 policytool rmid rmiregistry servertool tnameserv unpack200
So back to Google to find another StackExchange thread which indicated that the openjdk development tools also needed to be installed and apparently had not been as part of this AMI build.
[ec2-user@ip ~]$ sudo yum install java-1.7.0-openjdk-devel
Loaded plugins: priorities, update-motd, upgrade-helper
Resolving Dependencies
–> Running transaction check
—> Package java-1.7.0-openjdk-devel.x86_64 1:1.7.0.121-2.6.8.1.69.amzn1 will be installed
–> Finished Dependency Resolution
Dependencies Resolved
=============================================================================================================================================================================================================================================
Package Arch Version Repository Size
=============================================================================================================================================================================================================================================
Installing:
java-1.7.0-openjdk-devel x86_64 1:1.7.0.121-2.6.8.1.69.amzn1 amzn-updates 11 M
Transaction Summary
=============================================================================================================================================================================================================================================
Install 1 Package
Total download size: 11 M
Installed size: 36 M
Is this ok [y/d/N]: y
Downloading packages:
java-1.7.0-openjdk-devel-1.7.0.121-2.6.8.1.69.amzn1.x86_64.rpm | 11 MB 00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : 1:java-1.7.0-openjdk-devel-1.7.0.121-2.6.8.1.69.amzn1.x86_64 1/1
Verifying : 1:java-1.7.0-openjdk-devel-1.7.0.121-2.6.8.1.69.amzn1.x86_64 1/1
Installed:
java-1.7.0-openjdk-devel.x86_64 1:1.7.0.121-2.6.8.1.69.amzn1
Complete!
This did end up solving the problem
[ec2-user@ip ~]$ jps
5264 Jps
4248 NodeManager
3745 NameNode
4553 JobHistoryServer
3863 DataNode
4012 SecondaryNameNode
4152 ResourceManager
At least it is fixed now.
A simpler suggestion came from Professor Alexander Rasin which is simply to install Apache Ant by running:
sudo yum install ant
This will install the needed dependencies.