installing aws on free instance - part 2

second time on AWS

all AWS instances were stopped.
starting will give then new ID.

For convenience list host names below
Master
MasterID :- ec2-54-173-207-5.compute-1.amazonaws.com
Slaves
Slave1 :- ec2-54-165-137-226.compute-1.amazonaws.com
Slave2 :- ec2-54-173-232-142.compute-1.amazonaws.com

# connect masters
#go to directory where these PEM files are stored. enter following commands one after another. following commands shoudl be repeaseted each time terminal is opened.

eval `ssh-agent`
ssh-add awssecuritykey.pem

# Update following fields with new ID/IP

#core site
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://ec2-54-173-232-142.compute-1.amazonaws.com:8020</value>
<final>true</final>
</property>
</configuration>


#mapred-site.xml
----------

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>


<property>
<name>mapred.job.tracker</name>
<value>hdfs://ec2-54-173-232-142.compute-1.amazonaws.com:8021</value>
<final>true</final>
</property>

</configuration>


# transfer filws on slaves
scp core-site.xml mapred-site.xml ubuntu@ec2-54-172-79-68.compute-1.amazonaws.com:/home/ubuntu/hadoop/conf

scp core-site.xml mapred-site.xml ubuntu@ec2-54-174-72-143.compute-1.amazonaws.com:/home/ubuntu/hadoop/conf


#slaves file
#on Master machine; mention both "IP ID" in slaves file.
ec2-54-165-137-226.compute-1.amazonaws.com
ec2-54-173-232-142.compute-1.amazonaws.com


#on slaves machine; only that machines IP ID should be entered. connect to slaves and update slaves files
on slave 1; "slaves" file will have "ec2-54-165-137-226.compute-1.amazonaws.com" this line only.
on slave 2; "slaves" file will have "ec2-54-173-232-142.compute-1.amazonaws.com" this line only.

# on master
cd hadoop/bin
start-all.sh

# run following example for testing
hadoop jar hadoop-mapreduce-examples-2.1.0-beta.jar pi 16 100000