This means that for something is wrong in the ssh connections in the cluster. For rsync (or hadoop in cluster) to work, the you should be able to ssh between the following hadoop components without any password or prompts.
- Jobtracker to Tasktrackers
- Jobtracker to Namenode
- Namenode to DataNodes
- Namenode to Jobtracker
- Datanodes to NameNode
- Tasktrackers to Jobtracker
Once ssh is working between the above 6 directions, these errors should go away.
EXAMPLE
Supppose a hadoop cluster is composed of the following machines
j.jeka.com : Jobtracker
n.jeka.com: Name node
t1.jeka.com: Datanode and Tasktracker
t1.jeka.com: Datanode and Tasktracker
then from following ssh's should work
- Jobtracker to Tasktrackers
j.jeka.com > n.jeka.com
- Jobtracker to Tasktrackers
j.jeka.com > t1.jeka.com
j.jeka.com > t2.jeka.com
- Namenode to Jobtracker
n.jeka.com > j.jeka.com
- Namenode to DataNodes
n.jeka.com > t1.jeka.com
n.jeka.com > t2.jeka.com
- Datanodes to NameNode
t1.jeka.com > n.jeka.com
t2.jeka.com > n.jeka.com
- Tasktrackers to Jobtracker
t1.jeka.com > j.jeka.com
t2.jeka.com > j.jeka.com
Please note that it is not required to be able to ssh from one task tracker to another
No comments:
Post a Comment