2009年4月20日 星期一

hadoop cluster and eclipse plugin

Eclipse : 3.3.2
Hadoop: 0.19.1
Hadoop-eclipse-plugin: 0.19.1, in contrib/eclipse-plugin

cluster environment:
grid01(cluster head) has two network interfaces:
140.112.30.240 --> external
192.168.77.1 --> internal

grid02~grid05 are in the private network:
192.168.77.2~5.

Since we need to contact the internal nodes when running hadoop applications,
we need to setup proxy to have grid02~grid05 accessible to people outside.

Setup proxy port 54088:

ssh -L 140.112.30.240:12345:192.168.77.1:22 140.112.30.240 -n -N &
ssh -D 140.112.30.240:54088 -p 12345 140.112.30.240 -n -N &

And then in hadoop-eclipse plugin, you can set proxy host/port as 140.112.30.240:54088

沒有留言:

張貼留言