Arduino, Linux, Hadoop, Hive, H-Base, Postgres, Zabbix

Follow up on ETH'es not seen

Come on, help me.... If this helped you in any way and you have some spare BitCoins, you may donate them to me - 16tb2Rgn4uDptrEuR94BkhQAZNgfoMj3ug

This is a follow up on the post Problem with eth hardware not seen

This was causing my SLES build not to work, below is a script that will sort the entries in the udev rules out. Remember to reboot as soon as the script is done, make the script executable with chmod +x /opt/temp/sanity.sh

#!/bin/bash

# Hashing out entries for backup purposes
sed -i -e 's/SUBSYSTEM/#SUBSYSTEM/g' /etc/udev/rules.d/70-persistent-net.rules
echo "" >> /etc/udev/rules.d/70-persistent-net.rules
echo "# Added by the ETH MAC address sanity checker - King Rat" >> /etc/udev/rules.d/70-persistent-net.rules
echo "Adding the following entries into the udev rules 70-persistent-net.rules"

qq=`lspci -v | grep -i ethernet | wc -l`
for ((c=0; c<qq; c++ )); 
 do 
  macc=`ethtool -P eth$c | awk '{print $NF}'`
  echo "eth$c, $macc" >> /opt/temp/ETH_MAC_ADDRESSES
  echo 'SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="'$macc'", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth'$c'"'
  echo 'SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="'$macc'", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth'$c'"' >> /etc/udev/rules.d/70-persistent-net.rules
 done

Check the rules before rebooting as below

Ninja141:/opt/temp # cat /etc/udev/rules.d/70-persistent-net.rules
# This file was automatically generated by the /lib/udev/write_net_rules
# program, run by the persistent-net-generator.rules rules file.
#
# You can modify it, as long as you keep each rule on a single
# line, and change only the value of the NAME= key.

# PCI device 0x14e4:0x1657 (tg3)
#####SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="d8:9d:67:2a:10:f9", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

# PCI device 0x14e4:0x1657 (tg3)
#####SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="d8:9d:67:2a:10:fa", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth2"

# PCI device 0x14e4:0x1657 (tg3)
#####SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="d8:9d:67:2a:10:f8", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

# PCI device 0x14e4:0x1657 (tg3)
#####SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="d8:9d:67:2a:10:fb", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth3"

# Added by the ETH MAC address sanity checker - King Rat
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="d8:9d:67:2a:10:f8", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="d8:9d:67:2a:10:f9", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="d8:9d:67:2a:10:fa", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth2"
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="d8:9d:67:2a:10:fb", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth3"
Ninja141:/opt/temp #

Install Zabbix agent on SLES 11

Begging :.... If this helped you in any way and you have some spare BitCoins, you may donate them to me - 16tb2Rgn4uDptrEuR94BkhQAZNgfoMj3ug

Download the agent software from the Zabbix site :- http://www.zabbix.com/download.php
Pre-compiled Zabbix 2.2.1 (stable) agents are available for selected platforms

As root user
Make a directory to ftp the software to

mkdir -p /opt/temp

Add the group and user Zabbix : -useradd and groupadd zabbix
ftp file zabbix_agents_2.0.4.linux2_6_23.amd64.tar.gz, zabbix-agentd.conf and the .sh scripts to /opt/temp

svr3:~ # ls -ltr /opt/temp
total 4131360
-rw-r--r-- 1 root root 5703 Nov 24 09:08 zabbix_agentd.conf
-rw-r--r-- 1 root root 209205 Nov 24 09:09 zabbix_agents_1.8.3.linux2_6.amd64.tar.gz
svr3:~ # useradd zabbix
svr3:~ # groupadd zabbix
svr3:~ # cd /opt/temp
svr3:/opt/temp # tar -xvf zabbix_agents_2.0.4.linux2_6_23.amd64.tar.gz
sbin/zabbix_agent
sbin/zabbix_agentd
bin/zabbix_get
bin/zabbix_sender
conf/zabbix_agent.conf
conf/zabbix_agentd.conf
conf/zabbix_agentd/userparameter_examples.conf
conf/zabbix_agentd/userparameter_mysql.conf
svr3:/opt/temp #
svr3:/opt/temp # ls -ltr
total 328
-rw-r--r-- 1 root  root 286523 Feb  7 11:25 zabbix_agents_2.0.4.linux2_6_23.amd64.tar.gz
-rw-r--r-- 1 root  root   5999 Feb  7 11:30 zabbix_agentd.conf
drwxr-xr-x 2 root  root   4096 Feb  7 11:31 sbin
drwxr-xr-x 3 root  root   4096 Feb  7 11:31 conf
drwxr-xr-x 2 root  root   4096 Feb  7 11:31 bin
svr3:/opt/temp #

The code above will create sbin and bin directory under the directory /opt/temp, copy the contents of /opt/temp/sbin and /opt/temp/bin to /usr/local/sbin and /usr/local/bin respectively

svr3:/opt/temp # mv /opt/temp/sbin/* /usr/local/sbin/
svr3:/opt/temp # ls -ltr /opt/temp/sbin
total 0
szmtnsvr3:/opt/temp # ls -ltr /usr/local/sbin
total 432
-rwxr-xr-x 1 root sfcb 228624 Dec 10 13:41 zabbix_agentd
-rwxr-xr-x 1 root sfcb 199496 Dec 10 13:41 zabbix_agent
svr3:/opt/temp # mv /opt/temp/bin/* /usr/local/bin/
svr3:/opt/temp # ls -ltr /opt/temp/bin
total 0
svr3:/opt/temp # ls -ltr /usr/local/bin
total 224
-rwxr-xr-x 1 root sfcb 95464 Dec 10 13:41 zabbix_sender
-rwxr-xr-x 1 root sfcb 81560 Dec 10 13:41 zabbix_get
-rwx------ 1 root     root  8850 Feb  5 14:46 postsetup.sh
svr3:/opt/temp #

Change ownership of the copied files, touch the log and lock files, make directory /var/log/zabbix and copy /opt/temp/zabbix_agentd.conf to new directory /etc/zabbix

svr3:/opt/temp # chown zabbix:zabbix /usr/local/sbin/zabbix*
svr3:/opt/temp # chown zabbix:zabbix /usr/local/bin/zabbix*
svr3:/opt/temp # cd /var/log
svr3:/var/log # mkdir zabbix
svr3:/var/log # touch /var/log/zabbix/zabbix_agentd.log
svr3:/var/log # chown -R zabbix:zabbix zabbix/
svr3:/var/log # mkdir /etc/zabbix
svr3:/var/log # mv /opt/temp/zabbix_agentd.conf /etc/zabbix/
svr3:/var/log # ls -ltr /etc/zabbix/
total 8
-rw-r--r-- 1 root root 5999 Feb  7 11:30 zabbix_agentd.conf
svr3:/var/log # chown -R zabbix:zabbix /etc/zabbix/
svr3:/var/log # ls -ltr /etc/zabbix/
total 8
-rw-r--r-- 1 zabbix zabbix 5999 Feb  7 11:30 zabbix_agentd.conf
svr3:/var/log # touch /etc/zabbix/zabbix_agentd.pid
svr3:/var/log # chown zabbix:zabbix /etc/zabbix/zabbix_agentd.pid
svr3:/var/log #

vi the zabbix_agend.conf file and set the Zabbix Master server IP, also set the hostname of the server that you are installing Zabbix on, for example

vi /etc/zabbix/zabbix_agentd.conf
 
##### Passive checks related
 
### Option: Server
# List of comma delimited IP addresses (or hostnames) of Zabbix servers.
# No spaces allowed. First entry is used for receiving list of and sending active checks.
# Note that hostnames must resolve hostname->IP address and IP address->hostname.
# If IPv6 support is enabled then '127.0.0.1', '::127.0.0.1', '::ffff:127.0.0.1' are treated equally.
#
# Mandatory: yes
# Default:
# Server=
 
Server=xxx.xxx.xxx.xxx ## This is the Master Zabbix server IP
 
### Option: Hostname
# Unique, case sensitive hostname.
# Required for active checks and must match hostname as configured on the server.
#
# Default:
# Hostname=system.uname
 
Hostname=svr3.domain.com #

Make the startup and stop script, etc/init.d/zabbix-agent
vi /etc/init.d/zabbix-agent

#!/bin/bash
#
# Init file for Zabbix Agent
#
#
# chkconfig: - 80 12
# description: Zabbix agent startup
# processname: zabbix-agent
### BEGIN INIT INFO
# Provides: zabbix-agent
# Required-Start: $network $local_fs $remote_fs
# Required-Stop: $network $local_fs $remote_fs
# Default-Start: 3 5
# Default-Start: 0 6
# chkconfig: 12345 95 05
# Description: Supports the direct execution of binary formats.
### END INIT INFO
source /etc/rc.status
source /etc/profile.local
RETVAL=0
prog="zabbix_agent"
start() {
  runningCheck
  cd /usr/local/sbin
  echo -n $"Starting $prog: "
  ./zabbix_agentd -c /etc/zabbix/zabbix_agentd.conf
  rc_status -v
}
stop() {
        echo "Shutdown may take a while....";
        echo -n $"Shutting down $prog: "
         if [ `ps -ef | grep zabbix_agent | grep -v grep | wc -l` -eq 0 ]; then
            echo "No Zabbix agent running."
         else
            pkill zabbix_agentd
            rc_status -v
            RETVAL=0
            return $RETVAL
         fi
}
restart() {
  stop
  start
}
runningCheck() {
    rc=0
    if [ `ps -ef | grep zabbix_agent | grep -v grep | wc -l` -ge 1 ]; then
          rc=1
    fi
    if [ $rc -ne 0 ]; then
        echo "Zabbix agent is already running"
        exit 0
    fi
}
case "$1" in
  start)
  start
  ;;
  stop)
  stop
  ;;
  status)
  if [ `ps -ef | grep zabbix_agent | grep -v grep | wc -l` -eq 0 ]; then
         echo "No Zabbix agent running."
  else
      echo "Zabbix agent running(PID): `ps -ef | grep zabbix_agent | grep -v grep | grep -v tool | awk {'print $2'}`"
  fi
  RETVAL=$?
  ;;
  restart)
  restart
  ;;
   *)
  echo $"Usage: $0 {start|stop|status|restart}"
  RETVAL=1
esac
exit $RETVAL

Make the script executable

svr3:/var/log # chmod +x /etc/init.d/zabbix-agent

Start the Zabbix agent, monitor the logfile and add the server to the Zabbix server.

svr3:/var/log # ps -ef | grep zabbix
root 27908 27785 0 09:30 pts/3 00:00:00 grep zabbix
svr3:/var/log # /etc/init.d/zabbix-agent status
No Zabbix agent running.
svr3:/var/log # /etc/init.d/zabbix-agent start
Starting zabbix_agent:                                                                                                                                                                             done
svr3:/var/log # /etc/init.d/zabbix-agent status
Zabbix agent running(PID): 9596
9597
9598
9599
9600
svr3:/var/log #
svr3:/var/log # ps -ef | grep zabbix
zabbix    9596     1  0 11:42 ?        00:00:00 zabbix_agentd -c /etc/zabbix/zabbix_agentd.conf
zabbix    9597  9596  0 11:42 ?        00:00:00 zabbix_agentd -c /etc/zabbix/zabbix_agentd.conf
zabbix    9598  9596  0 11:42 ?        00:00:00 zabbix_agentd -c /etc/zabbix/zabbix_agentd.conf
zabbix    9599  9596  0 11:42 ?        00:00:00 zabbix_agentd -c /etc/zabbix/zabbix_agentd.conf
zabbix    9600  9596  0 11:42 ?        00:00:00 zabbix_agentd -c /etc/zabbix/zabbix_agentd.conf
root      9771  2257  0 11:43 pts/1    00:00:00 grep zabbix
svr3:/var/log # cd /var/log/zabbix/
svr3:/var/log/zabbix # ls -ltr
total 16
-rw-r--r-- 1 zabbix zabbix 12597 Feb  7 11:43 zabbix_agentd.log
svr3:/var/log/zabbix # tail -f zabbix_agentd.log
  9598:20130207:114346.259 Sending back [352714752]
  9597:20130207:114346.837 In update_cpustats()
  9597:20130207:114346.840 End of update_cpustats()
  9598:20130207:114347.176 Processing request.
  9598:20130207:114347.188 Requested [vfs.fs.size[/boot,used]]
  9598:20130207:114347.188 Sending back [73269248]
  9597:20130207:114347.840 In update_cpustats()
  9597:20130207:114347.843 End of update_cpustats()
  9597:20130207:114348.843 In update_cpustats()
  9597:20130207:114348.846 End of update_cpustats()
  9598:20130207:114349.181 Processing request.
  9598:20130207:114349.280 Requested [vfs.fs.size[/home,used]]
  9598:20130207:114349.280 Sending back [34963456]
  9597:20130207:114349.846 In update_cpustats()
  9597:20130207:114349.848 End of update_cpustats()
^C
svr3:/var/log/zabbix #

The following will make sure that Zabbix starts when the server is rebooted

svr1:~ # chkconfig --list | grep zabbix
svr3:/etc/zabbix/scripts # chkconfig --add zabbix-agent
insserv: Script jexec is broken: incomplete LSB comment.
insserv: missing `Required-Stop:'  entry: please add even if empty.
insserv: Script jexec is broken: incomplete LSB comment.
insserv: missing `Required-Stop:'  entry: please add even if empty.
insserv: Script jexec is broken: incomplete LSB comment.
insserv: missing `Required-Stop:'  entry: please add even if empty.
insserv: Script jexec is broken: incomplete LSB comment.
insserv: missing `Required-Stop:'  entry: please add even if empty.
insserv: Script jexec is broken: incomplete LSB comment.
insserv: missing `Required-Stop:'  entry: please add even if empty.
insserv: Script jexec is broken: incomplete LSB comment.
insserv: missing `Required-Stop:'  entry: please add even if empty.
insserv: Script jexec is broken: incomplete LSB comment.
insserv: missing `Required-Stop:'  entry: please add even if empty.
insserv: Script jexec is broken: incomplete LSB comment.
insserv: missing `Required-Stop:'  entry: please add even if empty.
insserv: Script jexec is broken: incomplete LSB comment.
insserv: missing `Required-Stop:'  entry: please add even if empty.
insserv: warning: script 'init.ohasd' missing LSB tags and overrides
insserv: Default-Start undefined, assuming default start runlevel(s) for script `init.ohasd'
insserv: Script jexec is broken: incomplete LSB comment.
insserv: missing `Required-Stop:'  entry: please add even if empty.
zabbix-agent              0:off  1:off  2:off  3:on   4:off  5:on   6:off
svr3:/etc/zabbix/scripts # chkconfig --level 35 zabbix-agent on
svr3:/etc/zabbix/scripts # chkconfig --list | grep zabbix
zabbix-agent              0:off  1:off  2:off  3:on   4:off  5:on   6:off
svr3:/etc/zabbix/scripts #

To check the Zabbix agent version, status etc, make sure that the Zabbix agent is the latest, on v2.0.4 at 07/Feb/2013

svr3:/etc/zabbix/scripts # /usr/local/sbin/zabbix_agentd -V
Zabbix Agent (daemon) v2.0.4 (revision 31984) (08 December 2012)
Compilation time: Dec 10 2012 13:41:24
svr3:/etc/zabbix/scripts # /etc/init.d/zabbix-agent status
Zabbix agent running(PID): 9596
9597
9598
9599
9600
svr3:/etc/zabbix/scripts #

BIG data - Setup Hadoop, HDFS, HBASE, hive - Installing Java and Hadoop - Part2

My wife takes all my money, so if this helped you in any way and you have some spare BitCoins, you may donate them to me - 16tb2Rgn4uDptrEuR94BkhQAZNgfoMj3ug

Keep this in mind

Host 1 - SuperNinja1 - 172.28.200.161 SuperMicro (The Master and running the NameNode. It also has H-Base and Hive installed)
Host 2 - SuperNinja2 - 172.28.200.163 SuperMicro (DataNode and a NodeManager)
Host 3 - SuperNinja3 - 172.28.200.165 SuperMicro (DataNode and a NodeManager, This node runs the Postgres instance for Hive)
Host 4 - SuperNinja4 - 172.28.200.150 HP Desktop (DataNode and a NodeManager)
Host 5 - SuperNinja5 - 172.28.200.153 HP Desktop (DataNode and a NodeManger)

For Hadoop and all the other stuff to work, you need java, seeing that I'm building on SLES11 SP3, I downloaded the latest Java RPM and installed it with rpm -ivh

SuperNinja5:/opt/temp # rpm -ivh jdk-2000\:1.7.0-fcs.x86_64.rpm
Preparing...                ########################################### [100%]
   1:jdk                    ########################################### [100%]
Unpacking JAR files...
    rt.jar...
    jsse.jar...
    charsets.jar...
    tools.jar...
    localedata.jar...
SuperNinja1:/opt/temp #
SuperNinja2:/opt/temp # which java
/usr/bin/java
SuperNinja2:/opt/temp # ls -ltr /usr/bin/java
lrwxrwxrwx 1 root root 26 May 28 10:07 /usr/bin/java -> /usr/java/default/bin/java
SuperNinja5:/opt/temp # ls -ltr /usr/java/latest
lrwxrwxrwx 1 root root 18 May 28 10:07 /usr/java/latest -> /usr/java/jdk1.7.0
SuperNinja2:/opt/temp #

I created 2 groups and 2 users, one for Hadoop and one for Hbase. The Hadoop user is called hduser and it belongs to the group hadoop, the other being hbuser, which is the H-Base user and belongs to the hbase group.

Let's start with Hadoop

Download the latest Hadoop from Apache's website
http://hadoop.apache.org/#Download+Hadoop
Place the downloaded file in /opt/temp

Make a directory for /opt/app, this is where we will place the Hadoop binaries. gunzip and untar the file, note the -C /opt/app, this means the tarred file contents will be placed in /opt/app

SuperNinja1:/opt/temp # mkdir -p /opt/app
SuperNinja1:/opt/temp # ls -ltr
total 135832
-rw-r--r-- 1 root root       194 May 14 15:29 ETH_MAC_ADDRESSES
-rw-r--r-- 1 root root 138943699 May 15 11:25 hadoop-2.4.0.tar.gz
SuperNinja1:/opt/temp # gunzip hadoop-2.4.0.tar.gz 
SuperNinja1:/opt/temp # tar -xvf hadoop-2.4.0.tar -C /opt/app
hadoop-2.4.0/
hadoop-2.4.0/bin/
hadoop-2.4.0/bin/mapred
hadoop-2.4.0/bin/hadoop
hadoop-2.4.0/bin/mapred.cmd
hadoop-2.4.0/bin/rcc
hadoop-2.4.0/bin/container-executor
hadoop-2.4.0/bin/hdfs
hadoop-2.4.0/bin/test-container-executor
Snip....Snip
hadoop-2.4.0/share/doc/hadoop/hadoop-streaming/images/icon_error_sml.gif
hadoop-2.4.0/share/doc/hadoop/hadoop-streaming/images/banner.jpg
hadoop-2.4.0/share/doc/hadoop/hadoop-streaming/images/bg.jpg
hadoop-2.4.0/share/doc/hadoop/hadoop-streaming/images/icon_info_sml.gif
hadoop-2.4.0/share/doc/hadoop/hadoop-streaming/images/expanded.gif
hadoop-2.4.0/share/doc/hadoop/hadoop-streaming/images/newwindow.png
hadoop-2.4.0/share/doc/hadoop/hadoop-streaming/images/maven-logo-2.gif
hadoop-2.4.0/share/doc/hadoop/hadoop-streaming/images/h3.jpg
hadoop-2.4.0/share/doc/hadoop/hadoop-streaming/images/breadcrumbs.jpg
hadoop-2.4.0/share/doc/hadoop/hadoop-streaming/images/h5.jpg
hadoop-2.4.0/share/doc/hadoop/hadoop-streaming/images/external.png
hadoop-2.4.0/share/doc/hadoop/hadoop-streaming/images/logos/
hadoop-2.4.0/share/doc/hadoop/hadoop-streaming/images/logos/build-by-maven-white.png
hadoop-2.4.0/share/doc/hadoop/hadoop-streaming/images/logos/build-by-maven-black.png
hadoop-2.4.0/share/doc/hadoop/hadoop-streaming/images/logos/maven-feather.png
hadoop-2.4.0/share/doc/hadoop/hadoop-streaming/images/icon_warning_sml.gif
hadoop-2.4.0/share/doc/hadoop/hadoop-streaming/images/collapsed.gif
hadoop-2.4.0/share/doc/hadoop/hadoop-streaming/images/logo_maven.jpg
hadoop-2.4.0/share/doc/hadoop/hadoop-streaming/images/logo_apache.jpg
hadoop-2.4.0/share/doc/hadoop/hadoop-streaming/images/icon_success_sml.gif
hadoop-2.4.0/share/doc/hadoop/hadoop-streaming/images/apache-maven-project-2.png
SuperNinja1:/opt/temp #

Lets see what happened, change directory to /opt/app

SuperNinja1:/opt/temp # cd /opt/app
SuperNinja1:/opt/app # ls
hadoop-2.4.0
SuperNinja1:/opt/app #

To make it more friendly, I renamed the hadoop-2.4.0 to hadoop

SuperNinja5:/opt/app # mv hadoop-2.4.0 hadoop
SuperNinja1:/opt/app # ls -ltr
total 8
drwxr-xr-x 9 67974 users 4096 Mar 31 11:15 hadoop
SuperNinja1:/opt/app #

Next we need a user for hadoop, I created a user called hduser with group hadoop, also create the user's home directory and set the permissions

SuperNinja1:/opt/app # groupadd hadoop
SuperNinja1:/opt/app # useradd -g hadoop hduser
SuperNinja1:/opt/app # mkdir -p /home/hduser
SuperNinja1:/opt/app # chown -R hduser:hadoop /home/hduser

We then login using the newly created user and generate the user's ssh keys, with this user you must be able to log into ALL the servers without any password

SuperNinja1:/opt/app # su - hduser
hduser@SuperNinja1:~> ssh-keygen -t rsa -P ""
Generating public/private rsa key pair.
Enter file in which to save the key (/home/hduser/.ssh/id_rsa): 
Created directory '/home/hduser/.ssh'.
Your identification has been saved in /home/hduser/.ssh/id_rsa.
Your public key has been saved in /home/hduser/.ssh/id_rsa.pub.
The key fingerprint is:
7e:ce:17:29:64:21:54:73:2c:fd:5c:64:96:b1:91:fc [MD5] hduser@SuperNinja1
The key's randomart image is:
+--[ RSA 2048]----+
|       ...oo. .+B|
|        . ooo  *+|
|         . o o o.|
|          o   o E|
|        So   .   |
|       .  . o    |
|        . .. .   |
|         +  .    |
|          o.     |
+--[MD5]----------+
hduser@SuperNinja1:~> ls -la .ssh
total 16
drwx------ 2 hduser hadoop 4096 May 15 11:29 .
drwxr-xr-x 3 hduser hadoop 4096 May 15 11:29 ..
-rw------- 1 hduser hadoop 1679 May 15 11:29 id_rsa
-rw-r--r-- 1 hduser hadoop  400 May 15 11:29 id_rsa.pub
hduser@SuperNinja1:~> echo $HOME
/home/hduser
hduser@SuperNinja1:~> cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys
hduser@SuperNinja1:~> cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys
ls -la .ssh
total 20
drwx------ 2 hduser hadoop 4096 May 15 11:30 .
drwxr-xr-x 3 hduser hadoop 4096 May 15 11:29 ..
-rw-r--r-- 1 hduser hadoop  400 May 15 11:30 authorized_keys
-rw------- 1 hduser hadoop 1679 May 15 11:29 id_rsa
-rw-r--r-- 1 hduser hadoop  400 May 15 11:29 id_rsa.pub
hduser@SuperNinja1:~> ssh localhost
The authenticity of host 'localhost (::1)' can't be established.
ECDSA key fingerprint is 06:a7:bc:61:a0:de:14:04:23:d9:2a:84:75:37:23:f4 [MD5].
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.

hduser@SuperNinja1:~> exit
logout
Connection to localhost closed.

hduser@SuperNinja1:~> exit
logout

Create the hduser on all the servers, using the procedure above. Then make a text file with all the servers authorized_keys in it and place this file containing all the servers authorized_keys on all the servers in /home/hduser/.ssh/authorized_keys. This ensure that the hduser can log into ALL servers with no password. Below is an example of what it looks like, yes I did change my keys for this printout below, so don't even try it...

SuperNinja1:~ # cd /home/hduser/.ssh/
SuperNinja1:/home/hduser/.ssh # cat authorized_keys
ssh-rsa jCfon0dWBqIffU9G3q+HVzYRs6FDNrov hduser@SuperNinja1
ssh-rsa n0fwO3pBo8bQc2bA9lvKEIHbTwmUWDcu hduser@SuperNinja2
ssh-rsa dwS0ltr6/H1VPaU1X/OS3/Jq83yxjAYT hduser@SuperNinja3
ssh-rsa u1HzxsOH8Leu07JQA3piUaB56B7eJNFz hduser@SuperNinja4
ssh-rsa pnbYOuKz093zZzSMt80AmijczuPctnaf hduser@SuperNinja5
SuperNinja1:/home/hduser/.ssh #

Next step is to login as hduser and set some variables in the .bashrc file on all the servers. Set the following in the .bashrc file in the hduser's home directory - See below

SuperNinja1:/home/hduser/.ssh # cd /
SuperNinja1:/ # su - hduser
hduser@SuperNinja1:~> pwd
/home/hduser
hduser@SuperNinja1:~> cat .bashrc
#Set Hadoop-related environment variables
export HADOOP_HOME=/opt/app/hadoop
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export HADOOP_YARN_HOME=$HADOOP_HOME
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export YARN_CONF_DIR=$HADOOP_HOME/etc/hadoop
export HIVE_HOME=/opt/app/hive
export PATH=$HADOOP_HOME/bin:$HIVE_HOME/bin:$PATH

# Set JAVA_HOME (we will also configure JAVA_HOME directly for Hadoop later on)
export JAVA_HOME=/usr/java/latest

# Some convenient aliases and functions for running Hadoop-related commands
unalias fs &> /dev/null
alias fs="hadoop fs"
unalias hls &> /dev/null
alias hls="fs -ls"

# If you have LZO compression enabled in your Hadoop cluster and
# compress job outputs with LZOP (not covered in this tutorial):
# Conveniently inspect an LZOP compressed file from the command
# line; run via:
#
# $ lzohead /hdfs/path/to/lzop/compressed/file.lzo
#
# Requires installed 'lzop' command.
#
lzohead () {
    hadoop fs -cat $1 | lzop -dc | head -1000 | less
}

# Add Hadoop bin/ directory to PATH
export PATH=$PATH:$HADOOP_HOME/bin
# For jps
export PATH=$PATH:$JAVA_HOME/bin
hduser@SuperNinja1:~>

Logout and log in again with the hduser and see if the .bashrc file is loaded

hduser@SuperNinja1:~> exit
logout
SuperNinja1:/ # su - hduser
hduser@SuperNinja1:~> echo $HADOOP_HOME
/opt/app/hadoop
hduser@SuperNinja1:~> echo $HIVE_HOME
/opt/app/hive
hduser@SuperNinja1:~>

Yea! we can start configuring HADOOP, all changes must be made as the hduser
All the files needed for Hadoop is in /opt/app/hadoop/etc/hadoop

SuperNinja1:/ # su - hduser
hduser@SuperNinja1:~> cd /opt/app/hadoop/etc/hadoop/
hduser@SuperNinja1:/opt/app/hadoop/etc/hadoop> ls -ltr
total 132
-rw-r--r-- 1 hduser hadoop  2268 Mar 31 10:49 ssl-server.xml.example
-rw-r--r-- 1 hduser hadoop  2316 Mar 31 10:49 ssl-client.xml.example
-rw-r--r-- 1 hduser hadoop 11169 Mar 31 10:49 log4j.properties
-rw-r--r-- 1 hduser hadoop  9257 Mar 31 10:49 hadoop-policy.xml
-rw-r--r-- 1 hduser hadoop  2490 Mar 31 10:49 hadoop-metrics.properties
-rw-r--r-- 1 hduser hadoop  3589 Mar 31 10:49 hadoop-env.cmd
-rw-r--r-- 1 hduser hadoop  2178 Mar 31 10:49 yarn-env.cmd
-rw-r--r-- 1 hduser hadoop  4113 Mar 31 10:49 mapred-queues.xml.template
-rw-r--r-- 1 hduser hadoop  1383 Mar 31 10:49 mapred-env.sh
-rw-r--r-- 1 hduser hadoop   918 Mar 31 10:49 mapred-env.cmd
-rw-r--r-- 1 hduser hadoop   620 Mar 31 10:49 httpfs-site.xml
-rw-r--r-- 1 hduser hadoop    21 Mar 31 10:49 httpfs-signature.secret
-rw-r--r-- 1 hduser hadoop  1657 Mar 31 10:49 httpfs-log4j.properties
-rw-r--r-- 1 hduser hadoop  1449 Mar 31 10:49 httpfs-env.sh
-rw-r--r-- 1 hduser hadoop  1774 Mar 31 10:49 hadoop-metrics2.properties
-rw-r--r-- 1 hduser hadoop   318 Mar 31 10:49 container-executor.cfg
-rw-r--r-- 1 hduser hadoop  1335 Mar 31 10:49 configuration.xsl
-rw-r--r-- 1 hduser hadoop  3589 Mar 31 10:49 capacity-scheduler.xml
-rw-r--r-- 1 hduser hadoop   206 May 15 12:28 mapred-site.xml
-rw-r--r-- 1 hduser hadoop  3512 May 15 12:54 hadoop-env.sh
-rw-r--r-- 1 hduser hadoop  4878 May 16 11:06 yarn-env.sh
-rw-r--r-- 1 hduser hadoop   679 May 16 11:27 yarn-site.xml
-rw-r--r-- 1 hduser hadoop   655 May 22 14:40 derby.log
drwxr-xr-x 5 hduser hadoop  4096 May 22 14:40 metastore_db
-rw-r--r-- 1 hduser hadoop   334 May 26 07:42 core-site.xml
-rw-r--r-- 1 hduser hadoop    60 May 28 11:58 slaves
-rw-r--r-- 1 hduser hadoop   510 May 29 11:14 hdfs-site.xml
hduser@SuperNinja1:/opt/app/hadoop/etc/hadoop>

Determine which process is using swap

Reason 2 why you should give me some BitCoins... I'm poor?.... If this helped you in any way and you have some spare BitCoins, you may donate them to me - 16tb2Rgn4uDptrEuR94BkhQAZNgfoMj3ug

My server recently ran out of swap space, causing slow calculations, slow response times etc

cd /opt/temp

vi a script called swap.sh, paste the following and save with ESC:wq!

#!/bin/bash
# Get current swap usage for all running processes
# This script will tell you the PID and the command line
# that stared the PID with how much of swap space is being used
# King Rat 08/02/2014
SUM=0
OVERALL=0
for DIR in `find /proc/ -maxdepth 1 -type d -regex "^/proc/[0-9]+"`
do
PID=`echo $DIR | cut -d / -f 3`
PROGNAME=`ps -p $PID -o comm --no-headers`
for SWAP in `grep Swap $DIR/smaps 2>/dev/null | awk '{ print $2 }'`
do
let SUM=$SUM+$SWAP
done
if (( $SUM > 0 )); then
echo "****************************"
echo "PID=$PID swapped $SUM in KB ($PROGNAME)"
echo "cmd line that activate the PID is below "
xargs -0 echo < /proc/$PID/cmdline
echo "*****************************************"
fi
let OVERALL=$OVERALL+$SUM
SUM=0
done
echo " "
echo "Overall swap used: $OVERALL in KB"
let OVERALL=$OVERALL/1024
echo "Overall swap used: $OVERALL in MB"
echo " "
echo "******** swapon -s *********"
swapon -s
echo " "
echo "********  free -m  *********" 
free -m

Make the script executable with chmod +x swap.sh Run the script with ./swap.sh, see output below

someserver:/opt/temp # ./swap.sh
****************************
PID=1 swapped 104 in KB (init)
cmd line that activate the PID is below
init [3] 
*****************************************
****************************
PID=708 swapped 16 in KB (oracle)
cmd line that activate the PID is below
ora_w006_DTE
*****************************************
****************************
PID=1504 swapped 28 in KB (cron)
cmd line that activate the PID is below
/usr/sbin/cron
*****************************************

Bla, Bla, Bla

****************************
PID=63545 swapped 12 in KB (oracle)
cmd line that activate the PID is below
oracle (LOCAL=NO)
*****************************************
****************************
PID=63568 swapped 12 in KB (oracle)
cmd line that activate the PID is below
ora_w007
*****************************************
****************************
PID=64038 swapped 16 in KB (oracle)
cmd line that activate the PID is below
ora_w00d
*****************************************
  
Overall swap used: 310456 in KB
Overall swap used: 303 in MB
  
******** swapon -s *********
Filename                Type        Size    Used    Priority
/dev/mapper/system-swap                 partition    26214392    4371104    -1
  
********  free -m  *********
             total       used       free     shared    buffers     cached
Mem:         96714      95321       1393          0         50      71053
-/+ buffers/cache:      24217      72497
Swap:        25599       4268      21331
someserver:/opt/temp #

Problem with ETH hardware not seen

Reason 1 why you should give me some BitCoins... uuhhh, mmmh, uuuhh, mmmh...... If this helped you in any way and you have some spare BitCoins, you may donate them to me - 16tb2Rgn4uDptrEuR94BkhQAZNgfoMj3ug

ETH0 to ETH3 not recognized

NIC board was replaced on a HP DL380 server

Situation arise were the NIC board was replace and after the board was replaced, ETH0, ETH1, ETH2 and ETH3 were activated as ETH4, ETH5, ETH6 and ETH7. The problem is in /etc/udev/rules.d/70-persistent-net.rules where the new board was identified, but because the old board was still in the configuration, ETH0 to ETH3 were added as ETH4 to ETH7. To fix this, log into ILOM and note the MAC addresses of the NICs

Set the file the same, IE change the NAME="ethx" to the correct one and set the MAC addresses as per the ILOM printout. Comment the old board entries.

someserver:~ # cat /etc/udev/rules.d/70-persistent-net.rules
# This file was automatically generated by the /lib/udev/write_net_rules
# program run by the persistent-net-generator.rules rules file.
#
# You can modify it, as long as you keep each rule on a single line.

# PCI device 0x14e4:0x1657 (tg3)
#SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="2c:76:8a:54:89:90", 
ATTR{type}=="1", KERNEL=="eth*", NAME="eth0" - Commented this

# PCI device 0x14e4:0x1657 (tg3)
#SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="2c:76:8a:54:89:92", 
ATTR{type}=="1", KERNEL=="eth*", NAME="eth2" - Commented this

# PCI device 0x14e4:0x1657 (tg3)
#SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="2c:76:8a:54:89:91", 
ATTR{type}=="1", KERNEL=="eth*", NAME="eth1" - Commented this

# PCI device 0x14e4:0x1657 (tg3)
#SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="2c:76:8a:54:89:93", 
ATTR{type}=="1", KERNEL=="eth*", NAME="eth3" - Commented this

# PCI device 0x14e4:0x1657 (tg3)
# Below is the new board settings, change the MAC address as per ILOM print, change 
the NAME="ethx" to correspond
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="ac:16:2d:70:11:4a", 
ATTR{type}=="1", KERNEL=="eth*", NAME="eth2"

# PCI device 0x14e4:0x1657 (tg3)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="ac:16:2d:70:11:4b", 
ATTR{type}=="1", KERNEL=="eth*", NAME="eth3"

# PCI device 0x14e4:0x1657 (tg3)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="ac:16:2d:70:11:49", 
ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

# PCI device 0x14e4:0x1657 (tg3)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="ac:16:2d:70:11:48", 
ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"
someserver:~ #

Reboot the server and make sure that the server is pingable afterwards

Arduino, Linux, Hadoop, Hive, H-Base, Postgres, Zabbix - Things that matter in life

Pages