Apr 282011
 

So, sometimes I write scripts to be a quick way to monitor or audit other systems.  The following script was written as a way to quickly audit a list of domain names, i.e. from a list of 1 domain per line.  Initially it was used to audit a list of subdomains from a nettica account to see if the name still resolved and if so, determine if the server was running ssh with a valid key.  I have realized the script has a lot of uses, as a way to validate hosts are up and also validate ssh is running.  You could also replace the “ls” command in order to monitor other services on a server or even top or similar.  So, this script could start as a foundation and easily expanded upon.

First, you need a text file containing 1 domain name per line, like this:

domain1.com
domain2.com
domain3.com
sub.domain4.com

Next, you need this script below….it is documented well enough to get you started.  Just be sure to set auditdom= to your file containing 1 domain per line.

#!/bin/bash
# By Ed Wiget for ........
# Audit a list of domains, validate the A record and then ping it to make sure it is up
# If the server is up, then do a ssh hostname "ls" and validate it is successful
# if ssh is also successful, then we can say server is a valid server we manage
# if it is not successful, the two reasons that may cause this are 1) not a valid server we manage or 2) ssh keys not installed
# If the server is not a server we manage, it should be removed from dns
# If the server is a valid server we manage but ssh does not work, keys should be installed and checked again
# 2011-04-28 - initial script - tested domains.com + subdomains completed - edw
# enable for debugging
#set -x
#####################################################################
############# USER  CONFIG VARIABLES BELOW LOGGING START ############
#####################################################################
#####################################################################
## set basedir and ctime because both required by logging
## set the current time and is used for consistency
#####################################################################
ctime=`date +%Y%m%d-%T`
basedir=/root
logdir=/root/installs
diags=/root/diags
mkdir -p $logdir
mkdir -p $diags
#####################################################################
## make sure we are running as root user
#####################################################################
if [ $(whoami) != "root" ]; then
echo "You need to run this script as root."
exit 1
fi
#####################################################################
## Set up logging the correct way
#####################################################################
BUILD_LOG=$logdir/`hostname | awk -F. '{print$1}'`-$ctime.auditdom.log
BUILD_PIPE=$logdir/build.pipe
if [ ! -e $BUILD_PIPE ]; then
mkfifo $BUILD_PIPE
fi
if [ -e $BUILD_LOG ]; then
rm $BUILD_LOG
fi
exec 3>&1 4>&2
tee $BUILD_LOG < $BUILD_PIPE >&3 &
tpid=$!
exec > $BUILD_PIPE 2>&1
#####################################################################
##
#####################################################################
# number of times to ping the domain
# recommend a value higher than 1, such as 2, but not any higher otherwise you spend time for ping command to complete
COUNT=2
# location of the file list of domains, 1 per line
# you can just copy and paste straight from company dns into a text file.
# Bulk remove the Edit Delete words, remove duplicates lines, make sure no line wrap
auditdom=`cat /home/user/subdomains.txt | awk -F" " '{print $1}'`
# email stuffs - can be enabled to email a report based on each failure
# email report when script completes
SUBJECT="Ping failed"
SUBJECT2="ssh failed"
EMAILID="[email protected]"
for i in `echo $auditdom`
do
# blank line for readability
echo ""
# here we echo the domain being checked
echo $i
# dig the A record of the domain and return short results
dig A $i +short
# here we set a count to see if the ping is successful or not, indicating host up or down
ping -c $COUNT $i | grep 'received' | awk -F',' '{ print $2 }' | awk '{ print $1 }' > /dev/null
# if count is 0, we echo host up
if [ $? -eq 0 ]; then
echo "Host : $i is up"
# next we want to validate it is in fact a server we manage, easy way...make sure ssh key works
ssh $i "ls" > /dev/null
if [ $? -eq 0 ]; then
echo "Host : $i ssh is successful"
else
echo "HOST : $i server needs validated as a server we manage"
# echo "Host : $i server needs validated as a server we manage on $(date)" | mail -s "$SUBJECT2" $EMAILID
fi
else
# 100% failed
echo "Host : $i is down (ping failed) at $(date)" # you could also pipe this to mail or similar
# echo "Host : $i is down (ping failed) at $(date)" | mail -s "$SUBJECT" $EMAILID
fi
# another blank line for readability
echo ""
done
###########################################################################
## LOGGING CLEANUP
###########################################################################
# 1>&3 and 2>&4 restore the original file descriptors for stdout and
# stderr from file descriptors 3 and 4.
# 3>&~ 4>&~: now that they are not needed, close file descriptors 3 and 4
exec 1>&3 3>&- 2>&4 4>&-
# waits on the tee pid to die before continuing
wait $tpid
# remove the named pipe which is no longer needed
sleep 5
rm $BUILD_PIPE
##############################################################################
## MAIL REPORT
##############################################################################
echo "using mutt to send log to $EMAILID"
# don't forget, gentoo, sabayon, and arch requires -- before email address to work correctly with mutt while redhat does not'
mutt -s "subdomain audit log for" -a $BUILD_LOG $EMAILID < /dev/null