Mar 062013
 

This was tested on centos 6.3.  It is running at approx 900 – 3,000+ log events per second from approx 30 hosts.

Current load is about 900 messages per second:  load average: 1.57, 1.35, 1.29 with 8GB memory.

With the above in mind, there was approx 165GB of log data after running for 4 days.

graylog.org web site

elasticsearch web site

mongo db

passenger phusion web site

logstash web site

I wrote a script in order to install a graylog2 central log server.  Its a one shot run and be done kinda thing…..

CHANGES20130309 – see notes in script

Continue reading »

Jul 122012
 

I had a few virtualboxes running centos 5.7 that I hadn’t updated in a while.  They were used for testing.  Today I went to update them and couldn’t because the 5.7 repo’s had been deprecated and were not longer available.  I found a quick way to do this, and this got my virtualboxes updated to 5.8……which is what I needed anyways.

Continue reading »

Aug 242011
 

Thought I would share this quick fix ref CVE-2011-3192 with the POC available here:
http://seclists.org/fulldisclosure/2011/Aug/175

First, if you are an apache admin, get this fixed ASAP. I was able to take down a small test server with 7 http get requests.

Login to the server and run this command to see if you have mod_headers installed:

locate mod_headers

If you see mod_headers.so in that list, you can continue to Configuring Apache.  Otherwise go to Compiling Mod_Headers

Continue reading »

Aug 112011
 

ï£ ¥ð ¢åñ’† rêåÐ ßå§h, gê† Ðå £µ¢k 𵆆å hêrê

#!/bin/bash
 
# By Ed Wiget
# This script automates downloading youtube video and converting to mp3 file
# I use it to grab new songs for my ipod that I am too lazy to insert store bought cd :)
 
# set this variable to the location of your scripts:
BASE_DIR=~/scripts
 
# this sets the location of the python youtube-dl script, relative to above
YOUTUBE_DL=${BASE_DIR}/youtube-dl/youtube-dl
 
# this sets the path of the downloaded files and temp directory
DL_FILES=~/Downloads
 
# this sets the path to were the final mp3 is stored
MP3_LOCATION=~/Music
 
# this checks for the youtube-dl script and if it doesn't exist, it gets it
# if it does exist, it checks to make sure its the latest version
if [ ! -d ${BASE_DIR}/youtube-dl ];
then
echo "grabbing the youtube-dl script"
cd ${BASE_DIR}
git clone git://github.com/rg3/youtube-dl.git youtube-dl
else
echo -e "youtube-dl already exists\n\nMaking sure we have the latest version"
${YOUTUBE_DL} -U
fi
 
# this checks to make sure we have ffmpeg and lame installed, and if not, grabs them
FFMPEG1=`which ffmpeg | wc -l`
LAME1=`which lame | wc -l`
 
if [ ${FFMPEG1} = 1 ];
then
echo "ffmpeg already exists"
else
echo "grabbing ffmpeg"
# for ubuntu based distros, use this line
sudo apt-get install ffmpeg
# for redhat based distros, use this line
#sudo yum install ffmpeg
# for arch
#pacman -S ffmpeg
fi
 
if [ ${LAME1} = 1 ];
then
echo "lame already exists"
else
echo "grabbing lame"
# for ubuntu use this line
sudo apt-get install lame
# for redhat use this line
# sudo yum install lame
# for arch linux
#pacman -S lame
fi
 
# next we ask the user for the video file, it should be in format like:
# http://www.youtube.com/watch?v=6E2hYDIFDIU
echo -e "What is the video to download, ie. http://www.youtube.com/watch?v=6E2hYDIFDIU"
read VIDEO_URL
 
echo "You entered ${VIDEO_URL} is this correct? ( y / n )"
read ANS
        if [ ${ANS} = "y" ];
                then
                        cd ${DL_FILES}
                        # grab the song title
                        SONG_TITLE=`${YOUTUBE_DL} --get-title ${VIDEO_URL}`
                        echo -e "the song title is ${SONG_TITLE}"
                        # downloading video
                        echo "downloading video....please wait"
                        ${YOUTUBE_DL} ${VIDEO_URL}
                        # we need to convert the dl url to a filename for later processing
                        # the url is like:  http://www.youtube.com/watch?v=6E2hYDIFDIU 
                        # the downloaded file will be 6E2hYDIFDIU.mp4
                        MP4_FILE=`echo ${VIDEO_URL} | awk -F/ '{print$4}' | awk -F= '{print$2}'`
                        # get the downloaded file extension
                        FILE_EXT=`ls ${DL_FILES}/${MP4_FILE}* | awk -F. '{print$2}'`
                        echo "your video is located in ${DL_FILES}/${MP4_FILE}.${FILE_EXT}"
                        echo ""
                        echo "converting ${DL_FILES}/${MP4_FILE}.${FILE_EXT} to wav.....please wait"
                        # ffmpeg -i 6E2hYDIFDIU.flv 6E2hYDIFDIU.wav
                        ffmpeg -i ${DL_FILES}/${MP4_FILE}.${FILE_EXT} ${DL_FILES}/"${SONG_TITLE}".wav
                        echo "video converted to wav file....converting wav to mp3"
                        lame -b 128 ${DL_FILES}/"${SONG_TITLE}".wav ${MP3_LOCATION}/"${SONG_TITLE}".mp3
                        echo "${SONG_TITLE} is now available at ${MP3_LOCATION}/${SONG_TITLE}.mp3"
                        echo -e "\n\nmoving video download file ${DL_FILES}/${MP4_FILE}.${FILE_EXT} to ${MP3_LOCATION}/${SONG_TITLE}.${FILE_EXT}"
                        mv ${DL_FILES}/${MP4_FILE}.${FILE_EXT} ${MP3_LOCATION}/"${SONG_TITLE}".${FILE_EXT}
                        echo "video file is now at ${MP3_LOCATION}/"${SONG_TITLE}".${FILE_EXT}"
                        echo -e "\n\ndone .... and enjoy"
        else
                echo "there was an error...."
                exit
fi

Apr 282011
 

So, sometimes I write scripts to be a quick way to monitor or audit other systems.  The following script was written as a way to quickly audit a list of domain names, i.e. from a list of 1 domain per line.  Initially it was used to audit a list of subdomains from a nettica account to see if the name still resolved and if so, determine if the server was running ssh with a valid key.  I have realized the script has a lot of uses, as a way to validate hosts are up and also validate ssh is running.  You could also replace the “ls” command in order to monitor other services on a server or even top or similar.  So, this script could start as a foundation and easily expanded upon.

First, you need a text file containing 1 domain name per line, like this:

domain1.com
domain2.com
domain3.com
sub.domain4.com

Continue reading »