Snippets of code

Ever wanted to get a line by specifying the line number?

This code does just that.

# awk 'NR==$line{print;exit}' $filename

Linux/UNIX: Find Out If File Exists With Conditional Expressions

This snippet shows you the script used to find if a file exists.

The script works as follows:

First it we declare the Variable FILE to equal what ever you specify on the command line when you execute the script.
Next step is the test function, we say if (-f) the $FILE (you entered) exists then,
print on the screen file exists else print it does not.

#!/bin/bash
FILE=$1  
 if [ -f $FILE ]; then
   echo \"File $FILE exists\"
    else
   echo \"File $FILE does not exists\"
 fi

Linux Script Test Conditions

Linux Script Test Conditions

There is a function provided by bash called test which returns a true or false value depending on the result of the tested expression. Its syntax is:

test expression

It can also be implied as follows:

[ expression ]

The tests below are test conditions provided by the shell:

* -b file = True if the file exists and is block special file.
* -c file = True if the file exists and is character special file.
* -d file = True if the file exists and is a directory.
* -e file = True if the file exists.
* -f file = True if the file exists and is a regular file
* -g file = True if the file exists and the set-group-id bit is set.
* -k file = True if the files\' \"sticky\" bit is set.
* -L file = True if the file exists and is a symbolic link.
* -p file = True if the file exists and is a named pipe.
* -r file = True if the file exists and is readable.
* -s file = True if the file exists and its size is greater than zero.
* -s file = True if the file exists and is a socket.
* -t fd = True if the file descriptor is opened on a terminal.
* -u file = True if the file exists and its set-user-id bit is set.
* -w file = True if the file exists and is writable.
* -x file = True if the file exists and is executable.
* -O file = True if the file exists and is owned by the effective user id.
* -G file = True if the file exists and is owned by the effective group id.
* file1 –nt file2 = True if file1 is newer, by modification date, than file2.
* file1 ot file2 = True if file1 is older than file2.
* file1 ef file2 = True if file1 and file2 have the same device and inode numbers.
* -z string = True if the length of the string is 0.
* -n string = True if the length of the string is non-zero.
* string1 = string2 = True if the strings are equal.
* string1 != string2 = True if the strings are not equal.
* !expr = True if the expr evaluates to false.
* expr1 –a expr2 = True if both expr1 and expr2 are true.
* expr1 –o expr2 = True is either expr1 or expr2 is true.

Custom bashrc and profile with colour PS1

Here is my custom login / profile script
It takes into account the os you are using before applying.

It is a work in progress let me know if there is anything I am missing
and I will get it added.

Copy this into the end of your ~/.bashrc or ~/.profile
If you want this script to be available for all users copy it to /etc/.profile

OS_NAME=`uname`
# Set  colors :
red='\e[0;31m'
RED='\e[1;31m'
green='\e[0;32m'
GREEN='\e[1;32m'
yellow='\e[0;33m'
YELLOW='\e[1;33m'
blue='\e[0;34m'
BLUE='\e[1;34m'
magenta='\e[0;35m'
MAGENTA='\e[1;35m'
cyan='\e[0;36m'
CYAN='\e[1;36m'
white='\e[0;37m'
WHITE='\e[1;37m'
NC='\e[0m'              # No Color
# Set up custom PATH
PATH=$PATH:$HOME
# functions
logintxt()
{
   echo "OS: "`uname -s`
   echo "VER: "`uname -r`
   echo "HOST: "`uname -n`
}
loginprompt()
{
if [ "$TERM" != 'dumb' ] && [ -n "$BASH" ] && [ -n "$PS1" ]
then
       if [ `id | awk -F\( '{print $2}'|awk -F\) '{print $1}'` = "root" ]
       then
               export PS1='\[\033[01;31m\]\h \[\033[01;34m\]\W \$ \[\033[00m\]'
       else
    export PS1="\[\033[0;34m\][\$(date +%H:%M)]\033[0;32m\]\u @
\033[0;33m\]\h:\033[0;37m\]\W$ \[\033[37m\]"
       fi
fi
  }
 
loginalias()
{
#My Aliases
alias mkdir='mkdir -p'
alias which='type -all'
alias ..='cd ..'
alias la='ls -Al'               # show hidden files
alias lc='ls -lcr'              # sort by change time
alias lu='ls -lur'              # sort by access time
alias lr='ls -lR'               # recursive ls
alias lt='ls -ltr'              # sort by date
alias lm='ls -al |more'         # pipe through 'more'
alias ld='ls -lp|grep ^d'       # List Directories Only
alias lf='ls -lp|grep -v ^d'    # List Files Only
alias lp='ls -lp'    # List with file type appended
alias rm='rm -i'    # Remove Interactive mode
alias hist='history | grep $1'  #Requires one input
alias ssh='ssh -X'    #SSH with X forwding enabled
alias hist='history | grep $1'  #Requires one input
set -o noclobber    # Do not over write files
}
# Begin case; Find OS type and apply settings
  case ${OS_NAME} in
    Linux) # linux - redhat
 
 # CUSTOM SETTINGS
   #  Linux only aliases
  alias ls='ls -hF --color'       # add colors for filetype recognition 
  alias lx='ls -lXB'              # sort by extension 
  alias lk='ls -lSr'              # sort by size
  alias iptables='sudo /sbin/iptables -L'    # Show IPTABLES rules
  alias vi='vim'
  ;;
    SunOS) # SunOS
 
 # CUSTOM SETTINGS   
  
 ;;
 
 FreeBSD) # FreeBSD
 
 # CUSTOM SETTINGS 
   alias vi='vim'
 ;;
    *)
     echo "$0 : unknown OS"
 
   ;;
esac
# Set up bash only settings
  if [ `echo $SHELL` == "/usr/local/bin/bash" -o "/bin/bash" ]; then
 shopt -s cdspell
 shopt -s extglob
 shopt -s dotglob
 shopt -s cmdhist
 shopt -s lithist
 shopt -s progcomp
 shopt -s checkhash
 shopt -s histreedit
 shopt -s promptvars
 shopt -s cdable_vars
 shopt -s checkwinsize
 shopt -s hostcomplete
 shopt -s expand_aliases
 shopt -s interactive_comments
 
 bind 'set show-all-if-ambiguous on'             # Tab once for complete
 bind 'set visible-stats on'                     # Show file info in complete
 
#Larger Bash History
HISTSIZE=10000
HISTFILESIZE=10000
     else
       echo "BASH not installed"
 fi
# enable programmable completion features (you don't need to enable
# this, if it's already enabled in /etc/bash.bashrc and /etc/profile
# sources /etc/bash.bashrc).
if [ -f /etc/bash_completion ]; then
   . /etc/bash_completion
fi
# Set VI as default editor
export EDITOR=vi
# Print OS details to tty
 logintxt
# set a fancy prompt
 loginprompt
# System Aliases
 loginalias
 
# END of Script

Mksysb backup shell script for Unix

Here is a shell script I wrote to perform a backup of an AIX system. It has been used but not fully tested and as such I would test running a backup and restore before using on a live server.

#!/bin/ksh

#AIX Backup Script
#Written on 27-11-08 

# Using mksysb and savevg for easier system rebuilds.
# Please see notes at the bottom for RESTORE HELP..
# Scheduled for every Friday. cpio used Monday - Thursday
#
# Please add volume groups to backup to the MY_VG below
# This script will ensure all non root groups are backed up

#Setup variables for script

TODAY=`date +%A`
WORKDIR="/home/scripts" # Scipt home directory
BCKLOG="backup.log"
LOG=${WORKDIR}/${BCKLOG} # File to log backup progress to
ERRORLOG=${WORKDIR}/${BCKLOG}.error
MY_VG="datavg " # Separate VG names with a space if more than one ot back up ie " datavg homevg logvg"
FSF_COUNT=0 # Set the FSF count during non root vg backups
MAIL_GROUP="user1@company.com;group@company.com" # Setup addresses for mail

# Setup The Functions >>>

date_func() {
# Find yesterdays date
DOW=`date +%u`
 case $DOW in

  1)
        YESTERDAY="Sunday"
        ;;
  2)
        YESTERDAY="Monday"
        ;;
  3)
        YESTERDAY="Tuesday"
        ;;
  4)
        YESTERDAY="Wednesday"
        ;;
  5)
        YESTERDAY="Thursday"
        ;;
  6)
        YESTERDAY="Friday"
        ;;
  7)
        YESTERDAY="Saturday"
        ;;
 esac
 }

# Function - Rotate backup logs
log_rotate()
{
   if [ -s ${LOG} ]
    then
     mv -f ${LOG} ${WORKDIR}/${YESTERDAY}.${BCKLOG}
         echo "Backup log rotated successfully" >> ${LOG}
    else
   echo "Error no Backup log found - Please investigate" >> ${LOG}
   fi
 }
# Function - Rotate Error logs
 errorlog_rotate()
{
   if [ -s ${ERRORLOG} ]
    then
     mv -f ${ERRORLOG} ${WORKDIR}/${YESTERDAY}.${BCKLOG}.error
         echo "ERROR log rotated successfully for ${YESTERDAY}" >> ${LOG}
    else
   echo "" >> /dev/null
   fi
 }

# Rewind function.
rmt_rewind()
{
# Ensure tape has rewound successfully
echo "Rewinding Tape... at "`date` >> ${LOG}
 mt -f /dev/rmt0 rewind 2>${ERRORLOG}
 if [ $? -ne 0 ]
  then
   echo "Error Tape Rewind Failed at "`date` >> ${LOG}
  else
   echo "Success Tape has Rewound at "`date` >> ${LOG}
 fi
 }

bck_mksysb() {
# The backup does not use the -i or -m flag as to not overwrite the custom files below
# /image.data and /bosinst.data files
# The mksysb uses the verify option -V
 echo "Starting Backup of rootvg at" `date` > ${LOG}
 echo "Running mksysb /dev/rmt0 -V " >> ${LOG}
/usr/bin/mksysb /dev/rmt0.1 -V >> ${LOG} 2>${ERRORLOG}
 if [ $? -ne 0 ]
  then
   echo "!!! Error mksysb Failed - Please investigate" >> ${LOG}
  else
   echo "* Finished Backup of rootvg at" `date` >> ${LOG}
 fi
 rmt_rewind
 }

bck_non_rootvgs() {
# Begin Non root Backup function
# Finds number of volume groups to backup
  NUM_VG=`print ${MY_VG}| wc -w`
  FSF_TOTAL=`expr ${NUM_VG} + 3`
# Begin volume backup loop
  for NRVG in ${MY_VG}
   do
    while [ $FSF_COUNT -lt $NUM_VG ]
do
     FSF_COUNT=`expr 1 + $FSF_COUNT`
     FSF=`expr $FSF_COUNT + 3`
# Skip tape forward to end of mksysb ready for next VG
      echo " Preparing to backup ${NRVG}" >> ${LOG}
      echo "Skipping forward ${FSF} blocks at "`date` >> ${LOG}
# Fast Forward tape position for non root vg
   /usr/bin/tctl -f /dev/rmt0.1 fsf $FSF 2>${ERRORLOG}
    if [ $? -ne 0 ]
  then
  echo "!!! Failed to skip to end of backup - Please investigate" `date` >> ${LOG}
  else
   echo "* Skipped forward ${FSF} Blocks" `date` >> ${LOG}
        fi
#  Beginning backup of vg's.
#  The flag -i Creates the data file by calling the mkvgdata command; -f specifies the device.
      echo "* Starting Backup of ${NRVG} at" `date` >> ${LOG}
       /usr/bin/savevg -i -f /dev/rmt0 ${NRVG} >> ${LOG} 2>${ERRORLOG}
            if [ $? -ne 0 ]
  then
  echo "!!! Failed to savevg for ${NRVG} - Please investigate" >> ${LOG}
  else
   echo "* Finished Backup of ${NRVG} at" `date` >> ${LOG}
        fi
rmt_rewind
     done
   echo "* Backup completed at" `date` >> ${LOG}
  done
}

# Backup test script
#test_backup() {
# Verify VG headers
# check
# Uncomment to enable backup verification
# This takes a little while to complete as it has to perform several checks to verify integrity.
# You should not rely on this test and should perform full test restores to ensure they work.
# lsmksysb -l -f /dev/rmt0.1
#rmt_rewind
#}

# Gather system information
vg_info()
{
 for VG_LIST in `/usr/sbin/lsvg`
  do
   lsvg -l ${VG_LIST} >> ${LOG} 2>${ERRORLOG}
   echo "\n\n" >> ${LOG}
  done
 for PV_LIST in `lspv | awk ' { print $1 } '`
  do
   lspv -l ${PV_LIST} >> ${LOG} 2>${ERRORLOG}
   echo "\n\n" >> ${LOG}
  done
 /usr/bin/df -m | tr -s " " | sed 's/ /, /g' | sed '1 s/, / /g' >> ${LOG} 2>${ERRORLOG}
  echo "\n\n" >> ${LOG}
}

bck_email()
#Mail Function configure names in MAIL_GROUP
{
# Mail LOG files if they exist
if [ - s ${LOG} ]
  then
   cat ${LOG} | mail -s "Backup of ${BCK_HOST} at `date +%A`" $MAIL_GROUP 2>${ERRORLOG}
  else
  mail -s "!Problem - ${BCK_HOST} backup cannot mail logs at `date +%A`" $MAIL_GROUP 2>${ERRORLOG}
        fi
# Mail Error LOG files if they exist
if [ - s ${ERRORLOG} ]
  then
   cat ${ERRORLOG} | mail -s "!Backup of ${BCK_HOST} at `date +%A` has an error" $MAIL_GROUP 2>${ERRORLOG}
  else
  mail -s "!Problem - ${BCK_HOST} backup cannot mail Error logs at `date +%A`" $MAIL_GROUP 2>${ERRORLOG}
        fi
}

# Functions Finish <<<

# Begin Backup Script MAIN >>>

# Run functions to clean up logs and rotate
date_func
log_rotate
errorlog_rotate
echo "\n\n" >> ${LOG}

# Run mksysb
bck_mksysb
echo "\n\n" >> ${LOG}

# Run all non-rootvg backup
bck_non_rootvgs
echo "\n\n" >> ${LOG}

# Compete log files with some useful info
echo "More Information on Volume groups" >> ${LOG}
vg_info

# Eject the tape and make device offline
mt -f /dev/rmt0 offline
echo "Tape Device rmt0 offline" `date` >> ${LOG}
echo " " >> ${LOG}
echo "Backup logs can be found" ${LOG} >> ${LOG}

# Send backup logs via email.
# Uncomment out if you wish to use it.
# bck_email
# Backup up script END <<<

###############################
#                             #
#  How to RESTORE the Backups #
#                             #
###############################

# If you are reading this then something has gone wrong with your backup
# First Rule don't panic !
# Second Rule understand the backup process and recovery.
# Third Rule I am not responsible for any data loss, All backups should be tested regularly to ensure you can recover
# from an unforeseen situation.
# The procedure above has been tested and worked for me, it may need some changing depending on your hardware set-up.

#  I will explain to you the process to restore from the above backup procedure
# NOTE OF CAUTION ALWAYS ENSURE THE BACKUP TAPE IS WRITE PROTECTED.
# DO NOT TOUCH THE SYSTEM UNTIL TAPE IS SECURE.

# Insert tape into server drive
# A mksysb only contains the rootvg, you will need to restore the data after this process.
# To ensure the server will boot from tape drive
# bootlist -m normal -o
# List the boot order
# bootlist -m normal rmt0 hdisk0 cd0
# Sets the boot order to tape disk then cd; Choose accordingly.
# If this fails or you can't run these commands you will need to boot from the CD media used to install the server,
# note it has to be the same version.
# The mksysb tape should load and ask if you are ok to install, accept defaults and 15mins later your system will reboot.
# You should now have the OS back

# Restore the datavg
# Rewind tape to the beginning
# mt -f /dev/rmt0 rewind
# Skip tape forwards to where the datavg is, this will be after the mksysb which is on blocks 0,1,2,3
#  tctl -f /dev/rmt0.1 fsf 4
# restvg -f /dev/rmt0 hdiskX replace with the drive you wish to restore too.
# This should restore the volume group and the data.
# If you are still reading this and our lost so am I and I wrote it!
# Some further commands to help.
# lspv ; List's the Physical Volumes, will show the volume groups on your server
# lspv -l hdisk3 ; list's the contents of the hard disk specified.
# lsvg -l datavg ; Lists the logical volume details in the vg specified.

Generating a random number / seed

You may on occasion need to generate a random number within a shell script. Maybe you have 10 servers that run an update at 1:00 you do not want them all downloading at the same time so you end up staggering each server to download 10 minutes apart, this piece of code allows a number to be generated at random that can be used as a delay so one server connect after a 1 minute wait another at maybe 4 minutes another at perhaps 1.5 minutes etc. the point is the same script can be used on all machines with out having to worry about the scripts clashing.

# Function - generate a random number, range from 30 to 336,
# using time of day and hostname string as seeds
get_random()
{

        random=`nawk -v maxnum=300 '
        BEGIN {
                # seed random number using time of day
                srand()
                # get a random number, range from 30 to 300
                range = maxnum - 30
                select = 30 + int(rand() * range)
                print select
        }'`

        # generate a number as second seed from hostname string
        # value ranges from 1 to 36
        # this is needed to avoid different hosts from generating
        # the same random number at the same time
        seed2=`echo $MYHOST | nawk '
        {
                # create array from a list of lower case alphabetic and numeric
                # characters. use array index number as value for a character
                # the array size is 36 ( = 10 numeric + 26 alphabetic )
                z=split("1,2,3,4,5,6,7,8,9,0,a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z", num, ",")

                str=tolower($0)
                strlength=length($0)

                # generate a number from given string, value range from 1 to 36
                for (i = 1; i <= strlength; ++i){
                        char=substr(str,i,1)
                        for (j = 1; j <= z; ++j){
                                if (char == num[j] ){
                                        sum+=j
                                        count++
                                }
                        }
                }
                print int(sum/count)
        }'`

        # generate a random number using the second seed
        # value ranges from 30 to 336
        random=`expr $random + $seed2`

}

<pre class="postCode">



# Function - generate a random number, range from 30 to 336,

# using time of day and hostname string as seeds

get_random()

{



        random=`nawk -v maxnum=300 '

        BEGIN {

                # seed random number using time of day

                srand()

                # get a random number, range from 30 to 300

                range = maxnum - 30

                select = 30 + int(rand() * range)

                print select

        }'`



        # generate a number as second seed from hostname string

        # value ranges from 1 to 36

        # this is needed to avoid different hosts from generating

        # the same random number at the same time

        seed2=`echo $MYHOST | nawk '

        {

                # create array from a list of lower case alphabetic and numeric

                # characters. use array index number as value for a character

                # the array size is 36 ( = 10 numeric + 26 alphabetic )

                z=split("1,2,3,4,5,6,7,8,9,0,a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z", num, ",")



                str=tolower($0)

                strlength=length($0)



                # generate a number from given string, value range from 1 to 36

                for (i = 1; i <= strlength; ++i){

                        char=substr(str,i,1)

                        for (j = 1; j <= z; ++j){

                                if (char == num[j] ){

                                        sum+=j

                                        count++

                                }

                        }

                }

                print int(sum/count)

        }'`



        # generate a random number using the second seed

        # value ranges from 30 to 336

        random=`expr $random + $seed2`



}



</pre>

First Post - Hello

Hello reader,

Once again I find my self needing a way to keep tabs on my Shell Scripts.

Here goes another way to keep them in order, feel free to add comments or suggestions and any scripts you would like to see here.

I will try to update this when I can and add scripts or parts of scripts that may be useful.

About Me...

I have been scripting for at least 6 years, I am of course still learning and picking up tips all the time. I started with unix on a AIX system and then looked after a SCO server during this time I started my love affair with scripting, as time went on I found myself wantign to ditch windows and use my new found skills at home, I took the leap into Linux and have player with many distros since then, my server tends to be fedora and my laptop seems happy on Ubuntu! Why Ubuntu I dont really know it is common which I am not so keen on to much gui perhaps?? I have used PCLinuxOS, DSL. I became Redhat qualified a few years ago, and have not looked back, I left my job in the UK after 10years and moved to New Zealand 4 months ago and find myself learning all over again what I though I knew.

Well here goes another way to share my knowledge. ( It's not much but may help someone one day, maybe even today?)


All the best

Paul