Review
Getting Help
- man Get off information about commands
- man -k Get off information via keyword
- info Read info documents
- help Show help on shell built-in commands
- command --help Help on command .i.e. cp --help Will give help on cp
- bash -c "help" Short help on bash
- bash -c "help set" Short help on bash options
Special Characters
Space Argument separator \ Quote a single character A \ followed by a carriage return extend the current line ' Quote rext with spaces in it i.e. 'Hello world' $' Quote allows string expansion, backslash-escaped characters " Quote rext, expands variables and command substitution ` Command substitution i.e. echo "The date is `date`" $ Denote a shell variable # Comments the rest of the line ; Commands separator (commands) Run multiple commands in a subshell Control-C Interrupt a command Control-D Sends End of File from terminal Control-U Erases the entire command line Control-\ Is a stronger terminate than Control-c Control-Z Suspend process Redirection, Pipes and Filters
- Standard In for some commands 0 Standard In 1 Standard Out 2 Standard Error >2&1 Redirect standard error to standard out &> Redirect standard error & standard out <>> Redirect output >> Concatenate output to file <<>| Pipe tee Command write to file and standard out tee -a FILE #Allow appending to FILE xargs Build command from standardin Wildcards
* Wildcard for any character(s) ? Wildcard for single character [set] Wildcard for character in set [^set ] Wildcard for not the character in the set [!set ] Wildcard for not the character in the set {ab,dc} Wildcard for alternate between commas All wildcard work with existing files Only {} alternate work to create files Process Control
(command1; command) Run command1, then command2 in subshell command1&&command Run command1, then command2 if command1 successes command1||command Run command1, then command2 if command1 fails Regular Expressions
Definition: Text pattern of text character and meta characters Some Meta characters - Escape character \
- Single Character Meta characters
- . Matches any one character
- [...] Matches any one character in a set
- [^...] Matches any one character not in the set
- Quantifiers
> - * Matches the previous character zero or more times
- \{n\} Matches the previous character n times
- \{n,m\} Matches the previous character at least n & at most m
- \{n,\} Matches the previous character n or more times
- Anchors
- ^ Matching at the start the line
- $ Matching at the end of line
- Grouping \( \)
Commands that use regular expressionsawk Pattern scanning and text processing language ed Line-oriented text editor egrep extended grep emacs Emacs full screen text editor ex Line-oriented text editor expr Command evaluates an expression fgrep Grep from patterns in a file gawk GNU pattern scanning and processing language grep Searches file for pattern (also see fgrep & egrep) grep [OPTIONS] PATTERN [INPUT-FILE...] - -E same as egrep
- -c Count
- -e pattern (for multiple pattern on line)
- -f same as fgrep
- -i Ignore case
- -l Only list files containing pattern
- -q Quit (No output, only Return Code)
- -v Invert sense mode
perl Perl scripting python Python scripting sed Applies a set of user-specified editing command to a file sed [OPTIONS] 'sed_command' [INPUT_FILE...] - -n Suppress automatic printing
- -e expression - sed_command
- substitute other_text for some_text sed 's/some_text/other_text/g' FILE > NEWFILE
- multiple changes sed -e 's@abc@def@g' -e 's@xyz@mno@g' FILE
- print out line with faq in them sed -n '/faq/p' FILE
- change Page ### to (Page ###) at end of line sed 's/Page [0-9]+$/(&)/' file # & replace the match
- delete blank lines sed '/^[ \t]*$/d
tcl Tool command language vi Full screen text editor Shell Variables
Built-in Shell Variables
- CDPATH Path of shortcuts for cd (like PATH)
- COLUMNS Numbers of columns on the display
- EDITOR Path for editor
- HISTSIZE Number of commands in command history (default 500)
- IFS Input Field Separator
- LINES Numbers of lines on the display
- OFS Output Field Separator
- SECONDS Seconds that this shell is running
- SHELLOPT Colon separate list of shell options
Environment Variables
export var Will make a variable an environment variable HOME User's home directory LOGNAME User's name MAIL Name of user's mailbox PATH List of directories to be search by the shell
to find programs whose names are type as commands PS1 String that is used by the shell prompt PWD Name of current directory SHELL Name of current shell TERM The kind of terminal being used Environment variable are global to shell and subshells User Variables
Can be either upper or lower case var=value Define a variable var="" Define a variable as null local var Define a variable local to its scope Positional Variables
$0 Name of function or script being called $1 ... $9 Replace by arguments to shell or function ${n} Replace by n-th arguments to shell or function required if number of argument is over 9 Special Variables
$? Exit status or return code of last command $# Number of arguments $@ Argument 1 thru n with Input Field Separator $* "$1" $2" ... $n $! Process id of last background process $$ Process id of shell running this script $- The current shell flags Some Command Useful With Scripts
basename Strip directory and option suffix declare Built-in command declares variable dirname Strip non-directory part echo Built-in command display message to standard out echo -n Built-in command display message to standard out without newline echo -e Builtin Command display message to standout with escape sequences enable Built-in command to enable/disable (-n) built-in commands env Built-in command displays environment variables eval Built-in command evaluate arguments before executing results exec Built-in command runs command as the shell process exit Built-in command exits shell expr Evaluates an expression and output its value false Built-in command always return false condition local Built-in command make variable local read Built-in command reads data into variable from standard in seq Print a sequence of numbers - seq [OPTION]... LAST
- seq [OPTION]... FIRST LAST
- seq [OPTION]... FIRST INCREMENT LAST
- -w equalize width by padding with leading zeroes
sleep Command causes execution to stop for a specified number of seconds test Built-in command tests for various conditions, such as existence of a file, useful for controlling conditional script execution time Built-in command times commands times Print accumulated user and system times. true Built-in command always return true condition type Built-in command show what word is wait wait for the completion of background processing; is used to ensure that critical processing is complete before proceeding in a script Making Scripts Colorful
tput sgr0 #Reset text attributes to normal without clearing screen Escape sequence to change colors #\e[${forground};${background}m example white on black sequence is \e[37;40m #Forground colors Backgrounfd Color black=30; bgblace=40 red=31; bgred=41 green=32; bggreen=42 yellow=33; bgyellow=43 blue=34; bgblue=44 magenta=35; bgmagenta=45 cyan=36; bgcyan=46 white=37; bgwhite=47 Turn text attribute off \e[0m Change text attribute bold \e[1m Change text attribute underline \e[4m Change text attributer everse \e[7m Shell Functions and Scripts
Functions
- Function must be sourced just like .bashrc
- type function will list the function
- function functionname { shell commands }
- functionname () { shell commands }
- Example - a helpful function function givehelp { exec $1 --help | more; }
Scripts
#!/bin/bash Script name run script is run if execute bit is set #!/bin/bash -x As above, but script lines are displayed bash -x script As above /usr/bin/env bash Use the environment bash for the script Script Debugging options
set -o OPTION Command line Action - set -o noexec sh -n Don't run command, just check for syntax
- set -o verbose sh -v Echo commands before running them
- set -o xtrace sh -x Echo commands after running them
- set +o OPTION Turns OPTION off
Operation With Variables
Variable Usage
- $var Value of variable var
- "$var" Is null if variable is undefined avoids some shell syntax errors if variable is undefined
- ${var} Value of variable var avoids confusion when concatenating with text
- ${#var} Gives the length of the string contained in var
- ${var:FIRST:N} Extract string from var starting at FIRST position and continuing for N-1 characters. Note FIRST starts at 0.
Passing an variable to a program or script
echo $var | command Arrays
declare -a name Declare an array name[index]=value Just assigning a value defines it Index starts at zero No maximum limit Need not be contiguous Setting Variable From Execution of Command
var=`command` var is set to output of command var=$(command) same as above Arithmetic Operation
$(( expression )) Almost the same as 'expr expression' $[ expression ] Same as above ( expression ) Groups expression within $(( ... )) expr expression Note must quote * Arithmetic operator
+ add - substract * multiply / divide % remainder # number conversion (Only in bash not expr) Test Operation
[ -option arg ] Same as 'test -option arg [[ -option arg ]] Same as 'test -option arg [ arg1 -option arg2 ] Same as 'test arg1 -option arg2' [[ arg1 -option arg2 ]] Same as 'test arg1 -option arg2' See test command for test complete list of options - string1 = string2 String 1 equals string 2
- string1 !=string2 String 1 not equals string 2
- -n string String is not zero length
- -z string String is zero length
- -d FILE File is a directory
- -e FILE File exists
- -f FILE File exists and is a regular file
- -r FILE File exists and is readable
- -s FILE File exists and has length greater than zero
- -w FILE File exists and is writable
- -x FILE File exists and is executable
- num1 -eq num2 Number 1 equals number 2
- num1 -ne num2 Number 1 not equals number 2
- num1 -lt num2 Number 1 less than number 2
- num1 -le num2 Number 1 less than or equals number 2
- num1 -gt num2 Number 1 greater than number 2
- num1 -ge num2 Number 1 greater than or equals number 2
String Operation
${varname:-word} Return var if exists and is not null, else word ${varname:+word} Return word if var exists and is not null, else null ${varname:?mess} Return var if exists and is not null, else display mess and return from script with error Pattern Operations
${variable#pattern} If the pattern matches the beginning of the variable value, delete the shortest part that matches and return the rest ${variable##pattern} If the pattern matches the beginning the of variable value, delete the longest part that matches and return the rest ${variable%pattern} If the pattern matches the end of the variable value, delete the shortest part that matches and return the rest ${variable%%pattern} If the pattern matches the end of the variable value, delete the longest part that matches and return the rest The handy 'expr'
expr string : regrep Anchored pattern match expr match string pattern Same as above expr substr string pos {len] Substring beginning at pos of length len expr length string Length of string Flow Control
if/then/else or if/then/elif..
if condition then statements... [else statements...] fi if condition then statements... [elif condition then statements...] [else] statements...] fi
return from Function
return [ numeric expression or variable ] This is the return code
for
for name [ in list ] do statements done for variable = start to end do statements done
while/until
while condition do statements done until condition do statements done
break/continue
break [n] Break out of loop & select continue [n] Continue next iteration of loop case
case expression in pattern1[|pattern11] } statements ;; pattern2[|pattern21] } statements ;; ... esac
User Interfaces
select
select name [in list] do statements that can use $name done
Generate a menu of each item in the list formatted with a number for each choice Prompt the user for the number Store the selected choice in name. The number is stored in REPLY execute the statements in the do Repeat the process again Command Line Options
shift Shift 1st argument form the argument list getopts Used to process command line options OPTIND Variable contains number of options OPTARG Default variable for option Example while getopts ":ab:c" opt; do case $opt in a) process_option_a;; b) process_option_b $OPTARG is the option's argument;; c) process_option_c;; esac done shift $(($OPTIND - 1)) I/O in Scripts
printf Built-in command for formatted output read Built-in command to read one line into variable(s) read #Everything entered goes to REPLY read var #Everything is read into var read a b #Read 1st word in to a and rest into b read -t 300 var #Read with 300 second timeout Example how to use read from $file while read line; do do_something_with_line done <$file Process Handling
Traps
trap "" signal-list Ignore signal trap "cmds" signal-list Execute commands if signal is caught trap signal-list Reset signal to original condition trap : signal-list (undocumented) ignore signal, pass to child Signal are normally not passed to subprocesses Examples trap 'rm tmpfile; exit' 0 1 2 #remove tmpfile on exit, logout, interrupt trap "echo 'You hit Control-C'" INT while true ; do sleep 60 done Example parent child process #!/bin/bash #parent echo parent running trap 'echo parent exiting; exit' 0 trap :2 child sleep 1000 #!/bin/bash #child echo child started. pid is $$ trap 'echo child got signal 2; exit' INT sleep 1000 Examples Useful Scripts or Function
# Function top5 Example how to set defaults # Usage top5 {n} #list n processes function top5 { ps -ef | head -${1:-5} } # Function hereis Example of HERE IS FILE and handling arguments # Usage hereis word1 word2 ... function hereis { for name in "$@" do cat <# Function pick Return selected items by user # Usage: .e.g var=`pick *` function pick { for name in $@ ; do #for each item in argument list echo -n "$name (y/n/q)?" >/dev/tty #ask user to select read ans #read answer from standard in case $ans in #Check choices y*) echo $name;; #selected q*) break;; #skip rest of arguments *) continue;; #skip item esac done } # Function acal Display a nicer calendar # but will accept Alphabetic month function acal { m="" case $# in 0) cal; return;; #no arguments 1) m=$1; y=`date +%Y`;; #1 argument 2) m=$1; y=$2;; #2 arguments esac case $m in Jan*|jan* ) m=1;; Feb*|feb* ) m=2;; Mar*|mar* ) m=3;; Apr*|apr* ) m=4;; May|may } m=5;; Jun*|jun* ) m=6;; Jul*|jul* ) m=7;; Aug*|aug* ) m=8;; Sep*|sep* } m=9;; Oct*|oct* ) m=10;; Nov*|nov* ) m=11;; Dec*|dec* ) m=12;; [1-9]|1[0-2] ) ;; #numeric month *) ) y=$m; m="";; esac cal $m $y } ## Function selectex - Example select # function selectex () { choices="/bin /usr /home" select selection in $choices; do if [ $selection ]; then ls $selection break else echo 'Invalid selection' fi done } # Function fwhich Which command in $PATH is executed # function fwhich { if [[ $# -eq 0 ]] ; then cat <<>&2; return 2 Usage fwhich command #Example of parsing the $PATH Return 0 - found Return 1 - not found Return 2 - No arguments EndOfHelp fi for path in `echo $PATH | sed 's/^:/.:/ s/::/.:/g s/:$/:./ s/:/ /g'` do if [[ -x $path/$1 ]] ; # does executable file exists here? then echo $path/$1 # found it return 0 fi done return 1 # not found } # Name: overwrite Copy standard input to output after EOF function overwrite { if [[ $# -lt 2 ]] ; then echo "Usage: overwrite file command [args]" 1>&2; return 2 fi file=$1; shift new=/tmp/overwrite1$$; old=/tmp/overwrite2$$ trap 'rm -f $new $old; return 1' 1 2 15 # clean up files "$@" > $new if [[ $? -eq 0 ]] ; # collect output then # command completed successfully cp $file $old # save original file trap '' 1 2 15 # we are committed; ignore signals cp $new $file # copy new file into file rm -f $new $old # remove temp files else echo "overwrite: $1 failed, $file unchanged" 1>$2 return 1 fi } # Name: zgrep # Purpose: caseless grep of gzip files # Usage: zgrep text files.gz # function zgrep { if [ $# -eq 0 ] ; then echo "Usage: zgrep grep_text files.gz" return 2 fi text=$1 shift while [ $# -gt 0 ] do echo $1 gzip -cd $1 | grep -i $text shift done } # Name: hgrep # Purpose: highlighting grep # Usage: hgrep pattern files # function hgrep { if [ $# -lt 2 ] ; then echo "Usage: hgrep pattern files" return 2 fi pattern=$1;shift sep=$'\001' #note use of $' ' to create control characters bold=$'\e[1m'; off=$'\e[0m' underline=$'\e[4m'; reverse=$'\e[7m' #other choices of highlighting sed -n "s${sep}${pattern}${sep}${reverse}&${off}${sep}gp" $* }
No comments:
Post a Comment