Linux Bash Scripting Tutorial


Regular Expressions

Regular Expression is a sequence of characters that have certain patterns of text (with meta-characters), which will be searched from larger text or file. Patterns of text is a combination of characters and meta characters. Regular expression searches a given pattern in a file or files,output mateched lines of text.Bash Regular expressions uses greedy,non-greedy algorithms to find a pattern in a given file. Regular Expressions in Bash mostly used with find,grep,awk and sed commands.

	^ (caret) Beginning of the line
	$ (dollar) End of the line
	. (dot)  Match single character
	* (asterisk) Match any number of character i.e zero or more characters
	[] (brackets) Enclose character set to match
	\ (backslash) Escape. interpret following character literally
	\<..\> (angel brackets, escaped)  word boundary
	
Character matches
-----------------
	\{..\}  Escaped curly brackets
	[::]  POSIX character classes
	

Character classes

"alnum", "alpha", "ascii", "blank", "cntrl", "digit", "graph", "lower", "print", "punct", "space", "upper", "word" or "xdigit".

Basic versus extended Regular Expressions

In basic regular expressions the metacharacters "?", "+", "{", "|", "(", and ")" lose their special meaning; instead use the backslashed versions "\?", "\+", "\{", "\|", "\(", and "\)". s expressions.

grep command

         grep command searches the files for lines containing a match for a given patterns.When it finds the pattern in a line, it copies that matched line to terminal(by default). String matching or searching algorithms try to find places where one or several strings (also called patterns) are found within a larger string (searched text)

grep command syntax:
	grep [OPTION]... PATTERNS [FILE]
PATTERNS

PATTERNS can be a simple text or it can be a combination of text and meta characters.

FILE

FILE can be single text file or multiple text files or even a directory

Grep command examples

grep options
	-w  

		searching whole word

		grep -w 'hello'  file

		grep '\' file
			matches whole word not Ohello

		grep 'hello/>'  file
			searches only for words ending   in 'hello' so it matches 'Ohello'

	-l
		listing only file names

		grep -l 'main'  *.c  
			search file names in the current directory whose has word 'main'

	-r    
		recursive search

		grep -r 'hello'  path

	-C
		context around the matching line


		grep -C 2 'hello'  
			
			prints 2 lines of context around each matching line.

	-H
	/dev/null

		how do i force to print the name of the file.

		grep 'fire' /etc/passwd /dev/null

		grep -H /etc/passwd

		/etc/passwd:firebird:x:977:974::/:/sbin/nologin

	fgrep,egrep

		fgrep stands for fixed grep
		egrep stands for extended grep

find e-mail address using grep command

 $ grep -i -E '[a-zA-Z0-9+_.-]+@[a-zA-Z0-9.-]+.[a-z]{3}' url_email.txt 

Find Command

   find command, finds or searches for files in the specified directories based on search condition. if search condition is ommitted displays are files in the directory and its sub directories. search condition can be a file type, time,name of the file with wild cards etc., for ex: find all graphic files, find all log files, find all files which modified yesterday etc.,

You can search a system for any type of file, including filesm directories,links,sockets,block special files,character special files and many other.You can search the system that are world-writable,world-readable,files with sticky bit set,as well as perform an action on each file found.

find command syntax:
	Usage: find [-H] [-L] [-P] [-Olevel] [-D help|tree|search|stat|rates|opt|exec] [path...] [expression]

default path is current directory, default expression is -print

positional options (always true): -daystart -follow -regextype

normal options (always true, specified before other expressions): -depth --help -maxdepth LEVELS -mindepth LEVELS -mount -noleaf --version -xautofs -xdev -ignore_readdir_race -noignore_readdir_race

expression may consists of operators, options, tests and actions

operators

  • not operator --- ! Expression or -not Expression
  • and operator --- expr1 -and expr2 or in simple expr1 -a expr2
  • or operator --- expr1 -or expr2 or in simple expr1 -o expr2

tests   

  • checking file permissions using -perm
  • check inode using -inum
  • check group name using -group NAME or -g groupid
  • check file type using -type i.e file,directory, link etc.,
  • check file size using -size
  • check user name using -user NAME or -user id
  • check files case-sensitive using -name , case-insensitive using -iname
  • check file access time using -atime
  • check file modified time using -mtime
  • check file changed time using -ctime
  • Find empty directories using -empty
  • using -regex
  • using -path

actions

  • -delete
  • -print0 & -print
  • -printf FORMAT
  • -fprintf FILE FORMAT
  • -ls -fls
  • -quit
  • -exec command
  • -exec command {}
  • -execdir command
  • + - ok command
  • + - okdir command

Find all .c extension files only

Finding all c files in the current directory and its sub directories

Following command . indicates current directory
-name indicates name of the file should be printed
"*.c" expression tells find command look for .c extension files

   find .  -name "*.c"

Find all files except .c extension files

Finding all files except c files in the current directory and its sub directories

Following command . indicates current directory
-not indicates not in the expression, here expression is "*.c"
-name indicates name of the file should be printed
"*.c" expression tells find command look for .c extension files

   find . -not -name "*.c"

-not option should come first then other options(normal options should be specified before other expressions). otherwise following error will be displayed. find: paths must precede expression: `*.c'

Find all files only

   find .  -type f

Find all directories only

   find .  -type d

Find first occurance of a file

   find .  -iname '*programming*' -print -quit

Above example finds a file name contains a word 'programming', prints it and then quits immediately.
Note:There may be multiple files containing word 'programming', but display only first match

Find all files modified in last seven days

	find . -mtime -7 | tra -cvf /dev/rmt0

Find all files that have sticky bit set by using -perm -1000 -type f

	find / -perm 1000 -type f
This find command searches the entire system, starting in the root directory, for
files of type file that have minimum permissions of the sticky bit se

Find all files that are world-writable

	find / -perm -2 -exec ls -l {} \;
Notice that we ended the -exec
switch option with a blank space followed by a backslash-semicolon, \; . This is a
requirement to terminate the -exec switch option

Find Multiple Files using -o (or) -or condition

 find  ~/Pictures     -iname '*.jpg' -o  -iname '*.gif' -o -iname '*.png'

Searching multiple files using or condition i.e -o , above example in Pictures directory search for .jpg files or .gif files or .png files.

Move files using find and -exec test condition

	find $DIR -maxdepth 1 -iname '*.txt' -exec mv -b '{}'  newdir  \;

Find command finds all .txt files in the current directory(-maxdepth 1), then moves these text files to 'newdir'. Note: if 'newdir' already has same file name -b option in mv command, takes backup(first time), otherwise error message will be displayed.

Delete all empty files or directories

	find $DIR -type f -empty -delete

The above command first finds all empty files i.e size 0 files in the current directory and its sub directories. then deletes it using -delete command.

Delete all empty directories
	find $DIR -type d -empty -delete

The above command first finds all empty directories and its sub directories. then deletes it using -delete command.Note:It deletes hidden directories also

Delete File by using inode number

    Each file or directory has inode number, These inodes can be obtained using ls -i command, -i option in ls command prints inode number and file name, inode number can be used when file name or directory name has special characters(such as spaces, ? ,* etc.,), Special attention should be given for those names. Operating System generates/maintains inodes for each entry in the directory.

	$ find .  -inum 500573  -delete

The above find command, looks for a file has inode number 500573 in the current directory and then deletes it. This number may differ in your system

Exclude directories


find $HOME -path "/home/sitapathipurru/.*" -prune -o -print find $HOME -path "/home/sitapathipurru/.*" -path "/home/sitapathipurru/python-programming/.*" -prune -o -print
-amin N
-atime N
-anewer FILE
-newer FILE

-cmin N
-cnewer FILE
-ctime N 

-mmin N 
-mtime N 

-readable -writable -executable
-path PATTERN

-gid N -group NAME
-used N -user NAME
-uid N
-nouser -nogroup 

-size N[bcwkMG]

-perm [-/]MODE

ADS