Linux Bash Scripting Tutorial
A Stream Editor that takes its commands from a file
Sed is noninteractive stream editor.
normally editing files with vi, requires files should be opened,
sed is nondescriptive editor.
sed makes changes to the file and displays the content on the screen.
if u want to save the changed file, then redirect sed output to file.
sed often used to find-and-replace actions on lines containg patterns.
In this tutorial Following table used as Input to SED command
Rank Gold Silver Bronze Total NOC 1 39 41 33 113 United States (USA) 2 38 32 18 88 China (CHN) 3 27 14 17 58 Japan (JPN)* 4 22 20 22 64 Great Britain (GBR) 5 20 28 23 71 ROC 6 17 7 22 46 Australia (AUS) 7 10 12 14 36 Netherlands (NED) 8 10 12 11 33 France (FRA) 9 10 11 16 37 Germany (GER) 10 10 10 20 40 Italy (ITA) 11-93 137 150 206 493 Remaining Countries Totals 340 337 402 1079 (93 NOCs)
| SED SYNTAX | Explanation | Example |
| sed -n /Regular_Expression/p' filename | This will print matching lines | |
| sed -n s/Regular_Expression/replacement_text' filename | This will replace the pattern matched string with repleacement text | |
| sed 'n,md' filename | Deletes line from n to m | |
| sed '/Regular Expression/d | This will delete all the lines matching Regular Expression | |
| sed '/Regular Expression/!d | This will deletall all lines except the line(s) contaning Regular Expression | |
sed '/GBR/p' olympics.txt
all the lines containing GBR will be printed
Note: GBR is Regular Expression Pattern , 'p' is a command for printing.In this example, telling sed command look for word 'GBR'(capitals), if found display matching lines.
In above example, GBR is a single word, below example searches for multi word, 'Great Britain'
$sed -n '/Great Britain/p' olympics.txt
Above 2 examples are case sensitive,below one is case-insensitive. i.e G or B can be small case letters
$sed -n '/[gG]reat [bB]ritain/p' olympics.txt
above 3 conditions displays following output.4 Great Britain (GBR) 22 20 22 64
display countries based on Medals
$sed -n '/[0-9][0-9]/p' olympics.txt
prints all lines except header.because every line has 2-digit numbers.
$sed -n '/[0-9][0-9][0-9]/p' olympics.txt
print 3 digit medal countries
First line has 113 3-digit number
second and third lines also has 3-digit numbers
1 United States (USA) 39 41 33 113 11–93 Remaining 137 150 206 493 Totals (93 NOCs) 340 337 402 1079
sed -n '/[0-9]\{4\}/p ' olympics.txt
Display 4-digit medal count
Totals (93 NOCs) 340 337 402 1079
above example range with Quantifier used. i.e [0-9]{4} here [0-9] range of numbers followed by quantifier surrounded by curly braces. {4},exactly 4-digits, In bash regular expressions, Curly braces should be escaped with forward slash(\)
Last line has 1079, these are the total medals in olympics.
Display Multiple Countries/Multiple Matches using -e expression
Multiple Options can be specified using -e expr option
$sed -n -e '/China/p' -e '/GBR/p' olympics.txt
Displays Lines containing 'China' or 'GBR'
2 China (CHN) 38 32 18 88 4 Great Britain (GBR) 22 20 22 64above sed command can be rewritten using single Regular Expression
$sed -n '/\(Japan\|ROC\|Germany\)/p' olympics.txt
3 Japan(JPN)* 27 14 17 58 5 ROC 20 28 23 71 9 Germany(GER) 10 11 16 37
Above sed command can be rewritten with Extented Regular Expression i.e -E option.
$sed -n -E '/(Japan|ROC|Germany)/p' olympics.txt
Gives same results.use 'awk' command to Filter columns based on some condition
display country details which got less than 10 Silver Medals.
$awk '$3<10 {print $0}' olympics.txt | sed -n 'p'
6 17 7 22 46 Australia (AUS)
display country details which got gold medals between 20 and 30
$awk '$2>=20 && $2 <=30 {print $0}' olympics.txt | sed -n 'p'
3 27 14 17 58 Japan (JPN)* 4 22 20 22 64 Great Britain (GBR) 5 20 28 23 71 ROC
display country details which got more gold medals Bronze Medals
$awk '$2>=$4 {print $0}' olympics.txt | sed -n 'p'
Rank Gold Silver Bronze Total NOC 1 39 41 33 113 United States (USA) 2 38 32 18 88 China (CHN) 3 27 14 17 58 Japan (JPN)*
sed '1,3d' filename > newfile
mv newfile filename
a command appends after the address or pattern match
i command Inserts before the address or pattern match
c command changes based on Address or Pattern Match
$(touch tennis.txt;echo " ">tennis.txt; sed -i '1i Australian Open' tennis.txt) creates a file called tennis.txt using touch command. File has zero length, Inorder to add some text, add empty space to tennis.txt using echo " "> tennis.txt, -i option updates the file in-place. i.e command output will not be redirected to Standard Output device, In other words command output will be directly written into file itself.
$ sed -i '1i Australia Open' tennis.txt $ sed -i '1a, French Open' tennis.txt $ sed -i '/French*/a Wimbledon' tennis.txt $ sed -i '/Wim*/a US Open' tennis.txtPrint tennis.txt file content using 'cat' command
$cat tennis.txt Australia Open French Open Wimbledon US OpenNow we will see how to change file content using c command. All these events are held different dates in a Calendar Year. In this we will update/change the file content using c command. While using c command, '/patten/c newText' means it looks for pattern in the file, if it finds , replaces entire line with newText.This is the drawback with c command.
$ sed -i '/Austra*/c Australia Open in January' tennis.txt $ sed -i '/French*/c French Open in May' tennis.txt $ sed -i '/US*/c US Open in August' tennis.txt $ sed -i '/Wim*/c Wimbledon in June' tennis.txt $ cat tennis.txt Australia Open in January French Open in May Wimbledon in June US Open in AugustNow replace Wimbledon in June , country UK
$ sed '/Wim*/c country UK' tennis.txt Australia Open in January French Open in May country UK US Open in Augustabove example, it replaced Wimbledon in June with 'Country UK', to avoid this, user should provide full text i.e 'Wimbledon in June, Country UK'
$ sed '/Wim*/c Wimbledon in June, Country UK' tennis.txt Australia Open in January French Open in May Wimbledon in June, Country UK US Open in AugustNote: Use s command for word(s) replacement/substitution
| sed delete Syntax | Explanation | Example |
| sed '3d' filename | deletes third line | |
| sed '3,$d' filename | deletes from third line to last line $ in the address indicates last line , is called range operator | |
| sed '$d' filename | deletes the last line | |
| sed '/RE/d' filename | The line(s) containing RE pattern deleted | |
| sed '/RE/!d' filename | The line(s) not conating RE patterns deleted | |
| sed '4,+5d' filename | This will delete from line 4 to next 5 lines | |
| sed '1,5!d' filename | This will keep lines from 1 to 5,deletes all other lines | |
| sed '1~3d' filename | This will delete lines 1,4,7,10 and so on... ~ is step increment | |
| sed '2~2d' | This will delete every other line starting with line 2 to be deleted. | |
sed 's/Cashew/Almonds/g' shopping.txt
sed 's/[0–9][0–9]$/&.5/' shopping.txt
this will replace 2 digit numbers at the end of the line .5 appended to them
Quit q command is used for Quitting the sed processing without proceeding to the next line.
TopCities.txtTokyo,Japan New York, USA London, UK Paris, France Istanbul,Turkey Beijing, China Los Angeleses, USA Moscow, Russia
$ sed '1q' citities.txt # prints first city $ sed '3q' citities.txt #prints first 3 cities $ sed '$q' cities.txt #prints all lines from cities file.
Remove Empty Lines from a Text File(s)
Following command looks for line containing any alphanumeric characters, if it finds, takes those lines into consideration,i.e ignores empty lines.finally updates the file$ sed -i -n '/^[0-9a-zA-Z]/p' file.txt
Remove Blank Lines from all files
same as above, but sed command takes 2 input files file1.txt file2.txt, removes/filters blank lines.$ sed -i -n '/[0-9a-zA-Z]/p' file1.txt file2.txt
find e-mail address using sed command
$ sed -n -E '/[a-zA-Z0-9+_.-]+@[a-zA-Z0-9.-]+.[a-z]{2,3}/p' url_email.txt
ADS