Sed by examples, Part 2

posted by Arkham on June 29th, 2009

sed_logo_white

sed (stream editor) is a Unix utility which (a) parses text files and (b) implements a programming language which can apply textual transformations to such files.
It reads input files line by line (sequentially), applying the operation which has been specified via the command line (or a sed script), and then outputs the line.

Substitution

Substitute, for every line, the first occurence of foo (if any) with spam:

# test.txt
foo bar spam foo
spam foo bar egg
sausage foo spam egg
spam spam spam

sed -e 's/foo/spam/' test.txt

Substitute every occurence of foo with spam:
```
sed -e 's/foo/spam/g' test.txt
```
Substitute every occurence of foo with spam in the first two lines:
```
sed -e '1,2s/foo/spam/g' test.txt
```
Substitute every occurence of foo with spam in every line that starts with sausage and ends with egg:
```
sed -e '/^sausage.*egg$/s/foo/spam/g' test.txt
```

Substitute paths:

sed -e 's|/usr/bin/|/usr/local/bin|g' script.sh

Remove html tags:

<html>
<body>
Hello World!
</body>
</html>

sed -e 's/<[^>]*>//g' hello.html

Advanced substitution

Append (that’s what she said) to every line ( & represents what matches):
```
sed -e "s/^.*$/& (that's what she said)/g" test.txt
```

Append lol, rofl and lmao to the first three words of every line:

sed -e 's/\(^[^ ]*\) \([^ ]*\) \([^ ]*\)/\1lol \2rofl \3lmao/' test.txt

Multiple Commands

Print a file alternating row numbers:
```
sed -n -e '=;p' test.txt
```

Print some information about your cpu first core:

# script.sed
1,/^$/{
    /model name/p
    /flags/p
    /bogomips/p
}

sed -n -f script.sed /proc/cpuinfo

Add a line before/after each line or replace line:

# script.sed
i\
This line will be inserted before each line

# script.sed
a\
This line will be inserted after each line

# script.sed
c\
This line will be inserted in each line

sed -f script.sed test.txt

Read full article | 1 Comment

Tags: cheatsheet, learning, linux, sed, shell

Sed by examples

posted by Arkham on June 29th, 2009

Canon Tosh SED

sed (stream editor) is a Unix utility which (a) parses text files and (b) implements a programming language which can apply textual transformations to such files.
It reads input files line by line (sequentially), applying the operation which has been specified via the command line (or a sed script), and then outputs the line.

Delete

Delete the first line of a file:
```
sed -e '1d' /etc/fstab
```
Delete from the second to the tenth line:
```
sed -e '2,10d' /etc/fstab
```
Delete lines starting with #:
```
sed -e '/^#/d' /etc/fstab
```

Print

Print lines starting with #:
```
sed -n -e '/^#/p' /etc/fstab
```
Print blocks of text enclosed by BEGIN and END:
```
sed -n -e '/BEGIN/,/END/p' script.awk
```

Print Device section in xorg.conf:

sed -n -e '/Section "Device"/,/EndSection/p' /etc/X11/xorg.conf

Print main function in a C source file:

sed -n -e '/main[[:space:]]*(/,/^}/p' source.c

Read full article | No Comments

Tags: cheatsheet, learning, linux, sed, shell

Awk by examples

posted by Arkham on April 14th, 2009

auk

AWK is a language for processing files of text. A file is treated as a sequence of records, and by default each line is a record. Each line is broken up into a sequence of fields, so we can think of the first word in a line as the first field, the second word as the second field, and so on. An AWK program is of a sequence of pattern-action statements. AWK reads the input a line at a time. A line is scanned for each pattern in the program, and for each pattern that matches, the associated action is executed.

Basics

Print /etc/mtab (awk reads a line at time and prints it) :
```
awk '{ print }' /etc/mtab
```
Print /etc/mtab ($0 denotes the whole line) :
```
awk '{ print $0 }' /etc/mtab
```
List mounted filesystems ($1 denotes the first element of the line) :
```
awk '{ print $1 }' /etc/mtab
```
List groups (-F chooses the Field Separator) :
```
awk -F":" '{ print $1 }' /etc/group
```
List groups and id (awk concatenates print() arguments) :
```
awk -F":" '{ print $1 " " $3 }' /etc/group
```

List groups and id (Nicer format) :

awk -F":" '{ print "group: " $1 "\tid: " $3 }' /etc/group

Launching external awk scripts :

# first.awk
BEGIN {
    FS=":"
}
{ print $1 }

awk -f first.awk /etc/mtab

List IPv4 addressess :
```
ifconfig | awk '/inet / { print $2 }'
```

List processes run by root :

# psroot.awk
$1 == "root" { 
    printf("ROOT: "); 
    for (i=11; i<=NF; i++) 
        printf("%s ", $i);
    printf("\n")
}

ps au | awk -f psroot.awk

Print X warnings :

awk '$1 ~ /(WW)/ { print }' /var/log/Xorg.0.log

Print a file removing comments :
```
awk '! /^#/ { print }' /etc/fstab
```

Print number of files/directories :

ls -lA |  awk 'BEGIN{ x=0 } { x=x+1 } END{ print x-1 }'

Awk as calculator :
```
echo | awk '{ print ((2*5^2-1)%7) }'
```

Count empty lines :

# blanklines.awk
BEGIN { x=0 } 
/^$/  { x=x+1 } 
END   { print "I found " x " blank lines :)" }

awk -f blanklines.awk /etc/profile

Use regexp as FS (note the difference though) :

echo '   a   b c  d '| awk -F"[ \t\n]+" '{ print $2 }'

echo '   a   b c  d '| awk -F" " '{ print $2 }'

Understand NR and NF :

# nrnf.awk
BEGIN { x=0 } 
      { print "Words on line " NR ": " NF ; x+=NF } 
END   { print "Total lines: " NR ; print "Total words: " x }

awk -f nrnf.awk nrnf.awk

Read full article | 3 Comments

Tags: awk, cheatsheet, learning, linux, shell

GNU Screen cheat-sheet

posted by Arkham on January 26th, 2008

Screen Home:
http://www.gnu.org/software/screen/

Basics:

Ctrl+a c -> create new window

Ctrl+a A -> set window name

Ctrl+a w -> show all window

Ctrl+a 1|2|3|… -> switch to window n

Ctrl+a “ -> choose window

Ctrl+a Ctrl+a -> switch between last two windows

Ctrl+a n -> switch to next window

Ctrl+a p -> switch to previous window

Ctrl+a d -> detach window

Ctrl+a ? -> help

Ctrl+a [ -> start copy, move cursor to the copy location, press ENTER, select the chars, press ENTER to copy the selected characters to the buffer

Ctrl+a ] -> paste from buffer

Ctrl+a : screen foo -> create a new window executing foo and set its name

How to use screen:

screen -ls -> list of detached screen

screen -r PID -> attach detached screen session

screen -S MySession -> start a screen session with name MySession

screen -r MySession -> attach screen session with name MySession

screen -S MySession -md /usr/bin/foo -> start a screen session called MySession, launching /usr/bin/foo inside of it, and then detach it

screen -DR -> if a session is running, reattach; if necessary detach and logout remotely first. If it was not running, create it

Advanced:

Ctrl+a S -> create split screen

Ctrl+a TAB -> switch between split screens
If you created a new split screen, the current window is empty. either select an existing window (ctrl a “) or create a new split screen (ctrl a n).

Ctrl+a Q -> Kill all regions but the current one.

Ctrl+a X -> remove active window from split screen

Ctrl+a O -> logout active window (disable output)

Ctrl+a I -> login active window (enable output)

Read full article | 1 Comment

Tags: cheatsheet, linux, screen

Arkham’s Eyrie