File Viewing

Below are the commands used to view the contents of a file, soft link or of a hard link without opening

(Explanations for these links are explained in a separate post ).


i) cat

cat filename

ii) more

more filename

iii) less

 less filename

(Note: Give Enter and Q accordingly to continue and terminate the execution)

Difference B/W more and less:

more just prints the text as is, stopping for page breaks, and does not clear the screen, it can be back grounded but it doesn't clear the screen where 

less is a full-screen application that gives you a searchable, scrollable window and clears the screen after you exit.

Difference B/W more and cat:

 more can be used to work with the standard output but not cat

For example:

We can execute 

ls -lrt | more 

but not ls -lrt|cat

Cut with Examples

Cut is a command to extract portion of text from a file or from the given text.

Below are some of the examples based on a file test.txt whose contents are as below.

$ cat test.txt

cat command for file oriented operations.

cp command for copy files or directories.

ls command to list out files and directories with its attributes.

--------------------------------------------

1. Select Column of Characters

To extract only a desired column(in this case column2) from a file use -c option.

$ cut -c2 test.txt

a

p

s

--------------------------------------------

2. Select Column of Characters using Range

The following example extracts first 3 characters of each line.

$ cut -c1-3 test.txt

cat

cp

ls

----------------------------------------

3. Select Column of Characters using either Start or End Position

The following specifies only the start position before the ‘-’. This example extracts from 3rd character to end of each line from test.txt file.

$ cut -c3- test.txt

t command for file oriented operations.

 command for copy files or directories.

 command to list out files and directories with its attributes.

The following specifies only the end position after the ‘-’. This example extracts 8 characters from the beginning of each line from test.txt file.

$ cut -c-8 test.txt

cat comm

cp comma

ls comma

The entire line would get printed when you don’t specify a number before or after the ‘-’ as shown below.

$ cut -c- test.txt

cat command for file oriented operations.

cp command for copy files or directories.

ls command to list out files and directories with its attributes.

------------------------------------------

4. Select a Specific Field from a File

Instead of selecting x number of characters, if you like to extract a whole field, you can combine option -f and -d.

The following example displays only first field of each lines from /etc/passwd file using the field delimiter : (colon).

$ cut -d':' -f1 /etc/passwd

root

daemon

bin

sys

sync

games

bala

----------------------------------------

5. Select Multiple Fields from a File

$ grep "/bin/bash" /etc/passwd | cut -d':' -f1,6

root:/root

bala:/home/bala

To display the range of fields specify start field and end field as shown below. In this example, we are selecting field 1 through 4, 6 and 7

$ grep "/bin/bash" /etc/passwd | cut -d':' -f1-4,6,7

root:x:0:0:/root:/bin/bash

bala:x:1000:1000:/home/bala:/bin/bash

------------------------------------------

6. Select Fields Only When a Line Contains the Delimiter

In our /etc/passwd example, if you pass a different delimiter other than : (colon), cut will just display the whole line.

In the following example, we’ve specified the delimiter as | (pipe), and cut command simply displays the whole line, even when it doesn’t find any line that has | (pipe) as delimiter.

$ grep "/bin/bash" /etc/passwd | cut -d'|' -f1

root:x:0:0:root:/root:/bin/bash

bala:x:1000:1000:bala,,,:/home/bala:/bin/bash

But, it is possible to filter and display only the lines that contains the specified delimiter using -s option.

The following example doesn’t display any output, as the cut command didn’t find any lines that has | (pipe) as delimiter in the /etc/passwd file.

$ grep "/bin/bash" /etc/passwd | cut -d'|' -s -f1

------------------------------------------

7. Complement :

Select All Fields Except the Specified Fields

using –complement option.

The following example displays all the fields from /etc/passwd file except field 7

$ grep "/bin/bash" /etc/passwd | cut -d':' --complement -s -f7

root:x:0:0:root:/root

bala:x:1000:1000:bala,,,:/home/bala

--------------------------------------

8. Output Delimiter:

To change the output delimiter use the option –output-delimiter as shown below. In this example, the input delimiter is : (colon), but the output delimiter is # (hash).

$ grep "/bin/bash" /etc/passwd | cut -d':' -s -f1,6,7 --output-delimiter='#'

root#/root#/bin/bash

bala#/home/bala#/bin/bash

-----------------------------------------

9. Change Output Delimiter to Newline

In this example, each and every field of the cut command output is displayed in a separate line. We still used –output-delimiter, but the value is $’\n’ which indicates that we should add a newline as the output delimiter.

$ grep bala /etc/passwd | cut -d':' -f1,6,7 --output-delimiter=$'\n'

bala

/home/bala

/bin/bash

----------------------------------------

10. Example of combining Cut with Other Unix Commands

$ ps axu | grep python | sed 's/\s\+/ /g' | cut -d' ' -f2,11-

2231 /usr/bin/python /usr/lib/unity-lens-video/unity-lens-video

2311 /usr/bin/python /usr/lib/unity-scope-video-remote/unity-scope-video-remote

2414 /usr/bin/python /usr/lib/ubuntuone-client/ubuntuone-syncdaemon

2463 /usr/bin/python /usr/lib/system-service/system-service-d

3274 grep --color=auto python

AWK with Examples

BASIC s ....

Lets take an example of a file abc.txt whose contents are 

$cat abc.txt

10 100 1000 one hundred
20 200 2000 two twohundred
30 300 3000 three threehundred

To print the first column

$awk '{print $1}' abc.txt
10
20
30

To print the first and second column

$awk '{print $1 $2}' abc.txt
10 100
20 200
30 300


By default space is the delimiter.You can change by flag -F like

$awk -OFS":" '{print $1 $4}' abc.txt

10:one
20:two
30:three

FLAGS:

NR                         Line Number
IFS                         Input File Seperator
OFS                       OutPut File Seperator

To count the no of  rows of a file(to replace wc -l abc.txt)

$awk 'BEGIN {count=0} {count=count+1} END {print count}' abc.txt
3

(OR)

simply

$ awk 'END {print NR}' abc.txt
3

To count the sum of  nth clolumns of a file(say 2 in this case)

$awk 'BEGIN {sum=0} {sum=sum+$2} END {print sum}' abc.txt
600

Lets go to some Advanced ...

Reading of input and printing can be managed by file seperators which is explained in a seperate post.

Loops, control statements, file handling and varible pasing can also be done in awk as explained below

<UNDER CONSTRUCTION> :)

Redirections, Standard I/O

BasicS  ...

Most of the cases we use this redirection operators in shell script or at prompt level.

./scriptname.sh > logfile         To send the standard output of ./scriptname by replacing the existing logfile

./scriptname.sh >> logfile     To send the standard output of ./scriptname by appending the existing logfile

./scriptname.sh 2> logfile       To send the standard error while running the scriptname.sh

./scriptname.sh 2>> logfile    To send the standard error while running the scriptname.sh with append

Lets go to some advanced...

To send standard error as well as standard output to logfile

./scriptname.sh  2> logfile >&1    

(OR)

./scriptname.sh  > logfile 2>&1