LINUX CLASSES - DATA MANIPULATION

Will Pipe Fitting Bring About World Peace?

Throughout this section, we've discussed how to manipulate a file with many different tools. But you can use each of these tools in a more powerful way by combining them into pipelines. Back in "Living in a Shell" you learned how to pump the output from one command to another by redirecting the input or output of those commands. Following are several examples that show how to combine the power of the tools described in this section.

· To find files that have not been accessed for over 30 days and print the first five lines of each:

find . -atime +30 -exec head -5 {} \;

· To find out if a process named netscape is running:

ps | grep netscape

· To print only the second and third lines of a file:

head -3 some.file | tail -2

Note that the usage changes slightly when a command is in the second or subsequent stages of a pipeline. No input file is specified, because the previous stage feeds the command.

At the beginning of this section, I said that it would be no problem to search within a bunch of files, pull out all lines that contain a certain keyword, sort those lines, eliminate duplicates, and then print just the third column of each line. Here's proof that you can do it all on one line:

grep 'stuff' *.data | sort +1 -2 | uniq | cut -f3

Seems almost too easy, doesn't it? Beats the heck out of writing a program several hundred lines long if you want to run it only once! Now let's use the rest of the commands from this section in another pipeline. Start by creating the file odds.ends containing the lines shown here:

Ford Cat 47
IBM Lion 152
Xerox Slug 31
Zenith Bear 26
Intel Cat 133
Hershey Lynx 28
Apple Panda 74

Then execute the following command. (The backslash at the end of a line tells the shell that you are continuing a command on the next line.) Can you figure out what the output will be?

head -5 odds.ends | sed s/Cat/Tigger/g | \
awk /Tigger/'{print "Buy",$1,"from",$2,"at",$3}' | \
tail -1

The correct answer is "Buy Intel from Tigger at 133"--can you prove it?

Previous Lesson: Finding Files
Next Lesson: Linux Shell Scripts

[ RETURN TO INDEX ]

Comments - most recent first
(Please feel free to answer questions posted by others!)

Niamul BAqui (25 Mar 2012, 15:17)

Thank you

Amit D (11 May 2011, 09:21)

Good one

Newbie (19 Nov 2010, 15:37)

Thank you!

I welcome your comments. However... I am puzzled by many people who say "Please send me the Linux tutorial." This website *is* your Linux Tutorial! Read everything here, learn all you can, ask questions if you like. But don't ask me to send what you already have. :-)

NO SPAM! If you post garbage, it will be deleted, and you will be banned.

Will Pipe Fitting Bring About World Peace?

Comments - most recent first(Please feel free to answer questions posted by others!)

Comments - most recent first
(Please feel free to answer questions posted by others!)