LL5 – Viewing, Searching and Redirection

Required Reading

Review previous reading and LLs as necessary

The Linux Command Line by Shotts – 1.6

 

Assignment Submission

In this lab we will use the submission process previously established. We will also use the files and directories we created in the previous labs.  If you need remote assistance please use Jing to record your session telling me what you are doing as you go along and please use ls -l (i.e. long listing) liberally so that I can see what is going on. Also, last reminder to research all of these commands in Wikipedia and through the man pages in advance.

Open a terminal as before but this time – Do not resize your terminal to make it larger.

 

Intro ScreenShot

Open or launch your SSH application (e.g. Mac Terminal or Win PuTTy). This time, please don’t resize your terminal. Sometimes I need a small Terminal window to demonstrate functionality.  Also, if possible please just take a screenshot/picture of your open terminal as if you include other items (like your entire desktop) the Terminal can be very small making it harder for me to see your commands. Paste this screenshot into your correctly named pdf document and label or title this screenshot “Intro Screenshot”.

 

5.1 Viewing file contents

cat (concatenate) – The cat command can be used to display the contents of a file on the screen. First list the contents of your home directory and this should now be automatic and then type:

$ cat lastnamefirstnamelink.txt # this is loobyjameslink.txt for me

First note you displayed the lastnamefirstnameLL.txt file using its soft link created in LL4.  The actual file is of course in your ciss100/FirstnameLastname subdirectory and this time you accessed it through the soft link.

If you received an error message you are either in the wrong directory or the file isn’t what you/the system thinks it is.  If you didn’t receive anything (i.e. no file contents) it is likely your link is broken and you should have caught this in LL4 but you can fix this so please return to LL4 after reading the following in italics.

I will repeat this one last time, you can verify a file’s existence with a ls – l so, 1st see if the soft link is correctly present in your home directory (see LL4 for color coding etc.) and then, see if the actual file is present in the subdirectory (i.e. the actual file that the link points to). While in the subdirectory verify the file contents can be viewed (of course using cat).  If it is a broken link (described in LL4) you will need to remove it and recreate it properly.

As you can see from the cat, the file is longer than the size of the window, so it scrolls past the beginning rendering it unreadable (of course if you have a large Terminal window you will see it all).

The cat command has many options and uses as it can copy a file, create a file, display line numbers or be redirected as needed.  A really good cat introduction and resource for many things Unix & Linux is here: http://www.cyberciti.biz/faq/howto-use-cat-command-in-unix-linux-shell-script/

less – The less command writes the contents of a file onto the screen a page at a time.  To see this enter:

$ less lastnamefirstnamelink.txt

Now press the [space-bar] if you want to see another page, type [q] if you want to quit reading. As you can see, less is used in preference to cat for long files.

head – The head command writes the first ten lines of a file to the screen.  First clear the screen as presented in previous labs then type

$ head lastnamefirstnamelink.txt

Note: if this didn’t work you likely have a broken link (you’ll see red in directory listing) or the link was not correctly setup and it points to another file or directory.  If the link is correctly color coded in your directory listing perform a head on the linked file.  You of course can perform a long listing and see the entire path to the linked file.

Now type

$ head -5 lastnamefirstnamelink.txt

What difference did the -5 do to the head command?  Now if you have me in class you know I never miss an opportunity to reinforce material so look at my use of terminology identifying the “head” command, the “-5” number of lines option and the “lastnamefirstnameLL.txt” argument.

tail – The tail command writes the last ten lines of a file to the screen. Clear the screen and type

$ tail lastnamefirstnamelink.txt

Exercise 5.1 – How can you view the last 15 lines of the file? Please figure this out either by researching the topic (man pages) or by intuition and when you have succeeded, please take a screenshot and paste it into your submission file.

Please clear your screen after taking your screenshot and before continuing.

 

5.2 Searching the contents of a file

Simple searching using less

Using less you can search though a text file for a keyword (pattern). For example, to search through lastnamefirstnameLL.txt for the word ’10’, type

$ less lastnamefirstnamelink.txt

then, still in less (i.e. don’t press [q] to quit), type a forward slash [/] followed by the word to search for so in this case enter  /10

As you can see less finds and highlights the keyword. Type [n] to search for the next occurrence of the word (I threw some Linux man page syntax at you there as items enclosed in square brackets are required so [n] meant type ‘n’ and hit enter).  Now when done type [q] if you are left with a colon prompt.

Now you’re thinking why does this guy think this is important or even useful.  I mentioned that Linux server environments may not have a graphical editor for both security (users who are not comfortable in a CLI environment cannot do anything to harm the system) and efficiency (graphical user interfaces consume resources).  Also over a network, you probably only have a GUI so on a Sunday morning at 4:30 a.m. when the CEO loses her password and cannot get into the system and calls you to reset her password… why CEOs get up at 4:30 a.m. on a Sunday remains a mystery but thats why they are CEOs 🙂 …   do you want to drive to the business and reset the password at 4:30 a.m. or do you want to stumble to your computer and do this over the network. Now that you are at the computer do you want to search for her name in the system settings line by line?

Now to further illustrate the need for CLI knowledge and digress even further – 🙂 … think about movies that show high tech hacking, what are they doing?  They are all correctly working from a CLI.  Can you expect to thwart malicious hacking if you don’t have equivalent knowledge to the hackers?

grep (don’t ask why it is called grep) – grep is one of many standard UNIX/Linux system administrator utilities. It searches files for specified words or patterns. The format is grep followed by the search phrase followed by the file to search.  Let’s look a this but first, clear the screen and then type

$ grep testing lastnamefirstnamelink.txt

As you can see, grep has printed out each line that contains the word “testing”

Or has it???? … Try typing

$ grep Testing lastnamefirstnamelink.txt

The grep command is case sensitive which makes sense because Linux is case sensitive therefore it distinguishes between Testing and testing.  To ignore upper/lower case distinctions, use the -i option, i.e. type

$ grep -i testing lastnamefirstnamelink.txt

To search for a phrase or pattern, you must enclose it in single quotes (the apostrophe symbol). For example to search for est, type

$ grep -i ‘est’ lastnamefirstnamelink.txt

Some of the other options of grep are:

-v display those lines that do NOT match
-n precede each maching line with the line number
-c print only the total count of matched lines

Try some of them and see the different results. Don’t forget, you can use more than one option at a time, for example, the number of lines without the words testing or Testing is

$ grep -ivc testing lastnamefirstnamelink.txt

Exercise 5.2  Please make your screen big enough and take a screenshot and paste it into your submission document.

Please clear your screen after taking your screenshot and before continuing.

 

5.3 File Utilities

wc (word count) – A handy little utility is the wc command, short for word count. To do a word count on lastnamefirstnameLL.txt type

$ wc -w lastnamefirstnamelink.txt

To find out how many lines the file has, type

$ wc -l lastnamefirstnamelink.txt

Exercise 5.3 Please take a screenshot and paste it into the .pdf document.

Please clear your screen after taking your screenshot and before continuing.

 

5.4 Redirection (chapter 2.6.1)

Most processes initiated by Linux commands write to the standard output (that is, they write to the terminal screen), and take their input from the standard input (that is, they read it from the keyboard). There is also the standard error, where processes write their error messages, by default, to the terminal screen.  So what do we have – input, processing and output – :).

First, please navigate to your ciss100/FirstnameLastname directory you created in previous labs.

We have already seen the use of the cat command to write the contents of a file to the screen. Now type cat without specifying a file to read

$ cat

Then type a few words on the keyboard and press the [Enter] key.

Finally hold the [Ctrl] key down and press [d] (written as ^D for short) to end the input. What happened?

If you run the cat command without specifing a file to read, it reads the standard input (the keyboard), and on receiving the’end of file’ (^D), copies it to the standard output (the screen).

In Linux, we can redirect both the input and the output of commands to files and this can be very useful.

 

5.5 Redirecting the Output

>   We use the > symbol to redirect the output of a command. For example, to create a file called list1 containing a list of fruit, type

$ cat > list1

Then type in the names of the following fruit. Press [Enter] after each one and please don’t worry if you make a spelling mistake or something of the sort.

pear
banana
apple
^D (Control D to stop)

What happens is the cat command reads the standard input (the keyboard) and the > redirects the output, which normally goes to the screen, into a file called list1.   Now recall that Linux treats files and directories the same,  could the same be said for devices?  We will explore this later but if you are ready for this see here: http://www.cyberciti.biz/faq/understanding-unix-linux-bsd-device-files/

To read the contents of the file, type

$ cat list1 #of course you could have used less, more, head or tail to view the file

Create another file using cat and redirection called list2 containing the following fruit: orange, plum, mango, grapefruit. Read the contents of list2 to verify your input and the file’s existence

>>    The form >> appends standard output to a file. So to add more items to the file list1, type

$ cat >> list1

Then type in the names of more fruit

peach
grape
orange
^D (Control D to stop)

To read the contents of the file, type

$ cat list1

You should now have two files. One contains six fruit, the other contains four fruit. We will now use the cat command to join (concatenate) list1 and list2 into a new file called biglist. Type

$ cat list1 list2 > biglist

What this is doing is reading the contents of list1 and list2 in turn, then outputting the text to the file biglist. 

Exercise 5.5 Display biglist on the terminal screen using a command other than “cat”, take a screen shot and paste this into your file. Note you should be looking at directory listings to verify everything is correct even if this was not specified as an exercise.

Please clear your screen after taking your screenshot and before continuing.

 

5.6 Redirecting the Input

<    We use the < symbol to redirect the input of a command.

The command sort alphabetically or numerically sorts a list. Type

$ sort

Then type in the names of some vegetables. Press [Return] after each one.

carrot
beetroot
artichoke
^D (control d to stop)

The output will be

artichoke
beetroot
carrot

Using < you can redirect the input to come from a file rather than the keyboard. For example, to sort the list of fruit, type

$ sort < biglist

and the sorted list will be output to the screen.

To output the sorted list to a file, type,

$ sort < biglist > slist

Exercise 5.6  Use a command other than “cat” and other than the command you used above in 5.5 to display the contents of the file slist. Take a screenshot and paste this into your .pdf document.

Please clear your screen after taking your screenshot and before continuing.

 

5.7 Pipes

|   The vertical bar is the Linux pipe symbol.  Pipes are similar to redirection however rather than redirecting output to a file, a pipe redirects output to another command.

First, to see who is on the system with you, type

$ who

Over time we may want to see who is logged on a system at a particular point in time.  One method to get a sorted list of these names is to type the following:

$ who > names.txt
$ sort < names.txt

This is a bit slow and too much work for a Sys Admin and we then have to remember to remove the temporary file called names when finished (please do this). What you really want to do is connect the output of the who command directly to the input of the sort command. This is exactly what pipes do. The symbol for a pipe is the vertical bar |

For example, typing the following will give the same result as above, but will execute quicker and be cleaner and save this to a file.

$ who | sort > users

To find out how many users are logged on, we could type

$ who | wc -l

Exercise 5.7  In this exercise you will use 2 pipes and a redirection on a single line. Using a pipe, sort a directory listing of your home directory (i.e. list contents of directory and sort)… then pipe this output into cat and redirect the output into a file named “exercise5.7.txt”.  Display the command used and the contents of the exercise5.7.txt file on the terminal screen, take a screenshot and paste this screenshot into your assignment submission file and submit this assignment file.

Hint: if this appears complex, try figuring it out one command/pipe/redirection at a time.  In other words, you may have to explore and try things to verify you are doing it properly.

Here’s a little demonstration of multiple pipes and redirection to assist but note – this demonstration is not Exercise 5.7:

 

Now in class or online you hear me present and query the class for the 3 types of computer interfaces that are: hardware (e.g. ports like USB & Ethernet), human computer (HCI) (e.g. keyboard, screen) and software.  It is possible you just programmed your first software interface as pipes send information between processes (of course if you are in CISS 110 you are creating software interfaces between objects).

Additional commands

Please research a2ps and lpr however we will not use these now.

 

Summary

If you are here… nicely done as you have now graduated to a first Linux user level – :). To understand the import of this, please look at the www.ciss100.com emergent topics “Jobs” and “Linux” tags as you will see Linux Sys Admins are in high demand and are compensated very very well.  As usual, please research the commands you used above in further detail on Wikipedia and the Linux man pages and continue to practice.

[table id=4 /]

Leave a Reply