Unix Primer - Basic Commands In the Unix Shell

The shell is a command programming language that provides an interface to the UNIX operating system. You can access your shell account via the SSH network protocol from anywhere on the internet.  Two free programs installed on most IUPUI computers are PuTTY and WinSCP.  Both are available through iuware.iu.edu.  To connect, use the hostname:  pegasus.cs.iupui.edu.

1. Directories

The shell should start you in your home directory. This is your individual space on the UNIX system for your files. You can find out the name of your current working directory by typing:

pegasus{jbeams}1: pwd
/users/username

(The stuff before the colon ':' designates your command line prompt, and you should type the letters 'p', 'w', 'd', and then "enter" - always conclude each command by pressing the "enter" key. The response that follows on the next line will be the name of your home directory, where the name following the last slash should be your username.) The directory structure can be conceptualized as an inverted tree.

No matter where in the directory structure you are, you can always get back to your home directory by typing:

pegasus{jbeams}3: cd

(without specifying a directory name).

From your home directory, create a new subdirectory named "primer" for working through this tutorial:

pegasus{jbeams}4: mkdir primer

You can remove an empty subdirectory with the following command (but don't do this right now):

pegasus{jbeams}5: rmdir primer

(Note: if you do remove "primer", please create it again.)

Now change to the "primer" subdirectory, making it your current working directory:

pegasus{jbeams}6: cd primer

2. Files

Files live within directories. You can see a list of the files in your "primer" directory (which should be your current working directory) by typing:

pegasus{jbeams}7: ls

Since you just created the directory, nothing will be listed because the directory is empty. Create your first file using the pico text editor:

pegasus{jbeams}8: pico first

The pico editor fills the entire console window. You can type text and move the cursor around with the arrow keys; the bottom of the screen presents the commands available. Type the sentence: "My first file." Then press "^O" (hold the left "control" key while pressing 'O') and "enter" to save the file, then "^X" (hold the left "control" key while pressing 'X') to exit pico. Now when you list your files, you will see file "first" listed:

pegasus{jbeams}9: ls
first

You can view a text file with the following command:

pegasus{jbeams}10: cat first
My first file.

("cat" is short for concatenate - you can use this to display multiple files together on the screen.) If you have a file that is longer than your 24-line console window, use instead "more" to list one page at a time or "less" to scroll the file down and up with the arrow keys. Don't use these programs to try to display binary (non-text) files on your console - the attempt to print the non-printable control characters might alter your console settings and render the console unusable.

Copy file "first" using the following command:

pegasus{jbeams}11: cp first 2nd

By doing this you have created a new file named "2nd" which is a duplicate of file "first". The file listing reveals:

pegasus{jbeams}12: ls
2nd first

Now rename the file "2nd" to "second":

pegasus{jbeams}13: mv 2nd second

Listing the files still shows two files because you haven't created a new file, just changed an existing file's name:

pegasus{jbeams}14: ls
first second

If you "cat" the second file, you'll see the same sentence as in your first file:

pegasus{jbeams}15: cat second
My first file.

The command more is essential for looking at the contents of long files. Pressing the space bar advances the display by one screen. To quit the display before the end press the q key. If you are using more to look at a file with the command

pegasus{jbeams}16: more filename

then you can go back a screen by pressing the b key.

The command cat (concatenate and display) has a broader usefulness than its name suggests. For example, the command:

pegasus{jbeams}17: cat filename

copies file filename to the standard output, the screen. If more than one file is given then they are copied to the screen successively. The standard output can be re-directed to a file so that the contents of two (or more) files can be concatenated into a third:

pegasus{jbeams}18: cat infile1 infile2 > outfile

The standard output can be redirected to append to an existing file, thus:

pegasus{jbeams}19: cat infile3 >> outfile

adds the contents of infile3 to the end of file outfile. If cat is invoked without an argument then it copies keyboard input to the screen. To escape from this enter <Ctrl D> to signal 'end of file'.

The command more is essential for looking at the contents of long files. Pressing the space bar advances the display by one screen. To quit the display before the end press the q key. If you are using more to look at a file with the command

pegasus{jbeams}20: more filename

then you can go back a screen by pressing the b key.

It is possible to 'pipe' the output of one process into the standard input of a second. So to examine the output of your own program a screen at a time, as it is produced, the command is:

pegasus{jbeams}21: a.out | more

3. Input and Output Redirection

A process usually has a standard input and a standard output associated with it. The default for standard input is the keyboard, while standard output goes to the screen. If there is an error then an error message may go to the standard error, which also defaults to the screen.

It is possible to change the standard input and output when a command is issued. For example, for a compiled program to take input from file indata the command would be:

pegasus{jbeams}22: a.out < indata

If we also want to redirect the output to file outdata the command becomes:

pegasus{jbeams}23: a.out < indata > outdata

"mv" will allow you to move files, not just rename them. Perform the following commands:

pegasus{jbeams}24: mkdir sub
pegasus{jbeams}25: mv second sub
pegasus{jbeams}26: ls sub
second
pegasus{jbeams}27: ls
first sub

This creates a new subdirectory named "sub", moves "second" into "sub", then lists the contents of both directories. You can list even more information about files by using the "-l" option with "ls":

pegasus{jbeams}28: ls -l
-rw-r--r-- 1 username group 15 May 22 16:26 first
drwxr-xr-x 2 username group 512 May 22 17:11 sub

(where "username" will be your username and "group" will be your group name). Among other things, this lists the creation date and time, file access permissions, and file size in bytes. The letter 'd' (the first character on the line) indicates the directory names.

Next perform the following commands:

pegasus{jbeams}29: cd sub
pegasus{jbeams}30: pwd
/Users/username/primer/sub
pegasus{jbeams}31: ls -l
-rw-r--r-- 1 username group 15 May 22 16:55 second
pegasus{jbeams}32: cd ..
pegasus{jbeams}33: pwd
/Users/username/primer

This changes your current working directory to the "sub" subdirectory under "primer", lists the files there, then changes you back up a level. The ".." always refers to the parent directory of the given subdirectory.

Finally, clean up the duplicate files by removing the "second" file and the "sub" subdirectory:

pegasus{jbeams}34: rm sub/second
pegasus{jbeams}35: rmdir sub
pegasus{jbeams}36: ls -l
-rw-r--r-- 1 username group 15 May 22 16:26 first

This shows that you can refer to a file in a different directory using the relative path name to the file (you can also use the absolute path name to the file - something like "/Users/username/primer/sub/second", depending on your home directory). You can also include the ".." within the path name (for instance, you could have referred to the file as "../primer/sub/second").

4. Programs
Programs are just files that contain lists of instructions in the form of binary numbers that can be run or 'executed' by the computer when they are loaded into memory, as opposed to files that contain lines of text. Typically executable files actually start as text files containing programs written in a high-level language like C. Compilers read these program files, and build an appropriate binary executable file. All the commands you have been typing: ls, cd, mkdir, etc. are just binary files derived from such programs, also often referred to as binaries or execuables. Many of the programs from which these executables are compiled are no longer than the programs you will write in this course. In this class you will learn how to create your own programs, which in UNIX can then be accessed and used like any other commands in the system. This simplicity and extensibility is what makes UNIX so powerful.

5. Getting Further Information

On-line manuals are a standard part of Unix. They may be consulted by use of the man command. For example, to see the on-line information on the chmod command described in section 4.6, enter the Unix command:

pegasus{jbeams}37: man chmod

The man command uses the more utility to show one screen of the manual at a time.

6. Concepts of file permissions.

Because there is typically more than one user on a Linux system, Linux provides a mechanism known as file permissions, which protect user files from tampering by other users. This mechanism lets files and directories be ``owned'' by a particular user. For example, because Larry created the files in his home directory, Larry owns those files and has access to them.

Linux also lets files be shared between users and groups of users. If Larry desired, he could cut off access to his files so that no other user could access them. However, on most systems the default is to allow other users to read your files but not modify or delete them in any way.

Every file is owned by a particular user. However, files are also owned by a particular group, which is a defined group of users of the system. Every user is placed into at least one group when that user's account is created. However, the system administrator may grant the user access to more than one group.

Groups are usually defined by the type of users who access the machine. For example, on a university Linux system users may be placed into the groups student, staff, faculty or guest. There are also a few system-defined groups (like bin and admin) which are used by the system itself to control access to resources--very rarely do actual users belong to these system groups.

Permissions fall into three main divisions: read, write, and execute. These permissions may be granted to three classes of users: the owner of the file, the group to which the file belongs, and to all users, regardless of group.

Read permission lets a user read the contents of the file, or in the case of directories, list the contents of the directory (using ls). Write permission lets the user write to and modify the file. For directories, write permission lets the user create new files or delete files within that directory. Finally, execute permission lets the user run the file as a program or shell script (if the file is a program or shell script). For directories, having execute permission lets the user cd into the directory in question.

7. Interpreting file permissions.

Let's look at an example that demonstrates file permissions. Using the ls command with the -l option displays a ``long'' listing of the file, including file permissions.

pegasus{jbeams}38: ls -l stuff
-rw-r--r-- 1 larry users 612661 May 16 15:31 stuff


The first field in the listing represents the file permissions. The third field is the owner of the file (larry) and the fourth field is the group to which the file belongs (users). Obviously, the last field is the name of the file (stuff). We'll cover the other fields later.

This file is owned by larry, and belongs to the group users. The string -rw-r-r- lists, in order, the permissions granted to the file's owner, the file's group, and everybody else.

The first character of the permissions string (``-'') represents the type of file. A ``-'' means that this is a regular file (as opposed to a directory or device driver). The next three characters (``rw-'') represent the permissions granted to the file's owner, larry. The ``r'' stands for ``read'' and the ``w'' stands for ``write''. Thus, larry has read and write permission to the file stuff.

As mentioned, besides read and write permission, there is also ``execute'' permission--represented by an ``x''. However, a ``-'' is listed here in place of an ``x'', so Larry doesn't have execute permission on this file. This is fine, as the file stuff isn't a program of any kind. Of course, because Larry owns the file, he may grant himself execute permission for the file if he so desires. (This will be covered shortly.)

The next three characters, (``r-''), represent the group's permissions on the file. The group that owns this file is users. Because only an ``r'' appears here, any user who belongs to the group users may read this file.

The last three characters, also (``r-''), represent the permissions granted to every other user on the system (other than the owner of the file and those in the group users). Again, because only an ``r'' is present, other users may read the file, but not write to it or execute it.

Here are some other examples of permissions:
-rwxr-xr-x The owner of the file may read, write, and execute the file. Users in the file's group and all other users may execute the file

-rw------- The owner of the file may read and write the file. No other user can access the file.

-rwxrwxrwx All users may read, write and execute the file.

8. Permissions Dependencies.

The permissions granted to a file also depend on the permissions of the directory in which the file is located. For example, even if a file is set to -rwxrwxrwx, other users cannot access the file unless they have read and execute access to the directory in which the file is located. For example, if Larry wanted to restrict access to all of his files, he could set the permissions to his home directory /home/larry to -rwx---. In this way, no other user has access to his directory, and all files and directories within it. Larry doesn't need to worry about the individual permissions on each of his files.

In other words, to access a file at all, you must have execute access to all directories along the file's pathname, and read (or execute) access to the file itself.

Typically, users on a Linux system are very open with their files. The usual set of permissions given to files is -rw-r-r-, which lets other users read the file but not change it in any way. The usual set of permissions given to directories is -rwxr-xr-x, which lets other users look through your directories, but not create or delete files within them.

However, many users wish to keep other users out of their files. Setting the permissions of a file to -rw---- will prevent any other user from accessing the file. Likewise, setting the permissions of a directory to -rwx--- keeps other users out of the directory in question.

9. Changing permissions.

The command chmod is used to set the permissions on a file. Only the owner of a file may change the permissions on that file. The syntax of chmod is
chmod {a,u,g,o}{+,-}{r,w,x} filenames

Briefly, you supply one or more of all, user, group, or other. Then you specify whether you are adding rights (+) or taking them away (-). Finally, you specify one or more of read, write, and execute. Some examples of legal commands are:

pegasus{jbeams}39: chmod a+r filename
gives all users read access to the file
pegasus{jbeams}40: chmod +r filename
same as above. if aug or o isn't specified, a is assumed
pegasus{jbeams}41: chmod og-x filename
remove execute permissions from people other than the owner
pegasus{jbeams}42: chmod u+rwx filename
let the owner read, write and execute the file

10. Jobs and processes.

Job control is a feature provided by many shells that let you control multiple running commands, or jobs, at once. Every time you run a program, you start what is called a process. The command ps displays a list of currently running processes.

The PID listed in the first column is the process ID, a unique number given to every running process. The last column, COMMAND, is the name of the running command. Here, we're looking only at the processes which Larry himself is currently running. (There are many other processes running on the system as well--``ps -aux'' lists them all.) These are csh (your shell program), and the ps command itself. As you can see, csh is running concurrently with the ps command. csh executed ps when you typed the command. After ps has finished running (after the table of processes is displayed), control is returned to the csh process, which displays the prompt, ready for another command.

A running process is also called a job. The terms process and job are interchangeable. However, a process is usually referred to as a ``job'' when used in conjunction with job control--a feature of the shell that lets you switch between several independent jobs.

In most cases users run only a single job at a time--whatever command they last typed to the shell. However, using job control, you can run several jobs at once, and switch between them as needed.

How might this be useful? Let's say you are editing a text file and want to interrupt your editing and do something else. With job control, you can temporarily suspend the editor, go back to the shell prompt and start to work on something else. When you're done, you can switch back to the editor and be back where you started, as if you didn't leave the editor. There are many other practical uses of job control.

11. Foreground and background.

Jobs can either be in the foreground or in the background. There can only be one job in the foreground at a time. The foreground job is the job with which you interact--it receives input from the keyboard and sends output to your screen, unless, of course, you have redirected input or output, as described above. On the other hand, jobs in the background do not receive input from the terminal--in general, they run along quietly without the need for interaction.

Some jobs take a long time to finish and don't do anything interesting while they are running. Compiling programs is one such job, as is compressing a large file. There's no reason why you should sit around being bored while these jobs complete their tasks; just run them in the background. While jobs run in the background, you are free to run other programs.

Jobs may also be suspended. A suspended job is a job that is temporarily stopped. After you suspend a job, you can tell the job to continue in the foreground or the background as needed. Resuming a suspended job does not change the state of the job in any way--the job continues to run where it left off.

Suspending a job is not equal to interrupting a job. When you interrupt a running process (by pressing the interrupt key, which is usually Ctrl-C), the process is killed, for good. Once the job is killed, there's no hope of resuming it. You'll must run the command again. Also, some programs trap the interrupt, so that pressing Ctrl-C won't immediately kill the job. This is to let the program perform any necessary cleanup operations before exiting. In fact, some programs don't let you kill them with an interrupt at all.

12. Backgrounding and killing jobs.

Let's begin with a simple example. The command yes is a seemingly useless command that sends an endless stream of y's to standard output. (This is actually useful. If you piped the output of yes to another command which asked a series of yes and no questions, the stream of y's would confirm all of the questions.)
The y's will continue ad infinitum. You can kill the process by pressing the interrupt key, which is usually Ctrl-C. So that we don't have to put up with the annoying stream of y's, let's redirect the standard output of yes to /dev/null. /dev/null is a special file in UNIX systems that acts as a ``black hole'' for data. Any data sent to it disappears. This is a very effective method of quieting an otherwise verbose program.

pegasus{jbeams}43: yes > /dev/null

Ah, much better. Nothing is printed, but the shell prompt doesn't come back. This is because yes is still running, and is sending those inane y's to /dev/null. Again, to kill the job, press the interrupt key.

Let's suppose that you want the yes command to continue to run but wanted to get the shell prompt back so that you can work on other things. You can put yes into the background, allowing it to run, without need for interaction.

One way to put a process in the background is to append an ``&'' character to the end of the command.

pegasus{jbeams}44: yes > /dev/null &
[1] 164
pegasus{jbeams}45:

As you can see, the shell prompt has returned. But what is this ``[1] 164''? And is the yes command really running?

The ``[1]'' represents the job number for the yes process. The shell assigns a job number to every running job. Because yes is the one and only job we're running, it is assigned job number 1. The ``164'' is the process ID, or PID, number given by the system to the job. You can use either number to refer to the job, as you'll see later.

You now have the yes process running in the background, continuously sending a stream of y's to /dev/null. To check on the status of this process, use the internal shell command jobs.
Sure enough, there it is. You could also use the ps command as demonstrated above to check on the status of the job.

To terminate the job, use the kill command. This command takes either a job number or a process ID number as an argument. This was job number 1, so using the command

pegasus{jbeams}46: kill %1

kills the job. When identifying the job with the job number, you must prefix the number with a percent (``%'') character.

Now that you've killed the job, use jobs again to check on it:
The job is in fact dead, and if you use the jobs command again nothing should be printed.

You can also kill the job using the process ID (PID) number, displayed along with the job ID when you start the job. In our example, the process ID is 164, so the command

pegasus{jbeams}47: kill 164

is equivalent to

pegasus{jbeams}48: kill %1

You don't need to use the ``%'' when referring to a job by its process ID.

13. Stopping and restarting jobs.

There is another way to put a job into the background. You can start the job normally (in the foreground), stop the job, and then restart it in the background.

First, start the yes process in the foreground, as you did before:
pegasus{jbeams}49: yes > /dev/null
Again, because yes is running in the foreground, you shouldn't get the shell prompt back.

Now, rather than interrupt the job with Ctrl-C, suspend the job. Suspending a job doesn't kill it: it only temporarily stops the job until you restart it. To do this, press the suspend key, which is usually Ctrl-Z.

While the job is suspended, it's simply not running. No CPU time is used for the job. However, you can restart the job, which causes the job to run again as if nothing ever happened. It will continue to run where it left off.

To restart the job in the foreground, use the fg command (for ``foreground'').

The shell displays the name of the command again so you're aware of which job you just put into the foreground. Stop the job again with Ctrl-Z. This time, use the bg command to put the job into the background. This causes the command to run just as if you started the command with ``&'' as in the last section.

And you have your prompt back. Jobs should report that yes is indeed running, and you can kill the job with kill as we did before.

How can you stop the job again? Using Ctrl-Z won't work, because the job is in the background. The answer is to put the job in the foreground with fg, and then stop it. As it turns out, you can use fg on either stopped jobs or jobs in the background.

There is a big difference between a job in the background and a job that is stopped. A stopped job is not running--it's not using any CPU time, and it's not doing any work (the job still occupies system memory, although it may have been swapped out to disk). A job in the background is running and using memory, as well as completing some task while you do other work.

However, a job in the background may try to display text on your terminal, which can be annoying if you're trying to work on something else. For example, if you used the command

pegasus{jbeams}50: yes &

without redirecting stdout to /dev/null, a stream of y's would be displayed on your screen, without any way for you to interrupt it. (You can't use Ctrl-C to interrupt jobs in the background.) In order to stop the endless y's, use the fg command to bring the job to the foreground, and then use Ctrl-C to kill it.

Another note. The fg and bg commands normally affect the job that was last stopped (indicated by a ``+'' next to the job number when you use the jobs command). If you are running multiple jobs at once, you can put jobs in the foreground or background by giving the job ID as an argument to fg or bg, as in

pegasus{jbeams}51: fg %2

(to put job number 2 into the foreground), or

pegasus{jbeams}52: bg %3
(to put job number 3 into the background). You can't use process ID numbers with fg or bg.

Furthermore, using the job number alone, as in
pegasus{jbeams}53: %2
is equivalent to
pegasus{jbeams}53: fg %2

14. Scripts

Let's say that you use a series of commands often and would like to save time by grouping all of them together into a single command. For example, the three commands

pegasus{jbeams}54: cat chapter1 chapter2 chapter3 > book
pegasus{jbeams}55: wc -l book
pegasus{jbeams}56: cat book

concatenates the files chapter1, chapter2, and chapter3 and places the result in the file book. The second command displays a count of the number of lines in book, and the third command cat book prints book to the screen.

Rather than type all these commands, you can group them into a shell script. The shell script used to run all these commands might look like this:

#!/bin/csh
# A shell script to create and show the book
cat chapter1 chapter2 chapter3 > book
wc -l book
cat book

Shell scripts are just plain text files; you can create them with an editor such as emacs, vi, or the pico editor discussed earlier.

Let's look at this shell script. The first line:

#!/bin/csh

identifies the file as a shell script and tells the shell how to execute the script. It instructs the shell to pass the script to /bin/csh for execution, where /bin/csh is the shell program itself.

The second line is a comment. Comments begin with the character # and continue to the end of the line. Comments are ignored by the shell--they are commonly used to identify the shell script to the programmer and make the script easier to understand.

The rest of the lines in the script are just commands, as you would type them to the shell directly. In effect, the shell reads each line of the script and runs that line as if you had typed it at the shell prompt.

Permissions are important for shell scripts. If you create a shell script, make sure that you have execute permission on the script in order to run it. When you create text files, the default permissions usually don't include execute permission, and you must set them explicitly. See the discussion of file permissions for details. Briefly, if this script were saved in the file called makebook, you could use the command

pegasus{jbeams}1: chmod u+x makebook

to give yourself execute permission for the shell script makebook.

You can use the command

pegasus{jbeams}57: makebook

to run all the commands in the script.

15. Getting Out of Trouble and Finishing off

In most cases if you get stuck, for example, in a looping program or in a utility that does not seem to be doing anything then pressing <Ctrl C> will get you back to the shell command prompt. If a process is taking terminal input then <Ctrl D> signals an 'end of file'. At the command prompt <Ctrl D> will exit from the shell, and on some systems if this is the login shell then you will be logged out.

If you have become nested in shells invoked within shells, then you will have to exit them one by one before you can logout with exit or the logout command. If Unix declines to log you out (because there are 'stopped jobs' for example), try issuing the logout command a second time.

If you are working remotely from a PC then breaking your connection or re-booting the PC will usually log you out. Do not re-boot or switch off a Unix workstation, except in extremis, but never walk away and leave yourself logged-in on a public workstation either!