Linux Commands
Getting Help
The Fabian command line is a Linux environment, which means Unix commands are supported. For example, to change to a directory/folder you would type cd foldername.
As we go through this documentation, we will briefly explain various commands and try to give you just enough information so that you can be comfortable using it. Much detail about each command will be omitted, such as various options and full descriptions of possible uses but it is easy to quickly find out more about a command you wish to use.
Manuals
man
The manual pages tell you which options a particular command can take, and how each option modifies the behaviour of the command. For instance, to learn about the ls command, type:
man ls
this will give you full documentation of the ls command. You can leave the manual page by simply typing q.
whatis
If, however, you only want a brief description of the command you can type:
whatis ls
apropos
If you are not sure of the exact name of a command, but have an idea of what you are looking for, you can type:
apropos keyword
This will list all the commands that have the keyword in their manual-page header.
For example, type:
apropos list
Again, type q to exit.
Exploring Files and Directories
Listing Files and Directories
ls
The ls command (short for list) lists the contents of the current working directory. To find out what is in your current directory, type:
ls
You can also provide the ls command with a path to some directory, and it will list the files in that directory. You can see this by typing:
ls path/to/dir/
By default the ls command only lists the names of folders and directories. It will also only list files which are not prefixed with a dot (.), aka hidden files. Getting more information from the ls command requires the usage of flags; some common ones are given below.
Common Flags
| Syntax | Description |
|---|---|
| -F | distinguishes directories |
| -a | shows hidden files/directories (written as .file or .dir) |
| -l | shows detailed information about files |
| -lt | Same as -l, except sorted by date/time |
| -lth | same as -lt, but with human-readable times |
Making Directories
mkdir
The mkdir command (short for make directory) is used to create subdirectories in your filesystem. This is useful for organizing your workflow and keeping files separated. To make a new subdirectory called learn_unix in your current directory, type:
mkdir learn_unix
You can check to see the directory has been created by using the ls command.
Changing Directories
cd
The cd command (short for change directory) changes the current working directory to the directory specified in the command. To change the current working directory to the one you just created (learn_unix), type:
cd learn_unix
If you now type ls, you will see that this directory is empty.
The Directories (.) and (..)
If you have not yet tried using the -a flag with the ls command, you can do so now in the learn_unix directory, type:
ls -a
You can see that even though the directory is supposed to be empty, there are two special directories there: (.) and (..).
The current directory (.)
In Unix, the special directory (.) is short for the current directory, so when you type:
cd .
You don't move anywhere in the file system. While this might not be useful for changing directories, having a shorthand for the current directory will be useful in many other commands.
The parent directory (..)
The other special directory (..) we see when running ls -a is shorthand for the parent or the current directory. Typing:
cd ..
will move you to one directory up in the file system's hierarchy. If you typed this from the learn_unix directory, this will take you to the home directory.
Path Names
pwd
The pwd command (short for print working directory) displays the full path of your current working directory. This is useful for knowing exactly where you are in your file system. This command will be particularly useful when transferring files to and from the cluster, as those operations will require the full path of the file or directory to be transferred. To get the full path of your home directory, type cd if you are not yet in your home directory, then type:
pwd
The path printed by this command is your full path.
~ (home directory)
You can reference your home directory anywhere in the file system by using the tilde (~) shorthand. If you move to a directory deep in the hierarchy cd path/deep/in/the/hierarchy and you want to list the contents of your home directory, you can type:
ls ~
Or you can also change to a subdirectory of the home directory using the cd command e.g.:
cd ~/learn_unix
Manipulating Files and Directories
Creating Empty Files
touch
The touch command is used to update timestamps on files. Every file in the Unix system has timestamp information for each file or folder, including access time, modify time and change time. It is possible to modify timestamps using the touch command, either to update a timestamp to the current time or to modify it to a date in the past.
If the specified file does not exist, the touch command will create that file and set the access and modification time for the file to the current time. To create a file, type:
touch file1
Copying Files
cp
The cp command (short for copy) makes a copy of a file. If we had a file called file1 in the current directory and we wanted to create a copy of this file in the same directory called file2, we could achieve this by typing:
cp file1 file2
If instead we had some file in our home directory and we wanted to copy it to our learn_unix directory while keeping the same name, we could do this in one of two ways depending on where we are in the file system. If we are in the home directory, type:
cp file1 learn_unix/
If we are currently in the learn_unix directory, we would type:
cp ../file1 .
or
cp ~/file1 .
Notice the difference in these two commands. In the former, we use the (..) shorthand to specify that file1 is located one folder up in the hierarchy. In the latter, we use the (~) for the home directory, which allows us to access file1 regardless of where we are in the file system as long as file1 is in the home directory.
Also note that in both instances we use the (.) shorthand to specify that we are copying the file into our current directory.
Moving Files
mv
The mv command (short for move) moves a file from one place to another. This differs from the cp command in that in the end there is still only one file. If we wanted to move file1 to file2 we could type:
mv file1 file2
This effectively renames the file from file1 to file2.
Alternatively, if file1 is in the home directory and we want to move it to the learn_unix directory, we would type:
mv file1 learn_unix/
(That is provided we were working from the home directory.)
Removing Files and Directories
rm
The rm command (short for remove) is used to delete a file or directory. If we have a file called file1 and we wanted to delete it, we would type:
rm file1
The rm command also works on directories, however the command differs a little when deleting an empty directory vs a non-empty directory. To remove an empty directory, dir1, type:
rm dir1
If dir1 is not empty, we would need to use the -r flag. The -r flag stands for recursive, and it means "delete this directory and everything in it". To delete the non-empty directory dir1, type:
rm -r dir1
Removing files in this way will PERMANENTLY DELETE them!
Command line Linux has no concept of a "recycle bin" so files should only be removed if you are certain you don’t need them any more, or if they can be easily regenerated
Viewing Files in the Terminal
There are a handful of commands that let you view a file, or portions of a file without having to open an editor. Before we get into those, it's useful to start with a cleared terminal window. To do that, we can use the command clear:
clear
The clear command will clear all the text in the terminal window, leaving you with a fresh view. To clear your window, type:
clear
Once the window is clear, we can start viewing files.
cat
The cat command (short for concatenate) allows you to view contents of a file called file1 on the screen. Type:
cat file1
You can also view multiple files by listing them after cat.
If you run the cat command without specifying a file to read, it reads the standard input (from the keyboard), and on receiving end-of-file (^d), copies it to the standard output (the screen). We will use this later when talking about redirection.
less
The less command writes a file to the screen one page at a time. Type:
less file1
Once the file is displayed, you can use the keyboard to navigate.
| Key | Navigation |
|---|---|
| space-bar | view next page |
| b | go back one page |
| enter | scroll down one line |
| = | displays information such as line numbers and percentage read |
| /word | searches for the word "word" following the backslash |
| n | after searching for a word, n will search for the next occurrence |
| q | quit reading |
head
The command head will display the first few lines of a file. Type:
head file1
By default, the first 10 lines of file1 are shown, but you can specify the number of lines to show by using the -n flag followed by the number.
tail
The command tail works in the same way as the head command except that it displays the last few lines of a file. Type:
tail file1
The next two commands are useful when comparing two files.
diff
The diff command (short for difference) compares the contents of two files and displays the differences. To find the difference between file1 and file2 you would type:
diff file1 file2
In the output, lines from file1 will be denoted by < and lines from file2 will be denoted by >.
paste
The paste command displays the contents of two or more files next to each other line by line. To view file1 and file2, type:
paste file1 file2
Searching Files
We've already seen how we can search for specific words in a file by first viewing the file with the less command, then typing / followed by the word we wish to find. What if we don't want to use less, or if what we're searching is not a file but the output of another command? There is a more general way to search in the terminal, and that is using the grep command.
grep
The grep command (short for global regular expression print) is one of many standard Unix utilities. It searches for specified words or patterns and displays the lines containing them to the screen. To search for the word "string" in the file file1, you would type:
grep string file1
One thing to keep in mind is that grep is case sensitive, meaning that it won't search for lines containing the word "String". You can also search for phrases and patterns by enclosing them in single quotes. To search for "this is a string", you would type:
grep 'this is a string' file1
You can gain more control over the grep command by using flags.
Common Flags
| Syntax | Description |
|---|---|
| -i | ignore case |
| -c | displays # of times the string is found |
| -v | displays all lines except those containing the string |
| -n | include line numbers where string is found |
| -B N | print N extra lines before a match |
| -A N | print N extra lines after a match |
wc
The wc command (short for word count) counts the lines, words, and characters in a file. For a file named file1, type:
wc file1
Redirection
Redirecting Output
We mentioned earlier that when using cat without specifying a file, the command reads the standard input and then displays it on the screen. However, we can actually redirect the cat command so that instead of displaying to the screen, it will write to a file. We can do this by using the > symbol.
Redirecting to a new file
To see this in action, we will redirect the output of the cat command to a file called list1, type:
cat > list1
Then type some words. Press enter after each one:
economics
political science
^d
Note that ^d means press both Ctrl and d at the same time. Each time we press enter, the word we typed gets written to the file list1.
You can see this by now typing:
cat list1
Appending a File
We can append an already existing file by using >>. Let's try appending list1: type:
cat >> list1
Then type some words. Press enter after each one:
module
server
application
^d
You can confirm the file has been appended by typing:
cat list1
Concatenating Files
Redirecting with > and >> is not only useful for writing to a file from the standard input, you can also take files as an input. First let's create a new file list2 by following the procedure above to create a new list.
Now we can concatenate list1 and list2 and redirect the output to a new file called biglist by typing:
cat list1 list2 > biglist
Confirm it worked by typing:
cat biglist
Redirecting Input
We can redirect the input of a command using the < symbol. To demonstrate this lets introduce a new command called sort.
sort
The sort command takes an input and sorts it, then prints it to the screen. Try it out by typing:
sort
Then type some words. Press enter after each one:
processor
gpu
compute
memory
^d
The output will be the list you just entered, sorted alphabetically.
Now we can try redirecting an input into the sort command by using <. Type:
sort < biglist
The sorted content of biglist should be printed to the screen.
We can do all sorts of chaining with the redirection symbols, let's create a new file slist that contains the sorted content of biglist, type:
sort < biglist > slist
Use cat to confirm.
Pipes
As we described earlier, one way to input a list of items and get a sorted list from them is to first redirect the output of the cat command to a file list1, then use that file as an input to the sort command.
This is a very roundabout way of doing things, and you can be left with random temporary files which will need to be removed. Ideally what we would like is to route the output of one command into the input of another command. This is accomplished with a pipe. The symbol for a pipe is a vertical bar |.
Try typing:
cat | sort
Now enter a list of items as you have done before:
processor
gpu
compute
memory
^d
You will see that once you press ^d the sorted list will be printed to the screen.
Pipes are a very useful tool for chaining various commands.
Processes and Jobs
A process is an executing program identified by a unique PID (process identifier).
ps
The ps command (short for process) is used to see information about your processes with their associated PID and status. To see the current running processes, type:
ps
A process may be in the foreground, in the background, or be suspended. In general the shell does not return the Unix prompt until the current process has finished executing.
Background Processes
Some processes take a long time to run and hold up the terminal. Backgrounding a long process has the effect that the Unix prompt is returned immediately, and other tasks can be carried out while the original process continues executing.
For example, typing:
sleep 10
will cause the terminal to wait 10 seconds before returning the prompt to the user. Nothing else can be done until the process has completed.
If you want to run processes that take time but retain use of the terminal window, you can run the processes in the background by typing & after the command, try typing:
sleep 10 &
After typing a command to be run in the background, the terminal will output the job number and the PID of the process, looking something like this:
[1] 12345
The job number is the number enclosed in brackets, and the PID is the number that follows. If you type ps while the process is still running you should now see the PID and command listed in the output.
Backgrounding a Running Process
What if you can't anticipate how long a process will run for, or if you decide after the fact that you need to run other processes? Don't worry, because it's possible to instruct a currently running process to run in the background instead.
To move a currently running process to the background, first suspend the process by typing ^z (hold down Ctrl and type z.
Let's start a long process by typing:
sleep 1000
Now suspend it with ^z. Once its suspended you can type:
bg
to move it to the background.
Listing Suspended and Background Processes
We saw one way of listing the current running processes, the ps command. The ps command can give you information about all the processes running on the system, including the PID. However, if you only want the processes you started in your current session, you can use the jobs command.
Try running a command in the background and then type:
jobs
You will get a list of all the running job with an output that looks like
[#] [state] [job]
[#] will be the job number, [state] will be the state of the job, either "Running" or "Suspended", and [job] will be the job that's running, ie "sleep 1000".
Move a Process to the Foreground
If you would like to move a background process to the foreground, you can to do so using the fg command along with the job number. Type:
fg $JOBNUMBER
For example, to move a job with job number of 1 to the foreground, type:
fg 1
If no jobnumber is given then the last suspended process will be foregrounded.
Killing Processes
There are many reasons why you might want to stop a process from running. To kill a job running in the foreground, type:
sleep 100
then Ctrl and c:
^c
If you want to kill a background process you have two choices, both involving the kill command. The first way is to specify the job number of the process you want to kill:
kill $JOBNUMBER
Alternatively, you can kill a process by specifying its PID:
kill PID
If a process is refusing to be killed, you can use the -9 option to force it:
kill -9 PID
Other Helpful Commands
alias
The alias command creates a temporary/easier/shorter name for a command or series of commands.
alias name="command"
chmod
chmod 0755 file1
Changes the write/read/execute permissions for a file so that anyone can execute it and you can edit/write to it
df
The df (stands for disk free) command displays the space left on the file system.
df .
Displays disk usage.
du
The du command displays the number of kilobytes used by each subdirectory.
du
You can specify a directory to be searched by typing it after the du command. For example, to summarise (-s) the space used, in human-readable units (-h), for each subdirectory of a directory dir1 and display a total (-c), type:
du -sch dir1/*
echo
The echo command prints the string following the command to the terminal. Try typing:
echo string
gzip
The gzip command is used to reduce the size of a file by compressing it. To reduce a file called file1 type:
gzip file1
The file is compressed and placed into a file called file1.gz.
To expand the compressed file you can use the gunzip command. To expand file1.gz type:
gunzip file1.gz
zcat
Once you've compressed a file, theres not much you can with it unless you expand it again. However, the zcat command allows you to read a compressed file. You can read file1.gz by typing:
zcat file1.gz
Just like the cat command, you can pipe the output of zcat into another command which makes it easier to manage. You can type:
zcat file1.gz | less
find
The find command searches through your directories for files and directories with a given name, date, size, or other attributes you can specify. For instance if you wanted to search for all files with the .txt extension starting at the current directory and print them to the screen, you can type:
find . -name "*.txt" -print
The wildcard symbol * is a way to include anything preceding .txt.
Finding files based on their size can be done by typing:
find . -size +1M -ls
This will display any file over 1Mb in size.
history
The commands you type into the command prompt are all stored in the shell. You can view the history of the commands you entered by using the history command. Type:
history
You can also use the ! key to recall previously used command; for instance, typing:
!!
will repeat the most recent command typed. You can also specify a command in the list in a few ways. To repeat the 3rd command in history, you can type:
!3
If you wanted to repeat the 5th most recent command type:
!-5
You can even repeat the last command starting with some command name. For instance if you wanted to repeat the last command starting with grep, type:
!grep
Editors
nano
nano
Opens a new nano editor, will save a new file.
nano file1
Opens file1 in the nano editor.
emacs
emacs
Opens a new emacs editor, will save a new file.
emacs file1
Opens file1 in the emacs editor.
vim
vim
Opens a new vim editor, will save a new file.
vim file1
Opens file1 in the vim editor.
Credit
This page is based on https://github.com/BinghamtonUniversity/SpiedieDocs
Unix Guides
Here are some additional links to help you get started with basic Unix commands:
-
University of Surrey has a beginners guide to Unix
-
Another list of Unix commands. This is particularly helpful if you have inherited a script and are wondering what the commands do.
-
A useful guide from the Berlin Institute of Medical System Biology (BIMSB).