class: center, middle # Introduction to UNIX I: Command Line --- # UNIX Command line * The UNIX Shell: command line interface * Navigating Directories and Files * Running applications * Reminder about helpful tutorial: http://korflab.ucdavis.edu/Unix_and_Perl/current.html --- class: left # Shell - Command Line Interface (CLI) .right[] * Interactive interface of UNIX operating system (Linux and OSX are a type of UNIX) * Access to this in the Terminal program on OSX. ** An alternative Terminal program is iTerm2 - https://www.iterm2.com/ * Windows: Use [Cygwin](https://www.cygwin.com/). Windows10 does have a [BASH shell](https://www.windowscentral.com/how-install-bash-shell-command-line-windows-10). --- # How to get to the shell On Mac: Open the Terminal (Spotlight - search for Terminal) .center[] --- # Anatomy of a command line * The Shell is the commandline program. There are several types, * `sh` - Bourne shell * `bash` - __Bourne again shell__ will use in class * `csh` - C shell * `zsh` - Z-shell ```shell /Users/me $ ls /Users/me $ date /Users/me $ ping www.ucr.edu /Users/me $ echo "hello there" /Users/me $ python test.py /Users/me $ python cmdline.py -i input -o out -d db ``` --- # Using CLI * Programs are run from the shell to interact with the filesystem, naviagate folders, read files * launch programs like file editors, run tools * cmd line programs can take arguments * some arguments are named or some are just in particluar order ```shell $ date Fri Sep 21 14:21:12 PDT 2018 $ date --help Usage: date [OPTION]... [+FORMAT] or: date [-u|--utc|--universal] [MMDDhhmm[[CC]YY][.ss]] Display the current time in the given FORMAT, or set the system date. ``` --- # Filename Reminders .right[] * Case MaTTerS * File Extensions (e.g. .txt .doc .docx) don't matter * Avoid spaces in filenames * Files starting with '.' are hidden by default * Backup files created by some editors will be appended with ~ --- # Directories and listing 1. ls - shows a listing of the files and folders ```shell $ ls $ ls -l ``` 3. ls -d will display a directory name only 4. ls -F will give a different symbol for directories 5. ls -l shows long listing 6. ls -lt shows long listing, ordered by newest to oldest 7. ls -a shows hidden files 7. How to know all the options? --- # Use `man` to find out more about a program ```shell $ man ls NAME ls -- list directory contents SYNOPSIS ls [-ABCFGHLOPRSTUW@abcdefghiklmnopqrstuwx1] [file ...] DESCRIPTION For each operand that names a file of a type other than directory, ls displays its name as well as any requested, associated information. ``` --- # cmdline help * Many programs also have a help built in * use -h or --help * Sometimes can run the program without any arguments and will show help ```shell $ sort --help sort --help Usage: sort [OPTION]... [FILE]... or: sort [OPTION]... --files0-from=F Write sorted concatenation of all FILE(s) to standard output. Mandatory arguments to long options are mandatory for short options too. Ordering options: ``` --- # Navigating paths 1. to go into a directory use cd 2. to go back a level use 'cd ..' Can specify a total path 'cd /tmp' 3. To go back to your home dir, type cd 5. Try to make a dir, go into it, make another directory. 4. On the Mac you can see the same directories and files organized graphically too 5. SWCarpentry has a nice intro with graphics [here](https://swcarpentry.github.io/shell-novice/02-filedir/)  --- # Directories: Creating and Deleting 1. mkdir - make directory ```shell $ mkdir MyDir $ mkdir MyDir # will give warning $ mkdir MyDir/UnderDir/DeepDir # will not work, UnderDir not created $ mkdir -p MyDir/UnderDir/DeepDir # -p option will create intermediate one ``` 2. rmdir - remove directory ```shell $ rmdir MyDir # won't let you unless it is empty $ rmdir MyDir/UnderDir/DeepDir ``` --- # Files 1. The whole point of all this is probably to create files with data 2. can make an empty file with ```shell $ touch file # will create an empty file ``` 3. Running programs can generate files 4. We will discuss output later but you can direct output of a program to create a file. ```shell $ echo "hello world" > hello.txt $ echo "goodbye cruel world" > goodbye.txt ``` --- # Permissions 1. `ls -l` will show a long version of listing output ```plain jstajich@pigeon:/bigdata/gen220/shared/data_files$ ls -l total 0 lrwxrwxrwx 1 jstajich gen220 25 Sep 25 13:53 Nc20H.expr.tab -> expression/Nc20H.expr.tab -rw-rw-r-- 1 jstajich gen220 609183 Sep 25 13:52 Spellman.csv drwxrwxr-x 2 jstajich gen220 4096 Jun 10 00:39 expression drwxr-xr-x 6 jstajich gen220 4096 Jun 10 00:39 sequences ``` 2. There is info on the file/folder and 3 sets of permissions listed there * 'd' -> It is a directory, 'l' -> it is a link, empty, it is a file * User -> rwx or rw- permissions for the user * Group * All 3. what are permissions * r - is it readable * w - is it writeable * x - is it executable 4. Directories have to be executable to be able to be used/entered. Programs/applications need to be executable to be able to run. --- # Deleting things (careful!) 1. `rm` - for removing files 2. `rm -r` -- _be careful_ removes recursively 3. `rm -f` - removes without prompting you 4. `rm -rf` - removes recurisvely without prompty __Use with care__ --- # File content Want to read the content of a text file you can use these commands 1. `cat` - will spit out the whole file on the screen 2. `more` - a paginator - will display one page at a time (based on your screen) 3. `less` - similar to more, but has additional options --- # Editors 1. On Biocluster `nano`. On other systems it is `pico` * Will be one of the easiest to try out on the command line * Follow commands at bottom of screen to figure out how it works 2. `emacs` [tutorial](https://www.masteringemacs.org/article/beginners-guide-to-emacs) [gnuguide](https://www.gnu.org/software/emacs/tour/) 3. `vi` [tutorial](https://www.cs.colostate.edu/helpdocs/vi.html) [vim book](ftp://ftp.vim.org/pub/vim/doc/book/vimbook-OPL.pdf) 4. [Atom](https://atom.io/), [TextMate](https://macromates.com/) on OSX 5. [Notepad++](https://notepad-plus-plus.org/) 5. Syntax highlighting useful when doing programming. We will do more on this later. --- # Practice with editors 1. Open an editor 2. Write a file with three lines that list the names of 3 book titles 3. Save the file --- # Compressed and uncompressed * Data files can be compressed to save disk space * typical compression is `zip`, `gzip`, and `bzip2` ```shell $ ls -l 1008 Sep 28 14:29 file.txt $ gzip file.txt $ ls -l 386 Sep 28 14:29 file.txt.gz $ gunzip file.txt.gz $ ls -l 1008 Sep 28 14:29 file.txt ``` * Can substitute `bzip2` and `bunzip2` * On some systems `pigz` and `pbzip2` are parallelized compressors that can use multiple processors (go faster!) --- # Downloading data from web Cmdline tools `curl` and `wget` can download data from the web/ftp * use `-O` to tell curl to save the data to a file, named by the file link ```shell $ curl -O http://www-personal.umich.edu/~jlawler/wordlist ``` ```shell $ curl -o ucr_index.html https://www.ucr.edu # OR $ curl https://www.ucr.edu > ucr_index_again.html ``` --- # Other resources 1. [Introduction to UNIX Shell - Software Carpentry](https://swcarpentry.github.io/shell-novice/)