Introduction to the command-line

Introduction to the command-line

Author

Affiliation

Stephanie Hicks

Department of Biostatistics, Johns Hopkins

Announcements

The next lecture falls on 👻 Halloween ! I plan to give the lecture dressed up in a costume. This is entirely optional, but I encourage students to come in costume if you wish! Candy 🍬 will be offered to anyone in costume!

Pre-lecture activities

Important

In advance of class, please follow the instructions on Software Carpentry: The Unix Shell to

1. Install a Unix shell on your computer (if it is not already installed)
2. How to access and open the Unix shell on your computer
3. How to type commands on the Unix shell

In addition, please read through Chapters 1-7

• https://swcarpentry.github.io/shell-novice/01-intro
• https://swcarpentry.github.io/shell-novice/02-filedir
• https://swcarpentry.github.io/shell-novice/03-create
• https://swcarpentry.github.io/shell-novice/04-pipefilter
• https://swcarpentry.github.io/shell-novice/05-loop
• https://swcarpentry.github.io/shell-novice/06-script
• https://swcarpentry.github.io/shell-novice/07-find

How much should I prepare for before class?

You should be comfortable with the meaning of most the commands and also executing commands in a Unix shell on your computer before class starts.

During class, I will give an overview of working in the Unix shell and then we will practice the commands with an in-class activity. If you have not opened a Unix shell before or you are not comfortable with executing commands, it will be challenging to participate in the activity.

Lecture

Acknowledgements

Material for this lecture was borrowed and adopted from

• Software Carpentry: The Unix Shell
• Data Science at the Command line
• Applied Computational Genomics

Learning objectives

Learning objectives

At the end of this lesson you will:

• Understand what is a command shell and why would use one.
• Create, copy, move, rename, and delete files and folders.
• Search for regular expressions in files.
• Execute R commands and scripts in the command line.
• Redirect a command’s output to a file with redirect operators (>, >>).
• Construct command pipelines with two or more stages with the pipe operator (|).
• Write a loop that applies one or more commands separately to each file in a set of files.

Tip

You can practice your command-line skills with the Command Challenge

Slides

• Lecture 01: Introduction to the command-line

Class activity

Objectives of the activity

For our in-class activity today, we will

• Practice writing commands in the Unix shell
• Enjoy a little fun with Taylor Swift lyrics

Find a partner and work on the following tasks. It’s unlikely you will finish all of them, but try to do as many as you can! Also, try to work out the solution for yourselves first and then check the solution afterwards.

• Create Taylor Swift’s Albums Directory. Navigate to your home directory. Create a directory called taylor_swift_discography. Inside taylor_swift_discography, create subdirectories for each of her albums: fearless, red, 1989, reputation, and lover.

Solution

Expected commands: cd, mkdir

• List all the album directories in taylor_swift_discography in a long format, showing permissions and modification dates.

Solution

ls -l

• Navigate to the red directory and create a file called all_too_well.txt. Add the first few lines of the song “All Too Well” to the file using a single Unix command.

Solution

A combo of touch, echo, cat

• Display the contents of all_too_well.txt without opening the file in a text editor.

Solution

Expected commands: cat, more, less

• Navigate to the 1989 directory. Create a file called blank_space.txt and add some of the lyrics. Count how many words are in the song lyrics using a Unix command.

Solution

Expected commands: wc

• In the reputation directory, create a file called delicate.txt and add some of the lyrics. Use a command to find all occurrences of the word “delicate” in the file.

Solution

Expected commands: grep

• Move the file all_too_well.txt from the red directory to a new directory called favorites within taylor_swift_discography. Copy blank_space.txt from 1989 to the favorites directory.

Solution

Expected commands: mv, cp

• Navigate to the favorites directory and compress both all_too_well.txt and blank_space.txt into a file called favorite_lyrics.tar.gz.

Solution

Expected commands: tar, gzip

• Uncompress the favorite_lyrics.tar.gz file and verify that the files have been restored correctly.

Solution

Expected commands: tar, gunzip

• In the taylor_swift_discography directory, find all the lines (representing song lyrics) that contain the word “love”. Output the results to a file called love_songs.txt in the favorites directory.

Solution

grep -r "love" >> favorites/love_songs.txt

• For each song in favorites, write a for loop to print to the screen the first two lines of each song in the files all_too_well.txt and blank_space.txt.

Solution

for in do done

• Create a shell script called top_favorites.sh and place the for loop in the shell script. Run the shell script.

Solution

Expected comments: touch

• In the taylor_swift_discography directory, create a file called song_metadata.csv that contains columns for album name, song title, duration (in minutes), and year with the following information.

Album,Song Title,Duration,Year
Fearless,Love Story,3.55,2008
Red,All Too Well,5.29,2012
1989,Blank Space,3.51,2014
Reputation,Delicate,3.52,2017
Lover,You Need to Calm Down,2.51,2019
Folklore,Cardigan,4.00,2020
Evermore,Willow,3.34,2020
Midnights,Anti-Hero,3.21,2022

Use awk to extract and display only the song titles and durations from this file.

Solution

awk -F, '{print $2, $3}' song_metadata.csv

Post-lecture

Summary

• Shell is a text based application for viewing, handling and manipulating files
• It is also known by the following names
  • CLI (Command Line Interface)
  • Terminal
  • Bash (Bourne Again Shell)
• Use Rscript -e or R -e to execute R scripts from the command line
• RStudio includes a Terminal (from version 1.1.383)
• Execute commands from shell script in RStudio using Ctrl + Enter
• RMarkdown and Quarto supports bash, sh and awk

Additional practice

Here are some additional practice questions to help you think about the material discussed.

Questions

1. Move around the computer, get used to moving in and out of directories, see how different file types appear in the Unix shell. Be sure to use the pwd and cd commands, and the different flags for the ls commands.
2. Practice using “Tab for Auto-complete” in the shell to autocomplete commands or file names.
3. Explore the manual pages of date in the command line to show you what that looks like. Try to figure out what is the argument to print the date since the Unix epoch or 00:00:00 UTC on 1 January 1970 as a function of the number of seconds. Then try to identify what is the argument to display the date in UTC.
4. Practice your command line knowledge with Command Challenge.