Command Line & Scripts¶

command line & shell commands
file paths (absolute and relative)
scripts and modules

Jupyter Notebooks are a helpful tool, but they’re bulky.

File Systems¶

Computers use a hierarchical file systems that organizes files & folders.

When you click through the folders (directories) on your computer, you’re interacting with this hierarchical system.

Command Line¶

A command line interface is a way to interact with a computer through written commands.

The command line allows us to:

create files
edit files
run python scripts
etc.

…without clicking on anything.

The Terminal¶

The terminal is where you can type these commands into the command line.

Accessing the terminal…

possible on your computer
and on datahub

Shell Commands¶

…can be run in the terminal and Jupyter notebooks

Do this by starting with !

Check current directory¶

# print working directory
!pwd

/Users/shannonellis/Desktop/Teaching/COGS18/Materials

An important aside: File Paths¶

The specific location of a file or folder on your computer.

When using a Graphical User Interface (GUI), you click on directories to access subdirectories and finally find the file you’re interested in.

When using the command line, you specify a file’s path explicitly with text.

Absolute vs. Relative Paths¶

The two ways to specify the path to your file of interest allow for flexibility in programming.

Absolute Paths¶

Absolute paths specify the full path for a given file system (starting from the root directory).

root specifies the ‘highest’ directory in the file structure (the start).

An absolute file path starts with a slash / specifying the root directory.

## absolute path
## this is specific to my computer
## look at the path output above for you computer
!ls /Users/shannonellis/Desktop/Teaching/COGS18/Materials

00-Introduction.ipynb          24-WrapUp.ipynb
01-Tools.ipynb                 A1-Syntax.ipynb
02-JupyterNotebooks.ipynb      A2-Examples.ipynb
03-Variables.ipynb             Check-In.ipynb
04-Operators.ipynb             E1-Answers.ipynb
05-Conditionals.ipynb          E1_Sp20_Answers.ipynb
06-DataTypes.ipynb             E1_Sp20_Practice_Answers.ipynb
07-Loops.ipynb                 E2-Answers.ipynb
08-Encodings.ipynb             Exam1-Practice.ipynb
09-FunctionsI.ipynb            Exam1-Review-Answers.ipynb
10-FunctionsII.ipynb           Exam1-Review-old.ipynb
11-Debugging.ipynb             Exam1-Review.ipynb
12-Algorithms.ipynb            Exam2-Practice.ipynb
13-Objects.ipynb               Exam2-Review-Answers.ipynb
14-Classes.ipynb               Exam2-Review.ipynb
15-Namespaces.ipynb            LICENSE.txt
16-CommandLine.ipynb           Midterm.ipynb
17-APIs.ipynb                  README.md
18-ScientificComputing.ipynb   Untitled.ipynb
19-Documentation.ipynb         XX-Extra-Clicker-Qs.ipynb
19-OpenSource.ipynb            [34m__pycache__[m[m
20-CodeStyle.ipynb             [34mimg[m[m
21-CodeTesting.ipynb           [34mmy_module[m[m
22-CodeProjects.ipynb          remote.py
23-AdvancedPython.ipynb        stw.py

Relative Paths¶

Relative paths specify the path to a file from your current working directory (where your computer is working right now).

# remind us of our current working directory
!pwd

/Users/shannonellis/Desktop/Teaching/COGS18/Materials

# relative path
# this is specific to my computer
!ls ../../COGS108/Lectures-Sp19

01_intro_ethics.pdf               15_machine_learning.pdf
02_version_control.pdf            16_ml_examples.pdf
03_data.pdf                       17_machine_learning.ipynb
[34m04_data_wrangling[m[m                 18_geospatial.pdf
04_data_wrangling.pdf             19_geospatial.ipynb
04_data_wrangling.zip             20_Nonparametric.pdf
05_dataviz.pdf                    21_dimensionality_reduction.pdf
[34m06_datavizII[m[m                      22_dimensionality_reduction.ipynb
07_analysis.pdf                   23_future.pdf
[34m08_descriptive[m[m                    EDA_CaseStudy_12PM.pdf
09_EDA.ipynb                      EDA_CaseStudy_2PM.pdf
10_inference.pdf                  Inference_CaseStudy_12PM.pdf
11_regression.ipynb               Inference_CaseStudy_2PM.pdf
12_correlation.ipynb              README.md
13_binary.ipynb                   XX_Guest_Merchant.pdf
13_text.pdf                       [34mimg[m[m
14_text.ipynb

.. specify you want to move one directory up in your hierarchy
COGS108/Lectures-Sp19 specifies the path to the directory I want to list files in
each directory is separated with a slash (/)

This relative path does not start with a leading slash (b/c it’s not an absolute path).

Clicker Question #1¶

Given the following file structure:

/
- scripts/
  - cool_thing.py
  - super_cool_thing.py
- images/
  - image1.png
  - image2.png
- notebooks/
  - 00_intro.ipynb
  - 01_variables.ipynb

If your current working directory is notebooks, what is the absolute path to cool_thing.py?

A) /scripts/cool_thing.py
B) scripts/cool_thing.py
C) scripts/cool_thing.py
D) ../scripts/cool_thing.py
E) ¯\_(ツ)_/¯

Clicker Question #2¶

Given the same file structure:

/
- scripts/
  - cool_thing.py
  - super_cool_thing.py
- images/
  - image1.png
  - image2.png
- notebooks/
  - 00_intro.ipynb
  - 01_variables.ipynb

If your current working directory is notebooks, what is the relative path to cool_thing.py?

A) /notebooks/../scripts/cool_thing.py
B) scripts/cool_thing.py
C) /scripts/cool_thing.py
D) ../scripts/cool_thing.py
E) ¯\_(ツ)_/¯

Shell Commands¶

…can be run in the terminal and Jupyter notebooks

Check current directory¶

# print working directory
!pwd

/Users/shannonellis/Desktop/Teaching/COGS18/Materials

Change directory¶

# change directory 
!cd ~/Desktop 

Here, we saw ~/Desktop/.

~ specifies the user’s home directory of your computer
Desktop is a directory in my home directory (~)
each directory is separated with a slash (/)

List files in a directory¶

# list files (list segments)
!ls

00-Introduction.ipynb          24-WrapUp.ipynb
01-Tools.ipynb                 A1-Syntax.ipynb
02-JupyterNotebooks.ipynb      A2-Examples.ipynb
03-Variables.ipynb             Check-In.ipynb
04-Operators.ipynb             E1-Answers.ipynb
05-Conditionals.ipynb          E1_Sp20_Answers.ipynb
06-DataTypes.ipynb             E1_Sp20_Practice_Answers.ipynb
07-Loops.ipynb                 E2-Answers.ipynb
08-Encodings.ipynb             Exam1-Practice.ipynb
09-FunctionsI.ipynb            Exam1-Review-Answers.ipynb
10-FunctionsII.ipynb           Exam1-Review-old.ipynb
11-Debugging.ipynb             Exam1-Review.ipynb
12-Algorithms.ipynb            Exam2-Practice.ipynb
13-Objects.ipynb               Exam2-Review-Answers.ipynb
14-Classes.ipynb               Exam2-Review.ipynb
15-Namespaces.ipynb            LICENSE.txt
16-CommandLine.ipynb           Midterm.ipynb
17-APIs.ipynb                  README.md
18-ScientificComputing.ipynb   Untitled.ipynb
19-Documentation.ipynb         XX-Extra-Clicker-Qs.ipynb
19-OpenSource.ipynb            [34m__pycache__[m[m
20-CodeStyle.ipynb             [34mimg[m[m
21-CodeTesting.ipynb           [34mmy_module[m[m
22-CodeProjects.ipynb          remote.py
23-AdvancedPython.ipynb        stw.py

More Shell Commands¶

Make a new directory¶

# make directory 
!mkdir dir_name

Create a file¶

# create an empty file
!touch new_file.py 

Move a file¶

# move file
# notice the relative file path
!mv new_file.py dir_name/

And Some More¶

Print out a message¶

!echo Hello World!

Hello World!

Print the contents of a file¶

!cat dir_name/new_file.py

# +
# add some comment 

Open to see and edit contents of a file¶

!open dir_name/new_file.py

Clicker Question #3¶

Which is the best description of the following command:

ls -l

A) ls -l is most analogous to a class in Python
B) The whole ls -l is like a function call
C) -l is analogous to a function call, and ls is like a parameter
D) ls is analogous to a function, and -l is like a parameter
E) There is no clear analogy here to Python

## test out here

Clicker Question #4¶

Which creates a file?

A) mkdir
B) pwd
C) cd
D) touch
E) cat

Windows Command Prompt¶

Some commands are slightly different if you are using windows command prompt:

dir : lists files in current directory
move : moves a file
copy : copies a file
rename : renames a file
type : can be used to print out a file

Note that pwd, cd, mkdir, echo are all the same in Windows command prompt.

If you want to make a new empty file, you can do:

'' > my_file.py

^This construction puts an empty string into a file. If the filename is not found, it will create a new (empty) file.

Python Files¶

Python files are plain-text files, with Python code in them, that can be executed and/or imported from.

Script vs. Module File¶

Scripts¶

A script is a Python file that can be run to execute a particular task.

Module Files¶

A module file is a file with Python code (typically functions & classes) that we can import and use.

Remember: if you’re writing code, you cannot just click through to the file you want. You need to specify using code where the file you want is.

This is where understanding file paths is critically important.

Text Editors¶

Text-editors are programs made for editing text. Many text-editors are designed for writing code specifically.

Terminal Based Text Editors¶

There are text editors designed to be used within a terminal, such as vim, emacs, or nano.

vim¶

vim is a terminal based text-editor. Type vim filename in command line to open a file in vim.

vim has different modes:

click escape + i to enter edit mode (insert mode).
- This will let you write text / code into the file
To escape vim, press escape then type :wq and enter to save and quit vim
- If you want to exit without saving, you can do :!q to force quit without saving

Non-Terminal Text Editors¶

For writing code (outside of notebooks and the terminal), you probably want a code focused stand-alone text editor, like Sublime.

Executing Python Files¶

From the command line, you can execute a Python script using the python command:

python dir_name/new_file.py

Clicker Question #5¶

To create a file and see its contents, which command line commands would you use (and in which order)?

A) mkdir > cd
B) pwd > ls
C) cd > pwd
D) touch > cat
E) cat > touch

Clicker Question #6¶

Given the file structure from earlier:

/
- scripts/
  - cool_thing.py
  - super_cool_thing.py
- images/
  - image1.png
  - image2.png
- notebooks/
  - 00_intro.ipynb
  - 01_variables.ipynb

Your currently working within notebooks and you want to execute the code in ‘cool_thing.py’ from the command line. How would you do that?

A) python cool_thing.py
B) python ../notebooks/cool_thing.py
C) python ../scripts/cool_thing.py
D) python ../cool_thing.py
E) ¯\_(ツ)_/¯

Introductory Python

Command Line & Scripts¶

File Systems¶

Command Line¶

The Terminal¶

Shell Commands¶

Check current directory¶

An important aside: File Paths¶

Absolute vs. Relative Paths¶

Absolute Paths¶

Relative Paths¶

Clicker Question #1¶

Clicker Question #2¶

Shell Commands¶

Check current directory¶

Change directory¶

List files in a directory¶

More Shell Commands¶

Make a new directory¶

Create a file¶

Move a file¶

And Some More¶

Print out a message¶

Print the contents of a file¶

Open to see and edit contents of a file¶

Clicker Question #3¶

Clicker Question #4¶

Windows Command Prompt¶

Python Files¶

Script vs. Module File¶

Scripts¶

Module Files¶

Text Editors¶

Terminal Based Text Editors¶

vim¶

Non-Terminal Text Editors¶

Executing Python Files¶

Clicker Question #5¶

Clicker Question #6¶