# Moving around and looking at things

```{objectives}
- Learn how to navigate through folders.
- Learn how to create, copy, move, rename, and remove files and folders.
- Learn how to see the contents of files in the terminal.
- Learn how to compare files.
```

```{instructor-note}
- Demo/teaching: 15 min
- Exercise: 15-20 min
```

````{note}
- First we will demonstrate a couple of commands and later we will use these
  in an exercise.
- If you want to type-along with the instructors, you can download and extract
  the example like this:
  ```
  cd
  wget https://gitlab.sigma2.no/training/tutorials/unix-for-hpc/-/raw/master/content/episodes/moving-around/demo-moving-around.tar.gz --no-check-certificate
  tar xvf demo-moving-around.tar.gz
  cd demo-moving-around
  ```
````


## Navigating in the file tree

I use the following commands often to find out where I am and what files and
folders are present in my current path/location:

This command **p**rints the current **w**orking **d**irectory:
```console
$ pwd

/cluster/home/user/demo-moving-around
```

With `cd` we can **c**hange **d**irectory:
```console
$ pwd
$ cd some-folder
$ pwd
$ cd ..
$ pwd
```
The `..` refers to one directory "up".


"" List what is in a directory

With `ls` we can **list** the current directory:
```console
$ ls

another-folder	numbers.out  poem.txt  some-folder  unfinished.txt
```

Most/all Unix commands have many options ("flags"). A flag is entered with a dash infront of the option: `-`. For example the **l**ong
format:
```console
$ ls -l

total 139
drwxr-xr-x 2 user user_g      1 Apr 28 21:03 another-folder
-rw-r--r-- 1 user user_g 140164 Apr 28 21:03 numbers.out
-rw-r--r-- 1 user user_g    359 Apr 28 21:03 poem.txt
drwxr-xr-x 2 user user_g      1 Apr 28 21:03 some-folder
-rw-r--r-- 1 user user_g    345 Apr 28 21:03 unfinished.txt
```

Or **h**uman-friendly output:
```console
$ ls -lh

total 139K
drwxr-xr-x 2 user user_g    1 Apr 28 21:03 another-folder
-rw-r--r-- 1 user user_g 137K Apr 28 21:03 numbers.out
-rw-r--r-- 1 user user_g  359 Apr 28 21:03 poem.txt
drwxr-xr-x 2 user user_g    1 Apr 28 21:03 some-folder
-rw-r--r-- 1 user user_g  345 Apr 28 21:03 unfinished.txt
```

We can list files replacing parts of the filename(s) by wildcards. All filenames ending in `txt` will be listed in this example:
```console
$ ls -l *txt

-rw-r--r-- 1 bast bast_g 359 Apr 28 20:25 poem.txt
-rw-r--r-- 1 bast bast_g 345 Apr 28 20:55 unfinished.txt
```

All linux commands have a man (manual) page. Let's have a look at the many options `ls` offers with the `man` command
(exit with "q" for quit):
```console
$ man ls
```




Instead of listing the current directory you can also give `ls` any other path
as argument.

```{discussion} Why do some commands have so short/cryptic/strange names?

Many Unix commands are over 40 years old and when they got introduced, screen
space was extremely limited. Some names were also meant to be funny or form
funny recursive acronyms. But unfortunately not all command names were
designed to be intuitive. Some names haven't been carefully designed but just
evolved from a personal tool that somebody developed over the weekend and
their colleagues started using and then quickly became too popular to be
changed.

Book recommendation to read about the early days of Unix: ["Unix: A History and a Memoir" by Brian Kernighan](https://www.cs.princeton.edu/~bwk/memoir.html)
```


## How to find back home

When you first connect to a cluster, you land in your home folder (try it and
verify the location with `pwd`).
These can tell us where our home folder is located:
```console
$ echo $HOME
$ echo ~
```

All of these can bring you back to your home folder root:
```console
$ cd $HOME
$ cd ~
$ cd
```


## Creating, copying, renaming, moving, and removing

````{note}
Before we continue let's make sure we are back in the demo folder:
  ```
  cd && cd demo-moving-around
  ```
````
This is also an example of concatenating two commands with `&&`: first `cd` is performed to place you in your home folder, then `cd demo-moving-around` which is a subfolder of your home directory.


Sometimes we need to **m**a**k**e new folder/**dir**ectory:
```console
$ mkdir new-folder
```

Let's **c**o**p**y a file:
```console
$ cp poem.txt poem2.txt
```

We can copy the file into another folder:
```console
$ cp poem.txt new-folder
```

You can also copy several files at once. The last argument is the destination folder (and must be a folder):
```console
$ cp poem.txt poem2.txt some-folder
```

The command `mv` can be used to **m**o**v**e files or folders or to rename them (for
Unix both are the same thing):
```console
$ mv poem2.txt poem3.txt
```

**R**e**m**oving a file:
```console
$ rm poem3.txt
```

Removing a directory (you can check what `-r` flag does with `man rm`):
```console
$ rm -r new-folder
```


## Printing and comparing file contents

The command `cat` can be used to concatenate several files into one but many
people use it on one file only to print its contents:
```console
$ cat poem.txt
```

Let's try the commands `more` and `less`. They behave similarly and allow to
page (scroll) through the file. They are useful if you don't want to see the
whole long file in one go. You can scroll with Enter and quit with "q":
```console
$ more numbers.out
$ less numbers.out
```

Commands `head` and `tail` only print the beginning and the end:
```console
$ head poem.txt
$ tail poem.txt
```

The command `diff` is super useful to compare files or to check whether files are identical:
```console
$ diff unfinished.txt poem.txt

1c1
< Working title
---
> Success is counted sweetest
```


## Exercise

`````{exercise} Exercise (20 min): Navigate in the file system and look at file content

It is perfectly fine to only do part of the steps below. They may be too many
for 15-20 minutes, so if you don't reach them all, do them as homework later on. 

1. Connect to the cluster - check the hostname and locate where in the file system you are:
   ```console
   $ hostname
   $ pwd
   ```
1. Download and extract the exercise folder:
   ```
   cd
   wget https://gitlab.sigma2.no/training/tutorials/unix-for-hpc/-/raw/master/content/episodes/moving-around/exercise-moving-around.tar.gz --no-check-certificate
   tar xvf exercise-moving-around.tar.gz
   ```
1. Step into the exercise folder:
   ```console
   $ cd exercise-moving-around
   ```
1. Make a drawing on paper (or a mental picture) of the folder structure that
   you find under exercise-moving-around.
1. Find out in which subfolder are example output files with the ending
   `.output`.
1. Let us look at one of the files , `computation-a.output` in  `./exercise-moving-around/results/` and compare the outputs
   of the following commands:
   ```console
   $ cat  computation-a.output
   $ more computation-a.output
   $ less computation-a.output
   $ head computation-a.output
   $ tail computation-a.output
   ```
1. Find a directory called "backup". Copy all the `.output` files to the
   backup folder.
1. In the folder where we found the `.output` files (not the backup folder),
   rename the output files to now have the ending `.out` instead of `.output`.
1. Among the output files there are two files with the same content (verify
   with `diff`), can you remove one of the duplicate files?
1. How would you copy the `inputs` folder under `backup`? How would you move
   the folder?

````{solution}
1. These commands will produce different outputs depending on where we are. Examples:
   ```console
   $ hostname

   login-4.saga
   ```
   ```console
   $ pwd

   /cluster/home/user
   ```
1. This step will hopefully produce the exercise folders. Nothing to change
   here.
1. Nothing to change here.
1. Drawing or mental picture of the folder structure. Something like this:
   ```
   exercise-moving-around
   ├── backup
   │   └── old
   │       └── old-computation.out
   ├── inputs
   │   └── example.in
   ├── results
   │   ├── computation-a.output
   │   ├── computation-b.output
   │   ├── computation-c.output
   │   ├── computation-c2.output
   │   ├── computation-d.output
   │   ├── computation-e.output
   │   └── computation-f.output
   └── scripts
       └── example.sh
   ```
1. One rudimentary was is to use `ls` to inspect the contents of the
   directories - combined with the wildcard symbol `* ` 
   ```console
   $ ls ./*
   $ ls ./*/*
   $ ls ./*/*/*
   ``` 
   There are better ways using the `find` command, which will be
   taught later. But to spoil the suprise, here it comes:
   ```console
   $ find . -name "*.output"
   ``` 
1. `cat` prints the whole file. `more` and `less` look similar and allow to
   page (scroll) through the file. You can scroll with Enter and quit with
   "q". `head` and `tail` only print the beginning and the end.
1. First navigate to the results folder:
   ```console
   $ cd results
   ```
   Then copy all files
   ```console
   $ cp *.output ../backup
   ```
1. Here is a manual way (later we will learn more elegant ways to rename all
   in one go):
   ```console
   $ mv computation-a.output computation-a.out
   $ mv computation-b.output computation-b.out

   ...
   ```
1. These two files have identical content (`diff` produces no output):
   ```console
   $ diff computation-c.output computation-c2.output
   ```
   You can remove one of them like this:
   ```console
   $ rm computation-c2.output
   ```
1. We can copy the entire folder like this:
   ```console
   $ cp -r inputs backup
   ```
   Or we can move it:
   ```console
   $ mv inputs backup
   ```
````
`````

```{keypoints}
- With a handful of commands we can navigate, move, rename, copy.
- In Unix there are often many ways to achieve something.
- Commands often have options/"flags". You can check manual pages with `man`.
```