Shell Course 2

Chapter 2: Navigation and File Manipulation

## The Art of Moving About

If Chapter 1 was about getting acquainted with the terminal, Chapter 2 is about learning to navigate it with confidence and precision. Think of your filesystem as a vast library with countless rooms and shelves - knowing how to move around efficiently is essential.

Navigation in the shell is rather like finding your way through an unfamiliar city. Initially confusing, perhaps, but ultimately liberating once you’ve mastered it.

Let’s expand our navigational toolkit beyond the basic cd command we’ve already met.

Understanding Paths: Absolute vs. Relative

There are two ways to tell the shell where to find something:

• Absolute paths start from the root directory (/) and give the complete address.
• Relative paths start from your current location.

For instance, if you’re in your home directory:

# Absolute path example
$ cd /home/username/Documents

# Relative path equivalent
$ cd Documents

Think of it like giving someone directions: An absolute path is saying “Starting from the town center, go north for 2 miles, then east for 1 mile.” A relative path is saying “Turn left at the next corner.”

Navigating Efficiently

Let’s revisit and expand on some navigation shortcuts:

$ cd ~            # Home directory (same as just 'cd')
$ cd ..           # Parent directory (one level up)
$ cd ../..        # Two levels up
$ cd -            # Previous directory (toggle between locations)
$ cd ~/Documents  # Documents inside home directory

Pro tip: If you’re not sure where a directory is, use the find command to track it down - rather like having a GPS for your filesystem.

$ find / -type d -name "Downloads" 2>/dev/null
/home/username/Downloads

The 2>/dev/null part simply tells the shell to discard error messages, which is particularly useful when searching system directories where you might not have permission to look everywhere.

---

## File Management: The Daily Bread of Shell Usage

Now that we can navigate our file system confidently, let’s look at how to manipulate files and directories - tasks you’ll likely perform daily.

Copying Files: The cp Command

The cp command copies files or directories from one location to another.

$ cp source.txt destination.txt        # Copy a file
$ cp -r source_dir destination_dir     # Copy a directory and its contents

Careful now: cp will overwrite the destination file without warning if it already exists. It’s rather like a helpful assistant who reorganizes your desk without asking first.

For the cautious:

$ cp -i source.txt destination.txt    # Interactive mode, asks before overwriting

Moving and Renaming: The mv Command

The mv command serves a dual purpose: moving files and renaming them (which is technically just “moving” a file to a new name in the same location).

$ mv oldname.txt newname.txt         # Rename a file
$ mv file.txt /path/to/destination/  # Move a file

Like cp, mv will overwrite destination files without asking. Use -i for interactive mode if you prefer to be asked first.

Removing Files: The rm Command

The rm command deletes files and directories. Use with appropriate care.

$ rm filename.txt             # Delete a file
$ rm -r directory_name        # Delete a directory and its contents

Warning: There is no recycle bin or trash can in the shell. Once you delete something with rm, it’s gone. Forever. Like that umbrella you left on the Tube.

For the cautious:

$ rm -i filename.txt         # Interactive mode, asks for confirmation

For those who prefer efficiency over safety, there’s rm -rf, which recursively deletes directories without asking. Just remember that ‘with great power comes the great ability to accidentally delete your entire home directory’ (LoserMan).

Creating Links: The ln Command

Links are like shortcuts or aliases to other files or directories.

Symbolic links (soft links) point to another file by name:

$ ln -s original.txt link.txt    # Create a symbolic link

If you move the original file, the link breaks - it’s pointing to a location that no longer exists.

Hard links point to the actual data on disk:

$ ln original.txt hardlink.txt   # Create a hard link

Move or rename the original, and the hard link still works. It’s less a shortcut and more like having two entrances to the same room.

---

## Understanding File Permissions

Unix-like systems have a permission system that can seem complex at first but makes perfect sense once you understand the logic behind it.

Viewing Permissions

The ls -l command shows detailed information about files, including their permissions:

$ ls -l file.txt
-rw-r--r-- 1 username group 123 Mar 4 10:30 file.txt

The permissions are represented by the first 10 characters:

• First character: File type (- for regular file, d for directory)
• Next three: Owner permissions (read, write, execute)
• Next three: Group permissions
• Last three: Everyone else’s permissions

Understanding Permission Notation

Each permission set has three positions:

• r (read): Can see the content
• w (write): Can modify
• x (execute): Can run as a program

So -rw-r--r-- means:

• It’s a regular file (-)
• The owner can read and write (rw-)
• Group members can only read (r--)
• Others can only read (r--)

Changing Permissions: The chmod Command

You can change permissions using the chmod command with either symbolic or numeric notation.

Symbolic notation uses letters and operators:

$ chmod u+x script.sh    # Add execute permission for the user
$ chmod g-w file.txt     # Remove write permission for the group
$ chmod o=r file.txt     # Set 'others' permission to read-only
$ chmod a+r file.txt     # Give read permission to everyone

Where:

• u = user (owner)
• g = group
• o = others
• a = all

Numeric notation uses a 3-digit octal number:

$ chmod 755 script.sh    # rwxr-xr-x
$ chmod 644 file.txt     # rw-r--r--

The digits represent:

1. Owner permissions
2. Group permissions
3. Others permissions

Each digit is calculated as:

• 4 for read
• 2 for write
• 1 for execute

So 7 (4+2+1) means read, write, and execute; 6 (4+2) means read and write; 5 (4+1) means read and execute; and 4 means read-only - rather like how your office security badge gives you access to different areas depending on your clearance level.

Changing Ownership: The chown Command

The chown command changes the owner and/or group of a file:

$ chown newowner file.txt             # Change owner
$ chown newowner:newgroup file.txt    # Change owner and group
$ chown :newgroup file.txt            # Change group only

You’ll typically need administrative privileges (via sudo) to change ownership:

$ sudo chown root:root system_file.txt

---

## Viewing and Examining Files

Let’s explore some sophisticated ways to peek inside files without modifying them.

Basic File Viewing

We’ve already met cat, which displays the entire file at once - perfect for small files, but less ideal for lengthy documents.

For more controlled viewing:

$ less file.txt       # View file with scrolling and search capabilities

Inside less, use:

• Space or f to page forward
• b to page backward
• / followed by a search term to find text
• q to quit

The less command is named as a play on the older more command, proving that Unix developers do indeed possess a sense of humor, however dry it might be.

For peeking at just parts of a file:

$ head -n 10 file.txt    # View first 10 lines
$ tail -n 10 file.txt    # View last 10 lines

These commands are particularly useful for large log files when you only need to see recent entries.

Examining File Types

The file command tells you what type of file you’re looking at:

$ file mystery_file
mystery_file: JPEG image data, JFIF standard 1.01

This is particularly useful when dealing with files without extensions or when you’re unsure of a file’s actual format.

Finding Text in Files: The grep Command

The grep command searches for patterns in files:

$ grep "search_term" file.txt           # Search for a term in a file
$ grep -i "case insensitive" file.txt   # Ignore case
$ grep -r "term" directory/             # Search recursively through directories
$ grep -v "exclude" file.txt            # Show lines that DON'T match

Combining powers: You can pipe the output of one command to grep:

$ ls -la | grep "Jan"        # Show only files modified in January
$ ps aux | grep "firefox"    # Check if Firefox is running

The pipe (|) symbol is one of shell’s most powerful concepts - it takes the output from one command and feeds it as input to another. Think of it as an assembly line where each command performs its specialized task on the data.

Counting Words, Lines, and Characters

The wc (word count) command gives you statistics about text:

$ wc file.txt
      7      46     237 file.txt

The output shows lines, words, and characters, respectively.

You can specify what to count:

$ wc -l file.txt    # Count lines only
$ wc -w file.txt    # Count words only
$ wc -c file.txt    # Count characters only

---

## Practical File Operations

Let’s look at some common everyday tasks and how to accomplish them efficiently.

Finding Files

The find command is your best friend for locating files:

$ find ~/Documents -name "*.pdf"              # Find PDFs in Documents folder
$ find /home -type f -size +100M              # Find files larger than 100MB
$ find . -type f -mtime -7                    # Files modified in the last 7 days
$ find /etc -type f -user root -perm 644      # Files owned by root with specific permissions

Power move: You can execute commands on found files:

$ find . -name "*.tmp" -exec rm {} \;        # Delete all .tmp files

The {} is replaced with each found filename, and \; marks the end of the command.

Comparing Files

The diff command shows the differences between files:

$ diff file1.txt file2.txt

For a more visual side-by-side comparison:

$ diff -y file1.txt file2.txt

Disk Usage

The du (disk usage) command shows space used by files and directories:

$ du -h file.txt              # Show file size in human-readable format
$ du -sh directory/           # Show total directory size (summary)
$ du -h --max-depth=1 /home/  # Show first-level directory sizes

The df command shows disk space available on mounted filesystems:

$ df -h                       # Show disk space in human-readable format

---

## Practical Exercises

Let’s practice what we’ve learned with some hands-on exercises:

1. Create a directory structure for a hypothetical project:

   $ mkdir -p Project/{src,docs,tests}

2. Create a few sample files:

   $ touch Project/src/{main.c,helper.c,utils.c}
   $ touch Project/docs/README.md
   $ echo "This is a test file" > Project/tests/test1.txt

3. Copy a file and examine the differences:

   $ cp Project/tests/test1.txt Project/tests/test2.txt
   $ echo "With some changes" >> Project/tests/test2.txt
   $ diff Project/tests/test1.txt Project/tests/test2.txt

4. Change file permissions:

   $ chmod 755 Project/src/*.c
   $ ls -l Project/src/

5. Find all text files in your project:

   $ find Project/ -name "*.txt" -o -name "*.md"

6. Count the words in your test files:

   $ wc Project/tests/*.txt

Challenge: Create a single command that finds all .c files in your project and counts their lines.

Solution: find Project/ -name "*.c" -exec wc -l {} \;

---

## Conclusion

In this chapter, we’ve expanded our shell skills to include navigating the filesystem efficiently, manipulating files and directories, understanding permissions, and examining file contents. These are the fundamental operations you’ll use daily when working with the shell.

Remember that the shell commands are tools in your toolkit. Like any good craftsperson, you’ll become more proficient as you use them regularly. Don’t be afraid to experiment in a safe environment (perhaps a dedicated practice directory), and make liberal use of the man pages when you need a refresher on specific options.

In the next chapter, we’ll explore how to work with text in more sophisticated ways, introducing powerful text processing tools that can help you extract and manipulate information from files.

Practice is key - blah blah, you know the drill…