Shell Course 4

Chapter 4: Processes and Tasks

## Understanding Processes

Every program running on your computer exists as one or more processes. When you launch an application, edit a document, or even use the shell itself, you’re creating and interacting with processes. Understanding how to monitor and control these processes gives you tremendous power over your system.

Think of processes as the living organisms of your operating system’s ecosystem - they’re born, they consume resources, they communicate with each other, and eventually, they die.

What is a Process?

A process is an instance of a running program. Each process has:

  • A unique Process ID (PID)
  • A parent process that created it
  • Allocated memory and resources
  • An owner (the user who started it)
  • A state (running, sleeping, stopped, zombie, etc.)

When you run a command in the shell, you’re creating a new process. For example, when you type ls, the shell creates a process to execute the ls program, which then terminates when its job is complete.

Process Hierarchy

Processes in Unix-like systems exist in a parent-child hierarchy. Every process (except the very first one, called init or systemd on modern systems) has a parent that created it.

This hierarchy begins when your computer boots:

  1. The kernel starts the init process (PID 1)
  2. init spawns system services and your login session
  3. Your login session spawns the shell
  4. The shell spawns commands you run

Understanding this family tree of processes is crucial for effective system management.

## Viewing Processes

Let’s explore the tools that let you see what’s happening under the hood of your system.

The ps Command

The ps command (short for “process status”) provides a snapshot of current processes:

1
2
3
4
$ ps
  PID TTY          TIME CMD
 3822 pts/0    00:00:00 bash
 5021 pts/0    00:00:00 ps

This basic output shows:

  • PID: Process ID
  • TTY: Terminal associated with the process
  • TIME: CPU time used
  • CMD: Command name

However, the real power comes with options:

1
$ ps aux

This comprehensive format shows all processes (not just yours), with detailed information including CPU and memory usage.

Let’s break down the output columns:

  • USER: Who owns the process
  • PID: Process ID
  • %CPU: Percentage of CPU used
  • %MEM: Percentage of physical memory used
  • VSZ: Virtual memory size
  • RSS: Resident set size (non-swapped physical memory)
  • TTY: Terminal associated with the process
  • STAT: Process state code (R=running, S=sleeping, Z=zombie, etc.)
  • START: Start time
  • TIME: Cumulative CPU time
  • COMMAND: Command with arguments

The ps command is like taking a photograph of your system’s processes at a moment in time. Sometimes useful, but if you want to see processes in motion, you need something more like a film.

Real-time Process Monitoring with top

While ps gives a static snapshot, top provides a dynamic, real-time view of processes:

1
$ top

This command refreshes every few seconds, showing the most resource-intensive processes at the top. The header provides system-wide information:

  • Load averages (system workload)
  • Tasks count by state
  • CPU usage breakdown
  • Memory usage statistics

Navigation in top:

  • Press q to quit
  • Press k to kill a process (you’ll be prompted for the PID)
  • Press r to renice a process (change its priority)
  • Press F to select which columns to display
  • Press u to filter by user

A Better Top: htop

If top is a film, then htop is that same film but in high definition with a better interface. It may not be installed by default, but it’s worth adding to your toolkit:

1
2
3
4
5
6
7
8
# On Debian/Ubuntu
$ sudo apt install htop

# On macOS with Homebrew
$ brew install htop

# Then run it
$ htop

htop offers several advantages over top:

  • Color-coded output for better readability
  • Visual indicators of CPU and memory usage
  • Ability to scroll horizontally and vertically
  • Mouse support for interaction
  • Built-in kill, nice, and other operations without needing to know PIDs

Using htop is like upgrading from a basic digital watch to a modern smartwatch - the core functionality is the same, but the interface and additional features make it much more pleasant to use.

## Process Control

Viewing processes is useful, but controlling them is where the real power lies.

Foreground vs Background Processes

By default, when you run a command in the shell, it runs in the foreground, meaning:

  • The shell waits for it to complete before giving you back the prompt
  • It receives input from and sends output to your terminal
  • You can interrupt it with keyboard shortcuts

But sometimes, you want to run a command and get back to work without waiting for it to finish. That’s where background processes come in.

Running Processes in the Background

To start a command in the background, add an ampersand (&) at the end:

1
2
$ long-running-command &
[1] 5243

The shell returns:

  • A job number in brackets [1]
  • The process ID (5243)

The command runs in the background, and you get your prompt back immediately.

Note: Background processes still output to your terminal by default, which can be disruptive. To prevent this, you can redirect output:

1
$ long-running-command > output.log 2>&1 &

This redirects both standard output and standard error to output.log.

Job Control

The shell maintains a list of jobs (commands) that you’ve started. You can view this list with the jobs command:

1
2
$ jobs
[1]+ Running    long-running-command &

Each job has a number that you can use to refer to it. You can bring a background job to the foreground with fg:

1
$ fg %1

This brings job 1 to the foreground. You can omit the % if there’s no ambiguity:

1
$ fg 1

Conversely, you can send a foreground process to the background with Ctrl+Z to suspend it, followed by bg:

1
2
3
4
[press Ctrl+Z]
[1]+ Stopped    long-running-command
$ bg
[1]+ long-running-command &

Think of foreground and background processes like tabs in a browser - foreground is the active tab you’re looking at, while background tabs are still running but not currently in focus. Job control is like the tab bar that lets you switch between them.

Terminating Processes

Sometimes, you need to stop a process before it completes naturally. The kill command sends signals to processes:

1
$ kill 5243

This sends the TERM (terminate) signal to the process with PID 5243, asking it to shut down gracefully.

If a process is stubborn, you can force it to quit:

1
$ kill -9 5243

This sends the KILL signal, which the process cannot ignore.

The -9 option should be a last resort. Always try a normal kill first, as it allows the process to clean up properly.

You can also kill processes by job number in the current shell:

1
$ kill %1

Common Signals

While there are many signals you can send with kill, the most common are:

  • SIGHUP (1): Hangup, often used to make a process reload its configuration
  • SIGINT (2): Interrupt, same as pressing Ctrl+C
  • SIGTERM (15): Terminate, the default signal sent by kill
  • SIGKILL (9): Kill immediately, cannot be caught or ignored
  • SIGSTOP (19): Stop execution, cannot be caught or ignored

To send a specific signal, use the signal number or name:

1
2
3
$ kill -SIGHUP 5243
# or
$ kill -1 5243

## Process Priorities

Not all processes are created equal. The system uses a concept called “niceness” to determine how much CPU time to allocate to each process.

Understanding Nice Values

The nice value ranges from -20 (highest priority) to 19 (lowest priority), with 0 being the default. A process with a lower nice value gets more CPU time compared to one with a higher value.

Think of nice values as queuing at a British pub - the default is to wait your turn (nice value 0), but some patrons might be especially polite and let others go first (positive nice value), while others might be rather forward and push to the front (negative nice value).

Viewing Process Nice Values

The -l option to ps shows the nice value (NI column):

1
2
3
4
$ ps -l
F S   UID    PID   PPID  C PRI  NI ADDR SZ WCHAN  TTY          TIME CMD
0 S  1000   3822   3821  0  80   0 -  6683 -      pts/0    00:00:00 bash
0 R  1000   6134   3822  0  80   0 -  7317 -      pts/0    00:00:00 ps

Setting Process Priorities

To start a new process with a specific nice value, use the nice command:

1
$ nice -n 10 ./large-computation.sh

This runs large-computation.sh with a niceness of 10 (lower priority).

To change the priority of a running process, use renice:

1
$ renice -n 15 -p 5243

This changes the nice value of process 5243 to 15.

Note: Regular users can only increase the nice value (lower the priority) of their processes, and can’t set negative values. Only the root user can decrease the nice value (increase priority).

## Monitoring System Resources

Understanding which processes are running is only part of the picture. You also need to know how they’re affecting your system’s resources.

Memory Usage

The free command shows memory usage:

1
2
3
4
$ free -h
              total        used        free      shared  buff/cache   available
Mem:           15Gi       4.2Gi       7.7Gi       386Mi       3.5Gi        10Gi
Swap:         4.0Gi          0B       4.0Gi

The -h option makes the output human-readable (with units like Gi for gibibytes).

Disk I/O

The iotop command (which may need to be installed) shows disk read/write activity:

1
$ sudo iotop

For a simpler view, you can use iostat (part of the sysstat package):

1
$ iostat -x 2

This shows extended statistics, refreshing every 2 seconds.

Network Usage

To monitor network traffic, you can use tools like nethogs or iftop:

1
$ sudo nethogs

These may need to be installed via your package manager.

Overall System Monitoring

For comprehensive system monitoring, you might consider more advanced tools:

  • glances: An all-in-one monitoring tool with a user-friendly interface
  • nmon: A performance monitor for Linux systems
  • dstat: A versatile replacement for various system monitoring tools

These tools often need to be installed, but they provide a wealth of information in a more digestible format than the native commands.

## Scheduling Tasks

As a system administrator or power user, you’ll often need to run commands at specific times rather than manually executing them. Linux and Unix-like systems provide several powerful tools for scheduling tasks.

The at Command

The at command allows you to schedule one-time tasks to run at a specific time:

1
2
3
4
5
6
$ at 10:00 PM
warning: commands will be executed using /bin/sh
at> echo "It's 10 PM!" > /tmp/at_test
at> find /home -name "*.tmp" -delete
at> <Ctrl+D>
job 3 at Thu Mar 04 22:00:00 2025

After typing at with a time specification, you’ll see the at> prompt. Type each command you want to run, then press Ctrl+D to finish.

Time can be specified in various formats:

1
2
3
4
5
6
7
$ at 10:00 PM                     # 10:00 PM today
$ at 10:00 PM tomorrow            # 10:00 PM tomorrow
$ at 10:00 PM next week           # 10:00 PM same day next week
$ at 10:00 PM 03/10/2025          # 10:00 PM on specific date
$ at now + 30 minutes             # 30 minutes from now
$ at now + 2 hours                # 2 hours from now
$ at now + 1 week                 # 1 week from now

The at command is perfect for those “I need to remember to do this later” moments, like a digital version of tying a string around your finger.

Managing Scheduled Jobs

To view pending at jobs:

1
2
$ atq
3   Thu Mar 04 22:00:00 2025 a username

To remove a scheduled job:

1
$ atrm 3

The cron System

While at handles one-time tasks, cron is designed for recurring tasks. It works by reading configuration files called “crontabs” that specify when and how often commands should run.

To edit your personal crontab:

1
$ crontab -e

This will open an editor where you can add scheduled tasks in the following format:

1
2
# minute hour day month weekday command
0 10 * * * echo "It's 10 AM!" > /tmp/cron_test

This example runs the echo command at 10:00 AM every day.

Understanding the crontab format:

1
2
3
4
5
6
7
* * * * * command
| | | | |
| | | | +-- Weekday (0-7) (0 or 7 is Sunday)
| | | +---- Month (1-12)
| | +------ Day of month (1-31)
| +-------- Hour (0-23)
+---------- Minute (0-59)

An asterisk (*) means “every” for that position. You can also use:

  • Commas for lists: 1,3,5
  • Hyphens for ranges: 1-5
  • Slashes for steps: */10 (every 10th unit)

Common crontab examples:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# Run every hour
0 * * * * /path/to/script.sh

# Run every 15 minutes
*/15 * * * * /path/to/script.sh

# Run at 2:30 AM every day
30 2 * * * /path/to/script.sh

# Run at midnight on the first of every month
0 0 1 * * /path/to/script.sh

# Run every weekday (Monday through Friday) at 8 AM
0 8 * * 1-5 /path/to/script.sh

# Run every Saturday and Sunday at 9 PM
0 21 * * 0,6 /path/to/script.sh

Viewing and Managing Crontabs

To list your current crontab:

1
$ crontab -l

To remove your crontab entirely:

1
$ crontab -r

System-wide cron jobs:

In addition to user-specific crontabs, system-wide cron jobs are stored in various locations:

  • /etc/crontab: The system crontab file
  • /etc/cron.d/: Directory for additional crontab files
  • /etc/cron.hourly/, /etc/cron.daily/, /etc/cron.weekly/, /etc/cron.monthly/: Directories for scripts to run at specific intervals

Many system maintenance tasks are scheduled through cron, such as log rotation, temporary file cleanup, and database backups. It’s the silent custodian of your system, diligently performing housekeeping while you’re away.

## Automating Process Management

Sometimes you need to take action when certain conditions are met, like restarting a crashed service or alerting when resources run low.

Basic Monitoring Script

Here’s a simple script to check if a critical process is running and restart it if needed:

1
2
3
4
5
6
7
8
9
#!/bin/bash

# Check if Apache web server is running
if ! pgrep -x "httpd" > /dev/null; then
    echo "Apache is not running. Restarting..."
    sudo systemctl start httpd
else
    echo "Apache is running normally."
fi

Save this as check_apache.sh, make it executable with chmod +x check_apache.sh, and run it periodically.

Using watch for Regular Monitoring

The watch command runs a command repeatedly, showing the output:

1
$ watch -n 5 "ps aux | grep mysql"

This runs ps aux | grep mysql every 5 seconds, highlighting differences between updates.

Running Commands at System Load Thresholds

The stress command allows you to run commands when the system load reaches certain levels:

1
2
3
4
$ sudo apt install stress  # Install if needed

# Then run
$ stress --cpu 8 --io 4 --vm 2 --vm-bytes 128M --timeout 30s

This simulates high CPU, I/O, and memory load for 30 seconds, useful for testing monitoring solutions.

## Practical Exercises

Let’s practice what we’ve learned with some hands-on exercises:

  1. View all your processes:

    1
    $ ps aux | grep $USER

  2. Start a background process that generates a large file:

    1
    $ dd if=/dev/zero of=largefile bs=1M count=1000 &

  3. Check its status in the jobs list:

    1
    $ jobs

  4. Monitor the process and overall system status:

    1
    $ top -p $(pgrep -d',' -f "dd if=/dev/zero")

  5. Bring the process to the foreground:

    1
    $ fg %1

  6. Stop it with Ctrl+C.

  7. Start a nice’d process in the background:

    1
    $ nice -n 15 find / -name "*.log" > logs_found.txt 2>/dev/null &

  8. Check its nice value:

    1
    $ ps -o pid,comm,nice -p $(pgrep -f "find / -name")

Advanced exercise: Write a simple shell script that monitors your system’s memory usage every 5 minutes and sends you a notification if it exceeds 80%.

Solution outline:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
#!/bin/bash

while true; do
    # Get memory usage percentage
    mem_usage=$(free | grep Mem | awk '{print $3/$2 * 100.0}')

    # Check if it exceeds threshold
    if (( $(echo "$mem_usage > 80" | bc -l) )); then
        # On Linux with notifications
        notify-send "Warning" "Memory usage is at ${mem_usage}%"
        # On macOS
        # osascript -e "display notification \"Memory usage is at ${mem_usage}%\" with title \"Warning\""
    fi

    # Wait 5 minutes
    sleep 300
done

## Conclusion

Understanding processes is fundamental to mastering the shell. With the knowledge and tools from this chapter, you can:

  • Monitor what’s happening on your system
  • Control processes through their lifecycle
  • Manage system resources effectively
  • Automate routine monitoring tasks

These skills are essential whether you’re managing a personal workstation or administering servers. Process management is the difference between being at the mercy of your computer and being firmly in control of it.

In the next chapter, we’ll explore shell scripts in greater depth, learning how to automate complex sequences of commands and create your own powerful tools.

Like a conductor leading an orchestra, you now have the ability to direct the symphony of processes running on your system, ensuring each plays its part at the right time and with the appropriate vigor.