Shell Scripting Fundamentals

Shell scripting transforms you from a passive user of the command line into an active automator, turning repetitive, error-prone tasks into reliable, one-command solutions. It is the glue of system administration, the backbone of deployment pipelines, and a critical skill for any developer or engineer working in Linux or Unix-like environments. Mastering the fundamentals allows you to manage files, monitor systems, and orchestrate complex workflows efficiently, directly from the terminal.

The Foundation: Scripts, Shells, and the Shebang

A shell script is a plain text file containing a sequence of commands for a command-line interpreter, or shell. The most common shell on Linux and macOS is bash (Bourne Again SHell), which this guide focuses on. The script itself is not compiled; it is executed line-by-line by the shell.

The first line of any script is crucial: the shebang (#!). It tells the operating system which interpreter to use to run the script. For a bash script, this line is:

#!/bin/bash

This path points to the bash executable. Without a correct shebang, your script may fail or behave unexpectedly depending on the user's default shell.

To make a script executable, you use the chmod command:

chmod +x my_script.sh

You can then run it with ./my_script.sh. This combination—shebang plus execute permission—is the entry point to all scripting.

Storing Data: Variables and User Input

Variables allow you to store and manipulate data. In bash, you create a variable by assigning a value with an equals sign (no spaces around it) and reference it with a dollar sign $.

filename="backup_$(date +%Y%m%d).tar.gz"
echo "Creating archive: $filename"

Here, $(date +%Y%m%d) is command substitution, where the output of the date command is captured and used as part of the string.

To make a variable available to subprocesses (scripts or commands launched from your script), you must export it:

export DATABASE_URL="postgres://localhost/mydb"

For interactivity, you can capture user input using the read command.

echo "Enter the target directory:"
read target_dir
if [ -d "$target_dir" ]; then
  echo "Directory exists."
fi

Notice the quotes around "$target_dir". This is essential to handle filenames or input with spaces correctly—a common pitfall we'll address later.

Controlling Execution Flow: Conditionals and Loops

Scripts need to make decisions and repeat actions. Conditionals use the if, then, elif, else, and fi structure. The test condition is placed within square brackets [ ] or double brackets [[ ]], which are preferred for fewer surprises with strings and patterns.

if [[ -f "/etc/config.yaml" ]]; then
  echo "Configuration file found."
elif [[ -d "/etc/config.d" ]]; then
  echo "Config directory found."
else
  echo "No config found." >&2
  exit 1
fi

Key test operators include -f (file exists), -d (directory exists), -z (string is empty), and comparison operators like -eq (equal) for numbers and == for strings within [[ ]].

Loops automate repetition. The for loop iterates over a list or a range.

# Iterate over a list of names
for user in alice bob charlie; do
  echo "Processing $user"
  mkdir -p "/home/projects/$user"
done

# Iterate over files matching a pattern
for logfile in /var/log/*.log; do
  echo "Analyzing: $logfile"
done

The while loop runs as long as a condition is true, perfect for reading a file line-by-line or waiting for a condition.

while read -r line; do
  echo "Read: $line"
done < "input.txt"

Building Blocks: Functions and Exit Codes

As scripts grow, functions help you organize code into reusable blocks. They are defined with a name and parentheses, and called like any other command.

# Function definition
create_backup() {
  local source_dir=$1  # 'local' restricts variable scope to the function
  local backup_name="backup___MATH_INLINE_0__source_dir").tgz"
  tar -czf "__MATH_INLINE_1__source_dir"
  echo "Backup created: $backup_name"
}

# Function call
create_backup "/home/user/documents"

Using local for variables inside functions prevents them from accidentally overwriting global variables.

Every command returns an exit code (0 for success, non-zero for failure). You can check this code in the special variable $? and use it for error handling. The exit command terminates the script with a given code.

grep "error" /var/log/syslog
if [[ $? -eq 0 ]]; then
  echo "Errors found in syslog."
  exit 1
fi

mkdir /tmp/backup && cp -r data /tmp/backup || echo "Backup failed!" >&2

Common Pitfalls

1. Unquoted Variables: Forgetting to quote variables is the top cause of bugs. Always use double quotes around variable expansions: "$myvar". Without quotes, word-splitting and globbing will occur on the variable's value, causing commands to break on filenames with spaces or wildcard characters to be unexpectedly expanded.

• Incorrect: rm $old_files
• Correct: rm "__MATH_INLINE_2__old_files; do rm "$f"; done

1. Ignoring Exit Codes: Assuming a command always succeeds leads to silent failures. Check critical commands' exit status or use set -e at the script's start to make it exit immediately on any error (though use this with caution, as some commands may return non-zero exit codes benignly).

1. Overlooking the Shebang: Running a script without a proper shebang (#!/bin/bash) means it will execute in the user's current shell, which could be sh, dash, or zsh. These shells have different features, leading to syntax errors. Always specify the interpreter explicitly.

1. Forgetting Execute Permissions: A script that runs with bash script.sh but fails with ./script.sh likely lacks execute permissions. Remember chmod +x is required to run a script as a direct command.

Summary

• Shell scripting is the art of automating command-line tasks by writing them into executable text files, primarily using the bash shell.
• Scripts combine core programming constructs—variables for data, conditionals (if/else) and loops (for/while) for logic, and functions for reusable code—to manage files, automate deployment, and monitor systems.
• The foundation of any script is the shebang (#!/bin/bash) and execute permissions (chmod +x), which allow the system to run it correctly.
• Robust scripts handle errors by checking exit codes ($?) and always quote variable expansions to prevent parsing errors caused by spaces or special characters.
• This knowledge is indispensable for DevOps, system administration, and general developer productivity, turning complex, multi-step procedures into simple, repeatable commands.