OS: Process Creation with Fork and Exec

Understanding how an operating system creates and manages processes is fundamental to systems programming. At the heart of this in Unix-like systems are two system calls, fork() and exec(), which provide a powerful, elegant model for spawning and transforming program execution. Mastering their use and interaction is key to building everything from simple scripts to complex applications like web servers and command-line shells.

The Process Abstraction and Fork

Before diving into the mechanics, you must understand what a process is: an instance of a running program, complete with its own memory space, register state, and system resources. In Unix, the primary way to create a new process is with the fork() system call.

When a process calls fork(), the OS creates an almost identical copy, known as the child process. This child gets its own unique Process ID (PID) but is a duplicate of the parent's memory, file descriptors, and register state. Crucially, fork() returns different values to each process: it returns 0 to the child and the new child's PID to the parent. This return value is how a program determines whether it's executing in the parent or child context.

#include <unistd.h>
#include <stdio.h>

int main() {
    pid_t pid = fork(); // The pivotal moment of creation

    if (pid == 0) {
        // This code runs ONLY in the child process
        printf("Child PID: %d\n", getpid());
    } else if (pid > 0) {
        // This code runs ONLY in the parent process
        printf("Parent PID: %d, Child's PID: %d\n", getpid(), pid);
    } else {
        // fork() failed
        perror("fork");
    }
    return 0;
}

This duality allows for parallel execution paths. However, a complete copy of the parent's memory would be inefficient, especially if the child immediately replaces itself with a new program. This is where copy-on-write (COW) optimization comes in. With COW, the parent and child initially share the same physical memory pages. The OS only makes a private copy for the child when either process attempts to modify a page. This makes fork() extremely fast and memory-efficient for the common case where the child calls exec().

Replacing the Process Image with Exec

While fork() creates a new process, it doesn't load a new program. That's the job of the exec() family of functions (e.g., execlp, execvp). Exec replaces the current process's memory image—its code, data, heap, and stack—with that of a brand new program loaded from an executable file.

The key distinction is that exec() does not create a new process; it transforms the calling process. The PID remains the same, but everything else changes. If exec() is successful, it never returns, as the old program's code is gone. Control passes to the entry point of the new program.

// In a child process after fork()
execlp("ls", "ls", "-l", NULL); // Replaces child with 'ls -l'
// Any code here will never be reached unless exec fails.

The exec functions differ in how they specify the program path and arguments. execlp and execvp are convenient because they search the directories listed in the PATH environment variable.

Managing the Parent-Child Relationship with Wait

When a child process terminates, it enters a zombie state until its parent collects its exit status. The parent uses the wait() or waitpid() system call for this purpose. Wait serves two critical functions: it blocks the parent until a child terminates (or changes state), and it retrieves the child's exit code, allowing the parent to know if the child succeeded or failed.

Properly calling wait() is essential for process hygiene. Failure to do so results in zombie processes that clutter the system's process table. In a typical pattern, the parent forks a child to perform a task, the child executes a new program, and the parent waits for the child to complete.

pid_t pid = fork();
if (pid == 0) {
    // Child: execute a command
    execlp("/bin/cat", "cat", "file.txt", NULL);
    exit(1); // Only reached if exec fails
} else if (pid > 0) {
    // Parent: wait for child
    int status;
    waitpid(pid, &status, 0);
    if (WIFEXITED(status)) {
        printf("Child exited with status %d\n", WEXITSTATUS(status));
    }
}

Building Process Trees and Simple Shells

Combining fork(), exec(), and wait() allows you to build hierarchical process trees. A parent can spawn multiple children, each of which can itself fork, creating a tree of executing programs. This is precisely how a command-line shell works.

A minimal shell operates in a loop: it prints a prompt, reads a command, forks a child process, has the child execute the command via exec(), and then waits for the child to finish before prompting again. This model allows users to run programs sequentially.

To enable more powerful operations like command pipelines (ls | grep txt), shells use pipes. A pipe is a unidirectional inter-process communication channel created with the pipe() system call, which returns two file descriptors: one for reading and one for writing. The shell creates a pipe before forking. It then orchestrates the children so the standard output of the first command (e.g., ls) is connected to the pipe's write end, and the standard input of the second command (e.g., grep) is connected to the pipe's read end. This involves careful manipulation of file descriptors using dup2() before calling exec().

Common Pitfalls

1. Ignoring Return Values and Error Conditions: Always check the return value of fork() and exec(). A negative return from fork() means it failed (often due to resource limits). exec() only returns if it fails; your code must handle this by printing an error and typically exiting the child process with a non-zero code.
2. Creating Zombie Processes: Forgetting to call wait() leads to zombies. In long-running programs like servers, you must also handle the SIGCHLD signal to asynchronously reap terminated children using waitpid() with the WNOHANG option to avoid blocking.
3. File Descriptor Leaks in Children: When a child calls exec(), all open file descriptors (including pipes) remain open by default. If they are not needed by the new program, they should be explicitly closed. Conversely, failing to close the unused ends of a pipe in the correct processes can cause programs to hang, waiting for EOF.
4. Misunderstanding Address Space After Fork: Because of copy-on-write, a child initially shares memory pages with the parent. A write to a global variable in the child will create a private copy, but this change will not be visible to the parent. Processes communicate via explicit IPC mechanisms (pipes, sockets, shared memory), not through simple global variables after the fork.

Summary

• The fork() system call creates a new child process that is a duplicate of the parent, differentiated only by its PID and the return value of fork().
• The exec() family of functions replaces the calling process's memory image with a new program, transforming the process without creating a new one.
• The copy-on-write optimization makes forking efficient by deferring memory duplication until a process writes to a page.
• The parent must use wait() or waitpid() to collect a child's exit status and prevent it from becoming a zombie process.
• Combining these primitives allows you to build process trees and sophisticated tools like shells, which use pipes (pipe()) to connect the standard input/output of multiple processes.