Shell Expansion
The program a line starts never sees the line you typed. Before ls gets a single argument, the shell has replaced each variable with its value, cut the unquoted values at their blanks, swapped each pattern for the file names it matches, and opened and copied file descriptors for every > and 2>&1, strictly left to right. The order is the whole subject. Put a variable holding *.txt twice through it and ls is handed every .txt file twice. Put 2>&1 on the wrong side of >file and the errors never reach the file.
New to the shell? Start here
A shell reads a line, works out which program to run and with what, and starts it. The program receives a list of strings, its arguments, with its own name first. It never sees the line you typed, the quotes in it, or the variables.
A running program reads and writes through small numbers called file descriptors: by agreement 0 is where it reads, 1 where it prints and 2 where it complains. Each is an entry in a table the kernel keeps for the process, pointing at an open file, a pipe or the terminal, and two entries can point at the same open file.
dup2(a, b) makes entry b point wherever entry a points, closing whatever b pointed at first. Every redirection a shell performs is an open followed by a dup2, a dup2 alone, or a close for n>&-; a pipe is made by pipe() and joined with one more dup2. Each command in a pipeline then runs as its own process, usually a fork of the shell (bash -c with one command execs in place), and is started by a system call; each descriptor names an open file, which names an inode.
Machines here that come first: The System Call, The Inode.
Five stages between the line you type and the program that runs
1 Expand: every $name outside single quotes and not after a backslash becomes its value
A shell reads a line and runs a program, and the program never sees the line. It sees a list of strings, built in stages. The first replaces each variable with what it holds. The Seventh Edition manual of January 1979 begins its account with “The character $ is used to introduce” what it calls substitutable parameters. A value goes in as text and nothing else happens to it yet, but the shell remembers which characters came out of a variable and which were inside quotes, because the next two stages treat them differently.
The variables, and the directory the line runs in
The directory, which every pattern is matched against:
typed quoted from a variable, unquoted from a variable, inside quotes
2 Split: what unquoted expansions produced is cut at its blanks
Splitting takes the characters from stage one and cuts them at blanks, but only blanks that came out of an unquoted variable. Blanks you typed already separated the words, and blanks inside quotes are kept. The 1979 manual: “After parameter and command substitution, any results of substitution are scanned for internal field separator” characters, which by default are space, tab and newline. An unquoted variable that held nothing leaves nothing behind; a quoted one leaves an empty argument. In the manual's words, explicit nulls “are retained. Implicit null arguments (those resulting from parameters that have no values) are removed.”
| word | fields |
|---|
3 Glob: each field with an unquoted *, ? or [ becomes the names it matches
Now each field holding an unquoted *, ? or [ is a pattern, and it is replaced by the names in the directory that match it. In 1979: “The word is replaced with alphabetically sorted file names that match the pattern.” If nothing matches, the field goes through as it was. “The character . at the start of a file name or immediately following a /, and the character /, must be matched explicitly.” Because this runs after splitting, a variable holding two patterns is two patterns, and each is matched on its own. A pattern is not a regular expression: * here is any run of characters, where in Regex it repeats the thing before it.
| field | a pattern? | becomes |
|---|
A redirection's target has its variables expanded too, but whether it is then split and globbed is where the shells part. POSIX: “Pathname expansion shall not be performed on the word by a non-interactive shell”, and it is not split either. The bash manual says the target goes through all of them, and “If it expands to more than one word, Bash reports an error.” Told to follow POSIX, bash agrees with dash: “Redirection operators do not perform filename expansion on the word in a redirection unless the shell is interactive.”
| redirection | in the shell chosen above | dash | bash | bash --posix |
|---|
4 Wire: the pipe, then each redirection left to right, as open and dup2
Each command in a pipeline is its own process, a fork of the shell, and each starts with the shell's three descriptors: 0 to read, 1 to print, 2 to complain. The pipe is wired first. POSIX: the standard input and output “shall be considered to be assigned by the pipeline before any redirection” the command carries. Then the redirections, and “the order of evaluation is from beginning to end”. A file target is an open, which takes the lowest free descriptor, then a dup2 onto the one named. The Seventh Edition dup(2): “Dup2 causes fildes2 to refer to the same file as fildes. If fildes2 already referred to an open file, it is closed first.” So 2>&1 copies whatever 1 is at that moment, and later changes to 1 do not follow it. The calls listed are the ones that change the table, not a full trace: dash, for one, also copies each descriptor it replaces out of the way, to put back later.
5 Exec: the fields left over are argv, and the program inherits the table
What is left of the words is the argument list, command name first, and the program is started with it by a system call, execve. The call is printed with the name and the arguments only; the real one is handed the path the shell found, such as /usr/bin/ls, and the environment as well. The program inherits the descriptor table as stage four left it, and each entry names an open file, which names an inode. None of the quoting, none of the variables and none of the redirections reach it. A redirection that failed stops the command before this stage, and its complaint goes to whatever descriptor 2 is at that moment.
What the line does to the directory:
Make one. Pick a brief, write a line in the box at the top, and check it here.
These ran in this browser when the page loaded. Each claim, whether it held, and the number behind it.
| claim | held | measured |
|---|---|---|
| $g holds "*.txt *.txt", so splitting makes two patterns before globbing, and ls is handed every .txt file twice | yes | 6 arguments, from 3 matches of *.txt |
| if globbing came first, the same line would pass ls two literal *.txt arguments and name no file | yes | *.txt, *.txt |
| quoted, "$g" reaches ls as one argument, stars and all | yes | 1 argument: *.txt *.txt |
| an unquoted empty $e disappears and a quoted one stays, as the 1979 manual says | yes | 2 arguments from three words |
| * matches no name that starts with a dot, and .* reaches "." and ".." in dash but not in bash | yes | 7 names for *; .* gives 4 in dash and 2 in bash |
| the names come back in byte order, so README sorts before a.txt | yes | README, a.txt, b.txt, ... |
| with 2 files matching *.xyz, "ls > *.xyz" creates a file literally named *.xyz in dash and in bash --posix, and bash refuses it | yes | bash: *.xyz: ambiguous redirect |
| when the pattern matches exactly one file, bash opens that file and empties it | yes | bash truncates README; dash creates a file named R* |
| >o 2>&1 leaves 1 and 2 sharing one open file, o; 2>&1 >o leaves 2 on the terminal | yes | fd 2 ends as o and as terminal (the shell's fd 1) |
| in ls 2>&1 | wc the pipe is connected before 2>&1, so ls's errors go into the pipe | yes | ls fd 2 is pipe 1, write end |
| replaying each printed list of calls on a fresh table gives the table shown, for every preset in every shell | yes | 32 tables, all equal |
| the engine reproduces every outcome recorded from dash 0.5.12-12ubuntu3 and bash 5.3.9(1)-release for the lines on this page | yes | 30 of 30 recorded outcomes |
What is real here, and what is not
What this page models, and what it refuses
From POSIX's list of expansions it models parameter expansion in two forms only, $name and ${name}, then field splitting with the default separators of space, tab and newline, then pathname expansion with *, ? and bracket expressions using ranges and !, then quote removal. It models the redirections <, >, >>, n>&m, n<&m and n>&-, on descriptors 0 to 9, and the pipe. It refuses, by name, tilde expansion, command substitution, arithmetic, every other ${...} form, the special parameters such as $@ and $?, a changed IFS, character classes such as [[:alpha:]], here-documents, <> and >|, lists, subshells, variable assignments, a backslash inside a variable's value, and any command it does not list. It never approximates one of those: the line is refused and the reason is printed.
Brace expansion is bash's, from the C shell, and it is left out
In bash, a{b,c}d becomes abd acd before anything else happens. The bash manual: “Brace expansion is performed before any other expansions”. It is not in the Seventh Edition shell or in dash, where the braces are ordinary characters. It is in the C shell's manual in the Third Berkeley Software Distribution, which describes the order it keeps: “Left to right order is preserved”. POSIX's 2024 edition makes room for an expansion of that shape without requiring one. This page treats an unquoted brace as ordinary in dash and refuses it in bash, rather than modelling a stage none of the other shells has.
Three shells, and where each difference comes from
dash 0.5.12, bash 5.3.9 and the same bash run with --posix differ in five places this engine models, each one setting, besides the braces above. A redirection target is split and globbed by bash only. A target that expands to nothing at all is an error called an ambiguous redirect in both modes of bash, while dash tries to open a file with an empty name and fails. [^a] means “not a” in bash and is an ordinary bracket holding ^ and a in dash. A pattern starting with a dot can match . and .. in dash and never does in bash 5.3. And when a duplication such as 2>&5 fails, dash has already closed descriptor 2 to set it aside for later, so its complaint about the failure goes nowhere, while bash prints it. That last one was found in the recordings, not in a manual, and strace shows the close.
Only non-interactive shells
POSIX lets an interactive shell glob a redirection target as long as the result is one word. Every line here was run with -c, which is not interactive, and nothing on this page predicts what a shell at a prompt does with one.
The outcomes the page holds itself against
The checks table compares this engine, on every preset line and in every shell, with what the real shells did with the same line: dash 0.5.12-12ubuntu3 and GNU bash 5.3.9(1)-release on Linux 7.0.0-31-generic, in the C locale, on 7 October 2026, in a directory holding exactly the nine files listed in stage one. Each command was a small program standing in for ls, wc and the rest, which recorded the arguments it was handed and its open descriptors, including which of them shared an open file, and exited. The preset lines are thirty of the outcomes recorded across the three shells; the rest of the recording is longer and is not printed here.
Nothing is run
No program is executed by this page and none of the commands it names is real here. What it computes is what a program would be handed. Whether ls would then succeed, what cat would read and what rm would delete is outside it.
A pipeline whose answer depends on timing is refused
Every command in a pipeline runs at once. If one of them creates a file that another reads, or that another's pattern would match, then whether the second sees it depends on which process gets there first. In the recordings, a file created by an earlier command did appear in a later command's pattern. The page does not pick an answer for those lines; it refuses them and names the file.
The pipe's other promise, measured on one kernel
The Seventh Edition pipe(2) promises that “Writes with a count of 4096 bytes or less are atomic”, meaning no other writer's bytes land in the middle of one. POSIX only requires PIPE_BUF to be 512. Linux's pipe(7): “On Linux, PIPE_BUF is 4096 bytes.” Measured on Linux 7.0.0-31-generic, with four separate processes each making one write to the same pipe at the same moment, 200 rounds for each size: no write of 512, 4,096, 4,097 or 65,536 bytes arrived in pieces, out of 800 at each size; at 65,537 bytes 470 of 800 did, in 188 of the 200 rounds; at 200,000 bytes 785 of 800 did. A second run gave 468 and 781. Of the sizes tried, the smallest that came apart was 65,537 bytes, one past the pipe's capacity. But every round began with an empty pipe, which takes 65,536 bytes before a writer has to wait, and above PIPE_BUF pipe(7) says only that “the kernel may interleave the data with data written by other processes”. So this is a measurement of one test on one kernel rather than a guarantee anyone makes, and not a sign that Linux keeps larger writes whole. Nothing else on this page depends on it: the pipe is drawn as one more dup2.
1979 is the date of a manual, not of an invention
The Seventh Edition manual's title page reads January, 1979, and the Plan 9 archive of the same manual gives 1979 for its copyright; the month rests on the title page alone. The shell in it is older than its manual, and globbing is older than this shell: in the Sixth Edition, the shell handed any argument containing a pattern to a separate program, of which its manual says “Glob is used to expand arguments to the shell containing” those characters. 1979 is the earliest manual this page reads that puts substitution, splitting and file name generation in one order, and nothing here says who first did any of it.
Sound: no
Asked and answered. Nothing on this page has a duration, and a click per stage would decorate a list that is already printed.
Sources
- sh(1), UNIX Programmer's Manual, Seventh Edition, January 1979. Substitution, blank interpretation and file name generation, in that order, and the input and output notation, including 2>&1.
- dup(2), Seventh Edition. dup2, and that the descriptor it replaces is closed first.
- pipe(2), Seventh Edition. The 4096-byte atomic write.
- The Seventh Edition manual's title page, in Bell Labs' own bundle of the Volume 1 introduction, mirrored at 9p.io. It prints January, 1979.
- The Seventh Edition manual as kept in the Plan 9 archive, with its 1979 copyright.
- glob(VIII), Sixth Edition. Globbing as a separate program the shell called.
- POSIX.1-2024, Shell Command Language: 2.6 Word Expansions, 2.7 Redirection, and 2.9.2 Pipelines.
- POSIX.1-2024, limits.h. PIPE_BUF, and its minimum of 512.
- Bash Reference Manual, Shell Expansions. bash's order, with brace expansion first.
- Bash Reference Manual, Redirections. The target is globbed and split, and more than one word is an error.
- Bash Reference Manual, Bash and POSIX. What --posix changes about a redirection target.
- Bash Reference Manual, Brace Expansion.
- csh(1), Third Berkeley Software Distribution, at the Unix Heritage Society. Brace notation in the C shell.
- dash(1). The pipeline is assigned before redirections, with the 2>&1 | example.
- pipe(7), the Linux man-pages project. PIPE_BUF on Linux, and the pipe's capacity.
- Logical Art, the studio this belongs to.