Linux/UNIX Pipes File Descriptors, and Process Substitution

  • POSIX Shell – A logical container providing an environment and a set of commands to interact with a UNIX and/or UNIX-like system.
    Examples include: sh(Bourne shell), bash, ash, csh, etc.
  • Pipe – a unidirectional connection between two processes utilized for inter-process communications.
  • File Descriptor (fd) – A dynamically created, locally significant and non-persistent file representation of a FIFO buffer which can be connected to a pipe, or by default to the parent process’ standard input, output, and error buffers. As an example, a (virtual) terminal’s input device (keyboard, mouse, etc.) and output device (display).
  • Process substitution – Allows the POSIX shell user the ability create scripted constructs that mimic functional and/or imperative programming syntax in-line.

Every interactive process on startup within a POSIX shell is attached dynamically to three pipes local to its memory space. Standard Input(0), Standard Output(1), and Standard Error(2). If the POSIX system supports file descriptors they are then created and attached to the pipes at ‘/dev/fd/’.

Name Abbrev. File Symbols
standard input stdin /dev/fd/0 < or <<
standard output stdout /dev/fd/1 1> or 1>>
standard error stderr /dev/fd/2 2> or 2>>

POSIX Shell Plumbing

The POSIX (Bourne) shell and its derivatives provides the ability to utilize various symbols to plumb processes together by their file descriptors.

Symbol Name _Symbol_Syntax_ ________Function________
pipe { block } | { block } attach stdout of left script block
to  stdin of right script block
cmd && { block } if cmd exits with success (0)
then execute block
cmd || { block } if cmd exits with error (not 0)
then execute block
{ block } #>file attach /dev/fd/#
of left script block to file
and overwrite with bytestream
{ block } #>&? attach /dev/fd/#
of left script block to /dev/fd/?
and overwrite with bytestream
append { block } #>>file attach /dev/fd/#
of left script block to file
and append bytestream
#>&- close /dev/fd/#
(no # specified defaults to stdout)
{ block } <file attach stdin of left script block
to file and read bytestream
{ block } <&? attach stdin of left script block
to /dev/fd/? and read bytstream
{ block } <<word attach stdin of left script block
to stdin of proc
and read until word
close stdin <&- close stdin

Process Substitution

The POSIX shell also provides the ability to perform inline substitution utilizing two methods:

Symbol_Syntax ________Function________
sub-shell exec ( block ) execute process proc without environment persistance
sub-block exec { block; } execute proc resulting inline at location within script with evnironment persistance

…for examples see below


Basic Pipeline

find / | grep '^\/bin' | sed -r 's/^/\/usr/g;'

This is a very basic example showing the connection of stdout(1) of the find utility to the stdin(0) of the grep utility to filter all paths that start with ‘/bin’ and prepending the paths with ‘/usr’. The intent is to translate the output of find with all ‘/bin’ paths to ‘/usr/bin’.

Dynamic File Descriptors

$ FOO=who                         # SET VARIABLE FOO TO who
$ cat <(                          # READ FD GENERATED BY <( ... )
         FOO=bar ;                # SET LOCAL VARIABLE FOO TO bar
         echo $FOO ;              # PRINT LOCAL VARIABLE FOO
        )                         # DESTROY ENVIRONMENT
$ echo $FOO                       # PRINT VARIABLE IN OUTER CONTEXT
$ echo <( FOO=bar ; echo $FOO )   # PRINT FD AS FILE PATH

The above is a great example that highlights what is returned by the <( … ) is actually a file path and not the contents. It is the responsibility of the script block, command, or executable to the left to read the contents of this file descriptor. It also shows how the contents within ( … ) are executed in a completely different context preventing access to variables in the executing script and vice versa.

Local execution of a Script generated on a Remote Server

$ /bin/sh <( curl ) 

Executes cURL retrieving contents generated by ‘’ on web-server hosted at and runs the script locally without saving the file to disk first. As an added inspection into whats happening, you can see the dynamic creation of a file descriptor by utilizing the following:



{ { { tee /dev/fd/3 /dev/fd/4 | command1 >&9;
 } 3>&1 | command2 >&9;
 } 4>&1 | command3 >&9;
} 9>&1

tee /dev/fd/3 /dev/fd/4‘ – takes stdin (/dev/fd/0) and duplicates it into three separate piped paths:

  1. stdout /dev/fd/1 -> command1 -> /dev/fd/9 -> stdout (/dev/fd/1)
  2. /dev/fd/3 -> command2 -> /dev/fd/9 -> stdout (/dev/fd/1)
  3. /dev/fd/4 -> command3 -> /dev/fd/9 -> stdout (/dev/fd/1)

NOTE: stderr (/dev/fd/2) is maintained end to end without redirection.

Other Interesting tidbits


$ echo '#!/bin/sh
echo[1]: first line of imported script
echo[2]: bla bla...' >

$ echo '#!/bin/sh
echo[1]: this is the first line of the main script
echo[2]: lets inline import
echo[4]: this is the fourth line of the main script' >


$ /bin/sh ./

. filename reads a filename as if it had occurred at that place in the script..