Original post is here: eklausmeier.goip.de
Process substitution is available in ksh, bash, and zsh. More examples can be found here: Process Substitution.
Process substitution uses
/dev/fd/<n>files to send the results of the process(es) within parentheses to another process.
Piping the output of one program to the input of another program is a very powerful way to run multiple programs at once without any auxiliary temporary files.
1. Sequential flow. Simple case:
1program1 | program2 | program3 > file1
[mermaid] flowchart LR A(program1) B(program2) C(program3) U@{ shape: flag, label: "file1" } A --> B --> C --> U [/mermaid]
2. Read from multiple programs. This is a quite common case.
1program1 <(program2) <(program3)
[mermaid] flowchart TD A(program1) B(program2) C(program3) B --> A C --> A [/mermaid] It looks very similar to the usual command line:
1program1 file1 file2
A real-word example of multiple programs started:
1cat <(cut -d\| -f1 WOLA_EXP_WOLA_ADRESSE_CZSWADTB.csv \
2 WOLA_EXP_WOLA_PARTYINFOS_CZSWPTTB.csv \
3 WOLA_EXP_WOLA_PARTY_BEZ_CZSWPBTB.csv \
4 WOLA_EXP_WOLA_VERFUEGUNGEN_CZSWVFTB.csv) \
5 <(cut -d\| -f7 WOLA_EXP_WOLA_GESCHAEFTS_VM_CZSWGVTB.csv | cut -b2-17) \
6 | sort -u > bpkennWOLA
3. Pipe to multiple programs. One program, here program2, takes multiple files as command-line arguments, i.e., it has multiple outputs.
1program1 | program2 >(program4 > file2) >(program5 > file3) \
2 | program3 > file1
[mermaid] flowchart LR A(program1) B(program2) C(program3) D(program4) E(program5) U@{ shape: flag, label: "file1" } V@{ shape: flag, label: "file2" } W@{ shape: flag, label: "file3" } A --> B --> C --> U B --> D --> V B --> E --> W [/mermaid]
Here is a real-world example for generating the statistics for this web-server:
1time blogconcatlog 57 | pv | tee /tmp/a2 | accesslogFilter -o >(blogstatcnt > /srv/http/statcnt.html) | tee /tmp/a1 | blogurlcnt -m70 > /srv/http/urlstat2-m100.html