Combining Commands with Piping in Bash

Piping in Bash

• Piping allows the output of one command to be used as the input for another command.
• The pipe operator (|) is used to connect commands.

Basic piping examples

• ls -l | grep "txt" – This command will list all files and directories in the current directory in a long format, and then pass the output to the grep command to search for any lines containing the string “txt”.

-rw-r--r-- 1 user group    0 Jan 13 14:22 file1.txt
-rw-r--r-- 1 user group    0 Jan 13 14:22 file2.txt
-rw-r--r-- 1 user group    0 Jan 13 14:22 file3.txt

• cat file.txt | sort | uniq – This command will display the contents of a text file, pass the output to the sort command to sort the lines alphabetically, and then pass the sorted output to the uniq command to remove any duplicate lines.

File content:

apple
banana
cherry
apple
banana
cherry

Output:

apple
banana
cherry

Advanced piping examples

• ps aux | awk '{print $1}' | sort | uniq -c | sort -n – This command will list all running processes, pass the output to awk to print the first column (the username), pass the output to sort to sort the usernames alphabetically, pass the output to uniq -c to count the number of occurrences of each username, and finally pass the output to sort -n to sort the results by the count.

      1 _apt
      1 avahi
      1 bin
      1 daemon
    ...

• find / -name "*.log" 2>/dev/null | xargs grep "error" – This command will search the entire file system for files with the “.log” extension, pass the output to xargs command which will run the grep command on each file found and search for the string “error”

/var/log/syslog:Jan 13 14:22:22 hostname kernel: [  123.456234] Error: Unable to access memory
/var/log/nginx/error.log:Jan 13 14:22:22 hostname nginx: [error] server closed connection while reading request header line