Sunday, March 04, 2007

10 useful bash utils

Bash Cures Cancer
Learn the UNIX/Linux command line
Home Search Man Pages RSS Feed Email: SpamDefeator

Title: 10 Linux commands you've never used
Published: 02-19-2007


It takes years maybe decades to master the commands available to you at the Linux shell prompt. Here are 10 that you will have never heard of or used. They are in no particular order. My favorite is mkfifo.

  1. pgrep, instead of:
    # ps -ef | egrep '^root ' | awk '{print $2}'
    1
    2
    3
    4
    5
    20
    21
    38
    39
    ...

    You can do this:

    # pgrep -u root
    1
    2
    3
    4
    5
    20
    21
    38
    39
    ...
  2. pstree, list the processes in a tree format. This can be VERY useful when working with WebSphere or other heavy duty applications.
    # pstree
    init-+-acpid
    |-atd
    |-crond
    |-cups-config-dae
    |-cupsd
    |-dbus-daemon-1
    |-dhclient
    |-events/0-+-aio/0
    | |-kacpid
    | |-kauditd
    | |-kblockd/0
    | |-khelper
    | |-kmirrord
    | `-2*[pdflush]
    |-gpm
    |-hald
    |-khubd
    |-2*[kjournald]
    |-klogd
    |-kseriod
    |-ksoftirqd/0
    |-kswapd0
    |-login---bash
    |-5*[mingetty]
    |-portmap
    |-rpc.idmapd
    |-rpc.statd
    |-2*[sendmail]
    |-smartd
    |-sshd---sshd---bash---pstree
    |-syslogd
    |-udevd
    |-vsftpd
    |-xfs
    `-xinetd
  3. bc is an arbitrary precision calculator language. Which is great. I found it useful in that it can perform square root operations in shell scripts. expr does not support square roots.
    # ./sqrt
    Usage: sqrt number
    # ./sqrt 64
    8
    # ./sqrt 132112
    363
    # ./sqrt 1321121321
    36347

    Here is the script:

    # cat sqrt
    #!/bin/bash
    if [ $# -ne 1 ]
    then
    echo 'Usage: sqrt number'
    exit 1
    else
    echo -e "sqrt($1)\nquit\n" | bc -q -i
    fi
  4. split, have a large file that you need to split into smaller chucks? A mysqldump maybe? split is your command. Below I split a 250MB file into 2 megabyte chunks all starting with the prefix LF_.
    # ls -lh largefile
    -rw-r--r-- 1 root root 251M Feb 19 10:27 largefile
    # split -b 2m largefile LF_
    # ls -lh LF_* | head -n 5
    -rw-r--r-- 1 root root 2.0M Feb 19 10:29 LF_aa
    -rw-r--r-- 1 root root 2.0M Feb 19 10:29 LF_ab
    -rw-r--r-- 1 root root 2.0M Feb 19 10:29 LF_ac
    -rw-r--r-- 1 root root 2.0M Feb 19 10:29 LF_ad
    -rw-r--r-- 1 root root 2.0M Feb 19 10:29 LF_ae
    # ls -lh LF_* | wc -l
    126
  5. nl numbers lines. I had a script doing this for me for years until I found out about nl.
    # head wireless.h
    /*
    * This file define a set of standard wireless extensions
    *
    * Version : 20 17.2.06
    *
    * Authors : Jean Tourrilhes - HPL
    * Copyright (c) 1997-2006 Jean Tourrilhes, All Rights Reserved.
    */

    #ifndef _LINUX_WIRELESS_H
    # nl wireless.h | head
    1 /*
    2 * This file define a set of standard wireless extensions
    3 *
    4 * Version : 20 17.2.06
    5 *
    6 * Authors : Jean Tourrilhes - HPL
    7 * Copyright (c) 1997-2006 Jean Tourrilhes, All Rights Reserved.
    8 */

    9 #ifndef _LINUX_WIRELESS_H
  6. mkfifo is the coolest one. Sure you know how to create a pipeline piping the output of grep to less or maybe even perl. But do you know how to make two commands communicate through a named pipe?

    First let me create the pipe and start writing to it:

    pipe" title="Making the pipe and writing to it" class="screenshot" src="http://bashcurescancer.com/media/10-linux-commands-youve-never-used/mkfifo-write-to-pipe.png" height="357" width="640">

    Then read from it:

    cat pipe
  7. ldd, want to know which Linux thread library java is linked to?
    # ldd /usr/java/jre1.5.0_11/bin/java
    libpthread.so.0 => /lib/tls/libpthread.so.0 (0x00bd4000)
    libdl.so.2 => /lib/libdl.so.2 (0x00b87000)
    libc.so.6 => /lib/tls/libc.so.6 (0x00a5a000)
    /lib/ld-linux.so.2 (0x00a3c000)
  8. col, want to save man pages as plain text?
    # PAGER=cat
    # man less | col -b > less.txt
  9. xmlwf, need to know if a XML document is well formed? (A configuration file maybe..)
    # curl -s 'http://bashcurescancer.com' > bcc.html
    # xmlwf bcc.html
    # perl -i -pe 's@
    @
    @g' bcc.html
    # xmlwf bcc.html
    bcc.html:104:2: mismatched tag
  10. lsof lists open files. You can do all kinds of cool things with this. Like find which ports are open:
    # lsof | grep TCP
    portmap 2587 rpc 4u IPv4 5544 TCP *:sunrpc (LISTEN)
    rpc.statd 2606 root 6u IPv4 5585 TCP *:668 (LISTEN)
    sshd 2788 root 3u IPv6 5991 TCP *:ssh (LISTEN)
    sendmail 2843 root 4u IPv4 6160 TCP badhd:smtp (LISTEN)
    vsftpd 9337 root 3u IPv4 34949 TCP *:ftp (LISTEN)
    cupsd 16459 root 0u IPv4 41061 TCP badhd:ipp (LISTEN)
    sshd 16892 root 3u IPv6 61003 TCP badhd.mshome.net:ssh->kontiki.mshome.net:4661 (ESTABLISHED)

    Note: OpenBSD 101 pointed out that "lsof -i TCP" a better way to obtain this same information. Thanks!


    Or find the number of open files a user has. Very important for running big applications like Oracle, DB2, or WebSphere:

    # lsof | grep ' root ' | awk '{print $NF}' | sort | uniq | wc -l
    179

    Note: an anonymous commenter pointed out that you can replace sort | uniq with "sort -u". This is true, I forgot about the -u flag. Thanks!

No comments: