bash.md

   1 ---
   2 title: Bash scripting
   3 category: CLI
   4 layout: 2017/sheet
   5 tags: [Featured]
   6 updated: 2020-07-04
   7 keywords:
   8   - Variables
   9   - Functions
  10   - Interpolation
  11   - Brace expansions
  12   - Loops
  13   - Conditional execution
  14   - Command substitution
  15 ---
  16
  17 Getting started
  18 ---------------
  19 {: .-three-column}
  20
  21 ### Example
  22
  23 ```bash
  24 #!/usr/bin/env bash
  25
  26 NAME="John"
  27 echo "Hello $NAME!"
  28 ```
  29
  30 ### Variables
  31
  32 ```bash
  33 NAME="John"
  34 echo $NAME
  35 echo "$NAME"
  36 echo "${NAME}!"
  37 ```
  38
  39 ### String quotes
  40
  41 ```bash
  42 NAME="John"
  43 echo "Hi $NAME"  #=> Hi John
  44 echo 'Hi $NAME'  #=> Hi $NAME
  45 ```
  46
  47 ### Shell execution
  48
  49 ```bash
  50 echo "I'm in $(pwd)"
  51 echo "I'm in `pwd`"
  52 # Same
  53 ```
  54
  55 See [Command substitution](http://wiki.bash-hackers.org/syntax/expansion/cmdsubst)
  56
  57 ### Conditional execution
  58
  59 ```bash
  60 git commit && git push
  61 git commit || echo "Commit failed"
  62 ```
  63
  64 ### Functions
  65 {: id='functions-example'}
  66
  67 ```bash
  68 get_name() {
  69   echo "John"
  70 }
  71
  72 echo "You are $(get_name)"
  73 ```
  74
  75 See: [Functions](#functions)
  76
  77 ### Conditionals
  78 {: id='conditionals-example'}
  79
  80 ```bash
  81 if [[ -z "$string" ]]; then
  82   echo "String is empty"
  83 elif [[ -n "$string" ]]; then
  84   echo "String is not empty"
  85 fi
  86 ```
  87
  88 See: [Conditionals](#conditionals)
  89
  90 ### Strict mode
  91
  92 ```bash
  93 set -euo pipefail
  94 IFS=$'\n\t'
  95 ```
  96
  97 See: [Unofficial bash strict mode](http://redsymbol.net/articles/unofficial-bash-strict-mode/)
  98
  99 ### Brace expansion
 100
 101 ```bash
 102 echo {A,B}.js
 103 ```
 104
 105 | Expression | Description         |
 106 | ---------- | ------------------- |
 107 | `{A,B}`    | Same as `A B`       |
 108 | `{A,B}.js` | Same as `A.js B.js` |
 109 | `{1..5}`   | Same as `1 2 3 4 5` |
 110
 111 See: [Brace expansion](http://wiki.bash-hackers.org/syntax/expansion/brace)
 112
 113
 114 Parameter expansions
 115 --------------------
 116 {: .-three-column}
 117
 118 ### Basics
 119
 120 ```bash
 121 name="John"
 122 echo ${name}
 123 echo ${name/J/j}    #=> "john" (substitution)
 124 echo ${name:0:2}    #=> "Jo" (slicing)
 125 echo ${name::2}     #=> "Jo" (slicing)
 126 echo ${name::-1}    #=> "Joh" (slicing)
 127 echo ${name:(-1)}   #=> "n" (slicing from right)
 128 echo ${name:(-2):1} #=> "h" (slicing from right)
 129 echo ${food:-Cake}  #=> $food or "Cake"
 130 ```
 131
 132 ```bash
 133 length=2
 134 echo ${name:0:length}  #=> "Jo"
 135 ```
 136
 137 See: [Parameter expansion](http://wiki.bash-hackers.org/syntax/pe)
 138
 139 ```bash
 140 STR="/path/to/foo.cpp"
 141 echo ${STR%.cpp}    # /path/to/foo
 142 echo ${STR%.cpp}.o  # /path/to/foo.o
 143 echo ${STR%/*}      # /path/to
 144
 145 echo ${STR##*.}     # cpp (extension)
 146 echo ${STR##*/}     # foo.cpp (basepath)
 147
 148 echo ${STR#*/}      # path/to/foo.cpp
 149 echo ${STR##*/}     # foo.cpp
 150
 151 echo ${STR/foo/bar} # /path/to/bar.cpp
 152 ```
 153
 154 ```bash
 155 STR="Hello world"
 156 echo ${STR:6:5}   # "world"
 157 echo ${STR:-5:5}  # "world"
 158 ```
 159
 160 ```bash
 161 SRC="/path/to/foo.cpp"
 162 BASE=${SRC##*/}   #=> "foo.cpp" (basepath)
 163 DIR=${SRC%$BASE}  #=> "/path/to/" (dirpath)
 164 ```
 165
 166 ### Substitution
 167
 168 | Code              | Description         |
 169 | ----------------- | ------------------- |
 170 | `${FOO%suffix}`   | Remove suffix       |
 171 | `${FOO#prefix}`   | Remove prefix       |
 172 | ---               | ---                 |
 173 | `${FOO%%suffix}`  | Remove long suffix  |
 174 | `${FOO##prefix}`  | Remove long prefix  |
 175 | ---               | ---                 |
 176 | `${FOO/from/to}`  | Replace first match |
 177 | `${FOO//from/to}` | Replace all         |
 178 | ---               | ---                 |
 179 | `${FOO/%from/to}` | Replace suffix      |
 180 | `${FOO/#from/to}` | Replace prefix      |
 181
 182 ### Comments
 183
 184 ```bash
 185 # Single line comment
 186 ```
 187
 188 ```bash
 189 : '
 190 This is a
 191 multi line
 192 comment
 193 '
 194 ```
 195
 196 ### Substrings
 197
 198 | Expression      | Description                    |
 199 | --------------- | ------------------------------ |
 200 | `${FOO:0:3}`    | Substring _(position, length)_ |
 201 | `${FOO:(-3):3}` | Substring from the right       |
 202
 203 ### Length
 204
 205 | Expression | Description      |
 206 | ---------- | ---------------- |
 207 | `${#FOO}`  | Length of `$FOO` |
 208
 209 ### Manipulation
 210
 211 ```bash
 212 STR="HELLO WORLD!"
 213 echo ${STR,}   #=> "hELLO WORLD!" (lowercase 1st letter)
 214 echo ${STR,,}  #=> "hello world!" (all lowercase)
 215
 216 STR="hello world!"
 217 echo ${STR^}   #=> "Hello world!" (uppercase 1st letter)
 218 echo ${STR^^}  #=> "HELLO WORLD!" (all uppercase)
 219 ```
 220
 221 ### Default values
 222
 223 | Expression        | Description                                              |
 224 | ----------------- | -------------------------------------------------------- |
 225 | `${FOO:-val}`     | `$FOO`, or `val` if unset (or null)                      |
 226 | `${FOO:=val}`     | Set `$FOO` to `val` if unset (or null)                   |
 227 | `${FOO:+val}`     | `val` if `$FOO` is set (and not null)                    |
 228 | `${FOO:?message}` | Show error message and exit if `$FOO` is unset (or null) |
 229
 230 Omitting the `:` removes the (non)nullity checks, e.g. `${FOO-val}` expands to `val` if unset otherwise `$FOO`.
 231
 232 Loops
 233 -----
 234 {: .-three-column}
 235
 236 ### Basic for loop
 237
 238 ```bash
 239 for i in /etc/rc.*; do
 240   echo $i
 241 done
 242 ```
 243
 244 ### C-like for loop
 245
 246 ```bash
 247 for ((i = 0 ; i < 100 ; i++)); do
 248   echo $i
 249 done
 250 ```
 251
 252 ### Ranges
 253
 254 ```bash
 255 for i in {1..5}; do
 256     echo "Welcome $i"
 257 done
 258 ```
 259
 260 #### With step size
 261
 262 ```bash
 263 for i in {5..50..5}; do
 264     echo "Welcome $i"
 265 done
 266 ```
 267
 268 ### Reading lines
 269
 270 ```bash
 271 cat file.txt | while read line; do
 272   echo $line
 273 done
 274 ```
 275
 276 ### Forever
 277
 278 ```bash
 279 while true; do
 280   ···
 281 done
 282 ```
 283
 284 Functions
 285 ---------
 286 {: .-three-column}
 287
 288 ### Defining functions
 289
 290 ```bash
 291 myfunc() {
 292     echo "hello $1"
 293 }
 294 ```
 295
 296 ```bash
 297 # Same as above (alternate syntax)
 298 function myfunc() {
 299     echo "hello $1"
 300 }
 301 ```
 302
 303 ```bash
 304 myfunc "John"
 305 ```
 306
 307 ### Returning values
 308
 309 ```bash
 310 myfunc() {
 311     local myresult='some value'
 312     echo $myresult
 313 }
 314 ```
 315
 316 ```bash
 317 result="$(myfunc)"
 318 ```
 319
 320 ### Raising errors
 321
 322 ```bash
 323 myfunc() {
 324   return 1
 325 }
 326 ```
 327
 328 ```bash
 329 if myfunc; then
 330   echo "success"
 331 else
 332   echo "failure"
 333 fi
 334 ```
 335
 336 ### Arguments
 337
 338 | Expression | Description                            |
 339 | ---        | ---                                    |
 340 | `$#`       | Number of arguments                    |
 341 | `$*`       | All arguments                          |
 342 | `$@`       | All arguments, starting from first     |
 343 | `$1`       | First argument                         |
 344 | `$_`       | Last argument of the previous command  |
 345
 346 See [Special parameters](http://wiki.bash-hackers.org/syntax/shellvars#special_parameters_and_shell_variables).
 347
 348 Conditionals
 349 ------------
 350 {: .-three-column}
 351
 352 ### Conditions
 353
 354 Note that `[[` is actually a command/program that returns either `0` (true) or `1` (false). Any program that obeys the same logic (like all base utils, such as `grep(1)` or `ping(1)`) can be used as condition, see examples.
 355
 356 | Condition                | Description           |
 357 | ---                      | ---                   |
 358 | `[[ -z STRING ]]`        | Empty string          |
 359 | `[[ -n STRING ]]`        | Not empty string      |
 360 | `[[ STRING == STRING ]]` | Equal                 |
 361 | `[[ STRING != STRING ]]` | Not Equal             |
 362 | ---                      | ---                   |
 363 | `[[ NUM -eq NUM ]]`      | Equal                 |
 364 | `[[ NUM -ne NUM ]]`      | Not equal             |
 365 | `[[ NUM -lt NUM ]]`      | Less than             |
 366 | `[[ NUM -le NUM ]]`      | Less than or equal    |
 367 | `[[ NUM -gt NUM ]]`      | Greater than          |
 368 | `[[ NUM -ge NUM ]]`      | Greater than or equal |
 369 | ---                      | ---                   |
 370 | `[[ STRING =~ STRING ]]` | Regexp                |
 371 | ---                      | ---                   |
 372 | `(( NUM < NUM ))`        | Numeric conditions    |
 373
 374 #### More conditions
 375
 376 | Condition            | Description              |
 377 | -------------------- | ------------------------ |
 378 | `[[ -o noclobber ]]` | If OPTIONNAME is enabled |
 379 | ---                  | ---                      |
 380 | `[[ ! EXPR ]]`       | Not                      |
 381 | `[[ X && Y ]]`       | And                      |
 382 | `[[ X || Y ]]`       | Or                       |
 383
 384 ### File conditions
 385
 386 | Condition               | Description             |
 387 | ---                     | ---                     |
 388 | `[[ -e FILE ]]`         | Exists                  |
 389 | `[[ -r FILE ]]`         | Readable                |
 390 | `[[ -h FILE ]]`         | Symlink                 |
 391 | `[[ -d FILE ]]`         | Directory               |
 392 | `[[ -w FILE ]]`         | Writable                |
 393 | `[[ -s FILE ]]`         | Size is > 0 bytes       |
 394 | `[[ -f FILE ]]`         | File                    |
 395 | `[[ -x FILE ]]`         | Executable              |
 396 | ---                     | ---                     |
 397 | `[[ FILE1 -nt FILE2 ]]` | 1 is more recent than 2 |
 398 | `[[ FILE1 -ot FILE2 ]]` | 2 is more recent than 1 |
 399 | `[[ FILE1 -ef FILE2 ]]` | Same files              |
 400
 401 ### Example
 402
 403 ```bash
 404 # String
 405 if [[ -z "$string" ]]; then
 406   echo "String is empty"
 407 elif [[ -n "$string" ]]; then
 408   echo "String is not empty"
 409 else
 410   echo "This never happens"
 411 fi
 412 ```
 413
 414 ```bash
 415 # Combinations
 416 if [[ X && Y ]]; then
 417   ...
 418 fi
 419 ```
 420
 421 ```bash
 422 # Equal
 423 if [[ "$A" == "$B" ]]
 424 ```
 425
 426 ```bash
 427 # Regex
 428 if [[ "A" =~ . ]]
 429 ```
 430
 431 ```bash
 432 if (( $a < $b )); then
 433    echo "$a is smaller than $b"
 434 fi
 435 ```
 436
 437 ```bash
 438 if [[ -e "file.txt" ]]; then
 439   echo "file exists"
 440 fi
 441 ```
 442
 443 Arrays
 444 ------
 445
 446 ### Defining arrays
 447
 448 ```bash
 449 Fruits=('Apple' 'Banana' 'Orange')
 450 ```
 451
 452 ```bash
 453 Fruits[0]="Apple"
 454 Fruits[1]="Banana"
 455 Fruits[2]="Orange"
 456 ```
 457
 458 ### Working with arrays
 459
 460 ```bash
 461 echo ${Fruits[0]}           # Element #0
 462 echo ${Fruits[-1]}          # Last element
 463 echo ${Fruits[@]}           # All elements, space-separated
 464 echo ${#Fruits[@]}          # Number of elements
 465 echo ${#Fruits}             # String length of the 1st element
 466 echo ${#Fruits[3]}          # String length of the Nth element
 467 echo ${Fruits[@]:3:2}       # Range (from position 3, length 2)
 468 echo ${!Fruits[@]}          # Keys of all elements, space-separated
 469 ```
 470
 471 ### Operations
 472
 473 ```bash
 474 Fruits=("${Fruits[@]}" "Watermelon")    # Push
 475 Fruits+=('Watermelon')                  # Also Push
 476 Fruits=( ${Fruits[@]/Ap*/} )            # Remove by regex match
 477 unset Fruits[2]                         # Remove one item
 478 Fruits=("${Fruits[@]}")                 # Duplicate
 479 Fruits=("${Fruits[@]}" "${Veggies[@]}") # Concatenate
 480 lines=(`cat "logfile"`)                 # Read from file
 481 ```
 482
 483 ### Iteration
 484
 485 ```bash
 486 for i in "${arrayName[@]}"; do
 487   echo $i
 488 done
 489 ```
 490
 491 Dictionaries
 492 ------------
 493 {: .-three-column}
 494
 495 ### Defining
 496
 497 ```bash
 498 declare -A sounds
 499 ```
 500
 501 ```bash
 502 sounds[dog]="bark"
 503 sounds[cow]="moo"
 504 sounds[bird]="tweet"
 505 sounds[wolf]="howl"
 506 ```
 507
 508 Declares `sound` as a Dictionary object (aka associative array).
 509
 510 ### Working with dictionaries
 511
 512 ```bash
 513 echo ${sounds[dog]} # Dog's sound
 514 echo ${sounds[@]}   # All values
 515 echo ${!sounds[@]}  # All keys
 516 echo ${#sounds[@]}  # Number of elements
 517 unset sounds[dog]   # Delete dog
 518 ```
 519
 520 ### Iteration
 521
 522 #### Iterate over values
 523
 524 ```bash
 525 for val in "${sounds[@]}"; do
 526   echo $val
 527 done
 528 ```
 529
 530 #### Iterate over keys
 531
 532 ```bash
 533 for key in "${!sounds[@]}"; do
 534   echo $key
 535 done
 536 ```
 537
 538 Options
 539 -------
 540
 541 ### Options
 542
 543 ```bash
 544 set -o noclobber  # Avoid overlay files (echo "hi" > foo)
 545 set -o errexit    # Used to exit upon error, avoiding cascading errors
 546 set -o pipefail   # Unveils hidden failures
 547 set -o nounset    # Exposes unset variables
 548 ```
 549
 550 ### Glob options
 551
 552 ```bash
 553 shopt -s nullglob    # Non-matching globs are removed  ('*.foo' => '')
 554 shopt -s failglob    # Non-matching globs throw errors
 555 shopt -s nocaseglob  # Case insensitive globs
 556 shopt -s dotglob     # Wildcards match dotfiles ("*.sh" => ".foo.sh")
 557 shopt -s globstar    # Allow ** for recursive matches ('lib/**/*.rb' => 'lib/a/b/c.rb')
 558 ```
 559
 560 Set `GLOBIGNORE` as a colon-separated list of patterns to be removed from glob
 561 matches.
 562
 563 History
 564 -------
 565
 566 ### Commands
 567
 568 | Command               | Description                               |
 569 | --------------------- | ----------------------------------------- |
 570 | `history`             | Show history                              |
 571 | `shopt -s histverify` | Don't execute expanded result immediately |
 572
 573 ### Expansions
 574
 575 | Expression   | Description                                          |
 576 | ------------ | ---------------------------------------------------- |
 577 | `!$`         | Expand last parameter of most recent command         |
 578 | `!*`         | Expand all parameters of most recent command         |
 579 | `!-n`        | Expand `n`th most recent command                     |
 580 | `!n`         | Expand `n`th command in history                      |
 581 | `!<command>` | Expand most recent invocation of command `<command>` |
 582
 583 ### Operations
 584
 585 | Code                 | Description                                                           |
 586 | -------------------- | --------------------------------------------------------------------- |
 587 | `!!`                 | Execute last command again                                            |
 588 | `!!:s/<FROM>/<TO>/`  | Replace first occurrence of `<FROM>` to `<TO>` in most recent command |
 589 | `!!:gs/<FROM>/<TO>/` | Replace all occurrences of `<FROM>` to `<TO>` in most recent command  |
 590 | `!$:t`               | Expand only basename from last parameter of most recent command       |
 591 | `!$:h`               | Expand only directory from last parameter of most recent command      |
 592
 593 `!!` and `!$` can be replaced with any valid expansion.
 594
 595 ### Slices
 596
 597 | Code     | Description                                                                              |
 598 | -------- | ---------------------------------------------------------------------------------------- |
 599 | `!!:n`   | Expand only `n`th token from most recent command (command is `0`; first argument is `1`) |
 600 | `!^`     | Expand first argument from most recent command                                           |
 601 | `!$`     | Expand last token from most recent command                                               |
 602 | `!!:n-m` | Expand range of tokens from most recent command                                          |
 603 | `!!:n-$` | Expand `n`th token to last from most recent command                                      |
 604
 605 `!!` can be replaced with any valid expansion i.e. `!cat`, `!-2`, `!42`, etc.
 606
 607
 608 Miscellaneous
 609 -------------
 610
 611 ### Numeric calculations
 612
 613 ```bash
 614 $((a + 200))      # Add 200 to $a
 615 ```
 616
 617 ```bash
 618 $((RANDOM%=200))  # Random number 0..200
 619 ```
 620
 621 ### Subshells
 622
 623 ```bash
 624 (cd somedir; echo "I'm now in $PWD")
 625 pwd # still in first directory
 626 ```
 627
 628 ### Redirection
 629
 630 ```bash
 631 python hello.py > output.txt   # stdout to (file)
 632 python hello.py >> output.txt  # stdout to (file), append
 633 python hello.py 2> error.log   # stderr to (file)
 634 python hello.py 2>&1           # stderr to stdout
 635 python hello.py 2>/dev/null    # stderr to (null)
 636 python hello.py &>/dev/null    # stdout and stderr to (null)
 637 ```
 638
 639 ```bash
 640 python hello.py < foo.txt      # feed foo.txt to stdin for python
 641 ```
 642
 643 ### Inspecting commands
 644
 645 ```bash
 646 command -V cd
 647 #=> "cd is a function/alias/whatever"
 648 ```
 649
 650 ### Trap errors
 651
 652 ```bash
 653 trap 'echo Error at about $LINENO' ERR
 654 ```
 655
 656 or
 657
 658 ```bash
 659 traperr() {
 660   echo "ERROR: ${BASH_SOURCE[1]} at about ${BASH_LINENO[0]}"
 661 }
 662
 663 set -o errtrace
 664 trap traperr ERR
 665 ```
 666
 667 ### Case/switch
 668
 669 ```bash
 670 case "$1" in
 671   start | up)
 672     vagrant up
 673     ;;
 674
 675   *)
 676     echo "Usage: $0 {start|stop|ssh}"
 677     ;;
 678 esac
 679 ```
 680
 681 ### Source relative
 682
 683 ```bash
 684 source "${0%/*}/../share/foo.sh"
 685 ```
 686
 687 ### printf
 688
 689 ```bash
 690 printf "Hello %s, I'm %s" Sven Olga
 691 #=> "Hello Sven, I'm Olga
 692
 693 printf "1 + 1 = %d" 2
 694 #=> "1 + 1 = 2"
 695
 696 printf "This is how you print a float: %f" 2
 697 #=> "This is how you print a float: 2.000000"
 698 ```
 699
 700 ### Directory of script
 701
 702 ```bash
 703 DIR="${0%/*}"
 704 ```
 705
 706 ### Getting options
 707
 708 ```bash
 709 while [[ "$1" =~ ^- && ! "$1" == "--" ]]; do case $1 in
 710   -V | --version )
 711     echo $version
 712     exit
 713     ;;
 714   -s | --string )
 715     shift; string=$1
 716     ;;
 717   -f | --flag )
 718     flag=1
 719     ;;
 720 esac; shift; done
 721 if [[ "$1" == '--' ]]; then shift; fi
 722 ```
 723
 724 ### Heredoc
 725
 726 ```sh
 727 cat <<END
 728 hello world
 729 END
 730 ```
 731
 732 ### Reading input
 733
 734 ```bash
 735 echo -n "Proceed? [y/n]: "
 736 read ans
 737 echo $ans
 738 ```
 739
 740 ```bash
 741 read -n 1 ans    # Just one character
 742 ```
 743
 744 ### Special variables
 745
 746 | Expression | Description                  |
 747 | ---------- | ---------------------------- |
 748 | `$?`       | Exit status of last task     |
 749 | `$!`       | PID of last background task  |
 750 | `$$`       | PID of shell                 |
 751 | `$0`       | Filename of the shell script |
 752
 753 See [Special parameters](http://wiki.bash-hackers.org/syntax/shellvars#special_parameters_and_shell_variables).
 754
 755 ### Go to previous directory
 756
 757 ```bash
 758 pwd # /home/user/foo
 759 cd bar/
 760 pwd # /home/user/foo/bar
 761 cd -
 762 pwd # /home/user/foo
 763 ```
 764
 765 ### Check for command's result
 766
 767 ```bash
 768 if ping -c 1 google.com; then
 769   echo "It appears you have a working internet connection"
 770 fi
 771 ```
 772
 773 ### Grep check
 774
 775 ```bash
 776 if grep -q 'foo' ~/.bash_history; then
 777   echo "You appear to have typed 'foo' in the past"
 778 fi
 779 ```
 780
 781 ## Also see
 782 {: .-one-column}
 783
 784 * [Bash-hackers wiki](http://wiki.bash-hackers.org/) _(bash-hackers.org)_
 785 * [Shell vars](http://wiki.bash-hackers.org/syntax/shellvars) _(bash-hackers.org)_
 786 * [Learn bash in y minutes](https://learnxinyminutes.com/docs/bash/) _(learnxinyminutes.com)_
 787 * [Bash Guide](http://mywiki.wooledge.org/BashGuide) _(mywiki.wooledge.org)_
 788 * [ShellCheck](https://www.shellcheck.net/) _(shellcheck.net)_