bash.md

   1 ---
   2 title: Bash scripting
   3 category: CLI
   4 layout: 2017/sheet
   5 tags: [Featured]
   6 updated: 2019-10-02
   7 keywords:
   8   - Variables
   9   - Functions
  10   - Interpolation
  11   - Brace expansions
  12   - Loops
  13   - Conditional execution
  14   - Command substitution
  15 ---
  16
  17 Getting started
  18 ---------------
  19 {: .-three-column}
  20
  21 ### Example
  22
  23 ```bash
  24 #!/usr/bin/env bash
  25
  26 NAME="John"
  27 echo "Hello $NAME!"
  28 ```
  29
  30 ### Variables
  31
  32 ```bash
  33 NAME="John"
  34 echo $NAME
  35 echo "$NAME"
  36 echo "${NAME}!"
  37 ```
  38
  39 ### String quotes
  40
  41 ```bash
  42 NAME="John"
  43 echo "Hi $NAME"  #=> Hi John
  44 echo 'Hi $NAME'  #=> Hi $NAME
  45 ```
  46
  47 ### Shell execution
  48
  49 ```bash
  50 echo "I'm in $(pwd)"
  51 echo "I'm in `pwd`"
  52 # Same
  53 ```
  54
  55 See [Command substitution](http://wiki.bash-hackers.org/syntax/expansion/cmdsubst)
  56
  57 ### Conditional execution
  58
  59 ```bash
  60 git commit && git push
  61 git commit || echo "Commit failed"
  62 ```
  63
  64 ### Functions
  65 {: id='functions-example'}
  66
  67 ```bash
  68 get_name() {
  69   echo "John"
  70 }
  71
  72 echo "You are $(get_name)"
  73 ```
  74
  75 See: [Functions](#functions)
  76
  77 ### Conditionals
  78 {: id='conditionals-example'}
  79
  80 ```bash
  81 if [[ -z "$string" ]]; then
  82   echo "String is empty"
  83 elif [[ -n "$string" ]]; then
  84   echo "String is not empty"
  85 fi
  86 ```
  87
  88 See: [Conditionals](#conditionals)
  89
  90 ### Strict mode
  91
  92 ```bash
  93 set -euo pipefail
  94 IFS=$'\n\t'
  95 ```
  96
  97 See: [Unofficial bash strict mode](http://redsymbol.net/articles/unofficial-bash-strict-mode/)
  98
  99 ### Brace expansion
 100
 101 ```bash
 102 echo {A,B}.js
 103 ```
 104
 105 | `{A,B}` | Same as `A B` |
 106 | `{A,B}.js` | Same as `A.js B.js` |
 107 | `{1..5}` | Same as `1 2 3 4 5` |
 108
 109 See: [Brace expansion](http://wiki.bash-hackers.org/syntax/expansion/brace)
 110
 111
 112 Parameter expansions
 113 --------------------
 114 {: .-three-column}
 115
 116 ### Basics
 117
 118 ```bash
 119 name="John"
 120 echo ${name}
 121 echo ${name/J/j}    #=> "john" (substitution)
 122 echo ${name:0:2}    #=> "Jo" (slicing)
 123 echo ${name::2}     #=> "Jo" (slicing)
 124 echo ${name::-1}    #=> "Joh" (slicing)
 125 echo ${name:(-1)}   #=> "n" (slicing from right)
 126 echo ${name:(-2):1} #=> "h" (slicing from right)
 127 echo ${food:-Cake}  #=> $food or "Cake"
 128 ```
 129
 130 ```bash
 131 length=2
 132 echo ${name:0:length}  #=> "Jo"
 133 ```
 134
 135 See: [Parameter expansion](http://wiki.bash-hackers.org/syntax/pe)
 136
 137 ```bash
 138 STR="/path/to/foo.cpp"
 139 echo ${STR%.cpp}    # /path/to/foo
 140 echo ${STR%.cpp}.o  # /path/to/foo.o
 141
 142 echo ${STR##*.}     # cpp (extension)
 143 echo ${STR##*/}     # foo.cpp (basepath)
 144
 145 echo ${STR#*/}      # path/to/foo.cpp
 146 echo ${STR##*/}     # foo.cpp
 147
 148 echo ${STR/foo/bar} # /path/to/bar.cpp
 149 ```
 150
 151 ```bash
 152 STR="Hello world"
 153 echo ${STR:6:5}   # "world"
 154 echo ${STR:-5:5}  # "world"
 155 ```
 156
 157 ```bash
 158 SRC="/path/to/foo.cpp"
 159 BASE=${SRC##*/}   #=> "foo.cpp" (basepath)
 160 DIR=${SRC%$BASE}  #=> "/path/to/" (dirpath)
 161 ```
 162
 163 ### Substitution
 164
 165 | Code | Description |
 166 | --- | --- |
 167 | `${FOO%suffix}` | Remove suffix |
 168 | `${FOO#prefix}` | Remove prefix |
 169 | --- | --- |
 170 | `${FOO%%suffix}` | Remove long suffix |
 171 | `${FOO##prefix}` | Remove long prefix |
 172 | --- | --- |
 173 | `${FOO/from/to}` | Replace first match |
 174 | `${FOO//from/to}` | Replace all |
 175 | --- | --- |
 176 | `${FOO/%from/to}` | Replace suffix |
 177 | `${FOO/#from/to}` | Replace prefix |
 178
 179 ### Comments
 180
 181 ```bash
 182 # Single line comment
 183 ```
 184
 185 ```bash
 186 : '
 187 This is a
 188 multi line
 189 comment
 190 '
 191 ```
 192
 193 ### Substrings
 194
 195 | `${FOO:0:3}`  | Substring _(position, length)_ |
 196 | `${FOO:-3:3}` | Substring from the right |
 197
 198 ### Length
 199
 200 | `${#FOO}` | Length of `$FOO` |
 201
 202 ### Manipulation
 203
 204 ```bash
 205 STR="HELLO WORLD!"
 206 echo ${STR,}   #=> "hELLO WORLD!" (lowercase 1st letter)
 207 echo ${STR,,}  #=> "hello world!" (all lowercase)
 208
 209 STR="hello world!"
 210 echo ${STR^}   #=> "Hello world!" (uppercase 1st letter)
 211 echo ${STR^^}  #=> "HELLO WORLD!" (all uppercase)
 212 ```
 213
 214
 215 ### Default values
 216
 217 | `${FOO:-val}`        | `$FOO`, or `val` if not set |
 218 | `${FOO:=val}`        | Set `$FOO` to `val` if not set |
 219 | `${FOO:+val}`        | `val` if `$FOO` is set |
 220 | `${FOO:?message}`    | Show error message and exit if `$FOO` is not set |
 221
 222 The `:` is optional (eg, `${FOO=word}` works)
 223
 224 Loops
 225 -----
 226 {: .-three-column}
 227
 228 ### Basic for loop
 229
 230 ```bash
 231 for i in /etc/rc.*; do
 232   echo $i
 233 done
 234 ```
 235
 236 ### C-like for loop
 237
 238 ```bash
 239 for ((i = 0 ; i < 100 ; i++)); do
 240   echo $i
 241 done
 242 ```
 243
 244 ### Ranges
 245
 246 ```bash
 247 for i in {1..5}; do
 248     echo "Welcome $i"
 249 done
 250 ```
 251
 252 #### With step size
 253
 254 ```bash
 255 for i in {5..50..5}; do
 256     echo "Welcome $i"
 257 done
 258 ```
 259
 260 ### Reading lines
 261
 262 ```bash
 263 < file.txt | while read line; do
 264   echo $line
 265 done
 266 ```
 267
 268 ### Forever
 269
 270 ```bash
 271 while true; do
 272   ···
 273 done
 274 ```
 275
 276 Functions
 277 ---------
 278 {: .-three-column}
 279
 280 ### Defining functions
 281
 282 ```bash
 283 myfunc() {
 284     echo "hello $1"
 285 }
 286 ```
 287
 288 ```bash
 289 # Same as above (alternate syntax)
 290 function myfunc() {
 291     echo "hello $1"
 292 }
 293 ```
 294
 295 ```bash
 296 myfunc "John"
 297 ```
 298
 299 ### Returning values
 300
 301 ```bash
 302 myfunc() {
 303     local myresult='some value'
 304     echo $myresult
 305 }
 306 ```
 307
 308 ```bash
 309 result="$(myfunc)"
 310 ```
 311
 312 ### Raising errors
 313
 314 ```bash
 315 myfunc() {
 316   return 1
 317 }
 318 ```
 319
 320 ```bash
 321 if myfunc; then
 322   echo "success"
 323 else
 324   echo "failure"
 325 fi
 326 ```
 327
 328 ### Arguments
 329
 330 | Expression | Description                        |
 331 | ---        | ---                                |
 332 | `$#`       | Number of arguments                |
 333 | `$*`       | All arguments                      |
 334 | `$@`       | All arguments, starting from first |
 335 | `$1`       | First argument                     |
 336
 337 See [Special parameters](http://wiki.bash-hackers.org/syntax/shellvars#special_parameters_and_shell_variables).
 338
 339 Conditionals
 340 ------------
 341 {: .-three-column}
 342
 343 ### Conditions
 344
 345 Note that `[[` is actually a command/program that returns either `0` (true) or `1` (false). Any program that obeys the same logic (like all base utils, such as `grep(1)` or `ping(1)`) can be used as condition, see examples.
 346
 347 | Condition                | Description           |
 348 | ---                      | ---                   |
 349 | `[[ -z STRING ]]`        | Empty string          |
 350 | `[[ -n STRING ]]`        | Not empty string      |
 351 | `[[ STRING == STRING ]]` | Equal                 |
 352 | `[[ STRING != STRING ]]` | Not Equal             |
 353 | ---                      | ---                   |
 354 | `[[ NUM -eq NUM ]]`      | Equal                 |
 355 | `[[ NUM -ne NUM ]]`      | Not equal             |
 356 | `[[ NUM -lt NUM ]]`      | Less than             |
 357 | `[[ NUM -le NUM ]]`      | Less than or equal    |
 358 | `[[ NUM -gt NUM ]]`      | Greater than          |
 359 | `[[ NUM -ge NUM ]]`      | Greater than or equal |
 360 | ---                      | ---                   |
 361 | `[[ STRING =~ STRING ]]` | Regexp                |
 362 | ---                      | ---                   |
 363 | `(( NUM < NUM ))`        | Numeric conditions    |
 364
 365 | Condition              | Description              |
 366 | ---                    | ---                      |
 367 | `[[ -o noclobber ]]`   | If OPTIONNAME is enabled |
 368 | ---                    | ---                      |
 369 | `[[ ! EXPR ]]`         | Not                      |
 370 | `[[ X ]] && [[ Y ]]`   | And                      |
 371 | `[[ X ]] || [[ Y ]]`   | Or                       |
 372
 373 ### File conditions
 374
 375 | Condition               | Description             |
 376 | ---                     | ---                     |
 377 | `[[ -e FILE ]]`         | Exists                  |
 378 | `[[ -r FILE ]]`         | Readable                |
 379 | `[[ -h FILE ]]`         | Symlink                 |
 380 | `[[ -d FILE ]]`         | Directory               |
 381 | `[[ -w FILE ]]`         | Writable                |
 382 | `[[ -s FILE ]]`         | Size is > 0 bytes       |
 383 | `[[ -f FILE ]]`         | File                    |
 384 | `[[ -x FILE ]]`         | Executable              |
 385 | ---                     | ---                     |
 386 | `[[ FILE1 -nt FILE2 ]]` | 1 is more recent than 2 |
 387 | `[[ FILE1 -ot FILE2 ]]` | 2 is more recent than 1 |
 388 | `[[ FILE1 -ef FILE2 ]]` | Same files              |
 389
 390 ### Example
 391
 392 ```bash
 393 # String
 394 if [[ -z "$string" ]]; then
 395   echo "String is empty"
 396 elif [[ -n "$string" ]]; then
 397   echo "String is not empty"
 398 fi
 399 ```
 400
 401 ```bash
 402 # Combinations
 403 if [[ X ]] && [[ Y ]]; then
 404   ...
 405 fi
 406 ```
 407
 408 ```bash
 409 # Equal
 410 if [[ "$A" == "$B" ]]
 411 ```
 412
 413 ```bash
 414 # Regex
 415 if [[ "A" =~ "." ]]
 416 ```
 417
 418 ```bash
 419 if (( $a < $b )); then
 420    echo "$a is smaller than $b"
 421 fi
 422 ```
 423
 424 ```bash
 425 if [[ -e "file.txt" ]]; then
 426   echo "file exists"
 427 fi
 428 ```
 429
 430 Arrays
 431 ------
 432
 433 ### Defining arrays
 434
 435 ```bash
 436 Fruits=('Apple' 'Banana' 'Orange')
 437 ```
 438
 439 ```bash
 440 Fruits[0]="Apple"
 441 Fruits[1]="Banana"
 442 Fruits[2]="Orange"
 443 ```
 444
 445 ### Working with arrays
 446
 447 ```bash
 448 echo ${Fruits[0]}           # Element #0
 449 echo ${Fruits[@]}           # All elements, space-separated
 450 echo ${#Fruits[@]}          # Number of elements
 451 echo ${#Fruits}             # String length of the 1st element
 452 echo ${#Fruits[3]}          # String length of the Nth element
 453 echo ${Fruits[@]:3:2}       # Range (from position 3, length 2)
 454 ```
 455
 456 ### Operations
 457
 458 ```bash
 459 Fruits=("${Fruits[@]}" "Watermelon")    # Push
 460 Fruits+=('Watermelon')                  # Also Push
 461 Fruits=( ${Fruits[@]/Ap*/} )            # Remove by regex match
 462 unset Fruits[2]                         # Remove one item
 463 Fruits=("${Fruits[@]}")                 # Duplicate
 464 Fruits=("${Fruits[@]}" "${Veggies[@]}") # Concatenate
 465 lines=(`cat "logfile"`)                 # Read from file
 466 ```
 467
 468 ### Iteration
 469
 470 ```bash
 471 for i in "${arrayName[@]}"; do
 472   echo $i
 473 done
 474 ```
 475
 476 Dictionaries
 477 ------------
 478 {: .-three-column}
 479
 480 ### Defining
 481
 482 ```bash
 483 declare -A sounds
 484 ```
 485
 486 ```bash
 487 sounds[dog]="bark"
 488 sounds[cow]="moo"
 489 sounds[bird]="tweet"
 490 sounds[wolf]="howl"
 491 ```
 492
 493 Declares `sound` as a Dictionary object (aka associative array).
 494
 495 ### Working with dictionaries
 496
 497 ```bash
 498 echo ${sounds[dog]} # Dog's sound
 499 echo ${sounds[@]}   # All values
 500 echo ${!sounds[@]}  # All keys
 501 echo ${#sounds[@]}  # Number of elements
 502 unset sounds[dog]   # Delete dog
 503 ```
 504
 505 ### Iteration
 506
 507 #### Iterate over values
 508
 509 ```bash
 510 for val in "${sounds[@]}"; do
 511   echo $val
 512 done
 513 ```
 514
 515 #### Iterate over keys
 516
 517 ```bash
 518 for key in "${!sounds[@]}"; do
 519   echo $key
 520 done
 521 ```
 522
 523 Options
 524 -------
 525
 526 ### Options
 527
 528 ```bash
 529 set -o noclobber  # Avoid overlay files (echo "hi" > foo)
 530 set -o errexit    # Used to exit upon error, avoiding cascading errors
 531 set -o pipefail   # Unveils hidden failures
 532 set -o nounset    # Exposes unset variables
 533 ```
 534
 535 ### Glob options
 536
 537 ```bash
 538 shopt -s nullglob    # Non-matching globs are removed  ('*.foo' => '')
 539 shopt -s failglob    # Non-matching globs throw errors
 540 shopt -s nocaseglob  # Case insensitive globs
 541 shopt -s dotglob     # Wildcards match dotfiles ("*.sh" => ".foo.sh")
 542 shopt -s globstar    # Allow ** for recursive matches ('lib/**/*.rb' => 'lib/a/b/c.rb')
 543 ```
 544
 545 Set `GLOBIGNORE` as a colon-separated list of patterns to be removed from glob
 546 matches.
 547
 548 History
 549 -------
 550
 551 ### Commands
 552
 553 | `history` | Show history |
 554 | `shopt -s histverify` | Don't execute expanded result immediately |
 555
 556 ### Expansions
 557
 558 | `!$` | Expand last parameter of most recent command |
 559 | `!*` | Expand all parameters of most recent command |
 560 | `!-n` | Expand `n`th most recent command |
 561 | `!n` | Expand `n`th command in history |
 562 | `!<command>` | Expand most recent invocation of command `<command>` |
 563
 564 ### Operations
 565
 566 | `!!` | Execute last command again |
 567 | `!!:s/<FROM>/<TO>/` | Replace first occurrence of `<FROM>` to `<TO>` in most recent command |
 568 | `!!:gs/<FROM>/<TO>/` | Replace all occurrences of `<FROM>` to `<TO>` in most recent command |
 569 | `!$:t` | Expand only basename from last parameter of most recent command |
 570 | `!$:h` | Expand only directory from last parameter of most recent command |
 571
 572 `!!` and `!$` can be replaced with any valid expansion.
 573
 574 ### Slices
 575
 576 | `!!:n` | Expand only `n`th token from most recent command (command is `0`; first argument is `1`) |
 577 | `!^` | Expand first argument from most recent command |
 578 | `!$` | Expand last token from most recent command |
 579 | `!!:n-m` | Expand range of tokens from most recent command |
 580 | `!!:n-$` | Expand `n`th token to last from most recent command |
 581
 582 `!!` can be replaced with any valid expansion i.e. `!cat`, `!-2`, `!42`, etc.
 583
 584
 585 Miscellaneous
 586 -------------
 587
 588 ### Numeric calculations
 589
 590 ```bash
 591 $((a + 200))      # Add 200 to $a
 592 ```
 593
 594 ```bash
 595 $((RANDOM%=200))  # Random number 0..200
 596 ```
 597
 598 ### Subshells
 599
 600 ```bash
 601 (cd somedir; echo "I'm now in $PWD")
 602 pwd # still in first directory
 603 ```
 604
 605 ### Redirection
 606
 607 ```bash
 608 python hello.py > output.txt   # stdout to (file)
 609 python hello.py >> output.txt  # stdout to (file), append
 610 python hello.py 2> error.log   # stderr to (file)
 611 python hello.py 2>&1           # stderr to stdout
 612 python hello.py 2>/dev/null    # stderr to (null)
 613 python hello.py &>/dev/null    # stdout and stderr to (null)
 614 ```
 615
 616 ```bash
 617 python hello.py < foo.txt      # feed foo.txt to stdin for python
 618 ```
 619
 620 ### Inspecting commands
 621
 622 ```bash
 623 command -V cd
 624 #=> "cd is a function/alias/whatever"
 625 ```
 626
 627 ### Trap errors
 628
 629 ```bash
 630 trap 'echo Error at about $LINENO' ERR
 631 ```
 632
 633 or
 634
 635 ```bash
 636 traperr() {
 637   echo "ERROR: ${BASH_SOURCE[1]} at about ${BASH_LINENO[0]}"
 638 }
 639
 640 set -o errtrace
 641 trap traperr ERR
 642 ```
 643
 644 ### Case/switch
 645
 646 ```bash
 647 case "$1" in
 648   start | up)
 649     vagrant up
 650     ;;
 651
 652   *)
 653     echo "Usage: $0 {start|stop|ssh}"
 654     ;;
 655 esac
 656 ```
 657
 658 ### Source relative
 659
 660 ```bash
 661 source "${0%/*}/../share/foo.sh"
 662 ```
 663
 664 ### printf
 665
 666 ```bash
 667 printf "Hello %s, I'm %s" Sven Olga
 668 #=> "Hello Sven, I'm Olga
 669
 670 printf "1 + 1 = %d" 2
 671 #=> "1 + 1 = 2"
 672
 673 printf "This is how you print a float: %f" 2
 674 #=> "This is how you print a float: 2.000000"
 675 ```
 676
 677 ### Directory of script
 678
 679 ```bash
 680 DIR="${0%/*}"
 681 ```
 682
 683 ### Getting options
 684
 685 ```bash
 686 while [[ "$1" =~ ^- && ! "$1" == "--" ]]; do case $1 in
 687   -V | --version )
 688     echo $version
 689     exit
 690     ;;
 691   -s | --string )
 692     shift; string=$1
 693     ;;
 694   -f | --flag )
 695     flag=1
 696     ;;
 697 esac; shift; done
 698 if [[ "$1" == '--' ]]; then shift; fi
 699 ```
 700
 701 ### Heredoc
 702
 703 ```sh
 704 cat <<END
 705 hello world
 706 END
 707 ```
 708
 709 ### Reading input
 710
 711 ```bash
 712 echo -n "Proceed? [y/n]: "
 713 read ans
 714 echo $ans
 715 ```
 716
 717 ```bash
 718 read -n 1 ans    # Just one character
 719 ```
 720
 721 ### Special variables
 722
 723 | `$?` | Exit status of last task |
 724 | `$!` | PID of last background task |
 725 | `$$` | PID of shell |
 726 | `$0` | Filename of the shell script |
 727
 728 See [Special parameters](http://wiki.bash-hackers.org/syntax/shellvars#special_parameters_and_shell_variables).
 729
 730 ### Go to previous directory
 731
 732 ```bash
 733 pwd # /home/user/foo
 734 cd bar/
 735 pwd # /home/user/foo/bar
 736 cd -
 737 pwd # /home/user/foo
 738 ```
 739
 740 ### Check for command's result
 741
 742 ```bash
 743 if ping -c 1 google.com; then
 744   echo "It appears you have a working internet connection"
 745 fi
 746 ```
 747
 748 ### Grep check
 749
 750 ```bash
 751 if grep -q 'foo' ~/.bash_history; then
 752   echo "You appear to have typed 'foo' in the past"
 753 fi
 754 ```
 755
 756 ## Also see
 757 {: .-one-column}
 758
 759 * [Bash-hackers wiki](http://wiki.bash-hackers.org/) _(bash-hackers.org)_
 760 * [Shell vars](http://wiki.bash-hackers.org/syntax/shellvars) _(bash-hackers.org)_
 761 * [Learn bash in y minutes](https://learnxinyminutes.com/docs/bash/) _(learnxinyminutes.com)_
 762 * [Bash Guide](http://mywiki.wooledge.org/BashGuide) _(mywiki.wooledge.org)_
 763 * [ShellCheck](https://www.shellcheck.net/) _(shellcheck.net)_