OSDN Git Service

Regular updates
[twpd/master.git] / bash.md
1 ---
2 title: Bash scripting
3 category: CLI
4 layout: 2017/sheet
5 tags: [Featured]
6 updated: 2020-07-05
7 keywords:
8   - Variables
9   - Functions
10   - Interpolation
11   - Brace expansions
12   - Loops
13   - Conditional execution
14   - Command substitution
15 ---
16
17 Getting started
18 ---------------
19 {: .-three-column}
20
21 ### Introduction
22 {: .-intro}
23
24 This is a quick reference to getting started with Bash scripting.
25
26 - [Learn bash in y minutes](https://learnxinyminutes.com/docs/bash/) _(learnxinyminutes.com)_
27 - [Bash Guide](http://mywiki.wooledge.org/BashGuide) _(mywiki.wooledge.org)_
28
29 ### Example
30
31 ```bash
32 #!/usr/bin/env bash
33
34 NAME="John"
35 echo "Hello $NAME!"
36 ```
37
38 ### Variables
39
40 ```bash
41 NAME="John"
42 echo $NAME
43 echo "$NAME"
44 echo "${NAME}!"
45 ```
46
47 ### String quotes
48
49 ```bash
50 NAME="John"
51 echo "Hi $NAME"  #=> Hi John
52 echo 'Hi $NAME'  #=> Hi $NAME
53 ```
54
55 ### Shell execution
56
57 ```bash
58 echo "I'm in $(pwd)"
59 echo "I'm in `pwd`"
60 # Same
61 ```
62
63 See [Command substitution](http://wiki.bash-hackers.org/syntax/expansion/cmdsubst)
64
65 ### Conditional execution
66
67 ```bash
68 git commit && git push
69 git commit || echo "Commit failed"
70 ```
71
72 ### Functions
73 {: id='functions-example'}
74
75 ```bash
76 get_name() {
77   echo "John"
78 }
79
80 echo "You are $(get_name)"
81 ```
82
83 See: [Functions](#functions)
84
85 ### Conditionals
86 {: id='conditionals-example'}
87
88 ```bash
89 if [[ -z "$string" ]]; then
90   echo "String is empty"
91 elif [[ -n "$string" ]]; then
92   echo "String is not empty"
93 fi
94 ```
95
96 See: [Conditionals](#conditionals)
97
98 ### Strict mode
99
100 ```bash
101 set -euo pipefail
102 IFS=$'\n\t'
103 ```
104
105 See: [Unofficial bash strict mode](http://redsymbol.net/articles/unofficial-bash-strict-mode/)
106
107 ### Brace expansion
108
109 ```bash
110 echo {A,B}.js
111 ```
112
113 | Expression | Description         |
114 | ---------- | ------------------- |
115 | `{A,B}`    | Same as `A B`       |
116 | `{A,B}.js` | Same as `A.js B.js` |
117 | `{1..5}`   | Same as `1 2 3 4 5` |
118
119 See: [Brace expansion](http://wiki.bash-hackers.org/syntax/expansion/brace)
120
121
122 Parameter expansions
123 --------------------
124 {: .-three-column}
125
126 ### Basics
127
128 ```bash
129 name="John"
130 echo ${name}
131 echo ${name/J/j}    #=> "john" (substitution)
132 echo ${name:0:2}    #=> "Jo" (slicing)
133 echo ${name::2}     #=> "Jo" (slicing)
134 echo ${name::-1}    #=> "Joh" (slicing)
135 echo ${name:(-1)}   #=> "n" (slicing from right)
136 echo ${name:(-2):1} #=> "h" (slicing from right)
137 echo ${food:-Cake}  #=> $food or "Cake"
138 ```
139
140 ```bash
141 length=2
142 echo ${name:0:length}  #=> "Jo"
143 ```
144
145 See: [Parameter expansion](http://wiki.bash-hackers.org/syntax/pe)
146
147 ```bash
148 STR="/path/to/foo.cpp"
149 echo ${STR%.cpp}    # /path/to/foo
150 echo ${STR%.cpp}.o  # /path/to/foo.o
151 echo ${STR%/*}      # /path/to
152
153 echo ${STR##*.}     # cpp (extension)
154 echo ${STR##*/}     # foo.cpp (basepath)
155
156 echo ${STR#*/}      # path/to/foo.cpp
157 echo ${STR##*/}     # foo.cpp
158
159 echo ${STR/foo/bar} # /path/to/bar.cpp
160 ```
161
162 ```bash
163 STR="Hello world"
164 echo ${STR:6:5}   # "world"
165 echo ${STR: -5:5}  # "world"
166 ```
167
168 ```bash
169 SRC="/path/to/foo.cpp"
170 BASE=${SRC##*/}   #=> "foo.cpp" (basepath)
171 DIR=${SRC%$BASE}  #=> "/path/to/" (dirpath)
172 ```
173
174 ### Substitution
175
176 | Code              | Description         |
177 | ----------------- | ------------------- |
178 | `${FOO%suffix}`   | Remove suffix       |
179 | `${FOO#prefix}`   | Remove prefix       |
180 | ---               | ---                 |
181 | `${FOO%%suffix}`  | Remove long suffix  |
182 | `${FOO##prefix}`  | Remove long prefix  |
183 | ---               | ---                 |
184 | `${FOO/from/to}`  | Replace first match |
185 | `${FOO//from/to}` | Replace all         |
186 | ---               | ---                 |
187 | `${FOO/%from/to}` | Replace suffix      |
188 | `${FOO/#from/to}` | Replace prefix      |
189
190 ### Comments
191
192 ```bash
193 # Single line comment
194 ```
195
196 ```bash
197 : '
198 This is a
199 multi line
200 comment
201 '
202 ```
203
204 ### Substrings
205
206 | Expression      | Description                    |
207 | --------------- | ------------------------------ |
208 | `${FOO:0:3}`    | Substring _(position, length)_ |
209 | `${FOO:(-3):3}` | Substring from the right       |
210
211 ### Length
212
213 | Expression | Description      |
214 | ---------- | ---------------- |
215 | `${#FOO}`  | Length of `$FOO` |
216
217 ### Manipulation
218
219 ```bash
220 STR="HELLO WORLD!"
221 echo ${STR,}   #=> "hELLO WORLD!" (lowercase 1st letter)
222 echo ${STR,,}  #=> "hello world!" (all lowercase)
223
224 STR="hello world!"
225 echo ${STR^}   #=> "Hello world!" (uppercase 1st letter)
226 echo ${STR^^}  #=> "HELLO WORLD!" (all uppercase)
227 ```
228
229 ### Default values
230
231 | Expression        | Description                                              |
232 | ----------------- | -------------------------------------------------------- |
233 | `${FOO:-val}`     | `$FOO`, or `val` if unset (or null)                      |
234 | `${FOO:=val}`     | Set `$FOO` to `val` if unset (or null)                   |
235 | `${FOO:+val}`     | `val` if `$FOO` is set (and not null)                    |
236 | `${FOO:?message}` | Show error message and exit if `$FOO` is unset (or null) |
237
238 Omitting the `:` removes the (non)nullity checks, e.g. `${FOO-val}` expands to `val` if unset otherwise `$FOO`.
239
240 Loops
241 -----
242 {: .-three-column}
243
244 ### Basic for loop
245
246 ```bash
247 for i in /etc/rc.*; do
248   echo $i
249 done
250 ```
251
252 ### C-like for loop
253
254 ```bash
255 for ((i = 0 ; i < 100 ; i++)); do
256   echo $i
257 done
258 ```
259
260 ### Ranges
261
262 ```bash
263 for i in {1..5}; do
264     echo "Welcome $i"
265 done
266 ```
267
268 #### With step size
269
270 ```bash
271 for i in {5..50..5}; do
272     echo "Welcome $i"
273 done
274 ```
275
276 ### Reading lines
277
278 ```bash
279 cat file.txt | while read line; do
280   echo $line
281 done
282 ```
283
284 ### Forever
285
286 ```bash
287 while true; do
288   Â·Â·Â·
289 done
290 ```
291
292 Functions
293 ---------
294 {: .-three-column}
295
296 ### Defining functions
297
298 ```bash
299 myfunc() {
300     echo "hello $1"
301 }
302 ```
303
304 ```bash
305 # Same as above (alternate syntax)
306 function myfunc() {
307     echo "hello $1"
308 }
309 ```
310
311 ```bash
312 myfunc "John"
313 ```
314
315 ### Returning values
316
317 ```bash
318 myfunc() {
319     local myresult='some value'
320     echo $myresult
321 }
322 ```
323
324 ```bash
325 result="$(myfunc)"
326 ```
327
328 ### Raising errors
329
330 ```bash
331 myfunc() {
332   return 1
333 }
334 ```
335
336 ```bash
337 if myfunc; then
338   echo "success"
339 else
340   echo "failure"
341 fi
342 ```
343
344 ### Arguments
345
346 | Expression | Description                                      |
347 | ---        | ---                                              |
348 | `$#`       | Number of arguments                              |
349 | `$*`       | All positional arguments  (as a single word)     |
350 | `$@`       | All positional arguments (as separate strings)  |
351 | `$1`       | First argument                                   |
352 | `$_`       | Last argument of the previous command            |
353
354 **Note**: `$@` and `$*` must be quoted in order to perform as described.
355 Otherwise, they do exactly the same thing (arguments as separate strings).
356
357 See [Special parameters](http://wiki.bash-hackers.org/syntax/shellvars#special_parameters_and_shell_variables).
358
359 Conditionals
360 ------------
361 {: .-three-column}
362
363 ### Conditions
364
365 Note that `[[` is actually a command/program that returns either `0` (true) or `1` (false). Any program that obeys the same logic (like all base utils, such as `grep(1)` or `ping(1)`) can be used as condition, see examples.
366
367 | Condition                | Description           |
368 | ---                      | ---                   |
369 | `[[ -z STRING ]]`        | Empty string          |
370 | `[[ -n STRING ]]`        | Not empty string      |
371 | `[[ STRING == STRING ]]` | Equal                 |
372 | `[[ STRING != STRING ]]` | Not Equal             |
373 | ---                      | ---                   |
374 | `[[ NUM -eq NUM ]]`      | Equal                 |
375 | `[[ NUM -ne NUM ]]`      | Not equal             |
376 | `[[ NUM -lt NUM ]]`      | Less than             |
377 | `[[ NUM -le NUM ]]`      | Less than or equal    |
378 | `[[ NUM -gt NUM ]]`      | Greater than          |
379 | `[[ NUM -ge NUM ]]`      | Greater than or equal |
380 | ---                      | ---                   |
381 | `[[ STRING =~ STRING ]]` | Regexp                |
382 | ---                      | ---                   |
383 | `(( NUM < NUM ))`        | Numeric conditions    |
384
385 #### More conditions
386
387 | Condition            | Description              |
388 | -------------------- | ------------------------ |
389 | `[[ -o noclobber ]]` | If OPTIONNAME is enabled |
390 | ---                  | ---                      |
391 | `[[ ! EXPR ]]`       | Not                      |
392 | `[[ X && Y ]]`       | And                      |
393 | `[[ X || Y ]]`       | Or                       |
394
395 ### File conditions
396
397 | Condition               | Description             |
398 | ---                     | ---                     |
399 | `[[ -e FILE ]]`         | Exists                  |
400 | `[[ -r FILE ]]`         | Readable                |
401 | `[[ -h FILE ]]`         | Symlink                 |
402 | `[[ -d FILE ]]`         | Directory               |
403 | `[[ -w FILE ]]`         | Writable                |
404 | `[[ -s FILE ]]`         | Size is > 0 bytes       |
405 | `[[ -f FILE ]]`         | File                    |
406 | `[[ -x FILE ]]`         | Executable              |
407 | ---                     | ---                     |
408 | `[[ FILE1 -nt FILE2 ]]` | 1 is more recent than 2 |
409 | `[[ FILE1 -ot FILE2 ]]` | 2 is more recent than 1 |
410 | `[[ FILE1 -ef FILE2 ]]` | Same files              |
411
412 ### Example
413
414 ```bash
415 # String
416 if [[ -z "$string" ]]; then
417   echo "String is empty"
418 elif [[ -n "$string" ]]; then
419   echo "String is not empty"
420 else
421   echo "This never happens"
422 fi
423 ```
424
425 ```bash
426 # Combinations
427 if [[ X && Y ]]; then
428   ...
429 fi
430 ```
431
432 ```bash
433 # Equal
434 if [[ "$A" == "$B" ]]
435 ```
436
437 ```bash
438 # Regex
439 if [[ "A" =~ . ]]
440 ```
441
442 ```bash
443 if (( $a < $b )); then
444    echo "$a is smaller than $b"
445 fi
446 ```
447
448 ```bash
449 if [[ -e "file.txt" ]]; then
450   echo "file exists"
451 fi
452 ```
453
454 Arrays
455 ------
456
457 ### Defining arrays
458
459 ```bash
460 Fruits=('Apple' 'Banana' 'Orange')
461 ```
462
463 ```bash
464 Fruits[0]="Apple"
465 Fruits[1]="Banana"
466 Fruits[2]="Orange"
467 ```
468
469 ### Working with arrays
470
471 ```bash
472 echo ${Fruits[0]}           # Element #0
473 echo ${Fruits[-1]}          # Last element
474 echo ${Fruits[@]}           # All elements, space-separated
475 echo ${#Fruits[@]}          # Number of elements
476 echo ${#Fruits}             # String length of the 1st element
477 echo ${#Fruits[3]}          # String length of the Nth element
478 echo ${Fruits[@]:3:2}       # Range (from position 3, length 2)
479 echo ${!Fruits[@]}          # Keys of all elements, space-separated
480 ```
481
482 ### Operations
483
484 ```bash
485 Fruits=("${Fruits[@]}" "Watermelon")    # Push
486 Fruits+=('Watermelon')                  # Also Push
487 Fruits=( ${Fruits[@]/Ap*/} )            # Remove by regex match
488 unset Fruits[2]                         # Remove one item
489 Fruits=("${Fruits[@]}")                 # Duplicate
490 Fruits=("${Fruits[@]}" "${Veggies[@]}") # Concatenate
491 lines=(`cat "logfile"`)                 # Read from file
492 ```
493
494 ### Iteration
495
496 ```bash
497 for i in "${arrayName[@]}"; do
498   echo $i
499 done
500 ```
501
502 Dictionaries
503 ------------
504 {: .-three-column}
505
506 ### Defining
507
508 ```bash
509 declare -A sounds
510 ```
511
512 ```bash
513 sounds[dog]="bark"
514 sounds[cow]="moo"
515 sounds[bird]="tweet"
516 sounds[wolf]="howl"
517 ```
518
519 Declares `sound` as a Dictionary object (aka associative array).
520
521 ### Working with dictionaries
522
523 ```bash
524 echo ${sounds[dog]} # Dog's sound
525 echo ${sounds[@]}   # All values
526 echo ${!sounds[@]}  # All keys
527 echo ${#sounds[@]}  # Number of elements
528 unset sounds[dog]   # Delete dog
529 ```
530
531 ### Iteration
532
533 #### Iterate over values
534
535 ```bash
536 for val in "${sounds[@]}"; do
537   echo $val
538 done
539 ```
540
541 #### Iterate over keys
542
543 ```bash
544 for key in "${!sounds[@]}"; do
545   echo $key
546 done
547 ```
548
549 Options
550 -------
551
552 ### Options
553
554 ```bash
555 set -o noclobber  # Avoid overlay files (echo "hi" > foo)
556 set -o errexit    # Used to exit upon error, avoiding cascading errors
557 set -o pipefail   # Unveils hidden failures
558 set -o nounset    # Exposes unset variables
559 ```
560
561 ### Glob options
562
563 ```bash
564 shopt -s nullglob    # Non-matching globs are removed  ('*.foo' => '')
565 shopt -s failglob    # Non-matching globs throw errors
566 shopt -s nocaseglob  # Case insensitive globs
567 shopt -s dotglob     # Wildcards match dotfiles ("*.sh" => ".foo.sh")
568 shopt -s globstar    # Allow ** for recursive matches ('lib/**/*.rb' => 'lib/a/b/c.rb')
569 ```
570
571 Set `GLOBIGNORE` as a colon-separated list of patterns to be removed from glob
572 matches.
573
574 History
575 -------
576
577 ### Commands
578
579 | Command               | Description                               |
580 | --------------------- | ----------------------------------------- |
581 | `history`             | Show history                              |
582 | `shopt -s histverify` | Don't execute expanded result immediately |
583
584 ### Expansions
585
586 | Expression   | Description                                          |
587 | ------------ | ---------------------------------------------------- |
588 | `!$`         | Expand last parameter of most recent command         |
589 | `!*`         | Expand all parameters of most recent command         |
590 | `!-n`        | Expand `n`th most recent command                     |
591 | `!n`         | Expand `n`th command in history                      |
592 | `!<command>` | Expand most recent invocation of command `<command>` |
593
594 ### Operations
595
596 | Code                 | Description                                                           |
597 | -------------------- | --------------------------------------------------------------------- |
598 | `!!`                 | Execute last command again                                            |
599 | `!!:s/<FROM>/<TO>/`  | Replace first occurrence of `<FROM>` to `<TO>` in most recent command |
600 | `!!:gs/<FROM>/<TO>/` | Replace all occurrences of `<FROM>` to `<TO>` in most recent command  |
601 | `!$:t`               | Expand only basename from last parameter of most recent command       |
602 | `!$:h`               | Expand only directory from last parameter of most recent command      |
603
604 `!!` and `!$` can be replaced with any valid expansion.
605
606 ### Slices
607
608 | Code     | Description                                                                              |
609 | -------- | ---------------------------------------------------------------------------------------- |
610 | `!!:n`   | Expand only `n`th token from most recent command (command is `0`; first argument is `1`) |
611 | `!^`     | Expand first argument from most recent command                                           |
612 | `!$`     | Expand last token from most recent command                                               |
613 | `!!:n-m` | Expand range of tokens from most recent command                                          |
614 | `!!:n-$` | Expand `n`th token to last from most recent command                                      |
615
616 `!!` can be replaced with any valid expansion i.e. `!cat`, `!-2`, `!42`, etc.
617
618
619 Miscellaneous
620 -------------
621
622 ### Numeric calculations
623
624 ```bash
625 $((a + 200))      # Add 200 to $a
626 ```
627
628 ```bash
629 $(($RANDOM%200))  # Random number 0..199
630 ```
631
632 ### Subshells
633
634 ```bash
635 (cd somedir; echo "I'm now in $PWD")
636 pwd # still in first directory
637 ```
638
639 ### Redirection
640
641 ```bash
642 python hello.py > output.txt   # stdout to (file)
643 python hello.py >> output.txt  # stdout to (file), append
644 python hello.py 2> error.log   # stderr to (file)
645 python hello.py 2>&1           # stderr to stdout
646 python hello.py 2>/dev/null    # stderr to (null)
647 python hello.py &>/dev/null    # stdout and stderr to (null)
648 ```
649
650 ```bash
651 python hello.py < foo.txt      # feed foo.txt to stdin for python
652 diff <(ls -r) <(ls)            # Compare two stdout without files
653 ```
654
655 ### Inspecting commands
656
657 ```bash
658 command -V cd
659 #=> "cd is a function/alias/whatever"
660 ```
661
662 ### Trap errors
663
664 ```bash
665 trap 'echo Error at about $LINENO' ERR
666 ```
667
668 or
669
670 ```bash
671 traperr() {
672   echo "ERROR: ${BASH_SOURCE[1]} at about ${BASH_LINENO[0]}"
673 }
674
675 set -o errtrace
676 trap traperr ERR
677 ```
678
679 ### Case/switch
680
681 ```bash
682 case "$1" in
683   start | up)
684     vagrant up
685     ;;
686
687   *)
688     echo "Usage: $0 {start|stop|ssh}"
689     ;;
690 esac
691 ```
692
693 ### Source relative
694
695 ```bash
696 source "${0%/*}/../share/foo.sh"
697 ```
698
699 ### printf
700
701 ```bash
702 printf "Hello %s, I'm %s" Sven Olga
703 #=> "Hello Sven, I'm Olga
704
705 printf "1 + 1 = %d" 2
706 #=> "1 + 1 = 2"
707
708 printf "This is how you print a float: %f" 2
709 #=> "This is how you print a float: 2.000000"
710 ```
711
712 ### Transform strings
713
714 | Command option     | Description                                         |
715 | ------------------ | --------------------------------------------------- |
716 | `-c`               | Operations apply to characters not in the given set |
717 | `-d`               | Delete characters                                   |
718 | `-s`               | Replaces repeated characters with single occurrence |
719 | `-t`               | Truncates                                           |
720 | `[:upper:]`        | All upper case letters                              |
721 | `[:lower:]`        | All lower case letters                              |
722 | `[:digit:]`        | All digits                                          |
723 | `[:space:]`        | All whitespace                                      |
724 | `[:alpha:]`        | All letters                                         |
725 | `[:alnum:]`        | All letters and digits                              |
726
727 #### Example
728
729 ```bash
730 echo "Welcome To Devhints" | tr [:lower:] [:upper:]
731 WELCOME TO DEVHINTS
732 ```
733
734 ### Directory of script
735
736 ```bash
737 DIR="${0%/*}"
738 ```
739
740 ### Getting options
741
742 ```bash
743 while [[ "$1" =~ ^- && ! "$1" == "--" ]]; do case $1 in
744   -V | --version )
745     echo $version
746     exit
747     ;;
748   -s | --string )
749     shift; string=$1
750     ;;
751   -f | --flag )
752     flag=1
753     ;;
754 esac; shift; done
755 if [[ "$1" == '--' ]]; then shift; fi
756 ```
757
758 ### Heredoc
759
760 ```sh
761 cat <<END
762 hello world
763 END
764 ```
765
766 ### Reading input
767
768 ```bash
769 echo -n "Proceed? [y/n]: "
770 read ans
771 echo $ans
772 ```
773
774 ```bash
775 read -n 1 ans    # Just one character
776 ```
777
778 ### Special variables
779
780 | Expression | Description                            |
781 | ---------- | -------------------------------------- |
782 | `$?`       | Exit status of last task               |
783 | `$!`       | PID of last background task            |
784 | `$$`       | PID of shell                           |
785 | `$0`       | Filename of the shell script           |
786 | `$_`       | Last argument of the previous command  |
787
788 See [Special parameters](http://wiki.bash-hackers.org/syntax/shellvars#special_parameters_and_shell_variables).
789
790 ### Go to previous directory
791
792 ```bash
793 pwd # /home/user/foo
794 cd bar/
795 pwd # /home/user/foo/bar
796 cd -
797 pwd # /home/user/foo
798 ```
799
800 ### Check for command's result
801
802 ```bash
803 if ping -c 1 google.com; then
804   echo "It appears you have a working internet connection"
805 fi
806 ```
807
808 ### Grep check
809
810 ```bash
811 if grep -q 'foo' ~/.bash_history; then
812   echo "You appear to have typed 'foo' in the past"
813 fi
814 ```
815
816 ## Also see
817 {: .-one-column}
818
819 * [Bash-hackers wiki](http://wiki.bash-hackers.org/) _(bash-hackers.org)_
820 * [Shell vars](http://wiki.bash-hackers.org/syntax/shellvars) _(bash-hackers.org)_
821 * [Learn bash in y minutes](https://learnxinyminutes.com/docs/bash/) _(learnxinyminutes.com)_
822 * [Bash Guide](http://mywiki.wooledge.org/BashGuide) _(mywiki.wooledge.org)_
823 * [ShellCheck](https://www.shellcheck.net/) _(shellcheck.net)_