OSDN Git Service

Regular updates
[twpd/master.git] / bash.md
1 ---
2 title: Bash scripting
3 category: CLI
4 layout: 2017/sheet
5 tags: [Featured]
6 updated: 2020-07-05
7 keywords:
8   - Variables
9   - Functions
10   - Interpolation
11   - Brace expansions
12   - Loops
13   - Conditional execution
14   - Command substitution
15 ---
16
17 Getting started
18 ---------------
19 {: .-three-column}
20
21 ### Introduction
22 {: .-intro}
23
24 This is a quick reference to getting started with Bash scripting.
25
26 - [Learn bash in y minutes](https://learnxinyminutes.com/docs/bash/) _(learnxinyminutes.com)_
27 - [Bash Guide](http://mywiki.wooledge.org/BashGuide) _(mywiki.wooledge.org)_
28 - [Bash Hackers Wiki](https://wiki.bash-hackers.org) _(wiki.bash-hackers.org)_
29
30 ### Example
31
32 ```bash
33 #!/usr/bin/env bash
34
35 NAME="John"
36 echo "Hello $NAME!"
37 ```
38
39 ### Variables
40
41 ```bash
42 NAME="John"
43 echo $NAME
44 echo "$NAME"
45 echo "${NAME}!"
46 ```
47
48 ### String quotes
49
50 ```bash
51 NAME="John"
52 echo "Hi $NAME"  #=> Hi John
53 echo 'Hi $NAME'  #=> Hi $NAME
54 ```
55
56 ### Shell execution
57
58 ```bash
59 echo "I'm in $(pwd)"
60 echo "I'm in `pwd`"
61 # Same
62 ```
63
64 See [Command substitution](http://wiki.bash-hackers.org/syntax/expansion/cmdsubst)
65
66 ### Conditional execution
67
68 ```bash
69 git commit && git push
70 git commit || echo "Commit failed"
71 ```
72
73 ### Functions
74 {: id='functions-example'}
75
76 ```bash
77 get_name() {
78   echo "John"
79 }
80
81 echo "You are $(get_name)"
82 ```
83
84 See: [Functions](#functions)
85
86 ### Conditionals
87 {: id='conditionals-example'}
88
89 ```bash
90 if [[ -z "$string" ]]; then
91   echo "String is empty"
92 elif [[ -n "$string" ]]; then
93   echo "String is not empty"
94 fi
95 ```
96
97 See: [Conditionals](#conditionals)
98
99 ### Strict mode
100
101 ```bash
102 set -euo pipefail
103 IFS=$'\n\t'
104 ```
105
106 See: [Unofficial bash strict mode](http://redsymbol.net/articles/unofficial-bash-strict-mode/)
107
108 ### Brace expansion
109
110 ```bash
111 echo {A,B}.js
112 ```
113
114 | Expression | Description         |
115 | ---------- | ------------------- |
116 | `{A,B}`    | Same as `A B`       |
117 | `{A,B}.js` | Same as `A.js B.js` |
118 | `{1..5}`   | Same as `1 2 3 4 5` |
119
120 See: [Brace expansion](http://wiki.bash-hackers.org/syntax/expansion/brace)
121
122
123 Parameter expansions
124 --------------------
125 {: .-three-column}
126
127 ### Basics
128
129 ```bash
130 name="John"
131 echo ${name}
132 echo ${name/J/j}    #=> "john" (substitution)
133 echo ${name:0:2}    #=> "Jo" (slicing)
134 echo ${name::2}     #=> "Jo" (slicing)
135 echo ${name::-1}    #=> "Joh" (slicing)
136 echo ${name:(-1)}   #=> "n" (slicing from right)
137 echo ${name:(-2):1} #=> "h" (slicing from right)
138 echo ${food:-Cake}  #=> $food or "Cake"
139 ```
140
141 ```bash
142 length=2
143 echo ${name:0:length}  #=> "Jo"
144 ```
145
146 See: [Parameter expansion](http://wiki.bash-hackers.org/syntax/pe)
147
148 ```bash
149 STR="/path/to/foo.cpp"
150 echo ${STR%.cpp}    # /path/to/foo
151 echo ${STR%.cpp}.o  # /path/to/foo.o
152 echo ${STR%/*}      # /path/to
153
154 echo ${STR##*.}     # cpp (extension)
155 echo ${STR##*/}     # foo.cpp (basepath)
156
157 echo ${STR#*/}      # path/to/foo.cpp
158 echo ${STR##*/}     # foo.cpp
159
160 echo ${STR/foo/bar} # /path/to/bar.cpp
161 ```
162
163 ```bash
164 STR="Hello world"
165 echo ${STR:6:5}   # "world"
166 echo ${STR: -5:5}  # "world"
167 ```
168
169 ```bash
170 SRC="/path/to/foo.cpp"
171 BASE=${SRC##*/}   #=> "foo.cpp" (basepath)
172 DIR=${SRC%$BASE}  #=> "/path/to/" (dirpath)
173 ```
174
175 ### Substitution
176
177 | Code              | Description         |
178 | ----------------- | ------------------- |
179 | `${FOO%suffix}`   | Remove suffix       |
180 | `${FOO#prefix}`   | Remove prefix       |
181 | ---               | ---                 |
182 | `${FOO%%suffix}`  | Remove long suffix  |
183 | `${FOO##prefix}`  | Remove long prefix  |
184 | ---               | ---                 |
185 | `${FOO/from/to}`  | Replace first match |
186 | `${FOO//from/to}` | Replace all         |
187 | ---               | ---                 |
188 | `${FOO/%from/to}` | Replace suffix      |
189 | `${FOO/#from/to}` | Replace prefix      |
190
191 ### Comments
192
193 ```bash
194 # Single line comment
195 ```
196
197 ```bash
198 : '
199 This is a
200 multi line
201 comment
202 '
203 ```
204
205 ### Substrings
206
207 | Expression      | Description                    |
208 | --------------- | ------------------------------ |
209 | `${FOO:0:3}`    | Substring _(position, length)_ |
210 | `${FOO:(-3):3}` | Substring from the right       |
211
212 ### Length
213
214 | Expression | Description      |
215 | ---------- | ---------------- |
216 | `${#FOO}`  | Length of `$FOO` |
217
218 ### Manipulation
219
220 ```bash
221 STR="HELLO WORLD!"
222 echo ${STR,}   #=> "hELLO WORLD!" (lowercase 1st letter)
223 echo ${STR,,}  #=> "hello world!" (all lowercase)
224
225 STR="hello world!"
226 echo ${STR^}   #=> "Hello world!" (uppercase 1st letter)
227 echo ${STR^^}  #=> "HELLO WORLD!" (all uppercase)
228 ```
229
230 ### Default values
231
232 | Expression        | Description                                              |
233 | ----------------- | -------------------------------------------------------- |
234 | `${FOO:-val}`     | `$FOO`, or `val` if unset (or null)                      |
235 | `${FOO:=val}`     | Set `$FOO` to `val` if unset (or null)                   |
236 | `${FOO:+val}`     | `val` if `$FOO` is set (and not null)                    |
237 | `${FOO:?message}` | Show error message and exit if `$FOO` is unset (or null) |
238
239 Omitting the `:` removes the (non)nullity checks, e.g. `${FOO-val}` expands to `val` if unset otherwise `$FOO`.
240
241 Loops
242 -----
243 {: .-three-column}
244
245 ### Basic for loop
246
247 ```bash
248 for i in /etc/rc.*; do
249   echo $i
250 done
251 ```
252
253 ### C-like for loop
254
255 ```bash
256 for ((i = 0 ; i < 100 ; i++)); do
257   echo $i
258 done
259 ```
260
261 ### Ranges
262
263 ```bash
264 for i in {1..5}; do
265     echo "Welcome $i"
266 done
267 ```
268
269 #### With step size
270
271 ```bash
272 for i in {5..50..5}; do
273     echo "Welcome $i"
274 done
275 ```
276
277 ### Reading lines
278
279 ```bash
280 cat file.txt | while read line; do
281   echo $line
282 done
283 ```
284
285 ### Forever
286
287 ```bash
288 while true; do
289   Â·Â·Â·
290 done
291 ```
292
293 Functions
294 ---------
295 {: .-three-column}
296
297 ### Defining functions
298
299 ```bash
300 myfunc() {
301     echo "hello $1"
302 }
303 ```
304
305 ```bash
306 # Same as above (alternate syntax)
307 function myfunc() {
308     echo "hello $1"
309 }
310 ```
311
312 ```bash
313 myfunc "John"
314 ```
315
316 ### Returning values
317
318 ```bash
319 myfunc() {
320     local myresult='some value'
321     echo $myresult
322 }
323 ```
324
325 ```bash
326 result="$(myfunc)"
327 ```
328
329 ### Raising errors
330
331 ```bash
332 myfunc() {
333   return 1
334 }
335 ```
336
337 ```bash
338 if myfunc; then
339   echo "success"
340 else
341   echo "failure"
342 fi
343 ```
344
345 ### Arguments
346
347 | Expression | Description                                      |
348 | ---        | ---                                              |
349 | `$#`       | Number of arguments                              |
350 | `$*`       | All positional arguments  (as a single word)     |
351 | `$@`       | All positional arguments (as separate strings)  |
352 | `$1`       | First argument                                   |
353 | `$_`       | Last argument of the previous command            |
354
355 **Note**: `$@` and `$*` must be quoted in order to perform as described.
356 Otherwise, they do exactly the same thing (arguments as separate strings).
357
358 See [Special parameters](http://wiki.bash-hackers.org/syntax/shellvars#special_parameters_and_shell_variables).
359
360 Conditionals
361 ------------
362 {: .-three-column}
363
364 ### Conditions
365
366 Note that `[[` is actually a command/program that returns either `0` (true) or `1` (false). Any program that obeys the same logic (like all base utils, such as `grep(1)` or `ping(1)`) can be used as condition, see examples.
367
368 | Condition                | Description           |
369 | ---                      | ---                   |
370 | `[[ -z STRING ]]`        | Empty string          |
371 | `[[ -n STRING ]]`        | Not empty string      |
372 | `[[ STRING == STRING ]]` | Equal                 |
373 | `[[ STRING != STRING ]]` | Not Equal             |
374 | ---                      | ---                   |
375 | `[[ NUM -eq NUM ]]`      | Equal                 |
376 | `[[ NUM -ne NUM ]]`      | Not equal             |
377 | `[[ NUM -lt NUM ]]`      | Less than             |
378 | `[[ NUM -le NUM ]]`      | Less than or equal    |
379 | `[[ NUM -gt NUM ]]`      | Greater than          |
380 | `[[ NUM -ge NUM ]]`      | Greater than or equal |
381 | ---                      | ---                   |
382 | `[[ STRING =~ STRING ]]` | Regexp                |
383 | ---                      | ---                   |
384 | `(( NUM < NUM ))`        | Numeric conditions    |
385
386 #### More conditions
387
388 | Condition            | Description              |
389 | -------------------- | ------------------------ |
390 | `[[ -o noclobber ]]` | If OPTIONNAME is enabled |
391 | ---                  | ---                      |
392 | `[[ ! EXPR ]]`       | Not                      |
393 | `[[ X && Y ]]`       | And                      |
394 | `[[ X || Y ]]`       | Or                       |
395
396 ### File conditions
397
398 | Condition               | Description             |
399 | ---                     | ---                     |
400 | `[[ -e FILE ]]`         | Exists                  |
401 | `[[ -r FILE ]]`         | Readable                |
402 | `[[ -h FILE ]]`         | Symlink                 |
403 | `[[ -d FILE ]]`         | Directory               |
404 | `[[ -w FILE ]]`         | Writable                |
405 | `[[ -s FILE ]]`         | Size is > 0 bytes       |
406 | `[[ -f FILE ]]`         | File                    |
407 | `[[ -x FILE ]]`         | Executable              |
408 | ---                     | ---                     |
409 | `[[ FILE1 -nt FILE2 ]]` | 1 is more recent than 2 |
410 | `[[ FILE1 -ot FILE2 ]]` | 2 is more recent than 1 |
411 | `[[ FILE1 -ef FILE2 ]]` | Same files              |
412
413 ### Example
414
415 ```bash
416 # String
417 if [[ -z "$string" ]]; then
418   echo "String is empty"
419 elif [[ -n "$string" ]]; then
420   echo "String is not empty"
421 else
422   echo "This never happens"
423 fi
424 ```
425
426 ```bash
427 # Combinations
428 if [[ X && Y ]]; then
429   ...
430 fi
431 ```
432
433 ```bash
434 # Equal
435 if [[ "$A" == "$B" ]]
436 ```
437
438 ```bash
439 # Regex
440 if [[ "A" =~ . ]]
441 ```
442
443 ```bash
444 if (( $a < $b )); then
445    echo "$a is smaller than $b"
446 fi
447 ```
448
449 ```bash
450 if [[ -e "file.txt" ]]; then
451   echo "file exists"
452 fi
453 ```
454
455 Arrays
456 ------
457
458 ### Defining arrays
459
460 ```bash
461 Fruits=('Apple' 'Banana' 'Orange')
462 ```
463
464 ```bash
465 Fruits[0]="Apple"
466 Fruits[1]="Banana"
467 Fruits[2]="Orange"
468 ```
469
470 ### Working with arrays
471
472 ```bash
473 echo ${Fruits[0]}           # Element #0
474 echo ${Fruits[-1]}          # Last element
475 echo ${Fruits[@]}           # All elements, space-separated
476 echo ${#Fruits[@]}          # Number of elements
477 echo ${#Fruits}             # String length of the 1st element
478 echo ${#Fruits[3]}          # String length of the Nth element
479 echo ${Fruits[@]:3:2}       # Range (from position 3, length 2)
480 echo ${!Fruits[@]}          # Keys of all elements, space-separated
481 ```
482
483 ### Operations
484
485 ```bash
486 Fruits=("${Fruits[@]}" "Watermelon")    # Push
487 Fruits+=('Watermelon')                  # Also Push
488 Fruits=( ${Fruits[@]/Ap*/} )            # Remove by regex match
489 unset Fruits[2]                         # Remove one item
490 Fruits=("${Fruits[@]}")                 # Duplicate
491 Fruits=("${Fruits[@]}" "${Veggies[@]}") # Concatenate
492 lines=(`cat "logfile"`)                 # Read from file
493 ```
494
495 ### Iteration
496
497 ```bash
498 for i in "${arrayName[@]}"; do
499   echo $i
500 done
501 ```
502
503 Dictionaries
504 ------------
505 {: .-three-column}
506
507 ### Defining
508
509 ```bash
510 declare -A sounds
511 ```
512
513 ```bash
514 sounds[dog]="bark"
515 sounds[cow]="moo"
516 sounds[bird]="tweet"
517 sounds[wolf]="howl"
518 ```
519
520 Declares `sound` as a Dictionary object (aka associative array).
521
522 ### Working with dictionaries
523
524 ```bash
525 echo ${sounds[dog]} # Dog's sound
526 echo ${sounds[@]}   # All values
527 echo ${!sounds[@]}  # All keys
528 echo ${#sounds[@]}  # Number of elements
529 unset sounds[dog]   # Delete dog
530 ```
531
532 ### Iteration
533
534 #### Iterate over values
535
536 ```bash
537 for val in "${sounds[@]}"; do
538   echo $val
539 done
540 ```
541
542 #### Iterate over keys
543
544 ```bash
545 for key in "${!sounds[@]}"; do
546   echo $key
547 done
548 ```
549
550 Options
551 -------
552
553 ### Options
554
555 ```bash
556 set -o noclobber  # Avoid overlay files (echo "hi" > foo)
557 set -o errexit    # Used to exit upon error, avoiding cascading errors
558 set -o pipefail   # Unveils hidden failures
559 set -o nounset    # Exposes unset variables
560 ```
561
562 ### Glob options
563
564 ```bash
565 shopt -s nullglob    # Non-matching globs are removed  ('*.foo' => '')
566 shopt -s failglob    # Non-matching globs throw errors
567 shopt -s nocaseglob  # Case insensitive globs
568 shopt -s dotglob     # Wildcards match dotfiles ("*.sh" => ".foo.sh")
569 shopt -s globstar    # Allow ** for recursive matches ('lib/**/*.rb' => 'lib/a/b/c.rb')
570 ```
571
572 Set `GLOBIGNORE` as a colon-separated list of patterns to be removed from glob
573 matches.
574
575 History
576 -------
577
578 ### Commands
579
580 | Command               | Description                               |
581 | --------------------- | ----------------------------------------- |
582 | `history`             | Show history                              |
583 | `shopt -s histverify` | Don't execute expanded result immediately |
584
585 ### Expansions
586
587 | Expression   | Description                                          |
588 | ------------ | ---------------------------------------------------- |
589 | `!$`         | Expand last parameter of most recent command         |
590 | `!*`         | Expand all parameters of most recent command         |
591 | `!-n`        | Expand `n`th most recent command                     |
592 | `!n`         | Expand `n`th command in history                      |
593 | `!<command>` | Expand most recent invocation of command `<command>` |
594
595 ### Operations
596
597 | Code                 | Description                                                           |
598 | -------------------- | --------------------------------------------------------------------- |
599 | `!!`                 | Execute last command again                                            |
600 | `!!:s/<FROM>/<TO>/`  | Replace first occurrence of `<FROM>` to `<TO>` in most recent command |
601 | `!!:gs/<FROM>/<TO>/` | Replace all occurrences of `<FROM>` to `<TO>` in most recent command  |
602 | `!$:t`               | Expand only basename from last parameter of most recent command       |
603 | `!$:h`               | Expand only directory from last parameter of most recent command      |
604
605 `!!` and `!$` can be replaced with any valid expansion.
606
607 ### Slices
608
609 | Code     | Description                                                                              |
610 | -------- | ---------------------------------------------------------------------------------------- |
611 | `!!:n`   | Expand only `n`th token from most recent command (command is `0`; first argument is `1`) |
612 | `!^`     | Expand first argument from most recent command                                           |
613 | `!$`     | Expand last token from most recent command                                               |
614 | `!!:n-m` | Expand range of tokens from most recent command                                          |
615 | `!!:n-$` | Expand `n`th token to last from most recent command                                      |
616
617 `!!` can be replaced with any valid expansion i.e. `!cat`, `!-2`, `!42`, etc.
618
619
620 Miscellaneous
621 -------------
622
623 ### Numeric calculations
624
625 ```bash
626 $((a + 200))      # Add 200 to $a
627 ```
628
629 ```bash
630 $(($RANDOM%200))  # Random number 0..199
631 ```
632
633 ### Subshells
634
635 ```bash
636 (cd somedir; echo "I'm now in $PWD")
637 pwd # still in first directory
638 ```
639
640 ### Redirection
641
642 ```bash
643 python hello.py > output.txt   # stdout to (file)
644 python hello.py >> output.txt  # stdout to (file), append
645 python hello.py 2> error.log   # stderr to (file)
646 python hello.py 2>&1           # stderr to stdout
647 python hello.py 2>/dev/null    # stderr to (null)
648 python hello.py &>/dev/null    # stdout and stderr to (null)
649 ```
650
651 ```bash
652 python hello.py < foo.txt      # feed foo.txt to stdin for python
653 diff <(ls -r) <(ls)            # Compare two stdout without files
654 ```
655
656 ### Inspecting commands
657
658 ```bash
659 command -V cd
660 #=> "cd is a function/alias/whatever"
661 ```
662
663 ### Trap errors
664
665 ```bash
666 trap 'echo Error at about $LINENO' ERR
667 ```
668
669 or
670
671 ```bash
672 traperr() {
673   echo "ERROR: ${BASH_SOURCE[1]} at about ${BASH_LINENO[0]}"
674 }
675
676 set -o errtrace
677 trap traperr ERR
678 ```
679
680 ### Case/switch
681
682 ```bash
683 case "$1" in
684   start | up)
685     vagrant up
686     ;;
687
688   *)
689     echo "Usage: $0 {start|stop|ssh}"
690     ;;
691 esac
692 ```
693
694 ### Source relative
695
696 ```bash
697 source "${0%/*}/../share/foo.sh"
698 ```
699
700 ### printf
701
702 ```bash
703 printf "Hello %s, I'm %s" Sven Olga
704 #=> "Hello Sven, I'm Olga
705
706 printf "1 + 1 = %d" 2
707 #=> "1 + 1 = 2"
708
709 printf "This is how you print a float: %f" 2
710 #=> "This is how you print a float: 2.000000"
711 ```
712
713 ### Transform strings
714
715 | Command option     | Description                                         |
716 | ------------------ | --------------------------------------------------- |
717 | `-c`               | Operations apply to characters not in the given set |
718 | `-d`               | Delete characters                                   |
719 | `-s`               | Replaces repeated characters with single occurrence |
720 | `-t`               | Truncates                                           |
721 | `[:upper:]`        | All upper case letters                              |
722 | `[:lower:]`        | All lower case letters                              |
723 | `[:digit:]`        | All digits                                          |
724 | `[:space:]`        | All whitespace                                      |
725 | `[:alpha:]`        | All letters                                         |
726 | `[:alnum:]`        | All letters and digits                              |
727
728 #### Example
729
730 ```bash
731 echo "Welcome To Devhints" | tr [:lower:] [:upper:]
732 WELCOME TO DEVHINTS
733 ```
734
735 ### Directory of script
736
737 ```bash
738 DIR="${0%/*}"
739 ```
740
741 ### Getting options
742
743 ```bash
744 while [[ "$1" =~ ^- && ! "$1" == "--" ]]; do case $1 in
745   -V | --version )
746     echo $version
747     exit
748     ;;
749   -s | --string )
750     shift; string=$1
751     ;;
752   -f | --flag )
753     flag=1
754     ;;
755 esac; shift; done
756 if [[ "$1" == '--' ]]; then shift; fi
757 ```
758
759 ### Heredoc
760
761 ```sh
762 cat <<END
763 hello world
764 END
765 ```
766
767 ### Reading input
768
769 ```bash
770 echo -n "Proceed? [y/n]: "
771 read ans
772 echo $ans
773 ```
774
775 ```bash
776 read -n 1 ans    # Just one character
777 ```
778
779 ### Special variables
780
781 | Expression         | Description                            |
782 | ------------------ | -------------------------------------- |
783 | `$?`               | Exit status of last task               |
784 | `$!`               | PID of last background task            |
785 | `$$`               | PID of shell                           |
786 | `$0`               | Filename of the shell script           |
787 | `$_`               | Last argument of the previous command  |
788 | `${PIPESTATUS[n]}` | return value of piped commands (array) |
789
790 See [Special parameters](http://wiki.bash-hackers.org/syntax/shellvars#special_parameters_and_shell_variables).
791
792 ### Go to previous directory
793
794 ```bash
795 pwd # /home/user/foo
796 cd bar/
797 pwd # /home/user/foo/bar
798 cd -
799 pwd # /home/user/foo
800 ```
801
802 ### Check for command's result
803
804 ```bash
805 if ping -c 1 google.com; then
806   echo "It appears you have a working internet connection"
807 fi
808 ```
809
810 ### Grep check
811
812 ```bash
813 if grep -q 'foo' ~/.bash_history; then
814   echo "You appear to have typed 'foo' in the past"
815 fi
816 ```
817
818 ## Also see
819 {: .-one-column}
820
821 * [Bash-hackers wiki](http://wiki.bash-hackers.org/) _(bash-hackers.org)_
822 * [Shell vars](http://wiki.bash-hackers.org/syntax/shellvars) _(bash-hackers.org)_
823 * [Learn bash in y minutes](https://learnxinyminutes.com/docs/bash/) _(learnxinyminutes.com)_
824 * [Bash Guide](http://mywiki.wooledge.org/BashGuide) _(mywiki.wooledge.org)_
825 * [ShellCheck](https://www.shellcheck.net/) _(shellcheck.net)_