OSDN Git Service

98d4807651302071649c3f4f00485670e9537fac
[twpd/master.git] / bash.md
1 ---
2 title: Bash scripting
3 category: CLI
4 layout: 2017/sheet
5 tags: [Featured]
6 updated: 2020-07-05
7 keywords:
8   - Variables
9   - Functions
10   - Interpolation
11   - Brace expansions
12   - Loops
13   - Conditional execution
14   - Command substitution
15 ---
16
17 Getting started
18 ---------------
19 {: .-three-column}
20
21 ### Introduction
22 {: .-intro}
23
24 This is a quick reference to getting started with Bash scripting.
25
26 - [Learn bash in y minutes](https://learnxinyminutes.com/docs/bash/) _(learnxinyminutes.com)_
27 - [Bash Guide](http://mywiki.wooledge.org/BashGuide) _(mywiki.wooledge.org)_
28 - [Bash Hackers Wiki](https://wiki.bash-hackers.org) _(wiki.bash-hackers.org)_
29
30 ### Example
31
32 ```bash
33 #!/usr/bin/env bash
34
35 name="John"
36 echo "Hello $name!"
37 ```
38
39 ### Variables
40
41 ```bash
42 name="John"
43 echo $name  # see below
44 echo "$name"
45 echo "${name}!"
46 ```
47 Generally quote your variables unless they contain wildcards to expand or command fragments.
48
49 ```bash
50 wildcard="*.txt"
51 option="iv"
52 cp -$options $wildcard /tmp
53 ```
54
55 ### String quotes
56
57 ```bash
58 name="John"
59 echo "Hi $name"  #=> Hi John
60 echo 'Hi $name'  #=> Hi $name
61 ```
62
63 ### Shell execution
64
65 ```bash
66 echo "I'm in $(pwd)"
67 echo "I'm in `pwd`"  # obsolescent
68 # Same
69 ```
70
71 See [Command substitution](http://wiki.bash-hackers.org/syntax/expansion/cmdsubst)
72
73 ### Conditional execution
74
75 ```bash
76 git commit && git push
77 git commit || echo "Commit failed"
78 ```
79
80 ### Functions
81 {: id='functions-example'}
82
83 ```bash
84 get_name() {
85   echo "John"
86 }
87
88 echo "You are $(get_name)"
89 ```
90
91 See: [Functions](#functions)
92
93 ### Conditionals
94 {: id='conditionals-example'}
95
96 ```bash
97 if [[ -z "$string" ]]; then
98   echo "String is empty"
99 elif [[ -n "$string" ]]; then
100   echo "String is not empty"
101 fi
102 ```
103
104 See: [Conditionals](#conditionals)
105
106 ### Strict mode
107
108 ```bash
109 set -euo pipefail
110 IFS=$'\n\t'
111 ```
112
113 See: [Unofficial bash strict mode](http://redsymbol.net/articles/unofficial-bash-strict-mode/)
114
115 ### Brace expansion
116
117 ```bash
118 echo {A,B}.js
119 ```
120
121 | Expression | Description         |
122 | ---------- | ------------------- |
123 | `{A,B}`    | Same as `A B`       |
124 | `{A,B}.js` | Same as `A.js B.js` |
125 | `{1..5}`   | Same as `1 2 3 4 5` |
126
127 See: [Brace expansion](http://wiki.bash-hackers.org/syntax/expansion/brace)
128
129
130 Parameter expansions
131 --------------------
132 {: .-three-column}
133
134 ### Basics
135
136 ```bash
137 name="John"
138 echo "${name}"
139 echo "${name/J/j}"    #=> "john" (substitution)
140 echo "${name:0:2}"    #=> "Jo" (slicing)
141 echo "${name::2}"     #=> "Jo" (slicing)
142 echo "${name::-1}"    #=> "Joh" (slicing)
143 echo "${name:(-1)}"   #=> "n" (slicing from right)
144 echo "${name:(-2):1}" #=> "h" (slicing from right)
145 echo "${food:-Cake}"  #=> $food or "Cake"
146 ```
147
148 ```bash
149 length=2
150 echo "${name:0:length}"  #=> "Jo"
151 ```
152
153 See: [Parameter expansion](http://wiki.bash-hackers.org/syntax/pe)
154
155 ```bash
156 str="/path/to/foo.cpp"
157 echo "${str%.cpp}"    # /path/to/foo
158 echo "${str%.cpp}.o"  # /path/to/foo.o
159 echo "${str%/*}"      # /path/to
160
161 echo "${str##*.}"     # cpp (extension)
162 echo "${str##*/}"     # foo.cpp (basepath)
163
164 echo "${str#*/}"      # path/to/foo.cpp
165 echo "${str##*/}"     # foo.cpp
166
167 echo "${str/foo/bar}" # /path/to/bar.cpp
168 ```
169
170 ```bash
171 str="Hello world"
172 echo "${str:6:5}"   # "world"
173 echo "${str: -5:5}"  # "world"
174 ```
175
176 ```bash
177 src="/path/to/foo.cpp"
178 base=${src##*/}   #=> "foo.cpp" (basepath)
179 dir=${src%$base}  #=> "/path/to/" (dirpath)
180 ```
181
182 ### Substitution
183
184 | Code              | Description         |
185 | ----------------- | ------------------- |
186 | `${foo%suffix}`   | Remove suffix       |
187 | `${foo#prefix}`   | Remove prefix       |
188 | ---               | ---                 |
189 | `${foo%%suffix}`  | Remove long suffix  |
190 | `${foo/%suffix}`  | Remove long suffix  |
191 | `${foo##prefix}`  | Remove long prefix  |
192 | `${foo/#prefix}`  | Remove long prefix  |
193 | ---               | ---                 |
194 | `${foo/from/to}`  | Replace first match |
195 | `${foo//from/to}` | Replace all         |
196 | ---               | ---                 |
197 | `${foo/%from/to}` | Replace suffix      |
198 | `${foo/#from/to}` | Replace prefix      |
199
200 ### Comments
201
202 ```bash
203 # Single line comment
204 ```
205
206 ```bash
207 : '
208 This is a
209 multi line
210 comment
211 '
212 ```
213
214 ### Substrings
215
216 | Expression      | Description                    |
217 | --------------- | ------------------------------ |
218 | `${foo:0:3}`    | Substring _(position, length)_ |
219 | `${foo:(-3):3}` | Substring from the right       |
220
221 ### Length
222
223 | Expression | Description      |
224 | ---------- | ---------------- |
225 | `${#foo}`  | Length of `$foo` |
226
227 ### Manipulation
228
229 ```bash
230 str="HELLO WORLD!"
231 echo "${str,}"   #=> "hELLO WORLD!" (lowercase 1st letter)
232 echo "${str,,}"  #=> "hello world!" (all lowercase)
233
234 str="hello world!"
235 echo "${str^}"   #=> "Hello world!" (uppercase 1st letter)
236 echo "${str^^}"  #=> "HELLO WORLD!" (all uppercase)
237 ```
238
239 ### Default values
240
241 | Expression        | Description                                              |
242 | ----------------- | -------------------------------------------------------- |
243 | `${foo:-val}`     | `$foo`, or `val` if unset (or null)                      |
244 | `${foo:=val}`     | Set `$foo` to `val` if unset (or null)                   |
245 | `${foo:+val}`     | `val` if `$foo` is set (and not null)                    |
246 | `${foo:?message}` | Show error message and exit if `$foo` is unset (or null) |
247
248 Omitting the `:` removes the (non)nullity checks, e.g. `${foo-val}` expands to `val` if unset otherwise `$foo`.
249
250 Loops
251 -----
252 {: .-three-column}
253
254 ### Basic for loop
255
256 ```bash
257 for i in /etc/rc.*; do
258   echo "$i"
259 done
260 ```
261
262 ### C-like for loop
263
264 ```bash
265 for ((i = 0 ; i < 100 ; i++)); do
266   echo "$i"
267 done
268 ```
269
270 ### Ranges
271
272 ```bash
273 for i in {1..5}; do
274     echo "Welcome $i"
275 done
276 ```
277
278 #### With step size
279
280 ```bash
281 for i in {5..50..5}; do
282     echo "Welcome $i"
283 done
284 ```
285
286 ### Reading lines
287
288 ```bash
289 while read -r line; do
290   echo "$line"
291 done <file.txt
292 ```
293
294 ### Forever
295
296 ```bash
297 while true; do
298   ยทยทยท
299 done
300 ```
301
302 Functions
303 ---------
304 {: .-three-column}
305
306 ### Defining functions
307
308 ```bash
309 myfunc() {
310     echo "hello $1"
311 }
312 ```
313
314 ```bash
315 # Same as above (alternate syntax)
316 function myfunc() {
317     echo "hello $1"
318 }
319 ```
320
321 ```bash
322 myfunc "John"
323 ```
324
325 ### Returning values
326
327 ```bash
328 myfunc() {
329     local myresult='some value'
330     echo "$myresult"
331 }
332 ```
333
334 ```bash
335 result=$(myfunc)
336 ```
337
338 ### Raising errors
339
340 ```bash
341 myfunc() {
342   return 1
343 }
344 ```
345
346 ```bash
347 if myfunc; then
348   echo "success"
349 else
350   echo "failure"
351 fi
352 ```
353
354 ### Arguments
355
356 | Expression | Description                                      |
357 | ---        | ---                                              |
358 | `$#`       | Number of arguments                              |
359 | `$*`       | All positional arguments  (as a single word)     |
360 | `$@`       | All positional arguments (as separate strings)  |
361 | `$1`       | First argument                                   |
362 | `$_`       | Last argument of the previous command            |
363
364 **Note**: `$@` and `$*` must be quoted in order to perform as described.
365 Otherwise, they do exactly the same thing (arguments as separate strings).
366
367 See [Special parameters](http://wiki.bash-hackers.org/syntax/shellvars#special_parameters_and_shell_variables).
368
369 Conditionals
370 ------------
371 {: .-three-column}
372
373 ### Conditions
374
375 Note that `[[` is actually a command/program that returns either `0` (true) or `1` (false). Any program that obeys the same logic (like all base utils, such as `grep(1)` or `ping(1)`) can be used as condition, see examples.
376
377 | Condition                | Description           |
378 | ---                      | ---                   |
379 | `[[ -z STRING ]]`        | Empty string          |
380 | `[[ -n STRING ]]`        | Not empty string      |
381 | `[[ STRING == STRING ]]` | Equal                 |
382 | `[[ STRING != STRING ]]` | Not Equal             |
383 | ---                      | ---                   |
384 | `[[ NUM -eq NUM ]]`      | Equal                 |
385 | `[[ NUM -ne NUM ]]`      | Not equal             |
386 | `[[ NUM -lt NUM ]]`      | Less than             |
387 | `[[ NUM -le NUM ]]`      | Less than or equal    |
388 | `[[ NUM -gt NUM ]]`      | Greater than          |
389 | `[[ NUM -ge NUM ]]`      | Greater than or equal |
390 | ---                      | ---                   |
391 | `[[ STRING =~ STRING ]]` | Regexp                |
392 | ---                      | ---                   |
393 | `(( NUM < NUM ))`        | Numeric conditions    |
394
395 #### More conditions
396
397 | Condition            | Description              |
398 | -------------------- | ------------------------ |
399 | `[[ -o noclobber ]]` | If OPTIONNAME is enabled |
400 | ---                  | ---                      |
401 | `[[ ! EXPR ]]`       | Not                      |
402 | `[[ X && Y ]]`       | And                      |
403 | `[[ X || Y ]]`       | Or                       |
404
405 ### File conditions
406
407 | Condition               | Description             |
408 | ---                     | ---                     |
409 | `[[ -e FILE ]]`         | Exists                  |
410 | `[[ -r FILE ]]`         | Readable                |
411 | `[[ -h FILE ]]`         | Symlink                 |
412 | `[[ -d FILE ]]`         | Directory               |
413 | `[[ -w FILE ]]`         | Writable                |
414 | `[[ -s FILE ]]`         | Size is > 0 bytes       |
415 | `[[ -f FILE ]]`         | File                    |
416 | `[[ -x FILE ]]`         | Executable              |
417 | ---                     | ---                     |
418 | `[[ FILE1 -nt FILE2 ]]` | 1 is more recent than 2 |
419 | `[[ FILE1 -ot FILE2 ]]` | 2 is more recent than 1 |
420 | `[[ FILE1 -ef FILE2 ]]` | Same files              |
421
422 ### Example
423
424 ```bash
425 # String
426 if [[ -z "$string" ]]; then
427   echo "String is empty"
428 elif [[ -n "$string" ]]; then
429   echo "String is not empty"
430 else
431   echo "This never happens"
432 fi
433 ```
434
435 ```bash
436 # Combinations
437 if [[ X && Y ]]; then
438   ...
439 fi
440 ```
441
442 ```bash
443 # Equal
444 if [[ "$A" == "$B" ]]
445 ```
446
447 ```bash
448 # Regex
449 if [[ "A" =~ . ]]
450 ```
451
452 ```bash
453 if (( $a < $b )); then
454    echo "$a is smaller than $b"
455 fi
456 ```
457
458 ```bash
459 if [[ -e "file.txt" ]]; then
460   echo "file exists"
461 fi
462 ```
463
464 Arrays
465 ------
466
467 ### Defining arrays
468
469 ```bash
470 Fruits=('Apple' 'Banana' 'Orange')
471 ```
472
473 ```bash
474 Fruits[0]="Apple"
475 Fruits[1]="Banana"
476 Fruits[2]="Orange"
477 ```
478
479 ### Working with arrays
480
481 ```bash
482 echo "${Fruits[0]}"           # Element #0
483 echo "${Fruits[-1]}"          # Last element
484 echo "${Fruits[@]}"           # All elements, space-separated
485 echo "${#Fruits[@]}"          # Number of elements
486 echo "${#Fruits}"             # String length of the 1st element
487 echo "${#Fruits[3]}"          # String length of the Nth element
488 echo "${Fruits[@]:3:2}"       # Range (from position 3, length 2)
489 echo "${!Fruits[@]}"          # Keys of all elements, space-separated
490 ```
491
492 ### Operations
493
494 ```bash
495 Fruits=("${Fruits[@]}" "Watermelon")    # Push
496 Fruits+=('Watermelon')                  # Also Push
497 Fruits=( "${Fruits[@]/Ap*/}" )          # Remove by regex match
498 unset Fruits[2]                         # Remove one item
499 Fruits=("${Fruits[@]}")                 # Duplicate
500 Fruits=("${Fruits[@]}" "${Veggies[@]}") # Concatenate
501 lines=(`cat "logfile"`)                 # Read from file
502 ```
503
504 ### Iteration
505
506 ```bash
507 for i in "${arrayName[@]}"; do
508   echo "$i"
509 done
510 ```
511
512 Dictionaries
513 ------------
514 {: .-three-column}
515
516 ### Defining
517
518 ```bash
519 declare -A sounds
520 ```
521
522 ```bash
523 sounds[dog]="bark"
524 sounds[cow]="moo"
525 sounds[bird]="tweet"
526 sounds[wolf]="howl"
527 ```
528
529 Declares `sound` as a Dictionary object (aka associative array).
530
531 ### Working with dictionaries
532
533 ```bash
534 echo "${sounds[dog]}" # Dog's sound
535 echo "${sounds[@]}"   # All values
536 echo "${!sounds[@]}"  # All keys
537 echo "${#sounds[@]}"  # Number of elements
538 unset sounds[dog]     # Delete dog
539 ```
540
541 ### Iteration
542
543 #### Iterate over values
544
545 ```bash
546 for val in "${sounds[@]}"; do
547   echo "$val"
548 done
549 ```
550
551 #### Iterate over keys
552
553 ```bash
554 for key in "${!sounds[@]}"; do
555   echo "$key"
556 done
557 ```
558
559 Options
560 -------
561
562 ### Options
563
564 ```bash
565 set -o noclobber  # Avoid overlay files (echo "hi" > foo)
566 set -o errexit    # Used to exit upon error, avoiding cascading errors
567 set -o pipefail   # Unveils hidden failures
568 set -o nounset    # Exposes unset variables
569 ```
570
571 ### Glob options
572
573 ```bash
574 shopt -s nullglob    # Non-matching globs are removed  ('*.foo' => '')
575 shopt -s failglob    # Non-matching globs throw errors
576 shopt -s nocaseglob  # Case insensitive globs
577 shopt -s dotglob     # Wildcards match dotfiles ("*.sh" => ".foo.sh")
578 shopt -s globstar    # Allow ** for recursive matches ('lib/**/*.rb' => 'lib/a/b/c.rb')
579 ```
580
581 Set `GLOBIGNORE` as a colon-separated list of patterns to be removed from glob
582 matches.
583
584 History
585 -------
586
587 ### Commands
588
589 | Command               | Description                               |
590 | --------------------- | ----------------------------------------- |
591 | `history`             | Show history                              |
592 | `shopt -s histverify` | Don't execute expanded result immediately |
593
594 ### Expansions
595
596 | Expression   | Description                                          |
597 | ------------ | ---------------------------------------------------- |
598 | `!$`         | Expand last parameter of most recent command         |
599 | `!*`         | Expand all parameters of most recent command         |
600 | `!-n`        | Expand `n`th most recent command                     |
601 | `!n`         | Expand `n`th command in history                      |
602 | `!<command>` | Expand most recent invocation of command `<command>` |
603
604 ### Operations
605
606 | Code                 | Description                                                           |
607 | -------------------- | --------------------------------------------------------------------- |
608 | `!!`                 | Execute last command again                                            |
609 | `!!:s/<FROM>/<TO>/`  | Replace first occurrence of `<FROM>` to `<TO>` in most recent command |
610 | `!!:gs/<FROM>/<TO>/` | Replace all occurrences of `<FROM>` to `<TO>` in most recent command  |
611 | `!$:t`               | Expand only basename from last parameter of most recent command       |
612 | `!$:h`               | Expand only directory from last parameter of most recent command      |
613
614 `!!` and `!$` can be replaced with any valid expansion.
615
616 ### Slices
617
618 | Code     | Description                                                                              |
619 | -------- | ---------------------------------------------------------------------------------------- |
620 | `!!:n`   | Expand only `n`th token from most recent command (command is `0`; first argument is `1`) |
621 | `!^`     | Expand first argument from most recent command                                           |
622 | `!$`     | Expand last token from most recent command                                               |
623 | `!!:n-m` | Expand range of tokens from most recent command                                          |
624 | `!!:n-$` | Expand `n`th token to last from most recent command                                      |
625
626 `!!` can be replaced with any valid expansion i.e. `!cat`, `!-2`, `!42`, etc.
627
628
629 Miscellaneous
630 -------------
631
632 ### Numeric calculations
633
634 ```bash
635 $((a + 200))      # Add 200 to $a
636 ```
637
638 ```bash
639 $(($RANDOM%200))  # Random number 0..199
640 ```
641
642 ```bash
643 declare -i count  # Declare as type integer 
644 count+=1          # Increment
645 ```
646
647 ### Subshells
648
649 ```bash
650 (cd somedir; echo "I'm now in $PWD")
651 pwd # still in first directory
652 ```
653
654 ### Redirection
655
656 ```bash
657 python hello.py > output.txt            # stdout to (file)
658 python hello.py >> output.txt           # stdout to (file), append
659 python hello.py 2> error.log            # stderr to (file)
660 python hello.py 2>&1                    # stderr to stdout
661 python hello.py 2>/dev/null             # stderr to (null)
662 python hello.py >output.txt 2>&1        # stdout and stderr to (file), equivalent to &>
663 python hello.py &>/dev/null             # stdout and stderr to (null)
664 echo "$0: warning: too many users" >&2  # print diagnostic message to stderr
665 ```
666
667 ```bash
668 python hello.py < foo.txt      # feed foo.txt to stdin for python
669 diff <(ls -r) <(ls)            # Compare two stdout without files
670 ```
671
672 ### Inspecting commands
673
674 ```bash
675 command -V cd
676 #=> "cd is a function/alias/whatever"
677 ```
678
679 ### Trap errors
680
681 ```bash
682 trap 'echo Error at about $LINENO' ERR
683 ```
684
685 or
686
687 ```bash
688 traperr() {
689   echo "ERROR: ${BASH_SOURCE[1]} at about ${BASH_LINENO[0]}"
690 }
691
692 set -o errtrace
693 trap traperr ERR
694 ```
695
696 ### Case/switch
697
698 ```bash
699 case "$1" in
700   start | up)
701     vagrant up
702     ;;
703
704   *)
705     echo "Usage: $0 {start|stop|ssh}"
706     ;;
707 esac
708 ```
709
710 ### Source relative
711
712 ```bash
713 source "${0%/*}/../share/foo.sh"
714 ```
715
716 ### printf
717
718 ```bash
719 printf "Hello %s, I'm %s" Sven Olga
720 #=> "Hello Sven, I'm Olga
721
722 printf "1 + 1 = %d" 2
723 #=> "1 + 1 = 2"
724
725 printf "This is how you print a float: %f" 2
726 #=> "This is how you print a float: 2.000000"
727
728 printf '%s\n' '#!/bin/bash' 'echo hello' >file
729 # format string is applied to each group of arguments
730 printf '%i+%i=%i\n' 1 2 3  4 5 9
731 ```
732
733 ### Transform strings
734
735 | Command option     | Description                                         |
736 | ------------------ | --------------------------------------------------- |
737 | `-c`               | Operations apply to characters not in the given set |
738 | `-d`               | Delete characters                                   |
739 | `-s`               | Replaces repeated characters with single occurrence |
740 | `-t`               | Truncates                                           |
741 | `[:upper:]`        | All upper case letters                              |
742 | `[:lower:]`        | All lower case letters                              |
743 | `[:digit:]`        | All digits                                          |
744 | `[:space:]`        | All whitespace                                      |
745 | `[:alpha:]`        | All letters                                         |
746 | `[:alnum:]`        | All letters and digits                              |
747
748 #### Example
749
750 ```bash
751 echo "Welcome To Devhints" | tr '[:lower:]' '[:upper:]'
752 WELCOME TO DEVHINTS
753 ```
754
755 ### Directory of script
756
757 ```bash
758 dir=${0%/*}
759 ```
760
761 ### Getting options
762
763 ```bash
764 while [[ "$1" =~ ^- && ! "$1" == "--" ]]; do case $1 in
765   -V | --version )
766     echo "$version"
767     exit
768     ;;
769   -s | --string )
770     shift; string=$1
771     ;;
772   -f | --flag )
773     flag=1
774     ;;
775 esac; shift; done
776 if [[ "$1" == '--' ]]; then shift; fi
777 ```
778
779 ### Heredoc
780
781 ```sh
782 cat <<END
783 hello world
784 END
785 ```
786
787 ### Reading input
788
789 ```bash
790 echo -n "Proceed? [y/n]: "
791 read -r ans
792 echo "$ans"
793 ```
794
795 The `-r` option disables a peculiar legacy behavior with backslashes.
796
797 ```bash
798 read -n 1 ans    # Just one character
799 ```
800
801 ### Special variables
802
803 | Expression         | Description                            |
804 | ------------------ | -------------------------------------- |
805 | `$?`               | Exit status of last task               |
806 | `$!`               | PID of last background task            |
807 | `$$`               | PID of shell                           |
808 | `$0`               | Filename of the shell script           |
809 | `$_`               | Last argument of the previous command  |
810 | `${PIPESTATUS[n]}` | return value of piped commands (array) |
811
812 See [Special parameters](http://wiki.bash-hackers.org/syntax/shellvars#special_parameters_and_shell_variables).
813
814 ### Go to previous directory
815
816 ```bash
817 pwd # /home/user/foo
818 cd bar/
819 pwd # /home/user/foo/bar
820 cd -
821 pwd # /home/user/foo
822 ```
823
824 ### Check for command's result
825
826 ```bash
827 if ping -c 1 google.com; then
828   echo "It appears you have a working internet connection"
829 fi
830 ```
831
832 ### Grep check
833
834 ```bash
835 if grep -q 'foo' ~/.bash_history; then
836   echo "You appear to have typed 'foo' in the past"
837 fi
838 ```
839
840 ## Also see
841 {: .-one-column}
842
843 * [Bash-hackers wiki](http://wiki.bash-hackers.org/) _(bash-hackers.org)_
844 * [Shell vars](http://wiki.bash-hackers.org/syntax/shellvars) _(bash-hackers.org)_
845 * [Learn bash in y minutes](https://learnxinyminutes.com/docs/bash/) _(learnxinyminutes.com)_
846 * [Bash Guide](http://mywiki.wooledge.org/BashGuide) _(mywiki.wooledge.org)_
847 * [ShellCheck](https://www.shellcheck.net/) _(shellcheck.net)_