Chapter 26. Arrays

Newer versions of Bash support one-dimensional arrays. Array elements may be initialized with the variable[xx] notation. Alternatively, a script may introduce the entire array by an explicit declare -a variable statement. To dereference (find the contents of) an array element, use curly bracket notation, that is, ${variable[xx]}.


Example 26-1. Simple array usage

   1 #!/bin/bash
   2 
   3 
   4 area[11]=23
   5 area[13]=37
   6 area[51]=UFOs
   7 
   8 # Array members need not be consecutive or contiguous.
   9 
  10 # Some members of the array can be left uninitialized.
  11 # Gaps in the array are o.k.
  12 
  13 
  14 echo -n "area[11] = "
  15 echo ${area[11]}    #  {curly brackets} needed
  16 
  17 echo -n "area[13] = "
  18 echo ${area[13]}
  19 
  20 echo "Contents of area[51] are ${area[51]}."
  21 
  22 # Contents of uninitialized array variable print blank.
  23 echo -n "area[43] = "
  24 echo ${area[43]}
  25 echo "(area[43] unassigned)"
  26 
  27 echo
  28 
  29 # Sum of two array variables assigned to third
  30 area[5]=`expr ${area[11]} + ${area[13]}`
  31 echo "area[5] = area[11] + area[13]"
  32 echo -n "area[5] = "
  33 echo ${area[5]}
  34 
  35 area[6]=`expr ${area[11]} + ${area[51]}`
  36 echo "area[6] = area[11] + area[51]"
  37 echo -n "area[6] = "
  38 echo ${area[6]}
  39 # This fails because adding an integer to a string is not permitted.
  40 
  41 echo; echo; echo
  42 
  43 # -----------------------------------------------------------------
  44 # Another array, "area2".
  45 # Another way of assigning array variables...
  46 # array_name=( XXX YYY ZZZ ... )
  47 
  48 area2=( zero one two three four )
  49 
  50 echo -n "area2[0] = "
  51 echo ${area2[0]}
  52 # Aha, zero-based indexing (first element of array is [0], not [1]).
  53 
  54 echo -n "area2[1] = "
  55 echo ${area2[1]}    # [1] is second element of array.
  56 # -----------------------------------------------------------------
  57 
  58 echo; echo; echo
  59 
  60 # -----------------------------------------------
  61 # Yet another array, "area3".
  62 # Yet another way of assigning array variables...
  63 # array_name=([xx]=XXX [yy]=YYY ...)
  64 
  65 area3=([17]=seventeen [24]=twenty-four)
  66 
  67 echo -n "area3[17] = "
  68 echo ${area3[17]}
  69 
  70 echo -n "area3[24] = "
  71 echo ${area3[24]}
  72 # -----------------------------------------------
  73 
  74 exit 0

Note

Bash permits array operations on variables, even if the variables are not explicitly declared as arrays.
   1 string=abcABC123ABCabc
   2 echo ${string[@]}               # abcABC123ABCabc
   3 echo ${string[*]}               # abcABC123ABCabc 
   4 echo ${string[0]}               # abcABC123ABCabc
   5 echo ${string[1]}               # No output!
   6                                 # Why?
   7 echo ${#string[@]}              # 1
   8                                 # One element in the array.
   9                                 # The string itself.
  10 
  11 # Thank you, Michael Zick, for pointing this out.
Once again this demonstrates that Bash variables are untyped.


Example 26-2. Formatting a poem

   1 #!/bin/bash
   2 # poem.sh: Pretty-prints one of the document author's favorite poems.
   3 
   4 # Lines of the poem (single stanza).
   5 Line[1]="I do not know which to prefer,"
   6 Line[2]="The beauty of inflections"
   7 Line[3]="Or the beauty of innuendoes,"
   8 Line[4]="The blackbird whistling"
   9 Line[5]="Or just after."
  10 
  11 # Attribution.
  12 Attrib[1]=" Wallace Stevens"
  13 Attrib[2]="\"Thirteen Ways of Looking at a Blackbird\""
  14 # This poem is in the Public Domain (copyright expired).
  15 
  16 echo
  17 
  18 for index in 1 2 3 4 5    # Five lines.
  19 do
  20   printf "     %s\n" "${Line[index]}"
  21 done
  22 
  23 for index in 1 2          # Two attribution lines.
  24 do
  25   printf "          %s\n" "${Attrib[index]}"
  26 done
  27 
  28 echo
  29 
  30 exit 0
  31 
  32 # Exercise:
  33 # --------
  34 # Modify this script to pretty-print a poem from a text data file.

Array variables have a syntax all their own, and even standard Bash commands and operators have special options adapted for array use.


Example 26-3. Various array operations

   1 #!/bin/bash
   2 # array-ops.sh: More fun with arrays.
   3 
   4 
   5 array=( zero one two three four five )
   6 # Element 0   1   2    3     4    5
   7 
   8 echo ${array[0]}       #  zero
   9 echo ${array:0}        #  zero
  10                        #  Parameter expansion of first element,
  11                        #+ starting at position # 0 (1st character).
  12 echo ${array:1}        #  ero
  13                        #  Parameter expansion of first element,
  14                        #+ starting at position # 1 (2nd character).
  15 
  16 echo "--------------"
  17 
  18 echo ${#array[0]}      #  4
  19                        #  Length of first element of array.
  20 echo ${#array}         #  4
  21                        #  Length of first element of array.
  22                        #  (Alternate notation)
  23 
  24 echo ${#array[1]}      #  3
  25                        #  Length of second element of array.
  26                        #  Arrays in Bash have zero-based indexing.
  27 
  28 echo ${#array[*]}      #  6
  29                        #  Number of elements in array.
  30 echo ${#array[@]}      #  6
  31                        #  Number of elements in array.
  32 
  33 echo "--------------"
  34 
  35 array2=( [0]="first element" [1]="second element" [3]="fourth element" )
  36 
  37 echo ${array2[0]}      # first element
  38 echo ${array2[1]}      # second element
  39 echo ${array2[2]}      #
  40                        # Skipped in initialization, and therefore null.
  41 echo ${array2[3]}      # fourth element
  42 
  43 
  44 exit 0

Many of the standard string operations work on arrays.


Example 26-4. String operations on arrays

   1 #!/bin/bash
   2 # array-strops.sh: String operations on arrays.
   3 # Script by Michael Zick.
   4 # Used with permission.
   5 
   6 #  In general, any string operation in the ${name ... } notation
   7 #+ can be applied to all string elements in an array
   8 #+ with the ${name[@] ... } or ${name[*] ...} notation.
   9 
  10 
  11 arrayZ=( one two three four five five )
  12 
  13 echo
  14 
  15 # Trailing Substring Extraction
  16 echo ${arrayZ[@]:0}     # one two three four five five
  17                         # All elements.
  18 
  19 echo ${arrayZ[@]:1}     # two three four five five
  20                         # All elements following element[0].
  21 
  22 echo ${arrayZ[@]:1:2}   # two three
  23                         # Only the two elements after element[0].
  24 
  25 echo "-----------------------"
  26 
  27 #  Substring Removal
  28 #  Removes shortest match from front of string(s),
  29 #+ where the substring is a regular expression.
  30 
  31 echo ${arrayZ[@]#f*r}   # one two three five five
  32                         # Applied to all elements of the array.
  33                         # Matches "four" and removes it.
  34 
  35 # Longest match from front of string(s)
  36 echo ${arrayZ[@]##t*e}  # one two four five five
  37                         # Applied to all elements of the array.
  38                         # Matches "three" and removes it.
  39 
  40 # Shortest match from back of string(s)
  41 echo ${arrayZ[@]%h*e}   # one two t four five five
  42                         # Applied to all elements of the array.
  43                         # Matches "hree" and removes it.
  44 
  45 # Longest match from back of string(s)
  46 echo ${arrayZ[@]%%t*e}  # one two four five five
  47                         # Applied to all elements of the array.
  48                         # Matches "three" and removes it.
  49 
  50 echo "-----------------------"
  51 
  52 # Substring Replacement
  53 
  54 # Replace first occurance of substring with replacement
  55 echo ${arrayZ[@]/fiv/XYZ}   # one two three four XYZe XYZe
  56                             # Applied to all elements of the array.
  57 
  58 # Replace all occurances of substring
  59 echo ${arrayZ[@]//iv/YY}    # one two three four fYYe fYYe
  60                             # Applied to all elements of the array.
  61 
  62 # Delete all occurances of substring
  63 # Not specifing a replacement means 'delete'
  64 echo ${arrayZ[@]//fi/}      # one two three four ve ve
  65                             # Applied to all elements of the array.
  66 
  67 # Replace front-end occurances of substring
  68 echo ${arrayZ[@]/#fi/XY}    # one two three four XYve XYve
  69                             # Applied to all elements of the array.
  70 
  71 # Replace back-end occurances of substring
  72 echo ${arrayZ[@]/%ve/ZZ}    # one two three four fiZZ fiZZ
  73                             # Applied to all elements of the array.
  74 
  75 echo ${arrayZ[@]/%o/XX}     # one twXX three four five five
  76                             # Why?
  77 
  78 echo "-----------------------"
  79 
  80 
  81 # Before reaching for awk (or anything else) --
  82 # Recall:
  83 #   $( ... ) is command substitution.
  84 #   Functions run as a sub-process.
  85 #   Functions write their output to stdout.
  86 #   Assignment reads the function's stdout.
  87 #   The name[@] notation specifies a "for-each" operation.
  88 
  89 newstr() {
  90     echo -n "!!!"
  91 }
  92 
  93 echo ${arrayZ[@]/%e/$(newstr)}
  94 # on!!! two thre!!! four fiv!!! fiv!!!
  95 # Q.E.D: The replacement action is an 'assignment.'
  96 
  97 #  Accessing the "For-Each"
  98 echo ${arrayZ[@]//*/$(newstr optional_arguments)}
  99 #  Now, if Bash would just pass the matched string as $0
 100 #+ to the function being called . . .
 101 
 102 echo
 103 
 104 exit 0

Command substitution can construct the individual elements of an array.


Example 26-5. Loading the contents of a script into an array

   1 #!/bin/bash
   2 # script-array.sh: Loads this script into an array.
   3 # Inspired by an e-mail from Chris Martin (thanks!).
   4 
   5 script_contents=( $(cat "$0") )  #  Stores contents of this script ($0)
   6                                  #+ in an array.
   7 
   8 for element in $(seq 0 $((${#script_contents[@]} - 1)))
   9   do                #  ${#script_contents[@]}
  10                     #+ gives number of elements in the array.
  11                     #
  12                     #  Question:
  13                     #  Why is  seq 0  necessary?
  14                     #  Try changing it to seq 1.
  15   echo -n "${script_contents[$element]}"
  16                     # List each field of this script on a single line.
  17   echo -n " -- "    # Use " -- " as a field separator.
  18 done
  19 
  20 echo
  21 
  22 exit 0
  23 
  24 # Exercise:
  25 # --------
  26 #  Modify this script so it lists itself
  27 #+ in its original format,
  28 #+ complete with whitespace, line breaks, etc.

In an array context, some Bash builtins have a slightly altered meaning. For example, unset deletes array elements, or even an entire array.


Example 26-6. Some special properties of arrays

   1 #!/bin/bash
   2 
   3 declare -a colors
   4 #  All subsequent commands in this script will treat
   5 #+ the variable "colors" as an array.
   6 
   7 echo "Enter your favorite colors (separated from each other by a space)."
   8 
   9 read -a colors    # Enter at least 3 colors to demonstrate features below.
  10 #  Special option to 'read' command,
  11 #+ allowing assignment of elements in an array.
  12 
  13 echo
  14 
  15 element_count=${#colors[@]}
  16 # Special syntax to extract number of elements in array.
  17 #     element_count=${#colors[*]} works also.
  18 #
  19 #  The "@" variable allows word splitting within quotes
  20 #+ (extracts variables separated by whitespace).
  21 #
  22 #  This corresponds to the behavior of "$@" and "$*"
  23 #+ in positional parameters. 
  24 
  25 index=0
  26 
  27 while [ "$index" -lt "$element_count" ]
  28 do    # List all the elements in the array.
  29   echo ${colors[$index]}
  30   let "index = $index + 1"
  31 done
  32 # Each array element listed on a separate line.
  33 # If this is not desired, use  echo -n "${colors[$index]} "
  34 #
  35 # Doing it with a "for" loop instead:
  36 #   for i in "${colors[@]}"
  37 #   do
  38 #     echo "$i"
  39 #   done
  40 # (Thanks, S.C.)
  41 
  42 echo
  43 
  44 # Again, list all the elements in the array, but using a more elegant method.
  45   echo ${colors[@]}          # echo ${colors[*]} also works.
  46 
  47 echo
  48 
  49 # The "unset" command deletes elements of an array, or entire array.
  50 unset colors[1]              # Remove 2nd element of array.
  51                              # Same effect as   colors[1]=
  52 echo  ${colors[@]}           # List array again, missing 2nd element.
  53 
  54 unset colors                 # Delete entire array.
  55                              #  unset colors[*] and
  56                              #+ unset colors[@] also work.
  57 echo; echo -n "Colors gone."			   
  58 echo ${colors[@]}            # List array again, now empty.
  59 
  60 exit 0

As seen in the previous example, either ${array_name[@]} or ${array_name[*]} refers to all the elements of the array. Similarly, to get a count of the number of elements in an array, use either ${#array_name[@]} or ${#array_name[*]}. ${#array_name} is the length (number of characters) of ${array_name[0]}, the first element of the array.


Example 26-7. Of empty arrays and empty elements

   1 #!/bin/bash
   2 # empty-array.sh
   3 
   4 #  Thanks to Stephane Chazelas for the original example,
   5 #+ and to Michael Zick for extending it.
   6 
   7 
   8 # An empty array is not the same as an array with empty elements.
   9 
  10 array0=( first second third )
  11 array1=( '' )   # "array1" consists of one empty element.
  12 array2=( )      # No elements . . . "array2" is empty.
  13 
  14 echo
  15 ListArray()
  16 {
  17 echo
  18 echo "Elements in array0:  ${array0[@]}"
  19 echo "Elements in array1:  ${array1[@]}"
  20 echo "Elements in array2:  ${array2[@]}"
  21 echo
  22 echo "Length of first element in array0 = ${#array0}"
  23 echo "Length of first element in array1 = ${#array1}"
  24 echo "Length of first element in array2 = ${#array2}"
  25 echo
  26 echo "Number of elements in array0 = ${#array0[*]}"  # 3
  27 echo "Number of elements in array1 = ${#array1[*]}"  # 1  (Surprise!)
  28 echo "Number of elements in array2 = ${#array2[*]}"  # 0
  29 }
  30 
  31 # ===================================================================
  32 
  33 ListArray
  34 
  35 # Try extending those arrays.
  36 
  37 # Adding an element to an array.
  38 array0=( "${array0[@]}" "new1" )
  39 array1=( "${array1[@]}" "new1" )
  40 array2=( "${array2[@]}" "new1" )
  41 
  42 ListArray
  43 
  44 # or
  45 array0[${#array0[*]}]="new2"
  46 array1[${#array1[*]}]="new2"
  47 array2[${#array2[*]}]="new2"
  48 
  49 ListArray
  50 
  51 # When extended as above; arrays are 'stacks'
  52 # The above is the 'push'
  53 # The stack 'height' is:
  54 height=${#array2[@]}
  55 echo
  56 echo "Stack height for array2 = $height"
  57 
  58 # The 'pop' is:
  59 unset array2[${#array2[@]}-1]	#  Arrays are zero-based,
  60 height=${#array2[@]}            #+ which means first element has index 0.
  61 echo
  62 echo "POP"
  63 echo "New stack height for array2 = $height"
  64 
  65 ListArray
  66 
  67 # List only 2nd and 3rd elements of array0.
  68 from=1		# Zero-based numbering.
  69 to=2		#
  70 array3=( ${array0[@]:1:2} )
  71 echo
  72 echo "Elements in array3:  ${array3[@]}"
  73 
  74 # Works like a string (array of characters).
  75 # Try some other "string" forms.
  76 
  77 # Replacement:
  78 array4=( ${array0[@]/second/2nd} )
  79 echo
  80 echo "Elements in array4:  ${array4[@]}"
  81 
  82 # Replace all matching wildcarded string.
  83 array5=( ${array0[@]//new?/old} )
  84 echo
  85 echo "Elements in array5:  ${array5[@]}"
  86 
  87 # Just when you are getting the feel for this . . .
  88 array6=( ${array0[@]#*new} )
  89 echo # This one might surprise you.
  90 echo "Elements in array6:  ${array6[@]}"
  91 
  92 array7=( ${array0[@]#new1} )
  93 echo # After array6 this should not be a surprise.
  94 echo "Elements in array7:  ${array7[@]}"
  95 
  96 # Which looks a lot like . . .
  97 array8=( ${array0[@]/new1/} )
  98 echo
  99 echo "Elements in array8:  ${array8[@]}"
 100 
 101 #  So what can one say about this?
 102 
 103 #  The string operations are performed on
 104 #+ each of the elements in var[@] in succession.
 105 #  Therefore : Bash supports string vector operations
 106 #+ if the result is a zero length string,
 107 #+ that element disappears in the resulting assignment.
 108 
 109 #  Question, are those strings hard or soft quotes?
 110 
 111 zap='new*'
 112 array9=( ${array0[@]/$zap/} )
 113 echo
 114 echo "Elements in array9:  ${array9[@]}"
 115 
 116 # Just when you thought you where still in Kansas . . .
 117 array10=( ${array0[@]#$zap} )
 118 echo
 119 echo "Elements in array10:  ${array10[@]}"
 120 
 121 # Compare array7 with array10.
 122 # Compare array8 with array9.
 123 
 124 # Answer: must be soft quotes.
 125 
 126 exit 0

The relationship of ${array_name[@]} and ${array_name[*]} is analogous to that between $@ and $*. This powerful array notation has a number of uses.

   1 # Copying an array.
   2 array2=( "${array1[@]}" )
   3 # or
   4 array2="${array1[@]}"
   5 
   6 # Adding an element to an array.
   7 array=( "${array[@]}" "new element" )
   8 # or
   9 array[${#array[*]}]="new element"
  10 
  11 # Thanks, S.C.

Tip

The array=( element1 element2 ... elementN ) initialization operation, with the help of command substitution, makes it possible to load the contents of a text file into an array.

   1 #!/bin/bash
   2 
   3 filename=sample_file
   4 
   5 #            cat sample_file
   6 #
   7 #            1 a b c
   8 #            2 d e fg
   9 
  10 
  11 declare -a array1
  12 
  13 array1=( `cat "$filename"`)                #  Loads contents
  14 #         List file to stdout              #+ of $filename into array1.
  15 #
  16 #  array1=( `cat "$filename" | tr '\n' ' '`)
  17 #                            change linefeeds in file to spaces. 
  18 #  Not necessary because Bash does word splitting,
  19 #+ changing linefeeds to spaces.
  20 
  21 echo ${array1[@]}            # List the array.
  22 #                              1 a b c 2 d e fg
  23 #
  24 #  Each whitespace-separated "word" in the file
  25 #+ has been assigned to an element of the array.
  26 
  27 element_count=${#array1[*]}
  28 echo $element_count          # 8

Clever scripting makes it possible to add array operations.


Example 26-8. Initializing arrays

   1 #! /bin/bash
   2 # array-assign.bash
   3 
   4 #  Array operations are Bash specific,
   5 #+ hence the ".bash" in the script name.
   6 
   7 # Copyright (c) Michael S. Zick, 2003, All rights reserved.
   8 # License: Unrestricted reuse in any form, for any purpose.
   9 # Version: $ID$
  10 #
  11 # Clarification and additional comments by William Park.
  12 
  13 #  Based on an example provided by Stephane Chazelas
  14 #+ which appeared in the book: Advanced Bash Scripting Guide.
  15 
  16 # Output format of the 'times' command:
  17 # User CPU <space> System CPU
  18 # User CPU of dead children <space> System CPU of dead children
  19 
  20 #  Bash has two versions of assigning all elements of an array
  21 #+ to a new array variable.
  22 #  Both drop 'null reference' elements
  23 #+ in Bash versions 2.04, 2.05a and 2.05b.
  24 #  An additional array assignment that maintains the relationship of
  25 #+ [subscript]=value for arrays may be added to newer versions.
  26 
  27 #  Constructs a large array using an internal command,
  28 #+ but anything creating an array of several thousand elements
  29 #+ will do just fine.
  30 
  31 declare -a bigOne=( /dev/* )
  32 echo
  33 echo 'Conditions: Unquoted, default IFS, All-Elements-Of'
  34 echo "Number of elements in array is ${#bigOne[@]}"
  35 
  36 # set -vx
  37 
  38 
  39 
  40 echo
  41 echo '- - testing: =( ${array[@]} ) - -'
  42 times
  43 declare -a bigTwo=( ${bigOne[@]} )
  44 #                 ^              ^
  45 times
  46 
  47 echo
  48 echo '- - testing: =${array[@]} - -'
  49 times
  50 declare -a bigThree=${bigOne[@]}
  51 # No parentheses this time.
  52 times
  53 
  54 #  Comparing the numbers shows that the second form, pointed out
  55 #+ by Stephane Chazelas, is from three to four times faster.
  56 #
  57 #  William Park explains:
  58 #+ The bigTwo array assigned as single string, whereas
  59 #+ bigThree assigned element by element.
  60 #  So, in essence, you have:
  61 #                   bigTwo=( [0]="... ... ..." )
  62 #                   bigThree=( [0]="..." [1]="..." [2]="..." ... )
  63 
  64 
  65 #  I will continue to use the first form in my example descriptions
  66 #+ because I think it is a better illustration of what is happening.
  67 
  68 #  The reusable portions of my examples will actual contain
  69 #+ the second form where appropriate because of the speedup.
  70 
  71 # MSZ: Sorry about that earlier oversight folks.
  72 
  73 
  74 #  Note:
  75 #  ----
  76 #  The "declare -a" statements in lines 31 and 43
  77 #+ are not strictly necessary, since it is implicit
  78 #+ in the  Array=( ... )  assignment form.
  79 #  However, eliminating these declarations slows down
  80 #+ the execution of the following sections of the script.
  81 #  Try it, and see what happens.
  82 
  83 exit 0

Note

Adding a superfluous declare -a statement to an array declaration may speed up execution of subsequent operations on the array.


Example 26-9. Copying and concatenating arrays

   1 #! /bin/bash
   2 # CopyArray.sh
   3 #
   4 # This script written by Michael Zick.
   5 # Used here with permission.
   6 
   7 #  How-To "Pass by Name & Return by Name"
   8 #+ or "Building your own assignment statement".
   9 
  10 
  11 CpArray_Mac() {
  12 
  13 # Assignment Command Statement Builder
  14 
  15     echo -n 'eval '
  16     echo -n "$2"                    # Destination name
  17     echo -n '=( ${'
  18     echo -n "$1"                    # Source name
  19     echo -n '[@]} )'
  20 
  21 # That could all be a single command.
  22 # Matter of style only.
  23 }
  24 
  25 declare -f CopyArray                # Function "Pointer"
  26 CopyArray=CpArray_Mac               # Statement Builder
  27 
  28 Hype()
  29 {
  30 
  31 # Hype the array named $1.
  32 # (Splice it together with array containing "Really Rocks".)
  33 # Return in array named $2.
  34 
  35     local -a TMP
  36     local -a hype=( Really Rocks )
  37 
  38     $($CopyArray $1 TMP)
  39     TMP=( ${TMP[@]} ${hype[@]} )
  40     $($CopyArray TMP $2)
  41 }
  42 
  43 declare -a before=( Advanced Bash Scripting )
  44 declare -a after
  45 
  46 echo "Array Before = ${before[@]}"
  47 
  48 Hype before after
  49 
  50 echo "Array After = ${after[@]}"
  51 
  52 # Too much hype?
  53 
  54 echo "What ${after[@]:3:2}?"
  55 
  56 declare -a modest=( ${after[@]:2:1} ${after[@]:3:2} )
  57 #                    ---- substring extraction ----
  58 
  59 echo "Array Modest = ${modest[@]}"
  60 
  61 # What happened to 'before' ?
  62 
  63 echo "Array Before = ${before[@]}"
  64 
  65 exit 0


Example 26-10. More on concatenating arrays

   1 #! /bin/bash
   2 # array-append.bash
   3 
   4 # Copyright (c) Michael S. Zick, 2003, All rights reserved.
   5 # License: Unrestricted reuse in any form, for any purpose.
   6 # Version: $ID$
   7 #
   8 # Slightly modified in formatting by M.C.
   9 
  10 
  11 # Array operations are Bash-specific.
  12 # Legacy UNIX /bin/sh lacks equivalents.
  13 
  14 
  15 #  Pipe the output of this script to 'more'
  16 #+ so it doesn't scroll off the terminal.
  17 
  18 
  19 # Subscript packed.
  20 declare -a array1=( zero1 one1 two1 )
  21 # Subscript sparse ([1] is not defined).
  22 declare -a array2=( [0]=zero2 [2]=two2 [3]=three2 )
  23 
  24 echo
  25 echo '- Confirm that the array is really subscript sparse. -'
  26 echo "Number of elements: 4"        # Hard-coded for illustration.
  27 for (( i = 0 ; i < 4 ; i++ ))
  28 do
  29     echo "Element [$i]: ${array2[$i]}"
  30 done
  31 # See also the more general code example in basics-reviewed.bash.
  32 
  33 
  34 declare -a dest
  35 
  36 # Combine (append) two arrays into a third array.
  37 echo
  38 echo 'Conditions: Unquoted, default IFS, All-Elements-Of operator'
  39 echo '- Undefined elements not present, subscripts not maintained. -'
  40 # # The undefined elements do not exist; they are not being dropped.
  41 
  42 dest=( ${array1[@]} ${array2[@]} )
  43 # dest=${array1[@]}${array2[@]}     # Strange results, possibly a bug.
  44 
  45 # Now, list the result.
  46 echo
  47 echo '- - Testing Array Append - -'
  48 cnt=${#dest[@]}
  49 
  50 echo "Number of elements: $cnt"
  51 for (( i = 0 ; i < cnt ; i++ ))
  52 do
  53     echo "Element [$i]: ${dest[$i]}"
  54 done
  55 
  56 # Assign an array to a single array element (twice).
  57 dest[0]=${array1[@]}
  58 dest[1]=${array2[@]}
  59 
  60 # List the result.
  61 echo
  62 echo '- - Testing modified array - -'
  63 cnt=${#dest[@]}
  64 
  65 echo "Number of elements: $cnt"
  66 for (( i = 0 ; i < cnt ; i++ ))
  67 do
  68     echo "Element [$i]: ${dest[$i]}"
  69 done
  70 
  71 # Examine the modified second element.
  72 echo
  73 echo '- - Reassign and list second element - -'
  74 
  75 declare -a subArray=${dest[1]}
  76 cnt=${#subArray[@]}
  77 
  78 echo "Number of elements: $cnt"
  79 for (( i = 0 ; i < cnt ; i++ ))
  80 do
  81     echo "Element [$i]: ${subArray[$i]}"
  82 done
  83 
  84 #  The assignment of an entire array to a single element
  85 #+ of another array using the '=${ ... }' array assignment
  86 #+ has converted the array being assigned into a string,
  87 #+ with the elements separated by a space (the first character of IFS).
  88 
  89 # If the original elements didn't contain whitespace . . .
  90 # If the original array isn't subscript sparse . . .
  91 # Then we could get the original array structure back again.
  92 
  93 # Restore from the modified second element.
  94 echo
  95 echo '- - Listing restored element - -'
  96 
  97 declare -a subArray=( ${dest[1]} )
  98 cnt=${#subArray[@]}
  99 
 100 echo "Number of elements: $cnt"
 101 for (( i = 0 ; i < cnt ; i++ ))
 102 do
 103     echo "Element [$i]: ${subArray[$i]}"
 104 done
 105 echo '- - Do not depend on this behavior. - -'
 106 echo '- - This behavior is subject to change - -'
 107 echo '- - in versions of Bash newer than version 2.05b - -'
 108 
 109 # MSZ: Sorry about any earlier confusion folks.
 110 
 111 exit 0

--

Arrays permit deploying old familiar algorithms as shell scripts. Whether this is necessarily a good idea is left to the reader to decide.


Example 26-11. An old friend: The Bubble Sort

   1 #!/bin/bash
   2 # bubble.sh: Bubble sort, of sorts.
   3 
   4 # Recall the algorithm for a bubble sort. In this particular version...
   5 
   6 #  With each successive pass through the array to be sorted,
   7 #+ compare two adjacent elements, and swap them if out of order.
   8 #  At the end of the first pass, the "heaviest" element has sunk to bottom.
   9 #  At the end of the second pass, the next "heaviest" one has sunk next to bottom.
  10 #  And so forth.
  11 #  This means that each successive pass needs to traverse less of the array.
  12 #  You will therefore notice a speeding up in the printing of the later passes.
  13 
  14 
  15 exchange()
  16 {
  17   # Swaps two members of the array.
  18   local temp=${Countries[$1]} #  Temporary storage
  19                               #+ for element getting swapped out.
  20   Countries[$1]=${Countries[$2]}
  21   Countries[$2]=$temp
  22   
  23   return
  24 }  
  25 
  26 declare -a Countries  #  Declare array,
  27                       #+ optional here since it's initialized below.
  28 
  29 #  Is it permissable to split an array variable over multiple lines
  30 #+ using an escape (\)?
  31 #  Yes.
  32 
  33 Countries=(Netherlands Ukraine Zaire Turkey Russia Yemen Syria \
  34 Brazil Argentina Nicaragua Japan Mexico Venezuela Greece England \
  35 Israel Peru Canada Oman Denmark Wales France Kenya \
  36 Xanadu Qatar Liechtenstein Hungary)
  37 
  38 # "Xanadu" is the mythical place where, according to Coleridge,
  39 #+ Kubla Khan did a pleasure dome decree.
  40 
  41 
  42 clear                      # Clear the screen to start with. 
  43 
  44 echo "0: ${Countries[*]}"  # List entire array at pass 0.
  45 
  46 number_of_elements=${#Countries[@]}
  47 let "comparisons = $number_of_elements - 1"
  48 
  49 count=1 # Pass number.
  50 
  51 while [ "$comparisons" -gt 0 ]          # Beginning of outer loop
  52 do
  53 
  54   index=0  # Reset index to start of array after each pass.
  55 
  56   while [ "$index" -lt "$comparisons" ] # Beginning of inner loop
  57   do
  58     if [ ${Countries[$index]} \> ${Countries[`expr $index + 1`]} ]
  59     #  If out of order...
  60     #  Recalling that \> is ASCII comparison operator
  61     #+ within single brackets.
  62 
  63     #  if [[ ${Countries[$index]} > ${Countries[`expr $index + 1`]} ]]
  64     #+ also works.
  65     then
  66       exchange $index `expr $index + 1`  # Swap.
  67     fi  
  68     let "index += 1"
  69   done # End of inner loop
  70 
  71 # ----------------------------------------------------------------------
  72 # Paulo Marcel Coelho Aragao suggests for-loops as a simpler altenative.
  73 #
  74 # for (( last = $number_of_elements - 1 ; last > 1 ; last-- ))
  75 # do
  76 #     for (( i = 0 ; i < last ; i++ ))
  77 #     do
  78 #         [[ "${Countries[$i]}" > "${Countries[$((i+1))]}" ]] \
  79 #             && exchange $i $((i+1))
  80 #     done
  81 # done
  82 # ----------------------------------------------------------------------
  83   
  84 
  85 let "comparisons -= 1" #  Since "heaviest" element bubbles to bottom,
  86                        #+ we need do one less comparison each pass.
  87 
  88 echo
  89 echo "$count: ${Countries[@]}"  # Print resultant array at end of each pass.
  90 echo
  91 let "count += 1"                # Increment pass count.
  92 
  93 done                            # End of outer loop
  94                                 # All done.
  95 
  96 exit 0

--

Is it possible to nest arrays within arrays?

   1 #!/bin/bash
   2 # "Nested" array.
   3 
   4 #  Michael Zick provided this example,
   5 #+ with corrections and clarifications by William Park.
   6 
   7 AnArray=( $(ls --inode --ignore-backups --almost-all \
   8 	--directory --full-time --color=none --time=status \
   9 	--sort=time -l ${PWD} ) )  # Commands and options.
  10 
  11 # Spaces are significant . . . and don't quote anything in the above.
  12 
  13 SubArray=( ${AnArray[@]:11:1}  ${AnArray[@]:6:5} )
  14 #  This array has six elements:
  15 #+     SubArray=( [0]=${AnArray[11]} [1]=${AnArray[6]} [2]=${AnArray[7]}
  16 #      [3]=${AnArray[8]} [4]=${AnArray[9]} [5]=${AnArray[10]} )
  17 #
  18 #  Arrays in Bash are (circularly) linked lists
  19 #+ of type string (char *).
  20 #  So, this isn't actually a nested array,
  21 #+ but it's functionally similar.
  22 
  23 echo "Current directory and date of last status change:"
  24 echo "${SubArray[@]}"
  25 
  26 exit 0

--

Embedded arrays in combination with indirect references create some fascinating possibilities


Example 26-12. Embedded arrays and indirect references

   1 #!/bin/bash
   2 # embedded-arrays.sh
   3 # Embedded arrays and indirect references.
   4 
   5 # This script by Dennis Leeuw.
   6 # Used with permission.
   7 # Modified by document author.
   8 
   9 
  10 ARRAY1=(
  11         VAR1_1=value11
  12         VAR1_2=value12
  13         VAR1_3=value13
  14 )
  15 
  16 ARRAY2=(
  17         VARIABLE="test"
  18         STRING="VAR1=value1 VAR2=value2 VAR3=value3"
  19         ARRAY21=${ARRAY1[*]}
  20 )       # Embed ARRAY1 within this second array.
  21 
  22 function print () {
  23         OLD_IFS="$IFS"
  24         IFS=$'\n'       #  To print each array element
  25                         #+ on a separate line.
  26         TEST1="ARRAY2[*]"
  27         local ${!TEST1} # See what happens if you delete this line.
  28         #  Indirect reference.
  29 	#  This makes the components of $TEST1
  30 	#+ accessible to this function.
  31 
  32 
  33         #  Let's see what we've got so far.
  34         echo
  35         echo "\$TEST1 = $TEST1"       #  Just the name of the variable.
  36         echo; echo
  37         echo "{\$TEST1} = ${!TEST1}"  #  Contents of the variable.
  38                                       #  That's what an indirect
  39                                       #+ reference does.
  40         echo
  41         echo "-------------------------------------------"; echo
  42         echo
  43 
  44 
  45         # Print variable
  46         echo "Variable VARIABLE: $VARIABLE"
  47 	
  48         # Print a string element
  49         IFS="$OLD_IFS"
  50         TEST2="STRING[*]"
  51         local ${!TEST2}      # Indirect reference (as above).
  52         echo "String element VAR2: $VAR2 from STRING"
  53 
  54         # Print an array element
  55         TEST2="ARRAY21[*]"
  56         local ${!TEST2}      # Indirect reference (as above).
  57         echo "Array element VAR1_1: $VAR1_1 from ARRAY21"
  58 }
  59 
  60 print
  61 echo
  62 
  63 exit 0
  64 
  65 #   As the author of the script notes,
  66 #+ "you can easily expand it to create named-hashes in bash."
  67 #   (Difficult) exercise for the reader: implement this.

--

Arrays enable implementing a shell script version of the Sieve of Eratosthenes. Of course, a resource-intensive application of this nature should really be written in a compiled language, such as C. It runs excruciatingly slowly as a script.


Example 26-13. Complex array application: Sieve of Eratosthenes

   1 #!/bin/bash
   2 # sieve.sh
   3 
   4 # Sieve of Eratosthenes
   5 # Ancient algorithm for finding prime numbers.
   6 
   7 # This runs a couple of orders of magnitude
   8 # slower than the equivalent C program.
   9 
  10 LOWER_LIMIT=1       # Starting with 1.
  11 UPPER_LIMIT=1000    # Up to 1000.
  12 # (You may set this higher...  if you have time on your hands.)
  13 
  14 PRIME=1
  15 NON_PRIME=0
  16 
  17 let SPLIT=UPPER_LIMIT/2
  18 # Optimization:
  19 # Need to test numbers only halfway to upper limit.
  20 
  21 
  22 declare -a Primes
  23 # Primes[] is an array.
  24 
  25 
  26 initialize ()
  27 {
  28 # Initialize the array.
  29 
  30 i=$LOWER_LIMIT
  31 until [ "$i" -gt "$UPPER_LIMIT" ]
  32 do
  33   Primes[i]=$PRIME
  34   let "i += 1"
  35 done
  36 # Assume all array members guilty (prime)
  37 # until proven innocent.
  38 }
  39 
  40 print_primes ()
  41 {
  42 # Print out the members of the Primes[] array tagged as prime.
  43 
  44 i=$LOWER_LIMIT
  45 
  46 until [ "$i" -gt "$UPPER_LIMIT" ]
  47 do
  48 
  49   if [ "${Primes[i]}" -eq "$PRIME" ]
  50   then
  51     printf "%8d" $i
  52     # 8 spaces per number gives nice, even columns.
  53   fi
  54   
  55   let "i += 1"
  56   
  57 done
  58 
  59 }
  60 
  61 sift () # Sift out the non-primes.
  62 {
  63 
  64 let i=$LOWER_LIMIT+1
  65 # We know 1 is prime, so let's start with 2.
  66 
  67 until [ "$i" -gt "$UPPER_LIMIT" ]
  68 do
  69 
  70 if [ "${Primes[i]}" -eq "$PRIME" ]
  71 # Don't bother sieving numbers already sieved (tagged as non-prime).
  72 then
  73 
  74   t=$i
  75 
  76   while [ "$t" -le "$UPPER_LIMIT" ]
  77   do
  78     let "t += $i "
  79     Primes[t]=$NON_PRIME
  80     # Tag as non-prime all multiples.
  81   done
  82 
  83 fi  
  84 
  85   let "i += 1"
  86 done  
  87 
  88 
  89 }
  90 
  91 
  92 # Invoke the functions sequentially.
  93 initialize
  94 sift
  95 print_primes
  96 # This is what they call structured programming.
  97 
  98 echo
  99 
 100 exit 0
 101 
 102 
 103 
 104 # ----------------------------------------------- #
 105 # Code below line will not execute.
 106 
 107 # This improved version of the Sieve, by Stephane Chazelas,
 108 # executes somewhat faster.
 109 
 110 # Must invoke with command-line argument (limit of primes).
 111 
 112 UPPER_LIMIT=$1                  # From command line.
 113 let SPLIT=UPPER_LIMIT/2         # Halfway to max number.
 114 
 115 Primes=( '' $(seq $UPPER_LIMIT) )
 116 
 117 i=1
 118 until (( ( i += 1 ) > SPLIT ))  # Need check only halfway.
 119 do
 120   if [[ -n $Primes[i] ]]
 121   then
 122     t=$i
 123     until (( ( t += i ) > UPPER_LIMIT ))
 124     do
 125       Primes[t]=
 126     done
 127   fi  
 128 done  
 129 echo ${Primes[*]}
 130 
 131 exit 0

Compare this array-based prime number generator with an alternative that does not use arrays, Example A-16.

--

Arrays lend themselves, to some extent, to emulating data structures for which Bash has no native support.


Example 26-14. Emulating a push-down stack

   1 #!/bin/bash
   2 # stack.sh: push-down stack simulation
   3 
   4 #  Similar to the CPU stack, a push-down stack stores data items
   5 #+ sequentially, but releases them in reverse order, last-in first-out.
   6 
   7 BP=100            #  Base Pointer of stack array.
   8                   #  Begin at element 100.
   9 
  10 SP=$BP            #  Stack Pointer.
  11                   #  Initialize it to "base" (bottom) of stack.
  12 
  13 Data=             #  Contents of stack location.  
  14                   #  Must use global variable,
  15                   #+ because of limitation on function return range.
  16 
  17 declare -a stack
  18 
  19 
  20 push()            # Push item on stack.
  21 {
  22 if [ -z "$1" ]    # Nothing to push?
  23 then
  24   return
  25 fi
  26 
  27 let "SP -= 1"     # Bump stack pointer.
  28 stack[$SP]=$1
  29 
  30 return
  31 }
  32 
  33 pop()                    # Pop item off stack.
  34 {
  35 Data=                    # Empty out data item.
  36 
  37 if [ "$SP" -eq "$BP" ]   # Stack empty?
  38 then
  39   return
  40 fi                       #  This also keeps SP from getting past 100,
  41                          #+ i.e., prevents a runaway stack.
  42 
  43 Data=${stack[$SP]}
  44 let "SP += 1"            # Bump stack pointer.
  45 return
  46 }
  47 
  48 status_report()          # Find out what's happening.
  49 {
  50 echo "-------------------------------------"
  51 echo "REPORT"
  52 echo "Stack Pointer = $SP"
  53 echo "Just popped \""$Data"\" off the stack."
  54 echo "-------------------------------------"
  55 echo
  56 }
  57 
  58 
  59 # =======================================================
  60 # Now, for some fun.
  61 
  62 echo
  63 
  64 # See if you can pop anything off empty stack.
  65 pop
  66 status_report
  67 
  68 echo
  69 
  70 push garbage
  71 pop
  72 status_report     # Garbage in, garbage out.      
  73 
  74 value1=23; push $value1
  75 value2=skidoo; push $value2
  76 value3=FINAL; push $value3
  77 
  78 pop              # FINAL
  79 status_report
  80 pop              # skidoo
  81 status_report
  82 pop              # 23
  83 status_report    # Last-in, first-out!
  84 
  85 #  Notice how the stack pointer decrements with each push,
  86 #+ and increments with each pop.
  87 
  88 echo
  89 # =======================================================
  90 
  91 
  92 # Exercises:
  93 # ---------
  94 
  95 # 1)  Modify the "push()" function to permit pushing
  96 #   + multiple element on the stack with a single function call.
  97 
  98 # 2)  Modify the "pop()" function to permit popping
  99 #   + multiple element from the stack with a single function call.
 100 
 101 # 3)  Using this script as a starting point,
 102 #   + write a stack-based 4-function calculator.
 103 
 104 exit 0

--

Fancy manipulation of array "subscripts" may require intermediate variables. For projects involving this, again consider using a more powerful programming language, such as Perl or C.


Example 26-15. Complex array application: Exploring a weird mathematical series

   1 #!/bin/bash
   2 
   3 # Douglas Hofstadter's notorious "Q-series":
   4 
   5 # Q(1) = Q(2) = 1
   6 # Q(n) = Q(n - Q(n-1)) + Q(n - Q(n-2)), for n>2
   7 
   8 # This is a "chaotic" integer series with strange and unpredictable behavior.
   9 # The first 20 terms of the series are:
  10 # 1 1 2 3 3 4 5 5 6 6 6 8 8 8 10 9 10 11 11 12 
  11 
  12 # See Hofstadter's book, "Goedel, Escher, Bach: An Eternal Golden Braid",
  13 # p. 137, ff.
  14 
  15 
  16 LIMIT=100     # Number of terms to calculate
  17 LINEWIDTH=20  # Number of terms printed per line
  18 
  19 Q[1]=1        # First two terms of series are 1.
  20 Q[2]=1
  21 
  22 echo
  23 echo "Q-series [$LIMIT terms]:"
  24 echo -n "${Q[1]} "             # Output first two terms.
  25 echo -n "${Q[2]} "
  26 
  27 for ((n=3; n <= $LIMIT; n++))  # C-like loop conditions.
  28 do   # Q[n] = Q[n - Q[n-1]] + Q[n - Q[n-2]]  for n>2
  29 # Need to break the expression into intermediate terms,
  30 # since Bash doesn't handle complex array arithmetic very well.
  31 
  32   let "n1 = $n - 1"        # n-1
  33   let "n2 = $n - 2"        # n-2
  34   
  35   t0=`expr $n - ${Q[n1]}`  # n - Q[n-1]
  36   t1=`expr $n - ${Q[n2]}`  # n - Q[n-2]
  37   
  38   T0=${Q[t0]}              # Q[n - Q[n-1]]
  39   T1=${Q[t1]}              # Q[n - Q[n-2]]
  40 
  41 Q[n]=`expr $T0 + $T1`      # Q[n - Q[n-1]] + Q[n - Q[n-2]]
  42 echo -n "${Q[n]} "
  43 
  44 if [ `expr $n % $LINEWIDTH` -eq 0 ]    # Format output.
  45 then   #     mod
  46   echo # Break lines into neat chunks.
  47 fi
  48 
  49 done
  50 
  51 echo
  52 
  53 exit 0
  54 
  55 # This is an iterative implementation of the Q-series.
  56 # The more intuitive recursive implementation is left as an exercise.
  57 # Warning: calculating this series recursively takes a *very* long time.

--

Bash supports only one-dimensional arrays, however a little trickery permits simulating multi-dimensional ones.


Example 26-16. Simulating a two-dimensional array, then tilting it

   1 #!/bin/bash
   2 # twodim.sh: Simulating a two-dimensional array.
   3 
   4 # A one-dimensional array consists of a single row.
   5 # A two-dimensional array stores rows sequentially.
   6 
   7 Rows=5
   8 Columns=5
   9 # 5 X 5 Array.
  10 
  11 declare -a alpha     # char alpha [Rows] [Columns];
  12                      # Unnecessary declaration. Why?
  13 
  14 load_alpha ()
  15 {
  16 local rc=0
  17 local index
  18 
  19 for i in A B C D E F G H I J K L M N O P Q R S T U V W X Y
  20 do     # Use different symbols if you like.
  21   local row=`expr $rc / $Columns`
  22   local column=`expr $rc % $Rows`
  23   let "index = $row * $Rows + $column"
  24   alpha[$index]=$i
  25 # alpha[$row][$column]
  26   let "rc += 1"
  27 done  
  28 
  29 #  Simpler would be
  30 #+   declare -a alpha=( A B C D E F G H I J K L M N O P Q R S T U V W X Y )
  31 #+ but this somehow lacks the "flavor" of a two-dimensional array.
  32 }
  33 
  34 print_alpha ()
  35 {
  36 local row=0
  37 local index
  38 
  39 echo
  40 
  41 while [ "$row" -lt "$Rows" ]   #  Print out in "row major" order:
  42 do                             #+ columns vary,
  43                                #+ while row (outer loop) remains the same.
  44   local column=0
  45 
  46   echo -n "       "            #  Lines up "square" array with rotated one.
  47   
  48   while [ "$column" -lt "$Columns" ]
  49   do
  50     let "index = $row * $Rows + $column"
  51     echo -n "${alpha[index]} "  # alpha[$row][$column]
  52     let "column += 1"
  53   done
  54 
  55   let "row += 1"
  56   echo
  57 
  58 done  
  59 
  60 # The simpler equivalent is
  61 #   echo ${alpha[*]} | xargs -n $Columns
  62 
  63 echo
  64 }
  65 
  66 filter ()     # Filter out negative array indices.
  67 {
  68 
  69 echo -n "  "  # Provides the tilt.
  70               # Explain why.
  71 
  72 if [[ "$1" -ge 0 &&  "$1" -lt "$Rows" && "$2" -ge 0 && "$2" -lt "$Columns" ]]
  73 then
  74     let "index = $1 * $Rows + $2"
  75     # Now, print it rotated.
  76     echo -n " ${alpha[index]}"
  77     #           alpha[$row][$column]
  78 fi    
  79 
  80 }
  81   
  82 
  83 
  84 
  85 rotate ()  #  Rotate the array 45 degrees --
  86 {          #+ "balance" it on its lower lefthand corner.
  87 local row
  88 local column
  89 
  90 for (( row = Rows; row > -Rows; row-- ))
  91   do       # Step through the array backwards. Why?
  92 
  93   for (( column = 0; column < Columns; column++ ))
  94   do
  95 
  96     if [ "$row" -ge 0 ]
  97     then
  98       let "t1 = $column - $row"
  99       let "t2 = $column"
 100     else
 101       let "t1 = $column"
 102       let "t2 = $column + $row"
 103     fi  
 104 
 105     filter $t1 $t2   # Filter out negative array indices. Why?
 106   done
 107 
 108   echo; echo
 109 
 110 done 
 111 
 112 #  Array rotation inspired by examples (pp. 143-146) in
 113 #+ "Advanced C Programming on the IBM PC," by Herbert Mayer
 114 #+ (see bibliography).
 115 #  This just goes to show that much of what can be done in C
 116 #+ can also be done in shell scripting.
 117 
 118 }
 119 
 120 
 121 #--------------- Now, let the show begin. ------------#
 122 load_alpha     # Load the array.
 123 print_alpha    # Print it out.  
 124 rotate         # Rotate it 45 degrees counterclockwise.
 125 #-----------------------------------------------------#
 126 
 127 exit 0
 128 
 129 # This is a rather contrived, not to mention inelegant simulation.
 130 
 131 # Exercises:
 132 # ---------
 133 # 1)  Rewrite the array loading and printing functions
 134 #     in a more intuitive and less kludgy fashion.
 135 #
 136 # 2)  Figure out how the array rotation functions work.
 137 #     Hint: think about the implications of backwards-indexing an array.
 138 #
 139 # 3)  Rewrite this script to handle a non-square array,
 140 #     such as a 6 X 4 one.
 141 #     Try to minimize "distortion" when the array is rotated.

A two-dimensional array is essentially equivalent to a one-dimensional one, but with additional addressing modes for referencing and manipulating the individual elements by "row" and "column" position.

For an even more elaborate example of simulating a two-dimensional array, see Example A-10.