Off Topic Bash Question

Home » CentOS » Off Topic Bash Question
CentOS 9 Comments

I have a simple script:
#!/bin/bash
#
index=0
total=0
names=()
ip=()
while read -r LINE
do NODENAME=` echo $LINE | cut -f 1 -d ‘,’`
IP=` echo $LINE | cut -f 2 -d ‘,’`
names[index]=”$NODENAME”
ip[index]=”$IP”
index=`expr index+1`
total=`expr total+1`
done <<< $(cat list.txt) simple file: more list.txt name1,ip1 name2,ip2 name3,ip3 output when running: sh -x ./test_bash.sh + index=0 + total=0 + names=() + ip=() ++ cat list.txt + read -r LINE ++ echo name1,ip1 name2,ip2 name3,ip3 ++ cut -f 1 -d , + NODENAME=name1 ++ echo name1,ip1 name2,ip2 name3,ip3 ++ cut -f 2 -d , + IP=’ip1 name2′ + names[index]=name1 + ip[index]=’ip1 name2′ ++ expr index+1 + index=index+1 ++ expr total+1 + total=total+1 + read -r LINE + echo name1 name1 Question is why is it not reading one line at a time ? All I get is the first one. I’m just trying to build the array of the items in the file and then list them at this point. Thanks Jerry

9 thoughts on - Off Topic Bash Question

  • Hi Jerry,

    See below, inline, for some comments.

    NODENAME=$(cut -d, -f1 <<< $LINE) Notes: use $( instead of backticks. There’s no need to quote the comma. Write less and write concisely. IP=$(cut -d, -f2 <<< $LINE) ip[$((index++))]=”$IP” This allows you to use the variable index, and then increment it by one for the next cycle of the loop. Not needed because of the post-increment (++) above. ((total++)) done < list.txt The “<<<" operator does all manner of expansion of the input and supplies it as a single line. That not what you want. Just redirect stdin from the file instead. This is happening because of the <<< operator.

  • Hi,

    Some remarks:
    – just try « expr ${VAR} + 1 » (with blanc between car)
    – use simple redirection (« < ») with a simple reference to file (« done < list.txt ») - use « ${VAR} » to manage variables - you can use numeric notation to increment VAR (p.e. « total=$(( ${total} + 1 )) ») An other way to do that: AWK. Regards

  • Thanks, when I change it do the following I get a syntax error

    #!/bin/bash
    #
    while read LINE
    do
    echo $LINE
    done < cat list.txt ./test_bash.sh ./test_bash.sh: line 6: syntax error near unexpected token `list.txt’ ./test_bash.sh: line 6: ` done < cat list.txt'

  • Hi Jerry,

    You can do even better:

    index=0
    total=0
    names=()
    ip=()

    IFS=,

    while read -r NODENAME IP
    do
    names[$index]=”$NODENAME”
    ip[$((index++))]=”$IP”
    ((total++))
    done < list.txt In this example, you set the input field separator (IFS) to the comma, so that the shell does the word splitting for you, and then allow the “read” call to assign to both variables in a single call. Now you don’t need any “cut” calls. In fact, this example avoids forks completely, by just using native features of bash. For small inputs, it doesn’t matter, but if you were processing a large file, and your script was forking for each call to “cut” and creating a new process, you would incur a horrible amount of overhead for no reason. Regards, Anand

  • I would add these considerations:
    1) -r means to not consider \ character as an escape character so possibly is not important to use it in your scenario
    2) read support multiple variables parsing
    3) read word separation (default space) can be modified using IFS (internal field separator)

    So I would change the script this way, together with other considerations

    #!/bin/bash
    #
    IFS=’,’

    index=0
    total=0
    names=()
    ip=()
    #while read -r LINE
    while read NODENAME IP
    do
    #NODENAME=$(echo $LINE | cut -f 1 -d ‘,’)
    #IP=$(echo $LINE | cut -f 2 -d ‘,’)
    names[$((index++))]=”$NODENAME”
    ip[$((index++))]=”$IP”
    ((total))
    done < /tmp/list.txt to verify you can add something like: for i in ${names[@]} ${ip[@]} do echo $i done HIH, Gianluca

  • You don’t use “cat” here; it’s not needed at all. You write:

    done < list.txt This tells the shell to redirect the stdin of the while loop from the file “list.txt”. People in the unix world have made a mess of code everywhere by superfluously using “cat”. In the old usenet days, anyone who posted shell code with unnecessary use of cat used to be awarded a prize (and it was not something to be proud of, but to be embarrassed about). “cat” is short for “concatenate”, and for that purpose, it is perfect. When you want to take two or more sources of data, and combine them, then cat is the perfect tool, eg: cat file1 file2 file3 > combined-file

    But for most other tasks, if you’re using “cat”, then you’re almost certainly misusing it. For example, people who do:

    cat file | grep something

    This makes the shell fork and run cat, and then the shell has to setup a pipe to pass the data to grep. Too much overhead. They don’t know that they can just do:

    grep something file

    and let the grep command read the file itself.

  • Sorry – I see it now “remove the cat”.

    Thanks “All” for the suggestions. I wasnt aware of the method to avoid the cut command.

    Jerry