[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Bash help requested: Capturing command errors within pipes



Steven W. Orr wrote:
On Friday, Mar 20th 2009 at 00:39 -0000, quoth Cameron Simpson:

=>On 19Mar2009 18:55, Daniel B. Thurman <dant cdkkt com> wrote:
=>> Cameron Simpson wrote:
=>>> Ok, but I strongly recommend you never us a regexp unquoted - the
=>>> necessary backslash nesting gets nasty real fast. A better way is like
=>>> this:
=>>>   re='s/b/h/'
=>>>   sed -e "$re"
=>[...]
=>>> The point here, unrelated to the pipe exit status issue, is to keep the
=>>> sed stuff easy to write and undamaged by shell substitution. [...]
=>>
=>> Another person sent me private email
=>> warning of the double-quotes, but thanks
=>> for your information, it is interesting!
=>>
=>> This is what I ended up doing, so please give
=>> your constructive critiques, if you so like?
=>
=>Untested, but here is how I would write each of you code pieces were I doing
=>it myself:
=>
=>> TRACKER="Tracker.log"
=>> SFILE="Bad_Hosts_File.txt"
=>> TFILE="Bad_Hosts_File_$$.txt"
=>
=>tracker=Tracker.log
=>sfile=Bad_Hosts_File.txt
=>tfile=Bad_Hosts_File_$$.txt
=>
=>Your quotes are unneeded here. Plenty of people find the "" marks and in
=>similar cases, to use ${foo} instead of $foo elsewhere, but I find the
=>syntactic noise annoying if there's no other necessity.
=>
=>Also, you should never use UPPER CASE names for script local variables (i.e.
=>that are not to be exported). I give some background why here:
=>
=>  http://markmail.org/message/awimhb3nmu6ssvwp

One historical tidbit I'd like to share with you. Back when the TRex was tromping around, I worked at Data General during Soul Of A New Machine. The deal there was that all external variables under AOS had to be uppercase. The reason for this was because we sometimes needed to access variables by name at link-time and the CLI (Command Line Interface, a sort of macro processor) was case insensitive, so all commands caused everything to be flipped to uppercase, no matter what we wanted. The result was that all external variables ended up as uppercase, just in case. :-)

=>
=>> pat="EACC"
=>
=>pat=EACC
=>
=>> re1="-e '/$pat/s/^.*Received:\[from\s\+\(.*\)\s\+(.*/\1/'"
=>> re2="-e '/^.*cdkkt.com$/d'"
=>
=>Note that this "$" should probably be backslashed. As it happens, $/ is
=>not a shell special variable, so it is left alone. However, has you used
=>$# or one of the others ($!, et al) there would have been substitution
=>done, mangling your sed code.
=>
=>> rex="sed $re1 $re2"
=>
=>I would strongly reommend putting this in a sed script file, perhaps
=>like this:
=>
=>  cat >sedf$$ <<X
=>    /$pat/s/^.*Received:\[from\s\+\(.*\)\s\+(.*/\1/
=>    /^.*cdkkt.com\$/d
=>  X
=>  rex="sed -f sedf$$"
=>
=>Hmm, actually, maybe not. Maybe like this:
=>
=>  sedline="/$pat/s/^.*Received:\[from\s\+\(.*\)\s\+(.*/\1/; /^.*cdkkt.com\$/d"
=>
=>and then:
=>
=>  sed -e "$sedline"
=>
=>in your pipeline lower down.
=>
=>> echo -en "\033[0;36m>> \033[0;35mExtracting data from:\n \
=>>  [\033[0;34m${TRACKER}\033[0;35m and appending to\n \
=>>  [\033[0;34m${TFILE}\033[0;35m]\033[0m"
=>
=>I usually put escape sequences into their own variables for readbility,
=>like this:
=>
=>  http://www.cskk.ezoshosting.com/cs/rc/shell/colour_ansi
=>
=>Then you can talk about ${tty_green}, ${tty_normal} etc in your text;
=>somewhat more readable. You can do even better than this, actually.
=>Since not all terminals use the ANSI colour escapes, it is better to use
=>tput, like this:
=>
=>  http://www.cskk.ezoshosting.com/cs/css/bin/with-colour
=>
=>and then in your script:
=>
=>  esc_tracker=`with-colour green echo "$TRACKER"`
=>
=>and then use ${esc_tracker} in messages. And so forth.
=>
=>> out=$(grep "$pat" "${TRACKER}" | \
=>>          eval "$rex" | sort -n | \
=>>          uniq >> "${TFILE}"); ret="$?";
=>>
=>> if [ "$ret" -gt 0 ] || \
=>>   [ "$PIPESTATUS[0]" -gt 0 ] || \
=>>   [ "$PIPESTATUS[1]" -gt 0 ] || \
=>>   [ "$PIPESTATUS[2]" -gt 0 ] || \
=>>   [ "$PIPESTATUS[3]" -gt 0 ]; then
=>>   echo -e " \033[0;31mFailed.\n   \
=>>   \033[0;35m-- ErrorStatus:<ret=$ret>,\
=>>  PipeStatus:<$PIPESTATUS[ ]>\n\
=>>  -- out:<$out>\033[0m"
=>>   echo -e "\033[0;31m>> \033[0;31m$PROG exited.\033[0m\n"
=>>   exit 1
=>> fi
=>> echo -e " \033[0;32mDone.\033[0m";
=>>
=>> I guess the ugly part is the multiple $PIPESTATUS[n]
=>> in the if statement test block. but perhaps this is the best
=>> I can do?
=>
=>Nah. You can do this:
=>
=>  if out=$(grep "$pat" "$tracker" | $rex | sort -un >>"$tfile")
Please note:

When I tried `sort -un', the data was truncated, i.e.
there is data loss.  So, when I went back to my original
code using 'sort -n | uniq',  there is no data loss.  There
seems to be a problem using the `sort -un' method.

What I do in my code, is to create a copy of the sorted
and uniq'd original file to a temp file, and then append
new data to the temp file, then sorted and uniq the temp
file back into the original file. The result was a file that
ended up much smaller than the original file!
=>  then
=>    case " $PIPESTATUS" in
=>      *' '[1-9]*)
=>        echo exit ok, but PIPESTATUS=$PIPESTATUS
=>        ;;
=>      *)echo exit ok and PIPESTATUS all zeroes
=>        ;;
=>    esac
=>  else
=>    echo exit not ok
=>  fi
=>
=>Um, you are aware that $out will always be empty? You have sent
=>standard out to the temp file. Standard error is not collected by back
=>ticks or $(); it will be displayed directly and not land in $out.
=>You probably need to go:
=>
=>  out=$(exec 2>&1; grep .... >>"$tfile")
=>
=>An easy test is like this:
=>
=>  out=$(ls /no/such/file >>temp)
=>  echo "out=[$out]"
=>
=>versus:
=>
=>  out=$(exec 2>&1; ls /no/such/file >>temp)
=>  echo "out=[$out]"
=>
=>Cheers,
=>-- =>Cameron Simpson <cs zip com au> DoD#743
=>http://www.cskk.ezoshosting.com/cs/
=>
=>The aim of AI is to make computers act like the ones in the movies.
=>        - Graham Mann
=>
=>-- =>fedora-list mailing list
=>fedora-list redhat com
=>To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
=>Guidelines: http://fedoraproject.org/wiki/Communicate/MailingListGuidelines
=>


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]