Bash help requested: Capturing command errors within pipes

[Steven W. Orr]

On Saturday, Mar 21st 2009 at 22:41 -0000, quoth Cameron Simpson:

=>On 21Mar2009 16:47, Daniel B. Thurman <dant at cdkkt.com> wrote:
=>>> =>> out=$(grep "$pat" "${TRACKER}" | \
=>>> =>>          eval "$rex" | sort -n | \
=>>> =>>          uniq >> "${TFILE}"); ret="$?";
=>[...]
=>>> =>  if out=$(grep "$pat" "$tracker" | $rex | sort -un >>"$tfile")
=>>>   
=>> Please note:
=>>
=>> When I tried `sort -un', the data was truncated, i.e.
=>> there is data loss.  So, when I went back to my original
=>> code using 'sort -n | uniq',  there is no data loss.  There
=>> seems to be a problem using the `sort -un' method.
=>
=>Well, they do mean slightly different things.
=>
=>"sort -un" sorts and returns the first row of each set of rows that
=>sorted equal. (i.e. "1 foo" and "1 bah" sort equal (numeric) and only "1
=>foo" is returned. (See "man sort" for the details, and "man 1p sort" for
=>what you may portably expect on multiple UNIX platforms.)
=>
=>"uniq" discards repeated identical lines. "1 foo" and "1 foo" are
=>identical, but not "1 bah". (And uniq requires sorted input; the
=>repeated lines must be adjacent in the input.)
=>
=>It is often correct to replace "sort -n | uniq" with "sort -un", but I was
=>clearly wrong to do so here.
=>
=>> What I do in my code, is to create a copy of the sorted
=>> and uniq'd original file to a temp file, and then append
=>> new data to the temp file, then sorted and uniq the temp
=>> file back into the original file. The result was a file that
=>> ended up much smaller than the original file!

Fascinating. I never noticed that behaviour.

given 

518 > cat > xxx
1 foo
1 bah
519 > sort -un xxx
1 foo

even though foo comes after bar. But look, it gets better.

522 > sort -k 1,1 -n xxx
1 bah
1 foo
*523 > sort -k 1,1 -nu xxx
1 foo

So, even if I sort on the first column, the -u still selects the second 
record.

I guess I'm convinced that the only safe way is to *never* use -u.

-- 
Time flies like the wind. Fruit flies like a banana. Stranger things have  .0.
happened but none stranger than this. Does your driver's license say Organ ..0
Donor?Black holes are where God divided by zero. Listen to me! We are all- 000
individuals! What if this weren't a hypothetical question?
steveo at syslang.net