[Tfug] rereading stdin

[Stephen Hooper]

Claude Rubinson wrote:
> I'm wondering how people handle situations in which they need to step
> through stdin more than once.  As an example, lets say that I want to
> compute some deviation statistics: x - mean(x).  The formula requires
> that I step through the input twice, once to compute mean(x) and once
> to compute x - mean(x) for every x.
> 
> If I'm working in a higher-level language (such as Python), this isn't
> an issue as I'll have already read the input into an object of some
> sort.  Where it rears its head is when I'm working in shell (as is my
> preference).  I can read the input into variables (or, more likely,
> arrays) or write it first to a temp file (possibly using tee).  I tend
> to prefer the second solution (with tee) but both feel kludgey to me.
> 
> Thoughts?
> 

If you are using Bourne shell:

<pre>
#!/bin/sh

# Get a tempfile
tmpfile=$(mktemp -p /tmp fooXXXXXXX) || exit 1

# Open up some fd's to it


# Fancier shells may allow you to "rewind" 
# a file descriptor. I don't think POSIX
# does, but as bash can write sockets,
# I wouldn't be surprised if it couldn't 
# do most anything.

exec 10>${tmpfile}
exec 11<${tmpfile}
exec 12<${tmpfile}

# Remove the file so that if something dies
# we don't have to worry about cleaning it up
rm ${tmpfile}

# Put stdin into it
cat >&10  # We opened 10 so that in case "cat" died, the tempfile would still be dissappeared.  In the case of "cat" this is probably a little silly.

# Read stdin from the file
(
read line
while  [ $? -eq 0 ] ; do
  echo "My super line: ${line}"
  read line
done
) <&11

# Read stding from the file
(
read line
while  [ $? -eq 0 ] ; do
  echo "My second super line: ${line}"
  read line
done
) <&12

</pre>

The limitations are that you need space to equal the amount you are reading in, and that you end up opening an fd for each time you want to read the data in (as there is no rewind).