[ast-developers] Problems with read -p and co-processes

[Terrence J. Doyle]

I kicked up a couple bugs while testing the variations on the way
"read -p" is now handled. In the following example I start by testing
"read -p" for generating a prompt -- a rather poor prompt, but it's a
lead-up to what I'm showing later. There are no problems with "read -p"
in prompt mode. Next, I start a co-process and do several successful
variations on printing to the co-process. But, there are problems when I
try to read back what was printed. As the comments in the example
describe "read x <&p" and "read -px" (no space between the p and x) have
problems.

$ read -p x
x1
$ print $REPLY
1
$ read -px
x2
$ print $REPLY
2
$ while read -r line

do
[[ "$line" == [Qq] ]] && break
print -r -- "$line" >> /tmp/test
print -r "read: $line"
done |&

[1] 40391
$ print -up 'Testing'
$ print 1 >&p
$ print -p 2
$ print -p 3
$ read -up x
$ print $x
read: Testing
#
# The next read hangs. Control-C returns the shell prompt. The
# co-process pipe is not read ($x still has the value of the previous
# read). <&p works in ksh-20120612, but not in ksh-20140929.
#
$ read x <&p

^C$ print $x
read: Testing
$ read -p x
$ print $x
read: 1
$ #
$ # The next read hangs hard. The system load jumps. After a long pause
$ # (> 1 minute) the error message appears followed by the bus error.
$ # Debugging with gdb reveals that the optget() loop in b_read() is
$ # being called repeatedly to case 'p' (line 230 in
$ # src/cmd/ksh93/bltins/read.c) because of the argv-- at line 237. The
$ # behavior with ksh-20120612 was to declare x an invalid option
$ # without hanging and without dieing with a bus error.
$ #
$ read -px
^C
^\
^Z

ksh: read: -px: invalid variable name
Bus error


I also did some digging with gdb for the "read x <&p" problem. I set
a breakpoint at line 337 in src/cmd/kdh93/bltins/read.c, which looks
like this:

r=sh_readline(shp,argv,readfn,fd,flags,len,timeout);

Then, I ran the shell, created the co-process as in the example above,
did the same prints to the co-process and ran "read -up x". At the
breakpoint the variable, fd, was 5, the same as shp->cpipe[0]. Then, I
continued and ran "read x <&p". This time the variable, fd, was 0
(zero), while shp->cpipe[0] was, of course, still 5. So, the problem
here appears to be that <&p is associated with stdin instead of the
co-process pipe. After I continued the read hung as before. I typed
control-C and did a where in gdb. I attached the output of the where
command in read_hang_where.txt.

Terrence Doyle

[Terrence J. Doyle]

I've come up with a fix for "read -px" (in co-process mode) and a
why for "read x <&p".

The fix for "read -px" is to simply put "read -p" into prompt mode
when an infinite loop is detected, return "x" as a prompt and read from
stdin instead of the co-process pipe. I was kind of disappointed with
this approach at first, and I abandoned it for a while. But, then I
realized that it's the only way to fix this problem (as far as I can see
-- if you have a better idea, post it). The patch to
src/cmd/ksh93/bltins/read.c is attached as read.c.diff.

As I looked more closely at the "read x <&p" hang, I began to notice
that it occurred only after a "read -up" is issued. If fact, if I do a
"print -p" or a "read -p" in between "read -up" and "read x <&p",
there's no problem with "read x <&p". It really looks like the -up
specification for read is the cause of this problem. When I looked at
case 'u' in the optget() switch in b_read(), I found an obscure little
mistake at the end of the case. Here's the section of code from read.c
with line numbers:

260 case 'u':
261 if(opt_info.arg[0]=='p' && opt_info.arg[1]==0)
262 {
263 if((fd = shp->cpipe[0])<=0)
264
errormsg(SH_DICT,ERROR_exit(1),e_query);
265 break;
266 }
267 fd = (int)strtol(opt_info.arg,&opt_info.arg,10);
268 if(*opt_info.arg)
269 fd = -1;
270 else if(!sh_iovalidfd(shp,fd))
271 fd = -1;
272 else if(!(shp->inuse_bits&(1<<fd)) &&
(sh_inuse(shp,fd)
|| (shp->gd->hist_ptr && fd==sffileno(shp->gd->hist_ptr->histfp))))
273 break;
274 case 'v':
275 flags |= V_FLAG;
276 break;

There appears to be at least a line of code missing after the "else if"
at line 272. With things as they are the break to end the case at line
273 gets executed only if the "else if" at line 272 is reached and
evaluates to true. It looks like at least sometimes case 'u' is falling
through to case 'v'.

Terrence Doyle

Post by Terrence J. Doyle
I kicked up a couple bugs while testing the variations on the way
"read -p" is now handled. In the following example I start by testing
"read -p" for generating a prompt -- a rather poor prompt, but it's a
lead-up to what I'm showing later. There are no problems with "read -p"
in prompt mode. Next, I start a co-process and do several successful
variations on printing to the co-process. But, there are problems when I
try to read back what was printed. As the comments in the example
describe "read x <&p" and "read -px" (no space between the p and x) have
problems.
$ read -p x
x1
$ print $REPLY
1
$ read -px
x2
$ print $REPLY
2
$ while read -r line

do
[[ "$line" == [Qq] ]] && break
print -r -- "$line" >> /tmp/test
print -r "read: $line"
done |&

[1] 40391
$ print -up 'Testing'
$ print 1 >&p
$ print -p 2
$ print -p 3
$ read -up x
$ print $x
read: Testing
#
# The next read hangs. Control-C returns the shell prompt. The
# co-process pipe is not read ($x still has the value of the previous
# read). <&p works in ksh-20120612, but not in ksh-20140929.
#
$ read x <&p
^C$ print $x
read: Testing
$ read -p x
$ print $x
read: 1
$ #
$ # The next read hangs hard. The system load jumps. After a long pause
$ # (> 1 minute) the error message appears followed by the bus error.
$ # Debugging with gdb reveals that the optget() loop in b_read() is
$ # being called repeatedly to case 'p' (line 230 in
$ # src/cmd/ksh93/bltins/read.c) because of the argv-- at line 237. The
$ # behavior with ksh-20120612 was to declare x an invalid option
$ # without hanging and without dieing with a bus error.
$ #
$ read -px
^C
^\
^Z
ksh: read: -px: invalid variable name
Bus error
I also did some digging with gdb for the "read x <&p" problem. I set
a breakpoint at line 337 in src/cmd/kdh93/bltins/read.c, which looks
r=sh_readline(shp,argv,readfn,fd,flags,len,timeout);
Then, I ran the shell, created the co-process as in the example above,
did the same prints to the co-process and ran "read -up x". At the
breakpoint the variable, fd, was 5, the same as shp->cpipe[0]. Then, I
continued and ran "read x <&p". This time the variable, fd, was 0
(zero), while shp->cpipe[0] was, of course, still 5. So, the problem
here appears to be that <&p is associated with stdin instead of the
co-process pipe. After I continued the read hung as before. I typed
control-C and did a where in gdb. I attached the output of the where
command in read_hang_where.txt.
Terrence Doyle