[ast-developers] Next alpha/beta and roadmap for ksh93v?

Post by Cedric Blancher
Glenn, when is the next alpha/beta due for AST and for UWIN? Is there
a roadmap what you are planning long-term for AST and UWIN?

I've been working on an ast-open pre-alpha that will (hopefully) clean up
pseudo devs, O_XATTR extended attributes, and *at() usage across ast by
these pseudo paths

/dev/<IP-PROTOCOL>/<HOST>/<PORT>
/dev/fd/<FD>[/...]
/proc/<PID>/fd/<FD>[/...]
/dev/file/<FLAG>[,<FLAG>...]:/<ABSOLUTE-PATH>
/dev/file/<FLAG>[,<FLAG>...]:<RELATIVE-PATH>
/dev/@/:/<ABSOLUTE-PATH>[/@/<RELATIVE-ATTRIBUTE-PATH>]
/dev/@/:<RELATIVE-PATH>[/@/<RELATIVE-ATTRIBUTE-PATH>]

currently ast + patches for the non-path variant of these pseudo paths
is morphing into an #ifdef mess making some portions unreadable and very fragile

the idea is to concentrate all of it in libast so it will be easy to
debug and extend -- in the current situation a bug fix for say O_XATTR
might involve a half-dozen or more files and that doesn't include the
ones that have yet to be patched for O_XATTR -- and these patches are rife
warnings that would surely get botched in future patches

the current pre-alpha has all ast commands, including cd, working
on a single O_XATTR entity without any patches to the commands
(of course making commands aware of the solaris-ish -@ options will
involve patches, but that's not part of the pre-alpha experiment)
"single O_XATTR entity" is key here too, e.g., pax does not yet handle
archiving a hierarchy+O_XATTR-counterparts, but the idea is that it
might be able to with a minimal patch involving a new FTS_XATTR
flag for ast::fts_open() detailed below

this is all in line with novel idea from 1970 that places as many arcane apis
as possible behind unix filesystem pathnames

dgk is concurrently working on ksh patches
there should be an ast-ksh package this week with those changes

I am currently deep inside ast::fts which is the directory traversal api
used by all ast utilities; part of this work involves the addition of
fgetcwd(int fd) which is a *at()-like variant of getcwd(), adding
FTSENT.fts_dirfd which is an open fd to the containg directory for
the current FTSENT; currently most ast commands that use fts do it
in the "chdir" mode where each FTSENT has "." set to the containing dir;
this includes the commands in -lcmd -- not a good thing in general;
fts_dirfd aware fts will allow the caller to fts_open(FTS_NOCHDIR) and use
*at(FTSENT->fts_dirfd, fts->fts_name) -- the challenge is to do this
without running out of fd's; along with this is a new fts_open() flag
FTS_XATTR that will traverse visible hierarchies and their hidden O_XATTR
counterparts, if any, within a single fts stream

re: uwin

we will probably put out a beta next week that does not include the latest ast
but does include patches for posix.dll, including signal upgrades and *at() syscalls

ast on uwin is an intersting beast because posix.dll co-opts and advertizes
some of ast for itself, and that includes vmalloc+aso which has changed
quite a bit this year and will take a least a week to soak in

so the uwin release will be in 3 stages
(1) posix.dll upgrade
(2) posix.dll + ast54.dll upgrade
(3) latest ast-open

Irek Szczesniak

2013-08-22 15:13:48 UTC

Post by Glenn Fowler

Post by Cedric Blancher
Glenn, when is the next alpha/beta due for AST and for UWIN? Is there
a roadmap what you are planning long-term for AST and UWIN?

<rant>
How often must people REPEAT that this attempt is USELESS? O_XATTR
doesn't have a name binding and cannot be assigned be without breaking
something else, usually catastrophically. This is one of the
experiments which just steal time, for example which could better be
invested to FIX the utterly broken job control in ksh93. Recall my
rant about the jobs variable being an utter misdesign because it is
accessed in unsafe manner from both signal handlers and normal code?
That code now breaks with clang -O2 or better, for good reasons
(again, ISO C violation).

FYI Merk Biostem uses files called @, @foo, foo@ and even brackets, so
/dev/@/: won't work. Almost every other valid ASCII character except
'\0' is allowed on Windows, rendering this attempt moot too.
</rant>

Irek

Glenn Fowler

2013-08-22 15:48:20 UTC

Post by Glenn Fowler

Post by Cedric Blancher
Glenn, when is the next alpha/beta due for AST and for UWIN? Is there
a roadmap what you are planning long-term for AST and UWIN?

<rant>
How often must people REPEAT that this attempt is USELESS? O_XATTR
doesn't have a name binding and cannot be assigned be without breaking
something else, usually catastrophically. This is one of the
experiments which just steal time, for example which could better be
invested to FIX the utterly broken job control in ksh93. Recall my
rant about the jobs variable being an utter misdesign because it is
accessed in unsafe manner from both signal handlers and normal code?
That code now breaks with clang -O2 or better, for good reasons
(again, ISO C violation).
'\0' is allowed on Windows, rendering this attempt moot too.
</rant>

dgk handles ksh issues and he is doing that as I type

like I said its pre-alpha
I'm aware of O_XATTR not residing the the namespace
that's the reason why code dealing with them is so crappy and fragile

the
/dev/@/:/<ABSOLUTE-PATH>
/dev/@/:<RELATIVE-PATH>
forms handle the -@ stuff, the second @ is experimental
since its experimental there may be some problems
in this case the existence of a non-O_XATTR file "@"
I'm aware of that possibility and don't have a solution to that working *yet*
but it is certainly possible to detect and differentiate a real "@" from an attribute file
what that differentiation looks like I don't know *yet*
that would be the next part of the experiment

btw windows is not the best reference because although it allows any ASCII char,
some of them have special windows-only meaning
< > : " \ | have special meaning
: is used for windows equivalent to O_XATTR
/ and \ can be path separators
aux, nul, prn, com[0-9] are special file names in *any* directory
not to mention case ignorance

Tina Harriott

2013-08-22 16:59:10 UTC

Post by Glenn Fowler

Post by Cedric Blancher
Glenn, when is the next alpha/beta due for AST and for UWIN? Is there
a roadmap what you are planning long-term for AST and UWIN?

Are there details of the changes for ast-ksh? I'm running with Roland
Mainz's patch for integer and floating-point constants and I'm quite
happy with it (except that I have to define the constants manually for
each project script).
I'd like to have it included for the next ast-ksh.

Tina

--
Tina Harriott - Women in Mathematics
Contact: tina.harriott.math at gmail.com

David Korn

2013-08-22 18:03:08 UTC

cc: ast-developers at research.att.com cedric.blancher at gmail.com
Subject: Re: Re: [uwin-developers] [ast-developers] Next alpha/beta and roadmap for ksh93v?
--------

Post by Tina Harriott
Are there details of the changes for ast-ksh? I'm running with Roland
Mainz's patch for integer and floating-point constants and I'm quite
happy with it (except that I have to define the constants manually for
each project script).
I'd like to have it included for the next ast-ksh.
Tina

The floating point constant and the sigqueue patches will not be in the
alpha this week. I expect they will be in the next alpha.

The floating point constants are likely only avaiable within ((...))
and the math constants may only be available for long double.

The next release will have many of the reported bugs fixed.

David Korn
dgk at research.att.com

Tina Harriott

2013-08-23 11:27:40 UTC

Post by David Korn
cc: ast-developers at research.att.com cedric.blancher at gmail.com
Subject: Re: Re: [uwin-developers] [ast-developers] Next alpha/beta and roadmap for ksh93v?
--------

The floating point constant and the sigqueue patches will not be in the
alpha this week. I expect they will be in the next alpha.
The floating point constants are likely only avaiable within ((...))

Why? How should I construct scripts from that using variables, i.e.
use typeset -s -E svar; printf 'typeset -s -E var; for ((var = %s; var
< %s; var+= %s)); do myfunc var; done' ${svar.MIN} ${svar.MAX}
$((2**24)) | source /dev/stdin

Post by David Korn
and the math constants may only be available for long double.

That part doesn't make sense. Like functions nextafter() and
nexttoward() the constants for a variable depend on a type for at
least minimum, maximum and epsilon or they are not usable.

Tina

--
Tina Harriott - Women in Mathematics
Contact: tina.harriott.math at gmail.com

David Korn

2013-08-23 12:38:40 UTC

cc: ast-developers at research.att.com cedric.blancher at gmail.com
Subject: Re: Re: Re: [uwin-developers] [ast-developers] Next alpha/beta and roadmap for ksh93v?
--------

Post by Tina Harriott
Why? How should I construct scripts from that using variables, i.e.
use typeset -s -E svar; printf 'typeset -s -E var; for ((var = %s; var
< %s; var+= %s)); do myfunc var; done' ${svar.MIN} ${svar.MAX}
$((2**24)) | source /dev/stdin

svar.MIN and svar.MAX will exist, but only in arithemtic expressions.
Thus, $((svar.MIN)) and $((svar.MAX)) instead of ${svar.MIN} ${svar.MAX}.
Only constants like PI, NaN, Inf, and others will not exist for each type.
The limits would.

David Korn
dgk at research.att.com

Tina Harriott

2013-08-23 13:52:43 UTC

Post by David Korn
cc: ast-developers at research.att.com cedric.blancher at gmail.com
Subject: Re: Re: Re: [uwin-developers] [ast-developers] Next alpha/beta and roadmap for ksh93v?
--------

I don't understand this. The patch
astksh20130814_math_constants001.diff.txt already allows MIN, MAX,
EPSILON etc if they are defined in .sh.mathconst. Why is the
functionality now restricted to $(())? It just makes it much harder to
use, if usable at all.

Tina

--
Tina Harriott - Women in Mathematics
Contact: tina.harriott.math at gmail.com

Cedric Blancher

2013-08-23 18:19:38 UTC

Post by David Korn
cc: ast-developers at research.att.com cedric.blancher at gmail.com
Subject: Re: Re: Re: [uwin-developers] [ast-developers] Next alpha/beta and roadmap for ksh93v?
--------

I don't understand it either.

David, is there a reason why you try to reinvent the wheel, or is
there anything fundamentally wrong with Roland Mainz's
astksh20130814_math_constants001.diff.txt patch?

I think that Roland's patch deals with all issues I can imagine of
(except predefining some constants, which is IMO easy to solve), adds
headroom for further enhancements (e.g. if we wish to implement
typeset -D to gain access to _Decimal32, _Decimal64 and _Decimal128
types to please COBOL migratory people), AFAIK can solve bool b;
b.true/b.false and other enum constants with little or no extra code,
supports ${var.fooconst} and supports user defined constants (IMO very
important)

Ced

--
Cedric Blancher <cedric.blancher at gmail.com>
Institute Pasteur

Simon Toedt

2013-08-26 04:29:31 UTC

On Fri, Aug 23, 2013 at 8:19 PM, Cedric Blancher

Post by Cedric Blancher

Post by David Korn
cc: ast-developers at research.att.com cedric.blancher at gmail.com
Subject: Re: Re: Re: [uwin-developers] [ast-developers] Next alpha/beta and roadmap for ksh93v?
--------

I don't understand it either.
David, is there a reason why you try to reinvent the wheel, or is
there anything fundamentally wrong with Roland Mainz's
astksh20130814_math_constants001.diff.txt patch?
I think that Roland's patch deals with all issues I can imagine of
(except predefining some constants, which is IMO easy to solve), adds
headroom for further enhancements (e.g. if we wish to implement
typeset -D to gain access to _Decimal32, _Decimal64 and _Decimal128
types to please COBOL migratory people), AFAIK can solve bool b;
b.true/b.false and other enum constants with little or no extra code,
supports ${var.fooconst} and supports user defined constants (IMO very
important)

+1

The other question if supporting var.fooconst in $(()) but not in
${var.fooconst} would be in the spirit of POSIX. I think not. It'll
look like a cumbersome hack, plus will break so many useful use cases.

BTW: bool is an integer type in ANSI C, right? So bool b;
${b.max}/$((b.max)) should work, right?

Simon

Irek Szczesniak

2013-08-26 07:54:46 UTC

On Fri, Aug 23, 2013 at 3:52 PM, Tina Harriott

Post by David Korn
cc: ast-developers at research.att.com cedric.blancher at gmail.com
Subject: Re: Re: Re: [uwin-developers] [ast-developers] Next alpha/beta and roadmap for ksh93v?
--------

Right. I don't like David's approach either, unless there is a very
damn good justification for castrating numeric constants to $(()) and
not for ${}. Given that Roland's patch works, minus having
preconfigured constants, and can be extended to cover enums (bool is
an enum) I don't see a reason for not taking that patch.

Irek

David Korn

2013-08-26 14:30:59 UTC

cc: ast-developers at research.att.com
Subject: Re: Re: [ast-developers] [uwin-developers] Next alpha/beta and roadmap for ksh93v?
--------

OK, let me preset some reasons:

1. These variables are only arithmetic related. Just like NaN, nan,
Inf, and inf which also don't give the same result for ${NaN} and
$((NaN)).

2. The implementation is inconsistant in that ${!foo*} does not
expand all integer variables starting with foo to each of the
18 integer or 27 possible float suffices for integer and float
respectively.

3. Rolands implementation used about 2500 bytes of read write
memory which will be copied for each fork(). Mine use 0 more
bytes of read/write memory.

4. There is no additional functionlity created by allowwing ${i.MAX}.

I am planning to only support enum.constant in numerical expressions
as well where it belongs.

The changes will be in the next alpha.

David Korn
dgk at research.att.com

Tina Harriott

2013-08-26 16:02:04 UTC

Post by David Korn
cc: ast-developers at research.att.com
Subject: Re: Re: [ast-developers] [uwin-developers] Next alpha/beta and roadmap for ksh93v?
--------

1. These variables are only arithmetic related. Just like NaN, nan,
Inf, and inf which also don't give the same result for ${NaN} and
$((NaN)).

That may be bad, but then it could be written as ${val.NAN}. NAN and
INF/-INF are specific to the floating point type anyway.

Post by David Korn
2. The implementation is inconsistant in that ${!foo*} does not
expand all integer variables starting with foo to each of the
18 integer or 27 possible float suffices for integer and float
respectively.

This would violate type-safety. The ${!foo.*} should only expand to
the names of the constants *valid* for the type of variable foo and
not all possible constants.

Post by David Korn
3. Rolands implementation used about 2500 bytes of read write
memory which will be copied for each fork(). Mine use 0 more
bytes of read/write memory.

I wouldn't mind of loosing 2k of memory if the code does what a
mathematician expects from the code

Post by David Korn
4. There is no additional functionlity created by allowwing ${i.MAX}.

I disagree. I even provided the example why it's important.

Post by David Korn
I am planning to only support enum.constant in numerical expressions
as well where it belongs.

Why? Like C++, JAVA, Fortess, python etc enums are numeric types and
thus should have the same behaviour as all other numeric types.

This example should IMO just work:
function f
{
nameref nv=$1
print "maxval=${nv.MAX}"
}
if (( condition1 ))
bool val
else
integer val
fi
f val

Tina

--
Tina Harriott - Women in Mathematics
Contact: tina.harriott.math at gmail.com

Wendy Lin

2013-08-26 17:17:41 UTC

Post by David Korn
cc: ast-developers at research.att.com
Subject: Re: Re: [ast-developers] [uwin-developers] Next alpha/beta and roadmap for ksh93v?
--------

1. These variables are only arithmetic related. Just like NaN, nan,
Inf, and inf which also don't give the same result for ${NaN} and
$((NaN)).

I like Mrs. Harriott's solution of x.NAN, i.e. make NAN a per type
constant analogous to
http://docs.oracle.com/javase/7/docs/api/java/lang/Float.html#NaN or
python float('nan')

Can this be added for Mr. Mainz's patch?

Post by David Korn
3. Rolands implementation used about 2500 bytes

2500 bytes per variable or 2500 bytes per shell session? 2500 bytes
isn't much - ksh93 requires at least 4MB to run on a 64bit machine, in
a world where mobile phones have 1GB and lowend desktops have 8GB
memory.

Post by David Korn
of read write
memory which will be copied for each fork(). Mine use 0 more
bytes of read/write memory.

What you loose is flexibility. The flexibility of user-defined
constants or even typed functions. I've experimented with Mr. Mainz's
patch a bit and found it useful to add functions to floats:
ksh -c 'compound .sh.mathconst=( float LDBL_MYVAL=nan ) ; function
.sh.mathconst.LDBL_MYVAL.getn { .sh.value=4 ; } ; float x=5.5 ; print
$((x.MYVAL)) ${x.MYVAL}'
4 4

Post by David Korn
4. There is no additional functionlity created by allowwing ${i.MAX}.

Why do you think that? Even C++ and python have implicit string
conversions and Java has the Object.toString() function, which is sh's
equivalent to float x; ${x}

Wendy

David Korn

2013-08-26 14:59:37 UTC

cc: ast-developers at lists.research.att.com
Subject: Re: [ast-developers] [uwin-developers] Next alpha/beta and roadmap for ksh93v?
--------

Post by Simon Toedt
BTW: bool is an integer type in ANSI C, right? So bool b;
${b.max}/$((b.max)) should work, right?
Simon

If max is an enumeration constant for the type of b what should the value be?

David Korn
dgk at research.att.com

Tina Harriott

2013-08-26 15:53:17 UTC

Post by David Korn
cc: ast-developers at lists.research.att.com
Subject: Re: [ast-developers] [uwin-developers] Next alpha/beta and roadmap for ksh93v?
--------

Post by Simon Toedt
BTW: bool is an integer type in ANSI C, right? So bool b;
${b.max}/$((b.max)) should work, right?
Simon

If max is an enumeration constant for the type of b what should the value be?

First rule: Numeric constants should be all uppercase
Second rule: Enumeration constants have priority over numeric constants

Second rule would mean that if an enumeration as en.max it takes
precedent over a constant of the same name (natural law, at least in
C++, java and python). The first rule would ensure that they stay out
of each others way unless the programmer asks for trouble.

Tina

--
Tina Harriott - Women in Mathematics
Contact: tina.harriott.math at gmail.com

David Korn

2013-08-26 18:53:25 UTC

cc: iszczesniak at gmail.com ast-developers at research.att.com
Subject: Re: Re: [ast-developers] [uwin-developers] Next alpha/beta and roadmap for ksh93v?
--------

Post by David Korn
cc: ast-developers at research.att.com
Subject: Re: Re: [ast-developers] [uwin-developers] Next alpha/beta and roadma

p for ksh93v?

Post by David Korn
--------

Post by David Korn
The limits would.

1. These variables are only arithmetic related. Just like NaN, nan,
Inf, and inf which also don't give the same result for ${NaN} and
$((NaN)).

I like Mrs. Harriott's solution of x.NAN, i.e. make NAN a per type
constant analogous to
http://docs.oracle.com/javase/7/docs/api/java/lang/Float.html#NaN or
python float('nan')

If you want a typeset -sE Nan, you can do
typeset -sE x=NaN
and x will be a float sized NaN.

Can this be added for Mr. Mainz's patch?

I suppose that it could, but not worth it.

Post by David Korn
3. Rolands implementation used about 2500 bytes

Post by David Korn
of read write
memory which will be copied for each fork(). Mine use 0 more
bytes of read/write memory.

What you loose is flexibility. The flexibility of user-defined
constants or even typed functions. I've experimented with Mr. Mainz's
ksh -c 'compound .sh.mathconst=( float LDBL_MYVAL=nan ) ; function
.sh.mathconst.LDBL_MYVAL.getn { .sh.value=4 ; } ; float x=5.5 ; print
$((x.MYVAL)) ${x.MYVAL}'
4 4

To me, this is a reason against it. I don't want constants that are not
constant and differ by type.

Post by David Korn
4. There is no additional functionlity created by allowwing ${i.MAX}.

Why do you think that? Even C++ and python have implicit string
conversions and Java has the Object.toString() function, which is sh's
equivalent to float x; ${x}

x=$((x)) gives is equivalent.
I don't think that i.MAX should be a real variable. If it is then how
to does Roland's code handle
nameref ref=i.MAX
print ${ref.__}

Post by David Korn
Wendy

David Korn
dgk at research.att.com

Wendy Lin

2013-08-26 19:00:48 UTC

Post by David Korn
cc: iszczesniak at gmail.com ast-developers at research.att.com
Subject: Re: Re: [ast-developers] [uwin-developers] Next alpha/beta and roadmap for ksh93v?
--------

Post by David Korn
cc: ast-developers at research.att.com
Subject: Re: Re: [ast-developers] [uwin-developers] Next alpha/beta and roadma

p for ksh93v?

Post by David Korn
--------

To me, this is a reason against it. I don't want constants that are not
constant and differ by type.

No no, you got it wrong. I wished to demonstrate that it is possible
(later!?) to implement per-type functions, e.g. typeset -E x; ((
c=x.hashCode() )) which calls a function hashCode which is specific
for this type. Like JAVA does in
http://docs.oracle.com/javase/6/docs/api/java/lang/Double.html#hashCode%28%29

Wendy

David Korn

2013-08-27 21:34:26 UTC