Discussion:
[ast-developers] AT&T Software Technology ast alpha software download update
Glenn Fowler
2013-09-13 22:15:15 UTC
Permalink
the AT&T Software Technology ast alpha 2013-09-13 source release
has been posted to the download site
http://www.research.att.com/sw/download/alpha/
the package names and md5 checksums are
INIT 327861e49e24dd51079c0a5316a4b2fe
ast-open dfb85d1dfb20acb8a1529bdf4b8cb89a
ast-ksh 746a556a2259aaa6d75468000e5bc36b
the md5 sums should match the ones listed on the download page

the change logs below are limited to ksh and libast
the libast changes involved a lot of meticulous multibyte code
that was hashed out off-list between { gsf roland olga }

there is a "news" link in the left side nav bar that will
be used to detail implemented and proposed ast features
as features mature the news info will migrate to the man pages

if your favorite bug/feature is not in the list below then it
hasn't been addressed yet and we don't know exactly when it will be

changes since 2013-08-29

:::::::: ksh93 ::::::::

13-09-13 --- Release ksh93v- ---
13-09-13 +The signal .sh.value variable is now a compound variable with the name
value.q corresponding to kill -q signed-integer and value.Q corresponding
to kill -Q unsigned-large-integer.
13-09-13 A bug in $(...) command substitution that corrupted a trailing
multibyte character in non-UTF-8 locales has been fixed.
13-09-13 Eliminted extranesous output of standard error when ksh is invoked
with the -v (verbose) option.
13-09-10 A bug in finding a function defined inside a type that was defined
in a namespace has been fixed.
13-09-10 A bug in the binding of function local variables inside arithmeitc
expression inside namespaces was fixed.
13-09-10 +A -Q option was added to kill to pass integers as large as pointers.
The -q option now only accepts integers as large as typeset -i.
13-09-09 A bug in command substitution has been fixed.
13-09-09 Qualified print format "%([no]unicode)q" added to prefer \u[...]
over \w[...] and override LC_OPTIONS=unicode.
13-09-04 +\w[hex] locale-specific code point literals have been added.
13-09-04 +The float(f) math function was added.
13-09-04 +The int(f) math function was fixed to return 0 for floating point
numbers larger than the maximum integer.
13-09-04 A bug in which assigning a compound indexed array a value of () did
not preserve the -C attribute has been fixed.
13-09-04 kill -q can now pass numbers as large as typeset -li and
.sh.sig.value is typeset -i rather than a compound variable.
13-09-04 kill -q yields the processor and returns 2 when siqueue fails with
EAGAIN and yield.
13-09-03 A bug in which $((x.xxx)) where x is a floating point variable and
xxx is not one of the known extensions yields a random value has
been fixed. It now is unset which has value 0 when set -u is off.
13-09-03 A bug in overriding discipline functions for types defined in
namespaces has been fixed.
13-09-03 A bug which on some systems caused a core dump for large <<< here
documents has been fixed.

:::::::: libast ::::::::

13-09-12 misc/fgetcwd.c: fix stat corruption bug on systems without fdopendir()
13-09-12 path/pathcanon.c: fix bug that added extra / when fgetcwd() returned /
13-09-09 comp/setlocale.c,port/codeset.c: consistent handling of US-ASCII + conformance(0,0) across all os's
13-09-09 string/utf8towc.c,string/wctoutf8.c: add { utf8toutf32() utf8towc() wctoutf8() }
13-09-07 include/ast_std.c,comp/setlocale.c,string/stresc.c: add ast.byte_max for single byte locales
13-09-06 comp/iconv.c: add sfclrerr() to iconv_move() if all input chars not consumed
13-09-06 port/codeset.h,port/codeset.c: internal api for retrieving locale codeset names
13-09-04 string/chresc.c,stresc.c: add \w[hex] support -- thanks Roland
13-09-04 string/utf32stowcs.c,string/wcstoutf32s.c: add -- thanks Roland
13-09-04 sfio/sfsetbuf.c: fix bug where SF_GETR mode was not cleared causing subsequent memory corruption
13-09-04 vmalloc/vmopen.c,vmdcsystem.c,vmstat.c: temporarily set vm->meth.meth=0 to disable vmstat() during init
13-09-04 port/intercept.c: include <ast_standards.h> to ensure fdopendir() prototype if _lib_fdopendir
13-09-04 include/ast_std.h,comp/setlocale.c: add LC_OPTIONS=unicode and AST_LC_unicode
13-09-01 path/pathcanon.c: O_* flags dev path: /dev/file/flags at flag[,flag...]@[/]path
13-09-01 path/pathcanon.c: limit NAMED_XATTR paths to /dev/file/xattr at canonical-path//@//[remainder]
13-08-29 cdt/dtstrhash.c: change sign-bit hitting fnv constants to hex to silence unsigned warnings
ольга крыжановская
2013-09-13 22:38:46 UTC
Permalink
Glenn, thanks for the alpha.

I've walked over Roland's notes about patches and found these easy-to-do items:
- sync syncfs() patch
- libcmd at()ify
- tar.sh localisation
- contribute script_l10n test module to dgk
- build ast-ksh C catalog files, by default, and install them in arch/

Does any of these patches require any updates, or changes?

Olga
Post by Glenn Fowler
the AT&T Software Technology ast alpha 2013-09-13 source release
has been posted to the download site
http://www.research.att.com/sw/download/alpha/
the package names and md5 checksums are
INIT 327861e49e24dd51079c0a5316a4b2fe
ast-open dfb85d1dfb20acb8a1529bdf4b8cb89a
ast-ksh 746a556a2259aaa6d75468000e5bc36b
the md5 sums should match the ones listed on the download page
the change logs below are limited to ksh and libast
the libast changes involved a lot of meticulous multibyte code
that was hashed out off-list between { gsf roland olga }
there is a "news" link in the left side nav bar that will
be used to detail implemented and proposed ast features
as features mature the news info will migrate to the man pages
if your favorite bug/feature is not in the list below then it
hasn't been addressed yet and we don't know exactly when it will be
changes since 2013-08-29
13-09-13 --- Release ksh93v- ---
13-09-13 +The signal .sh.value variable is now a compound variable with the name
value.q corresponding to kill -q signed-integer and value.Q corresponding
to kill -Q unsigned-large-integer.
13-09-13 A bug in $(...) command substitution that corrupted a trailing
multibyte character in non-UTF-8 locales has been fixed.
13-09-13 Eliminted extranesous output of standard error when ksh is invoked
with the -v (verbose) option.
13-09-10 A bug in finding a function defined inside a type that was defined
in a namespace has been fixed.
13-09-10 A bug in the binding of function local variables inside arithmeitc
expression inside namespaces was fixed.
13-09-10 +A -Q option was added to kill to pass integers as large as pointers.
The -q option now only accepts integers as large as typeset -i.
13-09-09 A bug in command substitution has been fixed.
13-09-09 Qualified print format "%([no]unicode)q" added to prefer \u[...]
over \w[...] and override LC_OPTIONS=unicode.
13-09-04 +\w[hex] locale-specific code point literals have been added.
13-09-04 +The float(f) math function was added.
13-09-04 +The int(f) math function was fixed to return 0 for floating point
numbers larger than the maximum integer.
13-09-04 A bug in which assigning a compound indexed array a value of () did
not preserve the -C attribute has been fixed.
13-09-04 kill -q can now pass numbers as large as typeset -li and
.sh.sig.value is typeset -i rather than a compound variable.
13-09-04 kill -q yields the processor and returns 2 when siqueue fails with
EAGAIN and yield.
13-09-03 A bug in which $((x.xxx)) where x is a floating point variable and
xxx is not one of the known extensions yields a random value has
been fixed. It now is unset which has value 0 when set -u is off.
13-09-03 A bug in overriding discipline functions for types defined in
namespaces has been fixed.
13-09-03 A bug which on some systems caused a core dump for large <<< here
documents has been fixed.
13-09-12 misc/fgetcwd.c: fix stat corruption bug on systems without fdopendir()
13-09-12 path/pathcanon.c: fix bug that added extra / when fgetcwd() returned /
13-09-09 comp/setlocale.c,port/codeset.c: consistent handling of US-ASCII + conformance(0,0) across all os's
13-09-09 string/utf8towc.c,string/wctoutf8.c: add { utf8toutf32() utf8towc() wctoutf8() }
13-09-07 include/ast_std.c,comp/setlocale.c,string/stresc.c: add ast.byte_max for single byte locales
13-09-06 comp/iconv.c: add sfclrerr() to iconv_move() if all input chars not consumed
13-09-06 port/codeset.h,port/codeset.c: internal api for retrieving locale codeset names
13-09-04 string/chresc.c,stresc.c: add \w[hex] support -- thanks Roland
13-09-04 string/utf32stowcs.c,string/wcstoutf32s.c: add -- thanks Roland
13-09-04 sfio/sfsetbuf.c: fix bug where SF_GETR mode was not cleared causing subsequent memory corruption
13-09-04 vmalloc/vmopen.c,vmdcsystem.c,vmstat.c: temporarily set vm->meth.meth=0 to disable vmstat() during init
13-09-04 port/intercept.c: include <ast_standards.h> to ensure fdopendir() prototype if _lib_fdopendir
13-09-04 include/ast_std.h,comp/setlocale.c: add LC_OPTIONS=unicode and AST_LC_unicode
13-08-29 cdt/dtstrhash.c: change sign-bit hitting fnv constants to hex to silence unsigned warnings
_______________________________________________
ast-developers mailing list
ast-developers at lists.research.att.com
http://lists.research.att.com/mailman/listinfo/ast-developers
--
, _ _ ,
{ \/`o;====- Olga Kryzhanovska -====;o`\/ }
.----'-/`-/ olga.kryzhanovska at gmail.com \-`\-'----.
`'-..-| / http://twitter.com/fleyta \ |-..-'`
/\/\ Solaris/BSD//C/C++ programmer /\/\
`--` `--`
ольга крыжановская
2013-09-13 23:08:15 UTC
Permalink
There is a BIG issue with signals in this release. A lot of signal
related tests fail, with memory corruption:

test basic begins at 2013-09-14+00:47:47
corrupted data:region=0xfffffd7ffb590eb0:block=0xfffffd7ffb49c890:bad
byte at=904:
basic.sh[320]: ALRM signal not working
test basic failed at 2013-09-14+00:48:46 with exit code 1 [ 109 tests 1 error ]
test basic(shcomp) begins at 2013-09-14+00:48:46
corrupted data:region=0xfffffd7ffb590eb0:block=0xfffffd7ffb49c890:bad
byte at=904:
shcomp-basic.ksh[320]: ALRM signal not working
test basic(shcomp) failed at 2013-09-14+00:49:46 with exit code 1 [
109 tests 1 error ]

Olga

P.S.: I dislike the idea of renaming .sh.sig.int to .sh.sig.q. Its a
torture for people. Could you not just accept .sh.sig.int and
.sh.sig.objref, or stick with .sh.sig.int and.sh.sig.ptr? The name
"ptr" is used even by perl and python.
Post by Glenn Fowler
the AT&T Software Technology ast alpha 2013-09-13 source release
has been posted to the download site
http://www.research.att.com/sw/download/alpha/
the package names and md5 checksums are
INIT 327861e49e24dd51079c0a5316a4b2fe
ast-open dfb85d1dfb20acb8a1529bdf4b8cb89a
ast-ksh 746a556a2259aaa6d75468000e5bc36b
the md5 sums should match the ones listed on the download page
the change logs below are limited to ksh and libast
the libast changes involved a lot of meticulous multibyte code
that was hashed out off-list between { gsf roland olga }
there is a "news" link in the left side nav bar that will
be used to detail implemented and proposed ast features
as features mature the news info will migrate to the man pages
if your favorite bug/feature is not in the list below then it
hasn't been addressed yet and we don't know exactly when it will be
changes since 2013-08-29
13-09-13 --- Release ksh93v- ---
13-09-13 +The signal .sh.value variable is now a compound variable with the name
value.q corresponding to kill -q signed-integer and value.Q corresponding
to kill -Q unsigned-large-integer.
13-09-13 A bug in $(...) command substitution that corrupted a trailing
multibyte character in non-UTF-8 locales has been fixed.
13-09-13 Eliminted extranesous output of standard error when ksh is invoked
with the -v (verbose) option.
13-09-10 A bug in finding a function defined inside a type that was defined
in a namespace has been fixed.
13-09-10 A bug in the binding of function local variables inside arithmeitc
expression inside namespaces was fixed.
13-09-10 +A -Q option was added to kill to pass integers as large as pointers.
The -q option now only accepts integers as large as typeset -i.
13-09-09 A bug in command substitution has been fixed.
13-09-09 Qualified print format "%([no]unicode)q" added to prefer \u[...]
over \w[...] and override LC_OPTIONS=unicode.
13-09-04 +\w[hex] locale-specific code point literals have been added.
13-09-04 +The float(f) math function was added.
13-09-04 +The int(f) math function was fixed to return 0 for floating point
numbers larger than the maximum integer.
13-09-04 A bug in which assigning a compound indexed array a value of () did
not preserve the -C attribute has been fixed.
13-09-04 kill -q can now pass numbers as large as typeset -li and
.sh.sig.value is typeset -i rather than a compound variable.
13-09-04 kill -q yields the processor and returns 2 when siqueue fails with
EAGAIN and yield.
13-09-03 A bug in which $((x.xxx)) where x is a floating point variable and
xxx is not one of the known extensions yields a random value has
been fixed. It now is unset which has value 0 when set -u is off.
13-09-03 A bug in overriding discipline functions for types defined in
namespaces has been fixed.
13-09-03 A bug which on some systems caused a core dump for large <<< here
documents has been fixed.
13-09-12 misc/fgetcwd.c: fix stat corruption bug on systems without fdopendir()
13-09-12 path/pathcanon.c: fix bug that added extra / when fgetcwd() returned /
13-09-09 comp/setlocale.c,port/codeset.c: consistent handling of US-ASCII + conformance(0,0) across all os's
13-09-09 string/utf8towc.c,string/wctoutf8.c: add { utf8toutf32() utf8towc() wctoutf8() }
13-09-07 include/ast_std.c,comp/setlocale.c,string/stresc.c: add ast.byte_max for single byte locales
13-09-06 comp/iconv.c: add sfclrerr() to iconv_move() if all input chars not consumed
13-09-06 port/codeset.h,port/codeset.c: internal api for retrieving locale codeset names
13-09-04 string/chresc.c,stresc.c: add \w[hex] support -- thanks Roland
13-09-04 string/utf32stowcs.c,string/wcstoutf32s.c: add -- thanks Roland
13-09-04 sfio/sfsetbuf.c: fix bug where SF_GETR mode was not cleared causing subsequent memory corruption
13-09-04 vmalloc/vmopen.c,vmdcsystem.c,vmstat.c: temporarily set vm->meth.meth=0 to disable vmstat() during init
13-09-04 port/intercept.c: include <ast_standards.h> to ensure fdopendir() prototype if _lib_fdopendir
13-09-04 include/ast_std.h,comp/setlocale.c: add LC_OPTIONS=unicode and AST_LC_unicode
13-08-29 cdt/dtstrhash.c: change sign-bit hitting fnv constants to hex to silence unsigned warnings
_______________________________________________
ast-developers mailing list
ast-developers at lists.research.att.com
http://lists.research.att.com/mailman/listinfo/ast-developers
--
, _ _ ,
{ \/`o;====- Olga Kryzhanovska -====;o`\/ }
.----'-/`-/ olga.kryzhanovska at gmail.com \-`\-'----.
`'-..-| / http://twitter.com/fleyta \ |-..-'`
/\/\ Solaris/BSD//C/C++ programmer /\/\
`--` `--`
Glenn Fowler
2013-09-16 06:19:37 UTC
Permalink
if you had built with nmake and had -g in CCFLAGS the src would have
compiled with -D_BLD_DEBUG and each malloc would be tagged with the
originating file and line number -- which would have lead you sh/init.c

if you are skipping nmake or hiding some options in a cc wrapper
(I'm guitly of this too) add -D_BLD_DEBUG when you compile with -g
(but don't use _BLD_DEBUG for production/timing)

the bug is indeed signal related but probably not for the reason you think
its an under-allocation related to the new siginfo code
here's a fix
homework assignment is to explain why adding 8 bytes works and 7 doesn't
(I'm not familiar with the svar_init() code or SH_SIG internal usage

--
static void siginfo_init(Shell_t *shp)
{
struct Svars *sp;
size_t dsize = sizeof(siginfo_t)+SIGNAME_MAX+8;
svar_init(shp,SH_SIG,shtab_siginfo,dsize);
sp = (struct Svars*)SH_SIG->nvfun->next;
sp->dsize = dsize;
}
--
Post by ольга крыжановская
There is a BIG issue with signals in this release. A lot of signal
test basic begins at 2013-09-14+00:47:47
corrupted data:region=0xfffffd7ffb590eb0:block=0xfffffd7ffb49c890:bad
basic.sh[320]: ALRM signal not working
test basic failed at 2013-09-14+00:48:46 with exit code 1 [ 109 tests 1 error ]
test basic(shcomp) begins at 2013-09-14+00:48:46
corrupted data:region=0xfffffd7ffb590eb0:block=0xfffffd7ffb49c890:bad
shcomp-basic.ksh[320]: ALRM signal not working
test basic(shcomp) failed at 2013-09-14+00:49:46 with exit code 1 [
109 tests 1 error ]
Cedric Blancher
2013-09-21 04:06:34 UTC
Permalink
Post by Glenn Fowler
if you had built with nmake and had -g in CCFLAGS the src would have
compiled with -D_BLD_DEBUG and each malloc would be tagged with the
originating file and line number -- which would have lead you sh/init.c
if you are skipping nmake or hiding some options in a cc wrapper
(I'm guitly of this too) add -D_BLD_DEBUG when you compile with -g
(but don't use _BLD_DEBUG for production/timing)
the bug is indeed signal related but probably not for the reason you think
its an under-allocation related to the new siginfo code
here's a fix
homework assignment is to explain why adding 8 bytes works and 7 doesn't
(I'm not familiar with the svar_init() code or SH_SIG internal usage
--
static void siginfo_init(Shell_t *shp)
{
struct Svars *sp;
size_t dsize = sizeof(siginfo_t)+SIGNAME_MAX+8;
svar_init(shp,SH_SIG,shtab_siginfo,dsize);
sp = (struct Svars*)SH_SIG->nvfun->next;
sp->dsize = dsize;
}
--
Glenn, could you send a diff to the list, please?

Ced
--
Cedric Blancher <cedric.blancher at gmail.com>
Institute Pasteur
Glenn Fowler
2013-09-24 21:14:11 UTC
Permalink
the AT&T Software Technology ast alpha 2013-09-24 source release
has been posted to the download site
http://www.research.att.com/sw/download/alpha/
the package names and md5 checksums are
INIT 3306a1ea5b0ccad851f984850235275a
ast-base 905d3125d8804cc53562d5f7ac2d834f
ast-open cc1f1bb8d20861eb1987a458c98548d0
ast-ksh 970460ec2c257b22bc4ad22e18046ebc
the md5 sums should match the ones listed on the download page

libcmd realpath, readlink, sync finally added
there is a clash between the ast and gnu interpretation of --logical
ast is sticking to its guns here because it matches the
--logical/--physical/--metaphysical options of other ast utilities
there's a bunch of regression tests but I'm sure a few dark corners were missed

the first phase of properly using mbstate_t mb/wc char in this release
second phase (next alpha) is to do the same for ksh93
there are probably some bugs lurking
hoping for feedback from our shift-state codeset locale friends

changes since 2013-09-13

:::::::: builtin ::::::::

13-09-22 readlink.rt,realpath.rt: add tests
13-09-19 head.tst,tail.tst: add incomplete line tests

:::::::: bzip ::::::::

13-09-19 codex.tst: update from obsolete "<meth" and ">meth" to --decode=meth and --encode=meth

:::::::: compress ::::::::

13-09-19 codex.tst: update from obsolete "<meth" and ">meth" to --decode=meth and --encode=meth

:::::::: gzip ::::::::

13-09-19 codex.tst: update from obsolete "<meth" and ">meth" to --decode=meth and --encode=meth

:::::::: sum ::::::::

13-09-19 codex.tst: update from obsolete "<meth" and ">meth" to --decode=meth and --encode=meth

:::::::: ksh93 ::::::::

13-09-23 --- Release ksh93v- ---
13-09-23 A bug in which a compound assignment to a type, [(T_t x=(i=2)] where
type T_t is defined in FPATH would fail with a syntx error has been
fixed
13-09-16 A bug in which a 0 value for variable in arithmetic expression inside
a function fails when set -u has been set has been fixed.
13-09-16 A bug which could cause a core dump when generating the man page
for user defined types containing compound variables has been fixed.
13-09-16 A bug in the handling of array instances of type variables in an
arithmetic expression has been fixed.
13-09-13 +The signal .sh.value variable is now a compound variable with the name
value.q for sending a signal with -q and value.Q for sending a value
with -Q.
13-09-13 A bug in $(...) command substitution that corrupted a trailing
multibyte character in non-UTF-8 locales has been fixed.
13-09-13 Eliminted extranesous output of standard error when ksh is invoked
with the -v (verbose) option.

:::::::: libast ::::::::

13-09-18 include/ast_std.h,ast.h: api>=20130913 add Mbstate_t and { mb*() wc*() } macros with Mbstate_t* last arg
13-09-18 path/pathcanon.c: handle more flags for /dev/file/flags at ***@path and do flag name lookup from table
13-09-16 include/ast.h: add PATH_EXCEPT_LAST for readlink(1) --canonicalize
13-09-16 path/pathcanon.c: handle PATH_EXCEPT_LAST

:::::::: libcmd ::::::::

13-09-22 realpath.c,readlink.c: add based on ast::pathdev()
13-09-22 sync.c: add roland's changes + --sfsync
13-09-19 xarrrrghs.c: halfscore d's be walkin' t'plank t'day: fix run() to work for context==0
13-09-19 head.c,tail.c: count incomplete lines too
13-09-19 do not check for command when /bin/echo implied
13-09-13 cat.c,cmdext.h,cmdlist.h,cut.c,join.c,od.c,paste.c,rev.c,tr.c,uniq.c,wc.c,wc.h,wclib.c: Mbstate_t update

:::::::: sort ::::::::

13-09-19 main.c: convert to mb*(Mbstate_t) api

:::::::: std ::::::::

13-09-19 ls.c: convert to mb*(Mbstate_t) api
Simon Toedt
2013-09-25 11:47:10 UTC
Permalink
Post by Glenn Fowler
the AT&T Software Technology ast alpha 2013-09-24 source release
has been posted to the download site
http://www.research.att.com/sw/download/alpha/
the package names and md5 checksums are
INIT 3306a1ea5b0ccad851f984850235275a
ast-base 905d3125d8804cc53562d5f7ac2d834f
ast-open cc1f1bb8d20861eb1987a458c98548d0
ast-ksh 970460ec2c257b22bc4ad22e18046ebc
the md5 sums should match the ones listed on the download page
libcmd realpath, readlink, sync finally added
there is a clash between the ast and gnu interpretation of --logical
ast is sticking to its guns here because it matches the
--logical/--physical/--metaphysical options of other ast utilities
there's a bunch of regression tests but I'm sure a few dark corners were missed
the first phase of properly using mbstate_t mb/wc char in this release
second phase (next alpha) is to do the same for ksh93
there are probably some bugs lurking
hoping for feedback from our shift-state codeset locale friends
changes since 2013-09-13
Has anyone been able to build the sources? Its failing for me with
ASTAPI symbol not defined on Linux and BSD.

Simon
Glenn Fowler
2013-09-25 13:17:51 UTC
Permalink
Post by Simon Toedt
Post by Glenn Fowler
the AT&T Software Technology ast alpha 2013-09-24 source release
has been posted to the download site
http://www.research.att.com/sw/download/alpha/
the package names and md5 checksums are
INIT 3306a1ea5b0ccad851f984850235275a
ast-base 905d3125d8804cc53562d5f7ac2d834f
ast-open cc1f1bb8d20861eb1987a458c98548d0
ast-ksh 970460ec2c257b22bc4ad22e18046ebc
the md5 sums should match the ones listed on the download page
libcmd realpath, readlink, sync finally added
there is a clash between the ast and gnu interpretation of --logical
ast is sticking to its guns here because it matches the
--logical/--physical/--metaphysical options of other ast utilities
there's a bunch of regression tests but I'm sure a few dark corners were missed
the first phase of properly using mbstate_t mb/wc char in this release
second phase (next alpha) is to do the same for ksh93
there are probably some bugs lurking
hoping for feedback from our shift-state codeset locale friends
changes since 2013-09-13
Has anyone been able to build the sources? Its failing for me with
ASTAPI symbol not defined on Linux and BSD.
thanks for reporting this
looking at a fix now
Glenn Fowler
2013-09-26 05:19:29 UTC
Permalink
the AT&T Software Technology ast alpha 2013-09-26 source release
has been posted to the download site
http://www.research.att.com/sw/download/alpha/
the package names and md5 checksums are
INIT 95da0c907a6ed25a601c373e16a1862e
ast-base 0aed6c2cd6bccf0ec23120179a1dfc38
ast-open 529b4c33049de4100bc6f5ecaafa7722
ast-ksh d0762a194fe508641fa0706294236d3f
the md5 sums should match the ones listed on the download page

note that these are binary md5sums :)

this is mainly to fix the 2 reported build problems in 2013-09-24

changes since 2013-09-24

:::::::: INIT ::::::::

13-09-25 iffe.sh: add _${API}_API_IMPLEMENT guards for implementation code

:::::::: builtin ::::::::

13-09-25 cksum.tst: adjust tests for gnu { md5 sha } text/binary indicator

:::::::: libast ::::::::

13-09-25 include/ast_std.h,comp/setlocale.c: AST_LC_unicode => AST_LC_unicodeliterals, LC_OPTIONS=...unicodeliterals...
13-09-25 include/ast_std.h,comp/setlocale.c: AST_LC_native LC_OPTIONS=...native... to bypass ast unicode iswalpha/wcwidth tables
13-09-25 comp/eaccess.c: resolve <ast_map.h> vs <ast_intercept.h> conflict -- _ast_eaccess wins

:::::::: libcmd ::::::::

13-09-25 shasum.c: add sha*sum variants
13-09-25 cksum.c: handle SUM_INDICATOR per-method { '*'(binary) ' '(text) } read mode indicator
13-09-25 cksum.c: fix --header to include "text" when --text specified
13-09-25 cksum.c: { -T, --text && -t, --total } => { -t, --text && -T, --total } for gnu compatibility

:::::::: libsum ::::::::

13-09-25 sum.h,sumlib.c: add Sum_t.flags and SUM_INDICATOR for gnu { '*'(binary) ' '(text) } read mode indicator

:::::::: libvcodex ::::::::

13-09-25 Vclz4/vclz4.c: add _PACKAGE_ast headers
!
Roland Mainz
2013-09-26 21:07:14 UTC
Permalink
Post by Glenn Fowler
the AT&T Software Technology ast alpha 2013-09-26 source release
has been posted to the download site
http://www.research.att.com/sw/download/alpha/
the package names and md5 checksums are
INIT 95da0c907a6ed25a601c373e16a1862e
ast-base 0aed6c2cd6bccf0ec23120179a1dfc38
ast-open 529b4c33049de4100bc6f5ecaafa7722
ast-ksh d0762a194fe508641fa0706294236d3f
the md5 sums should match the ones listed on the download page
note that these are binary md5sums :)
[snip]

Erm... there are a couple of problems:
1. ast-ksh doesn't build on Solaris 11/AMD64/64bit:
-- snip --
+ cc -KPIC -D_TS_ERRNO -D_REENTRANT -KPIC -g -xs -xspace -Xa
-xstrconst -z combreloc -xildoff -xcsi '-errtags=yes'
'-DSHOPT_CMDLIB_BLTIN=0' '-DSH_CMDLIB_DIR="/usr/ast/bin"' '-DSHO
PT_CMDLIB_HDR="/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/tmp_solaris_builtin_header.h"'
-DSHOPT_SYSRC '-D_map_libc=1' '-D_lib_socket=1' -lsocket -lnsl -I.
-I/hom
e/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum
-I/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/arch/sol11.i386-64/include/ast
-D_PACKAGE_ast
-c /home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c
"/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c",
line 198: undefined symbol: sha384_scale
"/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c",
line 198: non-constant initializer: op "NAME"
"/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c",
line 267: warning: statement not reached (E_STATEMENT_NOT_REACHED)
cc: acomp failed for
/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c
mamake [lib/libsum]: *** exit code 2 making sumlib.o
-- snip --


2. The patch for getconf XATTR_* was forgotten... again... see
http://lists.research.att.com/pipermail/ast-developers/2013q3/003483.html


3. The "wchar.sh" patch from
http://lists.research.att.com/pipermail/ast-developers/2013q3/003484.html
was forgotten, too...

----

Bye,
Roland
--
__ . . __
(o.\ \/ /.o) roland.mainz at nrubsig.org
\__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer
/O /==\ O\ TEL +49 641 3992797
(;O/ \/ \O;)
Glenn Fowler
2013-09-26 21:30:27 UTC
Permalink
patches not forgotten
just trying to get to a stable state
as you can see we're not there
Post by Roland Mainz
Post by Glenn Fowler
the AT&T Software Technology ast alpha 2013-09-26 source release
has been posted to the download site
http://www.research.att.com/sw/download/alpha/
the package names and md5 checksums are
INIT 95da0c907a6ed25a601c373e16a1862e
ast-base 0aed6c2cd6bccf0ec23120179a1dfc38
ast-open 529b4c33049de4100bc6f5ecaafa7722
ast-ksh d0762a194fe508641fa0706294236d3f
the md5 sums should match the ones listed on the download page
note that these are binary md5sums :)
[snip]
-- snip --
+ cc -KPIC -D_TS_ERRNO -D_REENTRANT -KPIC -g -xs -xspace -Xa
-xstrconst -z combreloc -xildoff -xcsi '-errtags=yes'
'-DSHOPT_CMDLIB_BLTIN=0' '-DSH_CMDLIB_DIR="/usr/ast/bin"' '-DSHO
PT_CMDLIB_HDR="/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/tmp_solaris_builtin_header.h"'
-DSHOPT_SYSRC '-D_map_libc=1' '-D_lib_socket=1' -lsocket -lnsl -I.
-I/hom
e/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum
-I/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/arch/sol11.i386-64/include/ast
-D_PACKAGE_ast
-c /home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c
"/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c",
line 198: undefined symbol: sha384_scale
"/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c",
line 198: non-constant initializer: op "NAME"
"/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c",
line 267: warning: statement not reached (E_STATEMENT_NOT_REACHED)
cc: acomp failed for
/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c
mamake [lib/libsum]: *** exit code 2 making sumlib.o
-- snip --
sum-sha2.c has
#define sha384_scale 0
sumlib.c has
#include "sum-sha2.c"
#undef METHOD
#define METHOD(x) x##_match,x##_description,x##_options,x##_open,x##_init,x##_block,x##_data,x##_print,x##_done,x##_scale,x##_flags
METHOD(sha384),

you'll have to track down why for that particular cc and options
x##_scale => sha384_scale doesn't expand to 0
give that a few other METHOD() expansions in that file also had 0 scale and went through fine

also, but unrelated, '-D_lib_socket=1' -lsocket -lnsl in cc options is a concern
those should have been handled by the build
Roland Mainz
2013-09-26 21:43:19 UTC
Permalink
Post by Glenn Fowler
Post by Roland Mainz
Post by Glenn Fowler
the AT&T Software Technology ast alpha 2013-09-26 source release
has been posted to the download site
http://www.research.att.com/sw/download/alpha/
the package names and md5 checksums are
INIT 95da0c907a6ed25a601c373e16a1862e
ast-base 0aed6c2cd6bccf0ec23120179a1dfc38
ast-open 529b4c33049de4100bc6f5ecaafa7722
ast-ksh d0762a194fe508641fa0706294236d3f
the md5 sums should match the ones listed on the download page
note that these are binary md5sums :)
[snip]
-- snip --
+ cc -KPIC -D_TS_ERRNO -D_REENTRANT -KPIC -g -xs -xspace -Xa
-xstrconst -z combreloc -xildoff -xcsi '-errtags=yes'
'-DSHOPT_CMDLIB_BLTIN=0' '-DSH_CMDLIB_DIR="/usr/ast/bin"' '-DSHO
PT_CMDLIB_HDR="/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/tmp_solaris_builtin_header.h"'
-DSHOPT_SYSRC '-D_map_libc=1' '-D_lib_socket=1' -lsocket -lnsl -I.
-I/hom
e/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum
-I/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/arch/sol11.i386-64/include/ast
-D_PACKAGE_ast
-c /home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c
"/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c",
line 198: undefined symbol: sha384_scale
"/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c",
line 198: non-constant initializer: op "NAME"
"/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c",
line 267: warning: statement not reached (E_STATEMENT_NOT_REACHED)
cc: acomp failed for
/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c
mamake [lib/libsum]: *** exit code 2 making sumlib.o
-- snip --
[snip]
Post by Glenn Fowler
patches not forgotten
just trying to get to a stable state
Oh please... I hereby swear to eat 2 inches of a broomstick if the
getconf table changes cause any trouble... :-)
Post by Glenn Fowler
as you can see we're not there
Grumpf... I'd prefer the "stuff all patches in"-approach as it gives
me the opportunity to fix more stuf... :-)
Post by Glenn Fowler
sum-sha2.c has
#define sha384_scale 0
sumlib.c has
#include "sum-sha2.c"
#undef METHOD
#define METHOD(x) x##_match,x##_description,x##_options,x##_open,x##_init,x##_block,x##_data,x##_print,x##_done,x##_scale,x##_flags
METHOD(sha384),
you'll have to track down why for that particular cc and options
x##_scale => sha384_scale doesn't expand to 0
Erm... because it uses the libmd codepath ?

Remember libmd is this monstrosity, originally conceived as all-in-one
digest library and as interface to the hardware accerlators:
-- snip --
Interface Libraries libmd(3LIB)

NAME
libmd - Message Digest library

SYNOPSIS
cc [ _f_l_a_g... ] _f_i_l_e... -lmd [ _l_i_b_r_a_r_y... ]
#include <md4.h>
#include <md5.h>
#include <sha1.h>
#include <sha2.h>


DESCRIPTION
Functions in this library provide hashing routines for MD4
(RFC1320), MD5 (RFC1321), SHA1 (RFC3174), SHA256 (FIPS 180-
2), SHA384 (FIPS 180-2), SHA512 (FIPS 180-2).

INTERFACES
The shared object libmd.so.1 provides the public interfaces
defined below. See intro(3) for additional information on
shared object interfaces.



MD4Final MD4Init
MD4Update md5_calc
MD5Final MD5Init
MD5Update SHA1Final
SHA1Init SHA1Update
SHA2Final SHA2Init
SHA2Update


SECURITY
The MD4 and MD5 algorithms are currently considered weak for
cryptographic use. The algorithms should be used only for
compatibility with legacy systems or protocols.


The SHA1 algorithm is also believed to have some weaknesses.
Migration to one of the SHA2 algorithms-including SHA256,
SHA386 or SHA512-is highly recommended when compatibility
with data formats and on wire protocols is permitted.

FILES
/lib/libmd.so.1 shared object


/lib/64/libmd.so.1 64-bit shared object


ATTRIBUTES


See attributes(5) for descriptions of the following attri-
butes:

____________________________________________________________
| ATTRIBUTE TYPE | ATTRIBUTE VALUE |
|______________________________|______________________________|
| Interface Stability | Committed |
|______________________________|______________________________|
| MT-Level | MT-Safe |
|______________________________|______________________________|

SunOS 5.11 Last change: 16 Jun 2008 2
-- snip --
Post by Glenn Fowler
give that a few other METHOD() expansions in that file also had 0 scale and went through fine
also, but unrelated, '-D_lib_socket=1' -lsocket -lnsl in cc options is a concern
those should have been handled by the build
I saw that the patch went in... and it seems to work... thanks... :-)

----

Bye,
Roland
--
__ . . __
(o.\ \/ /.o) roland.mainz at nrubsig.org
\__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer
/O /==\ O\ TEL +49 641 3992797
(;O/ \/ \O;)
Roland Mainz
2013-09-26 21:52:35 UTC
Permalink
Post by Roland Mainz
Post by Glenn Fowler
Post by Roland Mainz
Post by Glenn Fowler
the AT&T Software Technology ast alpha 2013-09-26 source release
has been posted to the download site
http://www.research.att.com/sw/download/alpha/
the package names and md5 checksums are
INIT 95da0c907a6ed25a601c373e16a1862e
ast-base 0aed6c2cd6bccf0ec23120179a1dfc38
ast-open 529b4c33049de4100bc6f5ecaafa7722
ast-ksh d0762a194fe508641fa0706294236d3f
the md5 sums should match the ones listed on the download page
note that these are binary md5sums :)
[snip]
-- snip --
+ cc -KPIC -D_TS_ERRNO -D_REENTRANT -KPIC -g -xs -xspace -Xa
-xstrconst -z combreloc -xildoff -xcsi '-errtags=yes'
'-DSHOPT_CMDLIB_BLTIN=0' '-DSH_CMDLIB_DIR="/usr/ast/bin"' '-DSHO
PT_CMDLIB_HDR="/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/tmp_solaris_builtin_header.h"'
-DSHOPT_SYSRC '-D_map_libc=1' '-D_lib_socket=1' -lsocket -lnsl -I.
-I/hom
e/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum
-I/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/arch/sol11.i386-64/include/ast
-D_PACKAGE_ast
-c /home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c
"/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c",
line 198: undefined symbol: sha384_scale
"/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c",
line 198: non-constant initializer: op "NAME"
"/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c",
line 267: warning: statement not reached (E_STATEMENT_NOT_REACHED)
cc: acomp failed for
/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c
mamake [lib/libsum]: *** exit code 2 making sumlib.o
-- snip --
[snip]
Post by Glenn Fowler
patches not forgotten
just trying to get to a stable state
Oh please... I hereby swear to eat 2 inches of a broomstick if the
getconf table changes cause any trouble... :-)
Post by Glenn Fowler
as you can see we're not there
Grumpf... I'd prefer the "stuff all patches in"-approach as it gives
me the opportunity to fix more stuf... :-)
Post by Glenn Fowler
sum-sha2.c has
#define sha384_scale 0
sumlib.c has
#include "sum-sha2.c"
#undef METHOD
#define METHOD(x) x##_match,x##_description,x##_options,x##_open,x##_init,x##_block,x##_data,x##_print,x##_done,x##_scale,x##_flags
METHOD(sha384),
you'll have to track down why for that particular cc and options
x##_scale => sha384_scale doesn't expand to 0
Erm... because it uses the libmd codepath ?
This patch should fix the issue:
-- snip --
diff -r -u original/src/lib/libsum/sum-lmd.c
build_i386_64bit_debug/src/lib/libsum/sum-lmd.c
--- src/lib/libsum/sum-lmd.c 2013-09-25 16:48:46.000000000 +0200
+++ src/lib/libsum/sum-lmd.c 2013-09-26 23:48:14.224181544 +0200
@@ -266,6 +266,7 @@
#define sha384_description "FIPS 180-2 SHA384 secure hash algorithm.
The block count is not printed."
#define sha384_options "[+(version)?sha384 (solaris -lmd) 2005-07-26]"
#define sha384_match "sha384|sha-384|SHA384|SHA-384"
+#define sha384_scale 0
#define sha384_flags SUM_INDICATOR
#define sha384_init lmd_init
#define sha384_block lmd_block
-- snip --

Still missing are the single command wrappers to get the digest part
of the utilities in sync with busybox and GNU coreutils, e.g. we need:
-- snip --
sha1sum(1)
sha2sum(1)
sha128sum(1)
sha224sum(1)
sha256sum(1)
sha384sum(1)
sha512sum(1)
-- snip --

sha224sum(1) is likely the largest effort...

----

Bye,
Roland
--
__ . . __
(o.\ \/ /.o) roland.mainz at nrubsig.org
\__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer
/O /==\ O\ TEL +49 641 3992797
(;O/ \/ \O;)
Glenn Fowler
2013-09-26 22:12:31 UTC
Permalink
Post by Roland Mainz
Post by Roland Mainz
Post by Glenn Fowler
Post by Roland Mainz
Post by Glenn Fowler
the AT&T Software Technology ast alpha 2013-09-26 source release
has been posted to the download site
http://www.research.att.com/sw/download/alpha/
the package names and md5 checksums are
INIT 95da0c907a6ed25a601c373e16a1862e
ast-base 0aed6c2cd6bccf0ec23120179a1dfc38
ast-open 529b4c33049de4100bc6f5ecaafa7722
ast-ksh d0762a194fe508641fa0706294236d3f
the md5 sums should match the ones listed on the download page
note that these are binary md5sums :)
[snip]
-- snip --
+ cc -KPIC -D_TS_ERRNO -D_REENTRANT -KPIC -g -xs -xspace -Xa
-xstrconst -z combreloc -xildoff -xcsi '-errtags=yes'
'-DSHOPT_CMDLIB_BLTIN=0' '-DSH_CMDLIB_DIR="/usr/ast/bin"' '-DSHO
PT_CMDLIB_HDR="/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/tmp_solaris_builtin_header.h"'
-DSHOPT_SYSRC '-D_map_libc=1' '-D_lib_socket=1' -lsocket -lnsl -I.
-I/hom
e/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum
-I/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/arch/sol11.i386-64/include/ast
-D_PACKAGE_ast
-c /home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c
"/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c",
line 198: undefined symbol: sha384_scale
"/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c",
line 198: non-constant initializer: op "NAME"
"/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c",
line 267: warning: statement not reached (E_STATEMENT_NOT_REACHED)
cc: acomp failed for
/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c
mamake [lib/libsum]: *** exit code 2 making sumlib.o
-- snip --
[snip]
Post by Glenn Fowler
patches not forgotten
just trying to get to a stable state
Oh please... I hereby swear to eat 2 inches of a broomstick if the
getconf table changes cause any trouble... :-)
Post by Glenn Fowler
as you can see we're not there
Grumpf... I'd prefer the "stuff all patches in"-approach as it gives
me the opportunity to fix more stuf... :-)
Post by Glenn Fowler
sum-sha2.c has
#define sha384_scale 0
sumlib.c has
#include "sum-sha2.c"
#undef METHOD
#define METHOD(x) x##_match,x##_description,x##_options,x##_open,x##_init,x##_block,x##_data,x##_print,x##_done,x##_scale,x##_flags
METHOD(sha384),
you'll have to track down why for that particular cc and options
x##_scale => sha384_scale doesn't expand to 0
Erm... because it uses the libmd codepath ?
-- snip --
diff -r -u original/src/lib/libsum/sum-lmd.c
build_i386_64bit_debug/src/lib/libsum/sum-lmd.c
--- src/lib/libsum/sum-lmd.c 2013-09-25 16:48:46.000000000 +0200
+++ src/lib/libsum/sum-lmd.c 2013-09-26 23:48:14.224181544 +0200
@@ -266,6 +266,7 @@
#define sha384_description "FIPS 180-2 SHA384 secure hash algorithm.
The block count is not printed."
#define sha384_options "[+(version)?sha384 (solaris -lmd) 2005-07-26]"
#define sha384_match "sha384|sha-384|SHA384|SHA-384"
+#define sha384_scale 0
#define sha384_flags SUM_INDICATOR
#define sha384_init lmd_init
#define sha384_block lmd_block
-- snip --
that was my edit error during the SUM_INDICATOR edit -- thx
Post by Roland Mainz
Still missing are the single command wrappers to get the digest part
-- snip --
sha1sum(1)
sha2sum(1)
sha128sum(1)
sha224sum(1)
sha256sum(1)
sha384sum(1)
sha512sum(1)
-- snip --
src/lib/libcmd/shasum.c was added in the last alpha with e.g.

int
b_sha512sum(int argc, register char** argv, Shbltin_t* context)
{
return b_cksum(argc, argv, context);
}

(except b_sha2sum() was omitted by error)
are you asking for something else
Roland Mainz
2013-09-26 22:26:30 UTC
Permalink
Post by Glenn Fowler
Post by Roland Mainz
Post by Roland Mainz
Post by Glenn Fowler
Post by Roland Mainz
Post by Glenn Fowler
the AT&T Software Technology ast alpha 2013-09-26 source release
has been posted to the download site
http://www.research.att.com/sw/download/alpha/
the package names and md5 checksums are
INIT 95da0c907a6ed25a601c373e16a1862e
ast-base 0aed6c2cd6bccf0ec23120179a1dfc38
ast-open 529b4c33049de4100bc6f5ecaafa7722
ast-ksh d0762a194fe508641fa0706294236d3f
the md5 sums should match the ones listed on the download page
note that these are binary md5sums :)
[snip]
-- snip --
+ cc -KPIC -D_TS_ERRNO -D_REENTRANT -KPIC -g -xs -xspace -Xa
-xstrconst -z combreloc -xildoff -xcsi '-errtags=yes'
'-DSHOPT_CMDLIB_BLTIN=0' '-DSH_CMDLIB_DIR="/usr/ast/bin"' '-DSHO
PT_CMDLIB_HDR="/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/tmp_solaris_builtin_header.h"'
-DSHOPT_SYSRC '-D_map_libc=1' '-D_lib_socket=1' -lsocket -lnsl -I.
-I/hom
e/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum
-I/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/arch/sol11.i386-64/include/ast
-D_PACKAGE_ast
-c /home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c
"/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c",
line 198: undefined symbol: sha384_scale
"/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c",
line 198: non-constant initializer: op "NAME"
"/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c",
line 267: warning: statement not reached (E_STATEMENT_NOT_REACHED)
cc: acomp failed for
/home/test001/ksh93/ast_ksh_20130926/build_i386_64bit_debug/src/lib/libsum/sumlib.c
mamake [lib/libsum]: *** exit code 2 making sumlib.o
-- snip --
[snip]
Post by Glenn Fowler
patches not forgotten
just trying to get to a stable state
Oh please... I hereby swear to eat 2 inches of a broomstick if the
getconf table changes cause any trouble... :-)
Post by Glenn Fowler
as you can see we're not there
Grumpf... I'd prefer the "stuff all patches in"-approach as it gives
me the opportunity to fix more stuf... :-)
Post by Glenn Fowler
sum-sha2.c has
#define sha384_scale 0
sumlib.c has
#include "sum-sha2.c"
#undef METHOD
#define METHOD(x) x##_match,x##_description,x##_options,x##_open,x##_init,x##_block,x##_data,x##_print,x##_done,x##_scale,x##_flags
METHOD(sha384),
you'll have to track down why for that particular cc and options
x##_scale => sha384_scale doesn't expand to 0
Erm... because it uses the libmd codepath ?
-- snip --
diff -r -u original/src/lib/libsum/sum-lmd.c
build_i386_64bit_debug/src/lib/libsum/sum-lmd.c
--- src/lib/libsum/sum-lmd.c 2013-09-25 16:48:46.000000000 +0200
+++ src/lib/libsum/sum-lmd.c 2013-09-26 23:48:14.224181544 +0200
@@ -266,6 +266,7 @@
#define sha384_description "FIPS 180-2 SHA384 secure hash algorithm.
The block count is not printed."
#define sha384_options "[+(version)?sha384 (solaris -lmd) 2005-07-26]"
#define sha384_match "sha384|sha-384|SHA384|SHA-384"
+#define sha384_scale 0
#define sha384_flags SUM_INDICATOR
#define sha384_init lmd_init
#define sha384_block lmd_block
-- snip --
that was my edit error during the SUM_INDICATOR edit -- thx
Post by Roland Mainz
Still missing are the single command wrappers to get the digest part
-- snip --
sha1sum(1)
sha2sum(1)
sha128sum(1)
sha224sum(1)
sha256sum(1)
sha384sum(1)
sha512sum(1)
-- snip --
src/lib/libcmd/shasum.c was added in the last alpha with e.g.
int
b_sha512sum(int argc, register char** argv, Shbltin_t* context)
{
return b_cksum(argc, argv, context);
}
(except b_sha2sum() was omitted by error)
are you asking for something else
Erm... I'm herely officially declared "dumb" (in this case) ... I
forgot that our list of path bindings on Solaris is static to match
exactly what is allowed by Solaris PSARC rules (e.g. PSARC/2006/550
and follow-ups) ... which includes all stuff in /usr/ast/bin/&&co. ...
I just have to update that list. /usr/include/ast/cmdext.h contains
all matching |b_*()| functions except |b_sha2()| ...

... the only missing bit is sha224sum(1) (AFAIK originally conceived
as method to have maximum security allowed by US export laws; sha256
was too strong so it seems they went right for the next lower possible
digest length... ;-/ ) for which libsum&&libcmd do not have any code
right now...

----

Bye,
Roland
--
__ . . __
(o.\ \/ /.o) roland.mainz at nrubsig.org
\__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer
/O /==\ O\ TEL +49 641 3992797
(;O/ \/ \O;)
Lionel Cons
2013-09-27 13:29:39 UTC
Permalink
Post by Glenn Fowler
patches not forgotten
just trying to get to a stable state
as you can see we're not there
Are the getconf(1) changes that risky?

Lionel
Glenn Fowler
2013-09-27 17:07:35 UTC
Permalink
Post by Lionel Cons
Post by Glenn Fowler
patches not forgotten
just trying to get to a stable state
as you can see we're not there
Are the getconf(1) changes that risky?
sometimes stability has little to do with risk

I mentioned a week or so ago that research in 1 building was moving
to 3 buildings (mutually ~40 miles apart)

in some cases people and their machines may be in different buildings

the ~40 machine ast build farm is part of the move

it will be this way for a week or so more
Simon Toedt
2013-09-27 08:29:47 UTC
Permalink
Anyone else seeing this test failure?

test basic begins at 2013-09-27+10:10:46
cat: /dev/fd/4: No such file or directory
basic.sh[372]: process substitution not working outside for or
while loop
test basic failed at 2013-09-27+10:11:32 with exit code 1 [ 109 tests 1 error ]

Simon
Post by Glenn Fowler
the AT&T Software Technology ast alpha 2013-09-26 source release
has been posted to the download site
http://www.research.att.com/sw/download/alpha/
the package names and md5 checksums are
INIT 95da0c907a6ed25a601c373e16a1862e
ast-base 0aed6c2cd6bccf0ec23120179a1dfc38
ast-open 529b4c33049de4100bc6f5ecaafa7722
ast-ksh d0762a194fe508641fa0706294236d3f
the md5 sums should match the ones listed on the download page
note that these are binary md5sums :)
this is mainly to fix the 2 reported build problems in 2013-09-24
changes since 2013-09-24
13-09-25 iffe.sh: add _${API}_API_IMPLEMENT guards for implementation code
13-09-25 cksum.tst: adjust tests for gnu { md5 sha } text/binary indicator
13-09-25 include/ast_std.h,comp/setlocale.c: AST_LC_unicode => AST_LC_unicodeliterals, LC_OPTIONS=...unicodeliterals...
13-09-25 include/ast_std.h,comp/setlocale.c: AST_LC_native LC_OPTIONS=...native... to bypass ast unicode iswalpha/wcwidth tables
13-09-25 comp/eaccess.c: resolve <ast_map.h> vs <ast_intercept.h> conflict -- _ast_eaccess wins
13-09-25 shasum.c: add sha*sum variants
13-09-25 cksum.c: handle SUM_INDICATOR per-method { '*'(binary) ' '(text) } read mode indicator
13-09-25 cksum.c: fix --header to include "text" when --text specified
13-09-25 cksum.c: { -T, --text && -t, --total } => { -t, --text && -T, --total } for gnu compatibility
13-09-25 sum.h,sumlib.c: add Sum_t.flags and SUM_INDICATOR for gnu { '*'(binary) ' '(text) } read mode indicator
13-09-25 Vclz4/vclz4.c: add _PACKAGE_ast headers
!
_______________________________________________
ast-developers mailing list
ast-developers at lists.research.att.com
http://lists.research.att.com/mailman/listinfo/ast-developers
Simon Toedt
2013-09-27 08:52:15 UTC
Permalink
Two more tests fail on my Fedora Linux, 64bit machine, native gcc:

1. test signal begins at 2013-09-27+10:34:20
./src/cmd/ksh93/tests/shtests[333]: eval: line 1: 3004: Terminated
test signal failed at 2013-09-27+10:35:47 with exit code 271 [ 49
tests 271 errors ]

2. test types begins at 2013-09-27+10:37:57
types.sh[895]: dynamic loading of type with assignment fails
test types failed at 2013-09-27+10:37:58 with exit code 1 [ 117 tests 1 error ]

Simon
Post by Simon Toedt
Anyone else seeing this test failure?
test basic begins at 2013-09-27+10:10:46
cat: /dev/fd/4: No such file or directory
basic.sh[372]: process substitution not working outside for or
while loop
test basic failed at 2013-09-27+10:11:32 with exit code 1 [ 109 tests 1 error ]
Simon
Post by Glenn Fowler
the AT&T Software Technology ast alpha 2013-09-26 source release
has been posted to the download site
http://www.research.att.com/sw/download/alpha/
the package names and md5 checksums are
INIT 95da0c907a6ed25a601c373e16a1862e
ast-base 0aed6c2cd6bccf0ec23120179a1dfc38
ast-open 529b4c33049de4100bc6f5ecaafa7722
ast-ksh d0762a194fe508641fa0706294236d3f
the md5 sums should match the ones listed on the download page
note that these are binary md5sums :)
this is mainly to fix the 2 reported build problems in 2013-09-24
changes since 2013-09-24
13-09-25 iffe.sh: add _${API}_API_IMPLEMENT guards for implementation code
13-09-25 cksum.tst: adjust tests for gnu { md5 sha } text/binary indicator
13-09-25 include/ast_std.h,comp/setlocale.c: AST_LC_unicode => AST_LC_unicodeliterals, LC_OPTIONS=...unicodeliterals...
13-09-25 include/ast_std.h,comp/setlocale.c: AST_LC_native LC_OPTIONS=...native... to bypass ast unicode iswalpha/wcwidth tables
13-09-25 comp/eaccess.c: resolve <ast_map.h> vs <ast_intercept.h> conflict -- _ast_eaccess wins
13-09-25 shasum.c: add sha*sum variants
13-09-25 cksum.c: handle SUM_INDICATOR per-method { '*'(binary) ' '(text) } read mode indicator
13-09-25 cksum.c: fix --header to include "text" when --text specified
13-09-25 cksum.c: { -T, --text && -t, --total } => { -t, --text && -T, --total } for gnu compatibility
13-09-25 sum.h,sumlib.c: add Sum_t.flags and SUM_INDICATOR for gnu { '*'(binary) ' '(text) } read mode indicator
13-09-25 Vclz4/vclz4.c: add _PACKAGE_ast headers
!
_______________________________________________
ast-developers mailing list
ast-developers at lists.research.att.com
http://lists.research.att.com/mailman/listinfo/ast-developers
Lionel Cons
2013-09-27 13:28:46 UTC
Permalink
Post by Glenn Fowler
the AT&T Software Technology ast alpha 2013-09-26 source release
has been posted to the download site
http://www.research.att.com/sw/download/alpha/
the package names and md5 checksums are
INIT 95da0c907a6ed25a601c373e16a1862e
ast-base 0aed6c2cd6bccf0ec23120179a1dfc38
ast-open 529b4c33049de4100bc6f5ecaafa7722
ast-ksh d0762a194fe508641fa0706294236d3f
the md5 sums should match the ones listed on the download page
The subshell.sh test runs out of memory on small Linux machines (<2GB
ram, 64bit, Suse 12.2):
test subshell(en_US.UTF-8) begins at 2013-09-27+11:50:19
subshell.sh[320]: test 2 '$(...) exit 0' failed -- got
'/home/lcons/newksh/arch/linux.i386-64/src/cmd/ksh93/ksh[8]: tr:
storage allocator out of space on 8590983168 byte request ( region
8608808960 segments 15 busy 0:0 free 0:0 ) [Cannot allocate memory]',
expected 'ok'
subshell.sh[320]: test 2 '$(...) exit 2' failed -- got
'/home/lcons/newksh/arch/linux.i386-64/src/cmd/ksh93/ksh[8]: tr:
storage allocator out of space on 8590983168 byte request ( region
8608808960 segments 15 busy 0:0 free 0:0 ) [Cannot allocate memory]',
expected 'ok'
subshell.sh[320]: test 2 '${ ...; } exit 0' failed -- got
'/home/lcons/newksh/arch/linux.i386-64/src/cmd/ksh93/ksh[8]: tr:
storage allocator out of space on 8590983168 byte request ( region
8608808960 segments 15 busy 0:0 free 0:0 ) [Cannot allocate memory]',
expected 'ok'
subshell.sh[320]: test 2 '${ ...; } exit 2' failed -- got
'/home/lcons/newksh/arch/linux.i386-64/src/cmd/ksh93/ksh[8]: tr:
storage allocator out of space on 8590983168 byte request ( region
8608808960 segments 15 busy 0:0 free 0:0 ) [Cannot allocate memory]',
expected 'ok'
subshell.sh[320]: test 2 '$(ulimit -c 0; ...) exit 0' failed
-- got '/home/lcons/newksh/arch/linux.i386-64/src/cmd/ksh93/ksh[8]:
tr: storage allocator out of space on 8590983168 byte request ( region
8608808960 segments 15 busy 0:0 free 0:0 ) [Cannot allocate memory]',
expected 'ok'
subshell.sh[320]: test 2 '$(ulimit -c 0; ...) exit 2' failed
-- got '/home/lcons/newksh/arch/linux.i386-64/src/cmd/ksh93/ksh[8]:
tr: storage allocator out of space on 8590983168 byte request ( region
8608808960 segments 15 busy 0:0 free 0:0 ) [Cannot allocate memory]',
expected 'ok'
subshell.sh[320]: test 2 '$( (...) ) exit 0' failed -- got
'/home/lcons/newksh/arch/linux.i386-64/src/cmd/ksh93/ksh[8]: tr:
storage allocator out of space on 8590983168 byte request ( region
8608808960 segments 15 busy 0:0 free 0:0 ) [Cannot allocate memory]',
expected 'ok'
subshell.sh[320]: test 2 '$( (...) ) exit 2' failed -- got
'/home/lcons/newksh/arch/linux.i386-64/src/cmd/ksh93/ksh[8]: tr:
storage allocator out of space on 8590983168 byte request ( region
8608808960 segments 15 busy 0:0 free 0:0 ) [Cannot allocate memory]',
expected 'ok'
subshell.sh[320]: test 2 '${ (...); } exit 0' failed -- got
'/home/lcons/newksh/arch/linux.i386-64/src/cmd/ksh93/ksh[8]: tr:
storage allocator out of space on 8590983168 byte request ( region
8608808960 segments 15 busy 0:0 free 0:0 ) [Cannot allocate memory]',
expected 'ok'
subshell.sh[320]: test 2 '${ (...); } exit 2' failed -- got
'/home/lcons/newksh/arch/linux.i386-64/src/cmd/ksh93/ksh[8]: tr:
storage allocator out of space on 8590983168 byte request ( region
8608808960 segments 15 busy 0:0 free 0:0 ) [Cannot allocate memory]',
expected 'ok'
subshell.sh[320]: test 2 '`...` exit 0' failed -- got
'/home/lcons/newksh/arch/linux.i386-64/src/cmd/ksh93/ksh[8]: tr:
storage allocator out of space on 8590983168 byte request ( region
8608808960 segments 15 busy 0:0 free 0:0 ) [Cannot allocate memory]',
expected 'ok'
subshell.sh[320]: test 2 '`...` exit 2' failed -- got
'/home/lcons/newksh/arch/linux.i386-64/src/cmd/ksh93/ksh[8]: tr:
storage allocator out of space on 8590983168 byte request ( region
8608808960 segments 15 busy 0:0 free 0:0 ) [Cannot allocate memory]',
expected 'ok'
subshell.sh[320]: test 2 '`(...)` exit 0' failed -- got
'/home/lcons/newksh/arch/linux.i386-64/src/cmd/ksh93/ksh[8]: tr:
storage allocator out of space on 8590983168 byte request ( region
8608808960 segments 15 busy 0:0 free 0:0 ) [Cannot allocate memory]',
expected 'ok'
subshell.sh[320]: test 2 '`(...)` exit 2' failed -- got
'/home/lcons/newksh/arch/linux.i386-64/src/cmd/ksh93/ksh[8]: tr:
storage allocator out of space on 8590983168 byte request ( region
8608808960 segments 15 busy 0:0 free 0:0 ) [Cannot allocate memory]',
expected 'ok'
subshell.sh[320]: test 2 '`ulimit -c 0; ...` exit 0' failed --
got '/home/lcons/newksh/arch/linux.i386-64/src/cmd/ksh93/ksh[8]: tr:
storage allocator out of space on 8590983168 byte request ( region
8608808960 segments 15 busy 0:0 free 0:0 ) [Cannot allocate memory]',
expected 'ok'
subshell.sh[320]: test 2 '`ulimit -c 0; ...` exit 2' failed --
got '/home/lcons/newksh/arch/linux.i386-64/src/cmd/ksh93/ksh[8]: tr:
storage allocator out of space on 8590983168 byte request ( region
8608808960 segments 15 busy 0:0 free 0:0 ) [Cannot allocate memory]',
expected 'ok'
test subshell(en_US.UTF-8) failed at 2013-09-27+13:34:53 with exit
code 16 [ 68 tests 16 errors ]

Lionel
Glenn Fowler
2013-09-27 17:44:01 UTC
Permalink
Post by Lionel Cons
Post by Glenn Fowler
the AT&T Software Technology ast alpha 2013-09-26 source release
has been posted to the download site
http://www.research.att.com/sw/download/alpha/
the package names and md5 checksums are
INIT 95da0c907a6ed25a601c373e16a1862e
ast-base 0aed6c2cd6bccf0ec23120179a1dfc38
ast-open 529b4c33049de4100bc6f5ecaafa7722
ast-ksh d0762a194fe508641fa0706294236d3f
the md5 sums should match the ones listed on the download page
The subshell.sh test runs out of memory on small Linux machines (<2GB
test subshell(en_US.UTF-8) begins at 2013-09-27+11:50:19
subshell.sh[320]: test 2 '$(...) exit 0' failed -- got
storage allocator out of space on 8590983168 byte request ( region
8608808960 segments 15 busy 0:0 free 0:0 ) [Cannot allocate memory]',
expected 'ok'
very strange

can you manually edit src/cmd/ksh93/tests/shtests

-[[ $VMALLOC_OPTIONS ]] || timesensitive=.
+[[ ,$VMALLOC_OPTIONS, == *,@(abort|check),* ]] || timesensitive=.

and then run the test with the new shtests and same env args as before except
insert this into the export list

VMALLOC_OPTIONS=usage ...

you should see lines like this in the output

vmalloc: 0x7fec39678000 1048576 region 0x007cf2a0 size=2097152 segs=1 packs=1 busy=24% cache=70736/363

if there aren't too many of them can you post the output
it looks like its one bogus 8Gib allocation instead of a bunch
so it should be reasonable size
my guess its the tr(1) misbehaving - this might help narrow it down
Loading...