[ast-developers] [ast-users] NaN and their payloads

[Roland Mainz]

I'm currently looking into a complex mathematical simulation (fluid
dynamics) and found references to NaN (Not A Number) with payloads.
The Nan values are used to represent 'no data' in the datasets and the
payloads are used for context information. The code has almost 2
million lines of code and removing the feature would be a major
effort.
As we are in the process of using ksh93 as "interface" to the
simulation using libshell we need access to Nan values and their
payloads.
Can anyone give an estimate how hard it would be to add support for
Nan payloads to ksh93?

I'm aware of NaN payload but have not seen C code that
* has MIN/MAX macros for the min/max payload value for a give FP type

See code attached code example... the maximum is the mask-1 ... the
minimum is either 0 or 1... I don't remember anymore...

* sets a NaN value with a specific payload
* extracts a payload from a NaN
are there standard api's for this?

Yes... the function |nan()| - see
http://pubs.opengroup.org/onlinepubs/009695399/functions/nan.html
-- snip --
double nan(const char *tagp);
float nanf(const char *tagp);
long double nanl(const char *tagp);
-- snip --

The trouble is... the exact string input is
compiler/libc/platforms-specific... and beyond highend HPC platforms
most platforms do not implement |nan()| with payload support.

The other issue is that often compiler/libc/platforms-combinations to
not preserve the payload when a value is casted between _different_
floating-point types. That problem isn't new and has haunted HPC
programmers since eternity... but the solution is to simply test with
C99 |isnan()| and then add some copy-payload code. Since |isnan()| is
basically a bit-pattern test the overhead is insignificant (on SX-8X
that's around 8 instructions more compared to a plain cast).

If we implement NaN's with payloads we have to make sure that _every_
cast between |Sfdouble| (basically a |long double|) and another
floating-point type is guarded with such copy-payload code... except
when we pass the value to a arithmetricfunction like |sin()|/|cos()|
etc. which (intentionally) discard the payload if they pass the Nan
through.

Attached (as "nan_payload_test1_20130904_001.c.txt") is some prototype
code which shows now to handle NaN payloads...

* Notes:
- The code explicitly requires ISO C99 semantics
- The code relies on |__int128_t| to handle |long double|. Originally
I thought about handling the code using byte-by-byte masking... but it
turned out that this creates a lot of mess and is endian-specific. The
final code may have to resort to that to be portable but for now I
use |__int128_t| to keep the code readable

----

Bye,
Roland
--
__ . . __
(o.\ \/ /.o) roland.mainz at nrubsig.org
\__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer
/O /==\ O\ TEL [phone]
(;O/ \/ \O;)
-------------- next part --------------
/***********************************************************************
* *
* This software is part of the ast package *
* Copyright (c) 2013 AT&T Intellectual Property *
* and is licensed under the *
* Common Public License, Version 1.0 *
* by AT&T Intellectual Property *
* *
* A copy of the License is available at *
* http://www.opensource.org/licenses/cpl1.0.txt *
* (with md5 checksum 059e8cd6165cb4c31e351f2b69388fd9) *
* *
* Information and Software Systems Research *
* AT&T Research *
* Florham Park NJ *
* *
* Roland Mainz <roland.mainz at nrubsig.org> *
* *
***********************************************************************/

#include <stdlib.h>
#include <stdint.h>
#include <stdio.h>
#include <math.h>
#include <float.h>
#include <limits.h>

/*
* Notes about portablity:
* - The ISO C99 standards supports a portable way to write the
* payload of a quiet NaN in ISO C.
* - The |nan()|/|nanf()|/|nanl()| functions (from <math.h>, see
* section 7.12.11.2 in the ISO C99 spec) accept strings as
* arguments.
* - |strtof()|/|strtod()|/|strtold()| functions (from <stdlib.h>,
* see section 7.20.1.3) accept strings in the form
* "NAN(character sequence)".
* - |fscanf()|+|sscanf()| follow |strtod()|
* - the character sequence in "NAN(character sequence)" is
* interpreted in an (libc+compiler) implementation-specific
* way (=not portable).
*/

void print_bits_32(uint32_t x)
{
int i;
for (i=0 ; i < 32 ; i++)
{
/* fixme: endian! */
printf("%d", (x & (1<<31))?1:0);
x = x << 1;
}
}
void print_bits_64(uint64_t x)
{
int i;
for (i=0 ; i < 64 ; i++)
{
/* fixme: endian! */
printf("%d", (x & (1LL<<63))?1:0);
x = x << 1;
}
}
void print_bits_128(__int128_t x)
{
int i;
for (i=0 ; i < 128 ; i++)
{
/* fixme: endian! */
printf("%d", (x & (((__int128_t)1)<<127))?1:0);
x = x << 1;
}
}

void set_nan_payload_binary32(float *val, unsigned int payload)
{
if (!isnanf(*val))
return;

volatile uint32_t *bitval_ptr = ((volatile uint32_t *)val);

const uint32_t exp_bits = log2f(FLT_MAX_EXP)+1;
const uint32_t frac_bits = 32-1-exp_bits;
const uint32_t frac_mask = powf(2, frac_bits-1)-1;

*bitval_ptr &= ~frac_mask;
*bitval_ptr |= (payload & frac_mask);
}

unsigned int get_nan_payload_binary32(float x)
{
unsigned int payload;

if(!isnanf(x))
return 0;

const uint32_t exp_bits = log2(FLT_MAX_EXP)+1;
const uint32_t frac_bits = 32-1-exp_bits;
const uint32_t frac_mask = pow(2, frac_bits-1)-1;

volatile uint32_t *bitval_ptr = ((volatile uint32_t *)&x);

payload = *bitval_ptr & frac_mask;

return payload;
}

void set_nan_payload_binary64(double *val, unsigned long payload)
{
if (!isnan(*val))
return;

volatile uint64_t *bitval_ptr = ((volatile uint64_t *)val);

const uint64_t exp_bits = log2(DBL_MAX_EXP)+1;
const uint64_t frac_bits = 64-1-exp_bits;
const uint64_t frac_mask = pow(2, frac_bits-1)-1;

*bitval_ptr &= ~frac_mask;
*bitval_ptr |= (payload & frac_mask);
}

unsigned long get_nan_payload_binary64(double x)
{
unsigned long payload;

if(!isnan(x))
return 0UL;

const uint64_t exp_bits = log2(DBL_MAX_EXP)+1;
const uint64_t frac_bits = 64-1-exp_bits;
const uint64_t frac_mask = pow(2, frac_bits-1)-1;

volatile uint64_t *bitval_ptr = ((volatile uint64_t *)&x);

payload = *bitval_ptr & frac_mask;

return payload;
}

const int ldbl_bits=80/*128*/;

void set_nan_payload_binary80(long double *val, unsigned long payload)
{
if (!isnanl(*val))
return;

volatile __int128_t *bitval_ptr = ((volatile __int128_t *)val);

/* what about Intel i387 80bit |long double|? ? */

const __int128_t exp_bits = log2l(LDBL_MAX_EXP)+1;
const __int128_t frac_bits = ldbl_bits-1-exp_bits;
const __int128_t frac_mask = ((__int128_t)powl(2, frac_bits-1))-1;

*bitval_ptr &= ~frac_mask;
*bitval_ptr |= (payload & frac_mask);
}

unsigned long get_nan_payload_binary80(long double x)
{
unsigned long payload;

if(!isnanl(x))
return 0;

const __int128_t exp_bits = log2l(LDBL_MAX_EXP)+1;
const __int128_t frac_bits = ldbl_bits-1-exp_bits;
const __int128_t frac_mask = ((__int128_t)powl(2, frac_bits-1))-1;

volatile __int128_t *bitval_ptr = ((volatile __int128_t *)&x);

payload = *bitval_ptr & frac_mask;

return payload;
}

float cast_binary64_to_binary32(double d)
{
float f;

if (isnan(d))
{
unsigned long payload;

payload = get_nan_payload_binary64(d);
f = (float)d;

set_nan_payload_binary32(&f, payload);
}
else
{
f = (float)d;
}

return f;
}


float cast_binary80_to_binary32(long double ld)
{
float f;

if (isnanl(ld))
{
unsigned long payload;

payload = get_nan_payload_binary80(ld);
f = (float)ld;

set_nan_payload_binary32(&f, payload);
}
else
{
f = (float)ld;
}

return f;
}


double cast_binary80_to_binary64(long double ld)
{
double d;

if (isnanl(ld))
{
unsigned long payload;

payload = get_nan_payload_binary80(ld);
d = (double)ld;

set_nan_payload_binary64(&d, payload);
}
else
{
d = (double)ld;
}

return d;
}

int main(int ac, char *av[])
{
volatile float val=NAN;

long i;

uint32_t *xp=(uint32_t *)&val;

printf("orig:\t" ); print_bits_32(*xp); printf("\tval=%f\n", val);

const uint32_t exp_bits = log2(FLT_MAX_EXP)+1;
const uint32_t frac_bits = 32-1-exp_bits;
const uint32_t frac_mask = pow(2, frac_bits-1)-1;

printf("exp_bits=%d, frac_bits=%d, frac_mask=%lx\n", exp_bits, frac_bits, (unsigned long)frac_mask);

printf("frcmsk:\t" ); print_bits_32(frac_mask); printf("\n");

*xp |= 0x01;
printf("0x01:\t" ); print_bits_32(*xp); printf("\tval=%f\n", val);

for (i=0 ; i <= frac_mask ; i++)
{
volatile uint32_t *bitval_ptr = ((volatile uint32_t *)&val);

val = NAN;
*bitval_ptr &= ~frac_mask;
*bitval_ptr |= (i & frac_mask);
if (!isnan(val))
printf("%f not a nan for %ld\n", val, (long)i);
if (isinf(val))
printf("%f is a inf for %ld\n", val, (long)i);
}

{
float fval;
fval = -NAN;
set_nan_payload_binary32(&fval, 667);
printf("flt: val with -NAN667=%f\n", fval);
printf("flt: payload of -NAN667=%d\n", (int)get_nan_payload_binary32(fval));
}

{
double dval;
dval = -NAN;
set_nan_payload_binary64(&dval, 667);
printf("dbl: val with -NAN667=%f\n", dval);
printf("dbl: payload of -NAN667=%lu\n", (unsigned long)get_nan_payload_binary64(dval));
printf("dbl: payload of sys_cast float(-NAN667)=%lu\n", (unsigned long)get_nan_payload_binary32((float)dval));
printf("dbl: payload of own_cast float(-NAN667)=%lu\n", (unsigned long)get_nan_payload_binary32(cast_binary64_to_binary32(dval)));
}

{
long double ldval;
ldval = -NAN;
set_nan_payload_binary80(&ldval, 667);
printf("ldbl: val with -NAN667=%Lf\n", ldval);
printf("ldbl: payload of -NAN667=%lu\n", (unsigned long)get_nan_payload_binary80(ldval));
printf("ldbl: payload of sys_cast double(-NAN667)=%lu\n", (unsigned long)get_nan_payload_binary64((double)ldval));
printf("ldbl: payload of own_cast double(-NAN667)=%lu\n", (unsigned long)get_nan_payload_binary64(cast_binary80_to_binary64(ldval)));
}

return EXIT_SUCCESS;
}

[Roland Mainz]

Post by Roland Mainz

I'm currently looking into a complex mathematical simulation (fluid
dynamics) and found references to NaN (Not A Number) with payloads.
The Nan values are used to represent 'no data' in the datasets and the
payloads are used for context information. The code has almost 2
million lines of code and removing the feature would be a major
effort.
As we are in the process of using ksh93 as "interface" to the
simulation using libshell we need access to Nan values and their
payloads.
Can anyone give an estimate how hard it would be to add support for
Nan payloads to ksh93?

I'm aware of NaN payload but have not seen C code that
* has MIN/MAX macros for the min/max payload value for a give FP type

See code attached code example... the maximum is the mask-1 ... the
minimum is either 0 or 1... I don't remember anymore...

* sets a NaN value with a specific payload
* extracts a payload from a NaN
are there standard api's for this?

Yes... the function |nan()| - see
http://pubs.opengroup.org/onlinepubs/009695399/functions/nan.html
-- snip --
double nan(const char *tagp);
float nanf(const char *tagp);
long double nanl(const char *tagp);
-- snip --
The trouble is... the exact string input is
compiler/libc/platforms-specific... and beyond highend HPC platforms
most platforms do not implement |nan()| with payload support.
The other issue is that often compiler/libc/platforms-combinations to
not preserve the payload when a value is casted between _different_
floating-point types. That problem isn't new and has haunted HPC
programmers since eternity... but the solution is to simply test with
C99 |isnan()| and then add some copy-payload code. Since |isnan()| is
basically a bit-pattern test the overhead is insignificant (on SX-8X
that's around 8 instructions more compared to a plain cast).
If we implement NaN's with payloads we have to make sure that _every_
cast between |Sfdouble| (basically a |long double|) and another
floating-point type is guarded with such copy-payload code... except
when we pass the value to a arithmetricfunction like |sin()|/|cos()|
etc. which (intentionally) discard the payload if they pass the Nan
through.
Attached (as "nan_payload_test1_20130904_001.c.txt") is some prototype
code which shows now to handle NaN payloads...
- The code explicitly requires ISO C99 semantics
- The code relies on |__int128_t| to handle |long double|. Originally
I thought about handling the code using byte-by-byte masking... but it
turned out that this creates a lot of mess and is endian-specific. The
final code may have to resort to that to be portable but for now I
use |__int128_t| to keep the code readable

Uhm... any feedback/suggestions ? Adding Nan payload support seems to
be easy and doesn't come with a measureable performance penalty (since
we only do extra work *if* there is a payload, otherwise most code has
only "wrapper" overhead) ... if there are no objections I can try my
luck and add this to ksh93's arithmetic engine...

----

Bye,
Roland

[Roland Mainz]

Post by Roland Mainz

Post by Roland Mainz

I'm currently looking into a complex mathematical simulation (fluid
dynamics) and found references to NaN (Not A Number) with payloads.
The Nan values are used to represent 'no data' in the datasets and the
payloads are used for context information. The code has almost 2
million lines of code and removing the feature would be a major
effort.
As we are in the process of using ksh93 as "interface" to the
simulation using libshell we need access to Nan values and their
payloads.
Can anyone give an estimate how hard it would be to add support for
Nan payloads to ksh93?

I'm aware of NaN payload but have not seen C code that
* has MIN/MAX macros for the min/max payload value for a give FP type

See code attached code example... the maximum is the mask-1 ... the
minimum is either 0 or 1... I don't remember anymore...

* sets a NaN value with a specific payload
* extracts a payload from a NaN
are there standard api's for this?

Yes... the function |nan()| - see
http://pubs.opengroup.org/onlinepubs/009695399/functions/nan.html
-- snip --
double nan(const char *tagp);
float nanf(const char *tagp);
long double nanl(const char *tagp);
-- snip --
The trouble is... the exact string input is
compiler/libc/platforms-specific... and beyond highend HPC platforms
most platforms do not implement |nan()| with payload support.
The other issue is that often compiler/libc/platforms-combinations to
not preserve the payload when a value is casted between _different_
floating-point types. That problem isn't new and has haunted HPC
programmers since eternity... but the solution is to simply test with
C99 |isnan()| and then add some copy-payload code. Since |isnan()| is
basically a bit-pattern test the overhead is insignificant (on SX-8X
that's around 8 instructions more compared to a plain cast).
If we implement NaN's with payloads we have to make sure that _every_
cast between |Sfdouble| (basically a |long double|) and another
floating-point type is guarded with such copy-payload code... except
when we pass the value to a arithmetricfunction like |sin()|/|cos()|
etc. which (intentionally) discard the payload if they pass the Nan
through.
Attached (as "nan_payload_test1_20130904_001.c.txt") is some prototype
code which shows now to handle NaN payloads...
- The code explicitly requires ISO C99 semantics
- The code relies on |__int128_t| to handle |long double|. Originally
I thought about handling the code using byte-by-byte masking... but it
turned out that this creates a lot of mess and is endian-specific. The
final code may have to resort to that to be portable but for now I
use |__int128_t| to keep the code readable

Uhm... any feedback/suggestions ? Adding Nan payload support seems to
be easy and doesn't come with a measureable performance penalty (since
we only do extra work *if* there is a payload, otherwise most code has
only "wrapper" overhead) ... if there are no objections I can try my
luck and add this to ksh93's arithmetic engine...

Attached (as "astksh20130913_nanpayload_prototype001.diff.txt") is a
_prototype_ for SuSE Linux 12.3/AMD64/32bit. 32bit will not work
because it lacks the |__int128_t| type... see below...

Example usage:
-- snip --
$ ksh -c 'typeset -X x ; typeset -sX y ; float z ; ((
x=ast_nan(0xbeef) )) ; ((y=x, z=y)); print -- $((z))'
ast_nan(0xbeef)
-- snip --

* Notes:
- This is a prototype
- The code was only tested on SuSE 12.3/AMD64/64bit with gcc4.7.
- Surprisingly the performance overhead is barely measureable...
basically we have a few extra |isnan()| calls or macros and that's it

* ToDo:
- Put the nan payload functions into their own source file
- ABI issue: |nan("")| on some platforms only sets the highest hit in
the payload instead of 0xFFFFFFFF (e.g. set all bits). How should we
deal with it ?
- Keep the nan payload function names (*binary@(32|64|80|128)* but add
platform-specific macros to map them properly to SFDOUBLE, |long
double|, |double| and |float|). The seperation is important to keep
the #ifdef/#else/#endif hackery to a minimum and in one place
- How should we deal with the usage of |__int128_t| ? It's only
available on 64bit plaftorms with "clang" and newer "gcc" versions...
but writing a "portable" version which does the operations on a
byte/word/longword level makes the code depending on the endian-ness
of the platform. Main issue here is testing... my SPARC machine is
broken and I need some time to find money to fix it.

----

Bye,
Roland
--
__ . . __
(o.\ \/ /.o) roland.mainz at nrubsig.org
\__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer
/O /==\ O\ TEL [phone]
(;O/ \/ \O;)
-------------- next part --------------
diff -r -u original/src/cmd/ksh93/data/math.tab build_nanpayload/src/cmd/ksh93/data/math.tab
--- src/cmd/ksh93/data/math.tab 2013-09-04 20:02:33.000000000 +0200
+++ src/cmd/ksh93/data/math.tab 2013-09-19 17:22:33.778814914 +0200
@@ -5,6 +5,7 @@
f 1 acosh
f 1 asin
f 1 asinh
+f 1 ast_nan
f 1 atan
f 2 atan2
f 1 atanh
diff -r -u original/src/cmd/ksh93/features/math.sh build_nanpayload/src/cmd/ksh93/features/math.sh
--- src/cmd/ksh93/features/math.sh 2013-09-07 06:31:37.000000000 +0200
+++ src/cmd/ksh93/features/math.sh 2013-09-19 17:14:10.339149139 +0200
@@ -25,7 +25,7 @@
command=$0
iffeflags="-n -v"
iffehdrs="math.h"
-iffelibs="-lm"
+iffelibs="-last -lm"
table=/dev/null

eval $1
@@ -38,11 +38,11 @@

: check ast_standards.h

-eval `iffe $iffeflags -F ast_standards.h -c "$cc" - tst use_ast_standards -lm 'note{' 'math.h needs ast_standards.h' '}end' 'link{' '#include <math.h>' '#ifndef isgreater' '#define isgreater(a,b) 0' '#endif' 'int main() { return isgreater(0.0,1.0); }' '}end'`
+eval `iffe $iffeflags -F ast_standards.h -c "$cc" - tst use_ast_standards -last -lm 'note{' 'math.h needs ast_standards.h' '}end' 'link{' '#include <math.h>' '#ifndef isgreater' '#define isgreater(a,b) 0' '#endif' 'int main() { return isgreater(0.0,1.0); }' '}end'`
case $_use_ast_standards in
1) iffeflags="$iffeflags -F ast_standards.h" ;;
esac
-eval `iffe $iffeflags -c "$cc" - tst use_ieeefp -lm 'note{' 'ieeefp.h plays nice' '}end' 'link{' '#include <math.h>' '#include <ieeefp.h>' 'int main() { return 0; }' '}end'`
+eval `iffe $iffeflags -c "$cc" - tst use_ieeefp -last -lm 'note{' 'ieeefp.h plays nice' '}end' 'link{' '#include <math.h>' '#include <ieeefp.h>' 'int main() { return 0; }' '}end'`
case $_use_ieeefp in
1) iffehdrs="$iffehdrs ieeefp.h" ;;
esac
diff -r -u original/src/cmd/ksh93/sh/name.c build_nanpayload/src/cmd/ksh93/sh/name.c
--- src/cmd/ksh93/sh/name.c 2013-09-04 22:01:08.000000000 +0200
+++ src/cmd/ksh93/sh/name.c 2013-09-19 20:33:34.566356847 +0200
@@ -1747,9 +1747,9 @@
if(flags&NV_LONG)
ld = *((Sfdouble_t*)sp);
else if(flags&NV_SHORT)
- ld = *((float*)sp);
+ ld = cast_binary32_to_binary80(*((float*)sp));
else
- ld = *((double*)sp);
+ ld = cast_binary64_to_binary80(*((double*)sp));
}
else
ld = sh_arith(shp,sp);
@@ -1765,9 +1765,9 @@
if(flags&NV_INTEGER)
{
if(flags&NV_LONG)
- d = (double)(*(Sfdouble_t*)sp);
+ d = cast_binary80_to_binary64((*(Sfdouble_t*)sp));
else if(flags&NV_SHORT)
- d = (double)(*(float*)sp);
+ d = cast_binary32_to_binary64((*(float*)sp));
else
d = *(double*)sp;
}
@@ -1783,7 +1783,7 @@
else if(flags&NV_APPEND)
od = *(up->dp);
if(nv_isattr(np,NV_SHORT))
- *(up->fp) = (float)od?d+od:d;
+ *(up->fp) = cast_binary64_to_binary32(od?d+od:d);
else
*(up->dp) = od?d+od:d;
}
@@ -1912,6 +1912,13 @@
{
if(flags&NV_LONG)
sfprintf(shp->strbuf,"%.*Lg",LDBL_DIG,*((Sfdouble_t*)sp));
+ else if(flags&NV_SHORT)
+ /*
+ * note: |float|2|double| promotion implicit,
+ * |cast_binary32_to_binary64() does it explicit
+ * to preserve nan payload
+ */
+ sfprintf(shp->strbuf,"%.*g",DBL_DIG, cast_binary32_to_binary64(*((float*)sp)));
else
sfprintf(shp->strbuf,"%.*g",DBL_DIG,*((double*)sp));
}
@@ -2920,7 +2927,7 @@
else
{
if(nv_isattr(np,NV_SHORT))
- d = *up->fp;
+ d = cast_binary32_to_binary64(*up->fp);
else
d = *up->dp;
if(nv_isattr (np,NV_EXPNOTE))
@@ -3038,9 +3045,9 @@
if(nv_isattr(np, NV_LONG))
r = *up->ldp;
else if(nv_isattr(np, NV_SHORT))
- r = *up->fp;
+ r = cast_binary32_to_binary80(*up->fp);
else
- r = *up->dp;
+ r = cast_binary64_to_binary80(*up->dp);
}
else if(nv_isattr(np, NV_UNSIGN))
{
diff -r -u original/src/lib/libast/include/ast.h build_nanpayload/src/lib/libast/include/ast.h
--- src/lib/libast/include/ast.h 2013-09-10 11:01:39.000000000 +0200
+++ src/lib/libast/include/ast.h 2013-09-19 20:09:50.862074657 +0200
@@ -411,6 +411,27 @@
extern ssize_t utf32stowcs(wchar_t*, uint32_t*, size_t);
extern ssize_t wcstoutf32s(uint32_t*, wchar_t*, size_t);

+#if 1
+/* wrong header file, hacking here for now */
+long double ast_nanl(long double payload);
+double ast_nan(double payload);
+float ast_nanf(float payload);
+
+void set_nan_payload_binary32(float *val, unsigned int payload);
+unsigned int get_nan_payload_binary32(float x);
+void set_nan_payload_binary64(double *val, unsigned long payload);
+unsigned long get_nan_payload_binary64(double x);
+void set_nan_payload_binary80(long double *val, unsigned long payload);
+unsigned long get_nan_payload_binary80(long double x);
+float cast_binary64_to_binary32(double d);
+long double cast_binary64_to_binary80(double d);
+double cast_binary32_to_binary64(float f);
+long double cast_binary32_to_binary80(float f);
+float cast_binary80_to_binary32(long double ld);
+double cast_binary80_to_binary64(long double ld);
+
+#endif
+
#undef extern

/*
diff -r -u original/src/lib/libast/sfio/sfcvt.c build_nanpayload/src/lib/libast/sfio/sfcvt.c
--- src/lib/libast/sfio/sfcvt.c 2012-06-19 09:27:53.000000000 +0200
+++ src/lib/libast/sfio/sfcvt.c 2013-09-19 23:27:01.433165300 +0200
@@ -24,6 +24,387 @@
#endif
#include "sfhdr.h"

+#if 1 /* nanpayload*/
+#include <stdint.h>
+#include <math.h>
+#include <limits.h>
+
+long double ast_nanl(long double payload)
+{
+ long double num=NAN;
+ set_nan_payload_binary80(&num, (unsigned long)payload);
+ return num;
+}
+double ast_nan(double payload)
+{
+ double num=NAN;
+ set_nan_payload_binary64(&num, (unsigned long)payload);
+ return num;
+}
+float ast_nanf(float payload)
+{
+ float num=NAN;
+ set_nan_payload_binary32(&num, (unsigned long)payload);
+ return num;
+}
+
+/*
+ * Notes about portablity:
+ * - The ISO C99 standards supports a portable way to write the
+ * payload of a quiet NaN in ISO C.
+ * - The |nan()|/|nanf()|/|nanl()| functions (from <math.h>, see
+ * section 7.12.11.2 in the ISO C99 spec) accept strings as
+ * arguments.
+ * - |strtof()|/|strtod()|/|strtold()| functions (from <stdlib.h>,
+ * see section 7.20.1.3) accept strings in the form
+ * "NAN(character sequence)".
+ * - |fscanf()|+|sscanf()| follow |strtod()|
+ * - the character sequence in "NAN(character sequence)" is
+ * interpreted in an (libc+compiler) implementation-specific
+ * way (=not portable).
+ */
+
+void print_bits_32(uint32_t x)
+{
+ int i;
+ for (i=0 ; i < 32 ; i++)
+ {
+ /* fixme: endian! */
+ printf("%d", (x & (1<<31))?1:0);
+ x = x << 1;
+ }
+}
+void print_bits_64(uint64_t x)
+{
+ int i;
+ for (i=0 ; i < 64 ; i++)
+ {
+ /* fixme: endian! */
+ printf("%d", (x & (1LL<<63))?1:0);
+ x = x << 1;
+ }
+}
+void print_bits_128(__int128_t x)
+{
+ int i;
+ for (i=0 ; i < 128 ; i++)
+ {
+ /* fixme: endian! */
+ printf("%d", (x & (((__int128_t)1)<<127))?1:0);
+ x = x << 1;
+ }
+}
+
+void set_nan_payload_binary32(float *val, unsigned int payload)
+{
+ if (payload == 0)
+ return;
+ if (!isnanf(*val))
+ return;
+
+ volatile uint32_t *bitval_ptr = ((volatile uint32_t *)val);
+
+ const uint32_t exp_bits = log2f(FLT_MAX_EXP)+1;
+ const uint32_t frac_bits = 32-1-exp_bits;
+ const uint32_t frac_mask = powf(2, frac_bits-1)-1;
+
+ *bitval_ptr &= ~frac_mask;
+ *bitval_ptr |= (payload & frac_mask);
+}
+
+unsigned int get_nan_payload_binary32(float x)
+{
+ unsigned int payload;
+
+ if(!isnanf(x))
+ return 0;
+
+ const uint32_t exp_bits = log2(FLT_MAX_EXP)+1;
+ const uint32_t frac_bits = 32-1-exp_bits;
+ const uint32_t frac_mask = pow(2, frac_bits-1)-1;
+
+ volatile uint32_t *bitval_ptr = ((volatile uint32_t *)&x);
+
+ payload = *bitval_ptr & frac_mask;
+
+ return payload;
+}
+
+void set_nan_payload_binary64(double *val, unsigned long payload)
+{
+ if (payload == 0)
+ return;
+ if (!isnan(*val))
+ return;
+
+ volatile uint64_t *bitval_ptr = ((volatile uint64_t *)val);
+
+ const uint64_t exp_bits = log2(DBL_MAX_EXP)+1;
+ const uint64_t frac_bits = 64-1-exp_bits;
+ const uint64_t frac_mask = pow(2, frac_bits-1)-1;
+
+ *bitval_ptr &= ~frac_mask;
+ *bitval_ptr |= (payload & frac_mask);
+}
+
+unsigned long get_nan_payload_binary64(double x)
+{
+ unsigned long payload;
+
+ if(!isnan(x))
+ return 0UL;
+
+ const uint64_t exp_bits = log2(DBL_MAX_EXP)+1;
+ const uint64_t frac_bits = 64-1-exp_bits;
+ const uint64_t frac_mask = pow(2, frac_bits-1)-1;
+
+ volatile uint64_t *bitval_ptr = ((volatile uint64_t *)&x);
+
+ payload = *bitval_ptr & frac_mask;
+
+ return payload;
+}
+
+const int ldbl_bits=80/*128*/;
+
+void set_nan_payload_binary80(long double *val, unsigned long payload)
+{
+ if (payload == 0)
+ return;
+ if (!isnanl(*val))
+ return;
+
+ volatile __int128_t *bitval_ptr = ((volatile __int128_t *)val);
+
+ /* what about Intel i387 80bit |long double|? ? */
+
+ const __int128_t exp_bits = log2l(LDBL_MAX_EXP)+1;
+ const __int128_t frac_bits = ldbl_bits-1-exp_bits;
+ const __int128_t frac_mask = ((__int128_t)powl(2, frac_bits-1))-1;
+
+ *bitval_ptr &= ~frac_mask;
+ *bitval_ptr |= (payload & frac_mask);
+}
+
+unsigned long get_nan_payload_binary80(long double x)
+{
+ unsigned long payload;
+
+ if(!isnanl(x))
+ return 0;
+
+ const __int128_t exp_bits = log2l(LDBL_MAX_EXP)+1;
+ const __int128_t frac_bits = ldbl_bits-1-exp_bits;
+ const __int128_t frac_mask = ((__int128_t)powl(2, frac_bits-1))-1;
+
+ volatile __int128_t *bitval_ptr = ((volatile __int128_t *)&x);
+
+ payload = *bitval_ptr & frac_mask;
+
+ return payload;
+}
+
+float cast_binary64_to_binary32(double d)
+{
+ float f;
+
+ if (isnan(d))
+ {
+ unsigned long payload;
+
+ payload = get_nan_payload_binary64(d);
+ f = (float)d;
+
+ set_nan_payload_binary32(&f, payload);
+ }
+ else
+ {
+ f = (float)d;
+ }
+
+ return f;
+}
+
+
+long double cast_binary64_to_binary80(double d)
+{
+ long double ld;
+
+ if (isnan(d))
+ {
+ unsigned long payload;
+
+ payload = get_nan_payload_binary64(d);
+ ld = (long double)d;
+
+ set_nan_payload_binary80(&ld, payload);
+ }
+ else
+ {
+ ld = (long double)d;
+ }
+
+ return ld;
+}
+
+
+double cast_binary32_to_binary64(float f)
+{
+ double d;
+
+ if (isnanf(f))
+ {
+ unsigned long payload;
+
+ payload = get_nan_payload_binary32(f);
+ d = (double)f;
+
+ set_nan_payload_binary64(&d, payload);
+ }
+ else
+ {
+ d = (double)f;
+ }
+
+ return d;
+}
+
+
+long double cast_binary32_to_binary80(float f)
+{
+ long double ld;
+
+ if (isnanf(f))
+ {
+ unsigned long payload;
+
+ payload = get_nan_payload_binary32(f);
+ ld = (long double)f;
+
+ set_nan_payload_binary80(&ld, payload);
+ }
+ else
+ {
+ ld = (long double)f;
+ }
+
+ return ld;
+}
+
+
+float cast_binary80_to_binary32(long double ld)
+{
+ float f;
+
+ if (isnanl(ld))
+ {
+ unsigned long payload;
+
+ payload = get_nan_payload_binary80(ld);
+ f = (float)ld;
+
+ set_nan_payload_binary32(&f, payload);
+ }
+ else
+ {
+ f = (float)ld;
+ }
+
+ return f;
+}
+
+
+double cast_binary80_to_binary64(long double ld)
+{
+ double d;
+
+ if (isnanl(ld))
+ {
+ unsigned long payload;
+
+ payload = get_nan_payload_binary80(ld);
+ d = (double)ld;
+
+ set_nan_payload_binary64(&d, payload);
+ }
+ else
+ {
+ d = (double)ld;
+ }
+
+ return d;
+}
+
+#if 0
+int main(int ac, char *av[])
+{
+ volatile float val=NAN;
+
+ long i;
+
+ uint32_t *xp=(uint32_t *)&val;
+
+ printf("orig:\t" ); print_bits_32(*xp); printf("\tval=%f\n", val);
+
+ const uint32_t exp_bits = log2(FLT_MAX_EXP)+1;
+ const uint32_t frac_bits = 32-1-exp_bits;
+ const uint32_t frac_mask = pow(2, frac_bits-1)-1;
+
+ printf("exp_bits=%d, frac_bits=%d, frac_mask=%lx\n", exp_bits, frac_bits, (unsigned long)frac_mask);
+
+ printf("frcmsk:\t" ); print_bits_32(frac_mask); printf("\n");
+
+ *xp |= 0x01;
+ printf("0x01:\t" ); print_bits_32(*xp); printf("\tval=%f\n", val);
+
+ for (i=0 ; i <= frac_mask ; i++)
+ {
+ volatile uint32_t *bitval_ptr = ((volatile uint32_t *)&val);
+
+ val = NAN;
+ *bitval_ptr &= ~frac_mask;
+ *bitval_ptr |= (i & frac_mask);
+ if (!isnan(val))
+ printf("%f not a nan for %ld\n", val, (long)i);
+ if (isinf(val))
+ printf("%f is a inf for %ld\n", val, (long)i);
+ }
+
+ {
+ float fval;
+ fval = -NAN;
+ set_nan_payload_binary32(&fval, 667);
+ printf("flt: val with -NAN667=%f\n", fval);
+ printf("flt: payload of -NAN667=%d\n", (int)get_nan_payload_binary32(fval));
+ }
+
+ {
+ double dval;
+ dval = -NAN;
+ set_nan_payload_binary64(&dval, 667);
+ printf("dbl: val with -NAN667=%f\n", dval);
+ printf("dbl: payload of -NAN667=%lu\n", (unsigned long)get_nan_payload_binary64(dval));
+ printf("dbl: payload of sys_cast float(-NAN667)=%lu\n", (unsigned long)get_nan_payload_binary32((float)dval));
+ printf("dbl: payload of own_cast float(-NAN667)=%lu\n", (unsigned long)get_nan_payload_binary32(cast_binary64_to_binary32(dval)));
+ }
+
+ {
+ long double ldval;
+ ldval = -NAN;
+ set_nan_payload_binary80(&ldval, 667);
+ printf("ldbl: val with -NAN667=%Lf\n", ldval);
+ printf("ldbl: payload of -NAN667=%lu\n", (unsigned long)get_nan_payload_binary80(ldval));
+ printf("ldbl: payload of sys_cast double(-NAN667)=%lu\n", (unsigned long)get_nan_payload_binary64((double)ldval));
+ printf("ldbl: payload of own_cast double(-NAN667)=%lu\n", (unsigned long)get_nan_payload_binary64(cast_binary80_to_binary64(ldval)));
+ }
+
+ return EXIT_SUCCESS;
+}
+
+#endif
+
+#endif /* nanpayload */
+
/* Convert a floating point value to ASCII.
**
** Written by Kiem-Phong Vo and Glenn Fowler (SFFMT_AFORMAT)
@@ -118,8 +499,8 @@
int x;
_ast_flt_unsigned_max_t m;

- static char lx[] = "0123456789abcdef";
- static char ux[] = "0123456789ABCDEF";
+ static const char lx[] = "0123456789abcdef";
+ static const char ux[] = "0123456789ABCDEF";

*sign = *decpt = 0;

@@ -129,13 +510,26 @@

if(isnanl(f))
{
+ unsigned long payload;
#if _lib_signbit
if (signbit(f))
#else
if (f < 0)
#endif
*sign = 1;
- return SF_NAN;
+#if 1
+ /* grrr... on some platforms the default |NAN| sets only one bit instead of using 0xFFFFFFFF */
+ payload = get_nan_payload_binary80(f);
+ if ((payload != 0) && (payload != get_nan_payload_binary80(nanl(""))))
+#else
+ if (payload = get_nan_payload_binary80(f))
+#endif
+ {
+ snprintf(buf, size, "ast_nan(0x%lx)", (unsigned long)payload);
+ return buf;
+ }
+ else
+ return SF_NAN;
}
#if _lib_isinf
if (n = isinf(f))
@@ -172,7 +566,7 @@
case FP_INFINITE:
return SF_INF;
case FP_NAN:
- return SF_NAN;
+ return SF_NAN; /* isn't this redundant since we handle nan already above ? */
case FP_ZERO:
return SF_ZERO;
}
@@ -306,14 +700,27 @@
{ double f = *(double*)vp;

if(isnan(f))
- {
+ {
+ unsigned long payload;
#if _lib_signbit
if (signbit(f))
#else
if (f < 0)
#endif
*sign = 1;
- return SF_NAN;
+#if 1
+ /* grrr... on some platforms the default |NAN| sets only one bit instead of using 0xFFFFFFFF */
+ payload = get_nan_payload_binary64(f);
+ if ((payload != 0) && (payload != get_nan_payload_binary64(nan(""))))
+#else
+ if (payload = get_nan_payload_binary64(f))
+#endif
+ {
+ snprintf(buf, size, "ast_nan(0x%lx)", (unsigned long)payload);
+ return buf;
+ }
+ else
+ return SF_NAN;
}
#if _lib_isinf
if (n = isinf(f))
@@ -530,3 +937,4 @@
ep = sp + 1;
goto done;
}
+

[Roland Mainz]

Post by Roland Mainz

Post by Roland Mainz

Post by Roland Mainz

I'm currently looking into a complex mathematical simulation (fluid
dynamics) and found references to NaN (Not A Number) with payloads.
The Nan values are used to represent 'no data' in the datasets and the
payloads are used for context information. The code has almost 2
million lines of code and removing the feature would be a major
effort.

[snip]

Post by Roland Mainz

Post by Roland Mainz

Post by Roland Mainz
Attached (as "nan_payload_test1_20130904_001.c.txt") is some prototype
code which shows now to handle NaN payloads...
- The code explicitly requires ISO C99 semantics
- The code relies on |__int128_t| to handle |long double|. Originally
I thought about handling the code using byte-by-byte masking... but it
turned out that this creates a lot of mess and is endian-specific. The
final code may have to resort to that to be portable but for now I
use |__int128_t| to keep the code readable

Uhm... any feedback/suggestions ? Adding Nan payload support seems to
be easy and doesn't come with a measureable performance penalty (since
we only do extra work *if* there is a payload, otherwise most code has
only "wrapper" overhead) ... if there are no objections I can try my
luck and add this to ksh93's arithmetic engine...

Attached (as "astksh20130913_nanpayload_prototype001.diff.txt") is a
_prototype_ for SuSE Linux 12.3/AMD64/32bit. 32bit will not work
because it lacks the |__int128_t| type... see below...
-- snip --
$ ksh -c 'typeset -X x ; typeset -sX y ; float z ; ((
x=ast_nan(0xbeef) )) ; ((y=x, z=y)); print -- $((z))'
ast_nan(0xbeef)
-- snip --
- This is a prototype
- The code was only tested on SuSE 12.3/AMD64/64bit with gcc4.7.
- Surprisingly the performance overhead is barely measureable...
basically we have a few extra |isnan()| calls or macros and that's it
- Put the nan payload functions into their own source file
- ABI issue: |nan("")| on some platforms only sets the highest hit in
the payload instead of 0xFFFFFFFF (e.g. set all bits). How should we
deal with it ?
platform-specific macros to map them properly to SFDOUBLE, |long
double|, |double| and |float|). The seperation is important to keep
the #ifdef/#else/#endif hackery to a minimum and in one place
- How should we deal with the usage of |__int128_t| ? It's only
available on 64bit plaftorms with "clang" and newer "gcc" versions...
but writing a "portable" version which does the operations on a
byte/word/longword level makes the code depending on the endian-ness
of the platform. Main issue here is testing... my SPARC machine is
broken and I need some time to find money to fix it.

Attached (as "astksh20130913_nanpayload_prototype005.diff.txt") is an
updated patch for ast-ksh.2013-09-13 and
"nan_payload_test1_20130922_001.c.txt" is standalone testcode for
debugging purposes...

* Notes:
- This is a PROTOTYPE, the code is NOT ready yet. There is a lot of
"spaghetti" which is caused by the issue that the master sources for
the nan payload code live outside the AST tree for now... and there
are unresolved issues (see below)

- The API is *layered* intentionally: First layer uses
|flt_binary@(32|64|80|128)|, 2nd layer uses ISO C names (|float|,
|double|, |long double|). This is done to seperate logical from
physical floating-point layouts... for example on some platforms |long
double| is the same type as |double| and some platforms only know
about |double| etc. etc. The other area of pain is x86/AMD64 with
80bit |long double| vs. other platforms with 128bit |long double|

- we need to write an iffe probe for the nan payload stuff, which
provides constants like number of bits, masks etc. and verifies that
the |float|, |double| and |long double| behave *EXACTLY* as defined in
IEEE754-2008. Some old implementations and software emulations take
shortcuts... or have "magic bits" which can cause trouble (i387's
IEEE754-1985-draft implementation with 80bit are such a special case).
We can only support payloads if we have a
fully-IEEE754-2008-conforming floating-point implementation (99% of
the modern platforms should fall into this category)

- x86/AMD84 i387 80bit floats are fully functional and have extra code
to support it even if |__int128_t| is not supported

- 128bit floats should be working now but require at least some testing

- SPARC needs to be tested

- We need per-floating-point datatype payload MIN/MAX constants

- On some OSes like Solaris |nanf("")|/|nan("")|/|nanl("")| return a
payload of 0xFFFFFFFFFF... (=all payload bits set) while the |NAN|
contanst returns a nan value with all payload bits not set

- AST |strtold()| on Solaris returns a value like
|nanf("")|/|nan("")|/|nanl("")|, which causes trouble. Replacing it
with |ast_nan()| causes sign reversal for (yet) unknown reasons (but
the same happens with |sin(inf)| ... which may be a hint that
something else is wrong in libast vs. nan sign handling) ... needs
more debugging...

- Glenn and I need to agree on a naming scheme for NaN's with payloads
(gsf: We need to chat... issue is that not all naming schemes assume
that the payload is an unsigned int). Currently we use
"ast_nan(hexval)" in ksh93 and |strtof()|/|strtod()|/|strtold()|

----

Bye,
Roland
--
__ . . __
(o.\ \/ /.o) roland.mainz at nrubsig.org
\__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer
/O /==\ O\ TEL [phone]
(;O/ \/ \O;)
-------------- next part --------------
diff -r -u original/src/cmd/ksh93/data/math.tab build_nanpayload/src/cmd/ksh93/data/math.tab
--- src/cmd/ksh93/data/math.tab 2013-09-04 20:02:33.000000000 +0200
+++ src/cmd/ksh93/data/math.tab 2013-09-23 06:11:21.588380094 +0200
@@ -5,6 +5,7 @@
f 1 acosh
f 1 asin
f 1 asinh
+f 1 ast_nan
f 1 atan
f 2 atan2
f 1 atanh
@@ -56,6 +57,7 @@
f 1 log1p
f 1 log2
f 1 logb
+f 1 nanintpayload
f 1 nearbyint
f 3 2 nextafter
f 1 2 nexttoward
diff -r -u original/src/cmd/ksh93/features/math.sh build_nanpayload/src/cmd/ksh93/features/math.sh
--- src/cmd/ksh93/features/math.sh 2013-09-07 06:31:37.000000000 +0200
+++ src/cmd/ksh93/features/math.sh 2013-09-23 06:11:21.589805502 +0200
@@ -25,7 +25,7 @@
command=$0
iffeflags="-n -v"
iffehdrs="math.h"
-iffelibs="-lm"
+iffelibs="-last -lm"
table=/dev/null

eval $1
@@ -38,11 +38,11 @@

: check ast_standards.h

-eval `iffe $iffeflags -F ast_standards.h -c "$cc" - tst use_ast_standards -lm 'note{' 'math.h needs ast_standards.h' '}end' 'link{' '#include <math.h>' '#ifndef isgreater' '#define isgreater(a,b) 0' '#endif' 'int main() { return isgreater(0.0,1.0); }' '}end'`
+eval `iffe $iffeflags -F ast_standards.h -c "$cc" - tst use_ast_standards -last -lm 'note{' 'math.h needs ast_standards.h' '}end' 'link{' '#include <math.h>' '#ifndef isgreater' '#define isgreater(a,b) 0' '#endif' 'int main() { return isgreater(0.0,1.0); }' '}end'`
case $_use_ast_standards in
1) iffeflags="$iffeflags -F ast_standards.h" ;;
esac
-eval `iffe $iffeflags -c "$cc" - tst use_ieeefp -lm 'note{' 'ieeefp.h plays nice' '}end' 'link{' '#include <math.h>' '#include <ieeefp.h>' 'int main() { return 0; }' '}end'`
+eval `iffe $iffeflags -c "$cc" - tst use_ieeefp -last -lm 'note{' 'ieeefp.h plays nice' '}end' 'link{' '#include <math.h>' '#include <ieeefp.h>' 'int main() { return 0; }' '}end'`
case $_use_ieeefp in
1) iffehdrs="$iffehdrs ieeefp.h" ;;
esac
diff -r -u original/src/cmd/ksh93/sh/name.c build_nanpayload/src/cmd/ksh93/sh/name.c
--- src/cmd/ksh93/sh/name.c 2013-09-04 22:01:08.000000000 +0200
+++ src/cmd/ksh93/sh/name.c 2013-09-23 06:11:21.594210264 +0200
@@ -1747,9 +1747,9 @@
if(flags&NV_LONG)
ld = *((Sfdouble_t*)sp);
else if(flags&NV_SHORT)
- ld = *((float*)sp);
+ ld = CAST_FLT2SFDBL(*((float*)sp));
else
- ld = *((double*)sp);
+ ld = CAST_DBL2SFDBL(*((double*)sp));
}
else
ld = sh_arith(shp,sp);
@@ -1765,9 +1765,9 @@
if(flags&NV_INTEGER)
{
if(flags&NV_LONG)
- d = (double)(*(Sfdouble_t*)sp);
+ d = CAST_SFDBL2DBL((*(Sfdouble_t*)sp));
else if(flags&NV_SHORT)
- d = (double)(*(float*)sp);
+ d = CAST_FLT2DBL((*(float*)sp));
else
d = *(double*)sp;
}
@@ -1783,7 +1783,7 @@
else if(flags&NV_APPEND)
od = *(up->dp);
if(nv_isattr(np,NV_SHORT))
- *(up->fp) = (float)od?d+od:d;
+ *(up->fp) = CAST_DBL2FLT(od?d+od:d);
else
*(up->dp) = od?d+od:d;
}
@@ -1912,6 +1912,13 @@
{
if(flags&NV_LONG)
sfprintf(shp->strbuf,"%.*Lg",LDBL_DIG,*((Sfdouble_t*)sp));
+ else if(flags&NV_SHORT)
+ /*
+ * note: |float|-to-|double| promotion implicit,
+ * |CAST_FLT2DBL() does it explicit
+ * to preserve nan payload
+ */
+ sfprintf(shp->strbuf,"%.*g",DBL_DIG, CAST_FLT2DBL(*((float*)sp)));
else
sfprintf(shp->strbuf,"%.*g",DBL_DIG,*((double*)sp));
}
@@ -2920,7 +2927,7 @@
else
{
if(nv_isattr(np,NV_SHORT))
- d = *up->fp;
+ d = CAST_FLT2DBL(*up->fp);
else
d = *up->dp;
if(nv_isattr (np,NV_EXPNOTE))
@@ -3038,9 +3045,9 @@
if(nv_isattr(np, NV_LONG))
r = *up->ldp;
else if(nv_isattr(np, NV_SHORT))
- r = *up->fp;
+ r = CAST_FLT2SFDBL(*up->fp);
else
- r = *up->dp;
+ r = CAST_DBL2SFDBL(*up->dp);
}
else if(nv_isattr(np, NV_UNSIGN))
{
diff -r -u original/src/lib/libast/features/common build_nanpayload/src/lib/libast/features/common
--- src/lib/libast/features/common 2013-08-27 18:32:00.000000000 +0200
+++ src/lib/libast/features/common 2013-09-23 06:11:21.594786705 +0200
@@ -1,7 +1,7 @@
iff AST_COMMON
hdr pthread,stdarg,stddef,stdint,inttypes,types,unistd
sys types
-typ long.double,size_t,ssize_t
+typ long.double,size_t,ssize_t,__int128_t
typ __va_list stdio.h

mac SF_APPEND,SF_CLOSE sys/stat.h sys/socket.h
diff -r -u original/src/lib/libast/features/float build_nanpayload/src/lib/libast/features/float
--- src/lib/libast/features/float 2013-08-26 21:25:36.000000000 +0200
+++ src/lib/libast/features/float 2013-09-23 06:11:21.595448128 +0200
@@ -2,6 +2,7 @@
set nooptimize
set stdio FEATURE/standards
hdr float,limits,math,values
+typ __int128_t
lib,npt frexp,frexpl,ldexp,ldexpl,finite,finitel,isinfl,isnanl,copysign,copysignl FEATURE/standards math.h -lm
lib fpclassify -lm note{ fpclassify present and works }end link{
#include <sys/types.h>
diff -r -u original/src/lib/libast/features/sfio build_nanpayload/src/lib/libast/features/sfio
--- src/lib/libast/features/sfio 2010-05-26 06:45:57.000000000 +0200
+++ src/lib/libast/features/sfio 2013-09-23 06:11:21.595839794 +0200
@@ -2,6 +2,7 @@
hdr float,floatingpoint,math,values
sys filio,ioctl
lib qfrexp,qldexp
+typ __int128_t
key signed

tst - note{ number of bits in pointer }end output{
diff -r -u original/src/lib/libast/include/ast.h build_nanpayload/src/lib/libast/include/ast.h
--- src/lib/libast/include/ast.h 2013-09-10 11:01:39.000000000 +0200
+++ src/lib/libast/include/ast.h 2013-09-23 06:24:58.636813211 +0200
@@ -38,6 +38,17 @@
#include <sfio.h>
#endif

+/*
+ * fixme: Solaris chokes on <float.h> in <ast.h> - we should
+ * move the whole nan payload support in a seperate
+ * header+iffe probe to avoid this
+ */
+#if (defined(__i386) || defined(__amd64)) && defined(__SunOS)
+#define LDBL_MAX_EXP 16384
+#else
+#include <float.h>
+#endif
+
#ifndef ast
#define ast _ast_info
#endif
@@ -411,6 +422,203 @@
extern ssize_t utf32stowcs(wchar_t*, uint32_t*, size_t);
extern ssize_t wcstoutf32s(uint32_t*, wchar_t*, size_t);

+#if defined(LDBL_MAX_EXP) && (__STDC_VERSION__ >= 199901L) && ((LDBL_MAX_EXP-0) == 16384)
+/*
+ * The test above should move into a iffe probe and should
+ * explicitly check for IEEE754-2008 conformance and whether
+ * the bit layout meets our expectations (e.g. |long double|
+ * on i387 and mv68881/2 are 80bit, sometimes with one or
+ * more "magic bits" in the middle)
+ */
+#define FP_SUPPORTS_NAN_PAYLOADS 1
+#endif
+
+#if FP_SUPPORTS_NAN_PAYLOADS
+/* wrong header file, hacking here for now */
+
+/*
+ * |strtod()| testcase:
+ *
+ * #include <stdio.h>
+ * #include <stdlib.h>
+ * #include <string.h>
+ * #include <math.h>
+ *
+ * int main(int ac, char *av[])
+ * {
+ * double x;
+ *
+ * printf("nan(7)=%f\n", strtod("nan(7)", NULL));
+ * printf("ast_nan(7)=%f\n", strtod("ast_nan(7)", NULL));
+ * printf("-nan(17)=%f\n", strtod("-nan(17)", NULL));
+ * printf("-ast_nan(17)=%f\n", strtod("-ast_nan(17)", NULL));
+ * return (EXIT_SUCCESS);
+ * }
+ */
+
+/*
+ * nan payload interfaces for libshell's arithmetric support
+ * |ast_nan()| creates a nan value
+ * |nanintpayload()| retrieves an integer payload
+ */
+long double ast_nanl(long double payload);
+double ast_nan(double payload);
+float ast_nanf(float payload);
+/* these three should return |unsigned long| but "math.tab"&&co. do not suppport this */
+Sfdouble_t nanintpayloadl(long double f);
+Sfdouble_t nanintpayload(double f);
+Sfdouble_t nanintpayloadf(float f);
+
+/* 80 for AMD64, 128 on SPARC, xxx on other platforms */
+#ifndef NAN_PAYLOAD_LDBL_BITS
+#if LDBL_MAX_EXP == 1024
+#define NAN_PAYLOAD_LDBL_BITS 64 /* this happens if |long double| is the same as |double| */
+#elif LDBL_MAX_EXP == 16384
+#define NAN_PAYLOAD_LDBL_BITS 80
+#elif LDBL_MAX_EXP > 16384
+#define NAN_PAYLOAD_LDBL_BITS 128
+#else
+#error Unsupported LDBL_MAX_EXP (cannot calculate |long double| bit size)
+#endif
+#endif /* NAN_PAYLOAD_LDBL_BITS */
+
+#ifndef _typ___int128_t
+#ifdef __SIZEOF_INT128__
+#define _typ___int128_t 1
+#endif
+#endif
+
+/* mapping C types to aliases which define their physical layout */
+typedef float flt_binary32;
+typedef double flt_binary64;
+#if defined(__i386) || defined(__amd64)
+#if (NAN_PAYLOAD_LDBL_BITS == 128)
+typedef __float128 flt_binary128;
+#else
+typedef long double flt_binary80;
+#endif
+#else
+#if (NAN_PAYLOAD_LDBL_BITS == 128)
+typedef long double flt_binary128;
+#elif (NAN_PAYLOAD_LDBL_BITS == 64)
+#error |long double| == |double| not implemented yet
+#else
+#error Unsupported NAN_PAYLOAD_LDBL_BITS
+#endif
+#endif
+
+
+
+#if 1 /* prototypes */
+void set_nan_payload_binary32(flt_binary32 *val, uint32_t payload);
+uint32_t get_nan_payload_binary32(flt_binary32 x);
+void set_nan_payload_binary64(flt_binary64 *val, uint64_t payload);
+uint64_t get_nan_payload_binary64(flt_binary64 x);
+#if NAN_PAYLOAD_LDBL_BITS == 128
+void set_nan_payload_binary128(flt_binary128 *val, __int128_t payload);
+__int128_t get_nan_payload_binary128(flt_binary128 x);
+#elif NAN_PAYLOAD_LDBL_BITS == 80
+#ifdef _typ___int128_t
+void set_nan_payload_binary80(flt_binary80 *val, uint64_t payload);
+uint64_t get_nan_payload_binary80(flt_binary80 x);
+#else
+#if (defined(__i386) || defined(__amd64)) && (NAN_PAYLOAD_LDBL_BITS == 80)
+#define set_nan_payload_binary80(v, p) set_nan_payload_binary80_i387((v), (p))
+#define get_nan_payload_binary80(v) get_nan_payload_binary80_i387(v)
+#endif
+#endif /* _typ___int128_t */
+#if (defined(__i386) || defined(__amd64)) && (NAN_PAYLOAD_LDBL_BITS == 80)
+void set_nan_payload_binary80_i387(flt_binary80 *val, uint64_t payload);
+uint64_t get_nan_payload_binary80_i387(flt_binary80 x);
+#endif /* (defined(__i386) || defined(__amd64)) && (NAN_PAYLOAD_LDBL_BITS == 80) */
+#endif
+
+flt_binary32 cast_binary64_to_binary32(flt_binary64 d);
+#if NAN_PAYLOAD_LDBL_BITS == 128
+flt_binary128 cast_binary64_to_binary128(flt_binary64 d);
+#elif NAN_PAYLOAD_LDBL_BITS == 80
+flt_binary80 cast_binary64_to_binary80(flt_binary64 d);
+#endif /* NAN_PAYLOAD_LDBL_BITS == 128 */
+flt_binary64 cast_binary32_to_binary64(flt_binary32 f);
+#if NAN_PAYLOAD_LDBL_BITS == 128
+flt_binary128 cast_binary32_to_binary128(flt_binary32 f);
+flt_binary32 cast_binary128_to_binary32(flt_binary128 ld);
+flt_binary64 cast_binary128_to_binary64(flt_binary128 ld);
+#elif NAN_PAYLOAD_LDBL_BITS == 80
+flt_binary80 cast_binary32_to_binary80(flt_binary32 f);
+flt_binary32 cast_binary80_to_binary32(flt_binary80 ld);
+flt_binary64 cast_binary80_to_binary64(flt_binary80 ld);
+#endif /* NAN_PAYLOAD_LDBL_BITS == 128 */
+#endif /* prototypes */
+
+
+/*
+ * External API:
+ *
+ * |flt_binary@(32|64|80|128)| to ISO C type mappings
+ */
+#define CAST_FLT2DBL(f) cast_binary32_to_binary64(f)
+#define CAST_DBL2FLT(f) cast_binary64_to_binary32(f)
+#if (NAN_PAYLOAD_LDBL_BITS == 128)
+#define CAST_FLT2LDBL(f) cast_binary32_to_binary128(f)
+#define CAST_DBL2LDBL(f) cast_binary64_to_binary128(f)
+#define CAST_LDBL2FLT(f) cast_binary128_to_binary32(f)
+#define CAST_LDBL2DBL(f) cast_binary128_to_binary64(f)
+#elif (NAN_PAYLOAD_LDBL_BITS == 80)
+#define CAST_FLT2LDBL(f) cast_binary32_to_binary80(f)
+#define CAST_DBL2LDBL(f) cast_binary64_to_binary80(f)
+#define CAST_LDBL2FLT(f) cast_binary80_to_binary32(f)
+#define CAST_LDBL2DBL(f) cast_binary80_to_binary64(f)
+#else
+#error Unsupported NAN_PAYLOAD_LDBL_BITS value
+#endif
+
+/*
+ * get payload
+ */
+#define GET_NAN_PAYLOAD_FLOAT(f) get_nan_payload_binary32(f)
+#define GET_NAN_PAYLOAD_DOUBLE(d) get_nan_payload_binary64(d)
+#if (NAN_PAYLOAD_LDBL_BITS == 128)
+#define GET_NAN_PAYLOAD_LONGDOUBLE(ld) get_nan_payload_binary128(ld)
+#elif (NAN_PAYLOAD_LDBL_BITS == 80)
+#define GET_NAN_PAYLOAD_LONGDOUBLE(ld) get_nan_payload_binary80(ld)
+#endif
+
+/*
+ * set payload
+ */
+#define SET_NAN_PAYLOAD_FLOAT(val, payload) set_nan_payload_binary32((val), (payload))
+#define SET_NAN_PAYLOAD_DOUBLE(val, payload) set_nan_payload_binary64((val), (payload))
+#if (NAN_PAYLOAD_LDBL_BITS == 128)
+#define SET_NAN_PAYLOAD_LONGDOUBLE(val, payload) set_nan_payload_binary128((val), (payload))
+#elif (NAN_PAYLOAD_LDBL_BITS == 80)
+#define SET_NAN_PAYLOAD_LONGDOUBLE(val, payload) set_nan_payload_binary80((val), (payload))
+#endif
+
+#else
+
+#define CAST_FLT2DBL(f) ((double)(f))
+#define CAST_FLT2LDBL(f) ((long double)(f))
+#define CAST_DBL2FLT(f) ((float)(f))
+#define CAST_DBL2LDBL(f) ((long double)(f))
+#define CAST_LDBL2FLT(f) ((float)(f))
+#define CAST_LDBL2DBL(f) ((double)(f))
+
+#endif
+
+/* Special macros for |Sfdouble_t|, which may be |double| or |long double| */
+#if LDBL_MAX_EXP == DBL_MAX_EXP
+#define CAST_SFDBL2FLT(f) CAST_DBL2FLT(f)
+#define CAST_SFDBL2DBL(f) (f)
+#define CAST_FLT2SFDBL(f) CAST_FLT2DBL(f)
+#define CAST_DBL2SFDBL(f) (f)
+#else
+#define CAST_SFDBL2FLT(f) CAST_LDBL2FLT(f)
+#define CAST_SFDBL2DBL(f) CAST_LDBL2DBL(f)
+#define CAST_FLT2SFDBL(f) CAST_FLT2LDBL(f)
+#define CAST_DBL2SFDBL(f) CAST_DBL2LDBL(f)
+#endif
+
#undef extern

/*
diff -r -u original/src/lib/libast/sfio/sfcvt.c build_nanpayload/src/lib/libast/sfio/sfcvt.c
--- src/lib/libast/sfio/sfcvt.c 2012-06-19 09:27:53.000000000 +0200
+++ src/lib/libast/sfio/sfcvt.c 2013-09-23 06:11:21.597362234 +0200
@@ -24,6 +24,519 @@
#endif
#include "sfhdr.h"

+#if FP_SUPPORTS_NAN_PAYLOADS
+
+#include <stdint.h>
+#include <math.h>
+#include <limits.h>
+
+/*
+ * Interfaces for libshell
+ */
+long double ast_nanl(long double payload)
+{
+ long double num=NAN;
+ SET_NAN_PAYLOAD_LONGDOUBLE(&num, (unsigned long)payload);
+ return num;
+}
+double ast_nan(double payload)
+{
+ double num=NAN;
+ SET_NAN_PAYLOAD_DOUBLE(&num, (unsigned long)payload);
+ return num;
+}
+float ast_nanf(float payload)
+{
+ float num=NAN;
+ SET_NAN_PAYLOAD_FLOAT(&num, (unsigned long)payload);
+ return num;
+}
+
+Sfdouble_t nanintpayloadl(long double f)
+{
+ return GET_NAN_PAYLOAD_LONGDOUBLE(f);
+}
+Sfdouble_t nanintpayload(double f)
+{
+ return GET_NAN_PAYLOAD_DOUBLE(f);
+}
+Sfdouble_t nanintpayloadf(float f)
+{
+ return GET_NAN_PAYLOAD_FLOAT(f);
+}
+
+/*
+ * libast interfaces for nan payloads
+ * these should be in src/lib/libast/math/nanpayload.c
+ */
+/*
+ * Notes about portablity:
+ * - The ISO C99 standards supports a portable way to write the
+ * payload of a quiet NaN in ISO C.
+ *
+ * - The |nan()|/|nanf()|/|nanl()| functions (from <math.h>, see
+ * section 7.12.11.2 in the ISO C99 spec) accept strings as
+ * arguments.
+ *
+ * - |strtof()|/|strtod()|/|strtold()| functions (from <stdlib.h>,
+ * see section 7.20.1.3) accept strings in the form
+ * "NAN(character sequence)".
+ *
+ * - |fscanf()|+|sscanf()| follow |strtod()|
+ *
+ * - the character sequence in "NAN(character sequence)" is
+ * interpreted in an (libc+compiler) implementation-specific
+ * way (=not portable).
+ *
+ * - The actual bit layout can differ from the recommended IEEE754-2008
+ * layout. For now we do not care about these special cases except
+ * the 80bit i387 FPU format (mc68881/2 *may* require similar
+ * attention) - see below...
+ *
+ * - The i387 FPU extended precision format defines one extra
+ * magic bit between exponent and fraction.
+ * The layout looks like this:
+ * <sign(1bit)><exponent(15bit)><magic(1bit)><fraction(63bit)>
+ *
+ * - The |long double| code relies on |__int128_t| which is a
+ * gcc+clang extension. It seems |__SIZEOF_INT128__| can be
+ * reliably used to test whether the type is available:
+ * -- snip --
+ * $ gcc -E -dM -x c /dev/null | fgrep 128
+ * #define __DEC128_MAX__ 9.999999999999999999999999999999999E6144DL
+ * #define __DEC128_EPSILON__ 1E-33DL
+ * #define __DEC128_MIN__ 1E-6143DL
+ * #define __DEC128_MIN_EXP__ (-6142)
+ * #define __DEC128_SUBNORMAL_MIN__ 0.000000000000000000000000000000001E-6143DL
+ * #define __FLT_MAX_EXP__ 128
+ * #define __SIZEOF_INT128__ 16
+ * #define __DEC128_MAX_EXP__ 6145
+ * #define __DEC128_MANT_DIG__ 34
+ * $ clang -E -dM -x c /dev/null | fgrep 128
+ * #define __FLT_MAX_EXP__ 128
+ * #define __SIZEOF_INT128__ 16
+ * -- snip --
+ *
+ */
+
+void set_nan_payload_binary32(flt_binary32 *val, uint32_t payload)
+{
+ if (payload == 0)
+ return;
+ if (!isnanf(*val))
+ return;
+
+ volatile uint32_t *bitval_ptr = ((volatile uint32_t *)val);
+
+ const uint32_t exp_bits = (uint32_t)log2f(FLT_MAX_EXP)+1;
+ const uint32_t frac_bits = 32-1-exp_bits;
+ const uint32_t payload_bits = frac_bits-1; /* one bit for quiet/signaling nan */
+ const uint32_t payload_mask = ((uint32_t)powf(2, payload_bits))-1;
+
+ *bitval_ptr &= ~payload_mask;
+ *bitval_ptr |= (payload & payload_mask);
+}
+
+
+uint32_t get_nan_payload_binary32(flt_binary32 x)
+{
+ uint32_t payload;
+
+ if (!isnanf(x))
+ return(0);
+
+ const uint32_t exp_bits = (uint32_t)log2f(FLT_MAX_EXP)+1;
+ const uint32_t frac_bits = 32-1-exp_bits;
+ const uint32_t payload_bits = frac_bits-1; /* one bit for quiet/signaling nan */
+ const uint32_t payload_mask = ((uint32_t)powf(2, payload_bits))-1;
+
+ volatile uint32_t *bitval_ptr = ((volatile uint32_t *)&x);
+
+ payload = *bitval_ptr & payload_mask;
+
+ return(payload);
+}
+
+
+void set_nan_payload_binary64(flt_binary64 *val, uint64_t payload)
+{
+ if (payload == 0)
+ return;
+ if (!isnan(*val))
+ return;
+
+ volatile uint64_t *bitval_ptr = ((volatile uint64_t *)val);
+
+ const uint64_t exp_bits = (uint64_t)log2(DBL_MAX_EXP)+1;
+ const uint64_t frac_bits = 64-1-exp_bits;
+ const uint64_t payload_bits = frac_bits-1; /* one bit for quiet/signaling nan */
+ const uint64_t payload_mask = ((uint64_t)pow(2, payload_bits))-1;
+
+ *bitval_ptr &= ~payload_mask;
+ *bitval_ptr |= (payload & payload_mask);
+}
+
+
+uint64_t get_nan_payload_binary64(flt_binary64 x)
+{
+ uint64_t payload;
+
+ if (!isnan(x))
+ return(0ULL);
+
+ const uint64_t exp_bits = (uint64_t)log2(DBL_MAX_EXP)+1;
+ const uint64_t frac_bits = 64-1-exp_bits;
+ const uint64_t payload_bits = frac_bits-1; /* one bit for quiet/signaling nan */
+ const uint64_t payload_mask = ((uint64_t)pow(2, payload_bits))-1;
+
+ volatile uint64_t *bitval_ptr = ((volatile uint64_t *)&x);
+
+ payload = *bitval_ptr & payload_mask;
+
+ return(payload);
+}
+
+
+#ifdef _typ___int128_t
+#if NAN_PAYLOAD_LDBL_BITS == 128
+void set_nan_payload_binary128(flt_binary128 *val, __int128_t payload)
+#elif NAN_PAYLOAD_LDBL_BITS == 80
+void set_nan_payload_binary80(flt_binary80 *val, uint64_t payload)
+#endif
+{
+ if (payload == 0)
+ return;
+ if (!isnanl(*val))
+ return;
+
+ volatile __int128_t *bitval_ptr = ((volatile __int128_t *)val);
+
+ const __int128_t exp_bits = (__int128_t)log2l(LDBL_MAX_EXP)+1;
+#if (defined(__i386) || defined(__amd64)) && (NAN_PAYLOAD_LDBL_BITS == 80)
+ /*
+ * The i387 FPU extended precision format defines one extra
+ * magic bit between exponent and fraction.
+ * The layout looks like this:
+ * <sign(1bit)><exponent(15bit)><magic(1bit)><fraction(63bit)>
+ */
+ const __int128_t frac_bits = NAN_PAYLOAD_LDBL_BITS-1-1-exp_bits;
+#else
+ const __int128_t frac_bits = NAN_PAYLOAD_LDBL_BITS-1-exp_bits;
+#endif
+ const __int128_t payload_bits = frac_bits-1; /* one bit for quiet/signaling nan */
+ const __int128_t payload_mask = ((__int128_t)powl(2, payload_bits))-1;
+
+ *bitval_ptr &= ~payload_mask;
+ *bitval_ptr |= (payload & payload_mask);
+}
+
+
+#if NAN_PAYLOAD_LDBL_BITS == 128
+__int128_t get_nan_payload_binary128(flt_binary128 x)
+#elif NAN_PAYLOAD_LDBL_BITS == 80
+uint64_t get_nan_payload_binary80(flt_binary80 x)
+#endif
+{
+#if NAN_PAYLOAD_LDBL_BITS == 128
+ __int128_t payload;
+#else
+ uint64_t payload;
+#endif
+ if (!isnanl(x))
+ return(0);
+
+ const __int128_t exp_bits = (__int128_t)log2l(LDBL_MAX_EXP)+1;
+#if (defined(__i386) || defined(__amd64)) && (NAN_PAYLOAD_LDBL_BITS == 80)
+ /*
+ * The i387 FPU extended precision format defines one extra
+ * magic bit between exponent and fraction.
+ * The layout looks like this:
+ * <sign(1bit)><exponent(15bit)><magic(1bit)><fraction(63bit)>
+ */
+ const __int128_t frac_bits = NAN_PAYLOAD_LDBL_BITS-1-1-exp_bits;
+#else
+ const __int128_t frac_bits = NAN_PAYLOAD_LDBL_BITS-1-exp_bits;
+#endif
+ const __int128_t payload_bits = frac_bits-1; /* one bit for quiet/signaling nan */
+ const __int128_t payload_mask = ((__int128_t)powl(2, payload_bits))-1;
+
+ volatile __int128_t *bitval_ptr = ((volatile __int128_t *)&x);
+
+ payload = *bitval_ptr & payload_mask;
+
+ return(payload);
+}
+#endif /* _typ___int128_t */
+
+
+#if (defined(__i386) || defined(__amd64)) && (NAN_PAYLOAD_LDBL_BITS == 80)
+void set_nan_payload_binary80_i387(flt_binary80 *val, uint64_t payload)
+{
+ if (payload == 0)
+ return;
+ if (!isnanl(*val))
+ return;
+
+ const uint64_t frac_bits = 63;
+ const uint64_t payload_bits = frac_bits-1;
+ const uint64_t payload_mask = ((uint64_t)powl(2, payload_bits))-1;
+
+ volatile uint64_t *bitval_ptr = ((volatile uint64_t *)val);
+
+ *bitval_ptr &= ~payload_mask;
+ *bitval_ptr |= (payload & payload_mask);
+}
+
+
+uint64_t get_nan_payload_binary80_i387(flt_binary80 x)
+{
+ uint64_t payload;
+
+ if (!isnanl(x))
+ return(0);
+
+ const uint64_t frac_bits = 63;
+ const uint64_t payload_bits = frac_bits-1;
+ const uint64_t payload_mask = ((uint64_t)powl(2, payload_bits))-1;
+
+ volatile uint64_t *bitval_ptr = ((volatile uint64_t *)&x);
+
+ payload = *bitval_ptr & payload_mask;
+
+ return(payload);
+}
+#endif /* (defined(__i386) || defined(__amd64)) && (NAN_PAYLOAD_LDBL_BITS == 80) */
+
+
+flt_binary32 cast_binary64_to_binary32(flt_binary64 d)
+{
+ flt_binary32 f;
+
+ if (isnan(d))
+ {
+ uint64_t payload;
+
+ payload = get_nan_payload_binary64(d);
+ f = (flt_binary32)d;
+
+ set_nan_payload_binary32(&f, payload);
+ }
+ else
+ {
+ f = (flt_binary32)d;
+ }
+
+ return(f);
+}
+
+#if NAN_PAYLOAD_LDBL_BITS == 128
+flt_binary128 cast_binary64_to_binary128(flt_binary64 d)
+{
+ flt_binary128 ld;
+
+ if (isnan(d))
+ {
+ __int128_t payload;
+
+ payload = get_nan_payload_binary64(d);
+ ld = (flt_binary128)d;
+
+ set_nan_payload_binary128(&ld, payload);
+ }
+ else
+ {
+ ld = (flt_binary128)d;
+ }
+
+ return(ld);
+}
+
+
+#elif NAN_PAYLOAD_LDBL_BITS == 80
+
+
+flt_binary80 cast_binary64_to_binary80(flt_binary64 d)
+{
+ flt_binary80 ld;
+
+ if (isnan(d))
+ {
+ uint64_t payload;
+
+ payload = get_nan_payload_binary64(d);
+ ld = (flt_binary80)d;
+
+ set_nan_payload_binary80(&ld, payload);
+ }
+ else
+ {
+ ld = (flt_binary80)d;
+ }
+
+ return(ld);
+}
+#endif /* NAN_PAYLOAD_LDBL_BITS == 80 */
+
+
+flt_binary64 cast_binary32_to_binary64(flt_binary32 f)
+{
+ flt_binary64 d;
+
+ if (isnanf(f))
+ {
+ uint32_t payload;
+
+ payload = get_nan_payload_binary32(f);
+ d = (flt_binary64)f;
+
+ set_nan_payload_binary64(&d, payload);
+ }
+ else
+ {
+ d = (flt_binary64)f;
+ }
+
+ return(d);
+}
+
+
+#if NAN_PAYLOAD_LDBL_BITS == 128
+flt_binary128 cast_binary32_to_binary128(flt_binary32 f)
+{
+ flt_binary128 ld;
+
+ if (isnanf(f))
+ {
+ uint32_t payload;
+
+ payload = get_nan_payload_binary32(f);
+ ld = (flt_binary128)f;
+
+ set_nan_payload_binary128(&ld, payload);
+ }
+ else
+ {
+ ld = (flt_binary128)f;
+ }
+
+ return(ld);
+}
+
+
+flt_binary32 cast_binary128_to_binary32(flt_binary128 ld)
+{
+ flt_binary32 f;
+
+ if (isnanl(ld))
+ {
+ __int128_t payload;
+
+ payload = get_nan_payload_binary128(ld);
+ f = (flt_binary32)ld;
+
+ set_nan_payload_binary32(&f, payload);
+ }
+ else
+ {
+ f = (flt_binary32)ld;
+ }
+
+ return(f);
+}
+
+
+flt_binary64 cast_binary128_to_binary64(flt_binary128 ld)
+{
+ flt_binary64 d;
+
+ if (isnanl(ld))
+ {
+ __int128_t payload;
+
+ payload = get_nan_payload_binary128(ld);
+ d = (flt_binary64)ld;
+
+ set_nan_payload_binary64(&d, payload);
+ }
+ else
+ {
+ d = (flt_binary64)ld;
+ }
+
+ return(d);
+}
+
+
+#elif NAN_PAYLOAD_LDBL_BITS == 80
+
+
+flt_binary80 cast_binary32_to_binary80(flt_binary32 f)
+{
+ flt_binary80 ld;
+
+ if (isnanf(f))
+ {
+ uint32_t payload;
+
+ payload = get_nan_payload_binary32(f);
+ ld = (flt_binary80)f;
+
+ set_nan_payload_binary80(&ld, payload);
+ }
+ else
+ {
+ ld = (flt_binary80)f;
+ }
+
+ return(ld);
+}
+
+
+flt_binary32 cast_binary80_to_binary32(flt_binary80 ld)
+{
+ flt_binary32 f;
+
+ if (isnanl(ld))
+ {
+ uint64_t payload;
+
+ payload = get_nan_payload_binary80(ld);
+ f = (flt_binary32)ld;
+
+ set_nan_payload_binary32(&f, payload);
+ }
+ else
+ {
+ f = (flt_binary32)ld;
+ }
+
+ return(f);
+}
+
+
+flt_binary64 cast_binary80_to_binary64(flt_binary80 ld)
+{
+ flt_binary64 d;
+
+ if (isnanl(ld))
+ {
+ uint64_t payload;
+
+ payload = get_nan_payload_binary80(ld);
+ d = (flt_binary64)ld;
+
+ set_nan_payload_binary64(&d, payload);
+ }
+ else
+ {
+ d = (flt_binary64)ld;
+ }
+
+ return(d);
+}
+#endif /* NAN_PAYLOAD_LDBL_BITS == 80 */
+#endif /* FP_SUPPORTS_NAN_PAYLOADS */
+
/* Convert a floating point value to ASCII.
**
** Written by Kiem-Phong Vo and Glenn Fowler (SFFMT_AFORMAT)
@@ -118,8 +631,8 @@
int x;
_ast_flt_unsigned_max_t m;

- static char lx[] = "0123456789abcdef";
- static char ux[] = "0123456789ABCDEF";
+ static const char lx[] = "0123456789abcdef";
+ static const char ux[] = "0123456789ABCDEF";

*sign = *decpt = 0;

@@ -128,14 +641,26 @@
{ Sfdouble_t f = *(Sfdouble_t*)vp;

if(isnanl(f))
- {
+ {
+ uint64_t payload; /* fixme: not big enough for 128bit float */
#if _lib_signbit
if (signbit(f))
#else
if (f < 0)
#endif
*sign = 1;
- return SF_NAN;
+#if FP_SUPPORTS_NAN_PAYLOADS
+ if (payload = GET_NAN_PAYLOAD_LONGDOUBLE(f))
+ {
+ snprintf(buf, size, "ast_nan(0x%lx)", (unsigned long)payload);
+ _Sfi = strlen(buf);
+ if (len)
+ *len = _Sfi;
+ return buf;
+ }
+ else
+#endif /* FP_SUPPORTS_NAN_PAYLOADS */
+ return SF_NAN;
}
#if _lib_isinf
if (n = isinf(f))
@@ -172,7 +697,7 @@
case FP_INFINITE:
return SF_INF;
case FP_NAN:
- return SF_NAN;
+ return SF_NAN; /* isn't this redundant since we handle nan already above ? */
case FP_ZERO:
return SF_ZERO;
}
@@ -306,14 +831,26 @@
{ double f = *(double*)vp;

if(isnan(f))
- {
+ {
+ uint32_t payload;
#if _lib_signbit
if (signbit(f))
#else
if (f < 0)
#endif
*sign = 1;
- return SF_NAN;
+#if FP_SUPPORTS_NAN_PAYLOADS
+ if (payload = GET_NAN_PAYLOAD_DOUBLE(f))
+ {
+ snprintf(buf, size, "ast_nan(0x%lx)", (unsigned long)payload);
+ _Sfi = strlen(buf);
+ if (len)
+ *len = _Sfi;
+ return buf;
+ }
+ else
+#endif /* FP_SUPPORTS_NAN_PAYLOADS */
+ return SF_NAN;
}
#if _lib_isinf
if (n = isinf(f))
@@ -530,3 +1067,4 @@
ep = sp + 1;
goto done;
}
+
diff -r -u original/src/lib/libast/sfio/sfstrtof.h build_nanpayload/src/lib/libast/sfio/sfstrtof.h
--- src/lib/libast/sfio/sfstrtof.h 2010-12-24 15:32:27.000000000 +0100
+++ src/lib/libast/sfio/sfstrtof.h 2013-09-23 07:15:46.861411025 +0200
@@ -72,7 +72,17 @@
#define S2F_ldexp ldexp
#define S2F_pow10 _Sffpow10
#define S2F_inf _Sffinf
+#if FP_SUPPORTS_NAN_PAYLOADS
+/*
+ * fixme: On Solaris |_Sffnan| has a payload value with all
+ * bits set while |NAN| as no bit set. We use |ast_nanf(0)|
+ * as workaround for now, but unfortunately this causes a
+ * sign reversal for yet unknown reasons
+ */
+#define S2F_nan ast_nanf(0)
+#else
#define S2F_nan _Sffnan
+#endif
#define S2F_min (FLT_MIN)
#define S2F_max (FLT_MAX)
#define S2F_exp_10_min (FLT_MIN_10_EXP)
@@ -85,7 +95,17 @@
#define S2F_ldexp ldexp
#define S2F_pow10 _Sfdpow10
#define S2F_inf _Sfdinf
+#if FP_SUPPORTS_NAN_PAYLOADS
+/*
+ * fixme: On Solaris |_Sfdnan| has a payload value with all
+ * bits set while |NAN| as no bit set. We use |ast_nan(0)|
+ * as workaround for now, but unfortunately this causes a
+ * sign reversal for yet unknown reasons
+ */
+#define S2F_nan ast_nan(0)
+#else
#define S2F_nan _Sfdnan
+#endif
#define S2F_min (DBL_MIN)
#define S2F_max (DBL_MAX)
#define S2F_exp_10_min (DBL_MIN_10_EXP)
@@ -98,7 +118,17 @@
#define S2F_ldexp ldexpl
#define S2F_pow10 _Sflpow10
#define S2F_inf _Sflinf
+#if FP_SUPPORTS_NAN_PAYLOADS
+/*
+ * fixme: On Solaris |_Sflnan| has a payload value with all
+ * bits set while |NAN| as no bit set. We use |ast_nanl(0)|
+ * as workaround for now, but unfortunately this causes a
+ * sign reversal for yet unknown reasons
+ */
+#define S2F_nan ast_nanl(0)
+#else
#define S2F_nan _Sflnan
+#endif
#define S2F_min (LDBL_MIN)
#define S2F_max (LDBL_MAX)
#define S2F_exp_10_min (LDBL_MIN_10_EXP)
@@ -364,18 +394,75 @@
PUT(s);
return negative ? -S2F_inf : S2F_inf;
}
- else if (c == 'n' || c == 'N')
+ else if (c == 'n' || c == 'N'
+#if FP_SUPPORTS_NAN_PAYLOADS
+
+ || c == 'a' || c == 'A'
+#endif
+ ) /* "nan"/"ast_nan" */
{
- if ((c = GET(s)) != 'a' && c != 'A' ||
- (c = GET(s)) != 'n' && c != 'N')
+ char buff[64], *bx;
+
+#if FP_SUPPORTS_NAN_PAYLOADS
+ if (c == 'a' || c == 'A')
+ {
+ if ( (c = GET(s)) != 's' && c != 'S' ||
+ (c = GET(s)) != 't' && c != 't' ||
+ (c = GET(s)) != '_' && c != '_' ||
+ (c = GET(s)) != 'n' && c != 'N' ||
+ (c = GET(s)) != 'a' && c != 'A' ||
+ (c = GET(s)) != 'n' && c != 'N')
+ {
+ REV(s, t, b);
+ PUT(s);
+ return 0;
+ }
+ }
+ else
+#endif
+ {
+ if ((c = GET(s)) != 'a' && c != 'A' ||
+ (c = GET(s)) != 'n' && c != 'N')
+ {
+ REV(s, t, b);
+ PUT(s);
+ return 0;
+ }
+ }
+
+ bx = buff;
+ do
+ {
+ c = GET(s);
+ *bx++ = c;
+ }
+ while (c && !isspace(c) && ((bx - buff) < (sizeof(buff)-2)));
+ *bx = '\0';
+#if FP_SUPPORTS_NAN_PAYLOADS
+ if (buff[0] == '(')
+ {
+ unsigned long long payload;
+
+ payload = atoll(&buff[1]);
+
+ {
+#if S2F_type == 0
+ float v = ast_nanf(payload);
+#elif S2F_type == 1
+ double v = ast_nan(payload);
+#elif S2F_type == 2
+ long double v = ast_nanl(payload);
+#endif
+ PUT(s);
+ return negative ? -v : v;
+ }
+ }
+ else
+#endif
{
- REV(s, t, b);
PUT(s);
- return 0;
+ return negative ? -S2F_nan : S2F_nan;
}
- do c = GET(s); while (c && !isspace(c));
- PUT(s);
- return negative ? -S2F_nan : S2F_nan;
}
else if (c < '1' || c > '9')
{
diff -r -u original/src/lib/libast/sfio/sfvprintf.c build_nanpayload/src/lib/libast/sfio/sfvprintf.c
--- src/lib/libast/sfio/sfvprintf.c 2012-06-27 19:06:45.000000000 +0200
+++ src/lib/libast/sfio/sfvprintf.c 2013-09-23 06:11:21.598770660 +0200
@@ -1202,7 +1202,12 @@
sp = endsp = buf+1; /* reserve space for sign */
*endsp++ = '0';
*endsp++ = fmt == 'a' ? 'x' : 'X';
- if (!isxdigit(*ep))
+ if (!isxdigit(*ep)
+#if FP_SUPPORTS_NAN_PAYLOADS
+ /* test for |ast_nan()| */
+ || ((n_s > 7) && ((!strncmp(ep, "ast_nan", 7)) || (!strncmp(ep, "AST_NAN", 7))))
+#endif
+ )
goto infinite;
if (base < 0)
base = 0;
@@ -1217,7 +1222,7 @@
&decpt, &sign, &n_s, v);
if(dval == 0.)
decpt = 1;
- else if(*ep == 'I')
+ else if(*ep == 'I') /* q. for gsf: Why only uppercase 'I' ? */
goto infinite;

if(!(flags&SFFMT_ALTER))
-------------- next part --------------
/***********************************************************************
* *
* This software is part of the ast package *
* Copyright (c) 2013 AT&T Intellectual Property *
* and is licensed under the *
* Common Public License, Version 1.0 *
* by AT&T Intellectual Property *
* *
* A copy of the License is available at *
* http://www.opensource.org/licenses/cpl1.0.txt *
* (with md5 checksum 059e8cd6165cb4c31e351f2b69388fd9) *
* *
* Information and Software Systems Research *
* AT&T Research *
* Florham Park NJ *
* *
* Roland Mainz <roland.mainz at nrubsig.org> *
* *
***********************************************************************/

#include <stdlib.h>
#include <stdint.h>
#include <stdio.h>
#include <math.h>
#include <float.h>
#include <limits.h>

/* use |__float128| on gcc/x86 for testing */
#if defined(__amd64)
//#define GCC__FLOAT128 1
#endif

#ifdef GCC__FLOAT128
#define NAN_PAYLOAD_LDBL_BITS 128
#else
#define NAN_PAYLOAD_LDBL_BITS 80
#endif

#ifndef _typ___int128_t
#ifdef __SIZEOF_INT128__
#define _typ___int128_t 1
#endif
#endif

#ifdef GCC__FLOAT128
#define FLT1282LDBL(flt128) ((long double)(flt128))
#else
#define FLT1282LDBL(flt128) (flt128)
#endif

/* mapping C types to aliases which define their physical layout */
typedef float flt_binary32;
typedef double flt_binary64;
#if defined(__i386) || defined(__amd64)
#if (NAN_PAYLOAD_LDBL_BITS == 128)
typedef __float128 flt_binary128;
#else
typedef long double flt_binary80;
#endif
#else
#if (NAN_PAYLOAD_LDBL_BITS == 128)
typedef long double flt_binary128;
#elif (NAN_PAYLOAD_LDBL_BITS == 64)
#error |long double| == |double| not implemented yet
#else
#error Unsupported NAN_PAYLOAD_LDBL_BITS
#endif
#endif



#if 1 /* prototypes */
void set_nan_payload_binary32(flt_binary32 *val, uint32_t payload);
uint32_t get_nan_payload_binary32(flt_binary32 x);
void set_nan_payload_binary64(flt_binary64 *val, uint64_t payload);
uint64_t get_nan_payload_binary64(flt_binary64 x);
#if NAN_PAYLOAD_LDBL_BITS == 128
void set_nan_payload_binary128(flt_binary128 *val, __int128_t payload);
__int128_t get_nan_payload_binary128(flt_binary128 x);
#elif NAN_PAYLOAD_LDBL_BITS == 80
#ifdef _typ___int128_t
void set_nan_payload_binary80(flt_binary80 *val, uint64_t payload);
uint64_t get_nan_payload_binary80(flt_binary80 x);
#else
#if (defined(__i386) || defined(__amd64)) && (NAN_PAYLOAD_LDBL_BITS == 80)
#define set_nan_payload_binary80(v, p) set_nan_payload_binary80_i387((v), (p))
#define get_nan_payload_binary80(v) get_nan_payload_binary80_i387(v)
#endif
#endif /* _typ___int128_t */
#if (defined(__i386) || defined(__amd64)) && (NAN_PAYLOAD_LDBL_BITS == 80)
void set_nan_payload_binary80_i387(flt_binary80 *val, uint64_t payload);
uint64_t get_nan_payload_binary80_i387(flt_binary80 x);
#endif /* (defined(__i386) || defined(__amd64)) && (NAN_PAYLOAD_LDBL_BITS == 80) */
#endif

flt_binary32 cast_binary64_to_binary32(flt_binary64 d);
#if NAN_PAYLOAD_LDBL_BITS == 128
flt_binary128 cast_binary64_to_binary128(flt_binary64 d);
#elif NAN_PAYLOAD_LDBL_BITS == 80
flt_binary80 cast_binary64_to_binary80(flt_binary64 d);
#endif /* NAN_PAYLOAD_LDBL_BITS == 128 */
flt_binary64 cast_binary32_to_binary64(flt_binary32 f);
#if NAN_PAYLOAD_LDBL_BITS == 128
flt_binary128 cast_binary32_to_binary128(flt_binary32 f);
flt_binary32 cast_binary128_to_binary32(flt_binary128 ld);
flt_binary64 cast_binary128_to_binary64(flt_binary128 ld);
#elif NAN_PAYLOAD_LDBL_BITS == 80
flt_binary80 cast_binary32_to_binary80(flt_binary32 f);
flt_binary32 cast_binary80_to_binary32(flt_binary80 ld);
flt_binary64 cast_binary80_to_binary64(flt_binary80 ld);
#endif /* NAN_PAYLOAD_LDBL_BITS == 128 */
#endif /* prototypes */


/*
* External API:
*
* |flt_binary@(32|64|80|128)| to ISO C type mappings
*/
#define CAST_FLT2DBL(f) cast_binary32_to_binary64(f)
#define CAST_DBL2FLT(f) cast_binary64_to_binary32(f)
#if (NAN_PAYLOAD_LDBL_BITS == 128)
#define CAST_FLT2LDBL(f) cast_binary32_to_binary128(f)
#define CAST_DBL2LDBL(f) cast_binary64_to_binary128(f)
#define CAST_LDBL2FLT(f) cast_binary128_to_binary32(f)
#define CAST_LDBL2DBL(f) cast_binary128_to_binary64(f)
#elif (NAN_PAYLOAD_LDBL_BITS == 80)
#define CAST_FLT2LDBL(f) cast_binary32_to_binary80(f)
#define CAST_DBL2LDBL(f) cast_binary64_to_binary80(f)
#define CAST_LDBL2FLT(f) cast_binary80_to_binary32(f)
#define CAST_LDBL2DBL(f) cast_binary80_to_binary64(f)
#else
#error Unsupported NAN_PAYLOAD_LDBL_BITS value
#endif

/*
* get payload
*/
#define GET_NAN_PAYLOAD_FLOAT(f) get_nan_payload_binary32(f)
#define GET_NAN_PAYLOAD_DOUBLE(d) get_nan_payload_binary64(d)
#if (NAN_PAYLOAD_LDBL_BITS == 128)
#define GET_NAN_PAYLOAD_LONGDOUBLE(ld) get_nan_payload_binary128(ld)
#elif (NAN_PAYLOAD_LDBL_BITS == 80)
#define GET_NAN_PAYLOAD_LONGDOUBLE(ld) get_nan_payload_binary80(ld)
#endif

/*
* set payload
*/
#define SET_NAN_PAYLOAD_FLOAT(val, payload) set_nan_payload_binary32((val), (payload))
#define SET_NAN_PAYLOAD_DOUBLE(val, payload) set_nan_payload_binary64((val), (payload))
#if (NAN_PAYLOAD_LDBL_BITS == 128)
#define SET_NAN_PAYLOAD_LONGDOUBLE(val, payload) set_nan_payload_binary128((val), (payload))
#elif (NAN_PAYLOAD_LDBL_BITS == 80)
#define SET_NAN_PAYLOAD_LONGDOUBLE(val, payload) set_nan_payload_binary80((val), (payload))
#endif



/*
* Notes about portablity:
* - The ISO C99 standards supports a portable way to write the
* payload of a quiet NaN in ISO C.
*
* - The |nan()|/|nanf()|/|nanl()| functions (from <math.h>, see
* section 7.12.11.2 in the ISO C99 spec) accept strings as
* arguments.
*
* - |strtof()|/|strtod()|/|strtold()| functions (from <stdlib.h>,
* see section 7.20.1.3) accept strings in the form
* "NAN(character sequence)".
*
* - |fscanf()|+|sscanf()| follow |strtod()|
*
* - the character sequence in "NAN(character sequence)" is
* interpreted in an (libc+compiler) implementation-specific
* way (=not portable).
*
* - The actual bit layout can differ from the recommended IEEE754-2008
* layout. For now we do not care about these special cases except
* the 80bit i387 FPU format (mc68881/2 *may* require similar
* attention) - see below...
*
* - The i387 FPU extended precision format defines one extra
* magic bit between exponent and fraction.
* The layout looks like this:
* <sign(1bit)><exponent(15bit)><magic(1bit)><fraction(63bit)>
*
* - The |long double| code relies on |__int128_t| which is a
* gcc+clang extension. It seems |__SIZEOF_INT128__| can be
* reliably used to test whether the type is available:
* -- snip --
* $ gcc -E -dM -x c /dev/null | fgrep 128
* #define __DEC128_MAX__ 9.999999999999999999999999999999999E6144DL
* #define __DEC128_EPSILON__ 1E-33DL
* #define __DEC128_MIN__ 1E-6143DL
* #define __DEC128_MIN_EXP__ (-6142)
* #define __DEC128_SUBNORMAL_MIN__ 0.000000000000000000000000000000001E-6143DL
* #define __FLT_MAX_EXP__ 128
* #define __SIZEOF_INT128__ 16
* #define __DEC128_MAX_EXP__ 6145
* #define __DEC128_MANT_DIG__ 34
* $ clang -E -dM -x c /dev/null | fgrep 128
* #define __FLT_MAX_EXP__ 128
* #define __SIZEOF_INT128__ 16
* -- snip --
*
*/

void set_nan_payload_binary32(flt_binary32 *val, uint32_t payload)
{
if (payload == 0)
return;
if (!isnanf(*val))
return;

volatile uint32_t *bitval_ptr = ((volatile uint32_t *)val);

const uint32_t exp_bits = (uint32_t)log2f(FLT_MAX_EXP)+1;
const uint32_t frac_bits = 32-1-exp_bits;
const uint32_t payload_bits = frac_bits-1; /* one bit for quiet/signaling nan */
const uint32_t payload_mask = ((uint32_t)powf(2, payload_bits))-1;

*bitval_ptr &= ~payload_mask;
*bitval_ptr |= (payload & payload_mask);
}


uint32_t get_nan_payload_binary32(flt_binary32 x)
{
uint32_t payload;

if (!isnanf(x))
return(0);

const uint32_t exp_bits = (uint32_t)log2f(FLT_MAX_EXP)+1;
const uint32_t frac_bits = 32-1-exp_bits;
const uint32_t payload_bits = frac_bits-1; /* one bit for quiet/signaling nan */
const uint32_t payload_mask = ((uint32_t)powf(2, payload_bits))-1;

volatile uint32_t *bitval_ptr = ((volatile uint32_t *)&x);

payload = *bitval_ptr & payload_mask;

return(payload);
}


void set_nan_payload_binary64(flt_binary64 *val, uint64_t payload)
{
if (payload == 0)
return;
if (!isnan(*val))
return;

volatile uint64_t *bitval_ptr = ((volatile uint64_t *)val);

const uint64_t exp_bits = (uint64_t)log2(DBL_MAX_EXP)+1;
const uint64_t frac_bits = 64-1-exp_bits;
const uint64_t payload_bits = frac_bits-1; /* one bit for quiet/signaling nan */
const uint64_t payload_mask = ((uint64_t)pow(2, payload_bits))-1;

*bitval_ptr &= ~payload_mask;
*bitval_ptr |= (payload & payload_mask);
}


uint64_t get_nan_payload_binary64(flt_binary64 x)
{
uint64_t payload;

if (!isnan(x))
return(0ULL);

const uint64_t exp_bits = (uint64_t)log2(DBL_MAX_EXP)+1;
const uint64_t frac_bits = 64-1-exp_bits;
const uint64_t payload_bits = frac_bits-1; /* one bit for quiet/signaling nan */
const uint64_t payload_mask = ((uint64_t)pow(2, payload_bits))-1;

volatile uint64_t *bitval_ptr = ((volatile uint64_t *)&x);

payload = *bitval_ptr & payload_mask;

return(payload);
}


#ifdef _typ___int128_t
#if NAN_PAYLOAD_LDBL_BITS == 128
void set_nan_payload_binary128(flt_binary128 *val, __int128_t payload)
#elif NAN_PAYLOAD_LDBL_BITS == 80
void set_nan_payload_binary80(flt_binary80 *val, uint64_t payload)
#endif
{
if (payload == 0)
return;
if (!isnanl(*val))
return;

volatile __int128_t *bitval_ptr = ((volatile __int128_t *)val);

const __int128_t exp_bits = (__int128_t)log2l(LDBL_MAX_EXP)+1;
#if (defined(__i386) || defined(__amd64)) && (NAN_PAYLOAD_LDBL_BITS == 80)
/*
* The i387 FPU extended precision format defines one extra
* magic bit between exponent and fraction.
* The layout looks like this:
* <sign(1bit)><exponent(15bit)><magic(1bit)><fraction(63bit)>
*/
const __int128_t frac_bits = NAN_PAYLOAD_LDBL_BITS-1-1-exp_bits;
#else
const __int128_t frac_bits = NAN_PAYLOAD_LDBL_BITS-1-exp_bits;
#endif
const __int128_t payload_bits = frac_bits-1; /* one bit for quiet/signaling nan */
const __int128_t payload_mask = ((__int128_t)powl(2, payload_bits))-1;

*bitval_ptr &= ~payload_mask;
*bitval_ptr |= (payload & payload_mask);
}


#if NAN_PAYLOAD_LDBL_BITS == 128
__int128_t get_nan_payload_binary128(flt_binary128 x)
#elif NAN_PAYLOAD_LDBL_BITS == 80
uint64_t get_nan_payload_binary80(flt_binary80 x)
#endif
{
#if NAN_PAYLOAD_LDBL_BITS == 128
__int128_t payload;
#else
uint64_t payload;
#endif
if (!isnanl(x))
return(0);

const __int128_t exp_bits = (__int128_t)log2l(LDBL_MAX_EXP)+1;
#if (defined(__i386) || defined(__amd64)) && (NAN_PAYLOAD_LDBL_BITS == 80)
/*
* The i387 FPU extended precision format defines one extra
* magic bit between exponent and fraction.
* The layout looks like this:
* <sign(1bit)><exponent(15bit)><magic(1bit)><fraction(63bit)>
*/
const __int128_t frac_bits = NAN_PAYLOAD_LDBL_BITS-1-1-exp_bits;
#else
const __int128_t frac_bits = NAN_PAYLOAD_LDBL_BITS-1-exp_bits;
#endif
const __int128_t payload_bits = frac_bits-1; /* one bit for quiet/signaling nan */
const __int128_t payload_mask = ((__int128_t)powl(2, payload_bits))-1;

volatile __int128_t *bitval_ptr = ((volatile __int128_t *)&x);

payload = *bitval_ptr & payload_mask;

return(payload);
}
#endif /* _typ___int128_t */


#if (defined(__i386) || defined(__amd64)) && (NAN_PAYLOAD_LDBL_BITS == 80)
void set_nan_payload_binary80_i387(flt_binary80 *val, uint64_t payload)
{
if (payload == 0)
return;
if (!isnanl(*val))
return;

const uint64_t frac_bits = 63;
const uint64_t payload_bits = frac_bits-1;
const uint64_t payload_mask = ((uint64_t)powl(2, payload_bits))-1;

volatile uint64_t *bitval_ptr = ((volatile uint64_t *)val);

*bitval_ptr &= ~payload_mask;
*bitval_ptr |= (payload & payload_mask);
}


uint64_t get_nan_payload_binary80_i387(flt_binary80 x)
{
uint64_t payload;

if (!isnanl(x))
return(0);

const uint64_t frac_bits = 63;
const uint64_t payload_bits = frac_bits-1;
const uint64_t payload_mask = ((uint64_t)powl(2, payload_bits))-1;

volatile uint64_t *bitval_ptr = ((volatile uint64_t *)&x);

payload = *bitval_ptr & payload_mask;

return(payload);
}
#endif /* (defined(__i386) || defined(__amd64)) && (NAN_PAYLOAD_LDBL_BITS == 80) */


flt_binary32 cast_binary64_to_binary32(flt_binary64 d)
{
flt_binary32 f;

if (isnan(d))
{
uint64_t payload;

payload = get_nan_payload_binary64(d);
f = (flt_binary32)d;

set_nan_payload_binary32(&f, payload);
}
else
{
f = (flt_binary32)d;
}

return(f);
}

#if NAN_PAYLOAD_LDBL_BITS == 128
flt_binary128 cast_binary64_to_binary128(flt_binary64 d)
{
flt_binary128 ld;

if (isnan(d))
{
__int128_t payload;

payload = get_nan_payload_binary64(d);
ld = (flt_binary128)d;

set_nan_payload_binary128(&ld, payload);
}
else
{
ld = (flt_binary128)d;
}

return(ld);
}


#elif NAN_PAYLOAD_LDBL_BITS == 80


flt_binary80 cast_binary64_to_binary80(flt_binary64 d)
{
flt_binary80 ld;

if (isnan(d))
{
uint64_t payload;

payload = get_nan_payload_binary64(d);
ld = (flt_binary80)d;

set_nan_payload_binary80(&ld, payload);
}
else
{
ld = (flt_binary80)d;
}

return(ld);
}
#endif /* NAN_PAYLOAD_LDBL_BITS == 80 */


flt_binary64 cast_binary32_to_binary64(flt_binary32 f)
{
flt_binary64 d;

if (isnanf(f))
{
uint32_t payload;

payload = get_nan_payload_binary32(f);
d = (flt_binary64)f;

set_nan_payload_binary64(&d, payload);
}
else
{
d = (flt_binary64)f;
}

return(d);
}


#if NAN_PAYLOAD_LDBL_BITS == 128
flt_binary128 cast_binary32_to_binary128(flt_binary32 f)
{
flt_binary128 ld;

if (isnanf(f))
{
uint32_t payload;

payload = get_nan_payload_binary32(f);
ld = (flt_binary128)f;

set_nan_payload_binary128(&ld, payload);
}
else
{
ld = (flt_binary128)f;
}

return(ld);
}


flt_binary32 cast_binary128_to_binary32(flt_binary128 ld)
{
flt_binary32 f;

if (isnanl(ld))
{
__int128_t payload;

payload = get_nan_payload_binary128(ld);
f = (flt_binary32)ld;

set_nan_payload_binary32(&f, payload);
}
else
{
f = (flt_binary32)ld;
}

return(f);
}


flt_binary64 cast_binary128_to_binary64(flt_binary128 ld)
{
flt_binary64 d;

if (isnanl(ld))
{
__int128_t payload;

payload = get_nan_payload_binary128(ld);
d = (flt_binary64)ld;

set_nan_payload_binary64(&d, payload);
}
else
{
d = (flt_binary64)ld;
}

return(d);
}


#elif NAN_PAYLOAD_LDBL_BITS == 80


flt_binary80 cast_binary32_to_binary80(flt_binary32 f)
{
flt_binary80 ld;

if (isnanf(f))
{
uint32_t payload;

payload = get_nan_payload_binary32(f);
ld = (flt_binary80)f;

set_nan_payload_binary80(&ld, payload);
}
else
{
ld = (flt_binary80)f;
}

return(ld);
}


flt_binary32 cast_binary80_to_binary32(flt_binary80 ld)
{
flt_binary32 f;

if (isnanl(ld))
{
uint64_t payload;

payload = get_nan_payload_binary80(ld);
f = (flt_binary32)ld;

set_nan_payload_binary32(&f, payload);
}
else
{
f = (flt_binary32)ld;
}

return(f);
}


flt_binary64 cast_binary80_to_binary64(flt_binary80 ld)
{
flt_binary64 d;

if (isnanl(ld))
{
uint64_t payload;

payload = get_nan_payload_binary80(ld);
d = (flt_binary64)ld;

set_nan_payload_binary64(&d, payload);
}
else
{
d = (flt_binary64)ld;
}

return(d);
}
#endif /* NAN_PAYLOAD_LDBL_BITS == 80 */



#define TESTCODE
#ifdef TESTCODE
void print_bits_32(uint32_t x)
{
int i;
for (i=0 ; i < 32 ; i++)
{
/* fixme: endian! */
(void)printf("%d", (x & (1UL<<31))?1:0);
x = x << 1;
}
}


void print_bits_64(uint64_t x)
{
int i;
for (i=0 ; i < 64 ; i++)
{
/* fixme: endian! */
(void)printf("%d", (x & (1ULL<<63))?1:0);
x = x << 1;
}
}


#ifdef _typ___int128_t
void print_bits_128(__int128_t x)
{
int i;
for (i=0 ; i < 128 ; i++)
{
/* fixme: endian! */
(void)printf("%d", (x & (((__int128_t)1)<<127))?1:0);
x = x << 1;
}
}
#endif /* _typ___int128_t */


int main(int ac, char *av[])
{
volatile flt_binary32 val = NAN;
long i;
volatile uint32_t *xp=(volatile uint32_t *)&val;

(void)printf("orig:\t" ); print_bits_32(*xp); (void)printf("\tval=%f\n", val);

const uint32_t exp_bits = ((uint32_t)log2(FLT_MAX_EXP))+1;
const uint32_t frac_bits = 32-1-exp_bits;
const uint32_t payload_bits = frac_bits-1;
const uint32_t frac_mask = ((uint32_t)pow(2, frac_bits))-1; /* upper bit is for quiet/signaning nans */
const uint32_t payload_mask = ((uint32_t)pow(2, payload_bits))-1; /* upper bit is for quiet/signaning nans */

(void)printf("exp_bits=%d, frac_bits=%d, frac_mask=%lx, payload_bits=%d, payload_mask=%lx\n",
exp_bits,
frac_bits,
(unsigned long)frac_mask,
payload_bits,
(unsigned long)payload_mask);

(void)printf("frcmsk:\t" ); print_bits_32(frac_mask); (void)printf("\n");
(void)printf("paymsk:\t" ); print_bits_32(payload_mask);(void)printf("\n");

val = +NAN;
*xp |= 0x01;
(void)printf("+NAN/0x01:\t" ); print_bits_32(*xp); (void)printf("\tval=%f\n", val);

val = -NAN;
*xp |= 0x01;
(void)printf("-NAN/0x01:\t" ); print_bits_32(*xp); (void)printf("\tval=%f\n", val);

/*
* basic test
* IEEE 764-2008 floating-point types must have a non-zero
* payload, otherwise they turn into an |inf|
*/
for (i=0 ; i <= frac_mask ; i++)
{
volatile uint32_t *bitval_ptr = ((volatile uint32_t *)&val);

val = NAN;
*bitval_ptr &= ~payload_mask;
*bitval_ptr |= (i & payload_mask);
if (!isnan(val))
(void)printf("FAIL: %f not a nan for %ld\n", val, (long)i);
if (isinf(val))
(void)printf("FAIL: %f is a inf for %ld\n", val, (long)i);
}

/* tests for type casting */
{
flt_binary32 fval;
fval = -NAN;
SET_NAN_PAYLOAD_FLOAT(&fval, 667);
(void)printf("flt: val with -NAN667=%f\n", fval);
(void)printf("flt: payload of -NAN667=%d\n", (int)GET_NAN_PAYLOAD_FLOAT(fval));
}

{
flt_binary64 dval;
dval = -NAN;
SET_NAN_PAYLOAD_DOUBLE(&dval, 667);
(void)printf("dbl: val with -NAN667=%f\n", dval);
(void)printf("dbl: payload of -NAN667=%lu\n", (unsigned long)GET_NAN_PAYLOAD_DOUBLE(dval));
(void)printf("dbl: payload of sys_cast float(-NAN667)=%lu\n", (unsigned long)GET_NAN_PAYLOAD_FLOAT((flt_binary32)dval));
(void)printf("dbl: payload of own_cast float(-NAN667)=%lu\n", (unsigned long)GET_NAN_PAYLOAD_FLOAT(CAST_DBL2FLT(dval)));
}

{
#if GCC__FLOAT128
__float128 ldval;
#else
long double ldval;
#endif

ldval = -NAN;
SET_NAN_PAYLOAD_LONGDOUBLE(&ldval, 667);
(void)printf("ldbl: val with -NAN667=%Lf\n", FLT1282LDBL(ldval));
(void)printf("ldbl: payload of -NAN667=%lu\n", (unsigned long)GET_NAN_PAYLOAD_LONGDOUBLE(ldval));
(void)printf("ldbl: payload of sys_cast double(-NAN667)=%lu\n", (unsigned long)GET_NAN_PAYLOAD_DOUBLE((flt_binary64)ldval));
(void)printf("ldbl: payload of own_cast double(-NAN667)=%lu\n", (unsigned long)GET_NAN_PAYLOAD_DOUBLE(CAST_LDBL2DBL(ldval)));
}

#if (defined(__i386) || defined(__amd64)) && (NAN_PAYLOAD_LDBL_BITS == 80)
/* special version of i387 */
{
flt_binary80 ldval;
ldval = -NAN;
set_nan_payload_binary80_i387(&ldval, 667);
(void)printf("ldbl(i387): val with -NAN667=%Lf\n", ldval);
(void)printf("ldbl(i387): payload of -NAN667=%lu\n", (unsigned long)get_nan_payload_binary80(ldval));
(void)printf("ldbl(i387): payload of -NAN667=%lu (i387)\n", (unsigned long)get_nan_payload_binary80_i387(ldval));
(void)printf("ldbl(i387): payload of sys_cast double(-NAN667)=%lu\n", (unsigned long)GET_NAN_PAYLOAD_DOUBLE((flt_binary64)ldval));
(void)printf("ldbl(i387): payload of own_cast double(-NAN667)=%lu\n", (unsigned long)GET_NAN_PAYLOAD_DOUBLE(cast_binary80_to_binary64(ldval)));
}
#endif /* (defined(__i386) || defined(__amd64)) && (NAN_PAYLOAD_LDBL_BITS == 80) */

{
float f;
double d;
long double ld;
f = NAN; (void)printf("flt: payload of plain NAN=%ld\n", (long)GET_NAN_PAYLOAD_FLOAT(f));
d = NAN; (void)printf("dbl: payload of plain NAN=%ld\n", (long)GET_NAN_PAYLOAD_DOUBLE(d));
ld = NAN; (void)printf("ldbl: payload of plain NAN=%ld\n", (long)GET_NAN_PAYLOAD_LONGDOUBLE(ld));
f = nanf(""); (void)printf("flt: payload of plain nanf("")=%lx\n", (long)GET_NAN_PAYLOAD_FLOAT(f));
d = nan(""); (void)printf("dbl: payload of plain nan("")=%lx\n", (long)GET_NAN_PAYLOAD_DOUBLE(d));
ld = nanl(""); (void)printf("ldbl: payload of plain nanl("")=%lx\n", (long)GET_NAN_PAYLOAD_LONGDOUBLE(ld));
f = sinf(INFINITY); (void)printf("flt: payload of nan=%f from sinf(inf)=%lx\n", f, (long)GET_NAN_PAYLOAD_FLOAT(f));
d = sin(INFINITY); (void)printf("dbl: payload of nan=%f from sin(inf)=%lx\n", d, (long)GET_NAN_PAYLOAD_DOUBLE(d));
ld = sinl(INFINITY); (void)printf("ldbl: payload of nan=%Lf from sinl(inf)=%lx\n", FLT1282LDBL(ld), (long)GET_NAN_PAYLOAD_LONGDOUBLE(ld));
}

return(EXIT_SUCCESS);
}

#endif /* TESTCODE */

[Roland Mainz]

Post by Roland Mainz

Post by Roland Mainz

Post by Roland Mainz

Post by Roland Mainz

I'm currently looking into a complex mathematical simulation (fluid
dynamics) and found references to NaN (Not A Number) with payloads.
The Nan values are used to represent 'no data' in the datasets and the
payloads are used for context information. The code has almost 2
million lines of code and removing the feature would be a major
effort.

[snip]

Post by Roland Mainz

Post by Roland Mainz

Post by Roland Mainz
Attached (as "nan_payload_test1_20130904_001.c.txt") is some prototype
code which shows now to handle NaN payloads...
- The code explicitly requires ISO C99 semantics
- The code relies on |__int128_t| to handle |long double|. Originally
I thought about handling the code using byte-by-byte masking... but it
turned out that this creates a lot of mess and is endian-specific. The
final code may have to resort to that to be portable but for now I
use |__int128_t| to keep the code readable

Uhm... any feedback/suggestions ? Adding Nan payload support seems to
be easy and doesn't come with a measureable performance penalty (since
we only do extra work *if* there is a payload, otherwise most code has
only "wrapper" overhead) ... if there are no objections I can try my
luck and add this to ksh93's arithmetic engine...

Attached (as "astksh20130913_nanpayload_prototype001.diff.txt") is a
_prototype_ for SuSE Linux 12.3/AMD64/32bit. 32bit will not work
because it lacks the |__int128_t| type... see below...
-- snip --
$ ksh -c 'typeset -X x ; typeset -sX y ; float z ; ((
x=ast_nan(0xbeef) )) ; ((y=x, z=y)); print -- $((z))'
ast_nan(0xbeef)
-- snip --
- This is a prototype
- The code was only tested on SuSE 12.3/AMD64/64bit with gcc4.7.
- Surprisingly the performance overhead is barely measureable...
basically we have a few extra |isnan()| calls or macros and that's it
- Put the nan payload functions into their own source file
- ABI issue: |nan("")| on some platforms only sets the highest hit in
the payload instead of 0xFFFFFFFF (e.g. set all bits). How should we
deal with it ?
platform-specific macros to map them properly to SFDOUBLE, |long
double|, |double| and |float|). The seperation is important to keep
the #ifdef/#else/#endif hackery to a minimum and in one place
- How should we deal with the usage of |__int128_t| ? It's only
available on 64bit plaftorms with "clang" and newer "gcc" versions...
but writing a "portable" version which does the operations on a
byte/word/longword level makes the code depending on the endian-ness
of the platform. Main issue here is testing... my SPARC machine is
broken and I need some time to find money to fix it.

Attached (as "astksh20130913_nanpayload_prototype005.diff.txt") is an
updated patch for ast-ksh.2013-09-13 and
"nan_payload_test1_20130922_001.c.txt" is standalone testcode for
debugging purposes...
- This is a PROTOTYPE, the code is NOT ready yet. There is a lot of
"spaghetti" which is caused by the issue that the master sources for
the nan payload code live outside the AST tree for now... and there
are unresolved issues (see below)
- The API is *layered* intentionally: First layer uses
|double|, |long double|). This is done to seperate logical from
physical floating-point layouts... for example on some platforms |long
double| is the same type as |double| and some platforms only know
about |double| etc. etc. The other area of pain is x86/AMD64 with
80bit |long double| vs. other platforms with 128bit |long double|
- we need to write an iffe probe for the nan payload stuff, which
provides constants like number of bits, masks etc. and verifies that
the |float|, |double| and |long double| behave *EXACTLY* as defined in
IEEE754-2008. Some old implementations and software emulations take
shortcuts... or have "magic bits" which can cause trouble (i387's
IEEE754-1985-draft implementation with 80bit are such a special case).
We can only support payloads if we have a
fully-IEEE754-2008-conforming floating-point implementation (99% of
the modern platforms should fall into this category)
- x86/AMD84 i387 80bit floats are fully functional and have extra code
to support it even if |__int128_t| is not supported
- 128bit floats should be working now but require at least some testing
- SPARC needs to be tested
- We need per-floating-point datatype payload MIN/MAX constants
- On some OSes like Solaris |nanf("")|/|nan("")|/|nanl("")| return a
payload of 0xFFFFFFFFFF... (=all payload bits set) while the |NAN|
contanst returns a nan value with all payload bits not set
- AST |strtold()| on Solaris returns a value like
|nanf("")|/|nan("")|/|nanl("")|, which causes trouble. Replacing it
with |ast_nan()| causes sign reversal for (yet) unknown reasons (but
the same happens with |sin(inf)| ... which may be a hint that
something else is wrong in libast vs. nan sign handling) ... needs
more debugging...
- Glenn and I need to agree on a naming scheme for NaN's with payloads
(gsf: We need to chat... issue is that not all naming schemes assume
that the payload is an unsigned int). Currently we use
"ast_nan(hexval)" in ksh93 and |strtof()|/|strtod()|/|strtold()|

Glenn: AFAIK I solved all technical issues (SPARC included) ... which
leaves the naming scheme as the only open question.
For now I would prefer to stick with |ast_nan(hexval)| and later
provide an env variable to switch between naming schemes (but always
accept |ast_nan(hexval)| as input).

What do you think ?

----

Bye,
Roland