lijo george
2015-02-10 20:07:09 UTC
I've observed some issues related to associative arrays in a
Solaris environment.
The issues are the following
1. unnecessary directory lookups when doing typeset/unset
2. memory leak: unset is not properly freeing members of an associative
array
3. ksh bypassing data ulimits when allocating associative arrays
All the 3 issues are reproducible in all ksh versions I've checked
2011-02-08, 2012-08-01, 2013-10-10 and 2014-12-24.
I've attached sample testscripts for each of the issues,
For issue 1, test2.ksh( this calls test2a.ksh and test2b.ksh)
This basically does a typeset and unset of a associative array in a loop.
We see that for each typeset/unset, we see that directory lookups are
happening as we can see from the getdents calls, i.e globbing/file name
generation is taking place.
The stack is as follows
<- getdents(3, 0xFFFFFD7FFF144000, 8192)
<- libc:getdents() = 0
<- libc:readdir() = 0
<- libast:gl_dirnext() = 0
<- libast:glob_dir() = 0
<- libast:_ast_glob() = 0
<- libshell:path_expand() = 1
<- libshell:path_generate() = 1
<- libshell:endfield() = 0
<- libshell:sh_macexpand() = 1
<- libshell:arg_expand() = 1
<- libshell:sh_argbuild
<- libshell:sh_exec
For issue 2, test3.ksh
This creates an associative array and sets the elements in a loop. For each
iteration the array is unset but it looks like the unset is not freeing the
memory and the process size grows which we can observe using the pmap
output.
For issue 3, test4.ksh
We set ulimit values for the data segment and then create an associative
array
in a loop and unset it at every iteration. But as we observed in the
previous
issue the memory is not being freed and the process size grows. As we can
see
from the pmap output at the end, the limits set by ulimit are bypassed.
I browsed through the archives but couldn't find any reference to these
issues.
Could someone let me know if there is any similar reported issue or fix.
I'm trying to rootcause the issues myself but being a beginner to the ksh
code
it may take a while before I can.
Thanks,
Lijo
Solaris environment.
The issues are the following
1. unnecessary directory lookups when doing typeset/unset
2. memory leak: unset is not properly freeing members of an associative
array
3. ksh bypassing data ulimits when allocating associative arrays
All the 3 issues are reproducible in all ksh versions I've checked
2011-02-08, 2012-08-01, 2013-10-10 and 2014-12-24.
I've attached sample testscripts for each of the issues,
For issue 1, test2.ksh( this calls test2a.ksh and test2b.ksh)
This basically does a typeset and unset of a associative array in a loop.
We see that for each typeset/unset, we see that directory lookups are
happening as we can see from the getdents calls, i.e globbing/file name
generation is taking place.
The stack is as follows
<- getdents(3, 0xFFFFFD7FFF144000, 8192)
<- libc:getdents() = 0
<- libc:readdir() = 0
<- libast:gl_dirnext() = 0
<- libast:glob_dir() = 0
<- libast:_ast_glob() = 0
<- libshell:path_expand() = 1
<- libshell:path_generate() = 1
<- libshell:endfield() = 0
<- libshell:sh_macexpand() = 1
<- libshell:arg_expand() = 1
<- libshell:sh_argbuild
<- libshell:sh_exec
For issue 2, test3.ksh
This creates an associative array and sets the elements in a loop. For each
iteration the array is unset but it looks like the unset is not freeing the
memory and the process size grows which we can observe using the pmap
output.
For issue 3, test4.ksh
We set ulimit values for the data segment and then create an associative
array
in a loop and unset it at every iteration. But as we observed in the
previous
issue the memory is not being freed and the process size grows. As we can
see
from the pmap output at the end, the limits set by ulimit are bypassed.
I browsed through the archives but couldn't find any reference to these
issues.
Could someone let me know if there is any similar reported issue or fix.
I'm trying to rootcause the issues myself but being a beginner to the ksh
code
it may take a while before I can.
Thanks,
Lijo