[ast-developers] /dev/@/ will work, not work, only half (was: Re: Next alpha/beta and roadmap for ksh93v?)

[Cedric Blancher]

Glenn, when is the next alpha/beta due for AST and for UWIN? Is there
a roadmap what you are planning long-term for AST and UWIN?

I've been working on an ast-open pre-alpha that will (hopefully) clean up
pseudo devs, O_XATTR extended attributes, and *at() usage across ast by
these pseudo paths
/dev/<IP-PROTOCOL>/<HOST>/<PORT>
/dev/fd/<FD>[/...]
/proc/<PID>/fd/<FD>[/...]
/dev/file/<FLAG>[,<FLAG>...]:/<ABSOLUTE-PATH>
/dev/file/<FLAG>[,<FLAG>...]:<RELATIVE-PATH>

We talked this through in our engineering meeting this morning.
There is no way (at least from testing on a whiteboard with 9 quite
much-smarter-than-I people) that
/dev/@/:/<ABSOLUTE-PATH>/@/<RELATIVE-ATTRIBUTE-PATH> will *EVER* work
without creating a high risk to blowing things up.

I said high, as Herv? M?nager pointed out that a lot of 'whacky'
characters (e.g. @) are used to implement locks (which may even change
state if the access time timestamp is changed!!!!!) and other special
things or are outright used on Windows by system software (installer)
or Internet explorer (e.g. which uses alternate streams with
@,<,>,&,%,#,$!,(,),... to store cookies for downloaded files).
That's the reason why so many GNU tools have options to use '\0' as
filename separator instead of the traditional $'\n'.

What *will* work is to use /dev/@/:/<ABSOLUTE-PATH> as directory name
to an NFSv4 attribute store, which could be used with redirect
{n}</dev/@/:/<ABSOLUTE-PATH> and
/dev/fd/${n}/<RELATIVE-ATTRIBUTE-PATH> or
~{n}/<RELATIVE-ATTRIBUTE-PATH>. That's the only "safe" way to access
these files.

currently ast + patches for the non-path variant of these pseudo paths
is morphing into an #ifdef mess making some portions unreadable and very fragile
the idea is to concentrate all of it in libast so it will be easy to
debug and extend -- in the current situation a bug fix for say O_XATTR
might involve a half-dozen or more files and that doesn't include the
ones that have yet to be patched for O_XATTR -- and these patches are rife
warnings that would surely get botched in future patches

A testsuite which tests the libast C API would prevent this, right?

...

I am currently deep inside ast::fts which is the directory traversal api
used by all ast utilities; part of this work involves the addition of
fgetcwd(int fd) which is a *at()-like variant of getcwd(), adding
FTSENT.fts_dirfd which is an open fd to the containing directory for
the current FTSENT; currently most ast commands that use fts do it
in the "chdir" mode where each FTSENT has "." set to the containing dir;
this includes the commands in -lcmd -- not a good thing in general;
fts_dirfd aware fts will allow the caller to fts_open(FTS_NOCHDIR) and use

Er... I think you got this backwards. By *default*, if the at() API is
available in libc, fts should NOT use chdir(). chdir() is a
pain-in-the-arse performance killer on networked filesystem and using
the at() API *is* much faster.
If course you could provide FTS_USECHDIR, FTS_NOCHDIR to force one of
the modes but the fts API should default to use at() and not chdir()
if at() is available.

*at(FTSENT->fts_dirfd, fts->fts_name) -- the challenge is to do this
without running out of fd's;

Why? Use a cache of 4-8 fds?

along with this is a new fts_open() flag
FTS_XATTR that will traverse visible hierarchies and their hidden O_XATTR
counterparts, if any, within a single fts stream

That raised a question here: Why?

Ced