several conditions are met as outlined below; for more background, refer to
http://www.FreeBSD.org/cgi/query-pr.cgi?pr=113860
We don’t yet optimise 「% sh -c '{ echo a; sleep 10;}&'; sleep 1; ps T」 so
the FreeBSD® sh approach cannot work for us, but scanning the “sh -c” argu‐
ment for disallowed characters and, if not, setting a flag that enables the
shell to exec the tree when parsed as TCOM *and not c_trap()* was possible.
Disallowed characters are currently C_QUOTE except space, that is:
Tab Newline " # $ & ' ( ) * ; < = > ? [ \ ] ` |
This should catch all cases of magic, variables, subshells, pipelines, etc.
XXX we could track whether tty_fd has already been successfully opened,
XXX the ttystate initialised, and then just never close it unless it is
XXX necessary, then we can keep COLUMNS/LINES accurate in scripts, even
remainging CIDs not listed are either
• false positive (bug in coverity)
• intentional (possibly with lint override coverity doesn't parse)
• VLA (XXX find out how to mix C99 and ANSI VLAs)
• things flagged as possible resource leaks I have no idea about
(no biggie though, and only in error cases I think)
This was actually more evil:
• use a recursive function to display blocks in reverse order,
so that local variable values overwrite global ones
• add array support to typeset -p (from typeset -p -)
• display 'set -A varname' line before setting values, for -p
• if -p got arguments, only display those (from the innermost scope)
Also, the usual amount of code cleanup…
to get rid of the bias introduced by making the hash never zero
… he also pointed out a memory (heap) usage optimisation… which
may impact code size a bit though as I’d need to pass an additional
argument on hashtable function calls… or, forgo the benefit of not
having to pointer-align the key in the structure, which can be as
much as 3/7 octets per item, heap storage… OTOH the saved space is
4/8 octets per not-allocated item, possibly some code (use of an
multiply-add opcode), but the function call overhead/cost would
possibly be quite a bit… I guess I’ll have to measure…
XXX in the future, the entire scheme must be rethinked when we need more
XXX entropy for the hash tables; possibly a cheap add using NZAT and re-
XXX initialise the LCG only on access and when added (so keep NZAT state
XXX separate from LCG state); also, then we will need a more elaborate
XXX scheme, such as adding from environment, editor keypresses and timing
• strlcpy
• utf_wcwidth
note strchr/strstr from misc.c are still #ifdef DEBUG only, as they are
not eligible: they’re for const-cleanliness debugging purposes
XXX get rid of multiple occurrences of binary search code, too…
• promote SCO OpenServer and UnixWare to !oswarn
• omit trying -O2/-O on OpenServer 5 and USL C
• cast mksh_ari_t to int, mksh_uari_t to unsigned int for printf
• skip ulimit-1 on syllable (which is still too broken)
• write ((mksh_ari_t)-2147483648) ipv UB ((mksh_ari_t)1 << 31)
and add a comment that that is actually meant
• rewrite functions returning !void ending in NOTREACHED
so they’ve got a jump target returning an error at the
end, to aid older compilers and just to be safe
• cast struct stat.st_size to off_t or size_t explicitly when needed
• shorten struct env by two bytes and an alignment, at least
also, optimise control flow and fix more paren matching cases
is not found, from a suggestion by RT (LP: #912691)
• try harder (in a loop) to acquire a file lock if the locking mechanism
documents EINTR is a possibility (fcntl always, flock on Linux not .Ox)
• use -std=c99 not -std=gnu99 if it must be at all
00:45 -!- variable [root@freebsd/developer/variable] has joined #!/bin/mksh
• +b *!*root@*, +b $a:root, +b $r:root on one more channel
• certain checks to prevent:
00:47 < variable> wjcw: sh.h:308: error: conflicting types for 'getrusage'
01:19 < variable> oh
01:19 < variable> I needed to run Build.sh
deep changes into R40-stable branch
• Version accordingly: HEAD gets 2012/03/24 (hi Eddy/Chris) so we backdate
$KSH_VERSION for R40-stable to 2012/03/20 (hi David) as that comes before
even though it includes today’s latest fixes
• Also, sync clog (including MFC indicators)
if we find any, but not later; do not check on every read
⇒ allows changing COLUMNS and LINES (independent of each other, or both)
for script shells by passing them in an environment setting, even if
we get a tty; interactive shells still check before each line is read…
reported by the PLD guys, thanks
prerequisites are actually fulfilled, i.e. evrn further down than with
the last commit doing this, and move some prerequisites of stuff that
has wandered outside the !INCLUDES_ONLY block with the compile-time
assert changes to the outside, too
fixes FTBFS on MSYS which has neither <stdint.h> nor uint32_t
reported by RT
handle any more, octal 010 style constants, as promised
• overhaul the manpage re. arithmetic expressions, make the guarantees
mksh code has explicitly, precisely, clear
• to reduce burden of the compiler, getint() now operates on mksh_uari_t
internally; it already applied the sign after operation, anyway (C99
guarantees wraparound on unsigned types, but for signed types we need
specific compiler support; apparently, this comes from hardware limits)
• use const and shuffle order of locals around while here
in the cases where they are defined unambiguously; bug reported by
Jilles Tjoelker in <20111129232526.GC14357@stack.nl> due to a report
by Stefano Lattarini on bug-autoconf
in the ambiguous case, I stick to traditional pdksh behaviour, which means
test ! a = b vs. test a = b
and
test ! a -o b vs. test a -o b
behave different from each other (in the second case, the NOT operator
binds strong; POSIX demands a reduction to 3 arguments and negating
that result in the first case), so we're at two known not-ok in the
FreeBSD® testsuite. (81 and 82 in regress.sh,v 1.3)
move /etc/{,suid_}profile to /system/etc/ for Android (but do not make
the location of /etc configurable); rewrite manpage section about
/etc/{,suid_}profile, .profile, .mkshrc
• while here, reformat 'struct tbl' comment-wise and placement-wise
and drop the Tflag typedef
• while here, write regression test for the "global" built-in, which
does what typeset is supposed to do except that it doubles as "local"
• IBM XL C: display version better (tested on V7.0 by cnuke@)
• do not 'IFS=: read nr name', Cygwin 1.7 dash fails it
• disable cd-pe, glob-range-3 on Cygwin (the former cannot
succeed because the mv fails, the latter fails from 1.7 on)
• mark heredoc-tmpfile-8 as need-pass: no
• apply __attribute__ only to a function prototype, not to
the body (even if static), since xlC fails that
• bump version to R40 (beta)
Testsuite:
• add new need-pass: {yes|no} attribute, default yes
• exit with 1 if a need-pass test failed unexpectedly
idea by Kacper Kornet <draenog@pld-linux.org>
• mark utf8bom-2 as need-pass: no
Infrstructure:
• add housekeeping function for making a tty raw
• switch functions with unused results to void
• struct op: u.charflag contains last char of ;; in TPAT
• var.c:arraysearch is now a global function
Language:
• add ;& (fall through) and ;| (examine next) delimiters
in addition to ;; (end case) as zsh extensions, because
POSIX standardised on ;& already
• add -A (read into array), -N (read exactly n bytes),
-n (read up to n bytes), -t (timeout) flags for read
from ksh93
• allow read -N -1 or -n -1 to slurp the entire input
• add -a (read into array the input characters) extension
specific to mksh to read, idea by David Korn
• add -e (exit with error if PWD was not set correctly
after a physical cd) to cd builtin, mandated by next
POSIX, and change error codes accordingly
Rewrites:
• full rewrite of read builtin and its manpage section
• add regression tetss for most of the new functionality
• duplicate hexdump demo tests for use of read -a
• use read -raN-1 in dot.mkshrc to get NUL safe base64,
DJB cdb hash and Jenkins one-at-a-time hash functions
• don’t leak memory parsing possible I/O redirection tokens
• get rid of volatile by using more const (also helps codegen, methinks)
• support empty here document markers (mksh extension)
• pimp the manpage
• ensure that bool/true/false are cpp macros, overriding any pre-defined
• document the requirement that tobool(x) must map any-type 'x' into bool
• document the requirement that a bool must only be true or false, and
that it (tobool() rather) must have an identity mapping to 'short'
• possibly fix ksh_func for/and fpFUNCTf – maybe spotted by cnuke@
a bit more with POSIX and the other shells
I considered http://austingroupbugs.net/view.php?id=253 but the use
of bi_errorf() is interesting, especially as it’s often enough a
noreturn function, and funnily enough, 'cd -P /foo' returns 0 while
'chdir -P /foo' fails (so idk where to put -e)…
⇒ consider merging simplify_path()
• move funcs.c:c_cd() to misc.c
• make misc.c:make_path() static, c_cd() is its only user
⇒ mark as obsolete
• tweak misc.c:set_current_wd() to drop ksh_get_wd() argument
should be no code change, but the entire path stuff is a mess…
so expect actual implementation changes or even rewrites shortly
and switches to the TARGET_OS=Linux
• introduce android as regression test suite category
• add an android specific standard alias
• clean up redundant ‘-o sh’ arg in a few checks
UTF-8 BOM instead (UTFMODE has a separate value now for activated
during BOM skipping)
• parsing a COMSUB now skips UTF-8 BOM, too, but only temporarily
(token stream, lexer output / parser input), EOS terminated, let
SASPAREN use the same lexing as SBASE (e.g. COMSUB recursively)
• make wdstrip recursive
• fix processing of COMSUB in wdstrip
⇒ pass comsub-1 test
• expose another debugging function
• use shf_putc (macro), shf_putchar (function) ipv tputc
• replace shf_putchar(x,y) calls for side-effect-less x with shf_putc
• plug another bug in the tree code – '\' → "\\" (backslashes must be
escaped inside double quotes, too)
• adjust testsuite (and, I _had_ wondered…)
EOF) # works again now, plugging a regression
• rewrite the here document parsing code to be *much* more efficient
(and a bit more readable too!) using goto, while here (no kidding)
① currently: ((cond) ? true : false) but (!!(cond)) and casting to bool,
the latter only if stdbool.h, would also work – which performs best on
(and across) all supported systems?
instead, but the parser for the so-called “backticks” (U+0060) still emits
plaintext COMSUB wdstrings, and the evaluation code emits plaintext if the
code is not run (‘-n’ option), so it’s not worth the effort and memory ma-
nagement issues, even though it _would_ optimise the most common case…
Bump version numbers, sync regression tests; add one testcase from the old
webpages too. Sync manpage, this now works, but keep the workaround in, as
“portability issue” with slightly changed wording.
Also, /bin/sleep must be used in one manpage example if sleep is built in.
• functions called by mksh’s grouping memory allocator
• functions called by mksh code itself
the latter may be changed to call the internal grouping allocator,
if a porter so desires (but if this were recommended, the code in
question would already do so, so…)
• PIPESTATUS now supported (like bash 2) whose last member
may actually differ from $? since the latter may not be the
result of a pipeline partial command
• add regression tests, documentation, etc.
• in interactive mode, always look up {LC_{ALL,CTYPE},LANG} environment
variables if setlocale/nl_langinfo(CODESET) doesn’t suffice
• add the ability to call any builtin (some don't make sense or wouldn't
work) directly by analysing argv[0]
• for direct builtin calls, the {LC_{ALL,CTYPE},LANG} environment
variables determine utf8-mode, even if MKSH_ASSUME_UTF8 was set
• when called as builtin, echo behaves POSIXish
• add domainname as alias for true on MirBSD only, to be able to link it
• sync mksh Makefiles with Build.sh output
• adjust manpage wrt release plans
• link some things to mksh now that we have callable builtins:
bin/echo bin/kill bin/pwd bin/sleep (exact matches)
bin/test bin/[ (were scripts before)
bin/domainname=usr/bin/true usr/bin/false (move to /bin/ now)
• drop linked utilities and, except for echo and kill, their manpages
• adjust instbin and link a few more there as well
a mirtoconf check, would’ve been a real problem on an LP64 platform
• sh.h: work around a bad interaction between -Wformat on gcc and manual
string pooling for T_synerr, which is used in place of a format string
in some places
– possible integer overflows in memory allocation, mostly
‣ multiplication: all are checked now
‣ addition: reviewed them, most were “proven” or guessed to be
“almost” impossible to run over (e.g. when we have a string
whose length is taken it is assumed that the length will be
more than only a few bytes below SIZE_MAX, since code and
stack have to fit); some are checked now (e.g. when one of
the summands is an off_t); most of the unchecked ones are
annotated now
⇒ cost (MirBSD/i386 static): +76 .text
⇒ cost (Debian sid/i386): +779 .text -4 .data
– on Linux targets, setuid() setresuid() setresgid() can fail
with EAGAIN; check for that and, if so, warn once and retry
infinitely (other targets to be added later once we know that
they are “insane”)
⇒ cost (Debian sid/i386): +192 .text (includes .rodata)
• setmode.c: Do overflow checking for realloc() too; switch back
from calloc() to a checked malloc() for simplification while there
• define -DIN_MKSH and let setmode.c look a tad nicer while here
we don’t get SIGWINCH when the window size changes during the runtime of
that, so, the signal is only usable reliably during editing in the shell
and we re-check the window size before each interactive edit line again
a string buffer whose window size is currently 32 (initial), your data
is 96 bytes, this routine used to resize the buffer to 64, append your
first 64 bytes to it (no matter if there's already something in it)
and then writes the remaining bytes to stdio fd instead of the string…
if it doesn’t SIGABRT before
discovered by wbx@ – thanks – bug inherited from pdksh 5.2.14 (AD 1999)
• deactivate %a and %A since our libc doesn’t have it
• rewrite the mksh integration code to use shf instead of stdio, removing
floating point support always in the process, as shf doesn’t support it
⇒ saves 11114 (6706 text, 168 data, 4240 bss) with dietlibc on Debian
• fix -Wall -Wextra -Wformat -Wstrict-aliasing=2 for gcc (Debian 4.4.4-7)
• fix these and -Wc++-compat for gcc version 4.6.0 20100711 (experimental)
[trunk revision 162057] (Debian 20100711-1) except:
– a few enum warnings that relate to eglibc’s {g,s}etrlimit() functions
taking an enum instead of an int because they’re too stupid to adhere
to POSIX interfaces they design by themselves
– all “request for implicit conversion” involving a "void *" on one side
• tweak the manual page somewhat more
• avoid calling realloc twice in sequence, since the final
size is known at the first call already
• do not lstat(2) the same path twice in the Hurd codepath
concurrently accessing the same $HISTFILE be more synchronised with
each other: empty lines (just pressing Return) and duplicates (that
are split and written twice by the lines loaded from $HISTFILE in
the meantime); requested by Maximilian “mxey” Gaß in #!/bin/mksh
some idiotic terminal emulators and/or people seem to use the es-
cape codes normally denoting Alt-Arrowkey instead so let's simply
bind them to the vt_hack as well... (untested)
• merge the rest of branch tg-wcswidth-behaviour
• enhance test cases for wcswidth-like behaviour
• switch hash table collision resolution algorithm to Python’s as announced
• bump vsn
│remember to restore errno (ie. stop someone from making a mistake later)
│ok guenther
check.t, sh.h: bump vsn
I wonder though why errno must be restored even if nothing was
called after reading it… moid?
which, in its latest sid incarnation, even received mksh's ability
to produce ${!foo[*]} array keys, wow!)
* plug a memory leak while here (ATEMP only, but still)
I read, IIRC in the Cederqvist, that 'cvs tag' sets a sticky tag onto
the cwd… it doesn’t, apparently. (I actually like it better this way,
but one needs to know!)
others (colon and equals sign need to be simply escaped, while dollar
sign and accent gravis need double escaping like opening square brak-
ket did back then); add = to C_QUOTE to simplify (doesn't break any-
thing) and sort these strings asciibetically while here
• use a combination of the one-at-a-time hash and an LCG for handling
the $RANDOM special if !HAVE_ARC4RANDOM instead of rand(3)/srand(3)
and get rid of time(3) usage to reduce import footprint
• raise entropy state (mostly in the !HAVE_ARC4RANDOM case though…)
• simplify handling of the $RANDOM_SPECIAL generally
• tweak hash() to save a temp var for non-optimising compilers
• some int → mksh_ari_t and other type fixes
• general tweaking of code and comments
just a "somewhat more POSIX" but also a "/bin/sh legacy kludge" mode
* consistently capitalise POSIX and SUSv3/SUSv4 (same as AT&T ksh) and
Bourne shell
to it are now either arc4random or rand/srand, but srand retains the old
state; set +o arc4random is no longer possible, but if it's there we use
arc4random(3), if not, we use rand(3) for $RANDOM reads; optimise special
variable handling too and fix a few consts and other minor things
MKSH_S_EDIT for small (Emacs) editing mode, MKSH_S_FEAT for all the dis-
abled language features), which can be set to 0 despite MKSH_SMALL being
defined to re-enable the Vi command line editing mode (which I wouldn't,
but fits into the general mastermind scheme)
some GNU bash extensions (suggested by cnuke@) and bind macros
* make the random cache more efficient (and the code potentially
smaller, although we have a new implementation of the oaat hash
function, alongside the old one, now) and pushb only if needed
(i.e. state has changed or user has set $RANDOM, but not onfork)
• shell flags are now handled in one single place (sh_flags.h)
• sync comments (between enum and array) and manpage with reality
• FMONITOR is now no longer needed for Hartz IV shells
tg