• merge the rest of branch tg-wcswidth-behaviour
• enhance test cases for wcswidth-like behaviour
• switch hash table collision resolution algorithm to Python’s as announced
• bump vsn
• use a combination of the one-at-a-time hash and an LCG for handling
the $RANDOM special if !HAVE_ARC4RANDOM instead of rand(3)/srand(3)
and get rid of time(3) usage to reduce import footprint
• raise entropy state (mostly in the !HAVE_ARC4RANDOM case though…)
• simplify handling of the $RANDOM_SPECIAL generally
• tweak hash() to save a temp var for non-optimising compilers
• some int → mksh_ari_t and other type fixes
• general tweaking of code and comments
just a "somewhat more POSIX" but also a "/bin/sh legacy kludge" mode
* consistently capitalise POSIX and SUSv3/SUSv4 (same as AT&T ksh) and
Bourne shell
call it only if $RANDOM is indeed set (although pool extension would be a
possibility we do have arc4random_atexit which does it nicely too)
• avoid calling setspec for int→str conversion just before execve()
to it are now either arc4random or rand/srand, but srand retains the old
state; set +o arc4random is no longer possible, but if it's there we use
arc4random(3), if not, we use rand(3) for $RANDOM reads; optimise special
variable handling too and fix a few consts and other minor things
MKSH_S_EDIT for small (Emacs) editing mode, MKSH_S_FEAT for all the dis-
abled language features), which can be set to 0 despite MKSH_SMALL being
defined to re-enable the Vi command line editing mode (which I wouldn't,
but fits into the general mastermind scheme)
some GNU bash extensions (suggested by cnuke@) and bind macros
* make the random cache more efficient (and the code potentially
smaller, although we have a new implementation of the oaat hash
function, alongside the old one, now) and pushb only if needed
(i.e. state has changed or user has set $RANDOM, but not onfork)
• shell flags are now handled in one single place (sh_flags.h)
• sync comments (between enum and array) and manpage with reality
• FMONITOR is now no longer needed for Hartz IV shells
• we must not set the item pointer to NULL, since subsequent ktscan()
would stop there and not find any later occurrences
possible resolution strategies:
‣ still keep tablep; store a dummy value (either (void *)-1 or, probably
more portable, &ktenter or something like that) as is-free marker
⇒ retains benefit of keeping count of actually used entries
⇒ see below for further discussion
‣ don't keep tablep; revert back to setting entry->flag = 0
⇒ need to ktwalk() or ktsort() for getting number of entries
⇒ most simple code
‣ same but with a twist: make ktscan() set pp to the first one with
!(entry->flag & DEFINED)¹ so that it can subsequently be re-used,
or, more accurate, free’d and the entry pointer re-used
⇒ less chance of texpand()ing when not needed
‣ similar (from kabelaffe@): in ktsearch(), move the one we DID find
to the first unused one
⇒ doesn’t need tablep or something, but has the overall best
memory use
⇒ more complicated ktscan(): needs to check pointer for NULL, for
dummyval, then entry->flag
⇒ makes lookup more expensive
⇒ benefit: self-optimising hash tables
⇒ loss: still need ktwalk() or ktsort()
• when afree()ing in ktremove(), …
① need to take FINUSE into account
• Python-2.5.4/Objects/dictnotes.txt talks about cache lines
‣ linear backward scan is much worse than linear forward scan
(even if we have to calculate the upper C-array bound)
‣ dereferencing the entry pointer in ktscan() is a penalty
• Python-2.5.4/Objects/dictobject.c has a lot of comments and
a rather interesting collision resolution algorithm, which
seems to de-cluster better than linear search at not much
more cost
• clib and libobjfw have unusable (for looking-at-for-ideas)
hash table implementations
this is a no-op change breaking ifdef-out-d code; the most likely
to happen is to switch to the following scheme:
• keep tablep in struct tbl
• use a magic pointer value for ktremove’d entries, deallocate
the struct tbl as soon as possible – if not FINUSE, immediately
inside ktremove()
‣ memory gain, despite needing to have tablep around
• nuke ktdelete, so that all ops go through kt{enter,remove}
‣ gains us accurate fill information
‣ speed gain: ktscan() needs no longer dereference removed entries
‣ memory (ktsort) and speed (ktwalk) gain: removed entries are now
ignored right from the beginning, so tstate->left and the size
of the sorted array are accurate
‣ removed entries no longer can cause texpand() to be invoked
⇒ this does not give us self-optimising tables, but a speed and
memory benefit plus, probably, simplicity of code; we accurately
know how many non-deleted entries are in a keytab so we can cal-
culate if we need to expand, how much space ktsort() is going to
need, and, for when indexed arrays will be converted to use key-
tabs instead of singly linked linear lists, ${#foo[*]} is fast
(although ${!foo[*]}² and ${foo[*]}³ will need some tweaking and
may run a little less quickly)
• shuffle code around, so that things like search/scan and garbage
collection can be re-used
• use Python’s collision resolution algorithm ipv linear search
② the list of keys needs to be sorted, at least for indexed arrays⁴
③ this needs to be sorted by keys, at least for indexed arrays⁴
④ … but this is a nice-to-have for associative arrays⁵ as well
⑤ which we however do not have
it with the array index; var.c says that
│ 1244 /* The table entry is always [0] */
so that we can have a special flag and a union which stores hval for
the table index, the array index otherwise (coïncidentally *hint hint*
they have the same size)
return information needed to do a real ktremove instead of the pseudo
ktdelete operation which merely unsets the DEFINED flag to mark it as
eligible for texpand garbage collection (even worse, !DEFINED entries
are still counted)
much better avalanche and no known funnels
• improve comments
• fix some types (uint32_t for hash, size_t for sizes)
• optimise ktsort()
no functional change, I think
Build.sh but use 'if defined(PRECOND) && !defined(TOBEDEFINED)'if possible
* for all of the source code, drop annotations "imake style" (if we check
for specific OSes, bad, instead of using mirtoconf checks proper) and
"conditions correct?" (if I'm not entirely sure if that #if catches all
cases and no false positives) where I can see it by grepping immediately
* bump mksh patchlevel
* refresh Makefiles
starting with an ‘!’ exclamation mark at the beginning of a com-
mand (PS1 not PS2), shall have the same effect as the predefined
“r” alias, to be compatible with csh and GNU bash’s “!string” to
«Execute last used command starting with string» – documentation
and feature request provided by wbx@ (Waldemar Brodkorb)
• expose “#ifdef MKSH_MIDNIGHTBSD01ASH_COMPAT” just in case they decide to
require it and show it in the ksh version automatically
• sync the use of non-ASCII characters over files (unification)
fix the regression test’s results while here, which have been
broken since cid 10049D9BE5254CE65B8
• get rid of separate copyright file which was intended for De-
bian; track down commits in all files of oksh-mirbsd and mksh
to get correct copyright years per-file, as is BSD custom
struct env (other structures defined have no "foreign type with pos-
sible alignment constraints" members) and take care of it while dea-
ling in a struct env instance
pass "xerrok" status across the execution call stack to more closely
match what both POSIX and [18]ksh.1 already describe in regards to set
-e/errexit's behavior in determining when to exit from nonzero return
values.
specifically, the truth values tested as operands to &&' and ||', as
well as the resulting compound expression itself, along with the truth
value resulting from a negated command (i.e. a pipeline prefixed !'),
should not make the shell exit when -e is in effect.
issue reported by matthieu.
testing matthieu, naddy.
ok miod (earlier version), otto.
man page ok jmc.
on Debian Lenny/amd64 (XXX need more verification; this
can be used for 64 bit arithmetics later too)
PPID, PGRP, RANDOM, USER_ID are now unsigned by default
was hard to type and hard to fix, galloc is also hard to fix, and some
things I learned will probably improve things more but make me use the
original form as base (especially for space savings)
* let sizeofN die though, remove even more casts
* optimise, polish
* regen Makefiles
* sprinkle a few /* CONSTCOND */ while here
tg