a couple of overnight, phone call and code/doku read ideas:

• we must not set the item pointer to NULL, since subsequent ktscan()
  would stop there and not find any later occurrences
  possible resolution strategies:
  ‣ still keep tablep; store a dummy value (either (void *)-1 or, probably
    more portable, &ktenter or something like that) as is-free marker
    ⇒ retains benefit of keeping count of actually used entries
    ⇒ see below for further discussion
  ‣ don't keep tablep; revert back to setting entry->flag = 0
    ⇒ need to ktwalk() or ktsort() for getting number of entries
    ⇒ most simple code
  ‣ same but with a twist: make ktscan() set pp to the first one with
    !(entry->flag & DEFINED)¹ so that it can subsequently be re-used,
    or, more accurate, free’d and the entry pointer re-used
    ⇒ less chance of texpand()ing when not needed
  ‣ similar (from kabelaffe@): in ktsearch(), move the one we DID find
    to the first unused one
    ⇒ doesn’t need tablep or something, but has the overall best
      memory use
    ⇒ more complicated ktscan(): needs to check pointer for NULL, for
      dummyval, then entry->flag
    ⇒ makes lookup more expensive
    ⇒ benefit: self-optimising hash tables
    ⇒ loss: still need ktwalk() or ktsort()
• when afree()ing in ktremove(), …
  ① need to take FINUSE into account
• Python-2.5.4/Objects/dictnotes.txt talks about cache lines
  ‣ linear backward scan is much worse than linear forward scan
    (even if we have to calculate the upper C-array bound)
  ‣ dereferencing the entry pointer in ktscan() is a penalty
• Python-2.5.4/Objects/dictobject.c has a lot of comments and
  a rather interesting collision resolution algorithm, which
  seems to de-cluster better than linear search at not much
  more cost
• clib and libobjfw have unusable (for looking-at-for-ideas)
  hash table implementations

this is a no-op change breaking ifdef-out-d code; the most likely
to happen is to switch to the following scheme:
• keep tablep in struct tbl
• use a magic pointer value for ktremove’d entries, deallocate
  the struct tbl as soon as possible – if not FINUSE, immediately
  inside ktremove()
  ‣ memory gain, despite needing to have tablep around
• nuke ktdelete, so that all ops go through kt{enter,remove}
  ‣ gains us accurate fill information
  ‣ speed gain: ktscan() needs no longer dereference removed entries
  ‣ memory (ktsort) and speed (ktwalk) gain: removed entries are now
    ignored right from the beginning, so tstate->left and the size
    of the sorted array are accurate
  ‣ removed entries no longer can cause texpand() to be invoked
⇒ this does not give us self-optimising tables, but a speed and
  memory benefit plus, probably, simplicity of code; we accurately
  know how many non-deleted entries are in a keytab so we can cal-
  culate if we need to expand, how much space ktsort() is going to
  need, and, for when indexed arrays will be converted to use key-
  tabs instead of singly linked linear lists, ${#foo[*]} is fast
  (although ${!foo[*]}² and ${foo[*]}³ will need some tweaking and
  may run a little less quickly)
• shuffle code around, so that things like search/scan and garbage
  collection can be re-used
• use Python’s collision resolution algorithm ipv linear search

② the list of keys needs to be sorted, at least for indexed arrays⁴
③ this needs to be sorted by keys, at least for indexed arrays⁴
④ … but this is a nice-to-have for associative arrays⁵ as well
⑤ which we however do not have

main.c

@@ -33,7 +33,7 @@
 #include <locale.h>
 #endif
 
-__RCSID("$MirOS: src/bin/mksh/main.c,v 1.142 2009/08/28 21:01:26 tg Exp $");
+__RCSID("$MirOS: src/bin/mksh/main.c,v 1.143 2009/08/29 11:26:44 tg Exp $");
 
 extern char **environ;
 
@@ -1380,9 +1380,11 @@ ktremove(struct tbl *p)
 	if (p->tablep && p->tablep->size && ktscan(p->tablep, p->name,
 	    p->ua.hval, &pp) == p) {
 		/* ktremove p */
+wontwork("cannot use NULL here, see r1.143 commit message");
 		*pp = NULL;
 		p->tablep->nfree++;
 		/* get rid of p */
+wontwork("need to check FINUSE, see texpand");
 		afree(p, p->areap);
 	} else {
 		/* mark p as free for garbage collection via texpand */