Commit Graph

269 Commits

Author SHA1 Message Date
tg e1cda74d04 SECURITY: fix integer overflows related to file descriptor parsing
bug initially found by Pawel Wylecial (LP#1440685)
additional bug found and suggested fix by enh (elliott hughes)

This commit also renames struct ioword.flag to ioflag to disambiguate
it from other members named “flag”, changes it to an unsigned type,
and packs ioflag and unit into shorts each, to make the struct smaller
(aligned even: 16 bytes on 32-bit systems) and reviews some of the
code involved in fd handling, though there wasn’t much to be found.
2015-04-11 22:03:32 +00:00
tg 8641829e21 fix comment no longer true since commit 1004DC49710562ACCB2 2015-03-20 23:37:39 +00:00
tg 0d72355038 “local x=$1” is supposed to not field-split and so 2015-03-14 05:23:18 +00:00
tg 8546943535 KNF 2015-03-01 16:02:48 +00:00
tg dcdc828c41 fix here string parsing problem found by Stéphane Chazelas 2014-12-15 22:50:11 +00:00
tg 9b31399aec fix buffer overflow found by zacts from IRC (bad length calculation)
now zsh and mksh are even… :|
2014-11-19 18:44:11 +00:00
tg e4c01375aa fix access-after-free crash spotted by Enjolras via IRC, thanks! 2014-06-29 11:28:28 +00:00
tg 270a86f895 • use BAFH for hash ipv NZAAT
• prep for release
• fix minor nits in manpage and tests
2014-01-11 18:09:43 +00:00
tg 2f7eec765d merge oksh’s “ctype cleanups” commit
(not much of it applies since we don’t use the OS’ table-based lookup)
2014-01-05 19:11:46 +00:00
tg 61886de5ee oksh sync (RCSID only) 2013-11-17 22:19:42 +00:00
tg 5eb6995d75 Ypnose discovered a problem with multi-line prompts and SIGWINCH
related to caching some values (downside, pprompt gets back doprint option)
2013-08-16 10:59:03 +00:00
tg c10e5bd237 reduce amount of .bss memory needed; initialise via AEDIT at x_init
or even first run of x_vi
2013-08-10 13:44:33 +00:00
tg 247e20c524 • Completely overhaul prompt length calculation and
prompt display routines; make Emacs and Vi share
  code, permitting reducing of duplication and code
  removal as well as more consistent behaviour; put
  some common code into shared helper functions, too
• New x_adjust() logic (Emacs mode): when determining
  what portion of a line to render use a much more
  sophisticated algorithm and try to fill up ⅔ of the
  total screen width (with line and prompt both) also
  as wished from Steffen Daode Nurpmeso
2013-07-26 20:33:39 +00:00
tg ec1bc74d41 pull some quoting fixes from oksh; original patch from Alexander Polakov 2013-06-01 00:15:58 +00:00
tg 1d0409d932 implement VALSUBs 2013-05-02 21:59:54 +00:00
tg ea657bd4d9 also do not, ever, use &> bashiop in lksh 2013-03-29 17:33:56 +00:00
tg a3b5e77c6c disable parsing of &> as GNU bash iop if -o posix or -o sh, based on IRC discussion where jilles(IIRC) pointed out it breaks existing valid scripts 2013-03-24 15:01:50 +00:00
tg 74e2ef8b0b one more int → bool; mention “set -o sh” may (on raare OSes) be enabled
automatically (and it differs between targets); test MidnightBSD 0.1 ash
compat code and adjust the testsuite so it passes with it enabled
2013-02-19 18:45:22 +00:00
tg e21a2166f7 use a different approach to the $"…" problem:
take the states that may open a string, only permit them by virtue of
moving the code out of Subst: to handle $+" and $+' at all; then add
S[Q]BRACE to that

side benefits:
• clearer, cleaner code flow
• smaller code
• better “business logic” ☺
• defuses one heavy use of duff’s device a bit
2013-02-17 06:05:02 +00:00
tg b684a7c6bd backpedal with $'…' and $"…" interpolation
it turns out this breaks more legacy scripts than anticipated
2013-02-17 05:40:17 +00:00
tg 1da94d6159 pure RCSID sync with oksh, who also added a tabcompletion fix now, with
less code, but, even according to their own comments, not as good as ours
2013-02-10 17:18:49 +00:00
tg 9111faeeb5 besides more code reuse, fix $'…' everywhere (but keep it ignored in here document bodies that aren’t here strings) 2013-01-19 19:47:13 +00:00
tg e307cb34c4 use the full parser code for handling here strings (and here document words)
as ormaaj requested, including a testcase
2013-01-19 18:32:56 +00:00
tg 72402c4990 permit $'\n' in the bar part of ${foo:-bar}
XXX this changes slightly the herestring part
2013-01-19 17:49:46 +00:00
tg c99a634f92 remove separate SHERESTRING parsing state, it’s the same as SHEREDELIM 2013-01-19 17:20:02 +00:00
tg f654a3dea4 fix libFirm/cparser -Wsign-compare 2012-12-05 19:38:25 +00:00
tg d8a6ce41de shuffle declarations around (all shared between even only two files go into sh.h even if they’re not supposed to be used elsewhere) and make some static; libFirm/cparser has -Wmissing-declarations against it 2012-12-05 18:54:10 +00:00
tg 0f3071a8b2 MKSH_DISABLE_EXPERIMENTAL is a NOP again; use ${ precmd;} in dot.mkshrc 2012-11-30 20:19:16 +00:00
tg 80e364b7ad use proper parenthesēs; int → bool; use same var (eno) for temp. errno 2012-11-30 19:02:10 +00:00
tg 5aa7842d33 who would’ve thought proper ^C handling be so hard? 2012-10-30 20:49:44 +00:00
tg bfe7d78d40 bring back ${ foo;} sans dot.mkshrc patch, using a temporary file, and as experimental feature 2012-10-22 20:19:18 +00:00
tg d8662eb228 even more hacks to pass “-O666 -fstrict-overflow -Wstrict-overflow=9 -flto=jobserver” with “gcc version 4.8.0 20120930 (experimental) [trunk revision 191865] (Debian 20120930-1)” ☹ plus make the rtchecks mandatory 2012-10-03 17:24:23 +00:00
tg bb5e56d4c4 remove ${ foo;} from mksh again due to buffering issues jilles found 2012-08-17 18:34:25 +00:00
tg 9b7b7f742e implement ksh93 feature ${ foo;} 2012-07-30 21:37:17 +00:00
tg 4778a2a62f lesson learned from http://k1024.org/~iusty/blog/entry/perf-null/ – add -DMKSH_SMALL_BUT_FAST which gives more speed (8/20K less cycles, 5/9K less insns, 1.8/2.4k less branches, 65/275 less branch misses) on Debian/amd64 (klibc-static/eglibc) at cost of 0/2 more page faults and 6K/6K more text size 2012-07-01 15:55:00 +00:00
tg ea01d80833 hackish work-around the lexer to make alias definitions in mksh -c
work (Closes: #517009) and mention in the manpage why they sometimes
do not work (doing so for COMSUBs is not worth the effort)
2012-06-28 20:05:11 +00:00
tg f34cae3997 keep up 2012-05-04 22:44:34 +00:00
tg 5780bc5eb7 new MKSH_NO_CMDLINE_EDITING to disable command line editing in its entirety
mainly for the Plan 9 port though it may also help the WinAPI variant,
other porting efforts, as well as a new project I cannot say yet
2012-05-04 20:49:08 +00:00
tg 5b4d0dbbc5 drop all deprecated code, you have been warned 2012-04-07 11:19:53 +00:00
tg e67b98e21b use _setjmp/_longjmp on NeXTstep: its Intel port always restores the signal mask on siglongjmp, which we never have 2012-03-31 17:30:00 +00:00
tg cf75e7b6ce couple of minor/cosmetic fixes from RT’s compile farm:
• promote SCO OpenServer and UnixWare to !oswarn
• omit trying -O2/-O on OpenServer 5 and USL C
• cast mksh_ari_t to int, mksh_uari_t to unsigned int for printf
• skip ulimit-1 on syllable (which is still too broken)
• write ((mksh_ari_t)-2147483648) ipv UB ((mksh_ari_t)1 << 31)
  and add a comment that that is actually meant
• rewrite functions returning !void ending in NOTREACHED
  so they’ve got a jump target returning an error at the
  end, to aid older compilers and just to be safe
• cast struct stat.st_size to off_t or size_t explicitly when needed
• shorten struct env by two bytes and an alignment, at least

also, optimise control flow and fix more paren matching cases
2012-03-29 19:23:01 +00:00
tg b3f4c69f05 #ifdef MKSH_DISABLE_DEPRECATED do not compile in features scheduled for removal 2011-11-26 17:56:30 +00:00
tg fed31331f4 rename more identifiers with trailing underscores 2011-10-25 22:36:39 +00:00
tg 9782f6b4d1 • access(2) is broken in at least kFreeBSD 9.0 as “modern” OS, so bring
back the wrapper code as well as refactor most other code calling it
• apparently, names can’t end in ‘_’ or contain ‘__’ anywhere…
2011-09-07 15:24:22 +00:00
tg 577c918beb patch most of Jerker Bäck’s concerns out, unless not applicable 2011-08-27 18:06:52 +00:00
tg 80223417d7 followup fix for 1004D90467358D6B13C regarding escaped substitutions
found in, out of all things, Debian/m68k, by Wouter Verhelst (Yoe)
fix hacked at DebConf 11, Бања Лука, Република Српска, Босна и Херцеговина
2011-07-26 16:57:28 +00:00
tg b3d38f9cdf better parsing for x=(…) – bug noted by Frank Terbeck 2011-06-04 16:11:20 +00:00
tg 2083c75c1c support ;;& as alternative name for ;| unless MKSH_SMALL 2011-05-29 06:19:27 +00:00
tg 2cfc3e5c3d mksh R40 Release Candidate 1
Testsuite:
• add new need-pass: {yes|no} attribute, default yes
• exit with 1 if a need-pass test failed unexpectedly
  idea by Kacper Kornet <draenog@pld-linux.org>
• mark utf8bom-2 as need-pass: no
Infrstructure:
• add housekeeping function for making a tty raw
• switch functions with unused results to void
• struct op: u.charflag contains last char of ;; in TPAT
• var.c:arraysearch is now a global function
Language:
• add ;& (fall through) and ;| (examine next) delimiters
  in addition to ;; (end case) as zsh extensions, because
  POSIX standardised on ;& already
• add -A (read into array), -N (read exactly n bytes),
  -n (read up to n bytes), -t (timeout) flags for read
  from ksh93
• allow read -N -1 or -n -1 to slurp the entire input
• add -a (read into array the input characters) extension
  specific to mksh to read, idea by David Korn
• add -e (exit with error if PWD was not set correctly
  after a physical cd) to cd builtin, mandated by next
  POSIX, and change error codes accordingly
Rewrites:
• full rewrite of read builtin and its manpage section
• add regression tetss for most of the new functionality
• duplicate hexdump demo tests for use of read -a
• use read -raN-1 in dot.mkshrc to get NUL safe base64,
  DJB cdb hash and Jenkins one-at-a-time hash functions
2011-05-29 02:18:57 +00:00
tg 40e914e8a5 • fix an inverted logic bug in the test I wrote
• only trigger deleting an alias in favour of a function for “()”, not
  just the opening parenthesis: “stop( )” is not a function definition
  (well, actually it seems to be, but… not according to POSIX, anyway)
• defer dropping the alias until the function is actually defined (õÕ)
2011-05-07 00:51:12 +00:00
tg c38dfbf51a fix problem #1 and a comment 2011-05-07 00:24:35 +00:00
tg c021aa4cad • use a flag for determining here strings, don’t parse delimiter every time
• don’t leak memory parsing possible I/O redirection tokens
• get rid of volatile by using more const (also helps codegen, methinks)
• support empty here document markers (mksh extension)
• pimp the manpage
2011-05-05 00:05:01 +00:00
tg 69f98bca9a • fix wtf(1) by keeping the paren in ${foo#\(} properly escaped in COMSUB
• merge tputS and wdstrip_internal
⇒ net save: 604 .text 0 .data 0 .bss (MirBSD/i386)
2011-05-02 22:52:54 +00:00
tg c0e1e7645a static initialisers implicitly null out a variable (I think)
so take care of that (for global variables) to avoid them
moving from .bss to .data
2011-04-22 12:10:14 +00:00
tg e19d8612ad another Debian workaround for /bin/sh: don’t pattern in trims 2011-03-28 08:27:09 +00:00
tg 84f7055f8f special-case another bizarre POSIX corner case
after david korn agreed to change ksh93 to match
2011-03-26 16:11:43 +00:00
tg 2dffc771ac new (some possible) testcases from Austin ML, and one fix 2011-03-26 15:32:37 +00:00
tg e4b0313044 sync with oksh (nop, RCS IDs only) 2011-03-13 16:35:54 +00:00
tg 4e7d9bdb33 we need not only store into the retrace buffers recursively but also ungetsc 2011-03-13 16:20:45 +00:00
tg 91ec264cb8 fix/sync comments 2011-03-13 16:07:36 +00:00
tg af53a7d16a • speed optimisation: drop SF_FIRST flag, factor out skipping the
UTF-8 BOM instead (UTFMODE has a separate value now for activated
  during BOM skipping)
• parsing a COMSUB now skips UTF-8 BOM, too, but only temporarily
2011-03-13 16:03:54 +00:00
tg e66eaab861 let SLETARRAY use SRETRACE, too 2011-03-13 15:57:23 +00:00
tg f16386954d drop dead code 2011-03-13 15:31:46 +00:00
tg 5710e78857 implement better SASPAREN handling (actually, reusable very good for the
case where SASPAREN→COMSUB downgrading is done) by “logging” input and
restarting from there if needed
2011-03-13 01:11:58 +00:00
tg 9b8fafb757 correct documentation 2011-03-12 23:16:51 +00:00
tg 75db4cdb56 • back out the EXPRSUB change
• optimise some code
• split testcase into two, one with expected-fail
2011-03-12 23:04:48 +00:00
tg ff0a3c8a8a SASPAREN changes have the same problem, revert too for now 2011-03-12 22:44:28 +00:00
tg fc42105199 revert SLETARRAY changes for now, this introduces problems 2011-03-12 22:40:02 +00:00
tg 17dbd3bd83 • make EXPRSUB not initiate an ASCIIZ string any more but a wdstring
(token stream, lexer output / parser input), EOS terminated, let
  SASPAREN use the same lexing as SBASE (e.g. COMSUB recursively)
• make wdstrip recursive
• fix processing of COMSUB in wdstrip
⇒ pass comsub-1 test
• expose another debugging function
2011-03-12 21:41:15 +00:00
tg fb1392fb6b even more test cases for comsub (2 breaking, but one of them fixed already) 2011-03-12 20:20:17 +00:00
tg 6894618bb9 • x=$(cat <<-EOF
EOF) # works again now, plugging a regression
• rewrite the here document parsing code to be *much* more efficient
  (and a bit more readable too!) using goto, while here (no kidding)
2011-03-07 20:32:50 +00:00
tg 5f8075fc82 introduce a tobool(cond) abstraction¹ and switch bool to char if !stdbool.h
① currently: ((cond) ? true : false) but (!!(cond)) and casting to bool,
  the latter only if stdbool.h, would also work – which performs best on
  (and across) all supported systems?
2011-03-07 20:30:41 +00:00
tg 009d4c7b75 drop more currently unused 2011-03-07 20:09:34 +00:00
tg 2075dd23b2 drop unused 2011-03-07 20:08:48 +00:00
tg bfe34ba110 reduce data size by 320 bytes by packing struct lex_state tighter
and reducing amount of them allocated in each “pack” from 31 to 7
since $(…) is recursive anyway (I tried hard to need even only 5)
2011-03-07 20:07:52 +00:00
tg 1f392ab09b • more fixes (some regression, some new)
• more testcases, stricter testcases
2011-03-06 17:08:14 +00:00
tg 25905b91a7 the long-awaited recursive parser for COMSUB $(…) expressions
fixes RedHat BZ#496791
2011-03-06 01:25:35 +00:00
tg b4948e430f this is a bashism but might be from ksh93 IIRC… still dead ugly as hell! 2010-12-19 20:00:56 +00:00
tg 667d792d6a • Address concerns of Chris Palmer from the Android security team
– possible integer overflows in memory allocation, mostly
    ‣ multiplication: all are checked now
    ‣ addition: reviewed them, most were “proven” or guessed to be
      “almost” impossible to run over (e.g. when we have a string
      whose length is taken it is assumed that the length will be
      more than only a few bytes below SIZE_MAX, since code and
      stack have to fit); some are checked now (e.g. when one of
      the summands is an off_t); most of the unchecked ones are
      annotated now
    ⇒ cost (MirBSD/i386 static): +76 .text
    ⇒ cost (Debian sid/i386): +779 .text  -4 .data
  – on Linux targets, setuid() setresuid() setresgid() can fail
    with EAGAIN; check for that and, if so, warn once and retry
    infinitely (other targets to be added later once we know that
    they are “insane”)
    ⇒ cost (Debian sid/i386): +192 .text (includes .rodata)
• setmode.c: Do overflow checking for realloc() too; switch back
  from calloc() to a checked malloc() for simplification while there
• define -DIN_MKSH and let setmode.c look a tad nicer while here
2010-09-14 21:26:19 +00:00
tg 06c6be0a70 shave off another 468 bytes: we’re 300 bytes smaller than BEFORE the
cat builtin was added now… (also removed utf-8 from source files, in
favour of just ASCII)
2010-08-28 20:22:24 +00:00
tg 3747722db4 improve string pooling: saves 316 bytes in .text 2010-08-28 18:50:58 +00:00
tg c187f3089f a tad annoying, when external programmes such as GNU screen are running,
we don’t get SIGWINCH when the window size changes during the runtime of
that, so, the signal is only usable reliably during editing in the shell
and we re-check the window size before each interactive edit line again
2010-07-25 11:35:43 +00:00
tg c11d35312a Michal Hlavinka from Red Hat found another SIGSEGV
fix: when iterating over all Sources, know when to stop…
2010-07-21 11:31:16 +00:00
tg 7c91e018f4 • merge printf from OpenBSD
• deactivate %a and %A since our libc doesn’t have it
• rewrite the mksh integration code to use shf instead of stdio, removing
  floating point support always in the process, as shf doesn’t support it
  ⇒ saves 11114 (6706 text, 168 data, 4240 bss) with dietlibc on Debian
• fix -Wall -Wextra -Wformat -Wstrict-aliasing=2 for gcc (Debian 4.4.4-7)
• fix these and -Wc++-compat for gcc version 4.6.0 20100711 (experimental)
  [trunk revision 162057] (Debian 20100711-1) except:
  – a few enum warnings that relate to eglibc’s {g,s}etrlimit() functions
    taking an enum instead of an int because they’re too stupid to adhere
    to POSIX interfaces they design by themselves
  – all “request for implicit conversion” involving a "void *" on one side
• tweak the manual page somewhat more
2010-07-17 22:09:40 +00:00
tg a516ea5d89 fix another initialisation error caught by Valgrind 2010-07-04 18:29:40 +00:00
tg 30046ffcf2 implement “live” SIGWINCH handling in the Emacs editing mode
for winstonw from IRC #!/bin/mksh
2010-07-04 17:45:17 +00:00
tg ff9f89f22f Fix two more POSIX conformance bugs and minor documentation/comment issues 2010-04-08 13:21:08 +00:00
tg 2557cfbbe5 move label, see <Pine.BSM.4.64L.1003311156380.24549@herc.mirbsd.org> (my
first paragraph in that) for reason: to make it more clear from the code
2010-03-31 12:20:04 +00:00
tg 401e262689 fix another expand-ugly SUSv4 bug 2010-03-27 16:53:17 +00:00
tg e5e6857c43 (hopefully) fix the substitution thing for good 2010-02-25 20:18:19 +00:00
tg ae2901146f begin fixing up ${foo+word}, see austin-group-l for discussion and DR 2010-02-23 21:51:49 +00:00
tg 5beee95d6b at least _parse_ the construct ${foo+(bar)} correctly
(whether the output is correct is still up for debate)
2010-02-18 17:31:23 +00:00
tg 4eb17f8752 remove some more of this ugliness 2010-01-29 09:34:31 +00:00
tg c67bed8790 • unbreak testsuite for MKSH_SMALL
• sync version and © year
2010-01-28 20:58:34 +00:00
tg 4aba1d69b5 DEC ucode cc politely reminds me this is not a good way of doing stuff, 10x 2010-01-28 20:52:08 +00:00
tg 2765a07564 when using persistent history (and not MKSH_SMALL), let the shells
concurrently accessing the same $HISTFILE be more synchronised with
each other: empty lines (just pressing Return) and duplicates (that
are split and written twice by the lines loaded from $HISTFILE in
the meantime); requested by Maximilian “mxey” Gaß in #!/bin/mksh
2010-01-25 16:12:57 +00:00
tg 703551bd4f attempt a better fix: break off widthadj upon encountering NUL,
add remaining columns as octets (should also speed optimise)
2009-12-05 20:17:59 +00:00
tg 86d4dee0da valgrind (thanks ciruZ for trying) found uninitialised memory use
in utf_skipcols by x_goto
2009-12-05 19:44:09 +00:00
tg 2094e3f25c since we have wcwidth anyway, expose it as ${%strvar} to the user, iff
utf8-mode is enabled (otherwise it'll be a synonym for ${#strvar} aka
the number of octets in it)
2009-11-21 23:23:20 +00:00
tg 5f58261287 fix long-standing parenthesēs problem: ((foo) || bar) is now parsed corr. 2009-10-04 12:45:23 +00:00