newlib/winsup/cygwin/how-cygtls-works.txt

Contributed by Max Kaehn

All cygwin threads have separate context in an object of class _cygtls.  The
storage for this object is kept on the stack in the bottom CYGTLS_PADSIZE
bytes.  Each thread references the storage via the Thread Environment Block
(aka Thread Information Block), which Windows maintains for each user thread
in the system, with the address in the FS segment register.  The memory
is laid out as in the NT_TIB structure from <w32api/winnt.h>:

typedef struct _NT_TIB {
	struct _EXCEPTION_REGISTRATION_RECORD *ExceptionList;
	PVOID StackBase;
	PVOID StackLimit;
	PVOID SubSystemTib;
	_ANONYMOUS_UNION union {
		PVOID FiberData;
		DWORD Version;
	} DUMMYUNIONNAME;
	PVOID ArbitraryUserPointer;
	struct _NT_TIB *Self;
} NT_TIB,*PNT_TIB;

Cygwin sees it like this:

extern exception_list *_except_list asm ("%fs:0");      // exceptions.cc
extern char *_tlsbase __asm__ ("%fs:4");                // cygtls.h
extern char *_tlstop __asm__ ("%fs:8");                 // cygtls.h

And accesses cygtls like this:

#define _my_tls (((_cygtls *) _tlsbase)[-1])            // cygtls.h


Initialization always goes through _cygtls::init_thread().  It works
in the following ways:

* In the main thread, _dll_crt0() provides CYGTLS_PADSIZE bytes on the stack
  and passes them to initialize_main_tls(), which calls _cygtls::init_thread().
  It then calls dll_crt0_1(), which terminates with cygwin_exit() rather than
  by returning, so the storage never goes out of scope.

  If you load cygwin1.dll dynamically from a non-cygwin application, it is
  vital that the bottom CYGTLS_PADSIZE bytes of the stack are not in use
  before you call cygwin_dll_init().  See winsup/testsuite/cygload for
  more information.

* Threads other than the main thread receive DLL_THREAD_ATTACH messages
  to dll_entry() (in init.cc).
  - dll_entry() calls munge_threadfunc(), which grabs the function pointer
    for the thread from the stack frame and substitutes threadfunc_fe(),
  - which then passes the original function pointer to _cygtls::call(),
  - which then allocates CYGTLS_PADSIZE bytes on the stack and hands them
    to call2(),
  - which allocates an exception_list object on the stack and hands it to
    init_exceptions() (in exceptions.cc), which attaches it to the end of
    the list of exception handlers, changing _except_list (aka
    tib->ExceptionList), then passes the cygtls storage to init_thread().
    call2() calls ExitThread() instead of returning, so the storage never
    goes out of scope.

Note that the padding isn't necessarily going to be just where the _cygtls
structure lives; it just makes sure there's enough room on the stack when the
CYGTLS_PADSIZE bytes down from there are overwritten.


Debugging

You can examine the segment registers in gdb via "info w32 selector $fs"
(which is using GetThreadSelectorEntry()) to get results like this:

    Selector $fs
    0x03b: base=0x7ffdd000 limit=0x00000fff 32-bit Data (Read/Write, Exp-up)
    Priviledge level = 3. Byte granular.

"x/3x 0x7ffdd000" will give you _except_list, _tlsbase, and _tlstop.